iTrend: under the hood

Currently, we are using Hadoop, Hive, Pig, HBase, Nutch, Lucene, Solr, and several other frameworks.

6152208054_2ef6114803_o

Custom data gathering software is written in Python or Java.  The software runs primarily on Dell hardware, and our typical dev server setup looks like this:

5981778256_3999cfc5aa_o

We are now relying on commercial “cloud” providers for additional capacity, but are planning to upgrade our own infrastructure over next few months to support full size data sets (billions of objects).

If you have practical “big data” experience and would like to join the iTrend team, send your resume to info at itrend.tv.

 

 

Technologist, parallel entrepreneur. Interests: travel, photography, big data, analytics, predictive modeling.

Tagged with: , , , , , , , , , , , , , , , ,
Posted in Uncategorized

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: