MS announced multiple updates to their Azure platform, including a limited preview of an Apache Hadoop based service for Windows Azure that would simplify deployment of Hadoop applications:
“To help bolster Big Data capabilities on Windows Azure, we are releasing a preview of the Hadoop based distribution on Windows Azure. This preview enables you to easily run Hadoop projects on Windows Azure and achieve some unique benefits such as ease of use, and advanced data analysis. Highlights include:
- Install and setup Hadoop on Windows Azure in hours instead of days thanks to a new set of installers that simplify setup and deployment.
- Hive ODBC Driver and Hive Add-in for Excel enable data analysis of unstructured data through familiar tools including Excel and PowerPivot.”
More information is available on the Windows Azure Team Blog.
December 19 2011 update: Microsoft’s “Big Data Solution” page has recently been updated, and I can say that, unlike many other companies jumping on the big data bandwagon, MS gets it. Big data frameworks are not here to replace traditional RDBMS but to complement them. The biggest technological challenge is in figuring out a way to marry traditional RDBMS technologies with the emerging open-source tech for loosely structured data (truly “unstructured” data is quite rare). Relational databases have been fine-tuned and perfected, while the Hadoop stack is buggy, lacks optimization capabilities, security features, and is generally difficult to install, configure, balance, and optimize. The two are hardly “compatible”. But, making the two work together opens up new capabilities in data analysis – while letting you scale out at low cost.
We recently designed a system to do just that – capturing and processing precise transactional data alongside loosely structured market, sentiment, and social media “conditions” at the exact moment in time the transaction took place. If your organization needs help with similar projects, feel free to drop us a line at firstname.lastname@example.org. We’ll be glad to help!
by Michael Alatortsev