Solution to Cloudera SCM Express problem

Ran into a strange issue installing Cloudera’s Hadoop Distribution using SCM Express on a cluster of 64-bit CentOS 5.7 machines recently.  

Install process would get to Hue common/proxy packages, fail, then proceed to roll back. I saw a number of similar reports in various message groups but no solution, so we are documenting our experience here:

Installing : hue-common 2/2/usr/share/hue/build/env/bin/python2.4: 
error while loading shared libraries: cannot open 
shared object file: No such file or directory 

Target machines had Python (64-bit library) installed:

[root@machine1 /]# locate 

… but it appeared as though the installation scriplet was expecting the 32-bit version.  Logs revealed that the script was trying to install both 32-bit and 64-bit packages that would then stomp on each other: 

Installing Hue common package… 

BEGIN yum list installed hue-common 
Loaded plugins: fastestmirror 
Error: No matching Packages to list 
END (1) 
BEGIN yum list available hue-common 
Loaded plugins: fastestmirror 
Loading mirror speeds from cached hostfile 
* base: 
* extras: 
* updates: 
Available Packages 
hue-common.i386 cloudera-cdh3 
hue-common.x86_64 cloudera-cdh3 
END (0) 
BEGIN yum -y install hue-common 
Loaded plugins: fastestmirror 
Loading mirror speeds from cached hostfile 
* base: 
* extras: 
* updates: 
Setting up Install Process 
Resolving Dependencies 
–> Running transaction check 
—> Package hue-common.i386 0: set to be updated 
—> Package hue-common.x86_64 0: set to be updated 
–> Finished Dependency Resolution 

According to Cloudera’s support team, there was a bug in the initial set of CDH3u2 packages that caused both the 32-bit and 64-bit packages to be installed, but this was fixed a few days after the initial packages were put out there.  For some unknow reason (oudated cache somewhere?) our installation process was running into the same bug.

Solution/workaround suggested by the support team – to add to the [main] section of your /etc/yum.conf the following line:  

exclude=*.i386 *.i586 *.i686

Note: this needs to be done before running any SCM instal scripts.  As a result, SCM only pulls 64-bit versions of the packages – and you avoid Python library conflicts.

Also – Cloudera tech support team is very responsive.

Other known CDH3 bugs and workarounds can be found here.

UPDATE: someone else also struggled with this issue, and by doing a remove of the i386  package and reinstalling the x86_64, they got Hue running without having  to reinstall SCM completely. 

December 19 2011 update: based on our Google Analytics reports, many companies are struggling with similar installation issues.  iTrend will be happy to assist you with your Big Data initiative – we have all necessary engineering resources in-house.  Email your requirements to and mention this blog.


Technologist, parallel entrepreneur. Interests: travel, photography, big data, analytics, predictive modeling.

Tagged with: , , , , , ,
Posted in Uncategorized

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s

%d bloggers like this: