본문 바로가기

오픈소스/hadoop

install hadoop 2.x on mac os x


http://zhongyaonan.com/hadoop-tutorial/setting-up-hadoop-2-6-on-mac-osx-yosemite.html


http://withsmilo.github.io/bigdata/2016/06/01/setup-hadoop-2.7.2-on-osx/

$ brew tap homebrew/versions $ brew install protobuf250 $ brew link --force --overwrite protobuf250 $ protoc --version libprotoc 2.5.0 $ brew install cmake $ wget http://www.eu.apache.org/dist/hadoop/common/hadoop-2.7.1/hadoop-2.7.1-src.tar.gz $ tar xvf hadoop-2.7.1-src.tar.gz $ cd hadoop-2.7.1-src $ mvn package -Pdist,native -DskipTests -Dtar $ mv $HADOOP_HOME/lib/native $HADOOP_HOME/lib/native.bak $ cd $HADOOP_HOME/lib $ ln $HADOOP_HOME/lib/native native Add the following lines in the hadoop-env.sh and yarn-env.sh export HADOOP_COMMON_LIB_NATIVE_DIR=${HADOOP_HOME}/lib/native export HADOOP_OPTS="${HADOOP_OPTS} -Djava.library.path=$HADOOP_HOME/lib/native"

http://stackoverflow.com/questions/20586920/hadoop-connecting-to-resourcemanager-failed

INFO ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8032. Already tried 0 time(s); 
retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1 SECONDS)
<property>
<name>yarn.resourcemanager.address</name>
<value>127.0.0.1:8032</value>
</property>
<property>
<name>yarn.resourcemanager.scheduler.address</name>
<value>127.0.0.1:8030</value>
</property>
<property>
<name>yarn.resourcemanager.resource-tracker.address</name>
<value>127.0.0.1:8031</value>
</property>


http://stackoverflow.com/questions/19943766/hadoop-unable-to-load-native-hadoop-library-for-your-platform-warning

http://stackoverflow.com/questions/20144371/hadoop-2-2-0-setup-pseudo-distributed-mode-error-warn-util-nativecodeloader