在mac os上安装hadoop的文章不多,赶紧分享一个,本文安装共分为7布,实验环境的操作系统是 MAC OS X 10.7 Lion,
第一步:确认JAVA是否已经安装,
可以在实用工具->终端中使用java -version命令来确认java的版本。如果没有安装java,也可以进入下面网址下载:http://support.apple.com/kb/dl1421。
第二步,去Hadoop官网下载,本文使用的Hadoop版本是1.0.4的stable版本。
下载完之后解压缩,我这里放置的目录是/users/chinastor/Hadoop。
第三步:设置环境变量
在实际启动Hadoop之前,有三个文件需要进行配置。
但在这之前,我们需要在mac os系统中设置一下几个类似Windows的环境变量,方便以后在命令行敲命令。
export HADOOP_HOME=/users/chinastor/hadoop
export PATH=$PATH:$HADOOP_HOME/bin
配置hadoop-env.sh
在Hadoop->conf目录下,找到hadoop-env.sh,打开编辑进行如下设置:
export JAVA_HOME=/library/Java/Home(去掉注释)
export HADOOP_HEAPSIZE=2000(去掉注释)
export HADOOP_OPTS="-Djava.security.krb5.realm=OX.AC.UK -Djava.security.krb5.kdc=kdc0.ox.ac.uk:kdc1.ox.ac.uk"(去掉注释)
注意第三个配置在OS X上最好进行配置,否则会报错“Unable to load realm info from SCDynamicStore”。
配置core-site.xml
<configuration>
<property>
<name>hadoop.tmp.dir</name>
<value>/users/chinastor/hadoop/tmp/hadoop-${user.name}</value>
<description>A base for other temporary directories.</description>
</property>
<property>
<name>fs.default.name</name>
<value>hdfs://localhost:8020</value>
</property>
</configuration>
配置hdfs-site.xml
<configuration>
<property>
<name>dfs.replication</name>
<value>1</value>
</property>
</configuration>
配置mapred-site.xml
<configuration>
<property>
<name>mapred.job.tracker</name>
<value>localhost:8021</value>
</property>
<property>
<name>mapred.tasktracker.map.tasks.maximum</name>
<value>2</value>
</property>
<property>
<name>mapred.tasktracker.reduce.tasks.maximum</name>
<value>2</value>
</property>
</configuration>
第四步:安装HDFS
经过以上的配置,就可以进行HDFS的安装了。
$HADOOP_HOME/bin/hadoop namenode -format
如果顺利的话,会得到类似如下的输出:
chinastorcom-Air:hadoop Billy$ $HADOOP_HOME/bin/hadoop namenode -format
Warning: $HADOOP_HOME is deprecated.
14/12/02 17:11:12 INFO namenode.NameNode: STARTUP_MSG:
/************************************************************
STARTUP_MSG: Starting NameNode
STARTUP_MSG: host = chinastorcom-Air.local/192.168.1.102
STARTUP_MSG: args = [-format]
STARTUP_MSG: version = 1.0.4
STARTUP_MSG: build = https://svn.apache.org/repos/asf/hadoop/common/branches/branch-1.0 -r 1393290; compiled by 'hortonfo' on Wed Oct 3 05:13:58 UTC 2012
************************************************************/
14/12/02 17:11:12 INFO util.GSet: VM type = 64-bit
14/12/02 17:11:12 INFO util.GSet: 2% max memory = 39.9175 MB
14/12/02 17:11:12 INFO util.GSet: capacity = 2^22 = 4194304 entries
14/12/02 17:11:12 INFO util.GSet: recommended=4194304, actual=4194304
14/12/02 17:11:12 INFO namenode.FSNamesystem: fsOwner=Billy
14/12/02 17:11:12 INFO namenode.FSNamesystem: supergroup=supergroup
14/12/02 17:11:12 INFO namenode.FSNamesystem: isPermissionEnabled=true
14/12/02 17:11:12 INFO namenode.FSNamesystem: dfs.block.invalidate.limit=100
14/12/02 17:11:12 INFO namenode.FSNamesystem: isAccessTokenEnabled=false accessKeyUpdateInterval=0 min(s), accessTokenLifetime=0 min(s)
14/12/02 17:11:13 INFO namenode.NameNode: Caching file names occuring more than 10 times
14/12/02 17:11:13 INFO common.Storage: Image file of size 111 saved in 0 seconds.
14/12/02 17:11:13 INFO common.Storage: Storage directory /users/chinastor/hadoop/tmp/hadoop-Billy/dfs/name has been successfully formatted.
14/12/02 17:11:13 INFO namenode.NameNode: SHUTDOWN_MSG:
/************************************************************
SHUTDOWN_MSG: Shutting down NameNode at chinastorcom-Air.local/192.168.1.102
************************************************************/
第五步:启动Hadoop
很简单,一条命令搞定。
$HADOOP_HOME/bin/start-all.sh
顺利的话,一般会让你输入三次账号的密码。
第六步:HADOOP的简单调试
如果想试试看是否已经成功启动,可以用自带的例子试验一下:
$hadoop jar $HADOOP_HOME/hadoop-example-1.0.4.jar pi 10 100
成功的话,会有类似结果:
chinastorcom-Air:hadoop Billy$ hadoop jar $HADOOP_HOME/hadoop-examples-1.0.4.jar pi 10 100
Warning: $HADOOP_HOME is deprecated.
Number of Maps = 10
Samples per Map = 100
Wrote input for Map #0
Wrote input for Map #1
Wrote input for Map #2
Wrote input for Map #3
Wrote input for Map #4
Wrote input for Map #5
Wrote input for Map #6
Wrote input for Map #7
Wrote input for Map #8
Wrote input for Map #9
至此,Hadoop在Mac OS X上的安装实验完成了,中存储米乐将与各位分享更多hadoop使用心得。