专业支持:(0512) 63016160 / 销售热线:(0512)63016068
当前在线: 288 今日上线: 1384 今日新增: 3

开源云计算技术系列(四)(Cloudera安装配置hadoop 0.20最新版配置)

文章来源:  中国云计算 发布时间: 2009年09月02日   浏览: 811   作者:中国云计算

接上文,我们继续体验Cloudera 0.20最新版。

wget hadoop-0.20-conf-pseudo_0.20.0-1cloudera0.5.0~lenny_all.deb

wget hadoop-0.20_0.20.0-1cloudera0.5.0~lenny_all.deb

debian:~# dpkg –i hadoop-0.20-conf-pseudo_0.20.0-1cloudera0.5.0~lenny_all.deb

dpkg –i hadoop-0.20_0.20.0-1cloudera0.5.0~lenny_all.deb

就这么简单。ok

如果不清楚安装到哪里了,可以用

debian:~# dpkg -L hadoop-0.20

可以看到清晰的安装目录结构。

启动:

debian:~# cd /etc/init.d/hadoop-0.20-
hadoop-0.20-datanode           hadoop-0.20-namenode           hadoop-0.20-tasktracker
hadoop-0.20-jobtracker         hadoop-0.20-secondarynamenode 
debian:~# cd /etc/init.d/hadoop-0.20-

debian:~# /etc/init.d/hadoop-0.20-namenode start

debian:~# /etc/init.d/hadoop-0.20-namenode status
hadoop-0.20-namenode is running

debian:~# /etc/init.d/hadoop-0.20-datanode start

debian:~# /etc/init.d/hadoop-0.20-datanode status
hadoop-0.20-datanode is running

 

debian:~# /etc/init.d/hadoop-0.20-jobtracker start

debian:~# /etc/init.d/hadoop-0.20-jobtracker status
hadoop-0.20-jobtracker is running

 

debian:~# /etc/init.d/hadoop-0.20-tasktracker start

debian:~# /etc/init.d/hadoop-0.20-tasktracker status
hadoop-0.20-tasktracker is running

启动完成。

接着可以进行常规的example的测试了。

值得测试的是

debian:~# sqoop --help
Usage: hadoop sqoop.jar org.apache.hadoop.sqoop.Sqoop (options)

Database connection options:
--connect (jdbc-uri)         Specify JDBC connect string
--driver (class-name)        Manually specify JDBC driver class to use
--username (username)        Set authentication username
--password (password)        Set authentication password
--local                      Use local import fast path (mysql only)

Import control options:
--table (tablename)          Table to read
--columns (col,col,col...)   Columns to export from table
--order-by (column-name)     Column of the table used to order results
--hadoop-home (dir)          Override $HADOOP_HOME
--warehouse-dir (dir)        HDFS path for table destination
--as-sequencefile            Imports data to SequenceFiles
--as-textfile                Imports data as plain text (default)
--all-tables                 Import all tables in database
                             (Ignores --table, --columns and --order-by)

Code generation options:
--outdir (dir)               Output directory for generated code
--bindir (dir)               Output directory for compiled objects
--generate-only              Stop after code generation do not import

Additional commands:
--list-tables                List tables in database and exit
--list-databases             List all databases available and exit
--debug-sql (statement)      Execute statement in SQL and exit

Generic Hadoop command-line options:
Generic options supported are
-conf <configuration file>     specify an application configuration file
-D <property=value>            use value for given property
-fs <local|namenode:port>      specify a namenode
-jt <local|jobtracker:port>    specify a job tracker
-files <comma separated list of files>    specify comma separated files to be copied to the map reduce cluster
-libjars <comma separated list of jars>    specify comma separated jar files to include in the classpath.
-archives <comma separated list of archives>    specify comma separated archives to be unarchived on the compute machines.

The general command line syntax is
bin/hadoop command [genericOptions] [commandOptions]

At minimum, you must specify --connect and either --table or --all-tables.
Alternatively, you can specify --generate-only or one of the additional
commands.

可以通过apt-get install mysql-server 安装debian下的mysql进行联合测试。

测试体验见前面的文章,到这里大家可以进行一个完整的体验了。Cloudera的出现的确对hadoop的配置进行了极大的简化,推动了开源云计算的发展。


一键分享:

在线客服