Kettle连接Hadoop

Linux软件包
1
yum -y install redhat-lsb
hdfs-site.xml
1
2
3
4
<property>
  <name>dfs.permissions.enabled</name>
  <value>false</value>
</property>
复制Hadoop配置文件到Kettle

  • core-site.xml

  • hdfs-site.xml

  • yarn-site.xml

  • mapred-site.xml

1
data-integration\plugins\pentaho-big-data-plugin\hadoop-configurations\cdh61
主对象树中配置HadoopCluster

image-20221213075351415

复制文件到HDFS

Local到HDFS

image-20221213075538074

image-20221213075203144

MapReduce

MapReduce Mapper

image-20221213082122602

image-20221213082202102

image-20221213082227629

image-20221213082250529

image-20221213082310633

MapReduce Reduce

image-20221213082430515

image-20221213082331351

image-20221213082354984

image-20221213082415349

MapReduce

image-20221213082505541

image-20221213082528100

image-20221213082544704

image-20221213082603211