Hadoop 常见问题汇总

Hadoop

datanode 重启后，由于 ip 变化导致连接 namenode 失败

现象：

Datanode denied communication with namenode because hostname cannot be resolved (ip=10.5.x.x, hostname=10.5.x.x): DatanodeRegistration(0.0.0.0, datanodeUuie12e2d92-e215-4af9-a09b-d22ef4e3cc5ainfoPort=9102, ipcPort=9201, storageInfo=lv=-55;cid=CID-f50d3808-5d7f-4520-b696-0234f07f89f7;nsid=1758720717;c=0)

原因：

“Hadoop provides a mechanism to prevent unauthorized access. User can add hosts.deny and hosts.allow files to limit the access to a hadoop cluster. When a data node joins the name node, the name node will check the DNS name of the data node, if the DNS name cannot be resolved or the host is not allowed based on hosts.deny and hosts.allow files, the NN will reject the data node join.

解决方案:

namenode 增加 hdfs-site.xml 配置

<property>
  <name>dfs.namenode.datanode.registration.ip-hostname-check</name>
  <value>false</value>
</property>