Pivotal Knowledge Base

Follow

Namenode startup fails with "Does not contain a valid host:port authority: ${secondarynamenode}:50090" on HA cluster after kerberos enablement

Environment:

PHD 1.1.x

Problem:

Namenode startup fails with Does not contain a valid host:port authority: ${secondarynamenode}:50090 on PHD HA (High Availability) cluster after kerberos enablement

2014-01-26 20:15:55,455 ERROR org.apache.hadoop.security.UserGroupInformation: PriviledgedActionException as:hdfs/phd11-nn.saturn.local@SATURN.LOCAL (auth:KERBEROS) cause:org.apache.hadoop.hdfs.server.namenode.TransferFsImage$HttpGetFailedException: Fetch of http://phd11-dn-1.saturn.local:8480/getJournal?jid=phd11krbha&segmentTxId=1&storageInfo=-40%3A848004040%3A0%3ACID-f9374af6-70d7-4498-bdd1-e407ea48939f failed with status code 500
Response message:
getedit failed. java.lang.IllegalArgumentException: Does not contain a valid host:port authority: ${secondarynamenode}:50090  at org.apache.hadoop.net.NetUtils.createSocketAddr(NetUtils.java:193)  at org.apache.hadoop.net.NetUtils.createSocketAddr(NetUtils.java:160)  at o 

Solution:

Once Kerberos is enabled on a PHD HA cluster, you may see the error reported above when namenode starts. Although, prior to kerberization, the cluster may be be working fine.

In order to workaround the issue, remove the below entries for secondary namenode parameter from /etc/gphd/hadoop/conf/hdfs-site.xml file on all the nodes and restart the services.

<property>
<name>dfs.namenode.secondary.http-address</name>
<value>${secondarynamenode}:50090</value>
<description>
The secondary namenode http server address and port.
</description>
</property>

This parameter is a part of Install & Configuration Manager (ICM) cluster configuration hdfs-site.xml template. If HA cluster has been deployed after removing it from the template, then you will not come across this error.

Miscellaneous:

  • Pivotal internal employees reference JIRA : HD-8652

Comments

Powered by Zendesk