Pivotal Knowledge Base


Hadoop commands fails with "java.lang.NullPointerException"


When executing hadoop read / write commands, a Null Pointer Exception is seen. 

[hive@aoabdlp00004 ~]$ hdfs dfs -get /app-logs/hive/logs/application_1430501760753_0003
get: java.lang.NullPointerException


Namenode logs can give insights into the probable cause of the error. In this specific instance, while running hadoop commands to pull data from HDFS, rack script was not resolving the client IP to any of the rack. The snippet below lists the reason "/etc/hadoop/conf/CentralPerkRack.pl returned 0 values when 1 were expected" when the rack resolve script was fed with an input IP

2015-05-01 14:03:42,299 ERROR net.ScriptBasedMapping (ScriptBasedMapping.java:resolve(198)) - Script /etc/hadoop/conf/CentralPerkRack.pl returned 0 values when 1 were expected.
2015-05-01 14:03:42,300 WARN  ipc.Server (Server.java:run(2058)) - IPC Server handler 31 on 8020, call org.apache.hadoop.hdfs.protocol.ClientProtocol.getBlockLocations from Call#
2 Retry#0
        at org.apache.hadoop.hdfs.server.blockmanagement.DatanodeManager.sortLocatedBlocks(DatanodeManager.java:359)
        at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocations(FSNamesystem.java:1799)
        at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.getBlockLocations(NameNodeRpcServer.java:543)
        at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.getBlockLocations(ClientNamenodeProtocolServerSideTranslatorPB.java:364)
        at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
        at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:619)


Rack resolving script must resolve the incoming client request to a rack. Either you can include the client to a specific rack or you can add a default rack to resolve any client IP to it if it does not matches any of the other configured rack IP ranges.


Powered by Zendesk