Pivotal Knowledge Base

Follow

Special chars decoding in HDFS command.

Environment

  • PHD 2.x

Problem

You create a directory name with a special character in it and when running HDFS ls command, the special character like # converting to %23 [ascii character], which impacts the functionality of the application.

Summary

Creating a directory:

[gpadmin@hdm1 ~]$ hadoop fs -mkdir /tmp/10.10.11.44#11

drwxr-xr-x - gpadmin hadoop 0 2014-10-28 10:53 /tmp/10.10.11.44#11

Creating a directory inside the created directoy

[gpadmin@hdm1 ~]$ hadoop fs -mkdir /tmp/10.10.11.44#11/hosts

Doing a listing on the directory structure using HDFS command

[gpadmin@hdm1 ~]$ hadoop fs -ls /tmp/10.10.11.44#11

Found 1 items
drwxr-xr-x - gpadmin hadoop 0 2014-10-28 10:54 /tmp/10.10.11.44%2311/hosts

Resolution/Alternative

Some of the special characters are not supported by the URIpath in hadoop shell[usually you can replace # with ! and try it]. Recommended to use only characters which are mentioned in JAVAURI.

http://docs.oracle.com/javase/7/docs/api/java/net/URI.html

category: Character categories

Java can encode these paths correctly but not the Hadoop shell.

Comments

Powered by Zendesk