Pivotal Knowledge Base

Follow

Sample WebHDFS C application using libcurl

If you are writing a c application and need to communicate with webhdfs then this article can help you get started. 

Get the code and build

  1. git clone https://github.com/randomtask1155/WebHDFS
  2. cd WebHDFS
  3. make

Try it out

In this example we will create a directory using the webhdfs rest api with webhdfsTest c applicaiton.  Please see the apache documentation to find more API calls for testing http://hadoop.apache.org/docs/r2.2.0/hadoop-project-dist/hadoop-hdfs/WebHDFS.html

First we create the directory using webhdfsTest

[gpadmin@hdm1 playground]$ ./webhdfsTest 172.28.16.158 50070 "webhdfs/v1/TESTDIR/rsasoc/v1/sessions/data/2014/5/17/15/27?user.name=gpadmin&op=MKDIRS&permission=777"

running http put: http://172.28.16.158:50070/webhdfs/v1/TESTDIR/rsasoc/v1/sessions/data/2014/5/17/15/27?user.name=gpadmin&op=MKDIRS&permission=777

Response:
{"boolean":true}

Note

172.28.16.158 - Namenode IP
50070 - (Default). As specified by dfs.http.address

Then we make sure the directory was created in hdfs  

[gpadmin@hdm1 playground]$ hdfs dfs -ls /TESTDIR/data/2014/5/17/15/
Found 1 items
drwxrwxrwx   - gpadmin hadoop          0 2014-06-09 15:56 /TESTDIR/data/2014/5/17/15/27

 

 

Comments

Powered by Zendesk