Pivotal Knowledge Base

Follow

How to configure HttpFS to access kerberized HDFS

Environment

Product Version
Pivotal HD 3.0.x
OS RHEL 6.x

Purpose

This article discusses how to configure HttpFS to access HDFS secured with Kerberos.

Please refer to the procedure in this article if you want to access kerberized HDFS via HttpFS.

Procedure

1. Create httpFS and HTTP service principals on KDC server

# addprinc -randkey httpfs/<FQDN>@<REALM>
# addprinc -randkey HTTP/<FQDN>@<REALM> 

Replace <FQDN> with a fully qualified domain name of  the host where the HttpFS server is running.

Replace <REALM> with the name of your Kerberos realm.

In the examples of this article FQDN will be admin.hadoop.local and REALM will be PIVOTAL.IO.

# kadmin.local -q "addprinc -randkey httpfs/admin.hadoop.local@PIVOTAL.IO "
# kadmin.local -q "addprinc -randkey HTTP/admin.hadoop.local@PIVOTAL.IO"

2. Create the keytab file with both principals

# kadmin.local -q "ktadd -k /etc/security/keytabs/httpfs.service.keytab httpfs/admin.hadoop.local@PIVOTAL.IO HTTP/admin.hadoop.local@pivotal.io"

3. Distribute the keytab file to the machine that run HttpFS server. Normally put the file to /etc/security/keytabs/

4. Change the ownership of keytab file to be owned by a user to run HttpFS server and group hadoop. And change permission of keytab file to 400

5. Test that the keytab file works

# kinit -kt /etc/security/keytabs/httpfs.service.keytab httpfs/admin.hadoop.local@PIVOTAL.IO
[root@admin ~]# klist
Ticket cache: FILE:/tmp/krb5cc_0
Default principal: httpfs/admin.hadoop.local@PIVOTAL.IO

Valid starting Expires Service principal
12/01/15 08:33:47 12/02/15 08:33:47 krbtgt/PIVOTAL.IO@PIVOTAL.IO
renew until 12/01/15 08:33:47

6. Add following properties to httpfs-site.xml (typically under /etc/hadoop-httpfs/conf)

<property>
  <name>httpfs.authentication.type</name>
  <value>kerberos</value>
</property> <property>
  <name>httpfs.hadoop.authentication.type</name>
  <value>kerberos</value>
</property> <property>
  <name>httpfs.authentication.kerberos.principal</name>
  <value>HTTP/admin.hadoop.local@PIVOTAL.IO</value>
</property> <property>
  <name>httpfs.authentication.kerberos.keytab</name>
  <value>/etc/security/keytabs/httpfs.service.keytab</value>
</property> <property>
  <name>httpfs.hadoop.authentication.kerberos.principal</name>
  <value>httpfs/admin.hadoop.local@PIVOTAL.IO</value>
</property> <property>
<name>httpfs.hadoop.authentication.kerberos.keytab</name>
<value>/etc/security/keytabs/httpfs.service.keytab</value>
</property> <property>
  <name>httpfs.authentication.kerberos.name.rules</name>
  <value>         RULE:[2:$1@$0](rm@.*PIVOTAL.IO)s/.*/yarn/
        RULE:[2:$1@$0](nm@.*PIVOTAL.IO)s/.*/yarn/
        RULE:[2:$1@$0](nn@.*PIVOTAL.IO)s/.*/hdfs/
        RULE:[2:$1@$0](dn@.*PIVOTAL.IO)s/.*/hdfs/
        RULE:[2:$1@$0](hbase@.*PIVOTAL.IO)s/.*/hbase/
        RULE:[2:$1@$0](hbase@.*PIVOTAL.IO)s/.*/hbase/
        RULE:[2:$1@$0](oozie@.*PIVOTAL.IO)s/.*/oozie/
        RULE:[2:$1@$0](jhs@.*PIVOTAL.IO)s/.*/mapred/
        RULE:[2:$1@$0](jn/_HOST@.*PIVOTAL.IO)s/.*/hdfs/
        RULE:[2:$1@$0](falcon@.*PIVOTAL.IO)s/.*/falcon/
        DEFAULT     </value> </property>

Note:

  • Replace principal names and path of keytab files accordingly
  • Use the value configured for hadoop.security.auth_to_local in core-site.xml for httpfs.authentication.kerberos.name.rules

7. Restart HttpFS service to take the configuration changes into effect

# service hadoop-httpfs restart

8. Test access to secured HDFS via HttpFS is successful. Following is a test to get Kerberos credential and list root directory in HDFS

[root@admin ~]# kinit -kt /etc/security/keytabs/httpfs.service.keytab HTTP/admin.hadoop.local@PIVOTAL.IO
[root@admin ~]# klist
Ticket cache: FILE:/tmp/krb5cc_0
Default principal: HTTP/admin.hadoop.local@PIVOTAL.IO Valid starting Expires Service principal
12/01/15 06:53:15 12/02/15 06:53:15 krbtgt/PIVOTAL.IO@PIVOTAL.IO
renew until 12/01/15 06:53:15
[root@admin ~]# curl --negotiate -i -L -u: 'http://admin.hadoop.local:14000/webhdfs/v1/?op=LISTSTATUS'
HTTP/1.1 401 Unauthorized
Server: Apache-Coyote/1.1
WWW-Authenticate: Negotiate
Set-Cookie: hadoop.auth=; Path=/; Expires=Thu, 01-Jan-1970 00:00:00 GMT; HttpOnly
Content-Type: text/html;charset=utf-8
Content-Length: 997
Date: Tue, 01 Dec 2015 11:53:23 GMT HTTP/1.1 200 OK
Server: Apache-Coyote/1.1
Set-Cookie: hadoop.auth="u=HTTP&p=HTTP/admin.hadoop.local@PIVOTAL.IO&t=kerberos&e=1449006803887&s=qeIsIBmD6POgBrciNHuMna2ifrY="; Path=/; Expires=Tue, 01-Dec-2015 21:53:23 GMT; HttpOnly
Content-Type: application/json
Transfer-Encoding: chunked
Date: Tue, 01 Dec 2015 11:53:23 GMT {"FileStatuses":{"FileStatus":[{"pathSuffix":"app-logs","type":"DIRECTORY","length":0,"owner":"yarn","group":"hadoop","permission":"777","accessTime":0,"modificationTime":1443577258258,"blockSize":0,"replication":0},{"pathSuffix":"apps","type":"DIRECTORY","length":0,"owner":"hdfs","group":"hdfs","permission":"755","accessTime":0,"modificationTime":1442907202986,"blockSize":0,"replication":0},{"pathSuffix":"hawq_data","type":"DIRECTORY","length":0,"owner":"postgres","group":"gpadmin","permission":"755","accessTime":0,"modificationTime":1442987912402,"blockSize":0,"replication":0},{"pathSuffix":"mapred","type":"DIRECTORY","length":0,"owner":"mapred","group":"hdfs","permission":"755","accessTime":0,"modificationTime":1442907099113,"blockSize":0,"replication":0},{"pathSuffix":"mr-history","type":"DIRECTORY","length":0,"owner":"hdfs","group":"hdfs","permission":"755","accessTime":0,"modificationTime":1442907099121,"blockSize":0,"replication":0},{"pathSuffix":"phd","type":"DIRECTORY","length":0,"owner":"hdfs","group":"hdfs","permission":"755","accessTime":0,"modificationTime":1442907132464,"blockSize":0,"replication":0},{"pathSuffix":"system","type":"DIRECTORY","length":0,"owner":"hdfs","group":"hdfs","permission":"755","accessTime":0,"modificationTime":1442907074968,"blockSize":0,"replication":0},{"pathSuffix":"tmp","type":"DIRECTORY","length":0,"owner":"hdfs","group":"hdfs","permission":"777","accessTime":0,"modificationTime":1442990872079,"blockSize":0,"replication":0},{"pathSuffix":"user","type":"DIRECTORY","length":0,"owner":"hdfs","group":"hdfs","permission":"755","accessTime":0,"modificationTime":1443576819986,"blockSize":0,"replication":0}]}}

Additional Information

 

Comments

Powered by Zendesk