Post

4 followers Follow
0
Avatar

How to integrate cdh4/hortonworks(HDP) with GemfireXD

Hi,

How to integrate the gemfireXD with CDH4/HDP. Pivotal hadoop stack gemfireXD is already integrated. What is the approach to get the data from CDH and load the data into gemfireXD with persistence mechanism. your inputs are really appreciated.

Regards,
Ram.

Ram

Please sign in to leave a comment.

3 comments

0
Avatar

Hey Ram,
At this point we only support production deployments of GemFireXD running over Pivotal HD. That being said, you may not have any problems running it over CDH4 or any other Apache based Hadoop implementation...

Official support for other Hadoop distros is being currently studied.

Thanks

William Markito 0 votes
0
Avatar

Hey Will, Great. So, guide me this strategy works or not? create hdfs persistance tables on CDH and then read them by using JDBC client and persist them in GemfireXD. But in order to create persistance tables on CDH do I need setup any thing on CDH? Please advice.

Ram.

Ramana G 0 votes
0
Avatar

Hi Ram,

Here is an example of creating a write-only hdfs-backed table: https://github.com/gopivotal/gfxd-demo/blob/master/scripts/schema-hadoop.sql#L16. All you should need to do is point your DDL at the namenode; no other config should be necessary. This example is for a write-only table which is not queryable from GFXD. For that, you need to use a read-write table and use the queryHDFS property for queries or on your connection. Check out this page: http://gemfirexd.docs.gopivotal.com/latest/userguide/disk_storage/persist-hdfs-topics.html

Jens Deppe 0 votes