Post

4 followers Follow
1
Avatar

Restore GemFire XD Tables using HDFS logs Files

Is there a possibility to restore the GemFire ​​tables XD using the data transferred to HDFS ?
In other words, if I create in GemFire ​​XD table for HDFS persistence (without local disk persistence), if ever GemFire ​​XD turns off, can i restore the table using its data persisted in HDFS ?

Regards,
Imane

Imane

Please sign in to leave a comment.

3 comments

0
Avatar

Currently, the short answer is no. The best way to do it right now would be to run a MapReduce job and then feed the results back into GFXD.

We are discussing ways to do this for future releases, so if you have a concrete use case we would love to know about it. Given that the in-memory, operational data set will be orders of magnitude smaller than what is on HDFS, knowing exactly what to read back into memory is the big question. Doing that as fast as possible is also important, but somewhat secondary.

Jens Deppe 0 votes
0
Avatar

Just adding that you can still operate over that data by using queryHDFS hint to your statements or even at the connection level (so all statements will be using the hint by default). Of course that it will have some performance implications when compared to data in-memory only.

William Markito 0 votes
0
Avatar

William, Jens, Thank you for your responses !

Currently, I don't have a concrete use case, but not for long !

regards

Imane 0 votes