Post

2 followers Follow
0
Avatar

Queue Persistence

Hello,

I wanted to know what is the role of Queue Persistence in GemFire XD Local Disk (in both HDFS Write-Only and Read/Write persistence), knowing that if HDFS crashes and restarts later, it can not receive datas sent by GemFire ​​XD meantime, which are always stored in the queue (theoretically)?

regards.

Imane

Please sign in to leave a comment.

2 comments

0
Avatar

Local disk persistence is used to ensure that if the entire GemFire XD cluster is shutdown the queued entries can be recovered and flushed to HDFS. If HDFS is unavailable the queued entries will remain in memory (and on disk if the queue is persistent) until HDFS comes back online. The queue is also fault tolerant--if a single node crashes another node will take over the queue of the failed node and flush any waiting entries.

Anthony Baker 0 votes
0
Avatar

Thank Anthony for your quick response !

Are the queue's entries really flushed to HDFS when it comes online ? Because I configured a HDFSTORE so that my queue persist on the local disk. I created a table having this HDFSSTORE, I turn off my HDFS, I continued to add data in my table (in HDFS Write-Only persistence case In HDFS Read/Write persistence case, i couldn't do it), when I restarted my HDFS, and after waiting several minutes, I found that there was no changes in my HDFSSTORE.

regards

Imane 0 votes