Local disk persistence is used to ensure that if the entire GemFire XD cluster is shutdown the queued entries can be recovered and flushed to HDFS. If HDFS is unavailable the queued entries will remain in memory (and on disk if the queue is persistent) until HDFS comes back online. The queue is also fault tolerant--if a single node crashes another node will take over the queue of the failed node and flush any waiting entries.
I wanted to know what is the role of Queue Persistence in GemFire XD Local Disk (in both HDFS Write-Only and Read/Write persistence), knowing that if HDFS crashes and restarts later, it can not receive datas sent by GemFire XD meantime, which are always stored in the queue (theoretically)?
Please sign in to leave a comment.
Thank Anthony for your quick response !
Are the queue's entries really flushed to HDFS when it comes online ? Because I configured a HDFSTORE so that my queue persist on the local disk. I created a table having this HDFSSTORE, I turn off my HDFS, I continued to add data in my table (in HDFS Write-Only persistence case In HDFS Read/Write persistence case, i couldn't do it), when I restarted my HDFS, and after waiting several minutes, I found that there was no changes in my HDFSSTORE.