Pivotal Knowledge Base


Backup disk store bloat with GemFire 7 when killing the GemFire processes

Applies to

GemFire 7.0.2.x and earlier


The purpose of this article is to describe why backup bloat can happen and how to avoid it.


When doing a full online backup the backup diskstore is bigger than the cache in memory plus the persisted data.

Gemfire creates 1 GB oplog files, but the filesystem does not necessarily allocate the space for the oplog. Every time the GemFire member is bounced it starts with a new oplog. So the old oplog would be left reporting it's size as 1 GB, even though GemFire maybe only wrote the first few hundred bytes and the rest of the file is unallocated space.

Because the filesystem didn't actually allocate that space, those 1 GB files wouldn't actually take up 1 GB as reported by the df command, but when copied all of the zeros will be included and the resulting file actually takes up 1 GB.

Check to see if this is the issue by doing an ls -l on each one of the copies of the disk store: online, backup. If the online disk store has a lot of 1 GB files that actually add up to a lot more that 6 GB when you look at what is reported by ls -l then you are probably experiencing this issue.


This issue was fixed in GemFire 8.0 with the fix of bug #44113.

Work Around

You only end up with an oplog with empty, unallocated space if you kill -9 the GemFire process. If shutting down the GemFire members using gfsh shutdown or Java commands this issue doesn't happen.


Powered by Zendesk