GemFire 7 - 8.2
The javadocs for online disk store compaction-threshold imply that GemFire compact as soon as the amount of garbage is equal or greater than the threshold.
When the amount of garbage in an oplog exceeds this percentage then when a compaction
* is done this garbage will be cleaned up freeing up disk space. Garbage is created by
* entry destroys, entry updates, and region destroys.
However GemFire actually trigger compaction when the amount of non-garbage data is equal or less than the threshold instead so online disk store compaction appears not to be working
If you set a threshold of 20% then GemFire doesn't trigger compaction until the *live* data is less than 20%. If you need a threashold of 20% you should be setting the threshold to 80% instead. This should be fixed in version 8.5 of GemFire