Greenplum database running on RHEL 6 or 7
The purpose of this guide is to provide an overview of memory tuneables which are required or recommended to adjust on a Greenplum Database cluster.
As with all kernel tuneable changes, please engage with your system administrator team to discuss best practices specific to your environment.
The first section will be required changes, followed by recommended changes, followed by a more in depth overview of the effect of these tuneables.
Overview of Tuneables
As discussed in our documentation, it is required to adjust two tuneables related to virtual memory overcommitting.
vm.overcommit_memory = 2 (default is 0)
vm.overcommit_ratio = 95 (default is 50)
This additional memory tuning is recommended:
vm.swappiness = 10 (default is 30)
vm.dirty_expire_centisecs = 500 (default is 3000)
vm.dirty_writeback_centisecs = 100 (default is 500)
For systems with 64GB of memory or less:
vm.dirty_background_ratio = 3 (default is 10)
vm.dirty_ratio = 10 (default is 30)
For systems with more than 64GB of memory:
vm.dirty_background_bytes = 1610612736 (1.5 GB - Default is 0, as dirty_background_ratio is applied instead)
vm.dirty_bytes = 4294967296 (4GB - Default is 0, as dirty_ratio is applied instead)
Please apply all tuneables by adding the relevant line to /etc/sysctl.conf. You can then apply the changes with the command # sysctl -p, or by restarting the system. It is not recommended to apply these tuneables while the database is running - at the minimum, please stop the database, apply the tuneables, then restart the database.
This section provides some documentation and further discussion on the tuneables being recommended.
For an overview or the overcommit memory tuneables, please see this guide:
You can find a calculator which recommends memory tuning settings for the overcommit_ratio value, as well as the gp_vmem_protect_limit, here:
The swappiness tuneable is adjusted to be lower so that swap usage is still permissible, but not recommended. An algorithm determines the tendency of the OS to swap at any given memory pressure.
Dirty Pages Tuneables
The goal of the additional tuneables (dirty writeback and dirty ratio/bytes), is to create more active memory management which pushes dirty pages to disk more frequently, and in smaller, more manageable increments.
Because Greenplum Database does not use direct IO, the Linux Kernel will manage writing data to disk. When the database determines that a page should be written to disk, the memory address will be marked "dirty." The tuneables related to dirty writeback and dirty ratio/bytes determine how often a memory management thread is awoken to push data to disk, or alternatively the amount of memory which must be marked "dirty" before more extensive action is taken to flush dirty pages. Please see the documentation on kernel.org for more thorough discussion on these tuneables.
For additional overview, please read the kernel document pages on kernel.org: