Pivotal Knowledge Base

Follow

Pivotal HDB Initialization or reload fails with OSError: [Errno 12] Cannot allocate memory

Problem

You may run into memory issues while initializing Pivotal HDB or performing other management operation like reloading configuration files using gpstop -u.

The error message may look similar to the below, but the key is "Cannot allocate memory"

20131126:14:09:45:169464 gpstart:cvlppvtlan4priv:gpadmin-[ERROR]:-[Errno 12] Cannot allocate memory
Traceback (most recent call last):
  File "/usr/local/hawq/lib/python/gppylib/commands/base.py", line 202, in run
    self.cmd.run()
  File "/usr/local/hawq/lib/python/gppylib/commands/base.py", line 687, in run
    self.exec_context.execute(self)
  File "/usr/local/hawq/lib/python/gppylib/commands/base.py", line 640, in execute
    LocalExecutionContext.execute(self,cmd)
  File "/usr/local/hawq/lib/python/gppylib/commands/base.py", line 421, in execute
    stdout=subprocess.PIPE, close_fds=True)
  File "/usr/local/hawq/ext/python/lib/python2.6/subprocess.py", line 595, in __init__
    errread, errwrite)
  File "/usr/local/hawq/ext/python/lib/python2.6/subprocess.py", line 1009, in _execute_child
    self.pid = os.fork()
OSError: [Errno 12] Cannot allocate memory
Cause
 
Such situation usually occur if a server has low RAM and SWAP and the processes running might be requesting more what could be allocated.
 
Workaround

You may workaround the above issue, by changing the value in /proc/sys/vm/overcommit_memory to 1 or 0 if it is set to 2 in the file. 

overcommit_memory: This value contains a flag that enables memory overcommitment.
- When this flag is 0, the kernel attempts to estimate the amount of free memory left when userspace requests more memory.
- When this flag is 1, the kernel pretends there is always enough memory until it actually runs out.
- When this flag is 2, the kernel uses a "never overcommit" policy that attempts to prevent any overcommit of memory.

Note: In case, the value is 1 or 0, then you have to definitely consider options to increase RAM / SWAP space to get back to a setting of 2 since that's recommended and identify if there is any other unnecessary process which is eating up the memory.
Please refer to the article https://support.pivotal.io/hc/en-us/articles/202703383 for recommended overcommit setting and more on it.
 

Comments

Powered by Zendesk