Pivotal Knowledge Base


MySQL Monitor VM shows High Memory Usage


Pivotal Cloud Foundry (PCF) versions 1.10, 1.11, 1.12, 2.0


MySQL Monitor Node shows high memory usage, caused by the replication_canary process and possibly the black box process after some time. 


The replication_canary process is the cause of the high memory usage. A bug causes this process to overwrite log files rather than appending to them, which then causes problems when a 'logrotate' occurs on the 'replication-canary.stdout.log' file. The black box process may also contribute to the high memory usage as it follows the large log file created by replication_canary.


Upgrade to the version of MySQL for Elastic Runtime (ERT) or Pivotal Application Service (PAS) which includes the mysql-monitoring release version 8.14.0. Below are the prerequisites:

  • Elastic Runtime versions 1.11.25+, 1.12.13+
  • PAS versions 2.0.4+, 2.1+
  • MySQL for PCF versions 1.9.18+, 1.10.11+

Note: If you are unable to upgrade, try the following steps manually:

  1. BOSH SSH to the mysql_monitor VM
  2. Edit /var/vcap/jobs/replication-canary/bin/replication-canary_ctl and remove the logLevel property.
  3. Restart the 'monit' services for the replication-canary and black box.

This should prevent the high memory utilization, however, the workaround will need to be carried out if the VM is recreated/upgraded.

Additional Information

Refer to these documents for guidance


Powered by Zendesk