This article will provide instructions on recovery MySQL for Pivotal Cloud Foundry Galera Cluster when the entire cluster crashes. Such situation may happen in case of datacenter power failure, virtual machines went down due to IaaS failure, etc.
In this case, we are not sure if all nodes were consistent with each other, hence it is crucial to find the most advanced one in order to bootstrap the cluster using it. Before starting MySQL instances on any node, we have to extract the highest sequence number and then bootstrap from the node.
Here are the steps to find out sequence number and restore the Galera Cluster.
In "Bootstrapping" section, it is mentioned that the operation requires root permission. As an easy alternative, setup BOSH CLI on Ops Manager and use it as a jumpbox, then run 'bosh ssh' to login to MySQL nodes. Switch account to root by issuing 'sudo -i'. Please find detailed steps to setup BOSH CLI jumpbox at Advanced Troubleshooting with the BOSH CLI.