Pivotal Knowledge Base

Follow

How to Recover the out of Sync VMs on vSphere from a Failed Bosh Deployment

Environment

  • Pivotal Cloud Foundry versions 1.5.X and 1.6.X
  • vSphere versions 5.X and 6.X

Purpose

This article will assist in bringing back virtual machines (VMs) that are out of sync after a failed deployment.

'bosh cck' will show the following where a disk is not mounted on the problem VM:

Cause 

A bosh deployment can fail to cause the VMs' to go out of sync:

Started preparing deployment > Binding existing deployment. Failed: VM `vm-ef777cc9-9780-4a9e-a4f1-dbfe9bbd2e5a' is out of sync: expected to be a part of deployment `cf-9c65e8a93ceb573a2c18' but is actually a part of deployment `' (00:00:01)
Error 400003: VM `vm-ef777cc9-9780-4a9e-a4f1-dbfe9bbd2e5a’ is out of sync: expected to be a part of deployment `cf-9c65e8a93ceb573a2c18' but is actually a part of deployment `'

Procedure

Below are the steps to resolve this issue:

1. Shutdown the vm in your IaaS.

2. Run the following the following command to list the VM details:

    'bosh instances --details':

 3. Ensure that the Disk CID in the 'unknown' instance (shown in the screenshot above) matches the 'Disk File' in the VM in vSphere:

4. If the disk CIDs' matches, run 'bosh cck' and select option 4:'Delete VM reference'. This will delete the reference for this VM from the microbosh database (Note: This will not remove the VM from your infrastructure): 

5. Once completed, run 'bosh deploy' to continue with the deployment.

Comments

Powered by Zendesk