Pivotal Knowledge Base

Follow

Cannot assign IP Addresses to tile VMs as they are already assigned to Compilation VMs

Environment

 Product  Version
 Ops Manager  All

Symptom

You are deploying Elastic Runtime (or any other tile) and trying to assign the first 4 IP addresses in the range to some of the components.

Ops Manager fails to deploy with an error similar to:

IP '192.0.2.12' is already taken, cannot be taken for '<VM>'

If you inspect the installation.yml:

https://<OPSMAN_FQDN>/debug/files

The compilation VMs are consuming the first 4 IP addresses after Director:

    ip_assignments:
      assignments:
        p-bosh-bcd86fffbee1d58fe13c:
          director-f11b525cbd035b03d83f:
            4d6c37861885eec8c3c7:
            - 192.0.2.11
          compilation-526375e6a58c312f0b78:
            4d6c37861885eec8c3c7:
            - 192.0.2.12
            - 192.0.2.13
            - 192.0.2.14
            - 192.0.2.15

Cause

The Ops Manager workflow generates the p-bosh manifest before it generates manifests for other tiles. Ops Manager Director requires the IP address for the Director and the compilation VMs. If you have 4 compilation VMs (default) then the first 4 available IP addresses from the subnet will be assigned to the compilation VMs. Because of this if you try to use these IP addresses in another tile, Ops Manager will return an error stating that they are already taken.

Workaround

This is a workaround and is not guaranteed to work in future Pivotal Cloud Foundry releases.

This procedure assumes you are using a clean install of Ops Manager where the Ops Manager Director tile has not yet been deployed.

In this example we will create the following configuration:

CIDR: 192.0.2.0/24
Reserved IP Ranges: 192.0.2.1-192.0.2.10 // Assuming we have other VMs outside of PCF using these IPs Ops Manager Director: 192.0.2.11 Compilation VMs: 192.0.2.241, 192.0.2.242, 192.0.2.243, 192.0.2.244

Steps:

  1. Go to the Create Networks page
  2. Click Add Network
  3. For CIDR enter:
    192.0.2.0/24
  4. For Reserved IP Ranges enter:
    192.0.2.1-192.0.2.10,192.0.2.12-192.0.2.240
    Note: This will force Ops Manager to select .11 for Director and .241 to .244 to the compilation VMs. You can use whatever exclude range necessary to force the compilation VMs to use specific IPs.
  5. Configure the other network options as required
  6. Click on Save
  7. Configure the rest of Ops Manager Director tile as required
  8. Browse to the debug page:
    https://<OPSMAN_FQDN>/debug/files
  9. Scroll to installation.yml and verify the IP assignments section:
        ip_assignments:
          assignments:
            p-bosh-bcd86fffbee1d58fe13c:
              director-f11b525cbd035b03d83f:
                4d6c37861885eec8c3c7:
                - 192.0.2.11
              compilation-526375e6a58c312f0b78:
                4d6c37861885eec8c3c7:
                - 192.0.2.241
                - 192.0.2.242
                - 192.0.2.243
                - 192.0.2.244
        
  10. Apply changes
  11. Modify the Reserved IP Ranges and remove the extra reservations that were added:
    192.0.2.1-192.0.2.10
  12. Apply changes

The compilation VMs will keep the 192.0.2.241-192.0.2.244 IP addresses.
Other IP addresses in the range are now free to be used by other tiles.

Additional Information

  • To have Ops Manager respect the statically defined IP addresses across all deployments before calculating the automatically assigned IP addresses would require major architectural changes.
  • The current plan is to resolve this in a future Pivotal Cloud Foundry (PCF) release where Ops Manager will not assign IP addresses. BOSH Director will assign IP addresses dynamically or a PCF operator would be able to assign static IP addresses.

Comments

Powered by Zendesk