Pivotal Knowledge Base

Follow

How to use a dual-homed Director with Ops Manager

Environment

Product Version
Pivotal Cloud Foundry® (PCF)  1.6.x and 1.8
Pivotal Cloud Foundry® (PCF)  1.7, but only with Ops Manager 1.7.8+

Purpose

In most cases, it is sufficient to use a single network interface when deploying the Ops Manager Director. This works at the networking layer by enabling routing between the Ops Manager Director's network and the networks of the IaaS and of the networks where the Ops Manager Director will be deploying VMs.

In some cases, it is not possible to route traffic to the IaaS and the Ops Manager Director must have a network interface on a special network to communicate with the IaaS.  For these cases, you can dual-home the Director and deploy it with two network interface cards.

Procedure 

In PCF 1.6, all versions of Ops Manager support use a dual-home director.  In PCF 1.7, and Ops Manager versions 1.7.0 - 1.7.7, this feature was removed.  It is present in PCF 1.7.8 and up.  This KB article is broken down into two scenarios: upgrading from PCF 1.6 when using a dual-homed setup and configuring a new PCF 1.7 environment with a dual-homed setup.

Upgrade from PCF 1.6

The instructions in this section are applicable if the following criteria apply:

  • You have configured Ops Manager Director to have two different networks, deployments, and infrastructure.
  • The deployment network is where all the products are deployed.
  • The infrastructure network is where Ops Manager and Ops Manager Director are deployed.

To perform the upgrade:

  • Export your current configuration from Ops Manager 1.6.
  • Import the dual-homed Ops Manager 1.6 "installation.zip" into OpsManager 1.7.8 or greater.  
    • Note that the Ops Manager 1.7.8+ UI will only show the deployment network listed in the “Assign AZs and Networks” form, but it will retain the infrastructure network setting. You can look at the generated BOSH manifest file to confirm that the BOSH Director actually has both networks (the ones that were defined in their 1.6 environment). The Create Networks screen will also show both networks as they were defined in Ops Manager 1.6.
  • Hit "Apply Changes" in Ops Manager.
  • To verify, connect via SSH to the Director VM and run "ifconfig".  It will report that the Director has two NICs.

New install of PCF 1.7

The instructions in this section are applicable if the following criteria apply:

  • You have decided that you need to use a dual-homed Ops Manager Director (please see the Purpose section above for details on why you might need to do this).
  • You are performing a new install of PCF with Ops Manager 1.7.8 or greater.

To configure Ops Manager so that it deploys the Ops Manager Director with two networks, follow these steps:

  • Go to the Ops Manager Director tile and click the Create Networks screen.  Define two networks.
  • On the “Assign AZs and Networks” screen, pick the deployment network.
  • Connect via SSH to the Ops Manager VM and run the following commands:

    uaac target https://<fqdn-ops-manager>/uaa

    uaac token owner get

    When prompted, enter opsman for the client and a blank value for the client secret. The username and password will be the username and password that you use for the admin account of the Ops Manager web UI.

    uaac curl https://<fqdn-ops-manager>/api/v0/staged/director/second_network -k -X PUT -H "Content-Type:application/json" -d '{"second_network": {"name": "second-network-name"}}'

    Where second_network is the name of the Infrastructure network. The expected response is a 200 OK API response. If you get a 4xx or 5xx value, please contact customer support.
  • Back in the Ops Manager UI, click Apply Changes.
  • To verify, connect via SSH to the Director VM and run "ifconfig". It will report that the Director has two Network Interface Cards (NICs). 

Impact/Risks

  • When possible, it is suggested to not use a dual-home setup. It adds complexity to the setup and unless that is necessary, we suggest that you do not take on the extra complexity. Instead, as described above, manage this at the network layer with routing and firewall rules.

  • With PCF 1.6 & Ops Manager 1.6, there are known problems with some environments where configuring a dual-homed Ops Manager Director results in asymmetric routing failures. These cases are not supported in 1.6, but should work with a new install in 1.7.

  • Prior to planning the upgrade, please make sure that you have sufficient IP addresses on the infrastructure network as the Ops Manager Director is assigned to this network and this is where its compilation VMs will run. 

Additional Information 

Here is a procedure to remove the dual homed NIC in PCF 1.7.8 or later.   This procedure can be applied If for any reason there are problems with deploying a dual homed NIC Director or you simply would like to remove the Directors second network.

  • Send api call to Operations Manager to delete the second_network field from the director config
uaac curl https://<fqdn-ops-manager>/api/v0/staged/director/second_network -k -X DELETE -H "Content-Type:application/json"
  • If DELETE call was successful then proceed to click apply changes in the Operations Manager UI.
  • To verify, connect via SSH to the Director VM and run "ifconfig". It will report that the Director has one Network Interface Cards (NICs). 

Comments

  • Avatar
    Todd Robbins

    With a dual-homed OpsManager, the compilation VM's will get created on infrastructure network. This could be worth noting above.

Powered by Zendesk