Pivotal Knowledge Base

Follow

Migrating Applications from DEAs to Diego

Purpose

Pivotal Web Services (PWS) is undergoing an important system upgrade. PWS is composed of several system components; the component responsible for running applications is currently known as a DEA. The DEA is being upgraded to a new runtime known as Diego, which is the new state-of-the-art application execution component that will power PWS going forward.

When this happens, any application not already being run by Diego will be automatically migrated by PWS.  This may result in some downtime for the application.  To avoid downtime for your application, follow the steps outlined in this article.

Migration Schedule

The instructions for migrating your app vary a bit based on the point in time when you go to make the switch.  The timeline below lists critical dates during the migration and how those affect the migration instructions.

  • October 15, 2015
    • Migration begins.  Developers are encouraged to re-deploy their application to Diego. 
    • DEAs remain the default for new application deployments.
    • Existing applications continue to run on the DEAs unless explicitly migrated.
  • October 19, 2015
    • Diego becomes the default application execution environment on PWS.
    • New applications run on Diego by default when pushed.
    • Developers can issue the cf disable-diego command to force an application to run on the DEAs.
  • November 2, 2015
    • Any application not manually moved to Diego, will be automatically moved to Diego.  This may result in up to 1 minute of downtime for your application.
    • Developers can no longer force an application to run on the DEAs

Instructions

Preparing to Migrate

Before you can switch your existing application to run on Diego, the first thing that you need to do is to install the Diego plugin for the cf client.  If you have not done so already, start by installing at least version 6.13 or later of the cf cli.  Then run the following commands, which will install the Diego plugin.

$ cf add-plugin-repo CF-Community http://plugins.cloudfoundry.org/
$ cf install-plugin Diego-Enabler -r CF-Community

With the cf client installed, you’re now ready to migrate your application to Diego.  While most applications will run fine on Diego, there are some differences between the DEAs and Diego.  As such, Pivotal recommends that you review the Additional Information section below to see how these changes might affect your application, prior to making the switch.  Once you have read through the changes, please proceed to the next section.

Migrating to Diego

If your application can tolerate up to a minute of of downtime, simply run the following command:

$ cf enable-diego your-app-name

This command will immediately instruct the Diego backend to run the required number of instances of your application. Within about a minute, the system will then stop the old instances of your application that were running on the DEAs.  This may occur without downtime, but the system does not guarantee it.

If your application cannot tolerate any downtime, you should use the blue-green deployment technique.  The process would look like this:

  1. Push your application to PWS with a new name and a new route.  This new application will run on Diego, since that is the default. Make sure to use the same version of your application that is currently deployed so that static assets like css and javascript that may be served from your application are consistent across both versions.
  2. Wait for the cf cli to report that your application has started.  Test your application and make sure it is working as you would expect.  If you encounter problems, refer to the migration guide listed in the Additional Information section below or contact PWS Support.
  3. Once you have confirmed that the new application deployed to Diego is operational, map the route for your application running on the DEAs to your application running on Diego.  This will result in a second route with half of your traffic going to the DEA’s app and half to Diego’s.

    Ex:
    $ cf map-route new-app my-domain.com -n www
  4. Unmap the route from the application running on the DEAs.  This will send all of the traffic to the application that’s running on Diego.

    Ex:
    $ cf unmap-route old-app my-domain.com -n www
  5. Stop the application that’s running on the DEAs. At this point, all of your traffic will be going to your app running on Diego.  Again, verify that your application is working properly.  Should you encounter any issues, you can rollback to your application on the DEAs by starting it and reversing the route map and unmap commands.

    Ex:
    $ cf start old-app
    $ cf map-route old-app my-domain.com -n www
    $ cf unmap-route new-app my-domain.com -n www
    Once you are satisfied that your application is working properly running on Diego, then you can delete the old application.

    Ex:
    $ cf delete old-app

Post Migration

Once you have migrated your applications to Diego, you no longer need the Diego-Enabler plugin.  This plugin is only necessary to migrate existing applications that are running on DEAs to Diego.  If you're pushing a new application, PWS will default to Diego and so you won't need this plugin.

Once you are done migrating your apps, you can remove the plugin by running this command.

$ cf uninstall-plugin Diego-Enabler

Impact / Risks

If you do not manually migrate your application to Diego prior to November 1, 2015 11:59 PM PDT, the system will automatically migrate your application.  The automatic migration will function as if you ran cf enable-diego <your-app-name>, which means there could be up to a minute of downtime during the migration.

Additional Information

Because Diego was written to be compatible with the DEA, we expect that most applications that were running on the DEA will continue to run on Diego without change. That said, there are some differences between the two components.  Here are the changes that we think will be commonly noticed.

  • The cf files command will no longer work.  The command still exists in the cf client, but running it will end up with an error saying Request failed for app: <app-name>, instance: <inst-num> and path: <path> as the instance is not found.  The new cf ssh <app-name> command can be used to list files in your application instances.
  • Worker applications, those that are pushed with the --no-route option, will likely fail when run on Diego.  This is because you must explicitly disable the port-based health check when running on Diego.  This is done by running cf set-health-check app-name to none.
  • Environment variables are no longer interpolated.  Previously you could set an environment variable with cf set-env or in your manifest file and reference other environment variable.  For example, “SOMEPATH=$HOME/some/path” and the result would be “SOMEPATH=/home/vcap/app/some/path”.  This is no longer possible.
  • Large applications may fail to stage as there is a known issue where disk usage is over-reported.  This typically manifests with the error "Copying into the container failed".  By default apps run on PWS have a 1GB disk quota, if your application files are close to this size, please use the -k option of cf push to increase your disk quota.  The disk quota on PWS can go up to 2GB.
  • Staging may take a bit longer when you do not specify a specific build pack.  The reason for this is explained here.  You can mitigate this by specifying a build pack with -b or in your manifest.yml file.
  • VCAP_APP_HOST and VCAP_APP_PORT environment variables have been removed.  VCAP_APP_HOST was always 0.0.0.0, so you can safely use that in place of the environment variable.  VCAP_APP_PORT is replaced by PORT and is set to 8080 by default.
  • Diego imposes a 4KB limit on the maximum size of an application's routes.   While this doesn't translate to a specific number of routes, as it would depend on how many characters are in each routes, we are estimating that gives you space for 40 to 50 medium sized (50 character) routes.  If you have too many routes bound to your application, you'll see the error Runner error: desire app failed: 503.  In some cases you can work around this by using a wildcard route (i.e. *.my-domain.com) instead of mapping individual routes.  When that is not possible, the only other option is to run multiple applications mapping 40 to 50 routes to each application.
  • Eclipse, STS, Maven, Gradle and anything else based off of the the Java cf-client will need to install the standard cf client to migrate, as the Java cf-client does not support the migration command at this time.  The Java client also does not support disabling or enabling Diego health checks.  If you have a worker application, like the second item above, then you will need to install the cf client to disable the health check.  Migration to Diego and disabling the health check only need to be done once per application.  Once migrated or disabled, the system will remember the setting until the application is deleted.

To read more about these and the other changes that might affect your application as it migrates to Diego, please see this migration guide.

Comments

Powered by Zendesk