The lucid64 stack, derived from Ubuntu Lucid 10.04, which has been part of Cloud Foundry for several years as the root file system for containers, will reach Canonical’s end of support for security fixes on April 29th, 2015.
Pivotal Web Services is migrating to a supported replacement stack called cflinuxfs2, derived from Ubuntu Trusty 14.04. This article explains who is affected by this change and what actions you need to take.
Who is affected
In order to avoid application downtime, developers with applications hosted on Pivotal Web Services must re-push their applications to use the new cflinuxfs2 stack by May 27, 2015 11:59 PM PDT.
The following timeline covers important milestones and the steps you can take to use the new stack.
April 10, 2015: cflinuxfs2 stack is available on PWS
- New applications can be pushed to use the new cflinuxfs2 stack.
- Existing applications can be re-pushed using the new cflinuxfs2 stack.
April 16, 2015 - May 27, 2015: cflinuxfs2 is the default stack on PWS
- New applications will use the new cflinuxfs2 stack by default.
- Developers with existing applications can choose to re-push using the new cflinuxfs2 stack.
May 28, 2015: all applications will be using cflinuxfs2
- Existing applications still using the lucid64 stack will be automatically re-deployed to use the new cflinuxfs2 stack. Note that this re-deployment may result in downtime as your application is re-staged.
- Stopped applications will be updated to use the new stack and will stage on the new stack if/when they are started in the future.
While we do not anticipate any issues with the new stack, we suggest deploying a separate instantiation of your application using the new stack to evaluate if there is any impact to your application. This can be done by pushing your existing application to a new name with a separate route. If you have questions about how to do this, please contact firstname.lastname@example.org. Once you have verified that your application has not been impacted, redeploy the app using the new cflinuxfs2 stack. Note that once an application has been deployed using the cflinuxfs2 stack, future pushes will remember the stack selection.
Below are detailed instructions regarding how to perform the various steps to migrate your application.
Step #1 - Determine your current stack
There are two ways that you can determine which stack the application is currently using.
Option #1 - via App Manager
Visit Pivotal Web Service’s Apps Manager UI at https://console.run.pivotal.io and log in. Navigate to your application details page. You will see a STACK field in the About section.
Option #2 - via command line tool
Make sure that you have at least cf version 6.11 (click here for the latest version) and run the following command:
cf app <app-name>
The output will include a line starting with stack:, what follows is the name of the stack currently being used.
Regardless of the way you find the current stack (cli or App Manager), if you see “Ubuntu 10.04” or "lucid64" listed then your app is using the soon-to-be-deprecated lucid64 stack. If you see “Cloud Foundry Linux-based filesystem” or "cflinuxfs2" then you are using the new cflinuxfs2 filesystem and no further action is required.
Step #2 - Determine if your build pack supports the new stack
All of the officially supported build packs support cflinuxfs2. Here is the list of officially supported build packs.
Custom and forked build packs should be tested with the new stack to ensure compatibility, particularly around the use of precompiled binaries, or the re-use of compiled staging artifacts that are stored in the “application cache”. Failure, should the buildpack be incompatible, will generally take the form of failure during staging, or failure to start the app.
Precompiled Binary Dependencies
If your custom buildpack is delivering precompiled binary dependencies, then you may need to package binaries for both stacks in order to accommodate the differences between the rootfses. Many binaries will work on both stacks; some will not.
If your custom buildpack is not forked from one of the recent supported buildpacks, you may need to add logic to use the appropriate binaries for the stack on which you’re staging and depoying. See the section below on the CF_STACK environment variable for details.
Instructions for generating stack-specific binaries for your custom build pack are out of scope for this document.
During staging, an “application cache” directory is mounted, which is commonly used to save staging artifacts (e.g., C extensions to the Node or Ruby interpreters) so that subsequent stagings don’t have to regenerate those artifacts.
When restaging your app onto a different stack, you may need to bust the cache to avoid using binary artifacts that were compiled for a different stack. Again, you should be able to inspect the value of the CF_STACK environment variable to add this behavior if your buildpacks is not (very-)recently forked from one of the supported buildpacks.
Inspecting The Stack
Buildpacks can inspect the value of the CF_STACK environment variable during staging to implement any stack-aware logic. Currently the values of this will be either lucid64 or cflinuxfs2.
Step #3 - Deploy an instance of your application using the new stack
Simply run this command.
cf push <APP_NAME> -s cflinuxfs2
Impact / Risks
The following risks are outlined above and included in summary below.
- The end of support for the Ubuntu Lucid stack is April 29th, 2015. Using this stack after that date could result in using a stack with known vulnerabilities.
- Build packs must specifically support the cflinuxfs2 stack. This generally requires that binaries used by the build must be compiled for the new environment, specifically if the binaries link to shared libraries that are provided by the stack. The most common example of this are binaries shipped with build packs that use OpenSSL. The OpenSSL libraries are provided by the stack and have been upgraded from lucid64 to cflinuxfs2.
- If your application utilizes or contains any binaries that have been precompiled you may need to recompile those binaries. The same rationale that applies to build pack provided binaries, applies to application provided binaries as well.
- If you have forked one of the official build packs, you need to make sure that you have recently pulled in upstream changes. The upstream changes should include support for the new stack. If you have not done this, it's likely that the build pack will not work on the cflinuxfs2 stack.
- If you do not test your application with the new cflinuxfs2 stack prior to April 16th, 2015, you may experience issues the next time you push to PWS. This is because after April 16th, the default stack will now be the cflinuxfs2 stack and your app will automatically start using it. If this happens, you can manually switch back to the lucid64 stack with the `-s` argument to `cf push`.
- Applications that are not running on the cflinuxfs2 stack by May 28th, 2015 will automatically be switched to the new stack. This may result in downtime for your application, both as the application is restaged and if the application fails to run on the new stack. Follow the timeline above to prevent this from happening.
The cf cli supports the use of third party plugins, and the CF App Stack Changer plugin was written to help users with large numbers of applications migrate stacks. If you have a lot of applications that need migrated or want to query the stack being used by your existing applications, this tool may be of assistance.
Please note, third party plugins are not supported by Pivotal. Questions or problems with a plugin should be reported to the plugin's author.