Pivotal Knowledge Base

Follow

BOSH Director Fails to Compile Packages - "No Space Left on Device"

Environment

 Product  Version
 Pivotal Cloud Foundry  1.8 and above
 Bosh Director  All

Symptom

During an Apply Changes run on Ops Manager, the BOSH Director fails to compile packages and returns an error.

Error messages

BOSH Director failed to compile packages:

Error 100: No space left on device @ io_write - /var/vcap/store/director/tasks/13211/cpi

Failed compiling packages:

Failed compiling packages > mariadb/40260d929b64e065a5bdaac45ccaa75c2f26474b: No space left on device @ io_write - /var/vcap/store/director/tasks/13211/cpi

 

Cause

When you login to the BOSH Director and see that there is no space available, run the following command:

/:/var/vcap/store/director# df -k
Filesystem 1K-blocks Used Available Use% Mounted on
udev 4075584 4 4075580 1% /dev
tmpfs 817560 680 816880 1% /run
/dev/sda1 3030960 1462172 1397216 52% /
none 4 0 4 0% /sys/fs/cgroup
none 5120 0 5120 0% /run/lock
none 4087784 0 4087784 0% /run/shm
none 102400 0 102400 0% /run/user
/dev/sdb2 301452400 1319956 284796476 1% /var/vcap/data
tmpfs 1024 44 980 5% /var/vcap/data/sys/run
/dev/sdc1 11111111 11111111 11111111 100% /var/vcap/store

You could see your VM has no space as the disk space is up to:

100% /var/vcap/store

The normal size of the persistent disk is configured while installing the VM. You need to change the persistent disk size per your requirement. 

Dependency

Confirm that you have adequate disk space for your upgrades. You need at least 20 GB of free disk space to upgrade PCF Ops Manager and Elastic Runtime. If you plan to upgrade other products, the amount of disk space required depends on how many tiles you plan to deploy to your upgraded PCF deployment.

To check current persistent disk usage, select the Ops Manager Director tile from the Installation Dashboard. Select Status and review the value of the PERS. DISK column. If persistent disk usage is higher than 50%, select Settings > Resource Config, and increase your persistent disk space to handle the size of the resources. If you do not know how much disk space to allocate, set the value to at least 100 GB. 

Resolution

1. Go to Ops Manager Director Tile > Resource Config. See below:

Screen_Shot_2017-03-02_at_11.02.45_AM.png

2. See the memory that is the Persistent Disk Type. You need to increase the disk space based on your requirement.

Screen_Shot_2017-03-02_at_11.07.51_AM.png

 

Comments

Powered by Zendesk