Pivotal Knowledge Base

Follow

Downgrading version of RabbitMQ

Environment

Product Version
RabbitMQ

all

Symptom

If for any reason you need to downgrade RabbitMQ you will encounter a schema_integrity_check_failed issue where the mnesia database can not be downgraded as the database schema only has "forward" migration steps.

BOOT FAILED 
===========
Error description:
{could_not_start,rabbit,
{{schema_integrity_check_failed,
[{table_attributes_mismatch,rabbit_user,
[username,password_hash,tags],
[username,password_hash,tags,hashing_algorithm]}]},
{rabbit,start,[normal,[]]}}}
Log files (may contain more information):
/logs/rabbit/log/rabbit@abc01.log
/logs/rabbit/log/rabbit@abc01-sasl.log
{"init terminating in do_boot",
{could_not_start,rabbit,{
{schema_integrity_check_failed,
[{table_attributes_mismatch,rabbit_user,
[username,password_hash,tags],
[username,password_hash,tags,hashing_algorithm]}]},
{rabbit,start,[normal,[]]}}}

Cause

The mnesia database doesn't support backward migration steps. This will mean that all message data and mnesia directory would will need to be removed from all nodes

Resolution

To prevent any data loss due to having to downgrade you can follow these step to ensure no data loss.

  • Export definitions JSON (queues, exchanges, bindings, users, virtual hosts, permissions, and parameters) from current broker using the management plugin [1
  • Setup you new cluster with the older version of RabbitMQ
  • Import the JSON definitions for queues, exchanges, etc.
  • Using shovel (dynamic or static) move messages from current cluster to the new cluster.
  • This can be fully automated using the HTTP API [2]
  • PUT /api/parameters/shovel/%2f/my-shovel
    {"value":{"src-uri": "amqp://", "src-queue": "my-queue", "dest-uri": "amqp://remote-server", "dest-queue": "another-queue"}}
  • Ensure that consumers and publisher are pointed to the newly created (older version of RabbitMQ) cluster.
  • Once all message are drained from the previous cluster you can shut that down.

Note: This process can also be used to upgrade RabbitMQ and avoid any downtime [3]. 

[1] https://www.rabbitmq.com/management.html

[2] https://www.rabbitmq.com/shovel-dynamic.html

[3] https://www.rabbitmq.com/clustering.html#upgrading

 

Comments

Powered by Zendesk