Pivotal Knowledge Base

Follow

How to remove segments from Pivotal HDB

Environment

Product Version
Pivotal HD (PHD) 3.0.x
Pivotal HDB 1.3.x
Ambari 1.7.1 / 2.1.2

Purpose

This article discusses the options available when the number of segments need to be reduced in Pivotal HDB 1.3.1.

NOTE: This KB ONLY applies to 1.3.x version of HAWQ. If you need to shrink a 2.x HAWQ cluster, please see the Pivotal HDB Documentation.

Cause

There is a need to reduce the amount of segments in a Pivotal HDB cluster.

Procedure

There is no option to remove a segment in Pivotal HDB or HAWQ 1.3.x, so the only two ways to reduce the number of segments in a Pivotal HDB 1.3.x cluster are: 

a) Migrate Data

Migrate data to a new cluster with less segments using the tools available with Pivotal HDB.

Or

B) Re-initialize the Pivotal HDB Cluster

WARNING: All data and configurations in HAWQ will be lost if this method is used.

This is the process to be followed:

1. Backup all Pivotal HDB data.

2. Stop all Pivotal HDB services via Ambari.

3. Remove Pivotal HDB data on the local file system and in HDFS.

4. Remove HAWQ from Ambari via the Ambari API with the following commands (replace AMBARI_FQDN with the Ambari server hostname and CLUSTER_NAME with the name of the cluster):

#STOP THE HAWQ SERVICE
curl -u admin:admin -H "X-Requested-By: ambari" -X PUT -d '{"RequestInfo":{"context":"Stop Service"},"Body":{"ServiceInfo":{"state":"INSTALLED"}}}' http://AMBARI_FQDN:8080/api/v1/clusters/CLUSTER_NAME/services/HAWQ

#STOP THE MASTER AND STANDBY HOST COMPONENTS:
curl -u admin:admin -H "X-Requested-By: ambari" -X PUT -d '{"RequestInfo":{"context":"Stop Component"},"Body":{"HostRoles":{"state":"INSTALLED"}}}' http://AMBARI_FQDN:8080/api/v1/clusters/CLUSTER_NAME/hosts/HAWQ_MASTER_FQDN/host_components/HAWQMASTER
curl -u admin:admin -H "X-Requested-By: ambari" -X PUT -d '{"RequestInfo":{"context":"Stop Component"},"Body":{"HostRoles":{"state":"INSTALLED"}}}' http://AMBARI_FQDN:8080/api/v1/clusters/CLUSTER_NAME/hosts/STANDBY_MASTER_FQDN/host_components/HAWQSTANDBY

#STOP THE HAWQ HOST SEGMENTS (REPEAT FOR EACH SEGMENT HOST):
curl -u admin:admin -H "X-Requested-By: ambari" -X PUT -d '{"RequestInfo":{"context":"Stop Component"},"Body":{"HostRoles":{"state":"INSTALLED"}}}' http://AMBARI_FQDN:8080/api/v1/clusters/CLUSTER_NAME/hosts/SEGMENT_FQDN/host_components/HAWQSEGMENT

#STOP EACH SERVICE COMPONENT:
curl -u admin:admin -H "X-Requested-By: ambari" -X PUT -d '{"RequestInfo":{"context":"Stop All Components"},"Body":{"ServiceComponentInfo":{"state":"INSTALLED"}}}' http://AMBARI_FQDN:8080/api/v1/clusters/CLUSTER_NAME/services/HAWQ/components/HAWQSEGMENT
curl -u admin:admin -H "X-Requested-By: ambari" -X PUT -d '{"RequestInfo":{"context":"Stop All Components"},"Body":{"ServiceComponentInfo":{"state":"INSTALLED"}}}' http://AMBARI_FQDN:8080/api/v1/clusters/CLUSTER_NAME/services/HAWQ/components/HAWQMASTER
curl -u admin:admin -H "X-Requested-By: ambari" -X PUT -d '{"RequestInfo":{"context":"Stop All Components"},"Body":{"ServiceComponentInfo":{"state":"INSTALLED"}}}' http://AMBARI_FQDN:8080/api/v1/clusters/CLUSTER_NAME/services/HAWQ/components/HAWQSTANDBY 

#REMOVE THE HAWQ SERVICE: 
curl -u admin:admin -H "X-Requested-By: ambari" -X DELETE http://amb171hawq:8080/api/v1/clusters/amb171hawq/services/HAWQ

5. Log out of Ambari then log back in and confirm that HAWQ is no longer seen in the Ambari GUI under the installed services.

6. Re-install HAWQ via Ambari and select the number of segments required in the new installation.

7. Restore data to the new Pivotal HDB instance with less segments.

 

Comments

  • Avatar
    Yadukula

    Adam,

    Thanks for documenting this. It is very helpful!

    Regards,
    Yadu

Powered by Zendesk