Pivotal Knowledge Base

Follow

How To Clean up Temporary Hive Folders

Environment

 Product  Version
 Pivotal HD, Pivotal HDP  2.x, 3.x
 OS  RHEL 6.x

Purpose

If the Hive CLI is being stopped incorrectly, Hive would not clean up its temporary directories after finishing its queries. Files and folders may be left in /tmp on the local file system and HDFS. This article tells you how to clean up Hive temporary files and folders to free up the client host disk space.

This article helps you to clean up hive temporary files and folders to free-up the client host disk space.

Cause 

The host is running out of disk space because of the large size of the /tmp folder.

Procedure

This script cleans up data in the /tmp folder that is older than N seconds:

find /tmp -mmin +N -exec rm {} \;

For example, this command will find files in /tmp that are older than 1 hour and remove them:

find /tmp -mmin +3600 -exec rm {} \;

IMPORTANT NOTE: This command will permanently remove files so it should be used with caution.

The command can be put into a cron-job so that it can run regularly on the system. 

NOTE: Temporary files may also be created in HDFS in /tmp and they might need to be cleaned up periodically also.

Comments

Powered by Zendesk