Pivotal Knowledge Base

Follow

Oozie server showing as stopped in Ambari immediately after trying to start up the service via Ambari

Environment

Product Version
Pivotal HD 3.x
Oozie  4.1.x

Symptom

Oozie server appears to start via Ambari:

But after a 1-2 minutes Ambari will show oozie server as stopped:

Error Message:

/var/log/oozie/catalina.out will show errors similar to this:

Caused by: java.io.FileNotFoundException: /var/log/oozie/oozie.log (Permission denied)
Caused by: java.io.FileNotFoundException: /AYCAP/hadoop/oozie/data/oozie-db/log/log2.dat (Permission denied)
at java.io.RandomAccessFile.open(Native Method)
at java.io.RandomAccessFile.<init>(RandomAccessFile.java:241)
at org.apache.derby.impl.io.DirRandomAccessFile.<init>(Unknown Source)
at org.apache.derby.impl.io.DirFile4.getRandomAccessFile(Unknown Source)
at org.apache.derby.impl.store.raw.log.LogToFile.run(Unknown Source)
at java.security.AccessController.doPrivileged(Native Method)
at org.apache.derby.impl.store.raw.log.LogToFile.privRandomAccessFile(Unknown Source)
at org.apache.derby.impl.store.raw.log.LogToFile.getLogFileAtPosition(Unknown Source)
at org.apache.derby.impl.store.raw.log.LogToFile.getLogFileAtBeginning(Unknown Source)
at org.apache.derby.impl.store.raw.log.Scan.getNextRecordForward(Unknown Source)
Caused by: java.sql.SQLException: Failed to start database '/AYCAP/hadoop/oozie/data/oozie-db' with class loader WebappClassLoader
context: /oozie
delegate: false
repositories:
/WEB-INF/classes/
----------> Parent Classloader:
org.apache.catalina.loader.StandardClassLoader@3b3eb5bd
, see the next exception for details.
at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
at org.apache.derby.impl.jdbc.SQLExceptionFact

Cause

Oozie was started as user root at somepoint in the past and this caused incorrect permissions to be set on a number of log and database files.

Resolution

All running oozie processes need to be stopped and correct permissions set on the files:

1. As user root try to stop oozie processes:

/usr/phd/current/oozie-server/bin/oozied.sh stop

2. Use ps -eaf | grep oozie and kill any remaining oozie processes

3. Find the value of configuration "Oozie Data Dir" and " Oozie Log Dir" via Ambari.

4. Correct permissions on the two directories with the following commands:

chown -R oozie:hadoop <oozie data dir>/oozie-db/log
chown -R oozie:hadoop <oozie data dir>/oozie-db/seg
chown -R oozie:hadoop <oozie data dir>/oozie-db/tmp
chown -R oozie:hadoop <oozie log dir>
chmod -R 644 oozie:hadoop <oozie data dir>/oozie-db/log
chmod -R 644  oozie:hadoop <oozie data dir>/oozie-db/seg
chmod -R 644 oozie:hadoop <oozie data dir>/oozie-db/tmp
chmod -R 644 oozie:hadoop <oozie log dir>

NOTE: the oozie-db will only exist and contain data if the derby database is being used.

5. Start Oozie via Ambari

Comments

Powered by Zendesk