Flink 1.2 does not start in HA Cluster mode - apache-flink

I've installed Flink 1.2 in HA cluster mode 2 JobManagers 1 TaskManager locally and it kept refusing to actually start in this mode showing
"Starting cluster." message instead of
"Starting HA cluster with 2 masters and 1 peers in ZooKeeper quorum."
Apparently in the bin/config.sh it reads the configuration like:
# High availability
if [ -z "${HIGH_AVAILABILITY}" ]; then
HIGH_AVAILABILITY=$(readFromConfig ${KEY_HIGH_AVAILABILITY} "" "${YAML_CONF}")
if [ -z "${HIGH_AVAILABILITY}" ]; then
# Try deprecated value
DEPRECATED_HA=$(readFromConfig "recovery.mode" "" "${YAML_CONF}")
if [ -z "${DEPRECATED_HA}" ]; then
HIGH_AVAILABILITY="none"
elif [ ${DEPRECATED_HA} == "standalone" ]; then
# Standalone is now 'none'
HIGH_AVAILABILITY="none"
else
HIGH_AVAILABILITY=${DEPRECATED_HA}
fi
else
HIGH_AVAILABILITY="none"
fi
fi
which means independently of what is configured for "high-availability" key in the configuration file (in my case value was "zookeeper") it will set that to "none" and in bin/start-cluster.sh
if [[ $HIGH_AVAILABILITY == "zookeeper" ]]; then
# HA Mode
readMasters
echo "Starting HA cluster with ${#MASTERS[#]} masters."
for ((i=0;i<${#MASTERS[#]};++i)); do
master=${MASTERS[i]}
webuiport=${WEBUIPORTS[i]}
ssh -n $FLINK_SSH_OPTS $master -- "nohup /bin/bash -l \"${FLINK_BIN_DIR}/jobmanager.sh\" start cluster ${master} ${webuiport} &"
done
else
echo "Starting cluster."
# Start single JobManager on this machine
"$FLINK_BIN_DIR"/jobmanager.sh start cluster
fi
will never get in the first if branch.
Anyone else faced this?

Yes I believe it is a bug: issues.apache.org/jira/browse/FLINK-6000.
It has already a pending PR.

Related

How can I cycle through an array in bash while also passing an argument to the script?

I have the following bash script I want to use as my "standard browser" with xdg-open.
It should prompt dmenu for me to choose the browser to open the url in.
Now xdg-open passes the url as an argument to the program (I suppose) and as I'm cycling through an array of browsers using the # symbol, it confuses this one with the argument (url) and errors on the dmenu command.
Is there a workaround to this problem or am I doing something completely wrong? --This problem was solved
#!/usr/bin/env bash
set -euo pipefail
_path="$(cd "$(dirname "${BASH_SOURCE[0]}")" && cd "$(dirname "$(readlink "${BASH_SOURCE[0]}" || echo ".")")" && pwd)"
if [[ -f "${_path}/_dm-helper.sh" ]]; then
# shellcheck disable=SC1090,SC1091
source "${_path}/_dm-helper.sh"
else
# shellcheck disable=SC1090
echo "No helper-script found"
fi
# script will not hit this if there is no config-file to load
# shellcheck disable=SC1090
source "$(get_config)"
main() {
if [ -t 0 ]
then
_url=$1
else
read _url
fi
_browser=$(printf '%s\n' "${!browsers[#]}" | sort | ${DMENU} 'Select browser: ') # "$#") ## Thx to #jhnc
_command=${browsers[${_browser}]}
if [[ -n ${_url} ]];then
$_command $_url
fi
}
[[ "${BASH_SOURCE[0]}" == "${0}" ]] && main "$#"
(get config) loads the dmenu command:
DMENU=dmenu -i -l 20 -p
as well as the array of browsers:
declare -A browsers
browsers[brave]="brave-browser"
browsers[firefox]="firefox"
browsers[opera]="opera"
browsers[badwolf]="badwolf"
from my config file.
Originally if i ran xdg-open "https://" or if I clicked on a url in some other program, brave was opened with on that site.
Now after xdg-settings set default-web-browser dmenu-script.desktop with the following .desktop file:
[Desktop Entry]
Version=1.0
Name=Dmenu Browser Script
GenericName=Web Browser
# Gnome and KDE 3 uses Comment.
Comment=Access the Internet
Exec=$HOME/.local/bin/dmenu-browser %U
StartupNotify=true
Terminal=false
Icon=brave-browser
Type=Application
Categories=Network;WebBrowser;
MimeType=application/pdf;application/rdf+xml;application/rss+xml;application/xhtml+xml;application/xhtml_xml;application/xml;image/gif;image/jpeg;image/png;image/webp;text/html;text/xml;x-scheme-handler/http;x-scheme-handler/https;x-scheme-handler/ipfs;x-scheme-handler/ipns;
Actions=new-window;new-private-window;
[Desktop Action new-window]
Name=New Window
Exec=$HOME/.local/bin/dmenu-browser
[Desktop Action new-private-window]
Name=New Incognito Window
Exec=$HOME/.local/bin/dmenu-browser --incognito
It only works if I execute xdg-open from my command line. (I modified the .desktop file of brave-browser, because I had no clue how to write one.)

Nomad task getting killed

I have two tasks in task group
1) a db task to bring up a db and
2) the app that needs the db to be up.
Both start in parallel and the db tasks takes a lil bit time but by then the app recognizes that db is not up and kills the db task. Any solutions? Please advise.
It's somewhat common to have an entrypoint script that checks if the db is healthy. Here's a script i've used before:
#!/bin/sh
set -e
cmd="$*"
postgres_ready() {
if test -z "${NO_DB}"
then
PGPASSWORD="${RDS_PASSWORD}" psql -h "${RDS_HOSTNAME}" -U "${RDS_USERNAME}" -d "${RDS_DB_NAME}" -c '\l'
return $?
else
echo "NO_DB Postgres will pretend to be up"
return 0
fi
}
until postgres_ready
do
>&2 echo "Postgres is unavailable - sleeping"
sleep 1
done
>&2 echo "Postgres is up - continuing..."
exec "${cmd}"
You could save it as entrypoint.sh and run it with your application start script as the argument. eg: entrypoint.sh python main.py

qsub array job submission

I am currently trying to run an array job on the "big-computer" at my Uni.
I'm new to Unix and bash and I've been having a hard time getting this to work.
The folder set up is as follow:
model1
- model1.inp
- model1.num
model2
- model2.inp
- model2.num
startup.sh
runAModel.sh
modelArray.sh
Due to restrictions on how long I can run a single job, I was asked to break up my simulations. So I need to run each model 5 times over, each time the model reads the input file .inp and outputs another input file for the subsequent run.
The code below used to work until a week a go or so but it doesn't seem to function anymore. I wonder if I didn't mess something up in there.
I suspected it might be in the line qcmd="qsub -N $modelName -t 1:5 ../../modelArray.sh" of runAModel.sh and that I should replace 1:5 to 1-5 but that didn't seem to work.
I use qstat to see my job and where I would expect to see a list of 5 queued jobs I only see one.
I was given three files to run:
startup.sh :
find . -mindepth 2 -type d -exec ./runAModel.sh {} \;
runAModel.sh :
#!/bin/bash
echo starting model in $1
cd $1 # go into the model directory
modelName=$(basename $PWD)
for f in *
do
dos2unix $f
done
qcmd="qsub -N $modelName -t 1:5 ../../modelArray.sh"
qq=`$qcmd` # runs a qsub command
# extract the job number
qt=`echo $qq | awk '{print $3}'`
jobid=${qt%%.*}
qrls $jobid.1
and modelArray.sh :
#!/bin/bash
# run program, invoke in model directory with input files.
# we want to run in the current working directory
#$ -cwd
# we want to run mpi with 4 cores on he same node:
#$ -pe sharedmem 4
# make a generous guess at the time we need
#$ -l h_rt=30:00:00
# force reservation
#$ -R y
# use 4G per process
#$ -l h_vmem=4G
# hold the array
#$ -h
echo I am task $SGE_TASK_ID in $JOB_ID with $SGE_TASK_LAST tasks in total
echo on $HOSTNAME
date
# run our model - set modules, then get the model name
echo "set modules"
. /etc/profile.d/modules.sh
PROGRAMBUILD=/exports/programlocation
. $PROGRAMBUILD/loadModules.sh
modelName=$(basename $PWD)
echo mpirun -np 4 $PROGRAMBUILD/bin/program $modelName
mpirun -np 4 $PROGRAMBUILD/bin/program $modelName
if [ $SGE_TASK_ID == $SGE_TASK_LAST ]
then
echo I am last task
else
# release the next task....
# next task in this array:
next=$((SGE_TASK_ID+1))
echo insert a test that this task in the array job was successful
echo if so, release next task
echo releasing $next
ssh login01.***.uk qrls $JOB_ID.$next
if [[ "$?" -ne 0 ]]; then
echo failed to qrls $pid
fi
fi

Debian package drop datebase on purge

i am on creating a deb package for ubuntu
in my postinst script i use
# Configure database
dbc_mysql_createdb_encoding="UTF8"
if ! dbc_go portal3 $# ; then
echo 'Automatic configuration using dbconfig-db_version 2.0common failed!'
fi
to create the database what works fine.
In the postrm file i have:
echo "Remove database"
if [ -f /usr/share/debconf/confmodule ]; then
. /usr/share/debconf/confmodule
fi
if [ -f /usr/share/dbconfig-common/dpkg/postrm ]; then
. /usr/share/dbconfig-common/dpkg/postrm
if ! dbc_go portal3 $# ; then
echo 'Automatic configuration using dbconfig-common failed!'
fi
fi
but this don’t drop the created user or database.
There is no response on console or anything else that helps me to debug the issue.
Has anyone an idea how to drop database and user created while installation?
It also requires an prerm script like
#!/bin/sh
set -e
. /usr/share/debconf/confmodule
. /usr/share/dbconfig-common/dpkg/prerm.mysql
if ! dbc_go portal3 $# ; then
echo 'Automatic configuration using dbconfig-common failed!'
fi
# dh_installdeb will replace this with shell code automatically
# generated by other debhelper scripts.
exit 0
Otherwise dbconfig-commond dont know what database needs to be dropt.

Facing some issues regarding tomcat startup and shutdown on ubuntu 10.04

I am facing some issues regarding tomcat startup and shutdown.
I am using
Tomcat v.6.0.32 (using the extracted the bundle downloaded from Apache website and not the one distributed by Ubuntu 10.04)
Ubuntu v.10.04
I start the tomcat using command:
$./catalina.sh jpda start
Then I try to stop it using command:
$./catalina.sh stop
and get the following message:
SEVERE: Catalina.stop:
java.net.ConnectException: Connection refused
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:351)
at java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:213)
at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:200)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:366)
at java.net.Socket.connect(Socket.java:529)
at java.net.Socket.connect(Socket.java:478)
at java.net.Socket.<init>(Socket.java:375)
at java.net.Socket.<init>(Socket.java:189)
at org.apache.catalina.startup.Catalina.stopServer(Catalina.java:422)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.catalina.startup.Bootstrap.stopServer(Bootstrap.java:338)
at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:416)
I also tried stopping it using command :
$./catalina.sh stop -force
but I got the message:
Kill failed: $CATALINA_PID not set
Then I again tried starting tomcat using command:
$./catalina.sh jpda start
In the /logs folder only catalina.out is generated with following error message:
ERROR: transport error 202: bind failed: Address already in use
ERROR: JDWP Transport dt_socket failed to initialize, TRANSPORT_INIT(510)
JDWP exit error AGENT_ERROR_TRANSPORT_INIT(197): No transports initialized [../../../src/share/back/debugInit.c:690]
FATAL ERROR in native method: JDWP No transports initialized, jvmtiError=AGENT_ERROR_TRANSPORT_INIT(197)
As it is indicated in the error "Address already in use" I verified no other application is configured to run on port 8080.
Everytime when I kill the process id and restart it, the startup works fine.This requires me to kill the process manually using "KILL" command, when I need to stop the tomcat server.
This same behavior I am encountering on other machines too where same environment as mentioned above is set up.
Can anybody please let me know what is causing this and how to resolve this?
Thanks,
Jignesh
Dont use /etc/bash.bashrc
just put this on your catalina.sh
After:
if [ -r "$CATALINA_BASE"/bin/setenv.sh ]; then
. "$CATALINA_BASE"/bin/setenv.sh
elif [ -r "$CATALINA_HOME"/bin/setenv.sh ]; then
. "$CATALINA_HOME"/bin/setenv.sh
fi
put this:
export CATALINA_PID="$CATALINA_HOME/catalina_pid.txt"
And now your tomcat will be killed when you use catalina.sh stop -force
Solution;
First set the pid of tomcat
CATALINA_PID="/id.pid"
export CATALINA_PID
then kill it
catalina.sh stop -force
source: http://confluence.atlassian.com/pages/viewpage.action?pageId=216959212
otherwise just as you mentioned, just kill it with the kill command
ps aux | grep catalina
kill <pid of tomcat>
if the above two solutions don't fit your needs try the following:
$ sudo service tomcat6 stop
* Stopping Tomcat servlet engine tomcat6 [ OK ]
$ sudo service tomcat6 start
* Starting Tomcat servlet engine tomcat6 [ OK ]
$
or of course the more traditional way:
$ sudo /etc/init.d/tomcat6 stop
* Stopping Tomcat servlet engine tomcat6 [ OK ]
$ sudo /etc/init.d/tomcat6 start
* Starting Tomcat ser
Fixed the tomcat shutdown issue.Posting the solution here for reference.
Do the following to fix it on your machine:
1) Replace $CATALINA_HOME/bin/catalina.sh with catalina.sh shown below.
2) Open /etc/bash.bashrc for edit
3) Add following into it, (AFTER export CATALINA_HOME)
export CATALINA_PID="$CATALINA_HOME/catalina_pid.txt",
where $CATALINA_HOME is the tomcat home path
4) Execute following :
$ source /etc/bash.bashrc
5) Start tomcat: $CATALINA_HOME/bin/catalina.sh start (or catalina.sh jpda start)
6) Shutdown tomcat using following command only for proper shutdown:
$CATALINA_HOME/bin/catalina.sh stop -force
catalina.sh - contains my inline comments
#!/bin/sh
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# -----------------------------------------------------------------------------
# Start/Stop Script for the CATALINA Server
#
# Environment Variable Prerequisites
#
# CATALINA_HOME May point at your Catalina "build" directory.
#
# CATALINA_BASE (Optional) Base directory for resolving dynamic portions
# of a Catalina installation. If not present, resolves to
# the same directory that CATALINA_HOME points to.
#
# CATALINA_OUT (Optional) Full path to a file where stdout and stderr
# will be redirected.
# Default is $CATALINA_BASE/logs/catalina.out
#
# CATALINA_OPTS (Optional) Java runtime options used when the "start",
# or "run" command is executed.
#
# CATALINA_TMPDIR (Optional) Directory path location of temporary directory
# the JVM should use (java.io.tmpdir). Defaults to
# $CATALINA_BASE/temp.
#
# JAVA_HOME Must point at your Java Development Kit installation.
# Required to run the with the "debug" argument.
#
# JRE_HOME Must point at your Java Development Kit installation.
# Defaults to JAVA_HOME if empty.
#
# JAVA_OPTS (Optional) Java runtime options used when the "start",
# "stop", or "run" command is executed.
#
# JAVA_ENDORSED_DIRS (Optional) Lists of of colon separated directories
# containing some jars in order to allow replacement of APIs
# created outside of the JCP (i.e. DOM and SAX from W3C).
# It can also be used to update the XML parser implementation.
# Defaults to $CATALINA_HOME/endorsed.
#
# JPDA_TRANSPORT (Optional) JPDA transport used when the "jpda start"
# command is executed. The default is "dt_socket".
#
# JPDA_ADDRESS (Optional) Java runtime options used when the "jpda start"
# command is executed. The default is 8000.
#
# JPDA_SUSPEND (Optional) Java runtime options used when the "jpda start"
# command is executed. Specifies whether JVM should suspend
# execution immediately after startup. Default is "n".
#
# JPDA_OPTS (Optional) Java runtime options used when the "jpda start"
# command is executed. If used, JPDA_TRANSPORT, JPDA_ADDRESS,
# and JPDA_SUSPEND are ignored. Thus, all required jpda
# options MUST be specified. The default is:
#
# -agentlib:jdwp=transport=$JPDA_TRANSPORT,
# address=$JPDA_ADDRESS,server=y,suspend=$JPDA_SUSPEND
#
# CATALINA_PID (Optional) Path of the file which should contains the pid
# of catalina startup java process, when start (fork) is used
#
# LOGGING_CONFIG (Optional) Override Tomcat's logging config file
# Example (all one line)
# LOGGING_CONFIG="-Djava.util.logging.config.file=$CATALINA_BASE/conf/logging.properties"
#
# LOGGING_MANAGER (Optional) Override Tomcat's logging manager
# Example (all one line)
# LOGGING_MANAGER="-Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager"
#
# $Id: catalina.sh 1040547 2010-11-30 14:47:49Z markt $
# -----------------------------------------------------------------------------
# OS specific support. $var _must_ be set to either true or false.
cygwin=false
os400=false
darwin=false
case "`uname`" in
CYGWIN*) cygwin=true;;
OS400*) os400=true;;
Darwin*) darwin=true;;
esac
# resolve links - $0 may be a softlink
PRG="$0"
while [ -h "$PRG" ]; do
ls=`ls -ld "$PRG"`
link=`expr "$ls" : '.*-> \(.*\)$'`
if expr "$link" : '/.*' > /dev/null; then
PRG="$link"
else
PRG=`dirname "$PRG"`/"$link"
fi
done
# Get standard environment variables
PRGDIR=`dirname "$PRG"`
# Only set CATALINA_HOME if not already set
[ -z "$CATALINA_HOME" ] && CATALINA_HOME=`cd "$PRGDIR/.." >/dev/null; pwd`
# Ensure that any user defined CLASSPATH variables are not used on startup,
# but allow them to be specified in setenv.sh, in rare case when it is needed.
CLASSPATH=
if [ -r "$CATALINA_BASE"/bin/setenv.sh ]; then
. "$CATALINA_BASE"/bin/setenv.sh
elif [ -r "$CATALINA_HOME"/bin/setenv.sh ]; then
. "$CATALINA_HOME"/bin/setenv.sh
fi
# For Cygwin, ensure paths are in UNIX format before anything is touched
if $cygwin; then
[ -n "$JAVA_HOME" ] && JAVA_HOME=`cygpath --unix "$JAVA_HOME"`
[ -n "$JRE_HOME" ] && JRE_HOME=`cygpath --unix "$JRE_HOME"`
[ -n "$CATALINA_HOME" ] && CATALINA_HOME=`cygpath --unix "$CATALINA_HOME"`
[ -n "$CATALINA_BASE" ] && CATALINA_BASE=`cygpath --unix "$CATALINA_BASE"`
[ -n "$CLASSPATH" ] && CLASSPATH=`cygpath --path --unix "$CLASSPATH"`
fi
# For OS400
if $os400; then
# Set job priority to standard for interactive (interactive - 6) by using
# the interactive priority - 6, the helper threads that respond to requests
# will be running at the same priority as interactive jobs.
COMMAND='chgjob job('$JOBNAME') runpty(6)'
system $COMMAND
# Enable multi threading
export QIBM_MULTI_THREADED=Y
fi
# Get standard Java environment variables
if $os400; then
# -r will Only work on the os400 if the files are:
# 1. owned by the user
# 2. owned by the PRIMARY group of the user
# this will not work if the user belongs in secondary groups
BASEDIR="$CATALINA_HOME"
. "$CATALINA_HOME"/bin/setclasspath.sh
else
if [ -r "$CATALINA_HOME"/bin/setclasspath.sh ]; then
BASEDIR="$CATALINA_HOME"
. "$CATALINA_HOME"/bin/setclasspath.sh
else
echo "Cannot find $CATALINA_HOME/bin/setclasspath.sh"
echo "This file is needed to run this program"
exit 1
fi
fi
if [ -z "$CATALINA_BASE" ] ; then
CATALINA_BASE="$CATALINA_HOME"
fi
# Add tomcat-juli.jar and bootstrap.jar to classpath
# tomcat-juli.jar can be over-ridden per instance
if [ ! -z "$CLASSPATH" ] ; then
CLASSPATH="$CLASSPATH":
fi
if [ "$CATALINA_BASE" != "$CATALINA_HOME" ] && [ -r "$CATALINA_BASE/bin/tomcat-juli.jar" ] ; then
CLASSPATH="$CLASSPATH""$CATALINA_BASE"/bin/tomcat-juli.jar:"$CATALINA_HOME"/bin/bootstrap.jar
else
CLASSPATH="$CLASSPATH""$CATALINA_HOME"/bin/bootstrap.jar
fi
if [ -z "$CATALINA_OUT" ] ; then
CATALINA_OUT="$CATALINA_BASE"/logs/catalina.out
fi
if [ -z "$CATALINA_TMPDIR" ] ; then
# Define the java.io.tmpdir to use for Catalina
CATALINA_TMPDIR="$CATALINA_BASE"/temp
fi
# Bugzilla 37848: When no TTY is available, don't output to console
have_tty=0
if [ "`tty`" != "not a tty" ]; then
have_tty=1
fi
# For Cygwin, switch paths to Windows format before running java
if $cygwin; then
JAVA_HOME=`cygpath --absolute --windows "$JAVA_HOME"`
JRE_HOME=`cygpath --absolute --windows "$JRE_HOME"`
CATALINA_HOME=`cygpath --absolute --windows "$CATALINA_HOME"`
CATALINA_BASE=`cygpath --absolute --windows "$CATALINA_BASE"`
CATALINA_TMPDIR=`cygpath --absolute --windows "$CATALINA_TMPDIR"`
CLASSPATH=`cygpath --path --windows "$CLASSPATH"`
JAVA_ENDORSED_DIRS=`cygpath --path --windows "$JAVA_ENDORSED_DIRS"`
fi
# Set juli LogManager config file if it is present and an override has not been issued
if [ -z "$LOGGING_CONFIG" ]; then
if [ -r "$CATALINA_BASE"/conf/logging.properties ]; then
LOGGING_CONFIG="-Djava.util.logging.config.file=$CATALINA_BASE/conf/logging.properties"
else
# Bugzilla 45585
LOGGING_CONFIG="-Dnop"
fi
fi
if [ -z "$LOGGING_MANAGER" ]; then
JAVA_OPTS="$JAVA_OPTS -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager"
else
JAVA_OPTS="$JAVA_OPTS $LOGGING_MANAGER"
fi
# ----- Execute The Requested Command -----------------------------------------
# Bugzilla 37848: only output this if we have a TTY
if [ $have_tty -eq 1 ]; then
echo "Using CATALINA_BASE: $CATALINA_BASE"
echo "Using CATALINA_HOME: $CATALINA_HOME"
echo "Using CATALINA_TMPDIR: $CATALINA_TMPDIR"
if [ "$1" = "debug" ] ; then
echo "Using JAVA_HOME: $JAVA_HOME"
else
echo "Using JRE_HOME: $JRE_HOME"
fi
echo "Using CLASSPATH: $CLASSPATH"
if [ ! -z "$CATALINA_PID" ]; then
# Added by Jignesh Gohel - Nov 08, 2011.
# There was an issue occuring when shutting down tomcat using
# command ./catalina.sh stop
# Issue mentioned at : http://stackoverflow.com/questions/7890197/facing-some-issues-regarding-tomcat-startup-and-shutdown-on-ubuntu-10-04
# Tomcat was shutting down properly but the process started using
# command ./catalina.sh start was not getting killed.
# To resolve this issue following steps taken :-
# 1) Exported an environment variable named CATALINA_PID which
# should point to a file in which the tomcat process id would be
# written.
# Note: Using the normal command to shutdown tomcat "./catalina.sh stop"
# following message was logged "Tomcat did not stop in time. PID file
# was not removed".Thus to shutdown the tomcat using following command
# "./catalina.sh stop -force" which forcefully kills the tomcat process
# and also removes the file pointed by environment variable
# "$CATALINA_PID".Thus executing a "touch" command when starting up
# tomcat to automatically create the file.
touch "$CATALINA_PID"
echo "Using CATALINA_PID: $CATALINA_PID"
fi
fi
if [ "$1" = "jpda" ] ; then
if [ -z "$JPDA_TRANSPORT" ]; then
JPDA_TRANSPORT="dt_socket"
fi
if [ -z "$JPDA_ADDRESS" ]; then
JPDA_ADDRESS="8000"
fi
if [ -z "$JPDA_SUSPEND" ]; then
JPDA_SUSPEND="n"
fi
if [ -z "$JPDA_OPTS" ]; then
JPDA_OPTS="-agentlib:jdwp=transport=$JPDA_TRANSPORT,address=$JPDA_ADDRESS,server=y,suspend=$JPDA_SUSPEND"
fi
CATALINA_OPTS="$CATALINA_OPTS $JPDA_OPTS"
shift
fi
if [ "$1" = "debug" ] ; then
if $os400; then
echo "Debug command not available on OS400"
exit 1
else
shift
if [ "$1" = "-security" ] ; then
if [ $have_tty -eq 1 ]; then
echo "Using Security Manager"
fi
shift
exec "$_RUNJDB" "$LOGGING_CONFIG" $JAVA_OPTS $CATALINA_OPTS \
-Djava.endorsed.dirs="$JAVA_ENDORSED_DIRS" -classpath "$CLASSPATH" \
-sourcepath "$CATALINA_HOME"/../../java \
-Djava.security.manager \
-Djava.security.policy=="$CATALINA_BASE"/conf/catalina.policy \
-Dcatalina.base="$CATALINA_BASE" \
-Dcatalina.home="$CATALINA_HOME" \
-Djava.io.tmpdir="$CATALINA_TMPDIR" \
org.apache.catalina.startup.Bootstrap "$#" start
else
exec "$_RUNJDB" "$LOGGING_CONFIG" $JAVA_OPTS $CATALINA_OPTS \
-Djava.endorsed.dirs="$JAVA_ENDORSED_DIRS" -classpath "$CLASSPATH" \
-sourcepath "$CATALINA_HOME"/../../java \
-Dcatalina.base="$CATALINA_BASE" \
-Dcatalina.home="$CATALINA_HOME" \
-Djava.io.tmpdir="$CATALINA_TMPDIR" \
org.apache.catalina.startup.Bootstrap "$#" start
fi
fi
elif [ "$1" = "run" ]; then
shift
if [ "$1" = "-security" ] ; then
if [ $have_tty -eq 1 ]; then
echo "Using Security Manager"
fi
shift
exec "$_RUNJAVA" "$LOGGING_CONFIG" $JAVA_OPTS $CATALINA_OPTS \
-Djava.endorsed.dirs="$JAVA_ENDORSED_DIRS" -classpath "$CLASSPATH" \
-Djava.security.manager \
-Djava.security.policy=="$CATALINA_BASE"/conf/catalina.policy \
-Dcatalina.base="$CATALINA_BASE" \
-Dcatalina.home="$CATALINA_HOME" \
-Djava.io.tmpdir="$CATALINA_TMPDIR" \
org.apache.catalina.startup.Bootstrap "$#" start
else
exec "$_RUNJAVA" "$LOGGING_CONFIG" $JAVA_OPTS $CATALINA_OPTS \
-Djava.endorsed.dirs="$JAVA_ENDORSED_DIRS" -classpath "$CLASSPATH" \
-Dcatalina.base="$CATALINA_BASE" \
-Dcatalina.home="$CATALINA_HOME" \
-Djava.io.tmpdir="$CATALINA_TMPDIR" \
org.apache.catalina.startup.Bootstrap "$#" start
fi
elif [ "$1" = "start" ] ; then
if [ ! -z "$CATALINA_PID" ]; then
if [ -f "$CATALINA_PID" ]; then
if [ -s "$CATALINA_PID" ]; then
echo "Existing PID file found during start."
if [ -r "$CATALINA_PID" ]; then
PID=`cat "$CATALINA_PID"`
ps -p $PID >/dev/null 2>&1
if [ $? -eq 0 ] ; then
echo "Tomcat appears to still be running with PID $PID. Start aborted."
exit 1
else
echo "Removing/clearing stale PID file."
rm -f "$CATALINA_PID" >/dev/null 2>&1
if [ $? != 0 ]; then
if [ -w "$CATALINA_PID" ]; then
cat /dev/null > "$CATALINA_PID"
else
echo "Unable to remove or clear stale PID file. Start aborted."
exit 1
fi
fi
fi
else
echo "Unable to read PID file. Start aborted."
exit 1
fi
else
rm -f "$CATALINA_PID" >/dev/null 2>&1
if [ $? != 0 ]; then
if [ ! -w "$CATALINA_PID" ]; then
echo "Unable to remove or write to empty PID file. Start aborted."
exit 1
fi
fi
fi
fi
fi
shift
touch "$CATALINA_OUT"
if [ "$1" = "-security" ] ; then
if [ $have_tty -eq 1 ]; then
echo "Using Security Manager"
fi
shift
"$_RUNJAVA" "$LOGGING_CONFIG" $JAVA_OPTS $CATALINA_OPTS \
-Djava.endorsed.dirs="$JAVA_ENDORSED_DIRS" -classpath "$CLASSPATH" \
-Djava.security.manager \
-Djava.security.policy=="$CATALINA_BASE"/conf/catalina.policy \
-Dcatalina.base="$CATALINA_BASE" \
-Dcatalina.home="$CATALINA_HOME" \
-Djava.io.tmpdir="$CATALINA_TMPDIR" \
org.apache.catalina.startup.Bootstrap "$#" start \
>> "$CATALINA_OUT" 2>&1 &
else
"$_RUNJAVA" "$LOGGING_CONFIG" $JAVA_OPTS $CATALINA_OPTS \
-Djava.endorsed.dirs="$JAVA_ENDORSED_DIRS" -classpath "$CLASSPATH" \
-Dcatalina.base="$CATALINA_BASE" \
-Dcatalina.home="$CATALINA_HOME" \
-Djava.io.tmpdir="$CATALINA_TMPDIR" \
org.apache.catalina.startup.Bootstrap "$#" start \
>> "$CATALINA_OUT" 2>&1 &
fi
if [ ! -z "$CATALINA_PID" ]; then
echo $! > "$CATALINA_PID"
fi
elif [ "$1" = "stop" ] ; then
shift
SLEEP=5
if [ ! -z "$1" ]; then
echo $1 | grep "[^0-9]" >/dev/null 2>&1
if [ $? -gt 0 ]; then
SLEEP=$1
shift
fi
fi
FORCE=0
if [ "$1" = "-force" ]; then
shift
FORCE=1
fi
if [ ! -z "$CATALINA_PID" ]; then
if [ -s "$CATALINA_PID" ]; then
if [ -f "$CATALINA_PID" ]; then
kill -0 `cat "$CATALINA_PID"` >/dev/null 2>&1
if [ $? -gt 0 ]; then
echo "PID file found but no matching process was found. Stop aborted."
exit 1
fi
else
echo "\$CATALINA_PID was set but the specified file does not exist. Is Tomcat running? Stop aborted."
exit 1
fi
else
echo "PID file is empty and has been ignored."
fi
fi
"$_RUNJAVA" $JAVA_OPTS \
-Djava.endorsed.dirs="$JAVA_ENDORSED_DIRS" -classpath "$CLASSPATH" \
-Dcatalina.base="$CATALINA_BASE" \
-Dcatalina.home="$CATALINA_HOME" \
-Djava.io.tmpdir="$CATALINA_TMPDIR" \
org.apache.catalina.startup.Bootstrap "$#" stop
if [ ! -z "$CATALINA_PID" ]; then
if [ -f "$CATALINA_PID" ]; then
while [ $SLEEP -ge 0 ]; do
kill -0 `cat "$CATALINA_PID"` >/dev/null 2>&1
if [ $? -gt 0 ]; then
rm -f "$CATALINA_PID" >/dev/null 2>&1
if [ $? != 0 ]; then
if [ -w "$CATALINA_PID" ]; then
cat /dev/null > "$CATALINA_PID"
else
echo "Tomcat stopped but the PID file could not be removed or cleared."
fi
fi
break
fi
if [ $SLEEP -gt 0 ]; then
sleep 1
fi
if [ $SLEEP -eq 0 ]; then
if [ $FORCE -eq 0 ]; then
echo "Tomcat did not stop in time. PID file was not removed."
fi
fi
SLEEP=`expr $SLEEP - 1 `
done
fi
fi
if [ $FORCE -eq 1 ]; then
if [ -z "$CATALINA_PID" ]; then
echo "Kill failed: \$CATALINA_PID not set"
else
if [ -f "$CATALINA_PID" ]; then
PID=`cat "$CATALINA_PID"`
echo "Killing Tomcat with the PID: $PID"
kill -9 $PID
rm -f "$CATALINA_PID" >/dev/null 2>&1
if [ $? != 0 ]; then
echo "Tomcat was killed but the PID file could not be removed."
fi
fi
fi
fi
elif [ "$1" = "version" ] ; then
"$_RUNJAVA" \
-classpath "$CATALINA_HOME/lib/catalina.jar" \
org.apache.catalina.util.ServerInfo
else
echo "Usage: catalina.sh ( commands ... )"
echo "commands:"
if $os400; then
echo " debug Start Catalina in a debugger (not available on OS400)"
echo " debug -security Debug Catalina with a security manager (not available on OS400)"
else
echo " debug Start Catalina in a debugger"
echo " debug -security Debug Catalina with a security manager"
fi
echo " jpda start Start Catalina under JPDA debugger"
echo " run Start Catalina in the current window"
echo " run -security Start in the current window with security manager"
echo " start Start Catalina in a separate window"
echo " start -security Start in a separate window with security manager"
echo " stop Stop Catalina, waiting up to 5 seconds for the process to end"
echo " stop n Stop Catalina, waiting up to n seconds for the process to end"
echo " stop -force Stop Catalina, wait up to 5 seconds and then use kill -KILL if still running"
echo " stop n -force Stop Catalina, wait up to n seconds and then use kill -KILL if still running"
echo " version What version of tomcat are you running?"
echo "Note: Waiting for the process to end and use of the -force option require that \$CATALINA_PID is defined"
exit 1
fi
Thanks,
Jignesh
The problem is caused because the address you use to start tomcat (in normal or debug mode) is already taken by another process.
You need to check the ports you are using in the you conf file (e.g. your_TOMCAT_HOME_DIR_/conf/server.xml), to see if they are not already used
Here you can look at the port used for
starting Tomcat: default value 8080
stopping Tomcat: default value 8005
using with AJP protocoll: default value 8009
and if you are using Tomcat in debug mode (through jdpa or jdwp ), please make sure to use a different port than all the previous configured ports
In my scenario to this problem, there were some threads not successfully terminated with shutdown message posted to tomcat, which made tomcat process still hanging around.
So I would start with plain tomcat instance..and keep adding the custom "wars" or changes one by one and make sure to stop tomcat successfully to narrow down the problem.
I think the officially recommended way (according to catalina.sh inline comments) is:
create a setenv.sh file under $CATALINA_HOME/bin:
#!/bin/sh
CATALINA_PID="$CATALINA_HOME/logs/catalina.pid"; export CATALINA_PID
chmod as executable:
chmod u+x setenv.sh
create the pid file by:
touch $CATALINA_HOME/logs/catalina.pid
then startup tomcat as usual, the pid file will be updated automatically. And you can stop tomcat by "-force" option as well.
Following command does the trick:
service tomcat8 stop
These ones would be helpful too:
service tomcat8 start
service tomcat8 restart

Resources