$ bin/start-cluster.sh
Starting cluster.
[INFO] 1 instance(s) of standalonesession are already running on centos1.
Starting standalonesession daemon on host centos1.
[ERROR] Could not get JVM parameters properly.
[ERROR] Could not get JVM parameters properly.
I have got the $JAVA_HOME in all the master and slaves
]$ echo $JAVA_HOME
/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.242.b08-0.el7_7.x86_64/
Below are the config file settings.
jobmanager.rpc.address: 10.0.2.4
# The RPC port where the JobManager is reachable.
jobmanager.rpc.port: 6123
taskmanager.numberOfTaskSlots: 5
parallelism.default: 2
JPS in master:
]# jps
30944 QuorumPeerMain
9600 StandaloneSessionClusterEntrypoint
31640 ConsoleProducer
32889 Jps
31278 Kafka
in slave I am not able to see jps command output:
# jps
-bash: jps: command not found
Also under the task-manager i don't see any entries.
http://10.x.x.x:8081/#/task-manager
Did you configure the conf/slaves file? See https://ci.apache.org/projects/flink/flink-docs-stable/ops/deployment/cluster_setup.html#configuring-flink.
The problem with jps doesn't look like a problem with Flink. Is there really a JDK on the slave?
UPDATE: I've tried starting TomCat from commandline. During the startup messages I get this:
15-Mar-2019 09:05:08.603 INFO [main] org.apache.catalina.startup.HostConfig.deployWAR Deploying web application archive [C:\Program Files\ASF\Tomcat9\webapps\thredds.war]
15-Mar-2019 09:05:15.900 INFO [main] org.apache.jasper.servlet.TldScanner.scanJars At least one JAR was scanned for TLDs yet contained no TLDs. Enable debug logging for this logger for a complete list of JARs that were scanned but no TLDs were found in them. Skipping unneeded JARs during scanning can improve startup time and JSP compilation time.
15-Mar-2019 09:05:18.286 INFO [main] org.hibernate.validator.internal.util.Version.<clinit> HV000001: Hibernate Validator 4.3.2.Final
15-Mar-2019 09:05:19.382 SEVERE [main] org.apache.catalina.core.StandardContext.startInternal One or more listeners failed to start. Full details will be found in the appropriate container log file
15-Mar-2019 09:05:19.383 SEVERE [main] org.apache.catalina.core.StandardContext.startInternal Context [/thredds] startup failed due to previous errors
15-Mar-2019 09:05:19.460 WARNING [main] org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesThreads The web application [thredds] appears to have started a thread named [Log4j2-TF-12-Scheduled-2] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
java.lang.Thread.run(Thread.java:748)
15-Mar-2019 09:05:19.469 INFO [main] org.apache.catalina.startup.HostConfig.deployWAR Deployment of web application archive [C:\Program Files\ASF\Tomcat9\webapps\thredds.war] has finished in [10,866] ms
I'm starting to believe that Gerhard is right that it is not a batch issue?
I'm trying to deploy the THREDDS server (version 4.6.13) for Windows using Tomcat 9. However, when I try to start the server I get:
FAIL - Application at context path [/thredds] could not be started
I've tried creating setenv.bat from a working setenv.sh on a Linux-machine, but my batch-scripting is more than rusty, but I certainly believe it's down to wrong syntax.
My script file is as follows:
#ECHO OFF
set "CATALINA_HOME=%ProgramFiles%/ASF/Tomcat9"
set "CATALINA_BASE=%ProgramFiles%/ASF/Tomcat9"
set "JAVA_HOME=%ProgramFiles%/AdoptOpenJDK/jdk8u202-b08-jre"
:: TDS specific ENVARS
::
:: Define where the TDS content directory will live
::
set "CONTENT_ROOT=-Dtds.content.root.path=%ProgramFiles%/ASF/Tomcat9/content"
:: set java prefs related variables (used by the wms service, for example)
set "JAVA_PREFS_ROOTS=-Djava.util.prefs.systemRoot=%CATALINA_HOME%/content/thredds/javaUtilPrefs -Djava.util.prefs.userRoot=%CATALINA_HOME%/content/thredds/javaUtilPrefs"
:: Some commonl used JAVA_OPTS settings:
set NORMAL="-d64 -Xmx4096m -Xms512m -server -ea"
set HEAP_DUMP="-XX:+HeapDumpOnOutOfMemoryError"
set HEADLESS="-Djava.awt.headless=true"
Where did I mess up?
My development environment setup: Windows 10 Enterprise Edition, 16GB RAM, 2.81GHz 64Bit OS. I installed Virtual Box and imported Ubuntu image in it. With in Ubuntu I installed Confluent CLI https://github.com/confluentinc/confluent-cli to run Kafka, zookeeper and other services.
Scenario: I want to write data from Apache Kafka topic to a text file. I am using Sink connector and following the below link to accomplish this task. Also, I didn't write any code to accomplish the same.
Using this Link to accomplish my task
http://bigdatums.net/2017/06/22/writing-data-from-apache-kafka-to-text-file/
Steps accomplished successfully so far:
Able to run Ubuntu image with in Virtual Box.
Able to run Confluent CLI.
Able to bring up Confluent Kafka, ZooKeeper and other services using bin/confluent start command.
Able to create Topics within Confluent CLI
Try to run the following message to read the text message from Kafka Topic
osboxes#osboxes:~/ganesh/confluent-5.1.0$ bin/connect-standalone /home/osboxes/ganesh/confluent-5.1.0/etc/kafka/csx-connect-standalone.properties /home/osboxes/ganesh/confluent-5.1.0/etc/kafka/csx-connect-file-sink.properties
Property configuration details below
connect-file-sink.properties (details)
name=local-file-sink
connector.class=FileStreamSink
tasks.max=1
file=/home/osboxes/ganesh/ptc/messages/output/trainstartevent/MBCDTSKB02.json
topics=TrainStartEvent
connect-file-source.properties (details)
name=local-file-source
connector.class=FileStreamSource
tasks.max=1
file=/home/osboxes/ganesh/ptc/messages/input/trainstartevent/MBCDTSKB02.json
topic=TrainStartEvent
connect-standalone.properties (details)
key.converter=org.apache.kafka.connect.storage.StringConverter
value.converter=org.apache.kafka.connect.storage.StringConverter
internal.key.converter=org.apache.kafka.connect.json.JsonConverter
internal.value.converter=org.apache.kafka.connect.json.JsonConverter
internal.key.converter.schemas.enable=false
internal.value.converter.schemas.enable=false
offset.flush.interval.ms=10000
plugin.path=share/java
Actual Error Message
[2019-01-20 21:14:17,413] INFO Started
o.e.j.s.ServletContextHandler#546394ed{/,null,AVAILABLE}
(org.eclipse.jetty.server.handler.ContextHandler:850)
[2019-01-20 21:14:17,428] ERROR Stopping after connector error
(org.apache.kafka.connect.cli.ConnectStandalone:113)
org.apache.kafka.connect.errors.ConnectException: Unable to start REST
server
[2019-01-20 21:13:19,927] INFO Kafka Connect standalone worker
initializing ... (org.apache.kafka.connect.cli.ConnectStandalone:67)
[2019-01-20 21:13:20,021] INFO WorkerInfo values:
jvm.args = -Xms256M, -Xmx2G, -XX:+UseG1GC, -XX:MaxGCPauseMillis=20,
-XX:InitiatingHeapOccupancyPercent=35, -XX:+ExplicitGCInvokesConcurrent, -Djava.awt.headless=true, -Dcom.sun.management.jmxremote, -Dcom.sun.management.jmxremote.authenticate=false, -Dcom.sun.management.jmxremote.ssl=false, -Dkafka.logs.dir=bin/../logs, -Dlog4j.configuration=file:bin/../etc/kafka/connect-log4j.properties
jvm.spec = Oracle Corporation, Java HotSpot(TM) 64-Bit Server VM,
1.8.0_201, 25.201-b09
[2019-01-20 21:14:13,427] WARN The configuration 'plugin.path' was
supplied but isn't a known config.
(org.apache.kafka.clients.admin.AdminClientConfig:287)
[2019-01-20 21:14:13,431] WARN The configuration 'value.converter' was
supplied but isn't a known config.
(org.apache.kafka.clients.admin.AdminClientConfig:287)
[2019-01-20 21:14:13,431] WARN The configuration
'internal.key.converter.schemas.enable' was supplied but isn't a known
config. (org.apache.kafka.clients.admin.AdminClientConfig:287)
[2019-01-20 21:14:13,432] WARN The configuration 'key.converter' was
supplied but isn't a known config.
(org.apache.kafka.clients.admin.AdminClientConfig:287)
[2019-01-20 21:14:13,433] INFO Kafka version : 2.1.0-cp1
(org.apache.kafka.common.utils.AppInfoParser:109)
[2019-01-20 21:14:13,433] INFO Kafka commitId : 3bce825d5f759863
(org.apache.kafka.common.utils.AppInfoParser:110)
[2019-01-20 21:14:14,047] INFO Kafka cluster ID:
jPHHwv39Riyn1krFQyhYkA (org.apache.kafka.connect.util.ConnectUtils:59)
[2019-01-20 21:14:14,139] INFO Logging initialized #55198ms to
org.eclipse.jetty.util.log.Slf4jLog (org.eclipse.jetty.util.log:193)
[2019-01-20 21:14:14,565] INFO Added connector for http://:8083
(org.apache.kafka.connect.runtime.rest.RestServer:119)
[2019-01-20 21:14:14,681] INFO Advertised URI: http://127.0.1.1:8083/
(org.apache.kafka.connect.runtime.rest.RestServer:267)
[2019-01-20 21:14:14,705] INFO Kafka version : 2.1.0-cp1
(org.apache.kafka.common.utils.AppInfoParser:109)
[2019-01-20 21:14:14,705] INFO Kafka commitId : 3bce825d5f759863
(org.apache.kafka.common.utils.AppInfoParser:110)
[2019-01-20 21:14:15,228] INFO JsonConverterConfig values:
converter.type = key
schemas.cache.size = 1000
schemas.enable = false
(org.apache.kafka.connect.json.JsonConverterConfig:279)
[2019-01-20 21:14:15,238] INFO JsonConverterConfig values:
converter.type = value
schemas.cache.size = 1000
schemas.enable = false
(org.apache.kafka.connect.json.JsonConverterConfig:279)
[2019-01-20 21:14:15,251] INFO Kafka Connect standalone worker
initialization took 55315ms
(org.apache.kafka.connect.cli.ConnectStandalone:92)
[2019-01-20 21:14:15,251] INFO Kafka Connect starting
(org.apache.kafka.connect.runtime.Connect:49)
[2019-01-20 21:14:15,256] INFO Herder starting
(org.apache.kafka.connect.runtime.standalone.StandaloneHerder:88)
[2019-01-20 21:14:15,256] INFO Worker starting
(org.apache.kafka.connect.runtime.Worker:172)
[2019-01-20 21:14:15,256] INFO Starting FileOffsetBackingStore with
file /tmp/connect.offsets
(org.apache.kafka.connect.storage.FileOffsetBackingStore:58)
[2019-01-20 21:14:15,258] INFO Worker started
(org.apache.kafka.connect.runtime.Worker:177)
[2019-01-20 21:14:15,259] INFO Herder started
(org.apache.kafka.connect.runtime.standalone.StandaloneHerder:90)
[2019-01-20 21:14:15,259] INFO Starting REST server
(org.apache.kafka.connect.runtime.rest.RestServer:163)
[2019-01-20 21:14:15,565] INFO jetty-9.4.12.v20180830; built:
2018-08-30T13:59:14.071Z; git:
27208684755d94a92186989f695db2d7b21ebc51; jvm 1.8.0_201-b09
(org.eclipse.jetty.server.Server:371)
[2019-01-20 21:14:15,733] INFO DefaultSessionIdManager
workerName=node0 (org.eclipse.jetty.server.session:365)
[2019-01-20 21:14:15,746] INFO No SessionScavenger set, using defaults
(org.eclipse.jetty.server.session:370)
[2019-01-20 21:14:15,748] INFO node0 Scavenging every 600000ms
(org.eclipse.jetty.server.session:149)
Jan 20, 2019 9:14:16 PM org.glassfish.jersey.internal.inject.Providers
checkProviderRuntime
WARNING: A provider
org.apache.kafka.connect.runtime.rest.resources.ConnectorPluginsResource
registered in SERVER runtime does not implement any provider
interfaces applicable in the SERVER runtime. Due to constraint
configuration problems the provider
org.apache.kafka.connect.runtime.rest.resources.ConnectorPluginsResource
will be ignored.
Jan 20, 2019 9:14:16 PM org.glassfish.jersey.internal.inject.Providers
checkProviderRuntime
WARNING: A provider
org.apache.kafka.connect.runtime.rest.resources.ConnectorsResource
registered in SERVER runtime does not implement any provider
interfaces applicable in the SERVER runtime. Due to constraint
configuration problems the provider
org.apache.kafka.connect.runtime.rest.resources.ConnectorsResource
will be ignored.
Jan 20, 2019 9:14:16 PM org.glassfish.jersey.internal.inject.Providers
checkProviderRuntime
WARNING: A provider
org.apache.kafka.connect.runtime.rest.resources.RootResource
registered in SERVER runtime does not implement any provider
interfaces applicable in the SERVER runtime. Due to constraint
configuration problems the provider
org.apache.kafka.connect.runtime.rest.resources.RootResource will be
ignored.
Jan 20, 2019 9:14:17 PM org.glassfish.jersey.internal.Errors logErrors
WARNING: The following warnings have been detected: WARNING: The
(sub)resource method listConnectors in
org.apache.kafka.connect.runtime.rest.resources.ConnectorsResource
contains empty path annotation.
WARNING: The (sub)resource method createConnector in
org.apache.kafka.connect.runtime.rest.resources.ConnectorsResource
contains empty path annotation.
WARNING: The (sub)resource method listConnectorPlugins in
org.apache.kafka.connect.runtime.rest.resources.ConnectorPluginsResource
contains empty path annotation.
WARNING: The (sub)resource method serverInfo in
org.apache.kafka.connect.runtime.rest.resources.RootResource contains
empty path annotation.
[2019-01-20 21:14:17,413] INFO Started
o.e.j.s.ServletContextHandler#546394ed{/,null,AVAILABLE}
(org.eclipse.jetty.server.handler.ContextHandler:850)
[2019-01-20 21:14:17,428] ERROR Stopping after connector error
(org.apache.kafka.connect.cli.ConnectStandalone:113)
org.apache.kafka.connect.errors.ConnectException: Unable to start REST
server
at
org.apache.kafka.connect.runtime.rest.RestServer.start(RestServer.java:214)
at org.apache.kafka.connect.runtime.Connect.start(Connect.java:53)
at
org.apache.kafka.connect.cli.ConnectStandalone.main(ConnectStandalone.java:95)
Caused by: java.net.BindException: Address already in use
at sun.nio.ch.Net.bind0(Native Method)
at sun.nio.ch.Net.bind(Net.java:433)
at sun.nio.ch.Net.bind(Net.java:425)
at
sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:223)
at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74)
at
org.eclipse.jetty.server.ServerConnector.openAcceptChannel(ServerConnector.java:339)
at
org.eclipse.jetty.server.ServerConnector.open(ServerConnector.java:307)
at
org.eclipse.jetty.server.AbstractNetworkConnector.doStart(AbstractNetworkConnector.java:80)
at
org.eclipse.jetty.server.ServerConnector.doStart(ServerConnector.java:235)
at
org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
at org.eclipse.jetty.server.Server.doStart(Server.java:395)
at
org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
at
org.apache.kafka.connect.runtime.rest.RestServer.start(RestServer.java:212)
... 2 more
[2019-01-20 21:14:17,437] INFO Kafka Connect stopping
(org.apache.kafka.connect.runtime.Connect:65)
[2019-01-20 21:14:17,437] INFO Stopping REST server
(org.apache.kafka.connect.runtime.rest.RestServer:223)
[2019-01-20 21:14:17,442] INFO Stopped
http_8083#1b90fee4{HTTP/1.1,[http/1.1]}{0.0.0.0:8083}
(org.eclipse.jetty.server.AbstractConnector:341)
[2019-01-20 21:14:17,460] INFO node0 Stopped scavenging
(org.eclipse.jetty.server.session:167)
[2019-01-20 21:14:17,493] INFO Stopped
o.e.j.s.ServletContextHandler#546394ed{/,null,UNAVAILABLE}
(org.eclipse.jetty.server.handler.ContextHandler:1040)
[2019-01-20 21:14:17,507] INFO REST server stopped
(org.apache.kafka.connect.runtime.rest.RestServer:241)
[2019-01-20 21:14:17,508] INFO Herder stopping
(org.apache.kafka.connect.runtime.standalone.StandaloneHerder:95)
[2019-01-20 21:14:17,509] INFO Worker stopping
(org.apache.kafka.connect.runtime.Worker:184)
[2019-01-20 21:14:17,510] INFO Stopped FileOffsetBackingStore
(org.apache.kafka.connect.storage.FileOffsetBackingStore:66)
[2019-01-20 21:14:17,522] INFO Worker stopped
(org.apache.kafka.connect.runtime.Worker:205)
[2019-01-20 21:14:17,523] INFO Herder stopped
(org.apache.kafka.connect.runtime.standalone.StandaloneHerder:112)
[2019-01-20 21:14:17,529] INFO Kafka Connect stopped
(org.apache.kafka.connect.runtime.Connect:70)
Caused by: java.net.BindException: Address already in use
Sound like you ran confluent start and a Kafka Connect server is therefore already running at port 8083.
You therefore need to use confluent load /home/osboxes/ganesh/confluent-5.1.0/etc/kafka/csx-connect-file-sink.properties, or convert the properties file into JSON, which you can then do curl -XPOST -d#csx-connect-file-sink.json http://localhost:8083
See Kafka Connect REST API
Note that to write to a file, you could also do it all from the console consumer
kafka-console-consumer --from-beginning --property print.key=true --topic x --bootstrap-server localhost:9092 --group to-file >> /tmp/file.txt
I wanted to run Security Authentication and Authorization in SOLR 5.3.1 and placed the security.json file in solr root folder but on running the CLI as below :
/cygdrive/c/xampp/htdocs/solr$ server/scripts/cloud-scripts/zkcli.bat -zkhost localhost:8983 -cmd putfile /security.json security.json
I am getting the below issue :
WARN - 2015-10-29 15:39:01.656; org.apache.zookeeper.ClientCnxn$SendThread; Session 0x0 for server frontend.dev/127.0.0.1:8983, unexpected error, closing socket connection and attempting reconnect
java.io.IOException: Packet len1213486160 is out of range!
at org.apache.zookeeper.ClientCnxnSocket.readLength(ClientCnxnSocket.java:112)
at org.apache.zookeeper.ClientCnxnSocketNIO.doIO(ClientCnxnSocketNIO.java:79)
at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:366)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081)
I run through the below article :
Zookeeper CLI failing - IOException Packet <len12343123123> is out of range
I followed the instruction to increase the buffer size.I increased it as below for zkcli.bat . The command is as below :
zkcli.bat
#echo off
REM You can override pass the following parameters to this script:
REM
set JVM=java
REM Find location of this script
set SDIR=%~dp0
if "%SDIR:~-1%"=="\" set SDIR=%SDIR:~0,-1%
"%JVM%" -Dlog4j.configuration="file:%SDIR%\log4j.properties" -Djute.maxbuffer="20213486160" -classpath "%SDIR%\..\..\solr-webapp\webapp\WEB-INF\lib\*;%SDIR%\..\..\lib\ext\*" org.apache.solr.cloud.ZkCLI %*
even I set the maxClientCnxns,minSessionTimeout and maxSessionTimeout in zoo.cfg under server\solr :
zoo.cfg
tickTime=2000
initLimit=10
syncLimit=5
maxClientCnxns=500
minSessionTimeout=100000
maxSessionTimeout=200000
security.json file is as follows :
{
"authentication":{
"class":"solr.BasicAuthPlugin",
"credentials":{"solr":"IV0EHq1OnNrj6gvRCwvFwTrZ1+z1oBbnQdiVC3otuq0= Ndd7LKvVBAaZIF0QAVi1ekCfAJXr1GGfLtRUXhgrF8c="}
},
"authorization":{
"class":"solr.RuleBasedAuthorizationPlugin",
"user-role":{"solr":"admin"},
"permissions":[{"name":"security-edit",
"role":"admin"}]
}}
I googled but aint get any solution.Please kindly assist what I need to do to make the security authentication and authorization work in SOLR.
I had the same problem, but in my case was that solr 6 was running on normal mode and not in cluster mode. To run solr in cluster mode you need the -c flag.
Then the zookeeper was running on port 911.
./solr status
And see if you have a zookeeper running and in which port
I'm having some problems restarting plesk on our server.
I increased the memory limit in our PHP.ini file and had stopped the server and was in the process of restarting when i've received the following error:
[root#server77-68-52-152 ~]# /etc/init.d/psa start
Starting xinetd service... done
Starting named service... done
Starting mysqld service... done
Starting postgresql service... done
Starting psa-spamassassin service... done
Plesk: Starting Mail Server... already started
Starting mail handlers tmpfs storage
Starting Plesk... failed
Starting drwebd service... failed
I'll be honest I'm very new to the dedicated server side of things so need all the help I can get!
Turns out it was an earlier SSL upgrade that broke plesk...it hadn't come to light because plesk hadn't been restarted until the point I updated the PHP file!
You should always check the log files for the exact reason. It will help in fixing the issue faster. Below the log files path of plesk. Check the reason there why plesk is failing to start.
Error log: /var/log/sw-cp-server/error_log
Access log: /usr/local/psa/admin/logs/httpsd_access_log
Panel log: /usr/local/psa/admin/logs/panel.log