I downloaded zeppelin binary package and started it. However it's disconnected.
Here is the log:
INFO [2016-01-12 14:37:56,592] ({main} QuartzScheduler.java[initialize]:305) - Scheduler meta-data: Quartz Scheduler (v2.2.1) 'Defau
ltQuartzScheduler' with instanceId 'NON_CLUSTERED'
Scheduler class: 'org.quartz.core.QuartzScheduler' - running locally.
NOT STARTED.
Currently in standby mode.
Number of jobs executed: 0
Using thread pool 'org.quartz.simpl.SimpleThreadPool' - with 10 threads.
Using job-store 'org.quartz.simpl.RAMJobStore' - which does not support persistence. and is not clustered.
INFO [2016-01-12 14:37:56,593] ({main} StdSchedulerFactory.java[instantiate]:1339) - Quartz scheduler 'DefaultQuartzScheduler' initi
alized from default resource file in Quartz package: 'quartz.properties'
INFO [2016-01-12 14:37:56,593] ({main} StdSchedulerFactory.java[instantiate]:1343) - Quartz scheduler version: 2.2.1
INFO [2016-01-12 14:37:56,593] ({main} QuartzScheduler.java[start]:575) - Scheduler DefaultQuartzScheduler_$_NON_CLUSTERED started.
INFO [2016-01-12 14:37:56,805] ({main} ServerImpl.java[initDestination]:94) - Setting the server's publish address to be /
INFO [2016-01-12 14:37:56,888] ({main} WebInfConfiguration.java[unpack]:478) - Extract jar:file:/data/users/huser/zeppelin/zeppelin-
web-0.5.5-incubating.war!/ to /tmp/jetty-0.0.0.0-8080-zeppelin-web-0.5.5-incubating.war-_-any-/webapp
INFO [2016-01-12 14:37:57,040] ({main} StandardDescriptorProcessor.java[visitServlet]:284) - NO JSP Support for /, did not find org.
apache.jasper.servlet.JspServlet
INFO [2016-01-12 14:37:57,941] ({main} AbstractConnector.java[doStart]:338) - Started SelectChannelConnector#0.0.0.0:8080
INFO [2016-01-12 14:37:57,941] ({main} ZeppelinServer.java[main]:108) - Started zeppelin server
I searched and found that port+1 is the websocket port. However, it's not listening:
netstat -na | grep 8080
tcp 0 0 :::8080 :::* LISTEN
netstat -na | grep 8081
noting...
And the web return the error:
WebSocket connection to 'ws://ip:8080/ws' failed: Establishing a tunnel via proxy server failed.
Can anyone help?
Thanks.
Make sure port forwarding command executes on yr client system all the time you use zeppelin
ssh -i ~/keys/mykey.pem -N -L 8122:xx.xxx.xxx.xxx:8890 hadoop#yy.yyy.yyy.yyy
Related
I try to start 2 containers with the following docker compose file:
version: '2'
services:
client-app:
image: client-app:latest
build: ./client-app/Dockerfile
volumes:
- ./client-app:/usr/src/app
ports:
- 3000:8000
spring-boot-server:
build: ./spring-boot-server/Dockerfile
volumes:
- ./spring-boot-server:/usr/src/app
ports:
- 7000:7000
The spring boot server tries to connect to a remote database server which is on another host and network. Docker successfully starts the client-app containers but fails to start the spring-boot-server. This log is showing that the server crashed because it has failed to connect to the remote database:
2021-01-25 21:02:28.393 INFO 1 --- [main] com.zaxxer.hikari.HikariDataSource: HikariPool-1 - Starting...
2021-01-25 21:02:29.553 ERROR 1 --- [main] com.zaxxer.hikari.pool.HikariPool: HikariPool-1 - Exception during pool initialization.
com.mysql.cj.jdbc.exceptions.CommunicationsException: Communications link failure
The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server.
The Dockerfiles of both containers create valid images by which I can run manually the containers. It looks like there are some default network restrictions on containers started by a composing file.
Docker compose version running on Ubuntu:
docker-compose version 1.8.0, build unknown
=============================================
FURTHER INVESTIGATIONS:
I had created a Dockerfile
FROM ubuntu
RUN apt-get update
RUN apt-get install -y mysql-client
CMD mysql -A -P 3306 -h 8.8.8.8 --user=root --password=mypassword -e "SELECT VERSION()" mydatabase
along with a docker-compose.yml
version: '2'
services:
test-remote-db-compose:
build: .
ports:
- 8000:8000
to test aside the connectivity alone with the remote database. The test passed with success.
The problem has been misteriously solved, after doing this , a host mashine reboot and docker-compose up --build.
I am trying to connect kafka-connect to my local mssql with localhost:3030
I am receiving this error when I try to make a new connection for mssql. in centos 7(linux). Mssql data is from an external IP(windows), my consumer is inside of linux environment.
"No suitable driver found for configuration".
connect-distributed.properties is shown below;
plugin.path=/usr/local/share/java,/usr/local/share/kafka/plugins,/opt/connectors,
I added "ojdbc7-12.1.0.2.jar" file under /opt/connectors/kafka-connect-jdbc/ but still I am receiving error messages. I dont know what is wrong.
Also my connect-console-source.properties
name=source-sqlserver-user
connector.class=io.confluent.connect.jdbc.JdbcSourceConnector
tasks.max=1
topic.prefix=my-timestamp
connection.url=jdbc:sqlserver://externalIP;database=database;username=username;password=password
version: '2'
services:
kafka-cluster:
image: landoop/fast-data-dev:cp3.3.0
environment:
ADV_HOST: 127.0.0.1
RUNTESTS: 0
ports:
- 2181:2181 # Zookeeper
- 3030:3030 # Landoop UI
- 8081-8083:8081-8083 # REST Proxy, Schema Registry, Kafka Connect ports
- 9581-9585:9581-9585 # JMX Ports
- 9092:9092 # Kafka Broker
ojdbc7-12.1.0.2.jar is the JDBC driver for Oracle.
For MS SQL you need the MS SQL JDBC driver
Edit: Since you're using Docker to run Kafka Connect, you need to make the JDBC JAR file available to the Kafka Connect worker before it runs. You can't just run the Docker container and copy the JDBC driver into it, because you need to restart Kafka Connect afterwards.
To workaround this, you can mount the JAR from your local machine to the relevant path in the container. The relevant path is whereever the Kafka Connect JDBC jar is. Looking at the fast-data-dev image it's in
root#fast-data-dev / $ ls -l /opt/confluent-3.3.0/share/java/kafka-connect-jdbc
total 6544
-rw-r--r-- 1 root root 133842 Jul 28 2017 kafka-connect-jdbc-3.3.0.jar
-rw-r--r-- 1 root root 658466 Jul 28 2017 postgresql-9.4-1206-jdbc41.jar
-rw-r--r-- 1 root root 5575351 Jul 28 2017 sqlite-jdbc-3.8.11.2.jar
So you can run
docker run --rm --net=host --volume ~/Downloads/mssql-jdbc-7.4.1.jre8.jar:/opt/confluent-3.3.0/share/java/kafka-connect-jdbc/mssql-jdbc-7.4.1.jre8.jar landoop/fast-data-dev:cp3.3.0
or mount it in your Docker Compose with a volumes config:
version: '2'
services:
kafka-cluster:
image: landoop/fast-data-dev:cp3.3.0
environment:
ADV_HOST: 127.0.0.1
RUNTESTS: 0
ports:
- 2181:2181 # Zookeeper
- 3030:3030 # Landoop UI
- 8081-8083:8081-8083 # REST Proxy, Schema Registry, Kafka Connect ports
- 9581-9585:9581-9585 # JMX Ports
- 9092:9092 # Kafka Broker
volumes:
- ~/Downloads/mssql-jdbc-7.4.1.jre8.jar:/opt/confluent-3.3.0/share/java/kafka-connect-jdbc/mssql-jdbc-7.4.1.jre8.jar
It's worth noting that Confluent Platform 3.3.0 is really old - the latest is 5.3.1. If you want to see an up-to-date example of running Kafka, Kafka Connect, SQL Server etc with JDBC driver automatically set up see this example here.
I am able to successfully connect to my local database, by specifying following URL in application.properties file of my spring boot application:
application.properties
spring.datasource.url=jdbc:sqlserver://localhost:1433;databaseName=mantaDB;integratedSecurity=true
And here is my output:
2019-Aug-29 11:14:41.298 INFO com.zaxxer.hikari.HikariDataSource - HikariPool-1 - Starting... -
2019-Aug-29 11:14:41.753 INFO com.zaxxer.hikari.HikariDataSource - HikariPool-1 - Start completed. -
2019-Aug-29 11:14:41.914 INFO o.h.jpa.internal.util.LogHelper - HHH000204: Processing PersistenceUnitInfo [_ name: default_ ...] -
2019-Aug-29 11:14:42.102 INFO org.hibernate.Version - HHH000412: Hibernate Core {5.3.7.Final} -
2019-Aug-29 11:14:42.396 INFO o.h.annotations.common.Version - HCANN000001: Hibernate Commons Annotations {5.0.4.Final} -
2019-Aug-29 11:14:42.827 INFO org.hibernate.dialect.Dialect - HHH000400: Using dialect: org.hibernate.dialect.SQLServerDialect -
2019-Aug-29 11:14:44.157 INFO o.s.o.j.LocalContainerEntityManagerFactoryBean - Initialized JPA EntityManagerFactory for persistence unit 'default' -
Now, I want to change application to point to my dev db box:
application.properties
spring.datasource.url=jdbc:sqlserver://STUDENT12.site,28001;databaseName=mantaDB;integratedSecurity=true
But, I am getting following error:
2019-Aug-29 11:16:48.089 INFO com.zaxxer.hikari.HikariDataSource - HikariPool-1 - Starting... -
2019-Aug-29 11:17:18.682 ERROR com.zaxxer.hikari.pool.HikariPool - HikariPool-1 - Exception during pool initialization. -
com.microsoft.sqlserver.jdbc.SQLServerException: The TCP/IP connection to the host STUDENT12.site,28001, port 1433 has failed. Error: "STUDENT12.site,28001. Verify the connection properties. Make sure that an instance of SQL Server is running on the host and accepting TCP/IP connections at the port. Make sure that TCP connections to the port are not blocked by a firewall.".
at com.microsoft.sqlserver.jdbc.SQLServerException.makeFromDriverError(SQLServerException.java:228)
at com.microsoft.sqlserver.jdbc.SQLServerException.ConvertConnectExceptionToSQLServerException(SQLServerException.java:285)
Clearly, I can get to the database.
Does anyone know what i am doing wrong?
The JDBC Url should look like:
jdbc:sqlserver://[serverName[\instanceName][:portNumber]][;property=value[;property=value]]
Building the Connection URL
And the error makes it clear that you are not connecting on port 28001:
connection to the host JD1LSTWLSLMC101.dcsr.site,28001, port 1433 has
failed
So
spring.datasource.url=jdbc:sqlserver://JD1LSTWLSLMC101.dcsr.site:28001; . . .
I have a SolrCloud cluster (6.6) setup with external Zookeeper Ensemble (3.4.8) of 5 nodes. Recently, one machine (ip1:port1) that run 1 Zookeeper with id=1 went down. This is what I've done to replace zookeeper:
Start zookeeper in another machine with the same id (=1).
Change zoo.cfg in 4 live zookeeper to match new zookeeper server and restart.
Update ZK_HOST variable in solr.in.sh to match new zookeeper server.
Restart solr.
After that, my solr cluster seemed to functioning well, but in solr.log, it looked like solr client and zookeeper servers still try to connect to the old zookeeper:
Solr log
2017-12-01 15:04:38.782 WARN (Timer-0-SendThread(ip1:port1)) [ ] o.a.z.ClientCnxn Client session timed out, have not heard from server in 30029ms for sessionid 0x0
2017-12-01 15:04:40.807 WARN (Timer-0-SendThread(ip1:port1)) [ ] o.a.z.ClientCnxn Client session timed out, have not heard from server in 31030ms for sessionid 0x0
Zookeeper log:
2017-12-01 13:53:57,972 [myid:] - INFO [main-SendThread(ip1:port1):ClientCnxn$SendThread#1032] - Opening socket connection to server ip1:port1. Will not attempt to authenticate using SASL (unknown error)
2017-12-01 13:54:03,972 [myid:] - WARN [main-SendThread(ip1:port1):ClientCnxn$SendThread#1162] - Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect
java.net.NoRouteToHostException: No route to host
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:744)
at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:361)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1141)
2017-12-01 13:54:05,074 [myid:] - INFO [main-SendThread(ip1:port1):ClientCnxn$SendThread#1032] - Opening socket connection to server ip1:port1. Will not attempt to authenticate using SASL (unknown error)
2017-12-01 13:54:06,974 [myid:] - WARN [main-SendThread(ip1:port1):ClientCnxn$SendThread#1162] - Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect
I've done some search in add/remove zookeeper but didn't find a document for it. My zookeeper version (3.4.7) is not supported for dynamic reconfiguration (which is in zookeeper 3.5).
Is there a way I can manually remove/add zookeeper server from ensemble?
Thanks for your attention!
When I try to run my flow on Apache Flink standalone cluster I see the following exception:
java.lang.IllegalStateException: Update task on instance aaa0859f6af25decf1f5fc1821ffa55d # app-2 - 4 slots - URL: akka.tcp://flink#192.168.38.98:46369/user/taskmanager failed due to:
at org.apache.flink.runtime.executiongraph.Execution$6.onFailure(Execution.java:954)
at akka.dispatch.OnFailure.internal(Future.scala:228)
at akka.dispatch.OnFailure.internal(Future.scala:227)
at akka.dispatch.japi$CallbackBridge.apply(Future.scala:174)
at akka.dispatch.japi$CallbackBridge.apply(Future.scala:171)
at scala.PartialFunction$class.applyOrElse(PartialFunction.scala:123)
at scala.runtime.AbstractPartialFunction.applyOrElse(AbstractPartialFunction.scala:28)
at scala.concurrent.Future$$anonfun$onFailure$1.apply(Future.scala:136)
at scala.concurrent.Future$$anonfun$onFailure$1.apply(Future.scala:134)
at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:32)
at scala.concurrent.impl.ExecutionContextImpl$AdaptedForkJoinTask.exec(ExecutionContextImpl.scala:121)
at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
Caused by: akka.pattern.AskTimeoutException: Ask timed out on [Actor[akka.tcp://flink#192.168.38.98:46369/user/taskmanager#1804590378]] after [10000 ms]
at akka.pattern.PromiseActorRef$$anonfun$1.apply$mcV$sp(AskSupport.scala:333)
at akka.actor.Scheduler$$anon$7.run(Scheduler.scala:117)
at scala.concurrent.Future$InternalCallbackExecutor$.unbatchedExecute(Future.scala:599)
at scala.concurrent.BatchingExecutor$class.execute(BatchingExecutor.scala:109)
at scala.concurrent.Future$InternalCallbackExecutor$.execute(Future.scala:597)
at akka.actor.LightArrayRevolverScheduler$TaskHolder.executeTask(Scheduler.scala:467)
at akka.actor.LightArrayRevolverScheduler$$anon$8.executeBucket$1(Scheduler.scala:419)
at akka.actor.LightArrayRevolverScheduler$$anon$8.nextTick(Scheduler.scala:423)
at akka.actor.LightArrayRevolverScheduler$$anon$8.run(Scheduler.scala:375)
at java.lang.Thread.run(Thread.java:745)
Seems like port 46369 blocked by firewall. It is true because I read configuration section and open these ports only:
6121:
comment: Apache Flink TaskManager (Data Exchange)
6122:
comment: Apache Flink TaskManager (IPC)
6123:
comment: Apache Flink JobManager
6130:
comment: Apache Flink JobManager (BLOB Server)
8081:
comment: Apache Flink JobManager (Web UI)
The same ports described in flink-conf.yaml:
jobmanager.rpc.address: app-1.stag.local
jobmanager.rpc.port: 6123
jobmanager.heap.mb: 1024
taskmanager.heap.mb: 2048
taskmanager.numberOfTaskSlots: 4
taskmanager.memory.preallocate: false
blob.server.port: 6130
parallelism.default: 4
jobmanager.web.port: 8081
state.backend: jobmanager
restart-strategy: none
restart-strategy.fixed-delay.attempts: 2
restart-strategy.fixed-delay.delay: 60s
So, I have two questions:
This exception related to blocked ports. Right?
Which ports should I open on firewall for standalone Apache Flink cluster?
UPDATE 1
I found configuration problem in masters and slaves files (I skip new line separators between hosts described in these files). I fixed it and now I see other exceptions:
flink--taskmanager-0-app-1.stag.local.log
flink--taskmanager-0-app-2.stag.local.log
I have 2 nodes:
app-1.stag.local (with running job and task managers)
app-2.stag.local (with running task manager)
As you can see from these logs the app-1.stag.local task manager can't connect to other task manager:
java.io.IOException: Connecting the channel failed: Connecting to remote task manager + 'app-2.stag.local/192.168.38.98:35806' has failed. This might indicate that the remote task manager has been lost.
but app-2.stag.local has open port:
2016-03-18 16:24:14,347 INFO org.apache.flink.runtime.io.network.netty.NettyServer - Successful initialization (took 39 ms). Listening on SocketAddress /192.168.38.98:35806
So, I think problem related to firewall but I don't understand where I can configure this port (or range of ports) in Apache Flink.
I have found a problem: taskmanager.data.port parameter was set to 0 by default (but documentation say what it should be set to 6121).
So, I set this port in flink-conf.yaml and now all works fine.