I'm attempting to write a dataframe to OmniSci's database, basically a GPU-backed SQL database. I'm receiving a strange error when attempting to do so:
subs_dim.write.format("jdbc").option("url", "jdbc:mapd:EC2ADDRESS.compute-1.amazonaws.com:9091:mapd").option("driver", "com.mapd.jdbc.MapDDriver").option("dbtable", "subs_dim").option("user", "mapd").option("password", "INSTANCEID").save()
and receiving this error:
java.sql.SQLException: Connection failed - org.apache.thrift.transport.TTransportException: java.net.ConnectException: Connection timed out (Connection timed out)
Py4JJavaError: An error occurred while calling o309.save.
: java.sql.SQLException: Connection failed - org.apache.thrift.transport.TTransportException: java.net.ConnectException: Connection timed out (Connection timed out)
at com.mapd.jdbc.MapDConnection.(MapDConnection.java:113)
at com.mapd.jdbc.MapDDriver.connect(MapDDriver.java:55)
at org.apache.spark.sql.execution.datasources.jdbc.DriverWrapper.connect(DriverWrapper.scala:45)
at org.apache.spark.sql.execution.datasources.jdbc.JdbcUtils$$anonfun$createConnectionFactory$1.apply(JdbcUtils.scala:63)
at org.apache.spark.sql.execution.datasources.jdbc.JdbcUtils$$anonfun$createConnectionFactory$1.apply(JdbcUtils.scala:54)
at org.apache.spark.sql.execution.datasources.jdbc.JdbcRelationProvider.createRelation(JdbcRelationProvider.scala:63)
at org.apache.spark.sql.execution.datasources.SaveIntoDataSourceCommand.run(SaveIntoDataSourceCommand.scala:45)
at org.apache.spark.sql.execution.command.ExecutedCommandExec.sideEffectResult$lzycompute(commands.scala:72)
at org.apache.spark.sql.execution.command.ExecutedCommandExec.sideEffectResult(commands.scala:70)
at org.apache.spark.sql.execution.command.ExecutedCommandExec.doExecute(commands.scala:88)
at org.apache.spark.sql.execution.SparkPlan$$anonfun$execute$1.apply(SparkPlan.scala:150)
at org.apache.spark.sql.execution.SparkPlan$$anonfun$execute$1.apply(SparkPlan.scala:138)
at org.apache.spark.sql.execution.SparkPlan$$anonfun$executeQuery$5.apply(SparkPlan.scala:190)
at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151)
at org.apache.spark.sql.execution.SparkPlan.executeQuery(SparkPlan.scala:187)
at org.apache.spark.sql.execution.SparkPlan.execute(SparkPlan.scala:138)
at org.apache.spark.sql.execution.QueryExecution.toRdd$lzycompute(QueryExecution.scala:108)
at org.apache.spark.sql.execution.QueryExecution.toRdd(QueryExecution.scala:108)
at org.apache.spark.sql.DataFrameWriter$$anonfun$runCommand$1.apply(DataFrameWriter.scala:683)
at org.apache.spark.sql.DataFrameWriter$$anonfun$runCommand$1.apply(DataFrameWriter.scala:683)
at org.apache.spark.sql.execution.SQLExecution$$anonfun$withCustomExecutionEnv$1.apply(SQLExecution.scala:89)
at org.apache.spark.sql.execution.SQLExecution$.withSQLConfPropagated(SQLExecution.scala:175)
at org.apache.spark.sql.execution.SQLExecution$.withCustomExecutionEnv(SQLExecution.scala:84)
at org.apache.spark.sql.execution.SQLExecution$.withNewExecutionId(SQLExecution.scala:126)
at org.apache.spark.sql.DataFrameWriter.runCommand(DataFrameWriter.scala:683)
at org.apache.spark.sql.DataFrameWriter.saveToV1Source(DataFrameWriter.scala:287)
at org.apache.spark.sql.DataFrameWriter.save(DataFrameWriter.scala:281)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244)
at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:380)
at py4j.Gateway.invoke(Gateway.java:295)
at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)
at py4j.commands.CallCommand.execute(CallCommand.java:79)
at py4j.GatewayConnection.run(GatewayConnection.java:251)
at java.lang.Thread.run(Thread.java:748)
Anyone have ideas? I've pinged the server and confirmed that port is open
Related
Overview
I learn to migrate an Amazon RDS for Oracle Database to Amazon Redshift referring this tutorial https://docs.aws.amazon.com/dms/latest/sbs/CHAP_RDSOracle2Redshift.html
Trouble
I had a trouble the below question.
I would like migrate Oralce DB to Amazon Redshift with AWS SCT
I managed to get out of trouble with the help of people. However, I had a new trouble.
I try to connect to Amazon Redshift with AWS SCT. AWS Redshift failed to connect.
images
I saw the log file. An error was recorded in the log.
2020-12-31 20:56:16.358 [ 78] LOADER ERROR Connection to 'jdbc:redshift://oracletoredshiftdwusingdms-redshiftcluster-1dll5wg4tqddk.cnxpo3loreqp.us-east-1.redshift.amazonaws.com:5439/test' wasn't established. ERROR: code: 500150; message: [Amazon](500150) Error setting/closing connection: Connection timed out: connect.
2020-12-31 20:56:16.358 [ 78] LOADER ERROR Error chain:
[Amazon](500150) Error setting/closing connection: Connection timed out: connect.
------------------------------
java.sql.SQLException: [Amazon](500150) Error setting/closing connection: Connection timed out: connect.
at com.amazon.redshift.client.PGClient.connect(Unknown Source)
at com.amazon.redshift.client.PGClient.<init>(Unknown Source)
at com.amazon.redshift.core.PGJDBCConnection.connect(Unknown Source)
at com.amazon.jdbc.common.BaseConnectionFactory.doConnect(Unknown Source)
at com.amazon.jdbc.common.AbstractDriver.connect(Unknown Source)
at com.amazon.sct.dbloader.JdbcDriverAdapter.connect(JdbcDriverAdapter.java:30)
at java.sql/java.sql.DriverManager.getConnection(DriverManager.java:677)
at java.sql/java.sql.DriverManager.getConnection(DriverManager.java:189)
at com.amazon.sct.dbloader.DbLoaderDataSource.testConnection(DbLoaderDataSource.java:109)
at com.amazon.sct.dbloader.SqlLoaderEngine.testConnection(SqlLoaderEngine.java:1706)
at com.amazon.sct.dbloader.DbLoader.checkConnection(DbLoader.java:706)
at com.amazon.sct.dbloader.DbLoader.connect(DbLoader.java:406)
at com.amazon.sct.dbloader.DbLoaderContainer.checkAccessibility(DbLoaderContainer.java:30)
at com.amazon.sct.task.TestConnectionTask.call(TestConnectionTask.java:40)
at com.amazon.sct.task.TestConnectionTask.call(TestConnectionTask.java:18)
at javafx.concurrent.Task$TaskCallable.call(Task.java:1425)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
Caused by: com.amazon.support.exceptions.GeneralException: [Amazon](500150) Error setting/closing connection: Connection timed out: connect.
... 17 more
Caused by: java.net.ConnectException: Connection timed out: connect
at java.base/sun.nio.ch.Net.connect0(Native Method)
at java.base/sun.nio.ch.Net.connect(Net.java:476)
at java.base/sun.nio.ch.Net.connect(Net.java:468)
at java.base/sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:694)
at java.base/sun.nio.ch.SocketAdaptor.connect(SocketAdaptor.java:100)
at com.amazon.redshift.client.PGClient.connect(Unknown Source)
at com.amazon.redshift.client.PGClient.<init>(Unknown Source)
at com.amazon.redshift.core.PGJDBCConnection.connect(Unknown Source)
at com.amazon.jdbc.common.BaseConnectionFactory.doConnect(Unknown Source)
at com.amazon.jdbc.common.AbstractDriver.connect(Unknown Source)
at com.amazon.sct.dbloader.JdbcDriverAdapter.connect(JdbcDriverAdapter.java:30)
at java.sql/java.sql.DriverManager.getConnection(DriverManager.java:677)
at java.sql/java.sql.DriverManager.getConnection(DriverManager.java:189)
at com.amazon.sct.dbloader.DbLoaderDataSource.testConnection(DbLoaderDataSource.java:109)
at com.amazon.sct.dbloader.SqlLoaderEngine.testConnection(SqlLoaderEngine.java:1706)
at com.amazon.sct.dbloader.DbLoader.checkConnection(DbLoader.java:706)
at com.amazon.sct.dbloader.DbLoader.connect(DbLoader.java:406)
at com.amazon.sct.dbloader.DbLoaderContainer.checkAccessibility(DbLoaderContainer.java:30)
at com.amazon.sct.task.TestConnectionTask.call(TestConnectionTask.java:40)
at com.amazon.sct.task.TestConnectionTask.call(TestConnectionTask.java:18)
at javafx.concurrent.Task$TaskCallable.call(Task.java:1425)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.lang.Thread.run(Thread.java:834)
2020-12-31 20:56:16.359 [ 78] GENERAL INFO Test connecting to Amazon Redshift database finished in 0:00:21.113 sec with memory consumption of 516.25 MB (497.79 MB .. 516.25 MB).
2020-12-31 20:56:16.359 [ 78] GENERAL INFO Test connecting to Amazon Redshift database statistics:
GENERAL: 0:00:21.113 sec
2020-12-31 20:56:16.365 [ 19] GENERAL ERROR com.amazon.sct.dbloader.DbLoaderHandledException: Connection wasn't established. Check connection properties.
java.util.concurrent.ExecutionException: com.amazon.sct.dbloader.DbLoaderHandledException: Connection wasn't established. Check connection properties.
at java.base/java.util.concurrent.FutureTask.report(FutureTask.java:122)
at java.base/java.util.concurrent.FutureTask.get(FutureTask.java:191)
at com.amazon.sct.task.launcher.CommonTaskLauncher.lambda$run$0(CommonTaskLauncher.java:39)
at com.sun.javafx.binding.ExpressionHelper$Generic.fireValueChangedEvent(ExpressionHelper.java:360)
at com.sun.javafx.binding.ExpressionHelper.fireValueChangedEvent(ExpressionHelper.java:80)
at javafx.beans.property.ObjectPropertyBase.fireValueChangedEvent(ObjectPropertyBase.java:106)
at javafx.beans.property.ObjectPropertyBase.markInvalid(ObjectPropertyBase.java:113)
at javafx.beans.property.ObjectPropertyBase.set(ObjectPropertyBase.java:147)
at javafx.concurrent.Task.setState(Task.java:698)
at javafx.concurrent.Task$TaskCallable.lambda$call$2(Task.java:1455)
at com.sun.javafx.application.PlatformImpl.lambda$runLater$10(PlatformImpl.java:428)
at java.base/java.security.AccessController.doPrivileged(Native Method)
at com.sun.javafx.application.PlatformImpl.lambda$runLater$11(PlatformImpl.java:427)
at com.sun.glass.ui.InvokeLaterDispatcher$Future.run(InvokeLaterDispatcher.java:96)
at com.sun.glass.ui.win.WinApplication._enterNestedEventLoopImpl(Native Method)
at com.sun.glass.ui.win.WinApplication._enterNestedEventLoop(WinApplication.java:201)
at com.sun.glass.ui.Application.enterNestedEventLoop(Application.java:509)
at com.sun.glass.ui.EventLoop.enter(EventLoop.java:107)
at com.sun.javafx.tk.quantum.QuantumToolkit.enterNestedEventLoop(QuantumToolkit.java:635)
at javafx.stage.Stage.showAndWait(Stage.java:465)
at javafx.scene.control.HeavyweightDialog.showAndWait(HeavyweightDialog.java:162)
at javafx.scene.control.Dialog.showAndWait(Dialog.java:346)
at com.amazon.sct.handler.CreateConnectionHandler.createConnection(CreateConnectionHandler.java:51)
at com.amazon.sct.viewmodel.AppViewModel.createConnection(AppViewModel.java:1406)
at com.amazon.sct.viewmodel.AppViewModel.createTargetConnection(AppViewModel.java:774)
at com.amazon.sct.view.AppView.loadTarget(AppView.java:348)
at com.amazon.sct.view.AppView.lambda$initConnectionMenusBindings$27(AppView.java:1077)
at com.sun.javafx.event.CompositeEventHandler.dispatchBubblingEvent(CompositeEventHandler.java:86)
at com.sun.javafx.event.EventHandlerManager.dispatchBubblingEvent(EventHandlerManager.java:238)
at com.sun.javafx.event.EventHandlerManager.dispatchBubblingEvent(EventHandlerManager.java:191)
at com.sun.javafx.event.CompositeEventDispatcher.dispatchBubblingEvent(CompositeEventDispatcher.java:59)
at com.sun.javafx.event.BasicEventDispatcher.dispatchEvent(BasicEventDispatcher.java:58)
at com.sun.javafx.event.EventDispatchChainImpl.dispatchEvent(EventDispatchChainImpl.java:114)
at com.sun.javafx.event.BasicEventDispatcher.dispatchEvent(BasicEventDispatcher.java:56)
at com.sun.javafx.event.EventDispatchChainImpl.dispatchEvent(EventDispatchChainImpl.java:114)
at com.sun.javafx.event.BasicEventDispatcher.dispatchEvent(BasicEventDispatcher.java:56)
at com.sun.javafx.event.EventDispatchChainImpl.dispatchEvent(EventDispatchChainImpl.java:114)
at com.sun.javafx.event.EventUtil.fireEventImpl(EventUtil.java:74)
at com.sun.javafx.event.EventUtil.fireEvent(EventUtil.java:54)
at javafx.event.Event.fireEvent(Event.java:198)
at javafx.scene.Scene$MouseHandler.process(Scene.java:3862)
at javafx.scene.Scene.processMouseEvent(Scene.java:1849)
at javafx.scene.Scene$ScenePeerListener.mouseEvent(Scene.java:2590)
at com.sun.javafx.tk.quantum.GlassViewEventHandler$MouseEventNotification.run(GlassViewEventHandler.java:409)
at com.sun.javafx.tk.quantum.GlassViewEventHandler$MouseEventNotification.run(GlassViewEventHandler.java:299)
at java.base/java.security.AccessController.doPrivileged(Native Method)
at com.sun.javafx.tk.quantum.GlassViewEventHandler.lambda$handleMouseEvent$2(GlassViewEventHandler.java:447)
at com.sun.javafx.tk.quantum.QuantumToolkit.runWithoutRenderLock(QuantumToolkit.java:412)
at com.sun.javafx.tk.quantum.GlassViewEventHandler.handleMouseEvent(GlassViewEventHandler.java:446)
at com.sun.glass.ui.View.handleMouseEvent(View.java:556)
at com.sun.glass.ui.View.notifyMouse(View.java:942)
at com.sun.glass.ui.win.WinApplication._runLoop(Native Method)
at com.sun.glass.ui.win.WinApplication.lambda$runLoop$3(WinApplication.java:174)
at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: com.amazon.sct.dbloader.DbLoaderHandledException: Connection wasn't established. Check connection properties.
at com.amazon.sct.dbloader.DbLoader.checkConnection(DbLoader.java:721)
at com.amazon.sct.dbloader.DbLoader.connect(DbLoader.java:406)
at com.amazon.sct.dbloader.DbLoaderContainer.checkAccessibility(DbLoaderContainer.java:30)
at com.amazon.sct.task.TestConnectionTask.call(TestConnectionTask.java:40)
at com.amazon.sct.task.TestConnectionTask.call(TestConnectionTask.java:18)
at javafx.concurrent.Task$TaskCallable.call(Task.java:1425)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
... 1 more
What I try
[Amazon](500150) Error setting/closing connection: Connection timed out: connect.
I think the above error is similar to the issues.
So, I launched the Security Groups console and add [Rule: Type=Redshift, Source=MyIP].
I saved it and I try to connect again.
Unfortunately, AWS Redshift failed to connect with same error.
I have no idea to do next time.
What I should do?
To troubleshoot this issue, you can use following steps;
(1) To troubleshoot faster, instead of your java application, you can use ping command to check network reachability from your host/machine/laptop/desktop.
(2) If step one is successful, then try following commands on your host/machine/laptop/desktop using Windows command prompt or Windows PowerShell or Any Linux shell/command line tool;
nc -vz -w 3 "${DB_HOST}" "${DB_PORT}"
OR
telnet "${DB_HOST}" "${DB_PORT}"
(3) If commands from step (1) or (2) are failing with "Connection timed out" error, then
(3.1) Use VPC Reachability Analyzer : Create and analyze path with Source type Network Interfaces and Source as Network Interface Id of the machine where you application is deployed. Furthermore, select Destination type as Network Interfaces and Destination as network Interface Id of the AWS Red Shift. Put Destination port as 5439, keep Protocol as TCP.
Note: It takes few minutes for completion of this path analysis.
OR
(3.2) If you have enabled VPC Flow Logs, then you can trace vpc flow logs to check, which AWS resource is rejecting the network traffic. For VPC flow logs information please refer this document from AWS.
I have a Spring boot + Spring Batch Application with External Source as : Oracle DB , Destination as : SQL Server in my Stage Server. I have a read user for oracle. I am getting the below error while reading the data from Source. Please find the below error logs.
Based on the error log, I added this property spring.batch.initializer.enabled=true
but it still gives me same error.
Can you please advise how to fix it ?
ERROR [org.springframework.scheduling.support.TaskUtils$LoggingErrorHandler] (pool-9-thread-1) Unexpected error occurred in scheduled task.: org.springframework.jdbc.BadSqlGrammarException: PreparedStatementCallback; bad SQL grammar [INSERT into BATCH_JOB_INSTANCE(JOB_INSTANCE_ID, JOB_NAME, JOB_KEY, VERSION) values (?, ?, ?, ?)]; nested exception is java.sql.SQLSyntaxErrorException: ORA-01950: no privileges on tablespace 'USERS'
at org.springframework.jdbc.support.SQLExceptionSubclassTranslator.doTranslate(SQLExceptionSubclassTranslator.java:91)
at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:73)
at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:81)
at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:649)
at org.springframework.jdbc.core.JdbcTemplate.update(JdbcTemplate.java:870)
at org.springframework.jdbc.core.JdbcTemplate.update(JdbcTemplate.java:931)
at org.springframework.jdbc.core.JdbcTemplate.update(JdbcTemplate.java:936)
at org.springframework.batch.core.repository.dao.JdbcJobInstanceDao.createJobInstance(JdbcJobInstanceDao.java:115)
at org.springframework.batch.core.repository.support.SimpleJobRepository.createJobExecution(SimpleJobRepository.java:135)
at sun.reflect.GeneratedMethodAccessor51.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:333)
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:190)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
at org.springframework.transaction.interceptor.TransactionInterceptor$1.proceedWithInvocation(TransactionInterceptor.java:99)
at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:282)
at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:96)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
at org.springframework.batch.core.repository.support.AbstractJobRepositoryFactoryBean$1.invoke(AbstractJobRepositoryFactoryBean.java:172)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:213)
at com.sun.proxy.$Proxy92.createJobExecution(Unknown Source)
at org.springframework.batch.core.launch.support.SimpleJobLauncher.run(SimpleJobLauncher.java:125)
at sun.reflect.GeneratedMethodAccessor48.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:333)
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:190)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
at org.springframework.batch.core.configuration.annotation.SimpleBatchConfiguration$PassthruAdvice.invoke(SimpleBatchConfiguration.java:127)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:213)
at com.sun.proxy.$Proxy90.run(Unknown Source)
at com.elm.salamah.scheduler.batch.configuration.EmployeeBatchConfiguration.performThirdJob(EmployeeBatchConfiguration.java:90)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at org.springframework.scheduling.support.ScheduledMethodRunnable.run(ScheduledMethodRunnable.java:65)
at org.springframework.scheduling.support.DelegatingErrorHandlingRunnable.run(DelegatingErrorHandlingRunnable.java:54)
at org.springframework.scheduling.concurrent.ReschedulingRunnable.run(ReschedulingRunnable.java:81)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.sql.SQLSyntaxErrorException: ORA-01950: no privileges on tablespace 'USERS'
at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:445)
at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:396)
at oracle.jdbc.driver.T4C8Oall.processError(T4C8Oall.java:879)
at oracle.jdbc.driver.T4CTTIfun.receive(T4CTTIfun.java:450)
at oracle.jdbc.driver.T4CTTIfun.doRPC(T4CTTIfun.java:192)
at oracle.jdbc.driver.T4C8Oall.doOALL(T4C8Oall.java:531)
at oracle.jdbc.driver.T4CPreparedStatement.doOall8(T4CPreparedStatement.java:207)
at oracle.jdbc.driver.T4CPreparedStatement.executeForRows(T4CPreparedStatement.java:1044)
at oracle.jdbc.driver.OracleStatement.doExecuteWithTimeout(OracleStatement.java:1329)
at oracle.jdbc.driver.OraclePreparedStatement.executeInternal(OraclePreparedStatement.java:3584)
at oracle.jdbc.driver.OraclePreparedStatement.executeUpdate(OraclePreparedStatement.java:3665)
at oracle.jdbc.driver.OraclePreparedStatementWrapper.executeUpdate(OraclePreparedStatementWrapper.java:1352)
at sun.reflect.GeneratedMethodAccessor54.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at org.apache.tomcat.jdbc.pool.StatementFacade$StatementProxy.invoke(StatementFacade.java:114)
at com.sun.proxy.$Proxy112.executeUpdate(Unknown Source)
at org.springframework.jdbc.core.JdbcTemplate$2.doInPreparedStatement(JdbcTemplate.java:877)
at org.springframework.jdbc.core.JdbcTemplate$2.doInPreparedStatement(JdbcTemplate.java:870)
at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:633)
... 45 more
To manage the job run status and to have restart facility Spring batch creates/uses these tables.
You can use MapJobRepositoryFactoryBean if you don’t want to persist these tables in database.
https://docs.spring.io/spring-batch/trunk/apidocs/org/springframework/batch/core/repository/support/MapJobRepositoryFactoryBean.html
sample java config for the same
#Bean
public MapJobRepositoryFactoryBean mapJobRepositoryFactoryBean(ResourcelessTransactionManager transactionManager) {
MapJobRepositoryFactoryBean mapJobRepositoryFactoryBean = new MapJobRepositoryFactoryBean(transactionManager);
mapJobRepositoryFactoryBean.setTransactionManager(transactionManager);
return mapJobRepositoryFactoryBean;
}
#Bean
public ResourcelessTransactionManager transactionManager() {
return new ResourcelessTransactionManager();
}
Other Solution
you can specify different datasource (for which you have insert access ) for creating job repository .
I'm having issues adding a MSSQL SERVER connection to Pentaho Data Intergration.
I tried the following:
I have downloaded the latest jdbc drivers (sqljdbc42) and moved them
to the lib folder
I have created a new user in SQL SERVER and gave it all rights...didn't help.
very sure about username/password and login settings
reinstalled pentaho multiple times, reinstalled the jdbc drivers
I'm have a mac but use windows on parallels, and I work from this environment.
This is the error that I'm getting:
Error connecting to database [Velo]
:org.pentaho.di.core.exception.KettleDatabaseException: Error
occurred while trying to connect to the database
Error connecting to database: (using class
com.microsoft.sqlserver.jdbc.SQLServerDriver) Login failed for user
'DBklaartje'. ClientConnectionId:33c3d571-4128-4c2d-b5a6-1290e06438e8
org.pentaho.di.core.exception.KettleDatabaseException: Error occurred
while trying to connect to the database
Error connecting to database: (using class
com.microsoft.sqlserver.jdbc.SQLServerDriver) Login failed for user
'DBklaartje'. ClientConnectionId:33c3d571-4128-4c2d-b5a6-1290e06438e8
at
org.pentaho.di.core.database.Database.normalConnect(Database.java:472)
at org.pentaho.di.core.database.Database.connect(Database.java:370)
at org.pentaho.di.core.database.Database.connect(Database.java:341)
at org.pentaho.di.core.database.Database.connect(Database.java:331)
at
org.pentaho.di.core.database.DatabaseFactory.getConnectionTestReport(DatabaseFactory.java:80)
at
org.pentaho.di.core.database.DatabaseMeta.testConnection(DatabaseMeta.java:2783)
at
org.pentaho.ui.database.event.DataHandler.testDatabaseConnection(DataHandler.java:597)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at
sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at
java.lang.reflect.Method.invoke(Unknown Source) at
org.pentaho.ui.xul.impl.AbstractXulDomContainer.invoke(AbstractXulDomContainer.java:313)
at
org.pentaho.ui.xul.impl.AbstractXulComponent.invoke(AbstractXulComponent.java:157)
at
org.pentaho.ui.xul.impl.AbstractXulComponent.invoke(AbstractXulComponent.java:141)
at
org.pentaho.ui.xul.swt.tags.SwtButton.access$500(SwtButton.java:43)
at
org.pentaho.ui.xul.swt.tags.SwtButton$4.widgetSelected(SwtButton.java:137)
at org.eclipse.swt.widgets.TypedListener.handleEvent(Unknown Source)
at org.eclipse.swt.widgets.EventTable.sendEvent(Unknown Source) at
org.eclipse.swt.widgets.Display.sendEvent(Unknown Source) at
org.eclipse.swt.widgets.Widget.sendEvent(Unknown Source) at
org.eclipse.swt.widgets.Display.runDeferredEvents(Unknown Source) at
org.eclipse.swt.widgets.Display.readAndDispatch(Unknown Source) at
org.eclipse.jface.window.Window.runEventLoop(Window.java:820) at
org.eclipse.jface.window.Window.open(Window.java:796) at
org.pentaho.di.ui.xul.KettleDialog.show(KettleDialog.java:80) at
org.pentaho.di.ui.xul.KettleDialog.show(KettleDialog.java:47) at
org.pentaho.di.ui.core.database.dialog.XulDatabaseDialog.open(XulDatabaseDialog.java:116)
at
org.pentaho.di.ui.core.database.dialog.DatabaseDialog.open(DatabaseDialog.java:60)
at
org.pentaho.di.ui.repository.repositoryexplorer.controllers.ConnectionsController.editConnection(ConnectionsController.java:363)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at
sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at
java.lang.reflect.Method.invoke(Unknown Source) at
org.pentaho.ui.xul.impl.AbstractXulDomContainer.invoke(AbstractXulDomContainer.java:313)
at
org.pentaho.ui.xul.impl.AbstractXulComponent.invoke(AbstractXulComponent.java:157)
at
org.pentaho.ui.xul.impl.AbstractXulComponent.invoke(AbstractXulComponent.java:141)
at
org.pentaho.ui.xul.swt.tags.SwtButton.access$300(SwtButton.java:43)
at
org.pentaho.ui.xul.swt.tags.SwtButton$2.mouseUp(SwtButton.java:103)
at org.eclipse.swt.widgets.TypedListener.handleEvent(Unknown Source)
at org.eclipse.swt.widgets.EventTable.sendEvent(Unknown Source) at
org.eclipse.swt.widgets.Display.sendEvent(Unknown Source) at
org.eclipse.swt.widgets.Widget.sendEvent(Unknown Source) at
org.eclipse.swt.widgets.Display.runDeferredEvents(Unknown Source) at
org.eclipse.swt.widgets.Display.readAndDispatch(Unknown Source) at
org.eclipse.jface.window.Window.runEventLoop(Window.java:820) at
org.eclipse.jface.window.Window.open(Window.java:796) at
org.pentaho.di.ui.xul.KettleDialog.show(KettleDialog.java:80) at
org.pentaho.di.ui.xul.KettleDialog.show(KettleDialog.java:47) at
org.pentaho.di.ui.repository.repositoryexplorer.RepositoryExplorer.show(RepositoryExplorer.java:146)
at org.pentaho.di.ui.spoon.Spoon$24$1.run(Spoon.java:4045) at
org.eclipse.swt.widgets.RunnableLock.run(Unknown Source) at
org.eclipse.swt.widgets.Synchronizer.runAsyncMessages(Unknown Source)
at org.eclipse.swt.widgets.Display.runAsyncMessages(Unknown Source)
at org.eclipse.swt.widgets.Display.readAndDispatch(Unknown Source)
at org.pentaho.di.ui.spoon.Spoon.readAndDispatch(Spoon.java:1366) at
org.pentaho.di.ui.spoon.Spoon.waitForDispose(Spoon.java:8022) at
org.pentaho.di.ui.spoon.Spoon.start(Spoon.java:9277) at
org.pentaho.di.ui.spoon.Spoon.main(Spoon.java:692) at
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at
sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at
java.lang.reflect.Method.invoke(Unknown Source) at
org.pentaho.commons.launcher.Launcher.main(Launcher.java:92) Caused
by: org.pentaho.di.core.exception.KettleDatabaseException: Error
connecting to database: (using class
com.microsoft.sqlserver.jdbc.SQLServerDriver) Login failed for user
'DBklaartje'. ClientConnectionId:33c3d571-4128-4c2d-b5a6-1290e06438e8
at
org.pentaho.di.core.database.Database.connectUsingClass(Database.java:585)
at
org.pentaho.di.core.database.Database.normalConnect(Database.java:456)
... 62 more Caused by:
com.microsoft.sqlserver.jdbc.SQLServerException: Login failed for user
'DBklaartje'. ClientConnectionId:33c3d571-4128-4c2d-b5a6-1290e06438e8
at
com.microsoft.sqlserver.jdbc.SQLServerException.makeFromDatabaseError(SQLServerException.java:217)
at
com.microsoft.sqlserver.jdbc.TDSTokenHandler.onEOF(tdsparser.java:279)
at com.microsoft.sqlserver.jdbc.TDSParser.parse(tdsparser.java:99)
at
com.microsoft.sqlserver.jdbc.SQLServerConnection.sendLogon(SQLServerConnection.java:4346)
at
com.microsoft.sqlserver.jdbc.SQLServerConnection.logon(SQLServerConnection.java:3160)
at
com.microsoft.sqlserver.jdbc.SQLServerConnection.access$100(SQLServerConnection.java:43)
at
com.microsoft.sqlserver.jdbc.SQLServerConnection$LogonCommand.doExecute(SQLServerConnection.java:3123)
at
com.microsoft.sqlserver.jdbc.TDSCommand.execute(IOBuffer.java:7505)
at
com.microsoft.sqlserver.jdbc.SQLServerConnection.executeCommand(SQLServerConnection.java:2445)
at
com.microsoft.sqlserver.jdbc.SQLServerConnection.connectHelper(SQLServerConnection.java:1981)
at
com.microsoft.sqlserver.jdbc.SQLServerConnection.login(SQLServerConnection.java:1628)
at
com.microsoft.sqlserver.jdbc.SQLServerConnection.connectInternal(SQLServerConnection.java:1459)
at
com.microsoft.sqlserver.jdbc.SQLServerConnection.connect(SQLServerConnection.java:773)
at
com.microsoft.sqlserver.jdbc.SQLServerDriver.connect(SQLServerDriver.java:1168)
at java.sql.DriverManager.getConnection(Unknown Source) at
java.sql.DriverManager.getConnection(Unknown Source) at
org.pentaho.di.core.database.Database.connectUsingClass(Database.java:561)
... 63 more
Hostname :DESKTOP-LM88EPQ Port :1433 Database name
:Velo
Apparently this issue was caused because of an authentication issue with the driver. If you download your drivers (sqljdbc), also copy the file 'sqljbdc_auth.dll' to %PENTAHO_HOME%\design-tools\data-integration\libswt\win64
This will make sure your users and logins get the proper authorisation.
On the Database connection definition, between the Test and the Explore button, you have a Feature List which gives you all the parameters used by the PDI for that connection.
Check the the driver class on line 20 (sometime you have more than one on your classpath).
Then drop the url on line 21, in your SQL-Developer or equivalent.
You can also write your own java class ; replace the driver, url, username, password by the walue ayou are using; compile it; run it; it tell you what went wrong.
import java.sql.DriverManager;
public class TestConnection {
public static void main(String[] args) throws Exception{
Class.forName("com.microsoft.sqlserver.jdbc.SQLServerDriver");
DriverManager.getConnection("url", "username", "password");
System.out.println("Ok");
}
}
I am getting below mentioned error very frequently at the time of sending builds to codename one server.
Building for username: myemail#gmail.com
javax.net.ssl.SSLHandshakeException: Remote host closed connection
during handshake at
sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:992) at
sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1375)
at
sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1403)
at
sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1387)
at
sun.net.www.protocol.https.HttpsClient.afterConnect(HttpsClient.java:559)
at
sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:185)
at
sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1513)
at
sun.net.www.protocol.http.HttpURLConnection.access$200(HttpURLConnection.java:90)
at
sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1433)
at
sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1431)
at java.security.AccessController.doPrivileged(Native Method) at
java.security.AccessController.doPrivilegedWithCombiner(AccessController.java:782)
at
sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1430)
at
sun.net.www.protocol.https.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:254)
at java.net.URL.openStream(URL.java:1045) at
com.codename1.build.client.BuildProcess.login(BuildProcess.java:119)
at
com.codename1.build.client.CodeNameOneBuildTask.execute(CodeNameOneBuildTask.java:249)
at
org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:292)
at sun.reflect.GeneratedMethodAccessor85.invoke(Unknown Source) at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498) at
org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:106)
at org.apache.tools.ant.Task.perform(Task.java:348) at
org.apache.tools.ant.Target.execute(Target.java:435) at
org.apache.tools.ant.Target.performTasks(Target.java:456) at
org.apache.tools.ant.Project.executeSortedTargets(Project.java:1393)
at org.apache.tools.ant.Project.executeTarget(Project.java:1364) at
org.apache.tools.ant.helper.DefaultExecutor.executeTargets(DefaultExecutor.java:41)
at org.apache.tools.ant.Project.executeTargets(Project.java:1248) at
org.apache.tools.ant.module.bridge.impl.BridgeImpl.run(BridgeImpl.java:286)
at
org.apache.tools.ant.module.run.TargetExecutor.run(TargetExecutor.java:555)
at
org.netbeans.core.execution.RunClassThread.run(RunClassThread.java:153)
Caused by: java.io.EOFException: SSL peer shut down incorrectly at
sun.security.ssl.InputRecord.read(InputRecord.java:505) at
sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:973) ...
31 more /home/hplus/NetBeansProjects/HplusNew/build.xml:221: Error in
server build process BUILD FAILED (total time: 15 minutes 19 seconds)
Please advice.
I'm trying to connect using kettle spoon installed in one of our db server but I'm getting the error below. I tried using the same .ktr file on my laptop and it is working fine..
I'm sure that the username and password are correct..
java.lang.Exception:
Error connecting to Salesforce!
; nested exception is:
java.net.ConnectException: Connection timed out: connect
at org.pentaho.di.ui.trans.steps.salesforceinput.SalesforceInputDialog.test(SalesforceInputDialog.java:1396)
at org.pentaho.di.ui.trans.steps.salesforceinput.SalesforceInputDialog.access$2000(SalesforceInputDialog.java:96)
at org.pentaho.di.ui.trans.steps.salesforceinput.SalesforceInputDialog$23.handleEvent(SalesforceInputDialog.java:1229)
at org.eclipse.swt.widgets.EventTable.sendEvent(Unknown Source)
at org.eclipse.swt.widgets.Widget.sendEvent(Unknown Source)
at org.eclipse.swt.widgets.Display.runDeferredEvents(Unknown Source)
at org.eclipse.swt.widgets.Display.readAndDispatch(Unknown Source)
at org.pentaho.di.ui.trans.steps.salesforceinput.SalesforceInputDialog.open(SalesforceInputDialog.java:1292)
at org.pentaho.di.ui.spoon.delegates.SpoonStepsDelegate.editStep(SpoonStepsDelegate.java:136)
at org.pentaho.di.ui.spoon.Spoon.editStep(Spoon.java:7756)
at org.pentaho.di.ui.spoon.trans.TransGraph.editStep(TransGraph.java:2756)
at org.pentaho.di.ui.spoon.trans.TransGraph.mouseDoubleClick(TransGraph.java:705)
at org.eclipse.swt.widgets.TypedListener.handleEvent(Unknown Source)
at org.eclipse.swt.widgets.EventTable.sendEvent(Unknown Source)
at org.eclipse.swt.widgets.Widget.sendEvent(Unknown Source)
at org.eclipse.swt.widgets.Display.runDeferredEvents(Unknown Source)
at org.eclipse.swt.widgets.Display.readAndDispatch(Unknown Source)
at org.pentaho.di.ui.spoon.Spoon.readAndDispatch(Spoon.java:1183)
at org.pentaho.di.ui.spoon.Spoon.start(Spoon.java:6968)
at org.pentaho.di.ui.spoon.Spoon.main(Spoon.java:567)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at org.pentaho.commons.launcher.Launcher.main(Launcher.java:134)
seems kettle is unable to reach the salesforce endpoint on the network.
try from the server where is kettle, open a console and try:
curl -i 'https://yourSalesforceWebservice.foo/services/Soap/u/21.0'
or
traceroute yourSalesforceWebservice.foo
(put the domain or the ip )