I'm trying to use Camel's hdfs2 component to read from a folder in HDFS, but my company uses Kerberos authentication and I get an error telling me that I need to provide authentication:
Caused by: org.apache.hadoop.ipc.RemoteException: SIMPLE authentication is not enabled. Available:[TOKEN, KERBEROS]
at org.apache.hadoop.ipc.Client.call(Client.java:1475) ~[hadoop-common-2.7.2.jar:na]
at org.apache.hadoop.ipc.Client.call(Client.java:1412) ~[hadoop-common-2.7.2.jar:na]
at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:229) ~[hadoop-common-2.7.2.jar:na]
at com.sun.proxy.$Proxy62.getFileInfo(Unknown Source) ~[na:na]
at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.getFileInfo(ClientNamenodeProtocolTranslatorPB.java:771) ~[hadoop-hdfs-2.7.2.jar:na]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_111]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_111]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_111]
at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_111]
at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:191) ~[hadoop-common-2.7.2.jar:na]
at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102) ~[hadoop-common-2.7.2.jar:na]
at com.sun.proxy.$Proxy63.getFileInfo(Unknown Source) ~[na:na]
at org.apache.hadoop.hdfs.DFSClient.getFileInfo(DFSClient.java:2108) ~[hadoop-hdfs-2.7.2.jar:na]
... 16 common frames omitted
I couldn't find any reference to authentication support in the component documentation.
I've got a workaround by using the jetty component to call WebHDFS and manually providing a token, but I'd prefer to use the built-in component. Is there any way to authenticate with Kerberos through the hdfs2 component?
Related
I am trying to submit a flink job of version 1.11.3 . The actual application is developed in flink 1.8.1.
Since the Hadoop cluster is 3.2.0 apache I downloaded flink 1.11.3 (flink-1.11.3-bin-scala_2.11.tgz) and tried to submit the job.
while submitting facing the below mentioned exception . I have set the HADOOP parameters :
export HADOOP_CONF_DIR=/etc/hadoop/conf
export HADOOP_CLASSPATH='hadoop classpath'
Is there any changes I need to do it the pom file to overcome this
org.apache.flink.client.program.ProgramInvocationException: The main method caused an error: No ExecutorFactory found to execute the application.
at org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:302)
at org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:198)
at org.apache.flink.client.ClientUtils.executeProgram(ClientUtils.java:149)
at org.apache.flink.client.cli.CliFrontend.executeProgram(CliFrontend.java:699)
at org.apache.flink.client.cli.CliFrontend.run(CliFrontend.java:232)
at org.apache.flink.client.cli.CliFrontend.parseParameters(CliFrontend.java:916)
at org.apache.flink.client.cli.CliFrontend.lambda$main$10(CliFrontend.java:992)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1730)
at org.apache.flink.runtime.security.contexts.HadoopSecurityContext.runSecured(HadoopSecurityContext.java:41)
at org.apache.flink.client.cli.CliFrontend.main(CliFrontend.java:992)
Caused by: java.lang.IllegalStateException: No ExecutorFactory found to execute the application.
at org.apache.flink.core.execution.DefaultExecutorServiceLoader.getExecutorFactory(DefaultExecutorServiceLoader.java:84)
at org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.executeAsync(StreamExecutionEnvironment.java:1809)
at org.apache.flink.client.program.StreamContextEnvironment.executeAsync(StreamContextEnvironment.java:128)
at org.apache.flink.client.program.StreamContextEnvironment.execute(StreamContextEnvironment.java:76)
at org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.execute(StreamExecutionEnvironment.java:1700)
at org.sapphire.appspayload.StreamingJob.main(StreamingJob.java:214)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:288)
I want to test Camel 2.24.x routes in Karaf 4.2.x. I use Pax-Exam 4.13.1 to run Karaf. The Camel Testing Manual does not cover Karaf testing. For older Karaf and Camel versions there are blog articles which demonstrate how to do that by extending a test class from CamelTestSupport:
Testing Camel JPA routes with Pax-Exam and Karaf
Testing (utest and itest) Apache Camel Blueprint route
However, they are so old that the code no longer runs without adjustments. The former is pretty hopeless, I have stopped trying to fix it. For the latter I was able to get it to work on Karaf 2.4 with Camel 2.12.1 on JDK 8, but not with the current Karaf 4.2.7 using Camel 2.24.2.
With the current Karaf 4 and Camel 2.24 I get the following error message when CamelTestSupport tries to create a JndiContext in CamelTestSupport#createRegistry.
java.lang.ClassNotFoundException: org.apache.camel.util.jndi.CamelInitialContextFactory
at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at org.apache.felix.framework.BundleWiringImpl.doImplicitBootDelegation(BundleWiringImpl.java:1859)
at org.apache.felix.framework.BundleWiringImpl.tryImplicitBootDelegation(BundleWiringImpl.java:1788)
at org.apache.felix.framework.BundleWiringImpl.searchDynamicImports(BundleWiringImpl.java:1741)
at org.apache.felix.framework.BundleWiringImpl.findClassOrResourceByDelegation(BundleWiringImpl.java:1617)
at org.apache.felix.framework.BundleWiringImpl.access$200(BundleWiringImpl.java:80)
at org.apache.felix.framework.BundleWiringImpl$BundleClassLoader.loadClass(BundleWiringImpl.java:2053)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:348)
at com.sun.naming.internal.VersionHelper12.loadClass(VersionHelper12.java:91)
at com.sun.naming.internal.VersionHelper12.loadClass(VersionHelper12.java:61)
at javax.naming.spi.NamingManager.getInitialContext(NamingManager.java:672)
at javax.naming.InitialContext.getDefaultInitCtx(InitialContext.java:313)
at javax.naming.InitialContext.init(InitialContext.java:244)
at javax.naming.InitialContext.<init>(InitialContext.java:216)
at org.apache.camel.test.junit4.CamelTestSupport.createJndiContext(CamelTestSupport.java:790)
at org.apache.camel.test.junit4.CamelTestSupport.createRegistry(CamelTestSupport.java:776)
at org.apache.camel.test.junit4.CamelTestSupport.createCamelContext(CamelTestSupport.java:770)
at org.apache.camel.test.junit4.CamelTestSupport.doSetUp(CamelTestSupport.java:325)
at org.apache.camel.test.junit4.CamelTestSupport.setUp(CamelTestSupport.java:265)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:24)
at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55)
at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55)
at org.junit.rules.RunRules.evaluate(RunRules.java:20)
The missing class is in camel-core and it used to be working with the older versions of Camel and Karaf, so I am at a loss why that class would not be available when using the latest Karaf and Camel.
What could be the reason why the CamelInitialContextFactory cannot be found? As in the older version, I install camel-blueprint and camel-test in my test, so I would expect that camel-core should be included.
Is it still a good idea to work with CamelTestSupport on Karaf? While there is a CamelKarafTestSupport which seems to demonstrate that approach, I could not find a working example for such a test and I could not get it to work myself either. With CamelKarafTestSupport I get an error telling me that ProbeInvoker never came up:
org.ops4j.pax.swissbox.tracker.ServiceLookupException: gave up waiting for service org.ops4j.pax.exam.ProbeInvoker
at org.ops4j.pax.swissbox.tracker.ServiceLookup.getService(ServiceLookup.java:199)
at org.ops4j.pax.swissbox.tracker.ServiceLookup.getService(ServiceLookup.java:136)
at org.ops4j.pax.exam.rbc.internal.RemoteBundleContextImpl.remoteCall(RemoteBundleContextImpl.java:81)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:357)
at sun.rmi.transport.Transport$1.run(Transport.java:200)
at sun.rmi.transport.Transport$1.run(Transport.java:197)
at java.security.AccessController.doPrivileged(Native Method)
at sun.rmi.transport.Transport.serviceCall(Transport.java:196)
at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:573)
at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:834)
at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.lambda$run$0(TCPTransport.java:688)
at java.security.AccessController.doPrivileged(Native Method)
at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:687)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
at sun.rmi.transport.StreamRemoteCall.exceptionReceivedFromServer(StreamRemoteCall.java:283)
at sun.rmi.transport.StreamRemoteCall.executeCall(StreamRemoteCall.java:260)
at sun.rmi.server.UnicastRef.invoke(UnicastRef.java:161)
at java.rmi.server.RemoteObjectInvocationHandler.invokeRemoteMethod(RemoteObjectInvocationHandler.java:227)
at java.rmi.server.RemoteObjectInvocationHandler.invoke(RemoteObjectInvocationHandler.java:179)
at com.sun.proxy.$Proxy15.remoteCall(Unknown Source)
at org.ops4j.pax.exam.rbc.client.intern.RemoteBundleContextClientImpl$1.invoke(RemoteBundleContextClientImpl.java:102)
at com.sun.proxy.$Proxy16.call(Unknown Source)
at org.ops4j.pax.exam.rbc.client.intern.RemoteBundleContextClientImpl.call(RemoteBundleContextClientImpl.java:290)
at org.ops4j.pax.exam.container.remote.RBCRemoteTarget.call(RBCRemoteTarget.java:60)
at org.ops4j.pax.exam.karaf.container.internal.KarafTestContainer.call(KarafTestContainer.java:665)
at org.ops4j.pax.exam.spi.reactors.EagerSingleStagedReactor.invoke(EagerSingleStagedReactor.java:109)
at org.ops4j.pax.exam.junit.impl.ProbeRunner$2.evaluate(ProbeRunner.java:267)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
at org.ops4j.pax.exam.junit.impl.ProbeRunner.run(ProbeRunner.java:98)
at org.ops4j.pax.exam.junit.PaxExam.run(PaxExam.java:93)
at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:68)
at com.intellij.rt.execution.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:47)
at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:242)
at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:70)
Should I go a different route, maybe the AbstractFeatureTest from camel-test-karaf, which is used extensively by camel-itest-karaf?
The advice in the pax-exam group was to use KarafTestSupport from org.apache.camel.itests:common for integration tests in pax-exam and stick to CamelTestSupport for unit tests.
With that, I got the camel itests running again: https://github.com/dschulten/blog-camel-blueprint/tree/karaf4-paxexam4.
In Karaf 4.2.7 KarafTestSupport requires numerous spring and activemq version system properties (see the configuration of the maven-surefire-plugin), that will go away as of Karaf 4.2.8.
It is still possible to use camel mock routes in KarafTestSupport-based integration tests.
Hi I am new to web development and am trying to get activejdbc to connect to a cloud sql instance... I am getting the following stacktrace:
[ERROR] Failed to execute goal org.javalite:db-migrator-maven-plugin:1.4.11:migrate (dev_migrations) on project activeweb-simple: Execution dev_migrations of go
al org.javalite:db-migrator-maven-plugin:1.4.11:migrate failed: com.mysql.jdbc.exceptions.jdbc4.MySQLNonTransientConnectionException: Could not create socket fa
ctory 'com.google.cloud.sql.mysql.SocketFactory' due to underlying exception: -> [Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal org.javalite:db-migrator-maven-plugin:1.4.11:migrate (dev_migrations) on project
activeweb-simple: Execution dev_migrations of goal org.javalite:db-migrator-maven-plugin:1.4.11:migrate failed: com.mysql.jdbc.exceptions.jdbc4.MySQLNonTransien
tConnectionException: Could not create socket factory 'com.google.cloud.sql.mysql.SocketFactory' due to underlying exception:
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:213)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:154)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:146)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:117)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:81)
at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:51)
at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:128)
at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:309)
at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:194)
at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:107)
at org.apache.maven.cli.MavenCli.execute(MavenCli.java:993)
at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:345)
at org.apache.maven.cli.MavenCli.main(MavenCli.java:191)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289)
at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229)
at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415)
at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356)
Caused by: org.apache.maven.plugin.PluginExecutionException: Execution dev_migrations of goal org.javalite:db-migrator-maven-plugin:1.4.11:migrate failed: com.m
ysql.jdbc.exceptions.jdbc4.MySQLNonTransientConnectionException: Could not create socket factory 'com.google.cloud.sql.mysql.SocketFactory' due to underlying ex
ception:
at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:145)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:208)
... 20 more
Caused by: org.javalite.db_migrator.MigrationException: com.mysql.jdbc.exceptions.jdbc4.MySQLNonTransientConnectionException: Could not create socket factory 'c
om.google.cloud.sql.mysql.SocketFactory' due to underlying exception:
at org.javalite.db_migrator.DbUtils.openConnection(DbUtils.java:183)
at org.javalite.db_migrator.maven.AbstractDbMigrationMojo.openConnection(AbstractDbMigrationMojo.java:178)
at org.javalite.db_migrator.maven.AbstractDbMigrationMojo.openConnection(AbstractDbMigrationMojo.java:170)
at org.javalite.db_migrator.maven.MigrateMojo.executeMojo(MigrateMojo.java:21)
at org.javalite.db_migrator.maven.AbstractDbMigrationMojo.executeCurrentConfiguration(AbstractDbMigrationMojo.java:96)
at org.javalite.db_migrator.maven.AbstractDbMigrationMojo.execute(AbstractDbMigrationMojo.java:80)
at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:134)
... 21 more
Caused by: com.mysql.jdbc.exceptions.jdbc4.MySQLNonTransientConnectionException: Could not create socket factory 'com.google.cloud.sql.mysql.SocketFactory' due
to underlying exception:
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at com.mysql.jdbc.Util.handleNewInstance(Util.java:377)
at com.mysql.jdbc.Util.getInstance(Util.java:360)
at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:956)
at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:935)
at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:924)
at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:870)
at com.mysql.jdbc.MysqlIO.createSocketFactory(MysqlIO.java:3194)
at com.mysql.jdbc.MysqlIO.<init>(MysqlIO.java:293)
at com.mysql.jdbc.ConnectionImpl.coreConnect(ConnectionImpl.java:2232)
at com.mysql.jdbc.ConnectionImpl.connectOneTryOnly(ConnectionImpl.java:2265)
at com.mysql.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:2064)
at com.mysql.jdbc.ConnectionImpl.<init>(ConnectionImpl.java:790)
at com.mysql.jdbc.JDBC4Connection.<init>(JDBC4Connection.java:44)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at com.mysql.jdbc.Util.handleNewInstance(Util.java:377)
at com.mysql.jdbc.ConnectionImpl.getInstance(ConnectionImpl.java:395)
at com.mysql.jdbc.NonRegisteringDriver.connect(NonRegisteringDriver.java:325)
at java.sql.DriverManager.getConnection(DriverManager.java:664)
at java.sql.DriverManager.getConnection(DriverManager.java:247)
at org.javalite.db_migrator.DbUtils.openConnection(DbUtils.java:181)
... 27 more
Caused by: java.lang.ClassNotFoundException: com.google.cloud.sql.mysql.SocketFactory
at org.codehaus.plexus.classworlds.strategy.SelfFirstStrategy.loadClass(SelfFirstStrategy.java:50)
at org.codehaus.plexus.classworlds.realm.ClassRealm.unsynchronizedLoadClass(ClassRealm.java:271)
at org.codehaus.plexus.classworlds.realm.ClassRealm.loadClass(ClassRealm.java:247)
at org.codehaus.plexus.classworlds.realm.ClassRealm.loadClass(ClassRealm.java:239)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:264)
at com.mysql.jdbc.MysqlIO.createSocketFactory(MysqlIO.java:3192)
... 43 more
My connection string in database.properties looks like this:
development.url=jdbc:mysql://google/database?cloudSqlInstance=project:location:instance&socketFactory=com.google.cloud.sql.mysql.SocketFactory&user=dbuser&password=dbuserpassword&useSSL=false
I have used the same connection string with spring-hibernate configuration with out a problem, but would like to try activeweb-activejdbc. Why the compiler cannot find the class? I have included google cloud sql dependency in pom.xml:
<dependency>
<groupId>com.google.cloud.sql</groupId>
<artifactId>mysql-socket-factory-connector-j-6</artifactId>
<version>1.0.2</version>
</dependency>
First, you need to include a fill stack trace. Second, if you get a ClassNotFoundException, this means you do not have that class on the classpath. You need to know for a fact that the class is there. Use Maven dependency plugin, or expand all your dependent jar files and ensure you have the class. No magic here:)
UPDATE:
Now that you provided the stack trace, I see that the exception happens when you run the migrator. If you look at the migrator configuration, you will see that it requires its own class path. For instance, the example here: https://github.com/javalite/activeweb-simple/blob/master/pom.xml#L39 lists the MySQL JDBC driver especially for the migrator, even if you have it configured already for the project runtime. Just copy the config, and replace the MySQL driver with your driver.
I'm deploying google search appliance connector for database(SQL). I'm filling the information in the GSA hostname and other required variable where I have specified the full class name of my JDBC Driver as:
com.microsoft.sqlserver.jdbc.SQLServerDriver
which is right. I have browsed my library of the JDBC to check that.
But after the installation, I am getting this error in the logs:
main .Application.realDaemonStart() INFO: about to init com.google.enterprise.adaptor.database.DatabaseAdaptor
11-04 14:05:27.314 main .Application.realDaemonStart() WARNING: Failed to initialize adaptor
java.lang.ClassNotFoundException: com.microsoft.sqlserver.jdbc.SQLServerDriver
at java.net.URLClassLoader.findClass(Unknown Source)
at java.lang.ClassLoader.loadClass(Unknown Source)
at sun.misc.Launcher$AppClassLoader.loadClass(Unknown Source)
at java.lang.ClassLoader.loadClass(Unknown Source)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Unknown Source)
at com.google.enterprise.adaptor.database.DatabaseAdaptor.init(DatabaseAdaptor.java:133)
at com.google.enterprise.adaptor.Application.realDaemonStart(Application.java:186)
at com.google.enterprise.adaptor.Application.daemonStart(Application.java:155)
at com.google.enterprise.adaptor.Application.start(Application.java:110)
at com.google.enterprise.adaptor.Application.main(Application.java:553)
at com.google.enterprise.adaptor.AbstractAdaptor.main(AbstractAdaptor.java:64)
at com.google.enterprise.adaptor.database.DatabaseAdaptor.main(DatabaseAdaptor.java:315)
What does this mean? I have specified the right class name in the JDBC. Any help is appreciated.
When you run the adaptor - is your JAR file in the same directory as your database adaptor JAR file?
If you are using the Google provided command line, be sure to modify the "classpath" to reference the JAR file for the ODBC driver.
ie.
-classpath="msqljdbc.jar,adaptor-database-4.1.1-withlib.jar"
or
-cp="msqljdbc.jar,adaptor-database-4.1.1-withlib.jar"
This code works fine:
.keyBy(value -> value.getSessionUid())
but this
.keyBy(LogItem::getSessionUid)
throw exception:
Exception in thread "main" java.lang.ArrayIndexOutOfBoundsException: -1
at org.apache.flink.api.java.typeutils.TypeExtractor.getUnaryOperatorReturnType(TypeExtractor.java:294)
at org.apache.flink.api.java.typeutils.TypeExtractor.getKeySelectorTypes(TypeExtractor.java:253)
at org.apache.flink.api.java.typeutils.TypeExtractor.getKeySelectorTypes(TypeExtractor.java:246)
at org.apache.flink.streaming.api.datastream.KeyedStream.<init>(KeyedStream.java:87)
at org.apache.flink.streaming.api.datastream.DataStream.keyBy(DataStream.java:241)
at streamer_sessions_aggregator.StreamerSessionAggregator.runFlow(StreamerSessionAggregator.java:40)
at streamer_sessions_aggregator.Main.main(Main.java:10)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at com.intellij.rt.execution.application.AppMain.main(AppMain.java:144)
As I understand Apache Flink 1.0.0 doesn't support reference methods now. Right?
Flink does currently not support reference methods. Here is the JIRA to add support for it: https://issues.apache.org/jira/browse/FLINK-3138