reference.conf exception when running flink application - apache-flink

I have a dropwizard application using Flink to read from Kafka but the application blows up with this exception when I start it:
java -jar my-app.jar server my-config.yaml
[2018-01-04T01:04:24,577Z](main)([]) INFO - FlinkMiniCluster - Stopping
FlinkMiniCluster.
[2018-01-04T01:04:24,591Z](main)([]) WARN - ROOT - unavailable
! com.typesafe.config.ConfigException$UnresolvedSubstitution:
reference.conf # jar:file:/my-app.jar!/reference.conf: 804: Could not
resolve substitution to a value: ${akka.stream.materializer}
at com.typesafe.config.impl.ConfigReference.resolveSubstitutions(ConfigReference.java:108)
at com.typesafe.config.impl.ResolveContext.realResolve(ResolveContext.java:179)
at com.typesafe.config.impl.ResolveContext.resolve(ResolveContext.java:142)
at com.typesafe.config.impl.SimpleConfigObject$ResolveModifier.modifyChildMayThrow(SimpleConfigObject.java:379)
at com.typesafe.config.impl.SimpleConfigObject.modifyMayThrow(SimpleConfigObject.java:312)
at com.typesafe.config.impl.SimpleConfigObject.resolveSubstitutions(SimpleConfigObject.java:398)
at com.typesafe.config.impl.ResolveContext.realResolve(ResolveContext.java:179)
at com.typesafe.config.impl.ResolveContext.resolve(ResolveContext.java:142)
at com.typesafe.config.impl.SimpleConfigObject$ResolveModifier.modifyChildMayThrow(SimpleConfigObject.java:379)
at com.typesafe.config.impl.SimpleConfigObject.modifyMayThrow(SimpleConfigObject.java:312)
at com.typesafe.config.impl.SimpleConfigObject.resolveSubstitutions(SimpleConfigObject.java:398)
at com.typesafe.config.impl.ResolveContext.realResolve(ResolveContext.java:179)
at com.typesafe.config.impl.ResolveContext.resolve(ResolveContext.java:142)
at com.typesafe.config.impl.SimpleConfigObject$ResolveModifier.modifyChildMayThrow(SimpleConfigObject.java:379)
at com.typesafe.config.impl.SimpleConfigObject.modifyMayThrow(SimpleConfigObject.java:312)
at com.typesafe.config.impl.SimpleConfigObject.resolveSubstitutions(SimpleConfigObject.java:398)
at com.typesafe.config.impl.ResolveContext.realResolve(ResolveContext.java:179)
at com.typesafe.config.impl.ResolveContext.resolve(ResolveContext.java:142)
at com.typesafe.config.impl.SimpleConfigObject$ResolveModifier.modifyChildMayThrow(SimpleConfigObject.java:379)
at com.typesafe.config.impl.SimpleConfigObject.modifyMayThrow(SimpleConfigObject.java:312)
at com.typesafe.config.impl.SimpleConfigObject.resolveSubstitutions(SimpleConfigObject.java:398)
at com.typesafe.config.impl.ResolveContext.realResolve(ResolveContext.java:179)
at com.typesafe.config.impl.ResolveContext.resolve(ResolveContext.java:142)
at com.typesafe.config.impl.SimpleConfigObject$ResolveModifier.modifyChildMayThrow(SimpleConfigObject.java:379)
at com.typesafe.config.impl.SimpleConfigObject.modifyMayThrow(SimpleConfigObject.java:312)
at com.typesafe.config.impl.SimpleConfigObject.resolveSubstitutions(SimpleConfigObject.java:398)
at com.typesafe.config.impl.ResolveContext.realResolve(ResolveContext.java:179)
at com.typesafe.config.impl.ResolveContext.resolve(ResolveContext.java:142)
at com.typesafe.config.impl.ResolveContext.resolve(ResolveContext.java:231)
at com.typesafe.config.impl.SimpleConfig.resolveWith(SimpleConfig.java:74)
at com.typesafe.config.impl.SimpleConfig.resolve(SimpleConfig.java:64)
at com.typesafe.config.impl.SimpleConfig.resolve(SimpleConfig.java:59)
at com.typesafe.config.impl.SimpleConfig.resolve(SimpleConfig.java:37)
at com.typesafe.config.impl.ConfigImpl$1.call(ConfigImpl.java:374)
at com.typesafe.config.impl.ConfigImpl$1.call(ConfigImpl.java:367)
at com.typesafe.config.impl.ConfigImpl$LoaderCache.getOrElseUpdate(ConfigImpl.java:65)
at com.typesafe.config.impl.ConfigImpl.computeCachedConfig(ConfigImpl.java:92)
at com.typesafe.config.impl.ConfigImpl.defaultReference(ConfigImpl.java:367)
at com.typesafe.config.ConfigFactory.defaultReference(ConfigFactory.java:413)
at akka.actor.ActorSystem$Settings.<init>(ActorSystem.scala:307)
at akka.actor.ActorSystemImpl.<init>(ActorSystem.scala:683)
at akka.actor.ActorSystem$.apply(ActorSystem.scala:245)
at akka.actor.ActorSystem$.apply(ActorSystem.scala:288)
at akka.actor.ActorSystem$.apply(ActorSystem.scala:263)
at akka.actor.ActorSystem$.create(ActorSystem.scala:191)
at org.apache.flink.runtime.akka.AkkaUtils$.createActorSystem(AkkaUtils.scala:106)
at org.apache.flink.runtime.minicluster.FlinkMiniCluster.startJobManagerActorSystem(FlinkMiniCluster.scala:300)
at org.apache.flink.runtime.minicluster.FlinkMiniCluster.singleActorSystem$lzycompute$1(FlinkMiniCluster.scala:329)
at org.apache.flink.runtime.minicluster.FlinkMiniCluster.org$apache$flink$runtime$minicluster$FlinkMiniCluster$$singleActorSystem$1(FlinkMiniCluster.scala:329)
at org.apache.flink.runtime.minicluster.FlinkMiniCluster$$anonfun$1.apply(FlinkMiniCluster.scala:343)
at org.apache.flink.runtime.minicluster.FlinkMiniCluster$$anonfun$1.apply(FlinkMiniCluster.scala:341)
at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)
at scala.collection.immutable.Range.foreach(Range.scala:160)
at scala.collection.TraversableLike$class.map(TraversableLike.scala:234)
at scala.collection.AbstractTraversable.map(Traversable.scala:104)
at org.apache.flink.runtime.minicluster.FlinkMiniCluster.start(FlinkMiniCluster.scala:341)
at org.apache.flink.runtime.minicluster.FlinkMiniCluster.start(FlinkMiniCluster.scala:323)
at org.apache.flink.streaming.api.environment.LocalStreamEnvironment.execute(LocalStreamEnvironment.java:107) ...
My Flink stream is pretty basic:
environment
.addSource(new FlinkKafkaConsumer010<>(...)
.name("source name 1")
.union(environment.addSource(new FlinkKafkaConsumer010<>(...)
.name("source name 2"))
.map(new MyMapFunction())
.addSink(new PrintSinkFunction<>())
.name("Sink: Print");
Strangely enough, the application runs just fine and successfully creates a FlinkMiniCluster when debugging in IDEA.
I'm using flink 1.4 and did not start a flink job manager when running from IDEA or command line.
Is there a configuration I need to be setting up to run from the command line?

FYI - I determined that the akka dependencies from Flink were not being recognized at runtime so I manually added them to my application's pom:
<dependency>
<groupId>com.typesafe.akka</groupId>
<artifactId>akka-actor_2.11</artifactId>
<version>${akka.version}</version>
</dependency>
<dependency>
<groupId>com.typesafe.akka</groupId>
<artifactId>akka-protobuf_2.11</artifactId>
<version>${akka.version}</version>
</dependency>
<dependency>
<groupId>com.typesafe.akka</groupId>
<artifactId>akka-stream_2.11</artifactId>
<version>${akka.version}</version>
</dependency>

As CPS said, the refernce.conf file in shaded JAR has to be a merged file of separated akka modules' reference.conf. To get it, the following shade transformer has to be added to the shade configuration :
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-shade-plugin</artifactId>
<version>3.0.0</version>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>shade</goal>
</goals>
<configuration>
<artifactSet>
...
</artifactSet>
<transformers>
<transformer implementation="org.apache.maven.plugins.shade.resource.AppendingTransformer">
<resource>reference.conf</resource>
</transformer>
</transformers>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
I experimented a small constraint. I also use the ManifestResourceTransformer to set the mainClass and I had to order the two transformers with ManifestResourceTransformer first and AppendingTransformer in second position (otherwise ManifestResourceTransformer modifies the entry method of the AppendingTransformer).

This answer to a similar question solves the issue. This has nothing to do with Flink, but has to do with how akka configuration is handled. From their documentation:
Akka’s configuration approach relies heavily on the notion of every module/jar having its own reference.conf file, all of these will be discovered by the configuration and loaded. Unfortunately this also means that if you put/merge multiple jars into the same jar, you need to merge all the reference.confs as well. Otherwise all defaults will be lost and Akka will not function.
Using the shade plugin to make the jar solves the problem.

It might be caused by forgetting about using build-jar build profile in
mvn clean install -Pbuild-jar
as documented in the flink documentation

Related

Apache Karaf feature offline issue

In karaf org.apache.karaf.features.cfg file
I have added
featuresRepositories=mvn:org.apache.cxf.karaf/apache-cxf/3.0.8/xml/features
featuresBoot = cxf-jaxws
The cxf feature could fetch and be installed when karaf started with the connection.
But it will fail without connection, how can I pre-install cxf feature?
This is likely far from most optimal solution for this (would love to hear about the better ones) but you could create offline-repository project using karaf-feature-archetype and configure karaf-maven-plugin use something like following configuration:
<plugin>
<groupId>org.apache.karaf.tooling</groupId>
<artifactId>karaf-maven-plugin</artifactId>
<configuration>
<startLevel>50</startLevel>
<aggregateFeatures>true</aggregateFeatures>
<checkDependencyChange>true</checkDependencyChange>
<failOnDependencyChange>false</failOnDependencyChange>
<logDependencyChanges>true</logDependencyChanges>
<overwriteChangedDependencies>true</overwriteChangedDependencies>
</configuration>
<executions>
<execution>
<id>features-add-to-repo</id>
<phase>generate-resources</phase>
<goals>
<goal>features-add-to-repository</goal>
</goals>
<configuration>
<descriptors>
<!-- Feature repository paths -->
<descriptor>mvn:groupId/artifactId/version/xml/features</descriptor>
</descriptors>
<features>
<!-- features and their artifacts + dependencies to add to offline repository-->
<feature>featureName</feature>
<feature>featureName/version</feature>
</features>
<repository>target/offline-repository</repository>
</configuration>
</execution>
</executions>
</plugin>
When packaging the project i.e with command maven clean install (in environment with online access) it'll generate offline-repository under target folder which you can copy to your offline environment and tell karaf to use it by adding it to org.ops4j.pax.url.mvn.defaultRepositories in file org.ops4j.pax.url.mvn.cfg i.e file:${user.home}/offline-repository#snapshots#id=local if its located in home directory.
features.xml itself can be empty this is just to use karaf-maven-plugin not to create an actual feature repository.
Just be careful if you need to create a new version of the offline-repository to replace the old one. If the new version is missing any of the artifacts that are currently installed to karaf it can cause issues when trying to remove/uninstall them.

Maven cxf plugin logging

I'm using the Apache cxf maven plugin (v3.3.0) to successfully to generate java wrappers.
However, the output from the maven build contains thousands of DEBUG logging lines from the wsdl2java which I am unable to remove. Is there an extraarg or other way to silence the process so I get just a success (or possibly failure) message?
<plugin>
<groupId>org.apache.cxf</groupId>
<artifactId>cxf-codegen-plugin</artifactId>
<version>${cxf.version}</version>
<executions>
<execution>
<id>generate-sources</id>
<phase>generate-sources</phase>
<configuration>
<sourceRoot>${project.build.directory}/generated-sources/cxf</sourceRoot>
<defaultOptions>
<autoNameResolution>true</autoNameResolution>
</defaultOptions>
<wsdlOptions>
<!--Some Web Service -->
<wsdlOption>
<wsdl>https://some/web/service.wsdl</wsdl>
<extraargs>
<extraarg>-client</extraarg>
<extraarg>-quiet</extraarg>
<extraarg>-p</extraarg>
<extraarg>com.foo.bar</extraarg>
</extraargs>
</wsdlOption>
</wsdlOptions>
</configuration>
<goals>
<goal>wsdl2java</goal>
</goals>
</execution>
</executions>
</plugin>
It appears that under Java 9+ the plugin forces code generation in a forked JVM regardless of the default being documented as false and regardless of any explicit configuration of this option. The plugin execution doesn't see any logging configuration from the project. CXF is logging using java.util.logging and any log down to FINER severity gets printed to the console.
I solved this by providing an explicit path to a logging configuration file to the forked JVM using the plugin's additionalJvmArgs configuration option:
<plugin>
<groupId>org.apache.cxf</groupId>
<artifactId>cxf-codegen-plugin</artifactId>
<version>${cxf-plugin.version}</version>
<configuration>
<additionalJvmArgs>-Dlogback.configurationFile=${project.basedir}/src/test/resources/logback-codegen.xml</additionalJvmArgs>
</configuration>
</plugin>
The system property for Logback (as in my case) is logback.configurationFile. For Log4j that would be log4j.configurationFile.
In the logging configuration file the following loggers can be added (Logback):
<!-- entries below silence excessive logging from cxf-codegen-plugin -->
<logger name="org.apache.cxf" level="info"/>
<logger name="org.apache.velocity" level="info"/>
This way the plugin execution will still print to the console all warnings and errors, but all the repetitive debug information goes away. The drawback is that you need to have such a logging configuration file visible in each of your projects. But then, you probably should have one anyway. The same one as for (unit) tests can often be used.
Sorry for not getting into the root solutions, but adding this in my dependencies help:
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter</artifactId>
<exclusions>
<exclusion>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-logging</artifactId>
</exclusion>
</exclusions>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-log4j2</artifactId>
<scope>provided</scope>
</dependency>
or put following in plugin execution helps too except for velocity logs
<additionalJvmArgs>
-Dorg.apache.cxf.Logger=null
</additionalJvmArgs>
Hope this would give a hints for someone to come out with a much proper solution.
Using maven-3.8.4 and cxf-codegen-plugin:3.5.1:wsdl2java, I've reduced log level with:
<configuration>
<additionalJvmArgs>-D.level=WARN</additionalJvmArgs>
</configuration>
More details here: Apache CXF

Eclipse/Maven: QueryDSL and SQL Server Issue Using Integrated Security

Relevant details for the IDE:
Eclipse 2018-09
m2e 1.9.1.20180912-1601
A few high level details:
I'm using Maven and QueryDSL to generate Data Transfer Objects and Query classes for two databases in SQL Server
The JDBC connection in Maven needs to use integrated security (i.e. no stored passwords/SQL Server database authentication). This requires use of the sqljdbc_auth.dll native library.
This setup largely works, but only for the first Maven -> 'Update Project' after starting Eclipse. Even on the first run of the project update, it appears that something in Maven runs more than once and causes an exception. If I make any schema changes, then I have to Restart Eclipse to successfully update the Maven project. On the first run and subsequent runs I have at least two persistent Maven errors, though they change from first to subsequent runs.
Changing IDEs (ex. Eclipse -> IntelliJ) has a high cost right now and not a good option.
I'm open to using something other than QueryDSL if there is an ORM that works better (note I primarily work in .Net, but have occasional Java projects such as this one).
In summary I have two questions - identified as Question 1 and Question 2 below in bold.
Below is a redacted pom.xml, but includes enough information to reproduce (or identify that it can't be reproduced):
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>REDACTED</groupId>
<artifactId>REDACTED</artifactId>
<version>REDACTED</version>
<name>REDACTED</name>
<description>REDACTED</description>
<properties>
<querydsl.version>4.2.1</querydsl.version>
<mssql.version>7.0.0.jre8</mssql.version>
</properties>
<build>
<sourceDirectory>src</sourceDirectory>
<plugins>
<plugin>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.8.0</version>
<configuration>
<source>1.10</source>
<target>1.10</target>
</configuration>
</plugin>
<plugin>
<groupId>com.querydsl</groupId>
<artifactId>querydsl-maven-plugin</artifactId>
<version>${querydsl.version}</version>
<executions>
<execution>
<id>database1</id>
<phase>validate</phase>
<configuration>
<jdbcDriver>com.microsoft.sqlserver.jdbc.SQLServerDriver</jdbcDriver>
<jdbcUrl>jdbc:sqlserver://server:1433;databaseName=database1;integratedSecurity=true;</jdbcUrl>
<packageName>REDACTED.database1</packageName>
<exportBeans>true</exportBeans>
<targetFolder>${project.basedir}/target/generated-sources/java</targetFolder>
</configuration>
<goals>
<goal>export</goal>
</goals>
</execution>
<execution>
<id>database2</id>
<phase>validate</phase>
<configuration>
<jdbcDriver>com.microsoft.sqlserver.jdbc.SQLServerDriver</jdbcDriver>
<jdbcUrl>jdbc:sqlserver://server:1433;databaseName=database2;integratedSecurity=true;</jdbcUrl>
<packageName>REDACTED.database2</packageName>
<exportBeans>true</exportBeans>
<targetFolder>${project.basedir}/target/generated-sources/java</targetFolder>
</configuration>
<goals>
<goal>export</goal>
</goals>
</execution>
<!-- <execution> <goals> <goal>export</goal> </goals> </execution> -->
</executions>
<dependencies>
<dependency>
<groupId>com.microsoft.sqlserver</groupId>
<artifactId>mssql-jdbc</artifactId>
<version>${mssql.version}</version>
</dependency>
</dependencies>
</plugin>
</plugins>
</build>
<dependencyManagement>
<dependencies>
<dependency>
<groupId>org.optaplanner</groupId>
<artifactId>optaplanner-bom</artifactId>
<type>pom</type>
<version>7.0.0.Final</version>
<scope>import</scope>
</dependency>
<dependency>
<groupId>org.apache.commons</groupId>
<artifactId>commons-lang3</artifactId>
<version>3.6</version>
<scope>import</scope>
</dependency>
<dependency>
<groupId>org.apache.commons</groupId>
<artifactId>commons-math3</artifactId>
<version>3.6</version>
</dependency>
<dependency>
<groupId>com.microsoft.sqlserver</groupId>
<artifactId>mssql-jdbc</artifactId>
<version>${mssql.version}</version>
</dependency>
<dependency>
<groupId>com.querydsl</groupId>
<artifactId>querydsl-jpa</artifactId>
<version>${querydsl.version}</version>
</dependency>
<dependency>
<groupId>com.querydsl</groupId>
<artifactId>querydsl-sql</artifactId>
<version>${querydsl.version}</version>
</dependency>
<dependency>
<groupId>javax</groupId>
<artifactId>javaee-api</artifactId>
<version>6.0</version>
</dependency>
</dependencies>
</dependencyManagement>
<dependencies>
<dependency>
<groupId>org.optaplanner</groupId>
<artifactId>optaplanner-core</artifactId>
</dependency>
<dependency>
<groupId>org.optaplanner</groupId>
<artifactId>optaplanner-persistence-jpa</artifactId>
</dependency>
<dependency>
<groupId>com.microsoft.sqlserver</groupId>
<artifactId>mssql-jdbc</artifactId>
</dependency>
<dependency>
<groupId>javax</groupId>
<artifactId>javaee-api</artifactId>
</dependency>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.12</version>
</dependency>
<dependency>
<groupId>com.querydsl</groupId>
<artifactId>querydsl-jpa</artifactId>
</dependency>
<dependency>
<groupId>com.querydsl</groupId>
<artifactId>querydsl-sql</artifactId>
</dependency>
</dependencies>
</project>
Eclipse produces two types of errors, an error involving multiple writes to a file and an error involving the native authentication library:
Execution database1 of goal com.querydsl:querydsl-maven-plugin:4.2.1:export failed: Attempted to write multiple times to <PATH>\CheckConstraints.java, please check your configuration (com.querydsl:querydsl-maven-plugin:4.2.1:export:database1:validate)
...
Caused by: java.lang.IllegalStateException: Attempted to write multiple times to <PATH>\CheckConstraints.java, please check your configuration
Question 1: Is there something off with the pom.xml file or an eclipse setting somewhere that would cause Maven to run the querydsl plugin executions more than once per validate phase of the maven build? Should I use a different phase than validate? Something else entirely?
Updating the maven project in eclipse more than once causes the following errors:
This driver is not configured for integrated authentication. ClientConnectionId:12c300a7-0674-41a1-8e64-16040e9dbde3 (com.querydsl:querydsl-maven-plugin:4.2.1:export:database2:validate)
org.apache.maven.plugin.MojoExecutionException: This driver is not configured for integrated authentication. ClientConnectionId:12c300a7-0674-41a1-8e64-16040e9dbde3
...
Caused by: com.microsoft.sqlserver.jdbc.SQLServerException: This driver is not configured for integrated authentication. ClientConnectionId:12c300a7-0674-41a1-8e64-16040e9dbde3
...
Caused by: java.lang.UnsatisfiedLinkError: Native Library C:\Windows\System32\sqljdbc_auth.dll already loaded in another classloader
This error is repeated for database2 as well. The ... are long sections of stack trace that might or might not be helpful. I've posted the full stack trace for both error types at the bottom of the question.
Research into this error indicates that there is likely a classloader/lifetime related issue to Maven and Eclipse. I'm not sure if Maven and Eclipse share a classloader, but the conclusion of the Maven -> 'Update Project' method is not causing the native library, sqljdbc_auth.dll, to be unloaded and subsequent updates are causing the Microsoft JDBC driver to attempt to reload the native library.
Question 2: How do I make Maven run in Eclipse in such a way as to have the native library, sqljdbc_auth.dll, unloaded (or the previously loaded native library reused)?
Below are the full stack traces for the two types of errors:
Error Type 1:
Execution database1 of goal com.querydsl:querydsl-maven-plugin:4.2.1:export failed: Attempted to write multiple times to <PATH>\CheckConstraints.java, please check your configuration (com.querydsl:querydsl-maven-plugin:4.2.1:export:database1:validate)
org.apache.maven.plugin.PluginExecutionException: Execution database1 of goal com.querydsl:querydsl-maven-plugin:4.2.1:export failed: Attempted to write multiple times to <Path>\CheckConstraints.java, please check your configuration
at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:148)
at org.eclipse.m2e.core.internal.embedder.MavenImpl.execute(MavenImpl.java:331)
at org.eclipse.m2e.core.internal.embedder.MavenImpl$11.call(MavenImpl.java:1362)
at org.eclipse.m2e.core.internal.embedder.MavenImpl$11.call(MavenImpl.java:1)
at org.eclipse.m2e.core.internal.embedder.MavenExecutionContext.executeBare(MavenExecutionContext.java:177)
at org.eclipse.m2e.core.internal.embedder.MavenExecutionContext.execute(MavenExecutionContext.java:112)
at org.eclipse.m2e.core.internal.embedder.MavenImpl.execute(MavenImpl.java:1360)
at org.eclipse.m2e.core.project.configurator.MojoExecutionBuildParticipant.build(MojoExecutionBuildParticipant.java:52)
at org.eclipse.m2e.core.internal.builder.MavenBuilderImpl.build(MavenBuilderImpl.java:137)
at org.eclipse.m2e.core.internal.builder.MavenBuilder$1.method(MavenBuilder.java:173)
at org.eclipse.m2e.core.internal.builder.MavenBuilder$1.method(MavenBuilder.java:1)
at org.eclipse.m2e.core.internal.builder.MavenBuilder$BuildMethod$1$1.call(MavenBuilder.java:116)
at org.eclipse.m2e.core.internal.embedder.MavenExecutionContext.executeBare(MavenExecutionContext.java:177)
at org.eclipse.m2e.core.internal.embedder.MavenExecutionContext.execute(MavenExecutionContext.java:112)
at org.eclipse.m2e.core.internal.builder.MavenBuilder$BuildMethod$1.call(MavenBuilder.java:106)
at org.eclipse.m2e.core.internal.embedder.MavenExecutionContext.executeBare(MavenExecutionContext.java:177)
at org.eclipse.m2e.core.internal.embedder.MavenExecutionContext.execute(MavenExecutionContext.java:151)
at org.eclipse.m2e.core.internal.embedder.MavenExecutionContext.execute(MavenExecutionContext.java:99)
at org.eclipse.m2e.core.internal.builder.MavenBuilder$BuildMethod.execute(MavenBuilder.java:87)
at org.eclipse.m2e.core.internal.builder.MavenBuilder.build(MavenBuilder.java:201)
at org.eclipse.core.internal.events.BuildManager$2.run(BuildManager.java:798)
at org.eclipse.core.runtime.SafeRunner.run(SafeRunner.java:45)
at org.eclipse.core.internal.events.BuildManager.basicBuild(BuildManager.java:219)
at org.eclipse.core.internal.events.BuildManager.basicBuild(BuildManager.java:262)
at org.eclipse.core.internal.events.BuildManager$1.run(BuildManager.java:315)
at org.eclipse.core.runtime.SafeRunner.run(SafeRunner.java:45)
at org.eclipse.core.internal.events.BuildManager.basicBuild(BuildManager.java:318)
at org.eclipse.core.internal.events.BuildManager.basicBuildLoop(BuildManager.java:370)
at org.eclipse.core.internal.events.BuildManager.build(BuildManager.java:391)
at org.eclipse.core.internal.events.AutoBuildJob.doBuild(AutoBuildJob.java:145)
at org.eclipse.core.internal.events.AutoBuildJob.run(AutoBuildJob.java:235)
at org.eclipse.core.internal.jobs.Worker.run(Worker.java:63)
Caused by: java.lang.IllegalStateException: Attempted to write multiple times to <PATH>\CheckConstraints.java, please check your configuration
at com.querydsl.sql.codegen.MetaDataExporter.write(MetaDataExporter.java:431)
at com.querydsl.sql.codegen.MetaDataExporter.serialize(MetaDataExporter.java:413)
at com.querydsl.sql.codegen.MetaDataExporter.handleTable(MetaDataExporter.java:393)
at com.querydsl.sql.codegen.MetaDataExporter.handleTables(MetaDataExporter.java:260)
at com.querydsl.sql.codegen.MetaDataExporter.export(MetaDataExporter.java:251)
at com.querydsl.maven.AbstractMetaDataExportMojo.execute(AbstractMetaDataExportMojo.java:538)
at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:137)
... 31 more
Error Type 2:
This driver is not configured for integrated authentication. ClientConnectionId:12c300a7-0674-41a1-8e64-16040e9dbde3 (com.querydsl:querydsl-maven-plugin:4.2.1:export:database2:validate)
org.apache.maven.plugin.MojoExecutionException: This driver is not configured for integrated authentication. ClientConnectionId:12c300a7-0674-41a1-8e64-16040e9dbde3
at com.querydsl.maven.AbstractMetaDataExportMojo.execute(AbstractMetaDataExportMojo.java:547)
at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:137)
at org.eclipse.m2e.core.internal.embedder.MavenImpl.execute(MavenImpl.java:331)
at org.eclipse.m2e.core.internal.embedder.MavenImpl$11.call(MavenImpl.java:1362)
at org.eclipse.m2e.core.internal.embedder.MavenImpl$11.call(MavenImpl.java:1)
at org.eclipse.m2e.core.internal.embedder.MavenExecutionContext.executeBare(MavenExecutionContext.java:177)
at org.eclipse.m2e.core.internal.embedder.MavenExecutionContext.execute(MavenExecutionContext.java:112)
at org.eclipse.m2e.core.internal.embedder.MavenImpl.execute(MavenImpl.java:1360)
at org.eclipse.m2e.core.project.configurator.MojoExecutionBuildParticipant.build(MojoExecutionBuildParticipant.java:52)
at org.eclipse.m2e.core.internal.builder.MavenBuilderImpl.build(MavenBuilderImpl.java:137)
at org.eclipse.m2e.core.internal.builder.MavenBuilder$1.method(MavenBuilder.java:173)
at org.eclipse.m2e.core.internal.builder.MavenBuilder$1.method(MavenBuilder.java:1)
at org.eclipse.m2e.core.internal.builder.MavenBuilder$BuildMethod$1$1.call(MavenBuilder.java:116)
at org.eclipse.m2e.core.internal.embedder.MavenExecutionContext.executeBare(MavenExecutionContext.java:177)
at org.eclipse.m2e.core.internal.embedder.MavenExecutionContext.execute(MavenExecutionContext.java:112)
at org.eclipse.m2e.core.internal.builder.MavenBuilder$BuildMethod$1.call(MavenBuilder.java:106)
at org.eclipse.m2e.core.internal.embedder.MavenExecutionContext.executeBare(MavenExecutionContext.java:177)
at org.eclipse.m2e.core.internal.embedder.MavenExecutionContext.execute(MavenExecutionContext.java:151)
at org.eclipse.m2e.core.internal.embedder.MavenExecutionContext.execute(MavenExecutionContext.java:99)
at org.eclipse.m2e.core.internal.builder.MavenBuilder$BuildMethod.execute(MavenBuilder.java:87)
at org.eclipse.m2e.core.internal.builder.MavenBuilder.build(MavenBuilder.java:201)
at org.eclipse.core.internal.events.BuildManager$2.run(BuildManager.java:798)
at org.eclipse.core.runtime.SafeRunner.run(SafeRunner.java:45)
at org.eclipse.core.internal.events.BuildManager.basicBuild(BuildManager.java:219)
at org.eclipse.core.internal.events.BuildManager.basicBuild(BuildManager.java:262)
at org.eclipse.core.internal.events.BuildManager$1.run(BuildManager.java:315)
at org.eclipse.core.runtime.SafeRunner.run(SafeRunner.java:45)
at org.eclipse.core.internal.events.BuildManager.basicBuild(BuildManager.java:318)
at org.eclipse.core.internal.events.BuildManager.basicBuildLoop(BuildManager.java:370)
at org.eclipse.core.internal.events.BuildManager.build(BuildManager.java:391)
at org.eclipse.core.internal.events.AutoBuildJob.doBuild(AutoBuildJob.java:145)
at org.eclipse.core.internal.events.AutoBuildJob.run(AutoBuildJob.java:235)
at org.eclipse.core.internal.jobs.Worker.run(Worker.java:63)
Caused by: com.microsoft.sqlserver.jdbc.SQLServerException: This driver is not configured for integrated authentication. ClientConnectionId:12c300a7-0674-41a1-8e64-16040e9dbde3
at com.microsoft.sqlserver.jdbc.SQLServerConnection.terminate(SQLServerConnection.java:2826)
at com.microsoft.sqlserver.jdbc.AuthenticationJNI.<init>(AuthenticationJNI.java:73)
at com.microsoft.sqlserver.jdbc.SQLServerConnection.logon(SQLServerConnection.java:3550)
at com.microsoft.sqlserver.jdbc.SQLServerConnection.access$000(SQLServerConnection.java:81)
at com.microsoft.sqlserver.jdbc.SQLServerConnection$LogonCommand.doExecute(SQLServerConnection.java:3541)
at com.microsoft.sqlserver.jdbc.TDSCommand.execute(IOBuffer.java:7240)
at com.microsoft.sqlserver.jdbc.SQLServerConnection.executeCommand(SQLServerConnection.java:2869)
at com.microsoft.sqlserver.jdbc.SQLServerConnection.connectHelper(SQLServerConnection.java:2395)
at com.microsoft.sqlserver.jdbc.SQLServerConnection.login(SQLServerConnection.java:2042)
at com.microsoft.sqlserver.jdbc.SQLServerConnection.connectInternal(SQLServerConnection.java:1889)
at com.microsoft.sqlserver.jdbc.SQLServerConnection.connect(SQLServerConnection.java:1120)
at com.microsoft.sqlserver.jdbc.SQLServerDriver.connect(SQLServerDriver.java:700)
at java.sql.DriverManager.getConnection(Unknown Source)
at java.sql.DriverManager.getConnection(Unknown Source)
at com.querydsl.maven.AbstractMetaDataExportMojo.execute(AbstractMetaDataExportMojo.java:536)
... 32 more
Caused by: java.lang.UnsatisfiedLinkError: Native Library C:\Windows\System32\sqljdbc_auth.dll already loaded in another classloader
at java.lang.ClassLoader.loadLibrary0(Unknown Source)
at java.lang.ClassLoader.loadLibrary(Unknown Source)
at java.lang.Runtime.loadLibrary0(Unknown Source)
at java.lang.System.loadLibrary(Unknown Source)
at com.microsoft.sqlserver.jdbc.AuthenticationJNI.<clinit>(AuthenticationJNI.java:52)
at com.microsoft.sqlserver.jdbc.SQLServerConnection.logon(SQLServerConnection.java:3549)
at com.microsoft.sqlserver.jdbc.SQLServerConnection.access$000(SQLServerConnection.java:81)
at com.microsoft.sqlserver.jdbc.SQLServerConnection$LogonCommand.doExecute(SQLServerConnection.java:3541)
at com.microsoft.sqlserver.jdbc.TDSCommand.execute(IOBuffer.java:7240)
at com.microsoft.sqlserver.jdbc.SQLServerConnection.executeCommand(SQLServerConnection.java:2869)
at com.microsoft.sqlserver.jdbc.SQLServerConnection.connectHelper(SQLServerConnection.java:2395)
at com.microsoft.sqlserver.jdbc.SQLServerConnection.login(SQLServerConnection.java:2042)
at com.microsoft.sqlserver.jdbc.SQLServerConnection.connectInternal(SQLServerConnection.java:1889)
at com.microsoft.sqlserver.jdbc.SQLServerConnection.connect(SQLServerConnection.java:1120)
at com.microsoft.sqlserver.jdbc.SQLServerDriver.connect(SQLServerDriver.java:700)
at java.sql.DriverManager.getConnection(Unknown Source)
at java.sql.DriverManager.getConnection(Unknown Source)
at com.querydsl.maven.AbstractMetaDataExportMojo.execute(AbstractMetaDataExportMojo.java:536)
at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:137)
at org.eclipse.m2e.core.internal.embedder.MavenImpl.execute(MavenImpl.java:331)
at org.eclipse.m2e.core.internal.embedder.MavenImpl$11.call(MavenImpl.java:1362)
at org.eclipse.m2e.core.internal.embedder.MavenImpl$11.call(MavenImpl.java:1)
at org.eclipse.m2e.core.internal.embedder.MavenExecutionContext.executeBare(MavenExecutionContext.java:177)
at org.eclipse.m2e.core.internal.embedder.MavenExecutionContext.execute(MavenExecutionContext.java:112)
at org.eclipse.m2e.core.internal.embedder.MavenImpl.execute(MavenImpl.java:1360)
at org.eclipse.m2e.core.project.configurator.MojoExecutionBuildParticipant.build(MojoExecutionBuildParticipant.java:52)
at org.eclipse.m2e.core.internal.builder.MavenBuilderImpl.build(MavenBuilderImpl.java:137)
at org.eclipse.m2e.core.project.configurator.AbstractLifecycleMapping.configure(AbstractLifecycleMapping.java:111)
at org.eclipse.m2e.core.internal.project.ProjectConfigurationManager$3.call(ProjectConfigurationManager.java:508)
at org.eclipse.m2e.core.internal.project.ProjectConfigurationManager$3.call(ProjectConfigurationManager.java:1)
at org.eclipse.m2e.core.internal.embedder.MavenExecutionContext.executeBare(MavenExecutionContext.java:177)
at org.eclipse.m2e.core.internal.embedder.MavenExecutionContext.execute(MavenExecutionContext.java:151)
at org.eclipse.m2e.core.internal.project.ProjectConfigurationManager.updateProjectConfiguration(ProjectConfigurationManager.java:501)
at org.eclipse.m2e.core.internal.project.ProjectConfigurationManager.updateProjectConfiguration0(ProjectConfigurationManager.java:439)
at org.eclipse.m2e.core.internal.project.ProjectConfigurationManager$2.call(ProjectConfigurationManager.java:352)
at org.eclipse.m2e.core.internal.project.ProjectConfigurationManager$2.call(ProjectConfigurationManager.java:1)
at org.eclipse.m2e.core.internal.embedder.MavenExecutionContext.executeBare(MavenExecutionContext.java:177)
at org.eclipse.m2e.core.internal.embedder.MavenExecutionContext.execute(MavenExecutionContext.java:151)
at org.eclipse.m2e.core.internal.embedder.MavenExecutionContext.execute(MavenExecutionContext.java:99)
at org.eclipse.m2e.core.internal.embedder.MavenImpl.execute(MavenImpl.java:1351)
at org.eclipse.m2e.core.internal.project.ProjectConfigurationManager.updateProjectConfiguration(ProjectConfigurationManager.java:349)
at org.eclipse.m2e.core.ui.internal.UpdateMavenProjectJob.runInWorkspace(UpdateMavenProjectJob.java:77)
at org.eclipse.core.internal.resources.InternalWorkspaceJob.run(InternalWorkspaceJob.java:42)
... 1 more
Thanks in advance for any help.

maven-pmd-plugin uses only the bundled rulesets

I am using the maven-pmd-plugin on my project and this is how I have configured it
<reporting>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-jxr-plugin</artifactId>
<version>2.3</version>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-pmd-plugin</artifactId>
<version>2.6</version>
<configuration>
<linkXref>true</linkXref>
<sourceEncoding>UTF-8</sourceEncoding>
<minimumTokens>100</minimumTokens>
<targetJdk>${targetJdk}</targetJdk>
<rulesets>
<ruleset>${maven.pmd.rulesetfiles}</ruleset>
</rulesets>
</configuration>
</plugin>
</plugins>
</reporting>
Here are the properties used in the above configuration
<properties>
<spring.version>3.0.6.RELEASE</spring.version>
<basedir>C:\Users\Q4\workspace\project</basedir>
<maven.pmd.rulesetfiles>${basedir}\pmdRuleset.xml</maven.pmd.rulesetfiles>
<targetJdk>1.5</targetJdk>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
</properties>
The problem is when I run mvn pmd:check, it gives me 8 violations -- only from the basic, unusedcode and imports. It simply doesn't use all the rules that I have listed in the custom ruleset file. I have even tried using the logging-java.xml and strings.xml directly in the ruleset without using the custom ruleset file and it still doesn't work.
When i run mvn pmd:pmd, i get a BUILD SUCCESS but the errors still show up in my target folder. Why do I get a build success here?
I solved this by simply adding the plugins in the build section along with the ones in the reporting section.
Somehow it needed to be in the as well to be able to run all the rulesets. Earlier I was under the impression that we put plugins in the build only if we want to run them during the build and deploy phase.

Maven building for GoogleAppEngine, forced to include JDO libraries?

I'm trying to build my application for GoogleAppEngine using maven. I've added the following to my pom which should "enhance" my classes after building, as suggested on the DataNucleus documentation
<plugin>
<groupId>org.datanucleus</groupId>
<artifactId>maven-datanucleus-plugin</artifactId>
<version>1.1.4</version>
<configuration>
<log4jConfiguration>${basedir}/log4j.properties</log4jConfiguration>
<verbose>true</verbose>
</configuration>
<executions>
<execution>
<phase>process-classes</phase>
<goals>
<goal>enhance</goal>
</goals>
</execution>
</executions>
</plugin>
According to the documentation on GoogleAppEngine, you have the choice to use JDO or JPA, I've chosen to use JPA since I have used it in the past. When I try to build my project (before I upload to GAE) using mvn clean package I get the following output
[ERROR] BUILD ERROR
[INFO] ------------------------------------------------------------------------
[INFO] Failed to resolve artifact.
Missing:
----------
1) javax.jdo:jdo2-api:jar:2.3-ec
Try downloading the file manually from the project website.
Then, install it using the command:
mvn install:install-file -DgroupId=javax.jdo -DartifactId=jdo2-api -Dversion=2.3-ec -Dpackaging=jar -Dfile=/path/to/file
Alternatively, if you host your own repository you can deploy the file there:
mvn deploy:deploy-file -DgroupId=javax.jdo -DartifactId=jdo2-api -Dversion=2.3-ec -Dpackaging=jar -Dfile=/path/to/file -Durl=[url] -DrepositoryId=[id]
Path to dependency:
1) org.datanucleus:maven-datanucleus-plugin:maven-plugin:1.1.4
2) javax.jdo:jdo2-api:jar:2.3-ec
----------
1 required artifact is missing.
for artifact:
org.datanucleus:maven-datanucleus-plugin:maven-plugin:1.1.4
from the specified remote repositories:
__jpp_repo__ (file:///usr/share/maven2/repository),
DN_M2_Repo (http://www.datanucleus.org/downloads/maven2/),
central (http://repo1.maven.org/maven2)
[INFO] ------------------------------------------------------------------------
[INFO] For more information, run Maven with the -e switch
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 3 seconds
[INFO] Finished at: Sat Apr 03 16:02:39 BST 2010
[INFO] Final Memory: 31M/258M
[INFO] ------------------------------------------------------------------------
Any ideas why I should get such an error? I've searched through my entire source code and I'm not referencing JDO anywhere, so unless the app engine libraries require it, I'm not sure why I get this message.
The DataNucleus Maven plugin requires the JDO2 API JAR (even for JPA) as documented here and as reported in the trace:
Path to dependency:
1) org.datanucleus:maven-datanucleus-plugin:maven-plugin:1.1.4
2) javax.jdo:jdo2-api:jar:2.3-ec
The odd part is that jdo2-api-2.3-ec.jar is in the DataNucleus Maven repository (that is declared in the POM of the plugin) and Maven has checked this repository as we can see in the trace.
Update: Ok, this is definitely weird and I don't know why the build is failing exactly (maybe a problem with dependencies ranges). As a workaround, declare the JDO2 API JAR as dependency in the plugin:
<project>
...
<build>
<plugins>
<plugin>
<groupId>org.datanucleus</groupId>
<artifactId>maven-datanucleus-plugin</artifactId>
<version>1.1.4</version>
<configuration>
<log4jConfiguration>${basedir}/log4j.properties</log4jConfiguration>
<verbose>true</verbose>
</configuration>
<executions>
<execution>
<phase>process-classes</phase>
<goals>
<goal>enhance</goal>
</goals>
</execution>
</executions>
<dependencies>
<dependency>
<groupId>javax.jdo</groupId>
<artifactId>jdo2-api</artifactId>
<version>2.3-ec</version>
<scope>runtime</scope>
</dependency>
</dependencies>
</plugin>
...
</plugins>
...
</build>
</project>
With this dependency declared, the JAR gets downloaded.

Resources