I am using Apache Camel 3.14.0 with SpringBoot and trying to generate code coverage report with JaCoCo xml so that SonarQube can use it to show code coverage. I followed all the steps mentioned in https://camel.apache.org/manual/camel-report-maven-plugin.html#_camelroute_coverage but still the jacoco xml file is not getting generated rather it shows the coverage in console.
I have below dependency in my pom.xml:
<dependency>
<groupId>org.apache.camel</groupId>
<artifactId>camel-management</artifactId>
<version>${camel.springboot.version}</version>
<scope>test</scope>
</dependency>
My test file has below annotations:
#CamelSpringBootTest
#EnableAutoConfiguration
#SpringBootTest(classes = MyApplication.class)
#MockEndpoints(value = "direct:first-route")
#EnableRouteCoverage
I executed mvn test command and all the tests are passed and generated individual test xml file inside target/camel-route-coverage directory. After this, I am executing mvn camel-report:route-coverage -DgenerateJacocoXmlReport=true command which shows code/route coverage on console but doesn't generate target/site/jacoco/xmlJacoco.xml file.
Due to this, I am unable to view code coverage in SonarQube.
Related
I need to save the table result to orc on S3, and this is how I do it:
tEnv.createTemporaryTable("my_output_table", TableDescriptor.forConnector("filesystem")
.schema(outputSchema)
.option("path", s3OutputPath)
.format(FormatDescriptor.forFormat("orc").build())
.build());
finalResultToInsert.executeInsert("my_output_table");
However, during runtime it throws error of
Caused by: org.apache.flink.table.api.ValidationException: Could not find any format factory for identifier 'orc' in the classpath.
at org.apache.flink.table.filesystem.FileSystemTableSink.<init>(FileSystemTableSink.java:128) ~[flink-table_2.12-1.14.2.jar:1.14.2]
at org.apache.flink.table.filesystem.FileSystemTableFactory.createDynamicTableSink(FileSystemTableFactory.java:87) ~[flink-table_2.12-1.14.2.jar:1.14.2]
at org.apache.flink.table.factories.FactoryUtil.createTableSink(FactoryUtil.java:179) ~[flink-table_2.12-1.14.2.jar:1.14.2]
at org.apache.flink.table.planner.delegation.PlannerBase.getTableSink(PlannerBase.scala:394) ~[flink-table_2.12-1.14.2.jar:1.14.2]
......
I have already included the relevant dependency
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-orc_2.11</artifactId>
<version>1.14.2</version>
</dependency>
And in the generated jar file I can see the flink orc classes:
org/apache/flink/orc/
org/apache/flink/orc/AbstractOrcFileInputFormat.class
org/apache/flink/orc/OrcFileFormatFactory$1.class
org/apache/flink/orc/OrcFilters$LessThanEquals.class
org/apache/flink/orc/AbstractOrcFileInputFormat$OrcVectorizedReader.class
org/apache/flink/orc/OrcFilters$Not.class
org/apache/flink/orc/OrcFileFormatFactory.class
org/apache/flink/orc/OrcColumnarRowSplitReader.class
org/apache/flink/orc/OrcColumnarRowSplitReader$ColumnBatchGenerator.class
org/apache/flink/orc/AbstractOrcFileInputFormat$OrcReaderBatch.class
org/apache/flink/orc/OrcFilters$In.class
......
So I really don't understand why it still can not find it in the classpath.
A side note, in the project I also include flink-avro dependency and if I change the output format from orc to avro it just works fine.
Also, I am running the job on AWS EMR. The EMR release is 6.6.0 which has Flink version of 1.14.2. https://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-660-release.html
Could anyone help with that? Thanks a lot!
If I'm not mistaken, ORC requires Hadoop. So you should also add org.apache.hadoop:hadoop-common and potentially other Hadoop dependencies too.
Ok, looks like I resolved the problem by placing
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-orc_2.11</artifactId>
<version>1.14.2</version>
</dependency>
To the top of the dependencies section in Maven pom, and it worked.
I think it is just due to the order of class loading, so put it in front might resolve the conflicts during the class loading for flink-orc dependency since it will be loaded first.
How do you install Apache-Camel?
I've just started reading 'Camel in Action' 2nd ed.
In section 1.2.1, is suggests downloading the binary distribution.
I found this link to from the releases page:
Release 2.24.1
But I can only see a source-code download.
I've tried to compile from the source, but that always encounters an error.
How do you just install the binary on either Ubuntu/Redhat/Fedora?
Or have I misunderstood Apache-Camel as being a library that you can just install?
Must it always be compiled with maven?
Camel, Java-projects and build-tools
Apache Camel works like any other Java library or framework, meaning that in order to use the framework you'll have to include its java binaries (i.e .jar files) to your project as dependencies. Now to make things simpler most developers use either Maven or Gradle to create, manage and build their java projects.
From these two I would recommend Maven as it seems to be the preferred option for Camel developers with most examples (including ones in official camel site) using it. To install maven follow the official how to install maven guide.
Maven archetypes
To create example camel project you can use maven archetypes which are basically project templates that can be used to create various types of new projects. If you're reading Camel in Action you might be better off using Camel version 2.x.x in your projects with Java Development Kit version 8. Camel 3.x.x is pretty similar so it should be fairly easy to learn that after learning the basics with 2.x.x.
After installing maven you can use your Java IDE (i.e IntelliJ, VSCode, Eclipse or Netbeans) to create project from maven archetype with groupId: org.apache.camel.archetypes artifactId: camel-archetype-java and version: 2.25.4
Or use maven command line command:
# Create new camel java project for Camel version 2.25.4
mvn archetype:generate -DarchetypeGroupId="org.apache.camel.archetypes" -DarchetypeArtifactId=camel-archetype-java -DarchetypeVersion="2.25.4"
Camel project
The new project should contain project file pom.xml where you can specify all the dependencies for your project. The the camel-archetype-java should have the following dependencies listed in the dependencies section of pom.xml.
<dependencies>
<dependency>
<groupId>org.apache.camel</groupId>
<artifactId>camel-core</artifactId>
</dependency>
<!-- logging -->
<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-api</artifactId>
<scope>runtime</scope>
</dependency>
<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-core</artifactId>
<scope>runtime</scope>
</dependency>
<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-slf4j-impl</artifactId>
<scope>runtime</scope>
</dependency>
<!-- testing -->
<dependency>
<groupId>org.apache.camel</groupId>
<artifactId>camel-test</artifactId>
<scope>test</scope>
</dependency>
</dependencies>
The example route in camel-archetype-java archetypes Myroutebuilder.java is pretty complex for beginners. Hello world on a timer is generally a more simpler test on to see if things work.
package com.example;
import org.apache.camel.LoggingLevel;
import org.apache.camel.builder.RouteBuilder;
public class MyRouteBuilder extends RouteBuilder {
public void configure() {
// from("file:src/data?noop=true")
// .choice()
// .when(xpath("/person/city = 'London'"))
// .log("UK message")
// .to("file:target/messages/uk")
// .otherwise()
// .log("Other message")
// .to("file:target/messages/others");
from("timer:timerName?period=3000")
.routeId("helloWorldTimer")
.log(LoggingLevel.INFO, "Hello world");
}
}
The project generated from archetype comes with exec-maven-plugin which allows you to run the project with mvn exec:java
Java development kit - JDK
If you're using JDK 11 instead of JDK 8 you'll have to modify maven-compiler-plugin configuration a bit.
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.8.0</version>
<configuration>
<!--
<source>1.8</source>
<target>1.8</target>
-->
<source>11</source>
<target>11</target>
</configuration>
</plugin>
If you've multiple versions of JDK installed you'll have to configure your IDE to use the correct one for the project. With IntelliJ you can configure JDK used by the project from project structure settings.
With VSCode you'll need both JDK 11 and JDK 8 as VSCode Java extensions require JDK 11 to run.
Example settings.json entries for OpenJDK:
"java.configuration.runtimes": [
{
"name": "JavaSE-11",
"path": "C:\\Program Files\\AdoptOpenJDK\\jdk-11.x.x.xxx-hotspot",
"default": true
},
{
"name": "JavaSE-1.8",
"path": "C:\\Program Files\\RedHat\\java-1.8.0-openjdk-1.8.0.xxx-x",
}
],
"java.home": "C:\\Program Files\\AdoptOpenJDK\\jdk-11.x.x.xxx-hotspot"
You might also want to setup your path variable so that java -version returns correct version from the command-line.
What you really need is download the binaries for Apache Camel. The best way to get them is make use of maven (https://maven.apache.org/install.html), and existing project from GitHub and get started.
You can go to the following link: https://github.com/dilipsundarraj1/TeachApacheCamel. You can either download the project as zip file or clone the project (you need to have git installed on your machine).
After you downloaded / cloned the project use go to one of the projects: learncamel-simple-file and open the folder in command prompt.
In the command prompt run the command mvn dependency:resolve. (I am assuming you have maven and java installed on your machine). This command will download all the required binaries in the folder: c:\user\<userid>\.m2\repository, where userid is specific to you machine.
Hope it helps.
I was going through the io.qameta.allure documentation and saw this option to generate allure report using "mvn allure:serve"
mvn allure:serve
The report will be generated into a temp folder. The web server with results will start. You can additionally configure the server timeout
regarding the below lines, can we modify in the configuration to generate reports under the particulate folder instead of the temp folder?
do anyone have information about it using this command only?
You can do it by defining a configuration > reportDirectory for allure-maven in your pom.xml file.
Example: in a java/selenium/testng project, here is the reporting section of pom.xml file:
<reporting>
<excludeDefaults>true</excludeDefaults>
<plugins>
<plugin>
<groupId>io.qameta.allure</groupId>
<artifactId>allure-maven</artifactId>
<version>2.10.0</version>
<configuration>
<reportVersion>${allure.version}</reportVersion>
<reportDirectory>allure-report/</reportDirectory>
</configuration>
</plugin>
</plugins>
</reporting>
In order to generate allure reports you need to use: mvn allure:report.
To specify the path of the report generated use the allure maven plugin and in the pom.xml file use this:
<configuration>
<reportVersion>2.14.0</reportVersion>
<resultsDirectory>*allure result path*</resultsDirectory>
<reportDirectory>*allure report path*</reportDirectory>
</configuration>
The following command: mvn allure:serve always uses a temp folder on your device and I haven't seen anywhere in their documentation the ability of defining the path when using the serve command.
I am working on a Rest-Assured project which consists of testng.xml file also. I am trying to run the testng.xml file through batch. But it gives me error as -
could not find or load main class org.testng.TestNG
Below is the code for the batch file :-
cd C:\Users\workspace\RestAssured
set projectPath=C:\Users\workspace\RestAssured
java org.testng.TestNG "%projectPath%\testng.xml"
pause
I went through numerous batch file creation solution but still I am not able to replicate it. And also, in some solutions I see they have added classpath also like - set classpath=C:\Users\workspace\RestAssured\bin. But this bin folder doesn't exist in my project.
And also, Let's say, I have my project at C drive location. How to take the path of the project dynamically for different users?
You need to ensure that you set the class path properly.
Here's what you need to do:
Download the latest version of TestNG 7.0.0 from here.
Download the dependencies that TestNG and have them made available in a folder (say lib). The easiest way of figuring out the actual download URLs is by doing
http://repo1.maven.org/maven2" + groupId + "/ + artifactId + "/" + version (replace all . with /)
<dependency>
<groupId>com.beust</groupId>
<artifactId>jcommander</artifactId>
<version>1.72</version>
</dependency>
<dependency>
<groupId>org.apache.ant</groupId>
<artifactId>ant</artifactId>
<version>1.10.3</version>
</dependency>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.12</version>
</dependency>
<dependency>
<groupId>org.yaml</groupId>
<artifactId>snakeyaml</artifactId>
<version>1.21</version>
</dependency>
Once you have everything in your lib folder, create a batch file wherein you set the classpath to refer to all the jars in the lib folder,
cd C:\Users\workspace\RestAssured
set projectPath=%cd%
java -classpath "lib/*" org.testng.TestNG "%projectPath%\testng.xml"
pause
This should take care of all of your questions.
I am using maven-surefire-plugin version 2.16 along with tests written using cucumber. The .xml and .txt reports do come up fine in the sure-fire reports folder. The text within the the .xml and .txt file along with the names has a lot of extra spaces that is proportional to the number of step definitions executed cumulatively. Also the filename has a number of spaces proportional to number of steps executed. In case I run a lot of tests then the file simply does not save with the following exception
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.16:test (default-test) on project **:
Execution default-test of goal org.apache.maven.plugins:maven-surefire-plugin:2.16:test failed: org.apache.maven.surefire.util.NestedRuntimeException: null;
nested exception is org.apache.maven.surefire.report.ReporterException:
Unable to create file for report: /Users/kgupta2/git/$$$$$$$$/target/surefire-reports/Scenario: Using $$$$$ .txt (File name too long);
nested exception is java.io.FileNotFoundException: /Users/kgupta2/git/$$$$$$$$/target/surefire-reports/Scenario: Using $$$$$$$$ .txt (File name too long)
-> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/PluginExecutionException
Clearly the filename becomes long and I have verified that it is proportional to the number of steps executed. I am running cucumber using JUnit.
Here is my configuration for maven-surefire-plugin
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.16</version>
<configuration>
<testSourceDirectory>${project.basedir}/src/test/java</testSourceDirectory>
<includes>
<include>**/CucumberJunitRun.java</include>
</includes>
</configuration>
<dependencies>
<dependency>
<groupId>org.apache.maven.surefire</groupId>
<artifactId>surefire-junit47</artifactId>
<version>2.16</version>
</dependency>
</dependencies>
</plugin>
I am unable to understand why these additional spaces pop up.
I got this to work. It was an issue with incorrect configuration of plugins. Since I was using JUnit to run my cucumber tests, I went ahead and removed all references of TestNG from my pom and that got it to work magically. Some references of TestNG were coming from my parent pom. Just checked that my effective pom had some dependency of testNG. Really do not have a great clue as to why this was happening. But the correct configuration fixed a bunch of other errors that were creeping in