I've a project that's checked by the circleci and fails if broke some test. I want the circleci to fails if the current branch coverage is smaller than the coverage from master. I'm using jest for the coverage. How can I do that?
You can use: https://facebook.github.io/jest/docs/configuration.html#coveragethreshold-object.
So, when the --coverage run in your CI it will throws an exception.
I created a script that:
1) Clone the project
2) Runs the coverage command to the master
3) Saves the result
4) Download the current branch
5) Runs the coverage command to the current branch
6) Saves the result
7) Compares the results
8) If the current branch result it's smaller than the master results, the circleci fails.
If you're interest in more information you can see the post that I created: https://medium.com/#marinahaack/fails-the-circleci-build-according-to-the-coverage-of-the-current-branch-with-the-master-5ca7a306a472
Related
I have started Appium server with the below lines of code in script step
#!/bin/bash
set -ex
npm install -g appium
appium &>/dev/null &
In Android Virtual Device testing step, getting the following error message. Is there any way to avoid this error
Validating Failed to get test status, error: Failed to get test status: INVALID(TEST_SAME_AS_APP)
The Virtual Device Testing (Firebase Test Lab) feature requires two separate APKs, one that will be tested and one that will drive the test. You can read more details about the setup here: https://blog.bitrise.io/introducing-solid-and-snappy-virtual-device-testing-for-android
See the Instrumentation section
For Instrumentation tests, you'll need to change your workflow:
Add an extra task assembleDebugAndroidTest in your gradle-runner's gradle task input, so it will be something like: assembleDebug assembleDebugAndroidTest.
Add Virtual Device testing for Android step after your gradle-runner step.
As a minimum, in Virtual Device testing for Android step set Test APK path inputs to your built debug apk (e.g.: ./app/build/outputs/apk/app-debug-androidTest-unaligned.apk) Make sure to configure only one section of the step, which matches your desired test type!
(APK path is exported by the gradle-runner step, and it is set as the default for the step's input, so if you want to use a different APK then don't forget to change the step's input accordingly!)
Step 1. I have written selenium tests and bundled it into executable jar.
Step 2. When I execute this jar tests runs normally. Invokes New browser and execute all steps.
Step 3. I created bat file to execute this jar. In this case also test execution happens normally.
Step 4. TFS Build integration. (TFS 2015)
- I created new build definition in TFS
- Added build step "Batch Script"
- Entered path of bat file created in step 3 in build step
- When I Queue this build. bat file gets called and test execution starts. But It does not open new browser..console says Started IE driver...it starts driver internal to build agent. and execution fails on setting value in textbox.
Click to see TFS Build Cosole
I can't use Visual studio tests. Requirement is to selenium tests should be triggered from jar created on Step 1.
Make sure the build agent is running in interactive mode.
First you could try to RDP to the remote build server(the build agent located) and manually run the batch file to see if it works.
If it's also not work then the issue may related the IE browser on the server.
If it works then maybe the TFS build service account don't have enough permission to run the IE driver. Please make sure the build service account has local Admin permission on the build server.
I have installed cobertura plugin in Jenkins and ran unit tests with below command #go test -cover -p 1 $PKGS | go-junit-report > report.xml. I could see report.xml is created with code coverage but I see below error in Jenkins console. Could you please let us know what is the issue and how to resolve this.
/bin/sh -xe /tmp/hudson1811800402022775798.sh
[Cobertura] Publishing Cobertura coverage report...
FATAL: Unable to find coverage results
java.io.IOException: <workspace>/report.xml is not a cobertura coverage report, please check your report pattern
at hudson.plugins.cobertura.CoberturaPublisher$ParseReportCallable.invoke(CoberturaPublisher.java:566)
at hudson.plugins.cobertura.CoberturaPublisher$ParseReportCallable.invoke(CoberturaPublisher.java:536)
at hudson.FilePath.act(FilePath.java:1018)
at hudson.FilePath.act(FilePath.java:996)
at hudson.plugins.cobertura.CoberturaPublisher.perform(CoberturaPublisher.java:343)
at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:779)
at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:720)
at hudson.model.Build$BuildExecution.post2(Build.java:185)
at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:665)
at hudson.model.Run.execute(Run.java:1766)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:98)
at hudson.model.Executor.run(Executor.java:410)
[Cobertura] No coverage results were found using the pattern '**/report.xml' relative to '<some directory>'. Did you enter a pattern relative to the correct directory? Did you generate the XML report(s) for Cobertura?
Build step 'Publish Cobertura Coverage Report' changed build result to FAILURE
Finished: FAILURE
I have followed below steps to generate coverage report (for go language)
gocovxml
Install gocov xml using below commands
$ go get github.com/axw/gocov/...
$ go get github.com/AlekSi/gocov-xml
execute go test command if you have multiple packages
gocov test $PKGS -p 1 | gocov-xml > report.xml
Provide report.xml in publish cobertura coverage report (Post build action in Jenkins)
**/report.xml
I'd like to run batch using jenkins. And the status of build depends of number of files created in a specific folder. My question is how could I manage Jenkins build status depending of number of files created?
You can execute a shell script to count the files and return 1, if the count isn't expected.
Another way would be to use the Text-finder Plugin searching for a pattern in the console log.
Groovy Postbuild Plugin is another alternative:
buildUnstable() - sets the build result to UNSTABLE.
If you like to use the CLI you can use the following command:
java -jar jenkins-cli.jar -s http://...:8080/ set-build-result
Sets the result of the current build. Works only if invoked from within a build.
What is the best way to run silverlight unit tests automatically using team city?
I have found StatLight which we had working well when we used cc.net, and it says that it has support for teamcity. Does this just mean the test results output file is compatible with teamcity? Do I need to create a command line runner to run the tests? If so how do I get the test results into team city?
Thanks
TeamCity has an extensibility feature where you can output special commands to the console and the TeamCity agent will capture the commands and publish the results within TeamCity.
If you get StatLight running on your desktop - do a regular console-run. Then do another run by giving it the "--teamcity" parameter. Notice the difference in the output?
In TeamCity you can setup a Command Line Build Runner
Command executable: "<Path to statlight.exe>"
Command parameters: "-x=%system.teamcity.build.checkoutDir%\PathToXap\SilverlightClient.Tests.xap --teamcity"
Hope this helps.
There's also a StatLight TeamCity plugin that adds a test runner.
You can try using Lighthouse Silverlight Unit Test Runner, it works with every Build Server including TeamCity and CCNet because it by default produces NUnit compatible xml results file:
http://lighthouse.codeplex.com/