Is it possible to write automatically test cases with the tool kiwi-tcms ?
write the automated code for testing and then connect it to the Kiwi API to send that test case ?
https://kiwitcms.org/
https://kiwitcms.readthedocs.io/en/latest/plugins/automation-frameworks.html
Kiwi TCMS can be used with test automation frameworks. Test names and results are fetched automatically from the test automation framework using a plugin! This is an additional software package that you have to install in your test environment and configure it to talk back to Kiwi TCMS.
Available plugins
tap-plugin: for reading Test Anything Protocol (TAP) files. Written
in Python
junit.xml-plugin: for reading junit.xml formatted files.
Written in Python
Native JUnit 5 plugin written in Java
Robot Framework plugin
Django test runner plugin
Related
Is it possible to have a kiwi-tcms test case launch an executable on a remote server in order to execute the test case and if so how could that be done ?
Short answer - NO!
Long answer:
what you are looking for is some kind of test runner or CI system - that will connect to a remote computer (or use an API directly) and launch automated test cases based on some parameters.
This brings up so many questions I can't even list all of them here but some of the most important ones:
who/where we store authentication credentials
how are progress and results monitored and reported
when are tests scheduled ? What kind of triggers are supported/desired ?
Kiwi TCMS takes a different approach when dealing with automated tests. You can schedule your tests in any way you like and then report the execution results back to Kiwi TCMS.
We are working on plugins for popular test runners, like JUnit, Python Nose, etc, that will automatically discover the result and name of your automated test cases and report them back to Kiwi TCMS.
If you do need a specific plugin/framework please open a request on GitHub and our team will take it into consideration.
Edit: upvote, comment & follow this feature request at https://github.com/kiwitcms/Kiwi/issues/914
I needed to do something similar: run automation on remote systems and report results back to Kiwi. I put together several components to get the entire system working. Here's what worked for me:
Jenkins to initiate test runs and manage remote machines
A Python script to create test runs against a test plan and write out a custom test run manifest, which is...
Passed to the automation system (.NET/C#) via more scripts to make sure the remote machine is configured correctly
Automation output is directly consumed by Jenkins to report test results for the build/job as well as consumed by another Python script that pushes results back to Kiwi
The automation system knows how to interpret the test run manifest and map test cases to test methods implementing the test. It's important to include the Kiwi caserunid throughout the pipeline so the result is associated with the correct Kiwi entry.
please open feature requests on GitHub if you'd like to get test runner adapters (essentially plugins) for Kiwi TCMS. GitHub is the only place where we can track who needs what and prioritize!
In the developer guide of Codename One, it's written so: «Codename One includes a built in testing framework and test recorder tool as part of the simulator. This allows developers to build both functional and unit test execution on top of Codename One. It even enables sending tests for execution on the device (pro-only feature).»
Using a pro account, what are the steps to do a test on a real device?
You need to set the build hint build.unitTest=1 which will build a version of the app that just executes the unit tests. This isn't as useful without continuous integration which is an enterprise feature...
To implement this you can do something like add a build target to your ant file:
<target name="test-for-ios-device" depends="clean,copy-ios-override,copy-libs,jar,clean-override">
<codeNameOne
jarFile="${dist.jar}"
displayName="${codename1.displayName}"
packageName = "${codename1.packageName}"
mainClassName = "${codename1.mainName}"
version="${codename1.version}"
icon="${codename1.icon}"
vendor="${codename1.vendor}"
subtitle="${codename1.secondaryTitle}"
buildArgs="build.unitTest=1"
targetType="iphone"
certificate="${codename1.ios.debug.certificate}"
certPassword="${codename1.ios.debug.certificatePassword}"
provisioningProfile="${codename1.ios.debug.provision}"
appid="${codename1.ios.appid}"
/>
</target>
With the enterprise version you can do a synchronized build, get the result then run it on devices using appium etc. We'll come up with a guide for that soon.
I have a application scenario where I am currently executing my test cases(written in selenium C#) using MTM, I want to know whether same can be achieved using selenium java from eclipse?
The same can be achieved using TFS 2015, in which we can create a new build of type ant/maven.Once the build has been created queueing the build would result in execution of test cases on pre configured agent machine.
While creating a build we need to provide build.xml (for ant) or pom.xml (for maven) which will contain all the details specific to the project and execute the same.
No
You can't use MTM to execute Java tests, however Microsoft is moving away form that model. This is likely because of this issue and they have already moved to using Team Build with VSTS & TFS 2015+ to execute all sorts of Functional Tests.
https://blogs.msdn.microsoft.com/dgartner/2016/02/23/running-selenium-tests-as-part-of-your-release-with-vsts-release-management/
They have a specific task for this and you can now associate all automated test results with Requirements for reporting without the need for Test Cases in tge middle.
I recently have implemented JBehave with webdriver for automation. I have few queries.
can JBehave store the results in DB after the suite is completed?
Can we modify the Jbehave report to display the buildnumber?
can we run webdriver tests to run from jbehave web runber.Example of etsy.com doesn't actually run the webdriver stories.
Can we integrate the results with web-runner. i.e instead of opening target/view.index.html , can we host it on any webserver along with web-runner.
To answer your question,
1) No, Not necessary at all.But you can write your own utility classes and call the methods for DB insertion as you wish. Here The Official selenium Database access Docs,But the DatabaseStorage interface is depreciated.
2) To make a templatable views , you need to look at here, the view generation can help you.
The solution for some of these things was to use continuous integration using Jenkins.
Create jenkins job as described here Integration of jbehave with jenkins
The results are available for all the test runs until the runs are purged in jenkins (which is configurable).
I have written quite a few Ruby programs for my clients test automation project.But the problem is my client wants to check the output to his PC without instilling ruby.He is asking me to make one web application where he can select functionality from drop down and run the automation by clicking run button.And after completion the test it will auto generate a test result.Is there any possible way to do so?
He is asking me to make one web application where he can select functionality from drop down and run the automation by clicking run button
A work around to let your client run the current tests without installing the environment on their PC is to use the CI tool Jenkins, and a few of the Jenkins plugins such as CI Reporter which is an add-on to Test::Unit and RSpec that allows you to generate XML reports of your test and/or spec runs.
You can set Jenkins up on a server that they can hit virtually from anywhere and trigger a shell / selenium script that can execute all the unit tests and functional tests against the latest build / code, and all your client has to do is hit a "Play" button to execute them.
And after completion the test it will auto generate a test result.Is there any possible way to do so?
Here are some example images that some of the data Jenkins can export. It's really easy to set up and the community is massive in regards to support.
Screenshots of Failed Tests Results and Passed Tests Results
Hope this helps!