Is the TestNG Executable JAR files alone is enough to work with TestNG framework or the TestNG plugin is essential? I have the JAR files, can i add to my library and work with TestNG?
Plugin just simplifies running things. You do not need it so to say. But, if you say want to run a single test or a suite file - a plugin would give you a simple way to right click and trigger it off. But with no plugin you will have to run with the java run command or through maven command if that's what you are running.
So the answer is yes - you can run but the plugin will always help while you are developing your tests.
Currently I have an Angular application built on top of NG6-starter. The NG6-Starter uses webpack to bundle the application. It also supports unit testing by default via Karma, but karma uses webpack as well in order to get all the sources injected to it's browser.
Now, my issue is, that I would like to add my application to a classic CI/CD pipeline: Some static analysis, then package build, then unit tests, then etc, and I do not want to break the principle of "Test against the artifact that you are going to deploy" principle. Since karma currently builds the application for itself, it does not really rely on the artifact, that is going to be deployed, even though by the stage karma runs, it is already built.
My question is, if you have any idea/practice/example/experience with this topic?
Okay, so it seems so, that in order to achieve this, the spec files should be built with webpack during build time. For this, the easiest way seemed to be to simple introduce a new chunk entry point to the application.
I am relatively new to Selenium WebDriver and a self learner. I have created a webDriver project which consists of different packages and have made use of Page Factory concepts extensively. I use TestNg framework to run the suite to generate my report. So my test cases are also following the TestNG framework concepts.
Now my team wants to use my script to be run at every build to test the sanity. The build team just wants to run my whole script by just a click. It can be shell script command or .exe or jar. Build team uses linux m/c and they dont have Ecplise , TestNg installed in their machine. Their intention is whenever build is given they want to run my script by just a click or a command in Command Prompt.( it has to be that simple for them) and a report should be generated in some location in hdfs
My script runs on FF version 32 and Selenium webDriver 2.44.0
Would really appreciate if someone could give me a solution that actually works for my requirement.
I found a similar query from someone but i am not sure if the answer still suits.
How to make Java executable Jar file of WebDriver project
Could somebody please give me a solution. Or the solution mentioned in the above link stil is the best?
Regards
There are few ways to do it:
Use CI tool (Jenkins): You can set up your project in Jenkins and it will allow you to run the project in single click. Jenkins also provides you an option to run the tests periodically so you can configure it in such a way that it will always get executed after certain time period.Jenkins is also capable to trigger the execution if there is any changes in source code of tests. I'd suggest to use this tool. https://jenkins-ci.org/
Use .bat or .sh file: I am not sure if you are using any build tool like ANT or Maven, if you are using any one of them then you can write a .bat or .sh file to run the tests. ANT
If you are not using any build tool then start using, it will help to run the tests in simplest way.For me to run entire suite, I just type below command in terminal from root directory of project
ant run -Dsuite=all
If you are more concerned about system configuration for e.g. the system which will be used by build team does not have a specific version of browser or specific required library. Then I'd suggest to keep all essentials like browser installation files, libraries in your project directory. This is not a good practice but yes it will help others to run the tests smoothly.
I would suggest just exporting a jar file for your project. All the libraries will be packaged together in the jar files(including TestNG) and you can simply double click on the jar file to start your tests.
External resources (if any), will have to be available for the jar file though. So, you can provide the jar file and the external resources together. The external resources might include your test data (if any) or Portable Firefox(if you are using the portable version).
Steps:
Right click on Project -> Export -> Runnable jar file
Give a name and file path for the jar file
Select option - Extract required libraries into generated JAR
And Finish
Troubleshooting:
Check the java version for the machines that you will be running your jar file on. Programs compiled with java 7 will mostly not run if the machine has java 6. Either compile with java 6 or update the jre on the target machines.
If the jar file does not launch, try using Jarfix.
I am trying to figure out a simple way to run tests on angularjs application.
I am new to the testing world, so it's a little hard to understand all the options and the difference between them.
My goal: to be able to run the tests simply from within my IDE - Eclipse.
And to tests the code on google chrome browser.
I found jasmine to be the obvious choice for writing js unit tests. The problem is choosing a runner both for the jasmine tests and for e2e tests.
Trying to keep it simple, I've come up with the following idea for a setup:
Write the unit tests in jasmine, and the e2e tests in phantomjs and syn.js.
Then configure eclipse to run phantomjs as an external tool, so that the output will go to the console in eclipse.
I also plan to have a CI job in Jenkins, and to my understanding Jenkins can also run phantom, so theoretically this solution will work the same for CI.
Alternatively, there are test running tools like Karma and Protractor. On one hand, they seem to be recommended, but on the other hand they seem to me like overkill in some cases. They require a lot of different tools/services/processes to be running in order to work, and it seems like a pain maintain all that setup if it breaks.
To my understanding: protractor runs on webdriverjs which runs on nodejs, and it requires a selenium server to be running in the background, and on top of all that the selenium opens real browser windows which seems a little pointless as opposed to headless browser testing.
Then there is Karma, that I did not yet fully understand what it's supposed to do. From what I've read it monitors the files in my project and whenever a file is changed it runs the tests. I'm not sure how it runs the tests - is it also using selenium?
And lastly, there are grunt and yeoman, which I did not understand at all what they do and how they interact or fit together with the other tools I've listed.
I would appreciate if someone could clarify what these different tools do, and how they fit together. Also, how would they fit with Jenkins as a CI server?
Also if you could comment on my "simpler setup" - does it make sense? Am I missing something?
Karma is for unit testing your JS, regardless of whether it is using Angular or not. The ins and outs of unit testing with Karma are covered very well here: http://www.yearofmoo.com/2013/01/full-spectrum-testing-with-angularjs-and-karma.html. Yes, Karma opens and closes browser windows as needed and specified in the configuration file. If you don't want any browser windows opened, you can use PhantomJS. You can run Karma from within most any IDE that is capable of running an external script, or run it via the command line.
Protractor is for end-to-end (or E2E) testing of your project as a whole. It will open a browser window and click through the pages as though it were a user, entering data where you tell it to and looking for the specified results. Protractor is a bit more complicated than just writing some Jasmine, but the results are worth it. Like Karma, you can run Protractor from within most any IDE that is capable of running external scripts or via the command line.
Yeoman is a process management system that incorporates dependency management via Bower, task automation via Grunt, and project management via Yo. It will run your tests in Karma and Protractor, minify your JS, CSS, and HTML, compile everything into appropriate files (internal JS, external libraries, and CSS) and provide you with a complete package that can be deployed. The beauty of Yeoman is that it is not specific to any one IDE. Everything it does can be done by scripting in your IDE or via the command line.
Now, having said all of this about Yeoman, you do still have to write the tests (it won't magically come up with them for you) and learn to integrate it into your development routine, but it is definitely the way to go for JS development. Eclipse is fine for JS development, but you'll get better performance and ease of use (IMHO) from WebStorm.
As for how these all fit into CI like Jenkins, I believe that both Karma and Protractor output test results in a format that Jenkins can read and display. With the scripting possibilities in Jenkins you can configure it to run the build process each time your source control repository (you are using some sort of source control, aren't you?) changes and show those results on the Jenkins page. My office has a very similar setup and we use it daily. I'm not the guy that has to do the Jenkins configuration, but I do work with Yeoman (and thus Karma and Protractor) via WebStorm on a regular basis and have had very good results.
I would say the clear choice here is Karma and Protractor. While it is true that they rely on a bunch of other stuff, they do so pretty antiseptically: protractor starts up the selenium server and then shuts it off when it's done. Once you have node installed, the other installations are all super simple. I would also install httpster, which will serve up your public director on port 3333.
Frankly, having come from a decade of doing TDD in the Java world, when I first looked at Javascript a few years ago (again), the testing picture was a complete joke. But now, I think the combination of Karma and Protractor is pretty fantastic. Inside IntelliJ, you can run the Karma tests and they are stupid fast and the results are presented in a runner that's as good as anything I've seen in the Java world (Xcode 5 has the best test integration). You can also install the ddescribe plugin in IntelliJ and have a ui for running individual tests or excluding tests.
On the protractor side, I found this post because I am at the point now where I am going to run my karma, protractor and then JUnit tests on a continuous integration server (either Jenkins or TeamCity). I was kind of surprised at the paucity of info on that leg of the trip, but the clear direction I see there is Grunt, because it will run your protractor tests then generate the JUnit-style output Jenkins wants. Grunt is also a pretty impressive addition to the JS world.
I know this sounds like a bunch of opinions, but I think that as happened in the Java world, the Javascript world has now reached that level of maturity where you are just going to have to expect things to drag other things in with them. Frankly, looks like node and npm do a pretty nice job of making that pretty seamless (vs. a decade down the drain on Maven in the Java world).
Updated: Sorry I did not read your question properly.
karma is a test runner, which is best suited for jasmine. For setting up is very very easy. Please download node, and install npm install karma. Follow the angular seed sandbox project it contains all the basic config set up for unit testing and end to end testing (in config folder).all you need is nodejs plugin installed in eclipse
Yeoman can be used for javascript minification, sass compilation e.t.c.
Install node eclipse and you can set all up in eclipse.
http://www.nodeclipse.org/
i'm looking for some plugin in hudson that runs unit tests, i get two names post build task, and CMake build, but by description anyone was created to run unit tests, them i want some indication about tools that help me to achieve this, or some tutorial or link that make one of these two tools work like expected to my case.
Hudson (or preferably Jenkins, which is much more actively maintained and improved) doesn't have plugins to run unit tests per se - you would normally do that as a step in your build script. I think the Cmake build would handle it in your case, e.g. "cmake test"
You'd need to look at a unit test framework that works with the language your code is written in - JUnit or TestNFG for Java, CPPUnit for C/C++, NUnit for .NET, etc.
Hudson / Jenkins do have plugins that will record the unit test results - the xUNit plugin is a good generic example.