I have had E2E tests set up to run in Chrome for a while, however they no longer work. I'm not sure when they stopped working as they've been neglected. The project uses grunt-protractor-runner (2.1.2) which uses protractor 2.5.1
I get the following error:
Uncaught exception: Error while waiting for Protractor to sync with the page: "angular could not be found on the window"
Below is screenshot of the output:
It actually opens up Chrome , however doesn't navigate anywhere. The omnibox says "data:," and nothing else:
One thing I'm wondering, is that chrome got somewhat locked down - and the omni box doesn't work as a google browser now - but I don't see why that would affect it as it should hit a URL directly.
Can anyone advise please?
Typically this error happens in one of two ways:
There is a race condition where protractor wants to start the tests before the page is fully loaded. In this case, make sure that your onPrepare function in your protractor conf returns a promise that gets resolved once all preparations work.
The test does not navigate to the app's url. In this case, make sure that you run `browser.get();
There is an error trying to start your app. This one's a little harder to fix. You can try running the protractor element explorer and see if there's an error in the browser console.
Ive seen this problem a few times myself. The last time I fixed it was by updating to the latest version of chrome driver and the latest version of protractor. Certain earlier combinations of chrome driver and protractor can cause this problem.
Grunt protractor runner can be used with latest versions of protractor according to its documentation. I suggest having a little read of how to update to the latest versions?
I'm using Karma and Jasmine for testing my Angular JS code. I managed to run all the tests in my project using Karma, but if I try to run just a single file I'm getting the following error: ReferenceError: describe is not defined. Is it possible to run just a single test file and if yes then how?
P.S. I'm using Intellij IDEA.
Although it's not ideal, you can replace describe (and it) with fdescribe (and fit) so only those tests will be executed
On the other hand, xdescribe / xit excludes the tests from your suite
If you use Gulp, you can combine gulp-karma & yargs to pass in a pattern as argument.
source: https://stackoverflow.com/a/27696472/1782659
If you are like me and you've inherited a suite of tests that are controlled by a task runner (e.g. gulp) and you are getting failures but are unsure which file is actually failing you can sanity check an individual file by running jasmine directly against it:
1.) Make sure you have jasmine installed (it may not be if wrapped inside your task runner configuration)
npm install jasmine
2.) Run jasmine against file:
./node_modules/.bin/jasmine /path/to/my/spec/file.js
or
./node_modules/jasmine/bin/jasmine.js /path/to/my/spec/file.js
I started adding e2e tests into our project and it seems a few things changed since the last time I used Protractor. For example elementExplorer still useful, but Elementor is way more cooler.
In its the readme is says to launch it by running e.g.: webmanager start and then elementor http://localhost:3000.
But this way it ignores protractor.conf. And since I have a binary path for Chrome browser in that file, along with a logic for clearing cookies and logging into the app in onPrepare function, protractor has to know where the file is. But I can't find a way.
Starting protractor as usual via protractor protractor.conf.coffee and then elementor localhost:3000 not working either. It says "Unable to start a WebDriver session"
Yes, you should launch it by running webdriver-manager start, I use this to run the scripts: protractor script_file_conf.js since I save my scripts as javascript code in my repository. Remember that the conf file should have a reference to the script file.
I am trying to figure out a simple way to run tests on angularjs application.
I am new to the testing world, so it's a little hard to understand all the options and the difference between them.
My goal: to be able to run the tests simply from within my IDE - Eclipse.
And to tests the code on google chrome browser.
I found jasmine to be the obvious choice for writing js unit tests. The problem is choosing a runner both for the jasmine tests and for e2e tests.
Trying to keep it simple, I've come up with the following idea for a setup:
Write the unit tests in jasmine, and the e2e tests in phantomjs and syn.js.
Then configure eclipse to run phantomjs as an external tool, so that the output will go to the console in eclipse.
I also plan to have a CI job in Jenkins, and to my understanding Jenkins can also run phantom, so theoretically this solution will work the same for CI.
Alternatively, there are test running tools like Karma and Protractor. On one hand, they seem to be recommended, but on the other hand they seem to me like overkill in some cases. They require a lot of different tools/services/processes to be running in order to work, and it seems like a pain maintain all that setup if it breaks.
To my understanding: protractor runs on webdriverjs which runs on nodejs, and it requires a selenium server to be running in the background, and on top of all that the selenium opens real browser windows which seems a little pointless as opposed to headless browser testing.
Then there is Karma, that I did not yet fully understand what it's supposed to do. From what I've read it monitors the files in my project and whenever a file is changed it runs the tests. I'm not sure how it runs the tests - is it also using selenium?
And lastly, there are grunt and yeoman, which I did not understand at all what they do and how they interact or fit together with the other tools I've listed.
I would appreciate if someone could clarify what these different tools do, and how they fit together. Also, how would they fit with Jenkins as a CI server?
Also if you could comment on my "simpler setup" - does it make sense? Am I missing something?
Karma is for unit testing your JS, regardless of whether it is using Angular or not. The ins and outs of unit testing with Karma are covered very well here: http://www.yearofmoo.com/2013/01/full-spectrum-testing-with-angularjs-and-karma.html. Yes, Karma opens and closes browser windows as needed and specified in the configuration file. If you don't want any browser windows opened, you can use PhantomJS. You can run Karma from within most any IDE that is capable of running an external script, or run it via the command line.
Protractor is for end-to-end (or E2E) testing of your project as a whole. It will open a browser window and click through the pages as though it were a user, entering data where you tell it to and looking for the specified results. Protractor is a bit more complicated than just writing some Jasmine, but the results are worth it. Like Karma, you can run Protractor from within most any IDE that is capable of running external scripts or via the command line.
Yeoman is a process management system that incorporates dependency management via Bower, task automation via Grunt, and project management via Yo. It will run your tests in Karma and Protractor, minify your JS, CSS, and HTML, compile everything into appropriate files (internal JS, external libraries, and CSS) and provide you with a complete package that can be deployed. The beauty of Yeoman is that it is not specific to any one IDE. Everything it does can be done by scripting in your IDE or via the command line.
Now, having said all of this about Yeoman, you do still have to write the tests (it won't magically come up with them for you) and learn to integrate it into your development routine, but it is definitely the way to go for JS development. Eclipse is fine for JS development, but you'll get better performance and ease of use (IMHO) from WebStorm.
As for how these all fit into CI like Jenkins, I believe that both Karma and Protractor output test results in a format that Jenkins can read and display. With the scripting possibilities in Jenkins you can configure it to run the build process each time your source control repository (you are using some sort of source control, aren't you?) changes and show those results on the Jenkins page. My office has a very similar setup and we use it daily. I'm not the guy that has to do the Jenkins configuration, but I do work with Yeoman (and thus Karma and Protractor) via WebStorm on a regular basis and have had very good results.
I would say the clear choice here is Karma and Protractor. While it is true that they rely on a bunch of other stuff, they do so pretty antiseptically: protractor starts up the selenium server and then shuts it off when it's done. Once you have node installed, the other installations are all super simple. I would also install httpster, which will serve up your public director on port 3333.
Frankly, having come from a decade of doing TDD in the Java world, when I first looked at Javascript a few years ago (again), the testing picture was a complete joke. But now, I think the combination of Karma and Protractor is pretty fantastic. Inside IntelliJ, you can run the Karma tests and they are stupid fast and the results are presented in a runner that's as good as anything I've seen in the Java world (Xcode 5 has the best test integration). You can also install the ddescribe plugin in IntelliJ and have a ui for running individual tests or excluding tests.
On the protractor side, I found this post because I am at the point now where I am going to run my karma, protractor and then JUnit tests on a continuous integration server (either Jenkins or TeamCity). I was kind of surprised at the paucity of info on that leg of the trip, but the clear direction I see there is Grunt, because it will run your protractor tests then generate the JUnit-style output Jenkins wants. Grunt is also a pretty impressive addition to the JS world.
I know this sounds like a bunch of opinions, but I think that as happened in the Java world, the Javascript world has now reached that level of maturity where you are just going to have to expect things to drag other things in with them. Frankly, looks like node and npm do a pretty nice job of making that pretty seamless (vs. a decade down the drain on Maven in the Java world).
Updated: Sorry I did not read your question properly.
karma is a test runner, which is best suited for jasmine. For setting up is very very easy. Please download node, and install npm install karma. Follow the angular seed sandbox project it contains all the basic config set up for unit testing and end to end testing (in config folder).all you need is nodejs plugin installed in eclipse
Yeoman can be used for javascript minification, sass compilation e.t.c.
Install node eclipse and you can set all up in eclipse.
http://www.nodeclipse.org/
While building a single page app with AngularJS, I'm trying to integrate Jasmine tests in my build.
I did something similar before with the Maven Jasmine plugin, but I don't like to wrap my project in maven just to run the Jasmine tests. It seems cleaner to use Karma (was Testacular) for this somehow.
I'm comfortable that I'll get things running from a shell command, and my guess is that I can then run the command from Bamboo.
My questions:
Am I on the right track?
How can I best fail the build from a script, or does Bamboo recognize the Karma output automatically?
Great question. Make sure testacular.conf.js is configured to output junit xml for consumption by bamboo
junitReporter = {
// will be resolved to basePath (in the same way as files/exclude patterns)
outputFile: 'test-results.xml'
};
You can configure Testacular to run against many browsers and is pre-configured to use Chrome, we've chosen to start going headless with PhantomJS to do unit testing. Testacular already has jasmine inside.
For CI we are following the recommendation in
// Continuous Integration mode
// if true, it capture browsers, run tests and exit
singleRun = true;
If you use Ant a lot (and we do) sometimes you just want to stick with what you know... so you may want to checkout ANT, Windows and NodeJS Modules. to run node modules (ie testacular).
One note, if you are running testacular on windows, the npm install of testacular fails on hiredis module, which seems to be just *nix friendly. So, far it works fine without it.
It took us a couple of hours to prove all of this works. Hope this helps
--dan