I've created an Angular 2 Typescript application. I'm using Protractor for running e2e test cases. I need to generated coverage report for e2e test cases using Istanbul or any other tool available.
I cannot use grunt-protractor-coverage. Also, protractor-istanbul-plugin is abandoned now and cannot be used for production level applications.
Is there any other tool available for generating coverage report for e2e test cases?
On our web application we are using protractor to test real user experiences and while it accurately tests the user flow they can be quite flaky for a multitude of reasons that could be out of our control. As a result it is hard to rely on the test results because the failures could be noise.
Is there a way to run just the flaky tests? I've tried to use the protractor-flakes but it doesn't seem to work when running in parallel.
Yes, there are ways to re-run flakey tests, but you will need to use a library/plugin outside of Protractor. It doesn't look like this functionality will be available in Protractor any time soon.
I use an node module called protractor-errors. This plugin will record when a test fails and allow you to re-run only failed tests. It supports running sharded tests in parallel. The catch is that it currently only supports tests written in Jasmine.
Can protractor be used for Test Driven Developement ?
i want to know whether we can use Protractor for TDD.
If yes ,please share an example.
TDD usually refers to unit testing, which can be achieved using Karma. Protractor which wraps Selenium is intended for functional testing - this basically makes sure that the views on your app are working as expected. You wouldn't usually use these kind of tests to drive your development process as they can't test the actual input/output of your methods, which is what you really want to be doing in a TDD workflow.
You can use Protractor to do the end to end testing. That complements the unit testing which is part of the TDD process.
While, a google search gives the purpose for Karma and Protractor, I am keen to know as to what are the best practices when it comes to writing automated tests. Is it a recommended practice to write both Karma and Protractor tests? Is this an overkill on the project. How can one find the optimal balance?
It is not an overkill!
Only Protractor runs end-to-end (e2e) tests, i.e. your complete application, so this is the only reliable way to test the end result.
However, errors due to individual pieces of your code are hard to track with e2e tests. Also Protractor is slow and not suitable for running in background upon every source file edit, as Karma can do.
See my answer here for more detailed discussion of use cases, advantages and limitations of Karma and Protractor.
There are ton of questions asking how to mock http responses in protractor tests. How to do this is not the question, should we do this is the question.
http://en.wikipedia.org/wiki/Test_fixture#Software
I've been a QA Engineer for over 4 years, and most of my automated test experience deals with both low level (unit) tests of controllers, models, etc and high level (integration) tests of full systems. In my ruby world experience, we used Capybara for integration tests along with blueprint and factorygirl (for different projects) to create mock database entries. This was our integration/E2E testing.
I've only recently moved to a javascript team using AngularJS. The original built-in testing framework (now deprecated) had a mock Backend module which seemed suitable for our needs. Protractor is now the standard. Only after protractor gained steamed, have I heard the backlash of using fixtures for E2E testing. Many posts are pointing out that E2E testing should be testing the full stack, so any backends should not be mocked and be accessible.
Should integration tests use fixtures, and why?
There is a vocabulary problem here. What is called "e2e" testing in the Angular world has nothing to do with end-to-end testing. It is an end-to-end of the UI part only, which means no e2e test at all. It is UI testing.
Gojko Adzic, in "spec by example" book, recommands to do functional, fixture-based testing "below the skin of the application", i.e. without the UI part.
To answer your question :
-Should UI tests have fixture? No, use mocks or stubs
-Should Backend tests have fixture ? Yes
You are asking 2 questions - about the e2e tests and the integration tests. :)
The e2e test, at least in Angular's world, is testing your complete application as a real user can interact with it. This includes testing your backend request and response. However, if that runs slow and requires resources, it makes perfect sense to switch to a smaller (or even fake) version of your backend for testing.
The integration test is about a part of your code, and unit test is about individual units. Both times some or all dependencies can be mocked to isolate the tests.
So in all cases using fixtures or mocks can be useful.
See my answer here for more detailed discussion of use cases, advantages and limitations of Karma and Protractor.
Yes we use ngMockE2E to mock the backend we then expose some helpers to window object so we can seed various mock data states. We also use sinon to force a specific time for testing date sensative UI so all new Date() calls returns what you want
I'm facing the same issue here with a personal code project. I'm using the MEAN stack, and my solution will be to:
use Grunt to run the tests.
before starting the Node server, use the mongoose fixtures to set up the Mongodb test db (https://github.com/powmedia/mongoose-fixtures)
start the Node server with a parameter to make it use the test db.
You could do something like this approach if on a different stack, although Grunt is very helpful as a general job runner.