I've created an Angular 2 Typescript application. I'm using Protractor for running e2e test cases. I need to generated coverage report for e2e test cases using Istanbul or any other tool available.
I cannot use grunt-protractor-coverage. Also, protractor-istanbul-plugin is abandoned now and cannot be used for production level applications.
Is there any other tool available for generating coverage report for e2e test cases?
Related
I am writing an integration test (not a unit test) for my AngularJS / CouchDB application using Karma. I realize that in unit tests mocking the database is important. I am deliberately NOT mocking the database since this is an integration test. I explicitly DO want to interact with the database. I also do NOT want to use protractor for these integration tests. The motivation for these integration tests is to move tests that don't necessarily rely on the UI away from protractor to increase the overall speed of our tests.
The following works when I run code in protractor exec("my command that loads data into the database").
In karma exec() is not available. How would you run a shell command in karma? Given how karma works, is this even possible?
If karma is not the best way to run integration tests, what would you recommend that I do?
Perhaps there is some way to run the shell command while karma is starting up?
I was wondering whether it is possible to connect my protractor tests with TFS and run it from Microsoft Test Manager.
Integration will be very limited. You can create a Generic Test in Visual Studio and associate that to your Test Case in MTM. The Generic Test can run the actual commandline that will trigger Protractor.
The only test runner that can currently directly integrate into MTM is MsTest, any other test runner needs to be wrapped by a Generic Test in order to integrate into MTM.
In previous engagements with customers i have used a T4 template to generate MS Test stubs for each of my tests in an unsupported third party tool. I then use the MTM command line tool to import the generated Tests as Test Cases in MTM. Then you can run them against an environment.
I'm using Chutzpah console to run all tests in teamcity.
Command line options:
/junit jasmine_report.html /teamcity /failOnError
This produces JUnit report in xml format. How can i visualize this as TeamCity tab (as Specflow, etc.).
It will be great to see all passed/not passed tests. I know there is Tests tab, but it is not human-readable one.
I don't know anything about Chutzpah, but have you looked at jasmine-reporters? It is a collection of reporters supporting Jasmine 1.x or 2.x, and it includes a TeamCity reporter. Maybe you could add this extra reporter to your spec runner to get better TeamCity reporting?
https://github.com/larrymyers/jasmine-reporters
To show some html as tab on build details page, follow next instructions.
Also jasmine-reporters #bloveridge mentioned allows on-the-fly reporting about tests and test would be shown on tests tab.
While, a google search gives the purpose for Karma and Protractor, I am keen to know as to what are the best practices when it comes to writing automated tests. Is it a recommended practice to write both Karma and Protractor tests? Is this an overkill on the project. How can one find the optimal balance?
It is not an overkill!
Only Protractor runs end-to-end (e2e) tests, i.e. your complete application, so this is the only reliable way to test the end result.
However, errors due to individual pieces of your code are hard to track with e2e tests. Also Protractor is slow and not suitable for running in background upon every source file edit, as Karma can do.
See my answer here for more detailed discussion of use cases, advantages and limitations of Karma and Protractor.
There are ton of questions asking how to mock http responses in protractor tests. How to do this is not the question, should we do this is the question.
http://en.wikipedia.org/wiki/Test_fixture#Software
I've been a QA Engineer for over 4 years, and most of my automated test experience deals with both low level (unit) tests of controllers, models, etc and high level (integration) tests of full systems. In my ruby world experience, we used Capybara for integration tests along with blueprint and factorygirl (for different projects) to create mock database entries. This was our integration/E2E testing.
I've only recently moved to a javascript team using AngularJS. The original built-in testing framework (now deprecated) had a mock Backend module which seemed suitable for our needs. Protractor is now the standard. Only after protractor gained steamed, have I heard the backlash of using fixtures for E2E testing. Many posts are pointing out that E2E testing should be testing the full stack, so any backends should not be mocked and be accessible.
Should integration tests use fixtures, and why?
There is a vocabulary problem here. What is called "e2e" testing in the Angular world has nothing to do with end-to-end testing. It is an end-to-end of the UI part only, which means no e2e test at all. It is UI testing.
Gojko Adzic, in "spec by example" book, recommands to do functional, fixture-based testing "below the skin of the application", i.e. without the UI part.
To answer your question :
-Should UI tests have fixture? No, use mocks or stubs
-Should Backend tests have fixture ? Yes
You are asking 2 questions - about the e2e tests and the integration tests. :)
The e2e test, at least in Angular's world, is testing your complete application as a real user can interact with it. This includes testing your backend request and response. However, if that runs slow and requires resources, it makes perfect sense to switch to a smaller (or even fake) version of your backend for testing.
The integration test is about a part of your code, and unit test is about individual units. Both times some or all dependencies can be mocked to isolate the tests.
So in all cases using fixtures or mocks can be useful.
See my answer here for more detailed discussion of use cases, advantages and limitations of Karma and Protractor.
Yes we use ngMockE2E to mock the backend we then expose some helpers to window object so we can seed various mock data states. We also use sinon to force a specific time for testing date sensative UI so all new Date() calls returns what you want
I'm facing the same issue here with a personal code project. I'm using the MEAN stack, and my solution will be to:
use Grunt to run the tests.
before starting the Node server, use the mongoose fixtures to set up the Mongodb test db (https://github.com/powmedia/mongoose-fixtures)
start the Node server with a parameter to make it use the test db.
You could do something like this approach if on a different stack, although Grunt is very helpful as a general job runner.