There are ton of questions asking how to mock http responses in protractor tests. How to do this is not the question, should we do this is the question.
http://en.wikipedia.org/wiki/Test_fixture#Software
I've been a QA Engineer for over 4 years, and most of my automated test experience deals with both low level (unit) tests of controllers, models, etc and high level (integration) tests of full systems. In my ruby world experience, we used Capybara for integration tests along with blueprint and factorygirl (for different projects) to create mock database entries. This was our integration/E2E testing.
I've only recently moved to a javascript team using AngularJS. The original built-in testing framework (now deprecated) had a mock Backend module which seemed suitable for our needs. Protractor is now the standard. Only after protractor gained steamed, have I heard the backlash of using fixtures for E2E testing. Many posts are pointing out that E2E testing should be testing the full stack, so any backends should not be mocked and be accessible.
Should integration tests use fixtures, and why?
There is a vocabulary problem here. What is called "e2e" testing in the Angular world has nothing to do with end-to-end testing. It is an end-to-end of the UI part only, which means no e2e test at all. It is UI testing.
Gojko Adzic, in "spec by example" book, recommands to do functional, fixture-based testing "below the skin of the application", i.e. without the UI part.
To answer your question :
-Should UI tests have fixture? No, use mocks or stubs
-Should Backend tests have fixture ? Yes
You are asking 2 questions - about the e2e tests and the integration tests. :)
The e2e test, at least in Angular's world, is testing your complete application as a real user can interact with it. This includes testing your backend request and response. However, if that runs slow and requires resources, it makes perfect sense to switch to a smaller (or even fake) version of your backend for testing.
The integration test is about a part of your code, and unit test is about individual units. Both times some or all dependencies can be mocked to isolate the tests.
So in all cases using fixtures or mocks can be useful.
See my answer here for more detailed discussion of use cases, advantages and limitations of Karma and Protractor.
Yes we use ngMockE2E to mock the backend we then expose some helpers to window object so we can seed various mock data states. We also use sinon to force a specific time for testing date sensative UI so all new Date() calls returns what you want
I'm facing the same issue here with a personal code project. I'm using the MEAN stack, and my solution will be to:
use Grunt to run the tests.
before starting the Node server, use the mongoose fixtures to set up the Mongodb test db (https://github.com/powmedia/mongoose-fixtures)
start the Node server with a parameter to make it use the test db.
You could do something like this approach if on a different stack, although Grunt is very helpful as a general job runner.
Related
We have used React JS for front end and we need to write end to end tests. After researching online, we came across 2 options :
1. Selenium WebDriver
2. React Test Utils (https://reactjs.org/docs/test-utils.html)
What I understood was that with React Test Utils you can simulate clicks and check the status of the HTML elements by using methods like findRenderedDOMComponentWithXXX and that you can run these tests from command line so they will be faster.
Selenium does the same thing but from within the browser, it will allow you to write test in Behavior Driven Development Style(making it more readable)
My confusion :
Can we use React Test Utils to test a complete web page (complex component) or it is better to only test simple custom made components.
For example: If we have a component like Tasks which allows you to
add tasks, remove tasks, change priority which uses components like Input, DropDown and Toggle.
So is it a good idea to use React Test Utils for the entire Tasks component or we should use it for smaller inidividual components like Input, DropDown, Toggle
To test the complete Tasks component write end to end tests using Selenium.
Some other points :
simulate method in React Test Utils requires to pass event data which can increase small amount of work.
It will be great if some one can help me understand the difference between two.
You can use Jest and Selenium together. Don’t limit yourself to just one. You probably will go even beyond those two when you do Test Driven Development, or even Behavior Driven Development, based on what specifically you need to test. Jest by itself can’t really simulate UAT in my opinion. You need to open the browser to fully simulate the user experience, but those browser tests could be a very small percentage of tests relative to all of your tests. The reason why so many people have moved away from selenium is the bulkyness of configuration and maintenance of it. Also speed and reliability.
There are newer tools coming to life for browser based testing in recent years:
Puppeteer
https://github.com/GoogleChrome/puppeteer
Nightwatch.js
http://nightwatchjs.org
Also, your specific programming language offers additional features and enhancements to testing. Many Java developers use Maven and TestNG with their testing to build and debug their tests. Ruby developers might use RSpec. Whatever test runner you use, it might incorporate several dependencies on the backend for all kinds of testing tools for linting, html proofing, functionality, DevSecOps, database migrations, or even spellcheck. The key is to group all the tests you think you will need together in a single test runner (preferably, or as few as possible). Then run them all, back to back in a CI/CD pipeline; like Jenkins prior to deployment of changes. If any one fails, then your build fails in real time.
If you want to run several API tests in the pipeline, Newman is looking pretty decent to incorporate Postman tests you may already have laying around into a CI/CD pipeline. Of course alternatively, you could also build http clients, but not everyone has a coding skill set, so the tools you pick first should compliment the skillsets you have available to you as well.
Smart Bear also has several tools now that will run in a CI/CD pipeline for SOAP and UI testing, but those are much more expensive than the open source alternatives.
I am writing an integration test (not a unit test) for my AngularJS / CouchDB application using Karma. I realize that in unit tests mocking the database is important. I am deliberately NOT mocking the database since this is an integration test. I explicitly DO want to interact with the database. I also do NOT want to use protractor for these integration tests. The motivation for these integration tests is to move tests that don't necessarily rely on the UI away from protractor to increase the overall speed of our tests.
The following works when I run code in protractor exec("my command that loads data into the database").
In karma exec() is not available. How would you run a shell command in karma? Given how karma works, is this even possible?
If karma is not the best way to run integration tests, what would you recommend that I do?
Perhaps there is some way to run the shell command while karma is starting up?
On our web application we are using protractor to test real user experiences and while it accurately tests the user flow they can be quite flaky for a multitude of reasons that could be out of our control. As a result it is hard to rely on the test results because the failures could be noise.
Is there a way to run just the flaky tests? I've tried to use the protractor-flakes but it doesn't seem to work when running in parallel.
Yes, there are ways to re-run flakey tests, but you will need to use a library/plugin outside of Protractor. It doesn't look like this functionality will be available in Protractor any time soon.
I use an node module called protractor-errors. This plugin will record when a test fails and allow you to re-run only failed tests. It supports running sharded tests in parallel. The catch is that it currently only supports tests written in Jasmine.
Can protractor be used for Test Driven Developement ?
i want to know whether we can use Protractor for TDD.
If yes ,please share an example.
TDD usually refers to unit testing, which can be achieved using Karma. Protractor which wraps Selenium is intended for functional testing - this basically makes sure that the views on your app are working as expected. You wouldn't usually use these kind of tests to drive your development process as they can't test the actual input/output of your methods, which is what you really want to be doing in a TDD workflow.
You can use Protractor to do the end to end testing. That complements the unit testing which is part of the TDD process.
While, a google search gives the purpose for Karma and Protractor, I am keen to know as to what are the best practices when it comes to writing automated tests. Is it a recommended practice to write both Karma and Protractor tests? Is this an overkill on the project. How can one find the optimal balance?
It is not an overkill!
Only Protractor runs end-to-end (e2e) tests, i.e. your complete application, so this is the only reliable way to test the end result.
However, errors due to individual pieces of your code are hard to track with e2e tests. Also Protractor is slow and not suitable for running in background upon every source file edit, as Karma can do.
See my answer here for more detailed discussion of use cases, advantages and limitations of Karma and Protractor.