Web components promise to be "the next big thing" in the web. I like the approach, but I wonder how to write end-to-end tests (ui tests). I do not talk about testing the component itself in isolation.
With Selenium WebDriver API it's not possible to access the shadow-dom. You have to rely on JavaScript and this circumvents a copule of the WebDriver features like checking if the element you interact with is visible/accessible.
I looked at Vaadin TestBench and they nearly don't use the mature WebDriver API anymore, but completely rely on JavaScript. They even have to emit chnage or blur events by hand.
I do not think web components have a bright future if writing of e2e tests is that hard. Opinions?
In 2021, several testing libraries aid in piercing the Shadow DOM, including Playwright, documented here.
Related
We use Angular for front end development and Java for backend development. We use Cucumber and Selenium for BDD and end to end testing. I have recently read about Protractor and I came to know it can be used for end to end testing. But I am not understanding why do I need it when I already have Cucumber and Selenium. Can somebody please help me understand what is the use of Protractor when I already have Cucumber and Selenium?
Thanks in Advance.
Cucumber
Cucumber supports behavior-driven development (BDD). It allows expected software behaviors to be specified in a logical language that customers can understand. Specifically, Cucumber allows the execution of feature documentation written in business-facing text. It runs automated acceptance tests written in a behavior-driven development (BDD) style. Cucumber BDD approach consists of its plain language parser called gherkin that Cucumber uses to define test cases which is designed to be non-technical and human readable, and collectively describes use cases relating to a software system. The main purpose behind Gherkin's syntax is to promote behavior-driven development practice across an entire team comprising of developers, business analysts and managers. It firmly establishes the unambiguous requirements starting in the initial phases of requirements definition by business management and in other stages of the development lifecycle.
Protractor
Protractor is an end-to-end test framework for angular and angularjs applications. Protractor works with AngularJS versions greater than 1.0.6/1.1.4, and is compatible with Angular applications. Protractor in it's basic form is a Node.js program built on top of WebDriverJS. Protractor runs tests against your application running in a real browser, interacting with it as a user would. Protractor uses the Jasmine test framework for its testing interface.
Conclusion
So the biggest difference seems to be scope of usage as Cucumber is mainly for the acceptance tests while Protractor is an end-to-end test framework specifically for the angular / angularjs based applications.
Protractor is tool just like Selenium, the difference between protractor and selenium is that protractor is better when you need to test Angular applications.
You can user Cucumber with Protractor also, just like you do with Selenium if you want to write your tests in BDD style.
Cucumber is a test runner that uses a BDD "Given, When, Then" approach to testing, allowing for descriptions and breakdowns of the tasks that make up a Scenario. You can wrap any form of test with these, and it increases the ability to communicate with the future dev/test team looking at a broken test, but also with the wider business.
Protractor is a library that allows for automation of websites, just like Selenium, with the key difference in their support for Angular.
Angular often loads in components after the page has already loaded, and Protractor will wait for these components to load without having to add in waits, which will end up speeding up your tests, or at least the writing of them.
Essentially, Protractor will make testing Angular easier out of the box, and there's no reason why you can't wrap Protractor with Cucumber if you want to take advantage of the communicative benefits of such a testing tool.
I have come to know that Selenium is the father of UI testing.
Now my question is why then Angular team has developed Protractor. Can't the same job (that Protractor does) be done with Selenium for AngularJS based web applications?
Further, people are talking about Nightwatch.js too. So why there are so many E2E testing libraries/frameworks like Protracor or Nightwatch.
Protractor combines powerful tools and technologies such as NodeJS, Selenium, webDriver, Jasmine, Cucumber and Mocha. Has a bunch of customizations from Selenium to easily create tests for AngularJS applications. Speeds up your testing as it avoids the need for a lot of “sleeps” and “waits” in your tests, as it optimizes sleep and wait times. Allows tests to be organized based on Jasmine, thus allowing you to write both unit and functional tests on Jasmine. It runs on real browsers and headless browsers...
You can find all these information here => http://www.thoughtworks.com/insights/blog/testing-angularjs-apps-protractor.
Protractor also provides you to wait for scope changes using protractor.waitForAngular() which I think may or may not be implemented using Selenium. You'll still have to bang around your head in order simulate the same feature using selenium.
My opinion
If you just want to do UI testing and not interested in listening for scope changes or any angular specific events, you can go ahead with selenium. But it's a HUGE advantage using Protractor for e2e tests for Angular.
What is the added value of Protractor over angular-scenario ? I tried to find out why was angular-scenario on the deprecation path, I have read this post, and watched this video
, but I am still unsatisfied by the fact there are no obvious advantage to use Protractor.
In Ruby On Rails land, capybara is one of the major integration test tools, Watir is another. Both utilize the WebDriver api to interact with the browsers. This is the standard way to interact with the browser/webpage for E2E testing.
Karma/ngScenarioE2E utilizes an iframe and interacts with browsers with karma plugins. While effective, its far from standard.
This to me in the main benefit of using protractor, the low-level access to the browser. Whether or not that's enough reason for them to switch, is really up to the google team, and it seems like they've made up their mind on this.
I'm sorry if this question isn't appropriate: I'm new to test-driven development, especially on JavaScript.
I have a working Backbone.js application, and I'd like to add some JavaScript tests to it. But I don't really know where to start.
Specifically, I'd like to write tests that are passed a URL, and check that the rendered page contains particular DOM elements. (So functional tests, rather than unit tests, I guess.)
What is a good way to do this? Could anyone recommend good practical resources for getting tests like this up and running?
Apologies for the newbie question.
I have faced the same problem lately and Ryan Roemer book Backbone.js Testing (http://backbone-testing.com/) is a great place to start.
His process is not purely TDD, but he gives some nice tips.
When taking it to practice, I will recommend a Yeoman generator like:
https://github.com/mrichard/generator-marionette (Marionette)
https://github.com/revathskumar/generator-backbone-mocha (Backbone and Mocha)
https://github.com/richistron/generator-cornelio (Karma and Backbone)
They will save you some time on the setup. Also, if you are using Rails, Konacha https://github.com/jfirebaugh/konacha has been pretty handy for me.
Good testing!
Selenium is on de-factor way to run functional tests in the browser. Selenium has a firefox plugin that will record actions and replay them. That would be the easiest way to start. You can look at Selenium's API docs to write code to interact with the browser.
Sause Labs (http://saucelabs.com/docs/quickstart) has selenium in the cloud which is useful so you don't need to have a browser window open on you desktop and run the tests. PhantomJS can also run a "headless" browser (it does't open a window on you computer, just runs in the background).
Once you have selenium working you might want to look into acceptance testing. The tests are written in plain english use Gherkin (or similar) syntax and it runs functions to run the test and output reports if there were errors or not.
I wrote an article how to use Selenium for Acceptance Testing http://alex-craft.com/blog/2014/acceptance-testing-for-web-app
Tools: WebDriver (Selenium or Phantom.js), wd.js, node.js, mocha.js.
There's an example with explanation how to use meaningful names instead of CSS selectors, eventual conditions, etc.
I need to test an angular application and I'm thinking which testing framework to use.
Can someone please list the differences between those 2 testing framework, what is good / bad in each. when should each of them should be used? can the one replace the second?
Edit 1:
I'll try to focus my question. what I'm really interested in is the technical abilities of the 2 frameworks.
for example, some of the differences I have found:
selenium webdriver has the ability to run on grid in parallel on multiple machines, on the other side, angular scenarios runs much faster than selenium.
with selenium I can use keyboard keys such as Enter, Backspace and more and also do drag and drop actions.
selenium is not dependent on the app technology, angular is for angular apps only
Those are the type of differences I'm looking for - what can be done with one and can't be done with the other
thanks
Both should be used for testing an angular application. However, in my experience, they serve different purposes.
Angular e2e tests are your best friends when integrating new features into your application, or changing already existing features. They are there to make sure your application still behaves as expected if you are making changes to your code. These tests are much faster than selenium, do not need to be as in-depth, and, in my experience, are best executed before you push a new feature to your version control server.
Selenium tests should be used for regression testing. These tests should be far more in-depth than Angular e2e tests, and should be performed before pushing code into production.
UPDATE: This question may be less relevant now. As of January 2014, Angular plans to move from their own E2E suite to the protractor library for function tests, which use Selenium Webdriver.
If you're starting a new Angular project, you may want to look into using Protractor, as it is going to replace the current method of E2E Testing in the near future.
(Source: http://docs.angularjs.org/guide/dev_guide.e2e-testing)
I am struggling with this question as well.
Here is my thinking today:
1) Use Angular e2e testing for integration/regression testing the UI with mocks of your external dependencies (like the app server).
2) Use Selenium for integration/regression testing the complete system in a test/qa environment.
It seems Angular e2e testing helps fill in the gaps that unit testing leaves in testing view and controller logic that is tied to UI events. All of the Angular tests should mock external dependencies.
Selenium seems like the best fit for testing the entire system as the user will experience it.
This is such a subjective question, and therefore I'd first tell you to do some research. There is not much difference.
It is a very new project, whereas Selenium is a lot more mature. For instance, the JSON Wire Protocol that WebDriver is built on top of, is an actual W3C spec. This shows the project is very mature and very stable.
Due to the above, Selenium has a much much wider set of documentation, and a much wider community.
With Selenium you have a variety of different languages to use to code the tests in, as well as use the IDE. Angular limits use to pure JavaScript. For a lot of manual testers, this is a skill most do not have.
Both Selenium and Angular are open source, which (believe me) helps tremendously in debugging problems and understanding design decisions in their respective API's.