I'm just getting into testing my node app with Protractor. Running into an issue with various resolutions, sometimes buttons are hidden/show depending on the screen size & making tests fail occasionally that shouldn't.
Is there a way to set up my tests so that certain tests run with certain resolutions & others are excluded?
Can someone point me to some resources they've found?
If you can group the the different types of specs by file then you should use multicapabilities. See https://stackoverflow.com/a/27489765/689411
If you need to mix tests in the same file that each have a special required resolution you can change the resolution once per test
var switchToMobileResolution = function() { browser.manage().window().setSize(500, 900); }
it('does something that only works on mobile',function(){
switchToMobileResolution();
//do something...
});
Another thing you could do is you could write a simple jasmine reporter that switches resolutions if it matches a special string in the spec name...
var mobileResolutionSpecKey = 'on mobile';
var myReporter = {
specStarted: function(result) {
if(result.fullName.match(mobileResoultionSpecKey) !== null){
switchToMobileResolution();
}
}
jasmine.getEnv().addReporter(myReporter);
Related
I am facing intermittent protractor test failures across a range of test suites without any real pattern in the fail cases to indicate what could be going on, for example it's not the same tests that are failing. Sometimes I will get many failures and on other occasions just a single fail case.
I should point out that this tends to only happen when performing test runs on a Jenkins CI server we have configured (running under linux). Locally on Windows dev machines we may get a single fail case after 30-40 runs which I can live with!
The application we are testing is currently built with angular 1.5 and we are using angular material 1.1.3
Due to the animations used in angular material and the performance hit these can have, we have already tried disabling animations following this approach here which certainly make the tests quicker but dont help with the fail cases we are seeing/
I am at a point now where I am running one test suite over and over, after 5 successful runs it then failed on it's 6th attempt on our Jenkins CI environment\linux box, locally I have run this test many times now and no failures yet.
The test suite in question is detailed below along with a page object file snippet:
//test suite
describe('Operators View', function () {
var operatorPage = require('./operators.po.js'),
loginView = require('../login/login.po.js'),
page = new operatorPage();
describe('Large screen tests', function () {
beforeAll(function () {
loginView.login();
});
afterAll(function () {
loginView.logout();
});
it('should create an operator', function () {
page.settlementBtn.click();
page.operatorsBtn.click();
page.fabBtn.click();
page.createOperator();
expect(page.headline.getText()).toEqual('Operators');
});
});
});
// operators.po.js
var operatorsSection = function() {
this.helper = new Helpers();
this.headline = element(by.css('.md-headline'));
this.settlementBtn = element(by.css('[ui-sref="settlement"]'));
this.operatorsBtn = element(by.css('[ui-sref="operators"]'));
this.fabBtn = element(by.css('.md-fab'));
// Form Elements
this.licenceNumber = element(by.model('vm.transportOperator.licenceNumber'));
this.tradingName = element(by.model('vm.tradingName'));
this.name = element(by.model('vm.name'));
this.operatorAddressFirstLine = element(by.model('vm.transportOperator.address.line1'));
this.operatorAddressCityTown = element(by.model('vm.transportOperator.address.line5'));
this.operatorAddressPostCode = element(by.model('vm.transportOperator.address.postcode'));
this.payeeAddressFirstLine = element(by.model('vm.transportOperator.payee.address.line1'));
this.payeeAddressCityTown = element(by.model('vm.transportOperator.payee.address.line4'));
this.payeeAddressPostCode = element(by.model('vm.transportOperator.payee.address.postcode'));
this.opID = element(by.model('vm.transportOperator.fields.opID'));
this.spID = element(by.model('vm.transportOperator.fields.spID'));
this.schemeSelect = element(by.model('reference.scheme'));
this.schemeOptions = element(by.exactRepeater('scheme in vm.schemes').row('0'));
this.alias = element(by.model('reference.alias'));
this.reference = element(by.model('reference.reference'));
this.saveBtn = element(by.css('.md-raised'));
this.createOperator = function() {
this.licenceNumber.sendKeys(this.helper.getRandomId(10));
this.tradingName.sendKeys('Protractor Trade Name LTD');
this.name.sendKeys('Protractor Trade Name');
this.operatorAddressFirstLine.sendKeys('Protractor Town');
this.operatorAddressCityTown.sendKeys('Cardiff');
this.operatorAddressPostCode.sendKeys('PT4 4TP');
this.payeeAddressFirstLine.sendKeys('Protractor Town');
this.payeeAddressCityTown.sendKeys('Cardiff');
this.payeeAddressPostCode.sendKeys('PT4 4TP');
this.opID.sendKeys('177');
this.spID.sendKeys('Protractor Spid');
this.schemeSelect.click();
this.schemeOptions.click();
this.alias.sendKeys('PTAlias');
this.reference.sendKeys('Protractor');
this.saveBtn.click();
}
};
module.exports = operatorsSection;
In this test suite after the call to createOperator from the PO file is invoked and the savteBtn is clicked, the application will transition to a state that shows a table of created entries (after successful creation of course). We are using angular ui-router also, currently on version 0.2.18
The expectation fails with:
Expected 'Create An Operator' to equal 'Operators'.
Yet the accompanying screenshot that was captured shows the table view with an 'Operators' heading, it seems the call to page.headline.getText() inside the expectation call is being invoked too soon, so before the database operation to create the item and the page change has had a chance to complete?
I have started wondering if this could be down to the order of promises executed by protractor. I have come across articles talking about control flow in protractor and why there may be occasions when you should hook into the result of a protractor call's promise using .then() - I found this
It got me wondering if I should move the call to my saveBtn.click(), that's called at the end of my page object's createOperator function, into the test suite, so doing something like:
it('should create an operator', function () {
page.settlementBtn.click();
page.operatorsBtn.click();
page.fabBtn.click();
page.createOperator();
page.saveBtn.click().then(function(){
expect(page.headline.getText()).toEqual('Operators');
});
});
I'm starting to clutch at straws here to be honest, so any thoughts\advice from the community here would be much appreciated.
Thanks!
As requested, here is the function I use for waiting for URLs to be as they should.
public waitForUrlToBeLike (urlPart: string, timeout: number = 10000) {
return browser.wait(() => {
return browser.driver.getCurrentUrl().then((url) => {
let regex = new RegExp(urlPart);
return regex.test(url);
});
}, timeout);
}
I also use the following a lot to wait for elements to be present before making assertions on them:
public waitTillPresent (element: ElementFinder, timeout: number = 10000) {
return browser.wait(() => {
return element.isPresent();
}, timeout);
}
I have been trying to figure out a way to test the focusElement function of my directive. But somehow, whenever I call the function in my test, the classElements variable is undefined. Anybody have a clue?
Here is the directive function
$scope.focusElement = function() {
if (attrs.focusToClass) {
$scope.classElements = angular.element(document.querySelectorAll('.' + attrs.focusToClass));
if (attrs.focusToClassIndex) {
// Focus to the class with the specified index.
// Index should be within the length of elements and must not be a negative number.
if (attrs.focusToClassIndex < $scope.classElements.length && attrs.focusToClassIndex >= 0) {
$scope.elementToFocus = $scope.classElements[attrs.focusToClassIndex];
}
} else {
// Goes to the first index if the index is not specified.
$scope.elementToFocus = $scope.classElements[0];
}
} else if (attrs.focusToId) {
// Focus to the given ID
$scope.elementToFocus = angular.element(document.querySelector('#' + attrs.focusToId))[0];
}
if ($scope.elementToFocus) {
$scope.elementToFocus.focus();
}
}
Here is the unit test code.
describe('focusElement function', function () {
var targetElement;
var element;
var elementScope;
var elementToFocus;
beforeEach(function() {
targetElement = $compile('<div id="targetDiv" class="targetClass"><span id="targetSpan" class="targetClass"></span></div>')($rootScope);
$rootScope.$apply();
});
it('should return the class element with index 0', function() {
element = $compile('<div next-focus focus-to-class="targetClass"></div>')($rootScope);
});
it('should return the class element with the specific index within the range', function() {
element = $compile('<div next-focus focus-to-class="targetClass" focus-to-class-index="1"></div>')($rootScope);
});
it('should return the class element with the specific index outside the range', function() {
element = $compile('<div next-focus focus-to-class="targetClass" focus-to-class-index="-1"></div>')($rootScope);
});
it('should return the id element', function() {
element = $compile('<div next-focus focus-to-id="targetDiv"></div>')($rootScope);
});
afterEach(function() {
elementScope = element.scope();
spyOn(elementScope, 'focusElement').and.callThrough();
elementScope.focusElement();
console.log(elementScope.classElements);
expect(elementScope.focusElement).toHaveBeenCalled();
expect(elementScope.elementToFocus).toBeDefined();
expect(elementScope.elementToFocus.focus).toHaveBeenCalled();
});
});
Here is the error
The error is the result of you using document directly in the code which comprises the unit under test. The solution is to refactor your code to not use document directly, rather use jQuery style $() syntax to get much the same behaviour, make the context which $() operates an injectable dependency and then in your unit test use things like fixtures to inject a well-known context during test. Since you are using Jasmine already, you probably want to look into jasmine-jquery for convenient API to do this easily.
(Alternatively, in this specific case you could also setup a stub/mock document.querySelectorAll in your beforeEach() callback.)
That's the where-does-the-issue-stem-from and the how-to-fix-it (high level) but it pays to understand Karma a bit better before proceeding.
Skipping over lots of finer points, basically karma consists of three things combined into one single application:
An extensible 'dummy' HTTP server to serve up content (as configured in files). This server is structured as an Express JS app, which is useful to remember if you ever want to integrate custom middleware. E.g. to expose additional paths on the server in order to, say, provide a dummy API server for your Angular app code to interact with. A particular path to remember is '/base' which corresponds to the project directory as defined in your Karma config (the working directory).
A driver to point a browser to a synthesied dummy 'index.html' kind of page on the HTTP server (which is how all entries in files for which included: true are loaded, basically as <script> tags).
An API/framework for integrating unit test logic, reporting, etc. This is where the karma-jasmine type plugins interact and also how Karma is able to get the output back out and determine whether tests succeeded or not.
Points #2 and #3 have implications, in particular that the environment (window) and the HTML (document) should essentially be treated as an internal implementation detail of Karma and are not to be relied on during tests. So that means your code under test also should not rely on these details either. This is why you should restructure your code to pass in the context for $() as a dependency and then you can pass a well-known fixture in your tests and the normal context in your actual app.
Is there way to merge 2 commands into 1 command
protractor protractor.conf --suite create_buyer --params.buyerName=Buyer1
protractor protractor.conf --suite create_buyer --params.buyerName=Buyer2
like
protractor protractor.conf --suite create_buyer,create_buyer --params.suites[0].buyerName=Buyer1 --params.suites[1].buyerName=Buyer2
to make this commad work i need to know the current suite index
is it possible?
may there is better way!
As far as i know, there is no way you can do it using Protractor. However, if you want to run the same suite twice, then there is a better way of handling this situation using data-providers. There are many ways to create a data driven framework for protractor, but the easiest one that i feel is using jasmine-data-provider, which is an npm package. Here's how you can do it -
Update conf.js file to include the suite and params -
suites: {
create_buyer: ['./firstSpec.js'] //All your specs
},
params: {
buyerName1: '',
buyerName2: ''
},
Update all test scripts file to include the dataprovider -
//Require the dataprovider
var dp = require('/PATH_TO/node_modules/jasmine-data-provider'); //Update your path where the jasmine-data-provider is installed
//Usage of dataprovider in your specs
var objectDataProvider = {
'Test1': {buyerName: browser.params.buyerName1},
'Test2': {buyerName: browser.params.buyerName2}
};
dp(objectDataProvider, function (data, description) {
//Whole describe or anything in the dp() runs twice
describe('First Suite Test: ', function(){
it('First Spec', function(){
element.sendKeys(data.buyerName); //usage of the data buyerNames
//Your code for the spec
});
//All your specs
});
});
Now pass in the parameters using command prompt -
protractor protractor.conf --suite create_buyer --params.buyerName1=Buyer1 --params.buyerName2=Buyer2
NOTE: However the issue here is that you cannot run one single suite at a single stretch with one buyerName. For ex: You cannot run all specs in the suite create_buyer at one stretch using buyerName1. Instead, one spec will run twice serially, once with buyerName1 and buyerName2, then it continues to next spec. But i guess, that should also work if your requirement is to not use a strict flow for one buyer (i.e, to complete end-to-end testing of suite create_buyer with buyerName1 and then run suite create_buyer with buyerName2 - this shouldn't be the case as the thumb rule of automation says that one test script shouldn't depend on another).
Hope it helps
I have created tests for my application. Everything works but it runs slow and even though only 1/3 of the application is tested it still takes around ten minutes for protrator to create the test data, fill out the fields, click the submit button etc.
I am using Google Crome for the testing. It seems slow as I watch protractor fill out the fields one by one.
Here's an example of my test suite:
suites: {
login: ['Login/test.js'],
homePage: ['Home/test.js'],
adminPage: ['Admin/Home/test.js'],
adminObjective: ['Admin/Objective/test.js'],
adminObjDetail: ['Admin/ObjectiveDetail/test.js'],
adminTopic: ['Admin/Topic/test.js'],
adminTest: ['Admin/Test/test.js'],
adminUser: ['Admin/User/test.js'],
adminRole: ['Admin/Role/test.js']
},
This is one test group:
login: ['Login/test.js'],
homePage: ['Home/test.js'],
adminUser: ['Admin/User/test.js'],
adminRole: ['Admin/Role/test.js']
This is another test group:
adminPage: ['Admin/Home/test.js'],
adminObjective: ['Admin/Objective/test.js'],
adminObjDetail: ['Admin/ObjectiveDetail/test.js'],
adminTopic: ['Admin/Topic/test.js'],
adminTest: ['Admin/Test/test.js'],
The two groups can run independently but they must run in the order I have above. After the answers I did read about sharing but I am not sure if this helps my situation as my tests need to be run in order. Ideally I would like to have one set of tests run in one browser and the other set in another browser.
I read about headless browsers such as PhantomJS. Does anyone have experience with these being faster?
Any advice on how I could do this would be much appreciated.
We currently use "shardTestFiles: true" which runs our tests in parallel, this could help if you have multiple tests.
I'm not sure what you are testing here, whether its the data creation or the end result. If the latter, you may want to consider mocking the data creation instead or bypassing the UI some other way.
Injecting in Data
One thing that you can do that will give you a major boost in performance is to not double test. What I mean by this is that you end up filling in dummy data a number of times to get to a step. Its also one of the major reasons that people need tests to run in a certain order (to speed up data entry).
An example of this is if you want to test filtering on a grid (data-table). Filling in data is not part of this action. Its just an annoying thing that you have to do to get to testing the filtering. By calling a service to add the data you can bypass the UI and seleniums general slowness (Id also recommend this on the server side by injecting values directly into the DB using migrations).
A nice way to do this is to add a helper to your pageobject as follows:
module.exports = {
projects: {
create: function(data) {
return browser.executeAsyncScript(function(data, callback) {
var api = angular.injector(['ProtractorProjectsApp']).get('apiService');
api.project.save(data, function(newItem) {
callback(newItem._id);
})
}, data);
}
}
};
The code in this isnt the cleanest but you get the general gist of it. Another alternative is to replace the module with a double or mock using [Protractor#addMockModule][1]. You need to add this code before you call Protractor#get(). It will load after your application services overriding if it has the same name as an existing service.
You can use it as follows :
var dataUtilMockModule = function () {
// Create a new module which depends on your data creation utilities
var utilModule = angular.module('dataUtil', ['platform']);
// Create a new service in the module that creates a new entity
utilModule.service('EntityCreation', ['EntityDataService', '$q', function (EntityDataService, $q) {
/**
* Returns a promise which is resolved/rejected according to entity creation success
* #returns {*}
*/
this.createEntity = function (details,type) {
// This is your business logic for creating entities
var entity = EntityDataService.Entity(details).ofType(type);
var promise = entity.save();
return promise;
};
}]);
};
browser.addMockModule('dataUtil', dataUtilMockModule);
Either of these methods should give you a significant speedup in your testing.
Sharding Tests
Sharding the tests means splitting up the suites and running them in parallel. To do this is quite simple in protractor. Adding the shardTestFiles and maxInstences to your capabilities config should allow you to (in this case) run at most two test in parrallel. Increase the maxInstences to increase the number of tests run. Note : be careful not to set the number too high. Browsers may require multiple threads and there is also an initialisation cost in opening new windows.
capabilities: {
browserName: 'chrome',
shardTestFiles: true,
maxInstances: 2
},
Setting up PhantomJS (from protractor docs)
Note: We recommend against using PhantomJS for tests with Protractor. There are many reported issues with PhantomJS crashing and behaving differently from real browsers.
In order to test locally with PhantomJS, you'll need to either have it installed globally, or relative to your project. For global install see the PhantomJS download page (http://phantomjs.org/download.html). For local install run: npm install phantomjs.
Add phantomjs to the driver capabilities, and include a path to the binary if using local installation:
capabilities: {
'browserName': 'phantomjs',
/*
* Can be used to specify the phantomjs binary path.
* This can generally be ommitted if you installed phantomjs globally.
*/
'phantomjs.binary.path': require('phantomjs').path,
/*
* Command line args to pass to ghostdriver, phantomjs's browser driver.
* See https://github.com/detro/ghostdriver#faq
*/
'phantomjs.ghostdriver.cli.args': ['--loglevel=DEBUG']
}
Another speed tip I've found is that for every test I was logging in and logging out after the test is done. Now I check if I'm already logged in with the following in my helper method;
# Login to the system and make sure we are logged in.
login: ->
browser.get("/login")
element(By.id("username")).isPresent().then((logged_in) ->
if logged_in == false
element(By.id("staff_username")).sendKeys("admin")
element(By.id("staff_password")).sendKeys("password")
element(By.id("login")).click()
)
I'm using grunt-protractor-runner v0.2.4 which uses protractor ">=0.14.0-0 <1.0.0".
This version is 2 or 3 times faster than the latest one (grunt-protractor-runner#1.1.4 depending on protractor#^1.0.0)
So I suggest you to give a try and test a previous version of protractor
Hope this helps
Along with the great tips found above I would recommend disabling Angular/CSS Animations to help speed everything up when they run in non-headless browsers. I personally use the following code in my Test Suite in the "onPrepare" function in my 'conf.js' file:
onPrepare: function() {
var disableNgAnimate = function() {
angular
.module('disableNgAnimate', [])
.run(['$animate', function($animate) {
$animate.enabled(false);
}]);
};
var disableCssAnimate = function() {
angular
.module('disableCssAnimate', [])
.run(function() {
var style = document.createElement('style');
style.type = 'text/css';
style.innerHTML = '* {' +
'-webkit-transition: none !important;' +
'-moz-transition: none !important' +
'-o-transition: none !important' +
'-ms-transition: none !important' +
'transition: none !important' +
'}';
document.getElementsByTagName('head')[0].appendChild(style);
});
};
browser.addMockModule('disableNgAnimate', disableNgAnimate);
browser.addMockModule('disableCssAnimate', disableCssAnimate);
}
Please note: I did not write the above code, I found it online while looking for ways to speed up my own tests.
From what I know:
run test in parallel
inject data in case you are only testing a UI element
use CSS selectors, no xpath (browsers have a native engine for CSS, and the xpath engine is not performance as CSS engine)
run them on high performant machines
use as much as possible beforeAll() and beforeEach() methods for instructions that you repeat often in multiple test
Using Phantomjs will considerably reduce the duration it takes in GUI based browser, but better solution I found is to manage tests in such a way that it can be run in any order independently of other tests, It can be achieved easily by use of ORM(jugglingdb, sequelize and many more) and TDB frameworks, and to make them more manageable one can use jasmine or cucumber framework, which has before and after hookups for individual tests. So now we can gear up with maximum instances our machine can bear with "shardTestFiles: true".
I currently run my angularjs testing environment using ngMockEE which, using a special testing module allows me to mock http responses as such;
var things = [thing01, thing02];
$httpBackend.whenGET('/v1/things?').respond(function() {return [200, things]});
In a separate file I define e2e tests which call a things.html file that includes that testing module.
I then define a e2e test like this;
browser().navigateTo('/test/thingTest.html');
expect(element('.table.things tbody tr').count()).toEqual(2);
Now, what would be nice is if I could somehow couple the two togther, so that I could write an assertion like this - avoiding hard coding that 2 which will need updating over time if I update my list of things
browser().navigateTo('/test/thingTest.html');
expect(element('.table.things tbody tr').count()).toEqual(things.length);
However I'm not quite sure if this is possible or advisable. Any thoughts?
Well, you could use an e2eData.js file containing those values, and which would be loaded both by your tests, and by the tested page:
In e2eData.js:
...
var e2eData = {
things : [thing01, thing02],
...
}
In your e2e module:
$httpBackend.whenGET('/v1/things?').respond(function() {return [200, e2eData.things]});
In your test:
expect(element('.table.things tbody tr').count()).toEqual(e2eData.things.length);