So I wrote some e2e tests with testcafe and they run perfectly on my local machine. Now I want them to run in a bitbucket pipeline, but I'm no able to get it to work.
My bitbucket-pipeline.yml looks like this
image: testcafe/testcafe
pipelines:
default:
- step:
caches:
- node
script:
- npm ci
- /opt/testcafe/docker/testcafe-docker.sh 'firefox:headless --no-sandbox --disable-dev-shm-usage' test/**
The error message I get on the pipeline:
A request to "http://127.0.0.1:3000/" has failed.
Use quarantine mode to perform additional attempts to execute this test.
You can find troubleshooting information for this issue at "https://go.devexpress.com/TestCafe_FAQ_ARequestHasFailed.aspx".
Error details:
Failed to find a DNS-record for the resource at"http://127.0.0.1:3000/".
Browser: Firefox 78.0 / Linux 0.0
Does anybody know what causes this exception and how to solve it?
When you're running your suite of e2e tests via a CI server like Bamboo; you expect to come back the next day or hour to see a report showing how many tests passed/failed.
In my case, my e2e tests stop running as soon as a single test fails so I come back the next morning to check the nightly build and do not have that nice report showing the stats on what passed/failed.
Checking the log file, I can see that there were some cucumber features that passed earlier on in the log prior to the failed feature; but again, a failing feature halts/stops the execution of any further tests in my situation.
The error would look something like:
AssertionError: expected 'Green' to equal 'Red'
[launcher] BUG: launcher exited with 1 tasks remaining
error 17:59:29
error 17:59:29 C:\Users\bamboo-agent-home\xml-data\build-dir\PROJ\gulp\e2e-tests.js:27
error 17:59:29 throw err;
error 17:59:29 ^
error 17:59:29 Error: protractor exited with code 100
How can I force the test suite to continue running even if some features fails so that I can finally make it to that end report with the stats: passed/failed?
Could it be a conflicting issue between how Protractor handles promises vs how chai/chai-as-promised does?
Protractor v 4.0.1
Chai v 3.4.0
Chai-as-promise v ^5.1.0
Windows 10
cucumber#0.6.0 - **`I found this with npm list cucumber`**
Node v 4.4.7
Gulp
CLI version 3.9.1
Local version 3.9.1
Problem
I'm trying to get AppVeyor running some e2e tests using Protractor and Selenium-webdriver.
Everything works fine on my local machine, but when building and testing on AppVeyor protractor exits with code 1 due to a not-found method in the selenium-webdriver, it seems. I have tried to get some information on the error message, but my search didn't get me anywhere. Most of the result come from backend testing with Java.
Here is a snippet of the buildlog:
node_modules\.bin\gulp protractor
[08:37:23] Using gulpfile C:\projects\kitos\gulpfile.js
[08:37:23] Starting 'protractor'...
Starting selenium standalone server...
[launcher] Running 1 instances of WebDriver
Selenium standalone server started at http://10.240.0.18:63619/wd/hub
..
Finished in 42.225 seconds
2 tests, 2 assertions, 0 failures
C:\projects\kitos\node_modules\protractor\node_modules\selenium-webdriver\lib\atoms\error.js:108
var template = new Error(this.message);
^
UnknownError: Can't obtain updateLastError method for class com.sun.jna.Native
at new bot.Error (C:\projects\kitos\node_modules\protractor\node_modules\selenium-webdriver\lib\atoms\error.js:108:18)
at Object.bot.response.checkResponse (C:\projects\kitos\node_modules\protractor\node_modules\selenium-webdriver\lib\atoms\response.js:109:9)
at C:\projects\kitos\node_modules\protractor\node_modules\selenium-webdriver\lib\webdriver\webdriver.js:379:20
at [object Object].promise.ControlFlow.runInFrame_ (C:/projects/kitos/node_modules/protractor/node_modules/selenium-webdriver/lib/goog/../webdriver/promise.js:1857:20)
at [object Object].goog.defineClass.notify (C:/projects/kitos/node_modules/protractor/node_modules/selenium-webdriver/lib/goog/../webdriver/promise.js:2448:25)
at [object Object].promise.Promise.notify_ (C:/projects/kitos/node_modules/protractor/node_modules/selenium-webdriver/lib/goog/../webdriver/promise.js:564:12)
at Array.forEach (native)
at [object Object].promise.Promise.notifyAll_ (C:/projects/kitos/node_modules/protractor/node_modules/selenium-webdriver/lib/goog/../webdriver/promise.js:553:15)
at goog.async.run.processWorkQueue (C:\projects\kitos\node_modules\protractor\node_modules\selenium-webdriver\lib\goog\async\run.js:130:15)
at process._tickCallback (node.js:368:9)
[launcher] Process exited with error code 1
[08:38:13] 'protractor' errored after 50 s
[08:38:13] Error in plugin 'gulp-protractor'
Message:
protractor exited with code 1
The selenium server is running, tests passes but when the server is about to shut down the error occurs.
Setup
On build start I run npm install which in turn runs node_modules\\.bin\\webdriver-manager update --standalone as a postinstall script.
Protractor is run with a gulp task with the gulp-protractor plugin and the selenium standalone jar is defined in the protractor configuration as seen here:
exports.config = {
seleniumServerJar: '../node_modules/protractor/selenium/selenium-server-standalone-2.47.1.jar',
specs: ['Tests/**/*e2e.spec.js'],
resultJsonOutputFile: 'results-protractor.json',
allScriptsTimeout: 30000
};
The complete buildlog can be seen here: https://ci.appveyor.com/project/Crevil/kitos/build/1.0.187
Apologizing for the noisy log. The deployment logging is very loud in the console and it is not configurable on AppVeyor afaict.
Local log information
As mentioned, everything runs fine on my local machine. Here is a log of protractor running.
C:\Users\BSO\Documents\Git repos\kitos>gulp protractor
[09:54:58] Using gulpfile ~\Documents\Git repos\kitos\gulpfile.js
[09:54:58] Starting 'protractor'...
Starting selenium standalone server...
[launcher] Running 1 instances of WebDriver
Selenium standalone server started at http://192.168.5.153:16772/wd/hub
..
Finished in 5.241 seconds
2 tests, 2 assertions, 0 failures
Shutting down selenium standalone server.
[launcher] 0 instance(s) of WebDriver still running
[launcher] chrome #1 passed
[09:55:08] Finished 'protractor' after 10 s
C:\Users\BSO\Documents\Git repos\kitos>
This issue is solve with the following parameter to the java runtime engine
localSeleniumStandaloneOpts: {
args: ['-Djna.nosys=true']
},
...
There are more details in the protractor issue here and the specific solution by Feodor Fitsner.
I followed this SO post
to set up my Gruntfile. If I manually downloaded Selenium standalone and specified its location in the file, my test runs successfully. Since I would like to automate this process, I tried the following configuration:
protractor_webdriver: {
start: {
options: {
path: 'node_modules/grunt-protractor-runner/node_modules/protractor/bin/',
command: 'webdriver-manager start'
}
}
};
grunt.loadNpmTasks('grunt-protractor-webdriver');
grunt.registerTask('test', ['protractor_webdriver:start','protractor:run'])
Is there a way to avoid downloading manually? I tried the above but when I ran it, I got the warning:
Running "protractor_webdriver:start" (protractor_webdriver) task
Verifying property protractor_webdriver.start exists in config...OK
File: [no files]
Options: path="node_modules/grunt-protractor-runner/node_modules/protractor/bin/", command="webdriver-manager start", keepAlive=false
Starting Selenium server
>> Selenium Standalone is not present.
Install with webdriver-manager update --standalone
So I still need to download the selenium standalone server manually?
Or maybe I missed some configuration here?
Protractor is a wrapper around WebDriverJS.
It's a nodejs program that interacts with Selenium Server and specific Browser drivers (e.g. ChromeDriver, IEDriver).
So, without using selenium server (at least for IE), you cannot run tests written with protractor. Test scripts send commands to the Selenium Server, which in turn then communicates with the browser driver. See this for a description of the architecture.
In a nutshell, without having started a Selenium server instance beforehand, nothing will happen.
You can run Protractor without Selenium by specifying
directConnect: true
in your respective Protractor configuration file (e.g. protractor.conf.js).
I'm trying to learn AngularJS. As part of this, I want to learn to use end-to-end testing. Currently, I have a directory structure like this:
node_modules
.bin
...
protractor
...
node_modules
.bin
adam-zip
glob
minijasminenode
optimist
saucelabs
selenium-webdriver
protractor
config.js
src
tests
test.e2e.js
My config.js file looks like the following:
exports.config = {
seleniumAddress: 'http://localhost:4444/wd/hub',
capabilities: {
'browserName': 'chrome'
},
specs: [
'../src/tests/test.e2e.js'
],
jasmineNodeOpts: {
showColors: true,
defaultTimeoutInterval: 30000
}
};
test.e2e.js looks like the following:
'use strict';
describe('My Sample', function () {
driver = protractor.getInstance();
beforeEach(function () {
driver.get('#/');
});
it('My First Test', function () {
message = "Hello.";
expect(message).toEqual('World.');
});
});
When I attempt to run my end-to-end tests using protractor, I run the following command from the command-line:
node_modules\.bin\protractor protractor\config.js
When I run that command, I receive the following error:
C:\Src\MyProject\node_modules\protractor\node_modules\selenium-webdriver\lib\webdriver\promise.js:1542
throw error;
^
Error: ECONNREFUSED connect ECONNREFUSED
at ClientRequest.<anonymous> (C:\Src\MyProject\node_modules\protractor\node_modules\selenium-webdriver\http\index.js:12
7:16)
at ClientRequest.EventEmitter.emit (events.js:95:17)
at Socket.socketErrorListener (http.js:1528:9)
at Socket.EventEmitter.emit (events.js:95:17)
at net.js:441:14
at process._tickCallback (node.js:415:13)
==== async task ====
WebDriver.createSession()
at Function.webdriver.WebDriver.acquireSession_ (C:\Src\MyProject\node_modules\protractor\node_modules\selenium-webdriv
er\lib\webdriver\webdriver.js:130:49)
at Function.webdriver.WebDriver.createSession (C:\Src\MyProject\node_modules\protractor\node_modules\selenium-webdriver
\lib\webdriver\webdriver.js:110:30)
at Builder.build (C:\Src\MyProject\node_modules\protractor\node_modules\selenium-webdriver\builder.js:105:20)
at runJasmineTests (C:\Src\MyProject\node_modules\protractor\lib\runner.js:191:45)
at C:\Src\MyProject\node_modules\protractor\lib\runner.js:255:5
at C:\Src\MyProject\node_modules\protractor\node_modules\selenium-webdriver\lib\goog\base.js:1178:15
at webdriver.promise.ControlFlow.runInNewFrame_ (C:\Src\MyProject\node_modules\protractor\node_modules\selenium-webdriv
er\lib\webdriver\promise.js:1438:20)
at notify (C:\Src\MyProject\node_modules\protractor\node_modules\selenium-webdriver\lib\webdriver\promise.js:328:12)
at then (C:\Src\MyProject\node_modules\protractor\node_modules\selenium-webdriver\lib\webdriver\promise.js:377:7)
What am I doing wrong?
I solved this with --standalone flag:
webdriver-manager start --standalone
I got it working by removing the following line from my config.js
seleniumAddress: 'http://localhost:4444/wd/hub',
Are you running a selenium server? The git README states the following:
WebdriverJS does not natively include the selenium server - you must start a standalone selenium server. All you need is the latest selenium-server-standalone.
source: https://github.com/angular/protractor
The error message is due to the following:
[ECONNREFUSED] The attempt to connect was ignored (because the target is not listening for connections) or explicitly rejected.
Check the URL of the Webdriver manager. The default URL is:
http://localhost:4444/wd/hub
Use a background process to run the webdriver-manager, then run protractor:
Start-Process webdriver-manager start -passthru
protractor conf.js
This will start up a Selenium Server and will output a bunch of info logs. Your Protractor test will send requests to this server to control a local browser. Leave this server running
References
Protractor Tutorial
Protractor Docs: Config File Reference
CONNECT Man Page
POSIX Man Page
For me, this had happened due to incompatible versions of Node and Protractor.
My fix-
Update Node to latest version (v7.0.0 in my case)
Follow steps given here https://stackoverflow.com/a/19333717/1902831
Install latest protractor version (4.0.10 in my case) using:
npm install -g protractor
Open another terminal and execute these command:
webdriver-manager update
webdriver-manager start
Run tests in another terminal using:
protractor conf.js
If you are using the npm protractor-webdriver grunt plugin (https://www.npmjs.org/package/grunt-protractor-webdriver) you may exeprience same kind of error.
This is due to webdriver termination just before test ends. The test runs successfully and then you have a message like :
Session deleted: Going to shut down the Selenium server
Shutting down Selenium server: http://127.0.0.1:4444
Shut down Selenium server: http://127.0.0.1:4444 (OKOK)
d:\Projets\Clouderial\nodeProjects\cld-apps\node_modules\grunt-protractor-runner\node_modules\protractor\node_modules\selenium-webdriver\http\index.js:145
callback(new Error(message));
^
Error: ECONNREFUSED connect ECONNREFUSED
at ClientRequest.<anonymous> (d:\Projets\Clouderial\nodeProjects\cld-apps\node_modules\grunt-protractor-runner\node_modules\protractor\node_modules\selenium-webdriver\http\index.js:145:16)
at ClientRequest.EventEmitter.emit (events.js:95:17)
at Socket.socketErrorListener (http.js:1547:9)
at Socket.EventEmitter.emit (events.js:95:17)
at net.js:440:14
at process._tickCallback (node.js:419:13)
I resolve this using the keepAlive option in the grunt plugin.
Here is my Gruntfile.js config :
protractor_webdriver: {
options: {
keepAlive : true // True to keep the webdriver alive
},
start: {
},
},
...
I hope this will help someone.
JM.
I also faced the same problem,the trick that worked for me is to use two cmd windows,keeping the one open after typing webdriver-manager start and without pressing enter key(if enter key is pressed the selenium server is shutdown,don't know why) open another cmd window and call your tests.
#Alexandros Spyropoulos, it took me quite some time to figure out how to run protractor and I think we had the same problem. You should open one terminal tab and run webdriver-manager start --standalone. Then open another terminal tag and run protractor ***.conf.js
In the hopes that it might help someone: I'd been having the same problem - encountering ECONNREFUSED using grunt-protractor-runner. The nuance to my case is that I was running my entire E2E environment (test files, web application and entire backend) within a Docker container.
I tried running protractor
with and without additional grunt-protractor-webdriver task to get webdriver up and running 'manually' (no difference);
with and without enabling directConnect and keepAlive settings (bypassing Selenium and resulting in crashes related to Chromedriver, one of which was described here).
The solution was rather simple: increase the amount of memory allocated to the container. On my Windows 10 host machine, I performed the following steps:
Run VBoxManage.exe modifyvm default --memory 8192 (via custom shell script) before starting the docker-machine (via Docker Quickstart script, which is equivalent to docker-machine start). (Thanks to this SO answer).
Changing my shell script to run my default container, adding the --shm-size=4G argument to my docker run command. (See docs)
You can verify if it worked by running df -h in your guest machine, by checking the amount of memory mounted on /dev/shm.
As a result, I no longer have seemingly inexplicable errors such as ECONNREFUSED.
If you run the provided protractor demo, you should try running the protractor config in the same command prompt as selenium. Try running both selenium server and protractor separately.
Make Sure first selenium runs by following command.
webdriver-manager start --standalone
And run the protractor in a separate command window.
protractor conf.js
(In my case conf.js was the config file )
I faced a similar issue to the one #David Remie faced with the Selenium Docker grid/standalone. With minimal RAM/CPU, the tests would start before the webdriver was up. A less resource consuming approach is to wait a few seconds before testing (run 'sleep 5' or something like that).
Increasing RAM was sometimes enough as a workaround for the issue, but the real problem was that the Selenium CMD (/opt/bin/entry_point.sh, starts a supervisor that runs the webdriver) from the image based on https://hub.docker.com/r/selenium/node-base/dockerfile was taking time to start the Selenium webdriver.
webdriver-manager start ----- didn't help, But below one helped
webdriver-manager start --standalone