Selenium Caching Issue w/ Nightwatch.js Integration Tests - selenium-webdriver

I have a small, but growing (hopefully) suite of integration tests that I've built using Nightwatch.js (v0.5.6) on top of the Selenium webdriver (v2.41.0). I've always gotten the occasional Element not found in the cache error, but I'm working on a large validation test case - a file that contains over 2 dozen individual tests plus setUp(). I haven't been able to finish running this test yet and that's a problem.
There was an error while executing the Selenium command - enabling the --verbose option might offer more details.
Element not found in the cache - perhaps the page has changed since it was looked up
The error always seems to occur right at the end of my setUp() function, but I can't find the answer that will prevent this caching from happening. Here's my setUp() function:
setUp: function(browser) {
console.log('Logging in & navigating to Eligibility Groups...');
login(browser, app.masterAdminUsername, app.masterAdminPassword)
// Navigate to Eligibility Groups
.waitForElementVisible('button[data-action="EligibilityGroups"]', 1000, function() {
browser
.click('button[data-action="EligibilityGroups"]', function() {
console.log('Link clicked. Waiting for #btnCreate to be visible');
browser
.waitForElementVisible('#btnCreate', 1000, function() {
console.log('Exiting setUp()');
});
});
})
}
Eventually, I get this:
There was an error while executing the Selenium command - enabling the --verbose option might offer more details.
Element not found in the cache - perhaps the page has changed since it was looked up
Command duration or timeout: 3.72 seconds
For documentation on this error, please visit: http://seleniumhq.org/exceptions/stale_element_reference.html
Build info: version: '2.41.0', revision: '3192d8a', time: '2014-03-27 17:17:32'
System info: host: 'robwilkerson.local', ip: '172.20.1.112', os.name: 'Mac OS X', os.arch: 'x86_64', os.version: '10.9.4', java.version: '1.6.0_65'
Session ID: 04f47f5c-fda0-f049-9ec1-1d3a40ac44fe
Driver info: org.openqa.selenium.firefox.FirefoxDriver
Capabilities [{platform=MAC, acceptSslCerts=true, javascriptEnabled=true, browserName=firefox, rotatable=false, locationContextEnabled=true, version=30.0, cssSelectorsEnabled=true, databaseEnabled=true, handlesAlerts=true, browserConnectionEnabled=true, nativeEvents=false, webStorageEnabled=true, applicationCacheEnabled=true, takesScreenshot=true}]
✖ Timed out while waiting for element <#btnCreate> to be visible for 1000 milliseconds. - expected "visible" but got: not visible
The test case varies, but it always seems to fail "while waiting for element <#btnCreate> to be visible for 1000 milliseconds". Changing the number of ms I wait only changes the number of ms reported in the error.
What can I do here? Is there something wrong with my scripts? Everything I've read and everything I've tried has gotten me nowhere.

In case anyone drops by, what I've found through trial and error is:
Virtually any time a .click() loads or reloads content, I need to add a .pause(). Usually, .pause(1000) is sufficient, but longer is occasionally necessary. Using .waitForElementVisible() alone rarely works consistently.
Whenever I do use .waitForElementVisible(), I've used a default timeout of 30000. There doesn't seem to be any penalty for larger values.
There may be better answers, but this has worked reasonably well for me so far.
I'm marking this as the answer for now, but am open to changing that if someone has a better strategy.

Related

[flink]Task manager initialization failed

I am new to flink. I am trying to run the flink example on my local PC(windows).
However, after I run the start-cluster.bat, I login to the dashboard, it shows the task manager is 0.
I checked the log and seems it fails to initialize:
2020-02-21 23:03:14,202 ERROR org.apache.flink.runtime.taskexecutor.TaskManagerRunner - TaskManager initialization failed.
org.apache.flink.configuration.IllegalConfigurationException: Failed to create TaskExecutorResourceSpec
at org.apache.flink.runtime.taskexecutor.TaskExecutorResourceUtils.resourceSpec.FromConfig(TaskExecutorResourceUtils.java:72)
at org.apache.flink.runtime.taskexecutor.TaskManagerRunner.startTaskManager(TaskManagerRunner.java:356)
at org.apache.flink.runtime.taskexecutor.TaskManagerRunner.<init>(TaskManagerRunner.java:152)
at org.apache.flink.runtime.taskexecutor.TaskManagerRunner.runTaskManager(TaskManagerRunner.java:308)
at org.apache.flink.runtime.taskexecutor.TaskManagerRunner.lambda$runTaskManagerSecurely$2(TaskManagerRunner.java:322)
at org.apache.flink.runtime.security.NoOpSecurityContext.runSecured(NoOpSecurityContext.java:30)
at org.apache.flink.runtime.taskexecutor.TaskManagerRunner.runTaskManagerSecurely(TaskManagerRunner.java:321)
at org.apache.flink.runtime.taskexecutor.TaskManagerRunner.main(TaskManagerRunner.java:287)
Caused by: org.apache.flink.configuration.IllegalConfigurationException: The required configuration option Key: 'taskmanager.cpu.cores' , default: null (fallback keys: []) is not set
at org.apache.flink.runtime.taskexecutor.TaskExecutorResourceUtils.checkConfigOptionIsSet(TaskExecutorResourceUtils.java:90)
at org.apache.flink.runtime.taskexecutor.TaskExecutorResourceUtils.lambda$checkTaskExecutorResourceConfigSet$0(TaskExecutorResourceUtils.java:84)
at java.util.Arrays$ArrayList.forEach(Arrays.java:3880)
at org.apache.flink.runtime.taskexecutor.TaskExecutorResourceUtils.checkTaskExecutorResourceConfigSet(TaskExecutorResourceUtils.java:84)
at org.apache.flink.runtime.taskexecutor.TaskExecutorResourceUtils.resourceSpecFromConfig(TaskExecutorResourceUtils.java:70)
... 7 more
2020-02-21 23:03:14,217 INFO org.apache.flink.runtime.blob.TransientBlobCache - Shutting down BLOB cache
Basically, it looks like a required option 'taskmanager.cpu.cores' is not set. However, I can't find this property in flink-conf.yaml and in the document(https://ci.apache.org/projects/flink/flink-docs-release-1.10/ops/config.html) either.
I am using flink 1.10.0. Any help would be highly appreciated!
That configuration option is intended for internal use only -- it shouldn't be user configured, which is why it isn't documented.
The windows start-cluster.bat is failing because of a bug introduced in Flink 1.10. See https://jira.apache.org/jira/browse/FLINK-15925.
One workaround is to use the bash script, start-cluster.sh, instead.
See also this mailing list thread: https://lists.apache.org/thread.html/r7693d0c06ac5ced9a34597c662bcf37b34ef8e799c32cc0edee373b2%40%3Cdev.flink.apache.org%3E

'amplify init' keeps failing

I recently got myself a new PC(Predator Helios 300) and I wanted to start using aws there but when I try to perform amplify init I get the error below even though I already did all the other steps such as configuration.
× Root stack creation failed
init failed
{ SignatureDoesNotMatch: Signature expired: 20190427T235724Z is now earlier than 20190428T094952Z (20190428T095452Z - 5 min.)
at Request.extractError (C:\Users\sahve\AppData\Roaming\npm\node_modules\#aws-amplify\cli\node_modules\aws-sdk\lib\protocol\query.js:50:29)
at Request.callListeners (C:\Users\sahve\AppData\Roaming\npm\node_modules\#aws-amplify\cli\node_modules\aws-sdk\lib\sequential_executor.js:106:20)
at Request.emit (C:\Users\sahve\AppData\Roaming\npm\node_modules\#aws-amplify\cli\node_modules\aws-sdk\lib\sequential_executor.js:78:10)
at Request.emit (C:\Users\sahve\AppData\Roaming\npm\node_modules\#aws-amplify\cli\node_modules\aws-sdk\lib\request.js:683:14)
at Request.transition (C:\Users\sahve\AppData\Roaming\npm\node_modules\#aws-amplify\cli\node_modules\aws-sdk\lib\request.js:22:10)
at AcceptorStateMachine.runTo (C:\Users\sahve\AppData\Roaming\npm\node_modules\#aws-amplify\cli\node_modules\aws-sdk\lib\state_machine.js:14:12)
at C:\Users\sahve\AppData\Roaming\npm\node_modules\#aws-amplify\cli\node_modules\aws-sdk\lib\state_machine.js:26:10
at Request.<anonymous> (C:\Users\sahve\AppData\Roaming\npm\node_modules\#aws-amplify\cli\node_modules\aws-sdk\lib\request.js:38:9)
at Request.<anonymous> (C:\Users\sahve\AppData\Roaming\npm\node_modules\#aws-amplify\cli\node_modules\aws-sdk\lib\request.js:685:12)
at Request.callListeners (C:\Users\sahve\AppData\Roaming\npm\node_modules\#aws-amplify\cli\node_modules\aws-sdk\lib\sequential_executor.js:116:18)
message:
'Signature expired: 20190427T235724Z is now earlier than 20190428T094952Z (20190428T095452Z - 5 min.)',
code: 'SignatureDoesNotMatch',
time: 2019-04-27T23:57:24.753Z,
requestId: 'ab179ef3-699b-11e9-bfe3-4ddc7ceb66ee',
statusCode: 403,
retryable: true }
After doing some research It seems to be a verification problem. Does anyone has experience with this or knows how to resolve this issue. Thanks a lot!
Any time you see an error like "is now earlier than" around some numbers that look like timestamps (20190427T235724Z -> 2019-04-27 23:57:24 UTC), that's an indicator that the error is time related. Time matters for cryptography in order to validate certificates (so that an attacker cannot break a certificate and use it after its expiration, among other reasons) [1]. In this case, either your clock or the remote server clock is wrongly set. Since the remote server in this case is AWS, it is highly unlikely that they have any significant clock drift, leaving you as the possible outlier.
Given that you mentioned a new computer, it is even more likely that this is due to an incorrectly set system clock.
Reset/synchronize your system clock and the error should disappear.
Reference [1]: https://security.stackexchange.com/q/72866/47422

Why some of the elements cannot be selected by Selenium webdriver?

I've been using Selenium webdriver for quite a while. Recently I came across this page: https://www.myagedcare.gov.au/service-finder
It seem very simple and I try to use id locator: bylocation
However, no matter how long I wait for the element to be ready by adding wait for some time, or wait for element to be ready like:
WebDriver driver = getDriver();
WebDriverWait wait = new WebDriverWait(driver, 10);
wait.until(
ExpectedConditions.elementToBeClickable(By.id("byname")))
.click();
the element is never ready, and I always get error like "no such element: Unable to locate element:"
What might be going on? I've used locator for quite a while and I do not know how this can happen.
I am using chromedriver 2.30 and Chrome browser v.60.
A simple id locator like this in www.google.com does not give me any trouble. I tested it so that I do not think it's driver or browser issue.
Logs
net.serenitybdd.core.exceptions.SerenityWebDriverException: Timed out after 10 seconds waiting for visibility of element located by By.xpath: //input[#id='bylocation']
Build info: version: '2.46.0', revision: '61506a4624b13675f24581e453592342b7485d71', time: '2015-06-04 10:22:50'
System info: host: 'yun-PC', ip: '192.168.1.14', os.name: 'Windows 7', os.arch: 'amd64', os.version: '6.1', java.version: '1.8.0_71'
Caused by: org.openqa.selenium.NoSuchElementException: no such element: Unable to locate element: {"method":"xpath","selector":"//input[#id='bylocation']"}
(Session info: chrome=60.0.3112.50)
(Driver info: chromedriver=2.30.477700 (0057494ad8732195794a7b32078424f92a5fce41),platform=Windows NT 6.1.7601 SP1 x86_64) (WARNING: The server did not provide any stacktrace information)
Command duration or timeout: 20.04 seconds
For documentation on this error, please visit: http://seleniumhq.org/exceptions/no_such_element.html
Try this xpath and let me know if it works,
//div[#id='edit-search-by--2']//label[normalize-space(text())='Location']/preceding-sibling::input[#type='radio']
Also try waiting for visibility of element and not for element to be clickable.
Edit:
The above Xpath is perfect. Problem was that the element was located in a frame.
Below code will get you going with the click part:
driver.get("https://www.myagedcare.gov.au/service-finder?tab=help-at-home");
new WebDriverWait(driver, 15).until(ExpectedConditions.visibilityOfElementLocated(By.xpath("//iframe[#id='content']")));
driver.switchTo().frame(driver.findElement(By.xpath("//iframe[#id='content']")));
new WebDriverWait(driver, 15).until(ExpectedConditions.visibilityOfElementLocated(By.xpath("//label[normalize-space(text())='Location']/preceding-sibling::input[#type='radio']")));
WebElement location = driver.findElement(By.xpath("//label[normalize-space(text())='Location']/preceding-sibling::input[#type='radio']"));
WebElement name = driver.findElement(By.xpath("//label[normalize-space(text())='Name']/preceding-sibling::input[#type='radio']"));
location.click();
Thread.sleep(3000);
name.click();
driver.switchTo().parentFrame();
Click working video - https://www.screencast.com/t/0Fsw7gGZ
As the Radio Buttons are inside input tags, instead of id use xpath locator as follows:
WebDriverWait wait6 = new WebDriverWait(driver, 10);
WebElement radio_name = wait6.until(ExpectedConditions.elementToBeClickable(By.xpath("//div[#id='byname']/input[#id='byname']")));
radio_name.click();
Let me know if this Answers your Question.

a error occur in remoteDriver function of RSelenium

I want to scrape data on dynamic web by the following code:
> URL<- "http://www.cbooo.cn/realtime"
> library(bitops)
> library(RCurl)
> library(XML)
> library(RSelenium)
> library(magrittr)
> checkForServer()
Warning message:
checkForServer is deprecated.
Users in future can find the function in file.path(find.package("RSelenium"), "example/serverUtils").
The sourcing/starting of a Selenium Server is a users responsiblity.
Options include manually starting a server see vignette("RSelenium-basics", package = "RSelenium")
and running a docker container see vignette("RSelenium-docker", package = "RSelenium")
> startServer()
$stop
function ()
{
tools::pskill(selPID)
}
<environment: 0x10991af0>
$getPID
function ()
{
return(selPID)
}
<environment: 0x10991af0>
Warning message:
startServer is deprecated.
Users in future can find the function in file.path(find.package("RSelenium"), "example/serverUtils").
The sourcing/starting of a Selenium Server is a users responsiblity.
Options include manually starting a server see vignette("RSelenium-basics", package = "RSelenium")
and running a docker container see vignette("RSelenium-docker", package = "RSelenium")
> remDrv <- remoteDriver()
> remDrv$browserName="Internet Explorer"
> remDrv$open()
[1] "Connecting to remote server"
Selenium message: The best matching driver provider org.openqa.selenium.ie.InternetExplorerDriver can't create a new driver instance for Capabilities [{nativeEvents=true, browserName=Internet Explorer, javascriptEnabled=true, version=, platform=ANY}]
Build info: version: '2.53.1', revision: 'a36b8b1', time: '2016-06-30 17:37:03'
System info: host: 'DESKTOP-J0D980N', ip: '10.36.17.76', os.name: 'Windows 10', os.arch: 'x86', os.version: '10.0', java.version: '1.8.0_77'
Driver info: driver.version: unknown
Error: Summary: UnknownError
Detail: An unknown server-side error occurred while processing the command.
class: org.openqa.selenium.WebDriverException
Further Details: run errorDetails method
There are the following questions which I can't solve:
1 checkForServer,startServer are deprecated.
2 connecting to remote server always fials,I don't konw how to set some argurment in this funciotn and what should be done
I hope to get a anwser as soon as possible ,thanks.
The author of RSelenium provides a solution as following (https://github.com/ropensci/RSelenium/issues/81):
From Firefox 48 on-wards the gecko driver/ marionette will be needed to run Firefox with Selenium.
If you have Firefox 48 you can run the gecko driver as follows:
Refer to the guidelines
https: //developer.mozilla.org/en-US/docs/Mozilla/QA/Marionette/WebDriver
Download the relevant gecko driver from
https: //github.com/mozilla/geckodriver/releases
Add it to your PATH or refer to the location when starting binary (see below)
# get beta selenium standalone
RSelenium::checkForServer(beta = TRUE)
# assume gecko driver is not in our path (assume windows and we downloaded to docs folder)
# if the driver is in your PATH the javaargs call is not needed
selServ <- RSelenium::startServer(javaargs = c("- Dwebdriver.gecko.driver=\"C:/Users/john/Documents/geckodriver.exe\""))
remDr <- remoteDriver(extraCapabilities = list(marionette = TRUE))
remDr$open()
....
....
remDr$close()
selServ$stop()
Well to produce a viable working solution I would use the old version of RSelenium and everything with this code.
if (!require("XML")) {
install.packages("XML",repos= 'https://cloud.r-project.org')
library("XML")
}
#XML is a dependency
if (!require("RSelenium")) {
install.packages("https://cran.r-project.org/src/contrib/Archive/RSelenium/RSelenium_1.3.5.tar.gz", repos=NULL, type="source", dependencies = TRUE)
library("RSelenium")
}
download.file('http://selenium-release.storage.googleapis.com/2.53/selenium-server-standalone-2.53.1.jar', destfile = "~/Documents/R/library/RSelenium/bin/selenium-server-standalone.jar")
#start server
system('java -jar "~/Documents/R/library/RSelenium/bin/selenium-server-standalone.jar"')
library(RSelenium)
checkForServer()
startServer()
That is not an optimal solution. But a working solution.

PhantomJSDriver (GhostDriver) can't get localStorage values using javascript

Using PhantomJsDriver (GhostDriver) for testing my web application.
Trying to access the local storage values using calls like:
(String) js.executeScript(String.format("return localStorage.getItem('%s');", key));
In webDriver initialisation I'm setting the capability:
DesiredCapabilities dCaps = new DesiredCapabilities()
dCaps.setJavascriptEnabled(true)
dCaps.setCapability("webStorageEnabled", true)
return new PhantomJSDriver(dCaps)
Receiving WebDriverException while trying to execution the JS above, from which it looks like the webStorageEnabled have been never set to true:
org.openqa.selenium.WebDriverException: {"errorMessage":"SECURITY_ERR:
DOM Exception 18","request":{"headers":{"Accept":"application/json,
image/png","Connection":"Keep-Alive","Content-Length":"98","Content-Type":"application/json;
charset=utf-8","Host":"localhost:20884"},"httpVersion":"1.1","method":"POST","post":"{\"args\":[],\"script\":\"return
localStorage.getItem('cartId_136d7735-13d7-8178-8abf-010beb62f8cf');\"}","url":"/execute","urlParsed":{"anchor":"","query":"","file":"execute","directory":"/","path":"/execute","relative":"/execute","port":"","host":"","password":"","user":"","userInfo":"","authority":"","protocol":"","source":"/execute","queryKey":{},"chunks":["execute"]},"urlOriginal":"/session/5f67ce30-310a-11e4-83c7-6d52cfce6b64/execute"}}
Command duration or timeout: 8 milliseconds Build info: version:
'2.39.0', revision: '14fa800511cc5d66d426e08b0b2ab926c7ed7398', time:
'2013-12-16 13:18:38'
Driver info: org.openqa.selenium.phantomjs.PhantomJSDriver
Capabilities [{platform=MAC, acceptSslCerts=false,
javascriptEnabled=true, browserName=phantomjs, rotatable=false,
driverVersion=1.1.0, locationContextEnabled=false, version=1.9.7,
cssSelectorsEnabled=true, databaseEnabled=false, handlesAlerts=false,
browserConnectionEnabled=false, webStorageEnabled=false,
nativeEvents=true, proxy={proxyType=direct},
applicationCacheEnabled=false, driverName=ghostdriver,
takesScreenshot=false}] at
sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
What could be the reason for the issues I experiencing? Should it be possible to set webStorageEnabled for PhantomJSDriver? If no, how can I work around this issue?
It looks like it's still not implemented:
https://github.com/detro/ghostdriver/blob/6c918e230cc98a7bce75e652c020c55cc3e0dd23/src/session.js#L65
https://github.com/detro/ghostdriver/issues/25
After some thinking I have managed to solve this issue.
In general, it's possible to access localStorage using using JavaScript. The issue described here happened because of something else.
Here is the flow I had:
1) Initialize WebDriver
2) Access localStorage
3) Doing driver.get(url)
The problem is, driver.get(url) must happen before accessing localStorage. That solved the issue.
Here is my LocalStorage driver implementation (in Scala), if you need one: LocalStorageDriver

Resources