I'm trying to do automated tests on a localhost site using the following settings:
Python 3.8.10
selenium 3.141.0
Firefox 90.0
Burp Suite Community Edition v2021.6.2
I'm using Burp proxy with the address 127.0.0.1:8080.
I tested several examples available here. The below code is the one that has worked best so far.
from selenium import webdriver
firefox_capabilities = webdriver.DesiredCapabilities.FIREFOX
firefox_capabilities['marionette'] = True
PROXY = "127.0.0.1:8080"
firefox_capabilities['proxy'] = {
"proxyType": "MANUAL",
"httpProxy": PROXY,
"sslProxy": PROXY
}
driver = webdriver.Firefox(capabilities=firefox_capabilities)
driver.get("http://127.0.0.1")
This code works fine when the url in driver.get("URL here") is not localhost. When I enter the url http://127.0.0.1, the access made by selenium does not appear in Burp Suite's HTTP history. Instead of the accessed url, "http://detectportal.firefox.com" appears.
Is this a problem in the code or some configuration that I need to do?
Burp HTTP History
Look into the Firefox proxy settings, there you can find a statement that localhost connections are never directed to a proxy.
But this can be changed by opening about:config and set the option network.proxy.allow_hijacking_localhost to true.
For Burp to track links on localhost just add ("network.proxy.allow_hijacking_localhost", True) to FirefoxProfile().
from selenium import webdriver
fp = webdriver.FirefoxProfile()
fp.set_preference("network.proxy.type", 1)
fp.set_preference("network.proxy.http", "127.0.0.1")
fp.set_preference("network.proxy.http_port", 8080)
fp.set_preference("network.proxy.allow_hijacking_localhost", True)
fp.update_preferences()
driver = webdriver.Firefox(firefox_profile=fp)
driver.get('http://127.0.0.1')
Related
I am running webdriver 2.53.1 against firefox 45.9.0ESR on Redhat Linux 6.6. FirefoxDriver object gets created successfully and firefox launches with blank page (about:blank) as expected. But when I do the 'get' to open the url, it fails with UnreachableBrowserException with underlying cause as org.apache.http.NoHttpResponseException: localhost:7055 failed to respond.
Preferences set for FirefoxProfile
app.update.auto = false
app.update.enabled = false
app.update.silent = false
media.gmp-provider.enabled = false
webdriver.log.file = webdriver_debug.log
webdriver.firefox.logfile = firefox_browser.log
My observations
Browser is running and is not killed
Browser has webdriver addon added.
By 'netstat' I see webdriver listening on port 7055
Though I configured to dump firefox and webdriver logs, nothing gets dumped.
What I tried so far
Handling the exception and retrying does not help
The firefox is a tar ball extract. I tried removing the folder and extracting again, but that did not help either.
Used navigate().to(url) instead of get(url) but result is same.
NOTE: The JRE 7 is used for running
Upgrading Selenium
We cannot upgrade the selenium or firefox as there are many other dependent layers to be upgraded for selenium/firefox to upgrade.
You need to update your jars files of selenium both server and client.
Download the latest jars from below link :-
http://www.seleniumhq.org/download/
You also need to update your gecko driver from below URL :-
https://github.com/mozilla/geckodriver/releases
Additionality update your firefox
Help -> About
I have an angular application on my local machine, I can open my SPA from webstorm over chrome, firefox or IE, but if I try to open the same html file from my windows explorer without using the IDE, it only opens and runs in firefox but nothing on chrome and IE just a blank page. I remember reading that firefox has an internal server or some sort, and chrome doesn't, I don't remember where I read that, if any body knows why please help.
Long term, you are best off running a local fileserver IMO. You can check out python's simple server example http://effbot.org/librarybook/simplehttpserver.htm, or I typically run something like this in node:
var http = require('http');
var express = require('express');
var app = express();
app.use('./', express.static(__dirname + './'));
app.get('*', function(req, res) {
return res.redirect('./');
});
// Create a server
var server = app.listen(3513, function () {
console.log('Server listening on', 3512)
});
If it is pure angular you are using, you don't need a server, however you might be running into some security problems, try running without security and see what happens.
For OSX, open Terminal and run:
$ open -a Google\ Chrome --args --disable-web-security
For Linux run:
$ google-chrome --disable-web-security
Also if you're trying to access local files for dev purposes like AJAX or JSON, you can use this flag too.
-–allow-file-access-from-files
For Windows go into the command prompt and go into the folder where Chrome.exe is and type
chrome.exe --disable-web-security
That should disable the same origin policy and allow you to access local files.
Otherwise taking a look at the console is the most sane thing to do, sometimes error information will pop-up there.
In my protractor config file, I had this line, seleniumAddress: 'http://localhost:4444/wd/hub'. On running Protractor I was getting an error "ECONNREFUSED connect ECONNREFUSED". After going through lot of other existing issues and solutions, I removed "seleniumAddress" property. That resolved the issue. Selenium standalone server gets started. "Selenium standalone server started at http://192.168.1.156:64477/wd/hub"
But when I turn on the VPN, then I get an error "Error: Timed out waiting for the WebDriver server at http://192.168.1.156:63199/wd/hub", which I have been not able to resolve.
I am on a HP laptop which has Windows 7 Professional and I am using Cisco VPN.
(Hi, so I can't comment yet (low rep)...)
Could you try running webdriver-manager start before running protractor? It will run in the address http://localhost:4444/wd/hub which is the seleniumAddress referred to in the protractor config. Does that change anything?
This might be related (VPN-workaround): protractor stand alone selenium fails: Error: Timed out waiting for the WebDriver server at
Check the settings of the firewall that stays between Selenium standalone server (which might run also on your local host) and your working station (usually your localhost).
In my case (running on local Linux station) I had a very restrictive iptables firewall rules such that the WebDriver process launched on localhost could not access the Selenium standalone server which also run on localhost at whatever TCP port.
Just try to turn it off and check if that is the case ; then accommodate your firewall settings such that the respective connection passes your firewall rules.
If you want your scripts communicate directly with the Firefox|Chrome Driver (bypassing the Selenium server entirely) then try adding the directConnect: true in your protractor.conf.js
Git and other tools, often use the git: protocol for accessing files
in remote repositories. Some firewall configurations are blocking
git:// URLs, which leads to errors when trying to clone repositories
or download dependencies. (For example corporate firewalls are
"notorious" for blocking git:.)
If you run into this issue, you can force the use of https: instead,
by running the following command: git config --global
url."https://".insteadOf git://
(see Common Issues on Angular tutorial)
I started playing with Seleniumlibrary tests (run with robotframework) and as our websites have ads and metrics and so on, every time I run a test those URLs get loaded.
Is there a way to tell selenium/robotframework to not load certain types of URLs or prevent it to load external resources (i.e. everything that is not from localhost).
You can do that with browsermob-proxy. Once you have that installed, you simply setup the proxy and set the blacklist.
ProxyServer server = new ProxyServer(9000)
server.start();
final DesiredCapabilities capabilities = new DesiredCapabilities();
capabilities.setCapability(CapabilityType.PROXY, server.seleniumProxy());
//Send a 200 for all requests to the facebook cdn
server.blacklistRequests("http://.*\\.fbcdn.net/.*", 200);
//Finish setting up your driver
WebDriver driver = new SomeDriverImpl(capabilities);
I believe the following will work with this python wrapper (the regex might be slightly different):
from browsermobproxy import Server
server = Server("path/to/browsermob-proxy")
server.start()
proxy = server.create_proxy()
proxy.blacklist('http://.*\\.fbcdn.net/.*', 200)
from selenium import webdriver
profile = webdriver.FirefoxProfile()
profile.set_proxy(proxy.selenium_proxy())
driver = webdriver.Firefox(firefox_profile=profile)
...
proxy.stop()
driver.quit()
This can now be done in two lines in Chrome:
driver.execute_cdp_cmd('Network.setBlockedURLs', {"urls": ["www.baidu.com"]})
driver.execute_cdp_cmd('Network.enable', {})
I am using selenium web driver (Not selenium RC). I need to download a xml file by clicking a link. I have did some Google search, and I have found in some answers an using AutoIT in order to handle OS related dialog boxes.
But is there any other option using selenium to handle this without using AutoIT tool.
Kindly suggest some ideas.
Can you be a bit more specific which browser you are using. If it's firefox you can have a greater control on the file download. Any other browser including firefox you can use robot class. This can be used to perform the click on ok button for downloads. If its chorme then file download happens automatically without any intervention.
For the newest version of Firefox (as of writing), these are the arguments that I needed to avoid the download box. Note that you need to specify a directory that you can write to as seen in the third statement:
FirefoxProfile profile = new FirefoxProfile();
profile.setPreference( "browser.download.folderList", 2 );
profile.setPreference( "browser.download.dir", <YOUR DOWNLOAD PATH> );
profile.setPreference( "plugin.disable_full_page_plugin_for_types", "application/pdf" );
profile.setPreference(
"browser.helperApps.neverAsk.saveToDisk",
"application/csv,text/csv,application/pdfss, application/excel" );
profile.setPreference( "browser.download.manager.showWhenStarting", false );
profile.setPreference( "pdfjs.disabled", true );
Note that the last line with pdfjs is required for the newer versions of Firefox, where it wasn't before. More info Here
I face Authentication Proxy pop-up issue in my project. So I tried below solution and it is working fine.
When we run Script from Selenium Web driver on Security Environment following Setup needs to be done to handle Authentication Proxy.
First you need to know below details,
network.proxy.autoconfig_url (Example: "http://example.com/abc.pac")
network.proxy.http (Example: abc-proxy.com)
network.proxy.http_port (Example: 8080)
private static WebDriver initFirefoxDriver(String appURL)
{
System.out.println("Launching Firefox browser..");
FirefoxProfile firefoxProfile = new FirefoxProfile();
firefoxProfile.setPreference("network.proxy.type", 1);
firefoxProfile.setPreference("network.proxy.autoconfig_url", "http://example.com/abc.pac");
firefoxProfile.setPreference("network.proxy.http", " abc-proxy.com");
firefoxProfile.setPreference("network.proxy.http_port", 8080);
WebDriver driver = new FirefoxDriver(firefoxProfile);
driver.manage().timeouts().implicitlyWait(30, TimeUnit.SECONDS);
driver.manage().window().maximize();
driver.navigate().to(appURL);
//driver.get(appURL);
return driver;
}