this code ran for last version I used in July:
fprof <- getFirefoxProfile("c:/Users/cp/AppData/Roaming/Mozilla/Firefox/Profiles/j7a5lq0r.selenium", useBase = T)
rS <- rsDriver(browser = "firefox", port = 4567L, extraCapabilities = fprof)
but now the output is:
[1] "Connecting to remote server"
$`moz:profile
[1] "C:\\Users\\cp\\AppData\\Local\\Temp\\rust_mozprofile.7n9y6eVCqnpt"
so it creates some temporary profile not loading the set one. Anyone knows what's the issue? This profile is valid and is ok while start FF manually.
Related
I'm using Python 3.8 with the pytest-docker-compose plugin -- https://pypi.org/project/pytest-docker-compose/ . Does anyone know how to write a URL that would eventually tell me if my SQL Server is running?
I have this docker-compose.yml file
version: "3.2"
services:
sql-server-db:
build: ./
container_name: sql-server-db
image: microsoft/mssql-server-linux:2017-latest
ports:
- "1433:1433"
environment:
SA_PASSWORD: "password"
ACCEPT_EULA: "Y"
but I don't know what URL to pass to my Retry object to test that the server is running. This fails ...
import pytest
import requests
from urllib3.util.retry import Retry
from requests.adapters import HTTPAdapter
...
#pytest.fixture(scope="function")
def wait_for_api(function_scoped_container_getter):
"""Wait for sql server to become responsive"""
request_session = requests.Session()
retries = Retry(total=5,
backoff_factor=0.1,
status_forcelist=[500, 502, 503, 504])
request_session.mount('http://', HTTPAdapter(max_retries=retries))
service = function_scoped_container_getter.get("sql-server-db").network_info[0]
api_url = "http://%s:%s/" % (service.hostname, service.host_port)
assert request_session.get(api_url)
return request_session, api_url
with this exception
raise ConnectionError(e, request=request)
E requests.exceptions.ConnectionError: HTTPConnectionPool(host='0.0.0.0', port=1433): Max retries exceeded with url: / (Caused by ProtocolError('Connection aborted.', RemoteDisconnected('Remote end closed connection without response')))
if connection.is_connected():
db_Info = connection.get_server_info()
print("Connected to MySQL Server version ", db_Info)
cursor = connection.cursor()
cursor.execute("select database();")
record = cursor.fetchone()
print("You're connected to database: ", record)
you could use something like this and it would output if it was connected
Here is a sample function that will retry to connect to the DB and won't return until it has successfully connected or the defined maxRetries is reached:
def waitDb(server, database, username, password, maxAttempts, waitBetweenAttemptsSeconds):
"""
Returns True if the connection is successfully established before the maxAttempts number is reached
Conversely returns False
pyodbc.connect has a built-in timeout. Use a waitBetweenAttemptsSeconds greater than zero to add a delay on top of this timeout
"""
for attemptNumber in range(maxAttempts):
cnxn = None
try:
cnxn = pyodbc.connect('DRIVER={ODBC Driver 17 for SQL Server};SERVER='+server+';DATABASE='+database+';UID='+username+';PWD='+ password)
cursor = cnxn.cursor()
except Exception as e:
print(traceback.format_exc())
finally:
if cnxn:
print("The DB is up and running: ")
return True
else:
print("DB not running yet on attempt numer " + str(attemptNumber))
time.sleep(waitBetweenAttemptsSeconds)
print("Max attempts waiting for DB to come online exceeded")
return False
I wrote a minimal example here: https://github.com/claudiumocanu/flow-pytest-docker-compose-wait-mssql.
I included the three actions that can be executed independently, but you can jump to the last step specifically for what you asked:
1. Connected from python to the mssql launched by the compose-file:
For me it was quite annoying to find and install the appropriate ODBC Driver and its dependencies - ODBC Driver 17 for SQL Server worked the best for me on Ubuntu 18.
To perform only this step, docker-compose up the the docker-compose.yml in my example, then run the example-connect.py
2. Created a function that attempts to connect to the DB with a maxAttemptsNumber and a delay between retries:
Just run this example-waitDb.py. You can play with the maxAttempts and the delayBetweenAttempts values, then bring up the database at randomly, to test it.
3. Put everything together in the test_db.py test suite:
the waitDb function described above.
same wrapper and annotations that you provided in your example to spin-up the resources defined in the compose-file
a dummy integration test that will not be executed before waitDb returns (if you want to block this tests completely, you can throw instead of returning False from the waitDb function)
PS: Keep using ENVs/vault etc rather than storing the real passwords like I did for the dummy example.
I'm trying to get trace logging information out of the gecko driver, in order to debug some issues we're having with moving to newer versions of Python, Robot Framework, Selenium, Firefox.
See in the DEBUG robot output below, I'm setting the capabilities to include log level arguments as part of moz:firefoxOptions.
When the capabilities are printed out for Open Browser, the moz:firefoxOptions value has been replaced with the profile info, so I never get tracing set.
20190219 13:07:22.461 - INFO - {'browserName': 'firefox', 'marionette': True, 'acceptInsecureCerts': True, 'moz:firefoxOptions': {'prefs': {'log': {'level': 'trace'}}, 'args': ['-headless', '-profile', '/tmp/tmpimda__f2']}, 'binary': '/usr/bin/firefox'}
20190219 13:07:22.461 - INFO - +--- END KW: BuiltIn.Log (1)
20190219 13:07:22.461 - INFO - +--- START KW: CustomLib.AdminClass.Open Browser [ ${LOGIN URL} | ${BROWSER} | desired_capabilities=${dc} | ff_profile_dir=${profile} ]
20190219 13:07:22.462 - INFO - Opening browser 'Firefox' to base url 'https://HOST:PORT/bmmadmin/login.jsp'.
20190219 13:07:22.480 - DEBUG - POST http://127.0.0.1:38132/session {"capabilities": {"firstMatch": [{}], "alwaysMatch": {"browserName": "firefox", "acceptInsecureCerts": true, "moz:firefoxOptions": {"profile": "UEsDBBQAAAAIAOtoU04+3NKsfgMAAM0MAAAHAAAAdXNlci5qc6VWTW/UMBC98ytQTyCx1rbABU6lFAkJUcSq4mg59mTjrmMbe7zp/nvGyaYsbb6AWz7e89hvZt44RQjcByhfnBXBNfTGlKtZo61yDVOp9gysKAyos1fPMSR4+f5ZesoxAiGEZE/ApTBxBG2hQVGwFAzhzkThEr4rjLC7sym8F1tYtn5EETB5lhmEXE+jKlfDEblgL49Z3O3p5FoBqyM6266it9YFGFmAPqkb+6MVOJ6vmbYY3O33L9Phc1Jqcc9lFSgwjzJoj5wU56jrHPT1epiwDBnAu4BcGMPvIod7CR61s3Ew6XdiL7plmetgLFau+Wyv6NEZGCRZwMaFHZOCKHtY5XjCrAiw13ImrxFkChoPTMa+HEeAj5PTgJFZr9lSm2IyoZTO5xQzizRQqECHC1zIrCBPlNsUERSXEHBYzRNSjClHfCBp+gBhhtXpwa3IsnLYgx0JJLxnySvqVCYSumEFT0CLWk3p6I04gLo+X2++OqRcEuHNCNg11jihqDAtaRzasvlRgd1k9bXdTse6vv1yyfp2m7SjFvmadVmYsy5td1THYKnsycS6JF/MYcmSOo8cB7uyJPxIpfagAEJlZQBB0k4/6RDxkrSQI230YEOihPaZVFtoiqeMWphGhKV2CiLI6lgWc1BTxuRzc/f2PuajEGO2DiSnJBXayi/J27gMIlYzUSowhskK5O4jlCIZ/ND9mabRDImMjm1v7JVxceYoJ+gbyvgwmBQRnXdmVSsQBqvunS1ytgl+8rlVrv+R3Ucvc0F9T3+/feO22/xRdp7+F/vwzmh5YPnjJhW1bvP8v/xv7cfL3w09tAzs0TkTGTmEC8eNT15f8ugjB2sNtG1rRXPGbnlWfjhG9gjbFm5roh878kYSO7vu+XoUXhgnd0ZHnK6IE0Zb4Feu9mTthTZ5AFq9rdAcZqlLLPwp3JJ/l4fb0UY6GfxTqvajvnP51dEHVzRiMcXhlXtKhUjXtkrH6rDKvz3NxJUBu8VsCRdv3/5J65emqUX5MMY1vDhw1XnC4N48WYzHmLub104Js/DmUWoD9MRc0NQWvKvRfJmawUcMWiJ/TBsKlW+NlqywhrqAsGlfR+TKZb7TyI6y9eP0WMqDB+8pCIYiYDiwTor25Bdz2EmRnsID3LXjbHAn916T7VO2WD4y1QS5PzF+Jh3GIvymNJWmMLmLBhi/AFBLAQIUAxQAAAAIAOtoU04+3NKsfgMAAM0MAAAHAAAAAAAAAAAAAACkgQAAAAB1c2VyLmpzUEsFBgAAAAABAAEANQAAAKMDAAAAAA=="}}}, "desiredCapabilities": {"browserName": "firefox", "acceptInsecureCerts": true, "moz:firefoxOptions": {"profile": "UEsDBBQAAAAIAOtoU04+3NKsfgMAAM0MAAAHAAAAdXNlci5qc6VWTW/UMBC98ytQTyCx1rbABU6lFAkJUcSq4mg59mTjrmMbe7zp/nvGyaYsbb6AWz7e89hvZt44RQjcByhfnBXBNfTGlKtZo61yDVOp9gysKAyos1fPMSR4+f5ZesoxAiGEZE/ApTBxBG2hQVGwFAzhzkThEr4rjLC7sym8F1tYtn5EETB5lhmEXE+jKlfDEblgL49Z3O3p5FoBqyM6266it9YFGFmAPqkb+6MVOJ6vmbYY3O33L9Phc1Jqcc9lFSgwjzJoj5wU56jrHPT1epiwDBnAu4BcGMPvIod7CR61s3Ew6XdiL7plmetgLFau+Wyv6NEZGCRZwMaFHZOCKHtY5XjCrAiw13ImrxFkChoPTMa+HEeAj5PTgJFZr9lSm2IyoZTO5xQzizRQqECHC1zIrCBPlNsUERSXEHBYzRNSjClHfCBp+gBhhtXpwa3IsnLYgx0JJLxnySvqVCYSumEFT0CLWk3p6I04gLo+X2++OqRcEuHNCNg11jihqDAtaRzasvlRgd1k9bXdTse6vv1yyfp2m7SjFvmadVmYsy5td1THYKnsycS6JF/MYcmSOo8cB7uyJPxIpfagAEJlZQBB0k4/6RDxkrSQI230YEOihPaZVFtoiqeMWphGhKV2CiLI6lgWc1BTxuRzc/f2PuajEGO2DiSnJBXayi/J27gMIlYzUSowhskK5O4jlCIZ/ND9mabRDImMjm1v7JVxceYoJ+gbyvgwmBQRnXdmVSsQBqvunS1ytgl+8rlVrv+R3Ucvc0F9T3+/feO22/xRdp7+F/vwzmh5YPnjJhW1bvP8v/xv7cfL3w09tAzs0TkTGTmEC8eNT15f8ugjB2sNtG1rRXPGbnlWfjhG9gjbFm5roh878kYSO7vu+XoUXhgnd0ZHnK6IE0Zb4Feu9mTthTZ5AFq9rdAcZqlLLPwp3JJ/l4fb0UY6GfxTqvajvnP51dEHVzRiMcXhlXtKhUjXtkrH6rDKvz3NxJUBu8VsCRdv3/5J65emqUX5MMY1vDhw1XnC4N48WYzHmLub104Js/DmUWoD9MRc0NQWvKvRfJmawUcMWiJ/TBsKlW+NlqywhrqAsGlfR+TKZb7TyI6y9eP0WMqDB+8pCIYiYDiwTor25Bdz2EmRnsID3LXjbHAn916T7VO2WD4y1QS5PzF+Jh3GIvymNJWmMLmLBhi/AFBLAQIUAxQAAAAIAOtoU04+3NKsfgMAAM0MAAAHAAAAAAAAAAAAAACkgQAAAAB1c2VyLmpzUEsFBgAAAAABAAEANQAAAKMDAAAAAA=="}, "binary": "/usr/bin/firefox", "marionette": true}}
Anyone have any ideas what I may be doing wrong here? And/OR some alternative methods of getting trace logging from geckodriver?
BTW: the moz:firefoxOptions I'm setting may not be correct, they're just the latest combination I have tried, but I can't tell what is going to work until the options I set are actually being sent to firefox.
Environment:
Python 3.6.3
geckodriver 0.24.0 ( 2019-01-28)
Mozilla Firefox 60.4.0
PIP:
requests (2.21.0)
robotframework (3.1.1)
robotframework-requests (0.5.0)
robotframework-selenium2library (3.0.0)
robotframework-seleniumlibrary (3.3.1)
selenium (3.141.0)
setuptools (40.8.0)
urllib3 (1.24.1)
ROBOT:
Open Browser To Login Page
${dc}= CustomLib.Get Desired Capabilities ${BROWSER}
Set To Dictionary ${dc}
... binary ${PATH TO MARIONETTE}
Log ${dc}
${profile}= CustomLib.Create FF Profile
Open Browser ${LOGIN URL} ${BROWSER} desired_capabilities=${dc} ff_profile_dir=${profile}
PYTHON:
#keyword
def get_desired_capabilities(browser):
_capabilities = {'ff' : DesiredCapabilities.FIREFOX,
'firefox' : DesiredCapabilities.FIREFOX
}
browserKey = str(browser).lower()
desired = _capabilities[browserKey]
BuiltIn().log("get_desired_capabilities B4 :" + str(desired))
#THIS IS ASSUMING FIREFOX
desired['marionette'] = True
desired['acceptSslCerts'] = True
desired['ignoreProtectedModeSettings'] = True
desired['headless'] = True
desired['moz:webdriverClick'] = True
desired['moz:firefoxOptions'] = {'log':{'level':'trace'}, 'args':['-headless']}
#desired['moz:firefoxOptions']['log'] = {'level':'trace'}
#desired['moz:firefoxOptions']['args'] = ['-headless']
BuiltIn().log("get_desired_capabilities returning :" + str(desired))
return desired
#keyword
def create_ff_download_profile(path):
from selenium import webdriver
BuiltIn().log('PATH: ' + path)
fp = webdriver.FirefoxProfile()
fp.set_preference("browser.download.folderList", 2)
fp.set_preference("browser.download.manager.showWhenStarting", False)
fp.set_preference("browser.helperApps.alwaysAsk.force", False);
fp.set_preference("browser.helperApps.neverAsk.saveToDisk", 'text/xml,text/plain,application/octet-stream')
fp.set_preference("browser.download.dir", path)
fp.update_preferences()
return fp.path
thanks,
Tom
This is how I set the log level to be trace in Firefox:
*** Settings ***
Library SeleniumLibrary
*** Keywords ***
${options}= Evaluate sys.modules['selenium.webdriver.firefox.options'].Options() sys
${options.log.level}= Set Variable trace
${driver}= Create Webdriver Firefox options=${options}
Go To https://www.google.com
It's basically following this python pattern from the firefox docs and translating it into Robotframework
Hope this helps!
I've tried following the guide at http://gatling.io/docs/2.2.3/realtime_monitoring/index.html to log my test results to influxdb and display the data in a grafana that I have previously set up. However I can't see any of the data that gatling is supposed to log anywhere in influxdb.
I've edited by influxdb.conf file so that it contains the following fields:
[[graphite]]
enabled = true
database = "gatlingdb"
bind-address = ":2003"
protocol = "tcp"
consistency-level = "one"
name-separator = "."
templates = [
"gatling.*.*.*.count measurement.simulation.request.status.field",
"gatling.*.*.*.min measurement.simulation.request.status.field",
"gatling.*.*.*.max measurement.simulation.request.status.field",
"gatling.*.*.*.percentiles50 measurement.simulation.request.status.field",
"gatling.*.*.*.percentiles75 measurement.simulation.request.status.field",
"gatling.*.*.*.percentiles95 measurement.simulation.request.status.field",
"gatling.*.*.*.percentiles99 measurement.simulation.request.status.field"
]
and my gatling.conf file contains the following fields:
data {
writers = [console, file, graphite] # The list of DataWriters to which Gatling write simulation data (currently supported : console, file, graphite, jdbc)
console {
#light = false # When set to true, displays a light version without detailed request stats
}
graphite {
#light = false # only send the all* stats
host = "127.0.0.1" # The host where the Carbon server is located
port = 2003 # The port to which the Carbon server listens to (2003 is default for plaintext, 2004 is default for pickle)
protocol = "tcp" # The protocol used to send data to Carbon (currently supported : "tcp", "udp")
rootPathPrefix = "gatling" # The common prefix of all metrics sent to Graphite
#bufferSize = 8192 # GraphiteDataWriter's internal data buffer size, in bytes
#writeInterval = 1 # GraphiteDataWriter's write interval, in seconds
}
Whenever i run my gatling tests I see no error messages or anything that indicates that anything is wrong, but I cannot see anything in the influxd logs that indicates that anything has been logged to influxdb, nor can I see any data in the gatlingdb database. I am using influxdb v0.10 and gatling v2.2.3 on Ubuntu
Can anyone help me figure out what I am doing wrong?
Updated to influxdb v1.1 and the problem seemed to have resolved itself from doing that
I'm trying to setup Xdebug because I'm tired of using echoes and var_dumps.
I'm using Atom as my IDE with the php-debug plugin.
I'm using the laravel Homestead vm as a server. forwarded port 9000 to 9999
Atom has the plugin installed with the following settings:
I set a breakpoint in Atom, I browse to my page but the breakpoint doesn't trigger.
I have a hunch that it has something to do with the IDE key but I don't know how to proceed.
Does anyone know the IDE key for atom?
Or is there something else I'm missing?
edit: I've added the following to xdebug.ini:
xdebug.auto_trace = 0
xdebug.collect_includes = 1
xdebug.collect_params = 1
xdebug.collect_return = 0
xdebug.collect_vars = "Off"
xdebug.default_enable = "On"
xdebug.dump.COOKIE = ""
xdebug.dump.FILES = ""
xdebug.dump.GET = ""
xdebug.dump.POST = ""
xdebug.dump.REQUEST = ""
xdebug.dump.SERVER = ""
xdebug.dump.SESSION = ""
xdebug.dump_globals = 1
xdebug.dump_once = 1
xdebug.dump_undefined = 0
xdebug.extended_info = 1
xdebug.file_link_format = ""
xdebug.idekey = "VVVDEBUG"
xdebug.manual_url = "http://www.php.net"
xdebug.max_nesting_level = 100
xdebug.overload_var_dump = 1
xdebug.profiler_append = 0
xdebug.profiler_enable = 0
xdebug.profiler_enable_trigger = 1
xdebug.profiler_output_dir = "/tmp"
xdebug.profiler_output_name = "cachegrind.out.%t-%s"
xdebug.remote_autostart = 1
xdebug.remote_enable = 1
xdebug.remote_handler = "dbgp"
xdebug.remote_host = "192.168.50.1"
xdebug.remote_log = /srv/log/xdebug-remote.log
xdebug.remote_mode = "req"
xdebug.remote_port = 9000
xdebug.show_exception_trace = 0
xdebug.show_local_vars = 0
xdebug.show_mem_delta = 0
xdebug.trace_format = 0
xdebug.trace_options = 0
xdebug.trace_output_dir = "/tmp"
xdebug.trace_output_name = "trace.%c"
xdebug.var_display_max_children = -1
xdebug.var_display_max_data = -1
xdebug.var_display_max_depth = -1
Like #sparkos72 says, the atom ide key xdebug.atom works for me on Ubuntu 16.04 and Debian 7 :-). I try to extend their answer.
I used php-debug atom extension with this config in the xdebug.ini (path: /etc/php5/apache2/conf.d/xdebug.ini).
xdebug.remote_enable=1
xdebug.remote_handler=dbgp
xdebug.remote_mode=req
xdebug.remote_host=172.17.0.1
xdebug.remote_port=9000
xdebug.idekey=xdebug.atom
xdebug.remote_autostart=true
I have a docker container with debian 7 + apache + php 5, that's the reason because I use ip 172.17.0.1 instead of 127.0.0.1. My "real" machine is an Ubuntu.
Hope it helps.
IDE key is xdebug.atom for me (mac/apache). In the Atom php-debug config, set the server port to 9000
None of the answers worked for me, so I'm leaving one of my own.
Setup
Make sure xdebug is enabled; sudo phpenmod xdebug or equivalent
Make sure xdebug is correctly configured:
xdebug.remote_enable=1
xdebug.remote_handler=dbgp
xdebug.remote_mode=req
xdebug.remote_host=172.17.0.1 (as per your setup)
xdebug.remote_port=9000 (as per your setup)
Didn't do xdebug.remote_autostart because it starts the debug stuff even for normal requests. So, even if you are not in the debug mode for the Chrome / Firefox extension, an attempt to convey debugging data will be made, which might slow things down without any reason.
Install Xdebug chrome/firefox extension and setup IDE Key as xdebug-atom (I found xdebug.atom to also be working).
Debugging
When you actually want to debug something,
Add a breakpoint where you want in your code; Alt + F9 or equivalent.
Open the PHP Debug panel in Atom; Ctrl + Alt + D or equivalent. If you are on certain versions Ubuntu, you might want to change the shortcut Ctrl + Alt + D might act as Show Desktop. In which case, click on the PHP Debug button in the IDE to toggle the debugger.
The debugger should say something like Listening to port 9000 or the port you setup in the PHP Debug settings.
Open the page you want to debug in your browser (if not already open) and click on the debugging extension icon to enable Debug mode. This actually sets a cookie in your document like XDEBUG_SESSION=xdebug-atom, etc.
Once the debug extension is enabled, refresh the page and you should be able to use the debugger (if everything went right).
Hope this helps. Took a while to get it working. Now I don't have to buy PHP Storm!
I know it is late but again:
Your .ini file have this:
xdebug.idekey = "VVVDEBUG"
which means you need to set the value "VVVDEBUG"(without quotes) in "The Easiest Xdebug" extension settings where you use "Atom" currently
IDE key is xdebug-atom for me on PHP3.2+ ATOM+ XAMPP+ XDEBUG Helper, strongly recommending to use XDEBUG Helper, you could set IDE key there.
I was trying the same thing, but never I found a real solution and I saw that xdebug started debug when find a error exeption I started put error in my code when I need debug and works well, example put this code in the line that you want debug.
#xdebug_start:
and watch the magic.
I did a video installing the plugin.
https://www.youtube.com/watch?v=jD0TIzYMFzQ
Using phantomjs page.evaluate to extract "resultStats" (div id) from http://www.google.com/search/?q=site:%s works on my local server but not on production server.
NOTE: I'm using the latest phantomjs 1.9.7, however I experienced the same issue with the previous version 1.9.6
NOTE: Phantomjs page.render (on Google home page as well as any other domain name) is working on both servers and creates nice screenshots.
On my production server (Debian stable 7.3 #linode.com) the PHP code below for a top level domain name as the "$url" returns:
TypeError: 'null' is not an object (evaluating 'document.getElementById('resultStats').textContent') phantomjs://webpage.evaluate():2 phantomjs://webpage.evaluate():3 phantomjs://webpage.evaluate():3 null
On my local server (debian testing) the PHP code below for the same "$url" returns:
About 43 results
This happens with any domain name/url I use as the argument - I've tested it on dozens.
What might cause this to occur in my remote production server and not my local server?
gsiteindex.js
var page = require('webpage').create(), site;
var site = phantom.args[0];
page.open("https://www.google.com/search?q=site:" + site, function (status) {
var result = page.evaluate(function () {
return document.getElementById('resultStats').textContent;
});
console.info(result);
phantom.exit();
});
.php
$phantomjs = "phantomjs";
$script = "gsiteindex.js";
$site = $url;
$command = "$phantomjs $script $site";
$googlestring = shell_exec($command);
echo $googlestring;
die();
Well, nrabinowitz was right. I tested it more on my own server using proxies, most timed out, some returned the above error, and a couple returned correct results (well I assume they were correct based on the location the IP address of the proxy - because the figures were a little different than using my ISPs public IP address (calif., USA)).
So it's simply a matter of google blocking certain types of requests from certain IP addresses.
Thanks again for the comment.
Incleude header with user-agent e.g.
header = {'user-asgent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64;
rv:68.0) Gecko/20100101 Firefox/68.0'}
Withuot user agent you get googles gefault style page without resultStats a also had this issue and adding header helped
Default google search page looks like this
enter image description here