‘platform’ import disappearing from selenium as a result of my script - selenium-webdriver

Linux ip-172-31-36-170 3.10.35-43.137.amzn1.x86_64 #1 SMP Wed Apr 2 09:36:59 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
Amazon Linux AMI release 2014.03
cpe:/o:amazon:linux:2014.03:ga
I've run into a weird problem with a script that uses selenium.webdriver.PhantomJS.
SYMPTOMS…
My script uses the following to start a phantomjs session
from selenium import selenium
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.common.action_chains import ActionChains
(…)
def load_driver(self, _driver = "phantomjs", _path = "./phantomjs"):
if "phantom" in str(_driver).lower():
self.driver = webdriver.PhantomJS(_path)
Fails with:
selenium.common.exceptions.WebDriverException: Message: 'Unable to start phantomjs with ghostdriver.' ; Screenshot: available via screen
HOWEVER, at the python command line, everything works fine…
from selenium import webdriver
_path = './phantomjs.exe'
driver = webdriver.PhantomJS(_path)
PLATFORM: Linux
platform.system() != 'Windows': True
(I’ll explain “PLATFORM: Linux”, and “platform.system() != 'Windows': True” below)
SO…
I traced the error to the “/usr/lib/python2.6/site-packages/selenium/webdriver/phantomjs/service.py” and specifically this code…
def start(self):
"""
"""
try:
print "PLATFORM:", platform.system() #ADDED BY ME
# FOLLOWING ADDED BY ME, NOW CAUSING THE ERROR
print "platform.system() != 'Windows': ", platform.system() != 'Windows'
self.process = subprocess.Popen(self.service_args, stdin=subprocess.PIPE,
close_fds=platform.system() != 'Windows', # <-- THIS CAUSED ORIG PROB
stdout=self._log, stderr=self._log)
except Exception as e:
raise WebDriverException("Unable to start phantomjs with ghostdriver.", e)
AND HERE’S THE ISSUE…
When I run the code at the python command line (see above) everything is fine, AND the responses to ‘platform.system()’ and “platform.system() != 'Windows' ” are correct.
However, when I run my script, the ‘platform.system()’ reports blank and “platform.system() != 'Windows' ” errors. (See actual output below).
So, for some reason, when my script loads selenium and runs…THE SELENIUM CODE loses the ‘platform’ import.
Thanks for the help!
ACTUAL OUTPUT BELOW (NOTICE that 'PLATFORM:' is followed by 'blank' (instead of 'Linux'), and the next line “platform.system() != 'Windows' ” triggers the error. These lines were added by me into the SELENIUM code, not my code!)
PLATFORM:
Traceback (most recent call last):
File "./agmarknet.py", line 834, in <module>
username = options.username # --username
File "./agmarknet.py", line 124, in __init__
self.load_driver(driver, driver_path)
File "./agmarknet.py", line 521, in load_driver
self.driver = webdriver.PhantomJS(_path)
File "/usr/lib/python2.6/site-packages/selenium/webdriver/phantomjs/webdriver.py", line 50, in __init__
self.service.start()
File "/usr/lib/python2.6/site-packages/selenium/webdriver/phantomjs/service.py", line 64, in start
print "PLATFORM:", platform.system() #333
File "/usr/lib64/python2.6/platform.py", line 1272, in system
return uname()[0]
File "/usr/lib64/python2.6/platform.py", line 1239, in uname
processor = _syscmd_uname('-p','')
File "/usr/lib64/python2.6/platform.py", line 995, in _syscmd_uname
output = string.strip(f.read())
File "./agmarknet.py", line 350, in _signal_handler
self._cleanup()
File "./agmarknet.py", line 194, in _cleanup
self.driver.close()
AttributeError: 'Agmarknet' object has no attribute 'driver'
2014-05-18 12:42:40,281 - Agmarknet - INFO - Closing WebDriver...
Error in atexit._run_exitfuncs:
Traceback (most recent call last):
File "/usr/lib64/python2.6/atexit.py", line 24, in _run_exitfuncs
func(*targs, **kargs)
File "./agmarknet.py", line 194, in _cleanup
self.driver.close()
AttributeError: 'Agmarknet' object has no attribute 'driver'
Error in sys.exitfunc:
Traceback (most recent call last):
File "/usr/lib64/python2.6/atexit.py", line 24, in _run_exitfuncs
func(*targs, **kargs)
File "./agmarknet.py", line 194, in _cleanup
self.driver.close()

Related

S3 permission error when running sagemaker python sdk sklearn in local mode

I created a training script with hard coded input. It works as expected using a training job but I couldn't make it work using local mode.
It brings up a container on my local docker and exits with code (1)
Code:
estimator = SKLearn(entry_point="train_model.py",
train_instance_type="local")
estimator.fit()
Here is the exception:
2020-02-22 06:21:05,470 sagemaker-containers INFO Imported framework sagemaker_sklearn_container.training
2020-02-22 06:21:05,480 sagemaker-containers INFO No GPUs detected (normal if no gpus installed)
2020-02-22 06:21:05,504 sagemaker_sklearn_container.training INFO Invoking user training script.
2020-02-22 06:21:06,407 sagemaker-containers ERROR Reporting training FAILURE
2020-02-22 06:21:06,407 sagemaker-containers ERROR framework error:
Traceback (most recent call last):
File "/miniconda3/lib/python3.7/site-packages/sagemaker_containers/_trainer.py", line 81, in train
entrypoint()
File "/miniconda3/lib/python3.7/site-packages/sagemaker_sklearn_container/training.py", line 36, in main
train(framework.training_env())
File "/miniconda3/lib/python3.7/site-packages/sagemaker_sklearn_container/training.py", line 32, in train
training_environment.to_env_vars(), training_environment.module_name)
File "/miniconda3/lib/python3.7/site-packages/sagemaker_containers/_modules.py", line 301, in run_module
_files.download_and_extract(uri, _env.code_dir)
File "/miniconda3/lib/python3.7/site-packages/sagemaker_containers/_files.py", line 129, in download_and_extract
s3_download(uri, dst)
File "/miniconda3/lib/python3.7/site-packages/sagemaker_containers/_files.py", line 164, in s3_download
s3.Bucket(bucket).download_file(key, dst)
File "/miniconda3/lib/python3.7/site-packages/boto3/s3/inject.py", line 246, in bucket_download_file
ExtraArgs=ExtraArgs, Callback=Callback, Config=Config)
File "/miniconda3/lib/python3.7/site-packages/boto3/s3/inject.py", line 172, in download_file
extra_args=ExtraArgs, callback=Callback)
File "/miniconda3/lib/python3.7/site-packages/boto3/s3/transfer.py", line 307, in download_file
future.result()
File "/miniconda3/lib/python3.7/site-packages/s3transfer/futures.py", line 106, in result
return self._coordinator.result()
File "/miniconda3/lib/python3.7/site-packages/s3transfer/futures.py", line 265, in result
raise self._exception
File "/miniconda3/lib/python3.7/site-packages/s3transfer/tasks.py", line 255, in _main
self._submit(transfer_future=transfer_future, **kwargs)
File "/miniconda3/lib/python3.7/site-packages/s3transfer/download.py", line 345, in _submit
**transfer_future.meta.call_args.extra_args
File "/miniconda3/lib/python3.7/site-packages/botocore/client.py", line 357, in _api_call
return self._make_api_call(operation_name, kwargs)
File "/miniconda3/lib/python3.7/site-packages/botocore/client.py", line 661, in _make_api_call
raise error_class(parsed_response, operation_name)
botocore.exceptions.ClientError: An error occurred (403) when calling the HeadObject operation: Forbidden
An error occurred (403) when calling the HeadObject operation: Forbidden
tmpe_msr8pi_algo-1-kt1vh_1 exited with code 1
I found out that docker restart solved the issue.
After a while it happened again - and it solved it again.
I'm using docker for windows, and the issue is probably related to the created container configuration

Executable path error while using chrome driver with selenium

I am trying to open chrome using selenium but getting this error . I have tried many ways but still this issue is not resolved.
Traceback (most recent call last):
File "sl.py", line 11, in <module>
chrome.Initialize()
File "/home/daffolap/api_test/api_test/utils.py", line 14, in Initialize
driver = webdriver.Chrome(executable_path=r"/home/daffolap/api_test/chromedriver2.exe")
File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/chrome/webdriver.py", line 73, in __init__
self.service.start()
File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/common/service.py", line 83, in start
os.path.basename(self.path), self.start_error_message)
selenium.common.exceptions.WebDriverException: Message: 'chromedriver`enter code here`2.exe' executable needs to be in PATH. Please see https://sites.google.com/a/chromium.org/chromedriver/home
driver=None
class Webbrowser:
#staticmethod
def Initialize():
global driver
driver = webdriver.Chrome(executable_path=r"/home/daffolap/api_test/chromedriver2.exe")
driver.maximize_window()
driver.implicitly_wait(5)
return driver

PloneIDE intallation Error

Running buildout. This might take a while...
While:
Installing ploneide.
An internal error occurred due to a bug in either zc.buildout or in a
recipe being used:
Traceback (most recent call last):
File "/Plone/buildout-cache/eggs/zc.buildout-2.2.1-py2.7.egg/zc/buildout/buildout.py", line 1942, in main
getattr(buildout, command)(args)
File "/Plone/buildout-cache/eggs/zc.buildout-2.2.1-py2.7.egg/zc/buildout/buildout.py", line 622, in install
installed_files = self[part]._call(recipe.install)
File "/Plone/buildout-cache/eggs/zc.buildout-2.2.1-py2.7.egg/zc/buildout/buildout.py", line 1366, in _call
return f()
File "/Plone/zinstance/src/collective.recipe.ploneide/collective/recipe/ploneide/__init__.py", line 200, in install
self.install_developer_manual()
File "/Plone/zinstance/src/collective.recipe.ploneide/collective/recipe/ploneide/__init__.py", line 107, in install_developer_manual
res = subprocess.Popen(cmd, **kwargs)
File "/usr/lib/python2.7/subprocess.py", line 710, in __init__
errread, errwrite)
File "/usr/lib/python2.7/subprocess.py", line 1327, in _execute_child
raise child_exception
OSError: [Errno 2] No such file or directory
Buildout.cfg
develop =
src/collective.ploneide
src/collective.recipe.ploneide
parts=
ploneide
[Instance]
[ploneide]
recipe = collective.recipe.ploneide
ploneide may not be super well maintained at this point - last commit from 2012 https://github.com/collective/collective.ploneide
It would help if you posted your buildout.cfg and mentioned which OS you're using.

Nose GAE : Cannot import dev_appserver, but app engine is still in PYTHONPATH

I am getting the following error when trying to run the nosetest from my GAE project:
nosetests --nologcapture --with-gae --without-sandbox --gae-lib-root=/Users/dsinha/Dropbox/code/google-cloud-sdk/platform/google_appengine
but I get the following error:
Traceback (most recent call last):
File "/usr/local/bin/nosetests", line 8, in <module>
load_entry_point('nose==1.3.4', 'console_scripts', 'nosetests')()
File "/Library/Python/2.7/site-packages/nose/core.py", line 121, in __init__
**extra_args)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/unittest/main.py", line 94, in __init__
self.parseArgs(argv)
File "/Library/Python/2.7/site-packages/nose/core.py", line 145, in parseArgs
self.config.configure(argv, doc=self.usage())
File "/Library/Python/2.7/site-packages/nose/config.py", line 346, in configure
self.plugins.configure(options, self)
File "/Library/Python/2.7/site-packages/nose/plugins/manager.py", line 284, in configure
cfg(options, config)
File "/Library/Python/2.7/site-packages/nose/plugins/manager.py", line 99, in __call__
return self.call(*arg, **kw)
File "/Library/Python/2.7/site-packages/nose/plugins/manager.py", line 167, in simple
result = meth(*arg, **kw)
File "/Library/Python/2.7/site-packages/nosegae.py", line 87, in configure
from google.appengine.tools import old_dev_appserver as dev_appserver
ImportError: cannot import name old_dev_appserver
The sys.path reads:
'/Users/dsinha/Downloads/eclipse/plugins/org.python.pydev_3.9.0.201411111611/pysrc', '/Users/dsinha/Dropbox/code/google-cloud-sdk/platform/google_appengine/lib/antlr3', '/Users/dsinha/Dropbox/code/google-cloud-sdk/platform/google_appengine/lib/fancy_urllib', '/Users/dsinha/Dropbox/code/google-cloud-sdk/platform/google_appengine/lib/ipaddr', '/Users/dsinha/Dropbox/code/google-cloud-sdk/platform/google_appengine/lib/yaml-3.10', '/Users/dsinha/Dropbox/code/google-cloud-sdk/platform/google_appengine/lib/rsa', '/Users/dsinha/Dropbox/code/google-cloud-sdk/platform/google_appengine/lib/pyasn1', '/Users/dsinha/Dropbox/code/google-cloud-sdk/platform/google_appengine/lib/pyasn1_modules', '/Users/dsinha/Dropbox/code/google-cloud-sdk/platform/google_appengine', '/Users/dsinha/Dropbox/code/google-cloud-sdk/platform/google_appengine/lib/simplejson', '/Users/dsinha/Dropbox/code/google-cloud-sdk/platform/google_appengine/lib/django-1.4', '/Users/dsinha/Dropbox/code/google-cloud-sdk/platform/goo...
so the app engine libraries should be getting pulled in. Actually its first failing to pull in dev_appserver, and then trying and failing to pull old_dev_appserver
The dir inside the appengine/lib file is:
bash-3.2$ ls
__init__.py appcfg.py backends_xml_parser.py dispatch_xml_parser.py handler_generator.py php_cli.py value_mixin.pyc
__init__.pyc appcfg_java.py boolean_action.py docker handler_generator.pyc queue_xml_parser.py web_xml_parser.py
adaptive_thread_pool.py appengine_rpc.py boolean_action.pyc dos_xml_parser.py jarfile.py queue_xml_parser.pyc web_xml_parser.pyc
api_server.py appengine_rpc.pyc bulkload_client.py download_appstats.py java_quickstart.py remote_api_shell.py xml_parser_utils.py
app_engine_config_exception.py appengine_rpc_httplib2.py bulkloader.py endpointscfg.py java_quickstart.pyc requeue.py xml_parser_utils.pyc
app_engine_config_exception.pyc augment_mimetypes.py cron_xml_parser.py gen_protorpc.py java_utils.py sdk_update_checker.py yaml_translator.py
app_engine_web_xml_parser.py augment_mimetypes.pyc dev-channel-js.js handler.py java_utils.pyc sdk_update_checker.pyc yaml_translator.pyc
app_engine_web_xml_parser.pyc backends_conversion.py devappserver2 handler.pyc os_compat.py value_mixin.py
bash-3.2$ pwd
/Users/dsinha/Dropbox/code/google-cloud-sdk/platform/google_appengine/google/appengine/tools
I also tried to find the modules available inside the google.appengine.tools package:
>>> import pkgutil
>>> [name for _, name, _ in pkgutil.iter_modules(['testpkg'])]
[]
This problem started occuring after I upgraded to App Engine 1.9.10 (to use the async search features). In a problem that I think is related, when I try to run the debug server from PyDev, it just silently terminated on any page request (localhost:8080).
Running dev_appserver . from the command line works fine though.
Nose-GAE broke with App Engine 1.9.17: https://github.com/Trii/NoseGAE/issues/6
Downgrading to 1.9.15 made the problem go away temporarily while waiting for the issue to be resolved by nose-gae

Exception in idle (python 2.7) - possible bug in idle?

I'm trying to run a meta-analysis on a database of fMRI data, using the neurosynth python library through idle. When I try to run even some of the most basic functions, I get an error, not an error my own code, or in the neurosynth modules, the error seems to be a bug in idle itself.
I uninstalled and reinstalled python 2.7, reinstalled neurosynth and its dependencies, and ran into the same error. I've pasted my code below, followed by the error message, which appears in the unix shell (not in the idle shell).
Has anybody come across this error before using idle and python 2.7?
The script:
from neurosynth.base.dataset import Dataset
from neurosynth.analysis import meta, decode, network
import neurosynth
neurosynth.set_logging_level('info')
dataset = Dataset('data/database.txt')
dataset.add_features('data/features.txt')
dataset.save('dataset.pkl')
print 'done'
The error message which appeared in the unix shell:
----------------------------------------
Unhandled server exception!
Thread: SockThread
Client Address: ('127.0.0.1', 46779)
Request: <socket._socketobject object at 0xcb8d7c0>
Traceback (most recent call last):
File "/usr/global/python/2.7.3/lib/python2.7/SocketServer.py", line 284, in _handle_request_noblock
self.process_request(request, client_address)
File "/usr/global/python/2.7.3/lib/python2.7/SocketServer.py", line 310, in process_request
self.finish_request(request, client_address)
File "/usr/global/python/2.7.3/lib/python2.7/SocketServer.py", line 323, in finish_request
self.RequestHandlerClass(request, client_address, self)
File "/usr/global/python/2.7.3/lib/python2.7/idlelib/rpc.py", line 503, in __init__
SocketServer.BaseRequestHandler.__init__(self, sock, addr, svr)
File "/usr/global/python/2.7.3/lib/python2.7/SocketServer.py", line 638, in __init__
self.handle()
File "/usr/global/python/2.7.3/lib/python2.7/idlelib/run.py", line 265, in handle
rpc.RPCHandler.getresponse(self, myseq=None, wait=0.05)
File "/usr/global/python/2.7.3/lib/python2.7/idlelib/rpc.py", line 280, in getresponse
response = self._getresponse(myseq, wait)
File "/usr/global/python/2.7.3/lib/python2.7/idlelib/rpc.py", line 300, in _getresponse
response = self.pollresponse(myseq, wait)
File "/usr/global/python/2.7.3/lib/python2.7/idlelib/rpc.py", line 424, in pollresponse
message = self.pollmessage(wait)
File "/usr/global/python/2.7.3/lib/python2.7/idlelib/rpc.py", line 376, in pollmessage
packet = self.pollpacket(wait)
File "/usr/global/python/2.7.3/lib/python2.7/idlelib/rpc.py", line 347, in pollpacket
r, w, x = select.select([self.sock.fileno()], [], [], wait)
error: (4, 'Interrupted system call')
*** Unrecoverable, server exiting!
----------------------------------------
Thanks in advance!
Idle is meant for interactive exploration in the shell, for editing in an editor, and for testing programs by running them from an editor. It is not meant for production running of programs once developed. If there is a problem, one should separate the Idle part from the running with Python part. So in the unix shell, run python -m idlelib (for instance) to see if Idle starts correctly. Then, in an appropriate directory, run python path-to-my-file.py. Which does not work?
The error message is definitely odd, as it has more than just the python traceback. On the other hand, it does not start with a line of your code. I have no idea why the select call would be interrupted.

Resources