I am testing pyflink on
os: centos7
flink version: flink-1.14.3
virtualenv python version: Python 3.6.8
pip list:
apache-beam 2.27.0
apache-flink 1.14.3
apache-flink-libraries 1.14.3
avro-python3 1.9.2.1
certifi 2021.10.8
charset-normalizer 2.0.11
cloudpickle 1.2.2
crcmod 1.7
dill 0.3.1.1
docopt 0.6.2
fastavro 0.23.6
future 0.18.2
grpcio 1.43.0
hdfs 2.6.0
httplib2 0.17.4
idna 3.3
mock 2.0.0
numpy 1.19.5
oauth2client 4.1.3
pandas 1.1.5
pbr 5.8.1
pip 21.3.1
protobuf 3.17.3
py4j 0.10.8.1
pyarrow 2.0.0
pyasn1 0.4.8
pyasn1-modules 0.2.8
pydot 1.4.2
pyflink 1.0
pymongo 3.12.3
pyparsing 3.0.7
python-dateutil 2.8.0
pytz 2021.3
requests 2.27.1
rsa 4.8
setuptools 59.6.0
six 1.16.0
typing-extensions 3.7.4.3
urllib3 1.26.8
wheel 0.37.1
I tried to run this command :
(virtualenv) [myuser#myvm flink-1.14.3] ./bin/flink run -py examples/python/table/word_count.py
And got the following error:
Caused by: java.io.IOException: Failed to execute the command: python -c import pyflink;import os;print(os.path.join(os.path.abspath(os.path.dirname(pyflink.file)), 'bin'))
output: Traceback (most recent call last):
File "", line 1, in
ImportError: No module named pyflink
I am sure pyflink package is already installed. Does anyone know why?
To install PyFlink, you only need to execute:
python -m pip install apache-flink
and make sure you have a compatible Python version (>= 3.5).
The problem may be the Python Virtual Environment， refer to https://nightlies.apache.org/flink/flink-docs-release-1.14/docs/dev/python/faq/#preparing-python-virtual-environment
Also may be you can add option '-pyexec venv.zip/venv/bin/python3' and have a try
You have to check if pyflink is well installed (in your venv)
also check if you are running Flink
if no, start it with :
start-cluster.sh
here is full documentation about PyFlink:
https://nightlies.apache.org/flink/flink-docs-master/docs/dev/python/overview/
I have a number of Python 3.7 apps on Google App Engine standard, all building and deploying fine. I'm trying to upgrade some of them to the new Python 3.8 runtime, but when I try to deploy, they fail in Cloud Build.
It looks like they're hitting this open pip bug (more background). Odd that only the Python 3.8 runtime triggers this bug, though, and 3.7 builds fine.
Full log below. (Note that it's happening in Cloud Build, not my local machine, so I can't upgrade pip or otherwise change any of the commands or environment.) Anyone know how I can fix or work around this?
File upload done.
Updating service [default]...failed.
ERROR: (gcloud.beta.app.deploy) Error Response: [9] Cloud build 83e346a0-7e88-43dd-b89c-a4820526e4a1 status: FAILURE
Error ID: f8df99ad
Error type: INTERNAL
Error message: ... (setup.py): started
Building wheel for webapp2 (setup.py): finished with status 'done'
Created wheel for webapp2: filename=webapp2-3.0.0b1-py3-none-any.whl size=68362 sha256=9dd9f3ab6a55404492a88eb9a6bacb00faa37efafbc41f21a24d21cfba0eaea3
Stored in directory: /layers/google.python.pip/pipcache/wheels/55/e9/4d/76b030f418cac0bef4a3dcc15ca95c9671f1e826731ce2bc0f
Building wheel for tlslite-ng (setup.py): started
Building wheel for tlslite-ng (setup.py): finished with status 'done'
Created wheel for tlslite-ng: filename=tlslite_ng-0.7.5-py3-none-any.whl size=199869 sha256=b9ead00f0832041fba1e9d3883e57847995c2d6f83ecb7ea87d09cf82c730e8b
Stored in directory: /layers/google.python.pip/pipcache/wheels/a6/e1/a6/09610854c3405202d0b71d8f869811781e40cd26ffb85eacf8
Successfully built gdata humanize mf2py mf2util python-tumblpy ujson webapp2 tlslite-ng
Installing collected packages: six, ecdsa, tlslite-ng, lxml, gdata, certifi, urllib3, chardet, idna, requests, setuptools, protobuf, googleapis-common-protos, pyasn1, pyasn1-modules, rsa, cachetools, google-auth, pytz, grpcio, google-api-core, google-cloud-core, google-cloud-logging, gunicorn, pbr, extras, linecache2, traceback2, python-mimeparse, argparse, unittest2, testtools, fixtures, mox3, soupsieve, beautifulsoup4, gdata-python3, redis, google-cloud-datastore, google-cloud-ndb, humanize, MarkupSafe, jinja2, webencodings, html5lib, mf2py, mf2util, oauthlib, prawcore, websocket-client, update-checker, praw, requests-oauthlib, python-tumblpy, tweepy, ujson, webob, webapp2, oauth-dropins
Running setup.py develop for oauth-dropins
ERROR: Command errored out with exit status 1:
command: /opt/python3.8/bin/python3 -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/workspace/setup.py'"'"'; __file__='"'"'/workspace/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' develop --no-deps --home /tmp/pip-target-zp53suvg
cwd: /workspace/
Complete output (6 lines):
usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
or: setup.py --help [cmd1 cmd2 ...]
or: setup.py --help-commands
or: setup.py cmd --help
error: option --home not recognized
----------------------------------------
ERROR: Command errored out with exit status 1: /opt/python3.8/bin/python3 -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/workspace/setup.py'"'"'; __file__='"'"'/workspace/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' develop --no-deps --home /tmp/pip-target-zp53suvg Check the logs for full command output.
WARNING: You are using pip version 20.1.1; however, version 20.2.2 is available.
You should consider upgrading via the '/opt/python3.8/bin/python3 -m pip install --upgrade pip' command.
Full build logs: https://console.cloud.google.com/cloud-build/builds/83e346a0-7e88-43dd-b89c-a4820526e4a1?project=216076569502
Here's my requirements.txt file. I suspect the -e . might be the problem...but it works with Python 3.7, so if so, that's disappointing.
git+https://github.com/dvska/gdata-python3.git#egg=gdata
google-cloud-logging~=1.14
gunicorn~=20.0
mox3~=0.28
# this includes everything in setup.py's install_requires.
# https://caremad.io/posts/2013/07/setup-vs-requirement/#developing-reusable-things-or-how-not-to-repeat-yourself
-e .
I checked pypi page of oauth-dropins (at which it is failing) and they're mentioning there exactly this issue being caused by -e
I am trying to run the solr server (latest version 8.5.0) on Mac High Sierra (10.13.6) and using the following getting started guide -https://lucene.apache.org/solr/guide/8_5/solr-tutorial.html
However, when I run the command
solr-8.5.0:$ ./bin/solr start -e cloud
Instead of the expected solr startup message, i get a following error message on colsole
ERROR: Failed to start Solr using command: <path-to-bin>start -cloud -p 8983 -s <path-to-bin> Exception : org.apache.commons.exec.ExecuteException: Process exited with an error: 1 (Exit value: 1)
Would greatly appreciate any feedback on this.
Operating system is Ubuntu 16.04
Python version is 3
Installed the AWS CLI using PIP
AWS CLI version is aws-cli/1.16.309 Python/3.6.9 Linux/4.20.17-042017-generic botocore/1.13.45
I set the HMAC creds correctly and type the following in my command line:
ncheaz#thinkburger:~/Projects/drupal-interact$ aws --endpoint-url 'https://control.cloud-object-storage.cloud.ibm.com/v2/endpoints' s3 ls
An HTTP Client raised and unhandled exception: __init__() got an unexpected keyword argument 'ssl_context'
Check your urllib3 version. Upgrading urllib3 to 1.25.11 (latest) solved the problem for me. https://pypi.org/project/urllib3/1.25.11/
You can load the urllib (or urllib3) module by loading it in the Python Interpreter and then checking it's version:
$ python
>>> import urllib
>>> urllib.__version__
'1.17'
I'm running Google Cloud SDK [212.0.0] on Linux. Things have worked fine until today. Now, when I run dev_server:
$ dev_appserver.py .
I get this error:
dev_appserver.py: error: unrecognized arguments:
--datastore_emulator_cmd=/usr/platform/cloud-datastore-emulator/cloud_datastore_emulator
I hadn't done any updates for quite awhile, but since getting the error I did try updating the SDK with sudo apt-get update && sudo apt-get update google-cloud-sdk
Have also tried other app engine directories and get the same error.
Thanks,
Dan
You may try not running the command with --datastore_emulator_cmd. Rather, you could from the start run dev_appserver.py in the /bin/dev_appserver.py directory, while not using the --datastore_emulator_cmd parameter any more.
I faced similar problem on gcloud sdk with following configuration:
gcloud --version
Google Cloud SDK 227.0.0
app-engine-python 1.9.80
app-engine-python-extras 1.9.74
bq 2.0.39
cloud-datastore-emulator 2.0.2
cloud_sql_proxy
core 2018.11.30
gsutil 4.34
As I've checked there is "_correct_datastore_emulator_cmd" method implemented in "DevelopmentServer" class of file "google/appengine/tools/devappserver2/devappserver2.p" which modifies cmd with that emulator flag.
It can be potentially reported to sdk developers.