When running Spark Streaming code in Zeppelin 0.7.1 it will throw an exception saying that the Jackson version is too old.
You have to replace these jar files with version 2.6.2 and copy them to zeppelin-0.7.1-bin-all/lib/.
wget http://central.maven.org/maven2/com/fasterxml/jackson/core/jackson-core/2.6.2/jackson-core-2.6.2.jar
wget http://central.maven.org/maven2/com/fasterxml/jackson/core/jackson-databind/2.6.2/jackson-databind-2.6.2.jar
wget http://central.maven.org/maven2/com/fasterxml/jackson/core/jackson-annotations/2.6.2/jackson-annotations-2.6.2.jar
Related
I am testing pyflink on
os: centos7
flink version: flink-1.14.3
virtualenv python version: Python 3.6.8
pip list:
apache-beam 2.27.0
apache-flink 1.14.3
apache-flink-libraries 1.14.3
avro-python3 1.9.2.1
certifi 2021.10.8
charset-normalizer 2.0.11
cloudpickle 1.2.2
crcmod 1.7
dill 0.3.1.1
docopt 0.6.2
fastavro 0.23.6
future 0.18.2
grpcio 1.43.0
hdfs 2.6.0
httplib2 0.17.4
idna 3.3
mock 2.0.0
numpy 1.19.5
oauth2client 4.1.3
pandas 1.1.5
pbr 5.8.1
pip 21.3.1
protobuf 3.17.3
py4j 0.10.8.1
pyarrow 2.0.0
pyasn1 0.4.8
pyasn1-modules 0.2.8
pydot 1.4.2
pyflink 1.0
pymongo 3.12.3
pyparsing 3.0.7
python-dateutil 2.8.0
pytz 2021.3
requests 2.27.1
rsa 4.8
setuptools 59.6.0
six 1.16.0
typing-extensions 3.7.4.3
urllib3 1.26.8
wheel 0.37.1
I tried to run this command :
(virtualenv) [myuser#myvm flink-1.14.3] ./bin/flink run -py examples/python/table/word_count.py
And got the following error:
Caused by: java.io.IOException: Failed to execute the command: python -c import pyflink;import os;print(os.path.join(os.path.abspath(os.path.dirname(pyflink.file)), 'bin'))
output: Traceback (most recent call last):
File "", line 1, in
ImportError: No module named pyflink
I am sure pyflink package is already installed. Does anyone know why?
To install PyFlink, you only need to execute:
python -m pip install apache-flink
and make sure you have a compatible Python version (>= 3.5).
The problem may be the Python Virtual Environment， refer to https://nightlies.apache.org/flink/flink-docs-release-1.14/docs/dev/python/faq/#preparing-python-virtual-environment
Also may be you can add option '-pyexec venv.zip/venv/bin/python3' and have a try
You have to check if pyflink is well installed (in your venv)
also check if you are running Flink
if no, start it with :
start-cluster.sh
here is full documentation about PyFlink:
https://nightlies.apache.org/flink/flink-docs-master/docs/dev/python/overview/
I have installed Apache drill version 1.16 but i got this error message when i bin/drill-embedded
this is my java -version
and this is my update-alternatives -config java
and this is my etc/environement
This has been closed, i have corrected that by just installing again without pointing to my JAVA HOME because it was already pointed, weird.
I have a flask app that used to run with python3.5 on apache2.
I am now trying to change it to run on 3.6 (i have sone type hinting in my code that is only supported on 3.6).
I have installed the module for python3.6, but when tried to restart apache and run my app, it fails.
When running
mod_wsgi-express module-location
I have the python3.5 version
/usr/local/lib/python3.5/dist-packages/mod_wsgi/server/mod_wsgi-py35.cpython-35m-x86_64-linux-gnu.so
The same file also existis on my server in python3.6 dir.
I also changed my $PATH to run python3.6 as a default for python.
How can I tell apache to run on python3.6 and not on 3.5?
WOHA! It's working!
I uninstalled the mod_wsgi (sudo apt get remove libapache2-mod-wsgi).
Then, Installed the module via pip specifically to python3.6 (python3.6 -m pip install mod_wsgi), but at this point the apache still ran python3.5.I added to the apache configuration file (in ubunutu 16.4 the file is located at: /etc/apache2/apache2.conf and /usr/local/lib64/python3.6/site-packages/mod_wsgi/server/mod_wsgi-py36.cpython-36m-x86_64-linux-gnu.so for RHEL/CentOS 7) the followings:
LoadModule wsgi_module "/usr/local/lib/python3.6/dist-packages/mod_wsgi/server/mod_wsgi-py36.cpython-36m-x86_64-linux-gnu.so"
and
WSGIPythonHome "/usr" and that did the trick.
I weeped a lot of tears over this issue, finding the apache2.conf file etc, so hopefully this will be helpful to anyone.
The LoadModule solution worked for me. However, I found it easiest to first ensure
a2dismod wsgi as provided by the default apache2 module. I spent 3 days looking at various solutions and answers and reviewing Graham's WSGI guide.
What I did in my case is was to remove python 3.6 from Django env and re-install python 3.5 and it worked.
I git clone the zeppelin from https://github.com/apache/incubator-zeppelin.git, and make the project by running:
mvn clean package -Pspark-1.5 -Dhadoop.version=2.2.0 -Phadoop-2.2 -Ppyspark -DskipTests
but i always get the error:
Most probably, this indicates a dependency version miss-match, in this case Apache Lense.
The best way is to try re-building Apache Zeppelin from latest master, and if the issue persists - file an issue on official project JIRA
I have downloaded mod_jk-1.2.28-httpd-2.2.X.so for Apache 2.2 running on CentOS, and set up as per http://tomcat.apache.org/connectors-doc/webserver_howto/apache.html. When I try to start httpd it fails with the following error:
"Starting httpd: httpd: Syntax error on line 993 of /etc/httpd/conf/httpd.conf: Syntax error on line 2 of /opt/apache-tomcat-6.0.26/conf/jk/mod_jk.conf-auto: Cannot load /etc/httpd/modules/mod_jk-1.2.28-httpd-2.2.X.so into server: /etc/httpd/modules/mod_jk-1.2.28-httpd-2.2.X.so: wrong ELF class: ELFCLASS32"
Does that mean that mod_jk-1.2.28-httpd-2.2.X.so has not been properly compiled?. What can I do about that?
Thanks
Peter
It means that you have a 32-bit module, but are not running a 32-bit httpd. Install the proper gcc package from the repositories and rebuild the module.
To fix, I installed the Tomcat Connector using yum:
yum search mod_jk
mod_jk-ap20.x86_64 : Tomcat mod_jk connector for Apache 2.0.x
mod_jk-ap22.x86_64 : Tomcat mod_jk connector for Apache 2.x
yum install mod_jk