So, I'm trying to connect to my local instance of SQL Server on windows from airflow dag.
The major error is that pyodbc or pymmssql libraries are not installed even after explicitly mentioning them in the docker file. I've followed the instructions from Microsoft ODBC Driver for SQL Server under Debian 9 and Medium as the image is built on it.
All the tasks in dag run fine except for the one I am trying to connect to the SQL Server with the error:
[2020-04-03 22:23:42,303] {{taskinstance.py:1128}} ERROR - No module
named 'pyodbc'
If I try pip install from the airflow container it is not installing in the /usr/lib/python3.7/site-packages and not able to import in the dag.
I have also included requirements.txt in the volume but had no luck.
Did anyone had a successful connection in this scenario?
I was able to solve this issue by building the airflow image form the Dockerfile. while building you need to add few dependencies.
sqlalchemy:
&& pip3 uninstall SQLAlchemy \
&& pip3 install SQLAlchemy==1.3.15 \
pyodbc:
&& apt-get install python3-dev -y\
&& apt-get install unixodbc-dev -y\
&& pip3 install pyodbc \
Another long way would be to change the USER as root and build the image and ap-get install from the container and change change the USER back to airflow but this is more efficient if you know what you are installing.
Related
I have created a database in MySQL using XAMPP But In Some How My XAMPP Server is Not Working Now I am trying to Get My data Back
It's in
xampp\mysql\data
folder. Or you can export the database in PHPmyadmin.
If you use Linux, you can install mysql-client and mysql-server:
sudo apt-get install mysql-client
sudo apt-get install mysql-server
and then access to phpMyAdmin with navigate to where you extract mysql. Then run this command:
php -S localhost:80
If you use windows, you can use an alternative software instead. I think Laragon is a good alternative software.
I enabled UFW on a ubuntu server setup by someone else following the documentation on digitalocean. This server already had virtualmin/webmin installed.
Enabling the UFW had no effect on traffic with only OpenSSH allowed, and defaults set to incoming (deny), outgoing (allow). I was still able to visit the website even when Apache as not added to the UFW rule.
I had to reboot the server to see if the UFW will kick in, but the status of the firewall was "inactive" and now I cannot connect to the SQL Server Instance via the application or SSMS even after disabling the firewall.
The application has been down for hours, the world is on my neck and am almost at the end of my rope.
Going on guesswork here after multiple comments.
Sounds like you have updated the version of mssql-server but haven't updated the msodbcsql17 package. The latest CU needs the latest version of msodbcsql17 (unsure why, but not having it installed causes the connection failures).
Firstly update your packages:
sudo apt update
Then install the latest version of msodbcsql17:
sudo apt install msodbcsql17
This should install version 17.4.2.1-1.
If this, for some reason fails. You could roll back the update to mssql-server:
sudo apt install mssql-server=14.0.3223.3-15
How to interact with mssql from python? I realy nead it to write my site.
I tried pyodbc
(
pip install pyodbc
)
but cmd says there is an error in setup.py.
Pleese, help me!
P.S. I am using windows and mssql.
[Updated Answer]
Follow these steps to install pyodbc and pymssql in 64-bit Windows with *Python 3.**
Update pip tool: python -m pip install pip -U
Update python setuptools : python -m pip install setuptools -U
Install pyodb : pip3 install pyodbc -U
Download and install MS Visual C++ 14.0 Build Tools. It's required for installing pymssql.
double click on downloaded file and select Workloads → Visual C++ build tools then in optional features tab select Windows 10 SDK(windows 8 user select Windows 8 SDK) and install.
Install pymssql: pip3 install pymssql -U
[Old Answer]
One can also try these unofficial wheel files, if above method doesn't work.
You can install the same using the unofficial pyodbc wheel for python 3.* from here, just download the appropriate wheel (*.whl) file form this link and install it using
pip install **downloaded_wheel_file_name**.whl
Note: Don't forget to change the downloaded_wheel_file_name with your downloaded wheel file's name.
You need to have Microsoft ODBC driver for sql in Windows to make
pip install pyodbc
work.
Try to follow this link and install required drivers.
Try this #Kostya2007: pip install --allow-external pyodbc --allow-unverified pyodbc pyodbc
or
pip install https://storage.googleapis.com/google-code-archive-downloads/v2/code.google.com/pyodbc/pyodbc-3.0.7.zip
depending on your Python version
or
easy_install pyodbc-3.0.7.win32-py2.7.exe
I'm trying to connect to the Microsoft SQL server.
I'm using python code to do so, and I want to download pyodbc.
I go to the Python33 directory by doing this: cd C:\Python33\, and I try typing in pip install pyodbc, but it doesn't recognize pip.
Did anyone else have this problem with Python 3.3?
I'm also trying to do this:
C:\Python34\Scripts>pip install pyodbc
But it says: "The system cannot find the path specified."
Pip is a stand-alone tool, you will have to install that as well in 3.3. Here is a link on a really good explanation:
(How do I install pip on Windows?)
In 3.4, pip was included, but you will have to call it with
python -m pip
inside your scripts directory
I need to access a Microsoft SQL Server with OGR2OGR from an Ubuntu Server 16.04. It is working on Microsoft, so the basic setup is fine.
My problem is that when I run my OGR2OGR command (after installing GDAL with sudo apt-get install gdal-bin):
ogr2ogr -overwrite -f MSSQLSpatial "MSSQL:server=tcp:<DATABASE_SERVER>,<PORT>;database=<DATABASE_NAME>;uid=<USER>;Pwd=<PASSWORD>;" "<IMPORT PATH FILE>" -nln "<TABLE NAME>" -progress
I get the error:
ERROR 1: Unable to initialize connection to the server for MSSQL:"DATABASE_SERVER";
[unixODBC][Driver Manager]Can't open lib 'SQL Server' : file not found
Try specifying the driver in the connection string from the list of available drivers:
I tried to install Microsoft ODBC Driver for SQL Server based on this instruction, but it cannot be installed when gdal is installed. The error is:
Some packages could not be installed. This may mean that you have
requested an impossible situation or if you are using the unstable
distribution that some required packages have not yet been created
or been moved out of Incoming.
The following information may help to resolve the situation:
The following packages have unmet dependencies:
msodbcsql : Depends: unixodbc-utf16 (>= 2.3.1-1) but it is not going to be installed
E: Unable to correct problems, you have held broken packages.
I managed to install the Microsoft ODBC driver after a while but it meant removing GDAL. So I can either have GDAL or the Microsoft ODBC driver. Is there a way to solve this problem?
Thanks for your help!
UPDATE (2017.01.03): I found the following solution and I also commented on the according article on the MSDN Blog (so far without reply) here
Install UnixODBC if not already there
sudo apt-get install unixodbc unixodbc-dev
Download from somewhere: msodbcsql-11.0.2270.0.tar.gz (it could be better to use msodbcsql-13.x.x.x.tar.gz but I was not able to find it) and
tar xvfz msodbcsql-11.0.2270.0.tar.gz
cd msodbcsql-11.0.2270.0
ldd lib64/libmsodbcsql-11.0.so.2270.0
If there are missing dependencies install them, in my case it could be done like this: (everything except the apt-get install is to fix the naming of the file by creating a link:)
sudo apt-get install libssl1.0.0 libssl-dev
cd /lib/x86_64-linux-gnu
sudo ln -s libssl.so.1.0.0 libssl.so.10
sudo ln -s libcrypto.so.1.0.0 libcrypto.so.10
Installing and cleaning up:
sudo bash ./install.sh install --force --accept-license
rm -rf /tmp/msodbcubuntu
If the Driver is not found by the tool using it, (e.g. ogr2ogr or pyodbc) edit /etc/odbcinst.ini and create a connector for [SQL Server]
[SQL Server]
Description=Microsoft ODBC Driver 11 for SQL Server
Driver=/opt/microsoft/msodbcsql/lib64/libmsodbcsql-11.0.so.2270.0
Threading=1
UsageCount=2
Shamelessly from : https://learn.microsoft.com/en-us/sql/connect/odbc/linux-mac/installing-the-microsoft-odbc-driver-for-sql-server?view=sql-server-ver15
sudo su
curl https://packages.microsoft.com/keys/microsoft.asc | apt-key add -
curl https://packages.microsoft.com/config/ubuntu/16.04/prod.list > /etc/apt/sources.list.d/mssql-release.list
sudo apt-get update
sudo ACCEPT_EULA=Y apt-get install msodbcsql17
Verify with
ogr2ogr --formats
Which should then list:
MSSQLSpatial -vector- (rw+): Microsoft SQL Server Spatial Database