I have gone through every possible solution on the internet, but I'm unable to make pyobdc get the drivers on heroku.
The steps I have used to create the app are as follows:
heroku create
heroku config:set FLASK_CONFIG=heroku
heroku buildpacks:add heroku/python
heroku buildpacks:add --index 1 heroku-community/apt
git push heroku master
I tried even with odbc buildpack but still no luck:
heroku buildpacks:add https://github.com/iFix/heroku-buildpack-odbc.git
After going through microsoft website, I trimmed down my Aptfile to instuct heroku to install the following packages:
# install msodbcsql17
https://packages.microsoft.com/ubuntu/16.04/prod/pool/main/m/msodbcsql17/msodbcsql17_17.4.2.1-1_amd64.deb
# install mssql-tools
https://packages.microsoft.com/ubuntu/16.04/prod/pool/main/m/mssql-tools/mssql-tools_17.4.1.1-1_amd64.deb
# install unixodbc-dev
unixodbc-dev
This makes pyodbc installation go without error. But when I run pyodbc.drivers(), it returns nothing. The same command on Ubuntu 16.04 returns "ODBC Driver 17 for SQL Server"
The source code for the project is at : https://github.com/IamVNIE/odbcTestHeroku
The hosted app is at : https://pyodbctest.herokuapp.com/
Can someone please provide some pointers to make this work.
I have solved this issue by precompiling ODBC Driver 17 for SQL Server on a machine running Ubuntu 18.04 and copying the libmsodbcsql-17.5.so.2.1 and msodbcsqlr17.rll files into appropriate directories via a Heroku buildpack. This is ODBC Driver 17.5 for SQL Server and I likely won't be compiling other versions of this driver, but I imagine the concept remains the same.
The Heroku buildpack and its requirements can be found here https://github.com/matt-bertoncello/python-pyodbc-buildpack.git
Previous answer worked for me until I had to switch to heroku stack 22. After that I started getting the following error:
pyodbc.OperationalError: ('08001', '[08001] [unixODBC][Microsoft][ODBC Driver 17 for SQL Server]Client unable to establish connection (0) (SQLDriverConnect)')
To solve this issue I switched to another library python pymssql, that handles connections to Microsoft SQL server in azure.
here is a simple example
conn = pymssql.connect(server=xxx,\
user=yyy, \
password=zzz, \
database=xxx)
cursor = conn.cursor(as_dict=True)
cursor.execute(query)
rows = cursor.fetchall()
cursor.close()
conn.close()
Related
i have made a QT Desktop application as an Appimage. On my development laptop it works fine connecting to the database flawless. But when i put the Appimage to my PC where also the MS SQL Server is hosted it does not work anymore.
My Laptop and PC are Linux Mint 19 Laptop is 19.1 and PC is 19.3 to be specific.
I have installed odbc and tds following this guide:
https://help.interfaceware.com/kb/904
Ubuntu
Run the following commands using a terminal window, or the CLI:
Get the latest package info:
sudo apt-get update
Install libodbc.so, libtdsS.so, isql, and isqlinst:
sudo apt-get install unixodbc
Install libtsdsodbc.so and tsql:
sudo apt-get install tdsodbc
Create a file called odbc.ini
i also edited the /etc/odbc.ini
like this:
Description = MS SQL Server Driver
Driver = /usr/lib/x86_64-linux-gnu/odbc/libtdsodbc.so
Setup = /usr/lib/x86_64-linux-gnu/odbc/libtdsS.so
Server = 192.168.0.10
Port = 1433
but when i run the application on the PC i always get:
QSqlError("0", "QODBC3: Unable to connect", "[unixODBC][Driver Manager]Can't open lib 'libtdsodbc.so' : file not found")
i also checked that the library is really under the specified path:
/usr/lib/x86_64-linux-gnu/odbc/libtdsodbc.so
yes it is really there.
so i asking what do i missing here?
help would be really appreciated
best regards
Rolf
I found the solution myself.
I tired to falsify the odbc.ini and the odbcinst.ini on my laptop trying to
reproduce the error there i found that it did took no effect.
After searching for the libtdsodbc.so on the laptop i found it also located at
/usr/local/lib/
so i copied the library from /usr/lib/x86_64-linux-gnu/odbc/libtdsodbc.so to
/usr/local/lib
and that solved the issue and the QT application is now working fine.
I have presto and apache superset hosted at GCP Cloud.
Presto server hosted at http://14.22.122.12:8088/ui/
But when i try to connect Presto to Superset it's giving me this error
Could not load database driver: presto
Already installed the presto driver using pip install pyhive.
Not sure what's wrong here ? My SQLAlchemyURL presto://14.22.122.12:8088/catalog_name
This error usually appears when the driver isn't available (so the connection string format isn't recognized).
I would confirm that the pip install happened correctly. If you're running everything within docker, make sure you rebuilt properly. You can enter the docker cli, fire up a Python shell, and try to import the presto driver (you can also try pip freeze in the docker cli).
After trying several approaches such as this one on Mac and I still get the error below when I do gcloud app deploy. I also tried manually adding the freetds driver to ODBC Manager GUI but once I cick 'Ok' it doesn't show in the list of drivers. Same thing happends for the DSN for both user and system. I would greatly appreciate any help.
Step 6/14 : ADD odbcinst.ini /etc/odbcinst.ini
ADD failed: stat /var/lib/docker/tmp/docker-builder438222044/odbcinst.ini: no such file or directory
ERROR
ERROR: build step 0 "gcr.io/cloud-builders/docker" failed: step exited with non-zero status: 1
I wanted to do a realtime deployment of my model on azure, so I plan to create an image which firsts queries an ID in azure SQL db to get the required features, then predicts using my model and returns the predictions. The error I get from PyODBC library is that drivers are not installed
I tried it on the azure ML jupyter notebook to establish the connection and found that no drivers are being installed in the environment itself. After some research i found that i should create a docker image and deploy it there, but i still met with the same results
driver= '{ODBC Driver 13 for SQL Server}'
cnxn = pyodbc.connect('DRIVER='+driver+';SERVER='+server+';PORT=1433;DATABASE='+database+';UID='+username+';PWD='+ password+';Encrypt=yes'+';TrustServerCertificate=no'+';Connection Timeout=30;')
('01000', "[01000] [unixODBC][Driver Manager]Can't open lib 'ODBC
Driver 13 for SQL Server' : file not found (0) (SQLDriverConnect)")
i want a result to the query instead i get this message
and/or you could use pymssql==2.1.1, if you add the following docker steps, in the deployment configuration (using either Environments or ContainerImages - preferred is Environments):
from azureml.core import Environment
from azureml.core.environment import CondaDependencies
conda_dep = CondaDependencies()
conda_dep.add_pip_package('pymssql==2.1.1')
myenv = Environment(name="mssqlenv")
myenv.python.conda_dependencies=conda_dep
myenv.docker.enabled = True
myenv.docker.base_dockerfile = 'FROM mcr.microsoft.com/azureml/base:latest\nRUN apt-get update && apt-get -y install freetds-dev freetds-bin vim gcc'
myenv.docker.base_image = None
Or, if you're using the ContainerImage class, you could add these Docker Steps
from azureml.core.image import Image, ContainerImage
image_config = ContainerImage.image_configuration(runtime= "python", execution_script="score.py", conda_file="myenv.yml", docker_file="Dockerfile.steps")
# Assuming this :
# RUN apt-get update && apt-get -y install freetds-dev freetds-bin vim gcc
# is in a file called Dockerfile.steps, it should produce the same result.
See this answer for more details on how I've done it using an Estimator Step and a custom docker container. You could use this Dockerfile to locally create a Docker container for that Estimator step (no need to do that if you're just using an Estimator run outside of a pipeline) :
FROM continuumio/miniconda3:4.4.10
RUN apt-get update && apt-get -y install freetds-dev freetds-bin gcc
RUN pip install Cython
For more details see this posting :using estimator in pipeline with custom docker images. Hope that helps!
Per my experience, I think the comment as #DavidBrowne-Microsoft said is right.
There is a similar SO thread I am getting an error while connecting to an sql DB in Jupyter Notebook answered by me, which I think it will help you to install the latest msodbcsql driver for Linux on Microsoft Azure Notebook or Docker.
Meanwhile, there is a detail about the connection string for Azure SQL Database which you need to carefully note, that you should use {ODBC Driver 17 for SQL Server} instead of {ODBC Driver 13 for SQL Server} if your Azure SQL Database had been created recently (ignore the connection string shown in Azure portal).
you can use AzureML built in solution dataset to connect to your SQL server.
To do so, you can first create an azure_sql_database datastore. reference here
Then create a dataset by passing the datastore you created and the query you want to run.
reference here
sample code
from azureml.core import Dataset, Datastore, Workspace
workspace = Workspace.from_config()
sql_datastore = Datastore.register_azure_sql_database(workspace = workspace,
datastore_name = 'sql_dstore',
server_name = 'your SQL server name',
database_name = 'your SQL database name',
tenant_id = 'your directory ID/tenant ID of the service principal',
client_id = 'the Client ID/Application ID of the service principal',
client_secret = 'the secret of the service principal')
sql_dataset = Dataset.Tabular.from_sql_query((sql_datastore, 'SELECT * FROM my_table'))
You can also do it via UI at ml.azure.com where you can register an azure SQL datastore using your user name and password.
I have a laravel 5.2 instance configured to run against MS SQL Server. I have downloaded from microsoft the drivers and the connection runs perfect DB::table('usuarios')->select('*')->get(); returns:
[{"id":"5","username":"jtd","password":"$2y$10$t8RXYy\/afaEuIpE8GIk2bOvbHsQoj0fhGPxmo\/3UqJABKVA4I4pHC","email":""}]
But when I try to php artisan migrate i get the:
[PDOException]
could not find driver
Why this is happening?
Stack trace:
c:\wamp\www\presence\webapp>php artisan migrate --verbose
[PDOException]
could not find driver
Exception trace:
() at C:\wamp\www\presence\webapp\vendor\laravel\framework\src\Illuminate\Datab
ase\Connectors\Connector.php:55
PDO->__construct() at C:\wamp\www\presence\webapp\vendor\laravel\framework\src\
Illuminate\Database\Connectors\Connector.php:55
Illuminate\Database\Connectors\Connector->createConnection() at C:\wamp\www\pre
sence\webapp\vendor\laravel\framework\src\Illuminate\Database\Connectors\SqlServ
erConnector.php:32
Illuminate\Database\Connectors\SqlServerConnector->connect() at C:\wamp\www\pre
sence\webapp\vendor\laravel\framework\src\Illuminate\Database\Connectors\Connect
ionFactory.php:61
Illuminate\Database\Connectors\ConnectionFactory->Illuminate\Database\Connector
s\{closure}() at n/a:n/a
call_user_func() at C:\wamp\www\presence\webapp\vendor\laravel\framework\src\Il
luminate\Database\Connection.php:924
Illuminate\Database\Connection->getPdo() at C:\wamp\www\presence\webapp\vendor\
laravel\framework\src\Illuminate\Database\Connection.php:792
Illuminate\Database\Connection->reconnectIfMissingConnection() at C:\wamp\www\p
resence\webapp\vendor\laravel\framework\src\Illuminate\Database\Connection.php:6
77
Illuminate\Database\Connection->run() at C:\wamp\www\presence\webapp\vendor\lar
avel\framework\src\Illuminate\Database\Connection.php:349
Illuminate\Database\Connection->select() at C:\wamp\www\presence\webapp\vendor\
laravel\framework\src\Illuminate\Database\Schema\Builder.php:55
Illuminate\Database\Schema\Builder->hasTable() at C:\wamp\www\presence\webapp\v
endor\laravel\framework\src\Illuminate\Database\Migrations\DatabaseMigrationRepo
sitory.php:141
Illuminate\Database\Migrations\DatabaseMigrationRepository->repositoryExists()
at C:\wamp\www\presence\webapp\vendor\laravel\framework\src\Illuminate\Database\
Migrations\Migrator.php:404
Illuminate\Database\Migrations\Migrator->repositoryExists() at C:\wamp\www\pres
ence\webapp\vendor\laravel\framework\src\Illuminate\Database\Console\Migrations\
MigrateCommand.php:103
Illuminate\Database\Console\Migrations\MigrateCommand->prepareDatabase() at C:\
wamp\www\presence\webapp\vendor\laravel\framework\src\Illuminate\Database\Consol
e\Migrations\MigrateCommand.php:58
Illuminate\Database\Console\Migrations\MigrateCommand->fire() at n/a:n/a
call_user_func_array() at C:\wamp\www\presence\webapp\vendor\laravel\framework\
src\Illuminate\Container\Container.php:507
Illuminate\Container\Container->call() at C:\wamp\www\presence\webapp\vendor\la
ravel\framework\src\Illuminate\Console\Command.php:169
Illuminate\Console\Command->execute() at C:\wamp\www\presence\webapp\vendor\sym
fony\console\Command\Command.php:256
Symfony\Component\Console\Command\Command->run() at C:\wamp\www\presence\webapp
\vendor\laravel\framework\src\Illuminate\Console\Command.php:155
Illuminate\Console\Command->run() at C:\wamp\www\presence\webapp\vendor\symfony
\console\Application.php:794
Symfony\Component\Console\Application->doRunCommand() at C:\wamp\www\presence\w
ebapp\vendor\symfony\console\Application.php:186
Symfony\Component\Console\Application->doRun() at C:\wamp\www\presence\webapp\v
endor\symfony\console\Application.php:117
Symfony\Component\Console\Application->run() at C:\wamp\www\presence\webapp\ven
dor\laravel\framework\src\Illuminate\Foundation\Console\Kernel.php:107
Illuminate\Foundation\Console\Kernel->handle() at C:\wamp\www\presence\webapp\a
rtisan:36
I found a solution. I post it for everyone. I use WAMP stack (PHP, Apache and MySQL). The fact is that wamp uses the 5.5.12 version for terminal CLI, so I have to install SQL Server PDO Extensions on the version I am using and (besides this) on the 5.5.12 version (I have several PHP versions installed). Can provide more information if needed.