RODBC MS SQL access from Ubuntu using FreeTDS - sql-server

I'm trying to access MS SQL server hosted on my Windows box from an Ubuntu machine. I have a shiny app that access MSSQL that works fine on Windows. I want to host this with shiny server on Ubuntu so that others can access the webpage and provide their SQL server as input to the app.
All of this works fine on Windows. Now I'm not able to get unixODBC working. I think I've got the odbc/freeTDS installation and configuration correct. I'm able to connect and query SQL DB instance from Ubuntu terminal using tsql -S . It took some googling but eventually it worked.
Now, when I try to connect from R I get this error.
sql <- odbcConnect("abc.xyz.com", "uname", "passwd")
In odbcDriverConnect("DSN=abc.xyz.com;UID=uname;PWD=passwd")
: [RODBC] ERROR: state 08001, code 0, message
[unixODBC][FreeTDS][SQL Server]Unable to connect to data source 2: In
odbcDriverConnect("DSN=abc.xyz.com;UID=uname;PWD=passwd")
: [RODBC] ERROR: state 01000, code 20002, message
[unixODBC][FreeTDS][SQL Server]Adaptive Server connection failed 3: In
odbcDriverConnect("DSN=abc.xyz.com;UID=uname;PWD=passwd")
: [RODBC] ERROR: state 01000, code 20017, message
[unixODBC][FreeTDS][SQL Server]Unexpected EOF from the server 4: In
odbcDriverConnect("DSN=abc.xyz.com;UID=uname;PWD=passwd")
: ODBC connection failed
freeTDS.conf
[abc.xyz.com]
host = abc.xyz.com
port = 49475
tds version = 8.0
odbcinst.ini
[FreeTDS]
Description = FreeTDS unixODBC Driver
Driver = /usr/local/lib/libtdsodbc.so
Setup = /usr/local/lib/libtdsodbc.so
odbc.ini
[abc.xyz.com]
Description = Shiny testing
Driver = FreeTDS
Trace = No
Server = abc.xyz.com\instance_name
Database = dbanme
port = 49475
This error "Unexpected EOF from the server" is not new to me. I got the same error when I was trying to connect using tsql. I was able to overcome this by adding "tds version = 8.0" line to freeTDS.conf. Not sure how I can get RODBC to use this config. I've read posts were other users were able to sql-server using freeTDS. Not sure what is missing here. I also tried re-installing RODBC.

I fixed this. TDS version was needed in odbc.ini file.
TDS_Version = 8.0
tsql reads version info from freeTDS.conf and worked. isql was failing with same error and it was also looking for this config in odbc.ini. So test with isql if you are configuring for R/Python.

I don't know what is causing the error on your side. Hope this helps:
These are my settings for RODBC 1.3-10 to connect to MSSQL Server 2012. I am also on ubuntu and this works for me.
I think I installed the RODBC package from the Ubuntu repos:
apt-cache policy r-cran-rodbc
r-cran-rodbc:
Installed: 1.3-10-1
Candidate: 1.3-10-1
Version table:
*** 1.3-10-1 0
500 ftp://ftp.fu-berlin.de/linux/ubuntu/ trusty/universe amd64 Packages
100 /var/lib/dpkg/statu
uname -a
Linux xxx 3.13.0-30-generic #54-Ubuntu SMP Mon Jun 9 22:45:01 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
odbc.ini
[mydsn]
APP = unixodbc
Description = master on vmXX
Driver = TDSdriver
Server = vmXX
Database = master
Port = 1433
#Trace = No
#TraceFile = /var/log/freetds/freetds--odbc.log
TDS Version = 7.2
freetds.conf
[global]
# TDS protocol version
; tds version = 4.2
# Whether to write a TDSDUMP file for diagnostic purposes
# (setting this to /tmp is insecure on a multi-user system)
#dump file = /tmp/freetds.log
#debug flags = 0xffff
# Command and connection timeouts
; timeout = 10
; connect timeout = 10
# If you get out-of-memory errors, it may mean that your client
# is trying to allocate a huge buffer for a TEXT field.
# Try setting 'text size' to a more reasonable limit
text size = 64512
[vmXX]
host = 111.222.333.555
port = 1433
tds version = 7.2
client charset = UTF-8
Output from
ps aux | grep "/R"
knb 56969 ....
lsof -p 56969 | grep -i odbc
R 56969 knb mem REG 8,1 72408 8528592 /usr/lib/x86_64-linux-gnu/libodbcinst.so.1.0.0
R 56969 knb mem REG 8,1 400608 8521896 /usr/lib/x86_64-linux-gnu/odbc/libtdsodbc.so
R 56969 knb mem REG 8,1 419680 8525415 /usr/lib/x86_64-linux-gnu/libodbc.so.1.0.0
R 56969 knb mem REG 8,1 96845 8933205 /usr/local/lib/R/site-library/RODBC/libs/RODBC.so

I had a similar error and realized R was just unable to read my user id correctly as it had a slash in it. I placed my credentials in a text file which solved the problem for me.
cred <- "/myPath/ODBC_cred.txt"
sql <- RODBC::odbcConnect("abc.xyz.com", uid=readLines(cred)[1], pwd=readLines(cred)[2])

Related

Connect to SQL Server database from RHEL with Windows Authentication

I have been working for two weeks in the installation of Superset (from Airbnb) for data visualization on a virtual RHEL machine and the connection with a SQL Server database. But I still cannot connect to this database because of a problem of driver I guess. I tried many things and I would like to know if you have a solution, about a driver I need, about modifications in my configuration etc...
Someone told me about jTDS driver. Maybe I need something like this but for python. If you have any idea, here is what I already did.
1) I tried to connect to the database from Superset :
SQL Alchemy URI : mssql://user:password#fr0-iacls-190.eu.company.corp:10001/dbname
ERROR : {"error": "Connection failed!\n\n
The error message returned was:\n(pyodbc.Error) ('IM002', '[IM002] [unixODBC][DriverManager]
Data source name not found, and no default driver specidied (0) (SQLDriverConnect)')"}
2) I tried almost the same with mssql+pymssql :
SQL Alchemy URI : mssql+pymssql://user:password#fr0-iacls-190.eu.company.corp:10001/dbname
ERROR:{"error":"Connection failed!\n\n
The error message returned was:\n(pymssql.OperationalError) (18456, 'DB-Lib error message 20018,
severity 14:\\nGeneral SQL Server error: Check messages from the SQL Server\\n
DB-Lib error message 20002, severity 9:\\n Adaptive Server connection failed (fr0-iacls-190.eu.company.corp:10001)\\n')"}
3) I tried to connect to the database from my terminal on virtual RHEL machine :
# tsql -S fr0-iacls-190.eu.company.corp -U user
Password:
locale is "en_US.UTF-8"
locale charset is "UTF-8"
using default charset "UTF-8"
20^C
Here I have a timer that increase the number every second. I stopped the example after 20 seconds.
4) Finally I tried a python script like this one :
import pyodbc
cnxn = pyodbc.connect('DRIVER={SQL Server};SERVER=fr0-iacls-190.eu.company.corp;DATABASE=dbname;UID=;PWD=password')
The empty UID is a tip used in another StackOverflow post.
# python connect.py
Traceback (most recent call last):
File "connect.py", line 2, in <module>
cnxn = pyodbc.connect('DRIVER={SQL Server};SERVER=fr0-iacls-190.eu.company.corp;DATABASE=dbname;UID=;PWD=password')
pyodbc.Error: ('01000', "[01000] [unixODBC][Driver Manager]Can't open lib 'SQL Server' : file not found (0) (SQLDriverConnect)")
To finish, I read about two files, odbc.ini and odbcinst.ini but I don't know if they are in the good directory (/etc). I am not working on root (~) but in in the parent directory of root (cd ~/..)...
Here are the two files if necessary :
odbc.ini
;
; odbc.ini
;
[ODBC Data Sources]
JDBC = Sybase JDBC Server
[JDBC]
Driver = /usr/local/lib/libtdsodbc.so
Description = Sybase JDBC Server
Trace = No
Servername = JDBC
Database = pubs2
UID = guest
[Default]
Driver = /usr/local/lib/libtdsodbc.so
odbcinst.ini
[PostgreSQL]
Description=ODBC for PostgreSQL
Driver=/usr/lib/psqlodbcw.so
Setup=/usr/lib/libodbcpsqlS.so
Driver64=/usr/lib64/psqlodbcw.so
Setup64=/usr/lib64/libodbcpsqlS.so
FileUsage=1
[MySQL]
Description=ODBC for MySQL
Driver=/usr/lib/libmyodbc5.so
Setup=/usr/lib/libodbcmyS.so
Driver64=/usr/lib64/libmyodbc5.so
Setup64=/usr/lib64/libodbcmyS.so
FileUsage=1
[MSSQLTest]
Driver = ODBC Driver 13 for SQL Server
Server = [http:]fr0-iacls-190.eu.company.corp[,10001]
#
# Note:
# Port is not a valid keyword in the ~/.odbc.ini file
# for the Microsoft ODBC driver on Linux
#
[ODBC Driver 13 for SQL Server]
Description=Microsoft ODBC Driver 13 for SQL Server
Driver=/opt/microsoft/msodbcsql/lib64/libmsodbcsql-13.1.so.4.0
UsageCount=1
Many thanks for your time and your help.
A few things to try:
Correct the python connection string:
You have the aliased the MS SQL driver in /etc/odbcinst.ini to [ODBC Driver 13 for SQL Server] therefore in your python code you should be using that, rather than SQL Server:
import pyodbc
cnxn = pyodbc.connect('Driver={ODBC Driver 13 for SQL Server};Server=fr0-iacls-190.eu.company.corp;Port=10001;Database=dbname;UID=user;PWD=password')
replacing user and password with correct credentials.
Use isql to test a connection using odbc.ini:
Generally you use the odbcinst.ini to setup driver configuration, and then odbc.ini for database instances (referencing the drivers), thus a valid entry to your odbc.ini could be:
[friendly_database_name]
Description=A description to help you remember what this connection is for
Server=fr0-iacls-190.eu.company.corp
Port=10001
Database=dbname
UID=user
PWD=password
If you have isql installed (comes as part of the unixODBC package if not), then you can test with:
$ isql -3 -v friendly_database_name
Ensure there's not a firewall blocking you:
I'm not sure why the tsql command is failing for you, it should return the 1> prompt. Ensure that you can establish a telnet connection to the database server:
$ telnet fr0-iacls-190.eu.company.corp 10001
which should give you something like:
Trying 12.34.56.78...
Connected to fr0-iacls-190.eu.company.corp.
Escape character is '^]'
and then to ctrl + ] and type quit to exit (or do ctrl + c to cancel if the telnet test fails).

Connect to a SQL Server database through RODBC

My question is related to
Trying to connect to an ODBC server using RODBC in ubuntu
and
How to specify include and lib directories when locally installing RODBC?
but I could not find suitable answers to my case.
I want to connect to a SQL Server database on a remote server using RODBC.
I have installed unixodbc and freetds, and can connect in the terminal with T-SQL, so the connection exists.
But when trying to connect in R with (all sensitive info have been replaced by ***):
odbcConnect(dsn="TESTSQL", uid=***, pwd=***)
I get:
Warning messages:
1: In RODBC::odbcDriverConnect("DSN=TESTSQL;UID=***;PWD=***") : [RODBC] ERROR: state 01000, code 0, message [unixODBC][Driver Manager]Can't open lib '/usr/local/Cellar/freetds/0.95.18/lib/libtdsodbc.so' : file not found
2: In RODBC::odbcDriverConnect("DSN=TESTSQL;UID=***;PWD=***") :
ODBC connection failed
The odbc.ini file being:
[ODBC Data Sources]
TESTSQL = Test database
[TESTSQL]
Driver = MSSQL
Servername = ***.**.**.**
Port = **
Database = ****
TDS_Version = 8.0
I had installed the latest version of freetds, that is 1.00.27, I am hence surprised that this library libtdsodbc.so is missing.
Is that normal? Would you recommend to install the version 0.95.18 or rather stay with 1.00.27 and look for that missing library?
I had to remove freetds:
brew remove freetds
then resintalling it, specifying --with-unixodbc to have the libtsdodbc.so created:
brew install freetds --with-unixodbc
In the odbc.ini, I had then to take care not to confuse "Server" and "Servername", and link the driver to the libtdsodbc.so, so that my odbc.ini looks like:
[ODBC Data Sources]
TESTSQL = Test database
[TESTSQL]
Driver = /usr/local/lib/libtdsodbc.so
Server = ***.**.**.**
Port = **
Database = ****
TDS_Version = 8.0
and connected using the RODBC package
ch1 <- odbcConnect(dsn="TESTSQL", uid=***, pwd=***)
> ch1
RODBC Connection 5
Details:
case=nochange
DSN=TESTSQL
UID=****
PWD=******
it works!
Further detailed informations from this page
http://eriqande.github.io/2014/12/19/setting-up-rodbc.html

FreeTDS connecton timeout in second attempt

I have installed FreeTDS on my linux. I have created dsn and get connected to mssql server once using isql.
But when I close the connection and try to reopen isql using same previous command I have used to connect it does not allow me and showing me error like
[S1000][unixODBC][FreeTDS][SQL Server]Unable to connect to data source
[08S01][unixODBC][FreeTDS][SQL Server]Read from the server failed
If i try after few mins then it automatically get connected.
I have tried same in php odbc_connect() as well. Same issue is there as well. my page showing once and if I refresh it lost connection. After few mins it start serving for once.
My freetds.conf file
[MSSQLExample]
host = 192.168.10.151
port = 49172
tds version = 8.0
instance=SQL2014
my odbc.ini file
[MSSQLExample]
Description = SQL server
Driver = FreeTDS
Trace = No
servername=MSSQLExample
Database = <dbname>
port = 49172
LoginTimeout=1200
QueryTimeout=1200
CPTimeout=1200
CPTimeToLive=100
my odbcinst.ini file
[FreeTDS]
Description = FreeTDS
Driver = /usr/lib/x86_64-linux-gnu/odbc/libtdsodbc.so
Setup = /usr/lib/odbc/libtdsS.so
CPTimeout = 120
Please help me to sortout this issue.
Help is highly appreciated.

tsql connects to ODBC but Perl DBI gets an error

Client is Ubuntu 14.04.1 LTS talking to an MS SQL Server 2005. Ubuntu has no registered domain. It's inside the local network as is the MS SQL Server.
Perl 5.18.2 using DBI 1.63 and DBD::ODBC 1.52.
Installed FreeTDS 0.92 (with TDS v4.2) and unixODBC 2.2.14.
isql DSN USERNAME PASSWD at the command line also doesn't work. Perhaps I should start with that. I still get the same error which is Unable to connect to data source (SQL-08001).
My related config files:
2015-07-09 11:55:14 AM
NOTE: TDS version from tsql -C must go into freetds.conf and odbc.ini files
=========================================
/etc/odbcinst.ini:
[FreeTDS]
Description = FreeTDS 0.91 for Microsoft SQL Server 2005
#Driver = /usr/lib/odbc/libtdsodbc.so
Driver = /usr/lib/x86_64-linux-gnu/odbc/libtdsodbc.so
Setup = /usr/lib/x86_64-linux-gnu/odbc/libtdsS.so
UsageCount = 1
#Threading = 2
fileusage = 1
dontdlclose = 1
=========================================
/etc/odbc.ini:
[ODBC Data Sources]
ResponseBDO = Response database desc in /etc/odbc/ini
[FreeTDS]
Description = FreeTDS 0.91 driver
Driver = /usr/lib/x86_64-linux-gnu/odbc/libtdsodbc.so
[responsebdo]
Driver = FreeTDS
Description = Response MSSQL 2005 ODBC driver
SERVER = sqlsvr.mydomain.com
PORT = 1433
USER = 'domain\domainuser'
Password = mypasswd
Database = r4w_004
# TDS_VERSION should be same as reported by 'tsql -C'.
TDS_Version = 4.2
[Default]
Driver = /usr/lib/x86_64-linux-gnu/odbc/libtdsodbc.so
Description = Default FreeTDS ODBC driver
SERVER = 10.20.30.40
USER = "domain\domainuser"
Database = r4w_4
=========================================
Relevant part of /etc/freetds/freetds.conf:
# $Id: freetds.conf,v 1.12 2007/12/25 06:02:36 jklowden Exp $
#
# This file is installed by FreeTDS if no file by the same
# name is found in the installation directory.
#
# For information about the layout of this file and its settings,
# see the freetds.conf manpage "man freetds.conf".
#
[responsetds]
#host = 10.20.30.40
host = sqlsvr.mydomain.com
port = 1433
tds version = 4.2
connect timeout = 20
timeout = 20
dump file = /tmp/freetds-resp.log
instance = MSSQLSERVER
use ntlmv2 = yes
=========================================
tsql -C:
Compile-time settings (established with the "configure" script)
Version: freetds v0.91
freetds.conf directory: /etc/freetds
MS db-lib source compatibility: no
Sybase binary compatibility: yes
Thread safety: yes
iconv library: yes
TDS version: 4.2
iODBC: no
unixodbc: yes
SSPI "trusted" logins: no
Kerberos: yes
=========================================
odbcinst --version:
unixODBC 2.2.14
Perl connection info:
$respdsn="dbi:ODBC:Driver={SQL Native Client};Server=10.20.30.40;".
"UID=$respdbuser;PWD=$respdbpw;Database=r4w_004";
$respdbh=DBI->connect($respdsn);
I can connect to the database by using tsql at the command line.
But using Perl I get this error:
DBI connect('Driver={SQL Native Client};Server=10.19.1.3;UID=gilson\mwreports;PWD=MR4gilson;Database=r4w_004','',...) failed:
[unixODBC][Driver Manager]Data source name not found, and no default
driver specified (SQL-IM002) at /home/chuck/perl/gilson/jimv/fedex/fedex.pl line 1557.
at /usr/lib/perl5/DBI.pm line 691.
DBI::__ANON__[/usr/lib/perl5/DBI.pm:744](undef, undef) called at /usr/lib/perl5/DBI.pm line 747
My questions:
Is the "{SQL Native Client}" in the Perl connection string correct? Does it refer to an entry in the odbc.ini or freetds.conf file?
What am I missing here?
Thank you.
If you're using FreeTDS, then your Driver is FreeTDS not "Sql Native Client".
Try this line to get your database handle:
my $dbh = DBI->connect("dbi:ODBC:Driver=FreeTDS;Server=10.20.30.40;database=r4w_004", $respdbuser, $respdbpw);
I had the same problem, when I using tsql works normally but using perl with dbi didn't. I lost many hours trying to figure out the problem, so I saw the limitations of Adaptive Server:
The name cannot be more than 30 characters long.
The first character must be a letter (ASCII a through z, A through Z).
The characters that follow must be letters, numbers, or underscores (_).
... So I tried to change the password that was 32 characters long to 30 characters and it works.
The tsql accept long password but perl dbi sybase accept ony 30 characters.
Command to connect to default instance on port 1433:
perl -e 'use DBI;DBI->connect("dbi:Sybase:server=10.0.0.1","sa","max_30_characters_longpassword");'
Command to connect to named instance (use port udp 1434 to discover the port):
perl -e 'use DBI;DBI->connect("dbi:Sybase:server=10.0.0.1\\SQL2014","sa","max_30_characters_longpassword");'
Command to connect to named instance when you know the port that sqlserver is listening:
perl -e 'use DBI;DBI->connect("dbi:Sybase:server=10.0.0.1:60036","sa","max_30characters_long_password");'
I used the tcpdump to monitoring the connection to the sqlserver:
tcpdump -qn dst host 10.0.0.1 &
Maybe it helps someone. Thanks

How to connect from ruby to MS Sql Server

I'm trying to connect to the sql server 2005 database from *NIX machine:
I have the following configuration: Linux 64bit
ruby -v
ruby 1.8.6 (2007-09-24 patchlevel 111) [x86_64-linux]
important gems:
dbd-odbc (0.2.4)
dbi (0.4.1)
active record sql server adapter - as plugin
ruby-odbc 0.9996 (installed without any options.)
unixODBC is installed
freeTDS is installed
cat /etc/odbcinst.ini
[FreeTDS]
Description = TDS driver (Sybase/MS SQL)
Driver = /usr/lib/libtdsodbc.so
Setup = /usr/lib/odbc/libtdsS.so
CPTimeout =
CPReuse =
FileUsage = 1
DSN:
DRIVER=FreeTDS;TDS_Version=8.0;SERVER=XXXX;DATABASE=XXX;Port=1433;uid=XXX;pwd=XXXX;"
or
DRIVER=/usr/lib/libtdsodbc.so;TDS_Version=8.0;SERVER=XXXX;DATABASE=XXX;Port=1433;uid=XXX;pwd=XXXX;"
I receive the following error:
>>ActiveRecord::Base.sqlserver_connection({"mode"=>"ODBC", "adapter"=>"sqlserver", "dsn"=>my_dns)
DBI::DatabaseError: IM002 (0) [unixODBC][Driver Manager]Data source name not found, and no default driver specified
from /usr/lib/ruby/1.8/DBD/ODBC/ODBC.rb:95:in `connect'
from /usr/lib/ruby/1.8/dbi.rb:424:in `connect'
from /usr/lib/ruby/1.8/dbi.rb:215:in `connect'
from /opt/ublip/rails/current/vendor/plugins/activerecord-sqlserver-adapter/lib/active_record/connection_adapters/sqlserver_adapter.rb:47:in `sqlserver_connection'
It looks like ODBC unable to find appropriate ODBC driver, but I have no ideas why.
I had a problem with /usr/lib/libtdsodbc.so which is empty in default debian package free-tds dev, but i solved it with remove broken package and installation from sources.
Will appreciate any thought!
Note:
I'm albe to connect using the same steps on mac 10.5
This looks close, and I've gotten the same type of setup to work on Ubuntu 8.04 (with Ruby 1.8.6.) Since this is on Debian, it should be the similar.
These packages should be installed: unixodbc unixodbc-dev freetds-dev libdbd-odbc-ruby tdsodbc
/etc/odbcinst.ini should have something like this:
[FreeTDS]
Description = FreeTDS unixODBC Driver
Driver = /usr/lib/odbc/libtdsodbc.so
Setup = /usr/lib/odbc/libtdsodbc.so
UsageCount = 1
And /etc/freetds/freetds.conf: (where XXX is the database name and NNN... is the IP of the server):
[XXX]
host = NNN.NNN.NNN.NNN
tds version = 8.0
client charset = UTF-8
text size = 262144
port = 1433
Then to connect:
ActiveRecord::Base.sqlserver_connection({"mode"=>"ODBC",
"adapter"=>"sqlserver",
"dsn"=>"DRIVER=FreeTDS;SERVERNAME=XXX;DATABASE=XXX;uid=XXX;pwd=XXX"
)

Resources