Connect to MS SQL Server Database Using isql on Mac - sql-server

Summary: tsql connects and isql doesn't.
Details: I have tried every official installation guide and Stack Overflow question I can find, but none of them produce joy. I'm trying to connect to a MS SQL Server database from my Mac (Mohave), and ultimately a Xubuntu 16.04 virtual machine as well, using pyodbc. Before getting into the Python, I just want to trouble-shoot the connection. The problem is, most of the documentation I'm seeing is missing important details such as, "Should I use quotes in usernames and passwords?" Or, "Should I include the domain in the username?" And if I should include the domain the username, "Should I escape the backslash character?"
Here is my current setup: tsql -C produces
Compile-time settings (established with the "configure" script)
Version: freetds v1.1.11
freetds.conf directory: /usr/local/etc
MS db-lib source compatibility: no
Sybase binary compatibility: yes
Thread safety: yes
iconv library: yes
TDS version: 7.3
iODBC: no
unixodbc: yes
SSPI "trusted" logins: no
Kerberos: yes
OpenSSL: yes
GnuTLS: no
MARS: yes
The contents of /usr/local/etc/freetds.conf are:
#
# This file is installed by FreeTDS if no file by the same
# name is found in the installation directory.
#
# For information about the layout of this file and its settings,
# see the freetds.conf manpage "man freetds.conf".
# Global settings are overridden by those in a database
# server specific section
[global]
# TDS protocol version
tds version = auto
# Whether to write a TDSDUMP file for diagnostic purposes
# (setting this to /tmp is insecure on a multi-user system)
; dump file = /tmp/freetds.log
; debug flags = 0xffff
# Command and connection timeouts
; timeout = 10
; connect timeout = 10
# To reduce data sent from server for BLOBs (like TEXT or
# IMAGE) try setting 'text size' to a reasonable limit
; text size = 64512
# If you experience TLS handshake errors and are using openssl,
# try adjusting the cipher list (don't surround in double or single quotes)
# openssl ciphers = HIGH:!SSLv2:!aNULL:-DH
# A typical Sybase server
[egServer50]
host = symachine.domain.com
port = 5000
tds version = 5.0
# A typical Microsoft server
[egServer73]
host = ntmachine.domain.com
port = 1433
tds version = 7.3
[my_server]
host = my_server
port = 1433
tds version = 7.3
The my_server is the server to which I'm trying to connect.
The command tsql -S my_server -U domain\\username -P password, with no quotes anywhere, connects just fine, with the result
locale is "en_US.UTF-8"
locale charset is "UTF-8"
using default charset "UTF-8"
1>
So much for FreeTDS. For the ODBC side, first note that FreeTDS is apparently already set up for ODBC, as it says unixodbc: yes in the results of tsql -C. My /etc/odbcinst.ini file contains:
[FreeTDS]
Description=FreeTDS Driver for Linux & MSSQL
Driver=/usr/local/lib/libtdsodbc.so
Setup=/usr/local/lib/libtdsodbc.so
UsageCount=1
My /etc/odbc.ini file contains
[my_server]
Description = "The Location of the EOAM_RA Database"
Driver = FREETDS
Trace = Yes
TraceFile = /tmp/sql.log
Database = my_database
Servername = my_server
UserName = domain\\username
Password = password
Port = 1433
Protocol = 7.3
ReadOnly = No
RowVersioning = No
ShowSystemTables = No
ShowOidColumn = No
FakeOidIndex = No
But when I try any version of isql -v my_server username password, I get the dreaded and incredibly unhelpful
[S1000][unixODBC][FreeTDS][SQL Server]Unable to connect to data source
[01000][unixODBC][FreeTDS][SQL Server]Unknown host machine name.
[ISQL]ERROR: Could not SQLConnect
So, while I certainly would LOVE to have the right settings for everything so that I can connect via isql and ultimately pyodbc, it would also help greatly if I could have the answers to the following questions:
For every place where I should have my username typed in, should it have quotes around it?
For every place where I should have my username typed in, should I have it prefixed with the domain?
For every place where I should have my username typed in, if I do need my domain included, should I escape the backslash in-between?
For servers, should I include quotes?
Many thanks for your time!

Have you tried connecting after using the fully qualified domain name (fqdn) for your SQL Server in the freetds.conf file:
[my_server]
host = my_server.<yourdomain.example>
port = 1433
tds version = 7.3

Related

freetds not recognizing version from any config files

Not sure why...
I have the same conf file in /etc/freetds/freetds.conf, in /usr/share/freetds/freetds.conf and ~/.freetds.conf
# $Id: freetds.conf,v 1.12 2007/12/25 06:02:36 jklowden Exp $
#
# This file is installed by FreeTDS if no file by the same
# name is found in the installation directory.
#
# For information about the layout of this file and its settings,
# see the freetds.conf manpage "man freetds.conf".
# Global settings are overridden by those in a database
# server specific section
[global]
# TDS protocol version
tds version = 7.4
# Whether to write a TDSDUMP file for diagnostic purposes
# (setting this to /tmp is insecure on a multi-user system)
; dump file = /tmp/freetds.log
; debug flags = 0xffff
# Command and connection timeouts
; timeout = 10
; connect timeout = 10
# If you get out-of-memory errors, it may mean that your client
# is trying to allocate a huge buffer for a TEXT field.
# Try setting 'text size' to a more reasonable limit
text size = 64512
# A typical Sybase server
[egServer50]
host = symachine.domain.com
port = 5000
tds version = 5.0
# A typical Microsoft server
[egServer70]
host = ntmachine.domain.com
port = 1433
tds version = 7.0
[cdcods]
host = *my_host*
port = 1433
tds version = 7.4
Where my_host is my actual mssql host. I'm connecting to a MSSQL 2012 so 7.4 is correct I think.
The command I'm using is $tsql -S cdcods -U 'my_domain\my_user' -P 'my_pass'
Here's the log:
log.c:196:Starting log file for FreeTDS 0.91
on 2017-02-03 13:20:19 with debug flags 0x4fff.
iconv.c:330:tds_iconv_open(0x15749f0, UTF-8)
iconv.c:187:local name for ISO-8859-1 is ISO-8859-1
iconv.c:187:local name for UTF-8 is UTF-8
iconv.c:187:local name for UCS-2LE is UCS-2LE
iconv.c:187:local name for UCS-2BE is UCS-2BE
iconv.c:349:setting up conversions for client charset "UTF-8"
iconv.c:351:preparing iconv for "UTF-8" <-> "UCS-2LE" conversion
iconv.c:391:preparing iconv for "ISO-8859-1" <-> "ISO-8859-1" conversion
iconv.c:394:tds_iconv_open: done
net.c:207:Connecting to *my_host* port 1433 (TDS version 4.2)
net.c:272:tds_open_socket: connect(2) returned "Operation now in progress"
net.c:312:tds_open_socket() succeeded
util.c:156:Changed query state from DEAD to IDLE
login.c:572:NT login not support using TDS 4.x or 5.0
util.c:156:Changed query state from IDLE to DEAD
util.c:331:tdserror(0x15749c0, 0x15749f0, 20002, 0)
util.c:361:tdserror: client library returned TDS_INT_CANCEL(2)
util.c:384:tdserror: returning TDS_INT_CANCEL(2)
mem.c:615:tds_free_all_results()
You're running FreeTDS v0.91 which does not support TDS version 7.4. IIRC the highest it supports is 7.2. Trying using version 7.2.
If you want to use TDS version 7.3, you'll need at least FreeTDS 0.95, and if you want 7.4, you'll need FreeTDS 1.0.
I have a PR to clarify this, but it hasn't been pushed out to the documentation.
Good luck!

Connect to SQL Server using SQLAlchemy

I'm trying to connect to a SQL Server Express database using SQLALchemy and pyodbc, but I'm continuously getting the error:
(pyodbc.Error) ('IM002', '[IM002] [unixODBC][Driver Manager]Data
source name not found, and no default driver specified (0)
(SQLDriverConnect)')
And I really don't understand if my engine url is wrong or what else.
My scenario is the following:
I'm on a Mac
I have a docker container (based on a Debian image with unixodbc and unixodbc-dev) in which my python app tries to connect to...
a virtualbox virtual machine running windows 8 with SQL express 2014...
I configured a user for the SQL express, with SQL Server authentication:
user: ar_user
password: ar_psw
...then:
I configured TCP ports as 1433 and disabled dynamic ports (SQL Server Configuration Manager > Network Configurations > Protocols).
I turned off Windows Firewall.
I used an Host-only adapter for the VM running windows8
now...
The VM is accessible from the host (my mac), since a:
ping -c 3 vm-ip
succeed!
But although I tried every possible permutation of user, password, ip, server name and port:
'mssql+pyodbc://ar_user:ar_psw#vm-ip/master'
'mssql+pyodbc://ar_user:ar_psw#vm-ip:1433/master'
'mssql+pyodbc://IE10WIN8\\SQLEXPRESS'
'mssql+pyodbc://ar_user:ar_psw#IE10WIN8\\SQLEXPRESS'
'mssql+pyodbc://ar_user:ar_psw#IE10WIN8\\SQLEXPRESS:1433'
'mssql+pyodbc://ar_user:ar_psw#IE10WIN8\\SQLEXPRESS:1433/master'
...and many more!
I always get the "datasource not found error".
What should I do?
ps: the vm is pingable even in the docker container!
UPDATE (solved but not 100%):
I solved in this way:
I configured FreeTDS driver using /etc/odbcinst.ini in this way:
[FreeTDS]
Description = TDS driver (Sybase/MS SQL)
Driver = /usr/lib/x86_64-linux-gnu/odbc/libtdsodbc.so
Setup = /usr/lib/x86_64-linux-gnu/odbc/libtdsS.so
client charset = UTF-8
and in /etc/freetds/freetds.conf:
[global]
tds version = 7.3
client charset = UTF-8
Then I created the engine using the following string:
'mssql+pyodbc://my_user:my_psw#machine_ip:1433/my_db?driver=FreeTDS'
It seems to work properly, but I get this warning:
SAWarning: Unrecognized server version info '95.12.255'. Version
specific behaviors may not function properly. If using ODBC with
FreeTDS, ensure TDS_VERSION 7.0 through 7.3, not 4.2, is configured in
the FreeTDS configuration.
I also defined the TDS version using environment variables but it doesn't fix the issue... any idea?
I wrote a tutorial here of how to do this. Essentially, you need to:
brew install unixodbc
brew install freetds --with-unixodbc
Add the freetds driver to odbcinst.ini
Add a DSN (Domain Source Name) to odbc.ini named "MY_DSN"
pip install pyodbc
e = create_engine("mssql+pyodbc://username:password#MY_DSN")
The walkthrough here does a much more thorough job of explaining this, including issues with SQL Server/FreeTDS Protocol Version Compatibility.

tsql connects to ODBC but Perl DBI gets an error

Client is Ubuntu 14.04.1 LTS talking to an MS SQL Server 2005. Ubuntu has no registered domain. It's inside the local network as is the MS SQL Server.
Perl 5.18.2 using DBI 1.63 and DBD::ODBC 1.52.
Installed FreeTDS 0.92 (with TDS v4.2) and unixODBC 2.2.14.
isql DSN USERNAME PASSWD at the command line also doesn't work. Perhaps I should start with that. I still get the same error which is Unable to connect to data source (SQL-08001).
My related config files:
2015-07-09 11:55:14 AM
NOTE: TDS version from tsql -C must go into freetds.conf and odbc.ini files
=========================================
/etc/odbcinst.ini:
[FreeTDS]
Description = FreeTDS 0.91 for Microsoft SQL Server 2005
#Driver = /usr/lib/odbc/libtdsodbc.so
Driver = /usr/lib/x86_64-linux-gnu/odbc/libtdsodbc.so
Setup = /usr/lib/x86_64-linux-gnu/odbc/libtdsS.so
UsageCount = 1
#Threading = 2
fileusage = 1
dontdlclose = 1
=========================================
/etc/odbc.ini:
[ODBC Data Sources]
ResponseBDO = Response database desc in /etc/odbc/ini
[FreeTDS]
Description = FreeTDS 0.91 driver
Driver = /usr/lib/x86_64-linux-gnu/odbc/libtdsodbc.so
[responsebdo]
Driver = FreeTDS
Description = Response MSSQL 2005 ODBC driver
SERVER = sqlsvr.mydomain.com
PORT = 1433
USER = 'domain\domainuser'
Password = mypasswd
Database = r4w_004
# TDS_VERSION should be same as reported by 'tsql -C'.
TDS_Version = 4.2
[Default]
Driver = /usr/lib/x86_64-linux-gnu/odbc/libtdsodbc.so
Description = Default FreeTDS ODBC driver
SERVER = 10.20.30.40
USER = "domain\domainuser"
Database = r4w_4
=========================================
Relevant part of /etc/freetds/freetds.conf:
# $Id: freetds.conf,v 1.12 2007/12/25 06:02:36 jklowden Exp $
#
# This file is installed by FreeTDS if no file by the same
# name is found in the installation directory.
#
# For information about the layout of this file and its settings,
# see the freetds.conf manpage "man freetds.conf".
#
[responsetds]
#host = 10.20.30.40
host = sqlsvr.mydomain.com
port = 1433
tds version = 4.2
connect timeout = 20
timeout = 20
dump file = /tmp/freetds-resp.log
instance = MSSQLSERVER
use ntlmv2 = yes
=========================================
tsql -C:
Compile-time settings (established with the "configure" script)
Version: freetds v0.91
freetds.conf directory: /etc/freetds
MS db-lib source compatibility: no
Sybase binary compatibility: yes
Thread safety: yes
iconv library: yes
TDS version: 4.2
iODBC: no
unixodbc: yes
SSPI "trusted" logins: no
Kerberos: yes
=========================================
odbcinst --version:
unixODBC 2.2.14
Perl connection info:
$respdsn="dbi:ODBC:Driver={SQL Native Client};Server=10.20.30.40;".
"UID=$respdbuser;PWD=$respdbpw;Database=r4w_004";
$respdbh=DBI->connect($respdsn);
I can connect to the database by using tsql at the command line.
But using Perl I get this error:
DBI connect('Driver={SQL Native Client};Server=10.19.1.3;UID=gilson\mwreports;PWD=MR4gilson;Database=r4w_004','',...) failed:
[unixODBC][Driver Manager]Data source name not found, and no default
driver specified (SQL-IM002) at /home/chuck/perl/gilson/jimv/fedex/fedex.pl line 1557.
at /usr/lib/perl5/DBI.pm line 691.
DBI::__ANON__[/usr/lib/perl5/DBI.pm:744](undef, undef) called at /usr/lib/perl5/DBI.pm line 747
My questions:
Is the "{SQL Native Client}" in the Perl connection string correct? Does it refer to an entry in the odbc.ini or freetds.conf file?
What am I missing here?
Thank you.
If you're using FreeTDS, then your Driver is FreeTDS not "Sql Native Client".
Try this line to get your database handle:
my $dbh = DBI->connect("dbi:ODBC:Driver=FreeTDS;Server=10.20.30.40;database=r4w_004", $respdbuser, $respdbpw);
I had the same problem, when I using tsql works normally but using perl with dbi didn't. I lost many hours trying to figure out the problem, so I saw the limitations of Adaptive Server:
The name cannot be more than 30 characters long.
The first character must be a letter (ASCII a through z, A through Z).
The characters that follow must be letters, numbers, or underscores (_).
... So I tried to change the password that was 32 characters long to 30 characters and it works.
The tsql accept long password but perl dbi sybase accept ony 30 characters.
Command to connect to default instance on port 1433:
perl -e 'use DBI;DBI->connect("dbi:Sybase:server=10.0.0.1","sa","max_30_characters_longpassword");'
Command to connect to named instance (use port udp 1434 to discover the port):
perl -e 'use DBI;DBI->connect("dbi:Sybase:server=10.0.0.1\\SQL2014","sa","max_30_characters_longpassword");'
Command to connect to named instance when you know the port that sqlserver is listening:
perl -e 'use DBI;DBI->connect("dbi:Sybase:server=10.0.0.1:60036","sa","max_30characters_long_password");'
I used the tcpdump to monitoring the connection to the sqlserver:
tcpdump -qn dst host 10.0.0.1 &
Maybe it helps someone. Thanks

RODBC MS SQL access from Ubuntu using FreeTDS

I'm trying to access MS SQL server hosted on my Windows box from an Ubuntu machine. I have a shiny app that access MSSQL that works fine on Windows. I want to host this with shiny server on Ubuntu so that others can access the webpage and provide their SQL server as input to the app.
All of this works fine on Windows. Now I'm not able to get unixODBC working. I think I've got the odbc/freeTDS installation and configuration correct. I'm able to connect and query SQL DB instance from Ubuntu terminal using tsql -S . It took some googling but eventually it worked.
Now, when I try to connect from R I get this error.
sql <- odbcConnect("abc.xyz.com", "uname", "passwd")
In odbcDriverConnect("DSN=abc.xyz.com;UID=uname;PWD=passwd")
: [RODBC] ERROR: state 08001, code 0, message
[unixODBC][FreeTDS][SQL Server]Unable to connect to data source 2: In
odbcDriverConnect("DSN=abc.xyz.com;UID=uname;PWD=passwd")
: [RODBC] ERROR: state 01000, code 20002, message
[unixODBC][FreeTDS][SQL Server]Adaptive Server connection failed 3: In
odbcDriverConnect("DSN=abc.xyz.com;UID=uname;PWD=passwd")
: [RODBC] ERROR: state 01000, code 20017, message
[unixODBC][FreeTDS][SQL Server]Unexpected EOF from the server 4: In
odbcDriverConnect("DSN=abc.xyz.com;UID=uname;PWD=passwd")
: ODBC connection failed
freeTDS.conf
[abc.xyz.com]
host = abc.xyz.com
port = 49475
tds version = 8.0
odbcinst.ini
[FreeTDS]
Description = FreeTDS unixODBC Driver
Driver = /usr/local/lib/libtdsodbc.so
Setup = /usr/local/lib/libtdsodbc.so
odbc.ini
[abc.xyz.com]
Description = Shiny testing
Driver = FreeTDS
Trace = No
Server = abc.xyz.com\instance_name
Database = dbanme
port = 49475
This error "Unexpected EOF from the server" is not new to me. I got the same error when I was trying to connect using tsql. I was able to overcome this by adding "tds version = 8.0" line to freeTDS.conf. Not sure how I can get RODBC to use this config. I've read posts were other users were able to sql-server using freeTDS. Not sure what is missing here. I also tried re-installing RODBC.
I fixed this. TDS version was needed in odbc.ini file.
TDS_Version = 8.0
tsql reads version info from freeTDS.conf and worked. isql was failing with same error and it was also looking for this config in odbc.ini. So test with isql if you are configuring for R/Python.
I don't know what is causing the error on your side. Hope this helps:
These are my settings for RODBC 1.3-10 to connect to MSSQL Server 2012. I am also on ubuntu and this works for me.
I think I installed the RODBC package from the Ubuntu repos:
apt-cache policy r-cran-rodbc
r-cran-rodbc:
Installed: 1.3-10-1
Candidate: 1.3-10-1
Version table:
*** 1.3-10-1 0
500 ftp://ftp.fu-berlin.de/linux/ubuntu/ trusty/universe amd64 Packages
100 /var/lib/dpkg/statu
uname -a
Linux xxx 3.13.0-30-generic #54-Ubuntu SMP Mon Jun 9 22:45:01 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
odbc.ini
[mydsn]
APP = unixodbc
Description = master on vmXX
Driver = TDSdriver
Server = vmXX
Database = master
Port = 1433
#Trace = No
#TraceFile = /var/log/freetds/freetds--odbc.log
TDS Version = 7.2
freetds.conf
[global]
# TDS protocol version
; tds version = 4.2
# Whether to write a TDSDUMP file for diagnostic purposes
# (setting this to /tmp is insecure on a multi-user system)
#dump file = /tmp/freetds.log
#debug flags = 0xffff
# Command and connection timeouts
; timeout = 10
; connect timeout = 10
# If you get out-of-memory errors, it may mean that your client
# is trying to allocate a huge buffer for a TEXT field.
# Try setting 'text size' to a more reasonable limit
text size = 64512
[vmXX]
host = 111.222.333.555
port = 1433
tds version = 7.2
client charset = UTF-8
Output from
ps aux | grep "/R"
knb 56969 ....
lsof -p 56969 | grep -i odbc
R 56969 knb mem REG 8,1 72408 8528592 /usr/lib/x86_64-linux-gnu/libodbcinst.so.1.0.0
R 56969 knb mem REG 8,1 400608 8521896 /usr/lib/x86_64-linux-gnu/odbc/libtdsodbc.so
R 56969 knb mem REG 8,1 419680 8525415 /usr/lib/x86_64-linux-gnu/libodbc.so.1.0.0
R 56969 knb mem REG 8,1 96845 8933205 /usr/local/lib/R/site-library/RODBC/libs/RODBC.so
I had a similar error and realized R was just unable to read my user id correctly as it had a slash in it. I placed my credentials in a text file which solved the problem for me.
cred <- "/myPath/ODBC_cred.txt"
sql <- RODBC::odbcConnect("abc.xyz.com", uid=readLines(cred)[1], pwd=readLines(cred)[2])

Issue with returning Cyrillic symbols from MSSQL via unixODBC and FreeTDS

I'm using django-pyodbc as database backend on Ubuntu 12.04 LTS and MSSQL 2008 on remote host. It works good except of returning Cyrillic symbols. Instead of them I see question marks - '?'. I have begin investigation what could cause this problem.
As far as I understand MSSQL-django chain looks so:
MSSQL <-> FreeTDS <-> unixODBC <-> pyodbc <-> django-pyodbc
So I have started from FreeTDS. When I run query in tsql - it works good I can see all symbols including Cyrillic.
The next one was isql - as far as I understand there I can test FreeTDS <-> unixODBC pair.
And there I didn't get proper data. In fact when I run query in isql columns that contain Cyrillic symbols are empty or consist of not visible symbols. I guess that problem in communication between FreeTDS <-> unixODBC. What could cause this problem? Btw, I have also tried iusql - nothing have changed.
MSSQL collation is Cyrillic_General_CI_AS.
Content of freetds.conf:
[global]
tds version = 4.2
dump file = /tmp/freetds.log
debug flags = 0xffff
timeout = 10
connect timeout = 10
client charset = UTF-8
text size = 64512
[egServer50]
host = symachine.domain.com
port = 5000
tds version = 5.0
[egServer70]
host = ntmachine.domain.com
port = 1433
tds version = 7.0
[rfxdigest]
host = mssql-iis-1
port = 1433
tds version = 8.0
client charset = UTF-8
Content of odbc.ini:
[RFX]
Description = Rfx digest server
Driver = FreeTDS
Database = RFXDB
Servername = rfxdigest
TDS_Version = 8.0
Edit1 15.08.12
In python using pyodbc I get '?' instead of Cyrillic symbols - I have tried both python versions: UCS2 and UCS4.
You get a ? returned for non-printable characters
Run the command below to see what unicode your Python setup supports :-
python -c "import sys;print(sys.maxunicode<66000)and'UCS2'or'UCS4'"
Next will need to set FreeTDS to use the same character set as Python. If FreeTDS does not support the unicode format you are using in Python you will need to change both Python and FreeTDS.
To rebuild Python from source with UCS2 enabled you need to do something like :-
$ ./configure --enable-unicode=ucs2
$ make
$ sudo make install
Ok, I have made all this modules chain work:
MSSQL <-> FreeTDS <-> unixODBC <-> pyodbc <-> django-pyodbc
I just have added 'unicode_results':True in DATABASES options in django settings:
DATABASES = {
'default': {
'ENGINE': 'sql_server.pyodbc', # Add 'postgresql_psycopg2', 'postgresql', 'mysql', 'sqlite3' or 'oracle'.
'NAME': 'name', # Or path to database file if using sqlite3.
'USER': 'user', # Not used with sqlite3.
'PASSWORD': 'pwd', # Not used with sqlite3.
'HOST': 'server-name', # Set to empty string for localhost. Not used with sqlite3.
'PORT': 'port', # Set to empty string for default. Not used with sqlite3.
'OPTIONS': {
'unicode_results':True,
'driver': 'FreeTDS',
'host_is_server': True,
'extra_params': 'TDS_VERSION=8.0'
}
But pyodbc and isql still doesn't work correctly - maybe I have missed other unicode-specific parameters. Going to check how odbc and pyodbc use this unicode_results parameter later. Anyway site now able to show Cyrillic symbols.
I was fighting with problem with cyrillic letters over week. I find diffrent solution, I just use microsoft mssql drivers for Linux, here is great how to install it on other than supported distributions (Suse, redhat): https://groups.google.com/forum/#!topic/shiny-discuss/AyFthz3UGwg
This driver returns normal utf-8, and everything working out of package.

Resources