Below script (fx is SQL Server's table):
LIBNAME SQL ODBC DSN='sql server' ;
DATA new;
SET SQL.fx;
WHERE repo_date = '2016-04-29 00:00:00.000';
RUN;
PROC PRINT DATA=new;
RUN;
returns me an error (in SAS log):
191 WHERE repo_date = '2016-04-29 00:00:00.000';
ERROR: WHERE clause operator requires compatible variables.
192 RUN;
Where can I check which data conversion I need (in this case and in others)?
In SQL Server 2008R2 repo_date is a datetime column.
You are comparing a string to a numeric value.
So your datetime-format is wrong (like Heinzi mentioned), and also you have to convert it to a datetime value (by adding a dt at the end)
Working should this:
WHERE repo_date ='29APR2016 00:00:00.000'dt;
If repo_time is datetime and the time is not relevant, you can just compare the date:
WHERE datepart(repo_date) = '29APR2016'd;
SAS is using the ODBC libname engine to translate SAS data step code into SQL code. Because you're writing it in SAS, SAS is assuming that you are looking for the string 2016-04-29 00:00:00.000. Instead, you want to put it in a SAS date literal so SAS knows how to translate the data.
LIBNAME SQL ODBC DSN='sql server' ;
DATA new;
SET SQL.fx;
WHERE repo_date = '29APR2016:00:00:00'dt;
RUN;
PROC PRINT DATA=new;
RUN;
If you were doing SQL passthrough to directly run SQL on the server, then your above code would work.
proc sql noprint;
connect to odbc(datasrc='sql server');
create table new as
select * from connection to odbc
(select * from schema.fx where repo_date='2016-04-29 00:00:00.000');
disconnect from odbc;
quit;
Basically, what the above is doing is having the SQL server pull the columns, then SAS simply pulls it all over to itself. Think of it as using SAS as a proxy program to run commands directly on the SQL server.
Related
I have a table stored in SQL Server Management Studio version 15.0.18369.0 and I have a working and established ODBC connection to SAS language program World Programming Software version 3.4.
Previous import/reading of this data has been successful but the operator of the SQL server may have recently converted their data type to NVARCHAR(max). It may also be due to a driver change (I got a new laptop and reinstalled what I thought was the exact same OBDC for SQl driver as I had before but who knows) I have 64-bit ODBC Driver 17 for SQL server.
The VARCHAR(max) type in SQL causes the data to only be 1 character long in every column.
I have tried to fix it by:
Adding the DB max text option to libname
libname odbclib odbc dsn="xxx" user="xxx" password="xxx" DBMAX_TEXT=8000;
This did nothing so I also tried to add DB type option:
data mydata (dbtype=(mycol='char(25)')) ;
set odbclib.'sql data'n
run;
And I get ERROR:
The option dbtype is not a valid output data set option.
I have also tried DBSASTYPE, and putting both options in the set statement and this yields the same error.
I also tried with proc SQL:
proc sql noprint;
CONNECT TO ODBC(dsn="xxx" user="xxx" password="xxx"); create table
extract(compress=no dbsastype=(mycol='CHAR(20)')) as select * from
connection to odbc ( select * from dbo.'sql data'n
); disconnect from odbc; quit;
And I get
NOTE: Connected to DB: BB64 (Microsoft SQL Server version 12.00.2148)
NOTE: Successfully connected to database ODBC as alias ODBC. 3915
create table extract(compress=no
dbsastype=(mycol='CHAR(20)')) as 3916 select *
from connection to odbc 3917 ( 3918 select *
from dbo.'sql data'n ERROR: The option dbsastype is not a valid
output data set option 3919 3920 ); 3921
disconnect from odbc; NOTE: ERRORSTOP was specified. The statement
will not be executed due to an earlier error NOTE: Statements not
executed because of errors detected 3922 quit;
One thing to try would be putting the DBTYPE or DBSASTYPE in the right place.
data mydata;
set odbclib.'sql data'n(dbtype=(mycol='char(25)'));
run;
It is an option that would go on the "set" not the "data" line.
I would also encourage you to contact the developers, as it seems like this is a bug in their implementation; or perhaps try a different ODBC SQL driver.
Maybe a workaround in the meanwhile is to use CONVERT in the passthrough?
proc sql noprint;
CONNECT TO ODBC(dsn="xxx" user="xxx" password="xxx"); create table
extract as select * from
connection to odbc ( select a,b,c,convert(NCHAR(20),mycol) from dbo.'sql data'n
); disconnect from odbc; quit;
I'm pulling data into SAS from a SQL server database with a (SAS DI) extract transformation. The data in the table goes back several years and has a little over 16 million rows; I only need data for the last few years which should amount to roughly 2.6 million rows.
So my extract uses a datetime filter. The autogenerated proc sql looks as follows (after obfuscating the libref, table, and column names):
%let SYSLAST = %nrquote(LIBREF.SRC_TBL);
proc sql;
create table work.WFJVX4PU as
select
col1,
col2,
dt_col,
col3
from &SYSLAST
where dt_col >= &start_dt
;
quit;
The code works and returns the desired rows, but it takes far too long to run when compared with executing a similar query directly on the SQL server. When inspecting the statistics of the execution I discovered that the entire table was brought into SAS before the where clause was applied.
I remember reading that this will occur when attempting to use filters that apply functions to source data (a la datepart(dt_col) >= input("&start_date",date9.)) which is why I tried to pass the datetime used by the filter directly to the SQL engine. I also tried the "datepart" approach and got the same result.
Is there something else I should be doing to apply the filter server side before bringing the data into SAS? This isn't the experience I've had in the past when working with other database tables (e.g. Teradata, MySQL, Oracle, etc).
Additional Details:
The macro variable start_date is defined using the SAS DI prompt creation tool; the auto-generated code for this is
%let start_date_label = 730 days ago (July 02, 2017);
%let start_date_rel = D-730D;
%let start_date = 02Jul2017;
I then create the datetime macro variable start_dt by executing the following in the precode of the job:
%let start_dt = %sysfunc(dhms("&start_date"d,0,0,0));
Below is the libname statement (after obfuscation):
LIBNAME MYLIB SQLSVR CONNECTION=SHARED PRESERVE_TAB_NAMES=YES dbconinit="set ansi_warnings off;use MY_DB_NAME;set nocount on;" Datasrc=MY_DATA_SRC SCHEMA=dbo AUTHDOMAIN="my_sql_authentication" ;
If I modify the where clause to use a literal of the form '2019-06-25' then SAS throws the error
ERROR: Expression using greater than or equal (>=) has components that
are of different data types.
because the dt_col field is of type numeric (format datetime22.3) on the SAS instance of the SQL server table that is registered from the library. If I use a literal as follows:
dt_col >= '25JUN2019'D
then I do get the desired set of rows (despite comparison between a datetime field and a date literal), but the query still takes a long time to execute and the job statistics indicate SAS is still grabbing all 16 million rows to perform this task.
UPDATE
I'm having issues following Tom's advice below. If I execute the following code:
LIBNAME MYLIB SQLSVR CONNECTION=SHARED PRESERVE_TAB_NAMES=YES dbconinit="set ansi_warnings off;use MYDB;set nocount on;" Datasrc=MYSRC SCHEMA=dbo AUTHDOMAIN="my_sql_authentication" ;
/*---- Map the columns ----*/
proc datasets lib = work nolist nowarn memtype = (data view);
delete sql_psthru2;
quit;
proc sql;
create table work.sql_psthru2 as
select
col1,
col2,
dt_col,
col3
from MYLIB.MYTBL
where dt_col>= '25JUN2019'd
;
quit;
then I get data from the database, but if I execute
LIBNAME MYLIB SQLSVR CONNECTION=SHARED PRESERVE_TAB_NAMES=YES dbconinit="set ansi_warnings off;use MYDB;set nocount on;" Datasrc=MYSRC SCHEMA=dbo AUTHDOMAIN="my_sql_authentication" ;
proc sql;
connect using MYLIB;
create table work.sql_psthru as
select * from connection to MYLIB
(select
col1
,col2
,dt_col
from MYTBL
where dt_col >= '2019-06-25'
)
;
quit;
then I receive the error
ERROR: CLI error trying to establish connection: [DataDirect][ODBC
lib] Data source name not found and no default driver specified
immediately after the connect using MYLIB; line.
I've tried many variants of the explicit passthru which I've found all over the internet which I won't post here, but none worked.
An interesting side note is that I believe the reason the SAS DI statistics are indicating that all 16 million rows are returned is that the extract transformation auto-generates the following macro:
%macro etls_recordCheck;
%let etls_recCheckExist = %eval(%sysfunc(exist(MYLIB.MYTBL, DATA)) or
%sysfunc(exist(MYLIB.MYTBL, VIEW)));
%if (&etls_recCheckExist) %then
%do;
proc sql noprint;
select count(*) into :etls_recnt from MYLIB.MYTBL;
quit;
%end;
%mend etls_recordCheck;
%etls_recordCheck;
So the culprit in the long execution time is not that the full dataset is being returned to SAS (I removed the macro and the code still takes far too long to run).
You could try explicitly writing the code in the remote database. So if you already have a libref named LIBREF defined you can use that in the CONNECT statement in PROC SQL.
proc sql;
connect using libref ;
create table WFJVX4PU as
select * from connection to libef
(select
col1
,col2
,dt_col
,col3
from SRC_TBL
where dt_col >= &start_dt
)
;
quit;
Just make sure everything inside the () is valid syntax for that database system. Included the values of the macro variable START_DT.
It seemed to be an easy task but I fail and do not find a solution for my problem: I have a local table in Access 2010 with a date/time column and I want to update a column in a SQL Server table with a datatype date.
Sending the date/time values direct to the SQL Server table fails, same with converting the date/time column with this VBA function:
Function DateForSQL(dteDate) As String
DateForSQL = "'" & Format(CDate(dteDate), "yyyy-mm-dd") & "'"
End Function
which gives
DateForSQL(Date()) = '2016-01-14'
and should work, I assumed.
The update command is this:
UPDATE SQL_table
INNER JOIN local_table ON SQL_table.ID = local_table.ID
SET SQL_table.DateField = DateForSQL(local_table.DateField)
But it fails again in Access with a type conversion error.
Even when changing the SQL Server table column to datetime I get the same error.
Same with sending to SQL a string like '14/01/2016' or '01/14/2016'.
The only thing I could do - eventually - is to change the datetime to text in Access and try again, but this could not be the only solution.
Any help?
Thanks
Michael
First of all, I recommend to use the ISO-8601 format for your date - which is YYYYMMDD (no dashes, nothing) - this works for all regional & language settings in SQL Server.
Next, I'm not sure about Access' SQL syntax, but in SQL Server, your UPDATE statement would be to be something like this:
UPDATE sql
SET sql.DateField = DateForSQL(local_table.DateField)
FROM local_table local
INNER JOIN SQL_table sql ON local.ID = sql.ID
First UPDATE, then SET, then FROM and INNER JOIN ...
I'd like to insert a date field into a SQL server table form Proc SQL in SAS. Here is my code for Proc SQL:
proc sql;
insert into CFS_SQL.Data_DSB_Raw(sasdatefmt=(TheDate='mmddyy10.'))
select TheDateIncoming
from Work.Upload;
quit;
According to the SAS help documentation (http://support.sas.com/kb/6/450.html), this should work as long as TheDateIncoming also has format mmddyy10.. I've verified that the format on TheDateIncoming is correct, so I think this should work.
Unfortunately, however, I'm getting a "Value 1 on the SELECT clause does not match the data type of the corresponding column" error.
Any thoughts?
Annnnnd... solved. It actually had nothing to do with the code. It was a driver problem. Switching to the SQL Server Native Client 11.0 ODBC driver fixed the issue.
I need to be able to specify the schema that I want to access in SAS. I have used a connection string with the following schema=?? but SAS will not let me select or print the contents of any object in the named schema. Has anyone been able to write a PROC SQL statement selecting objects in a schema other than dbo?
Thank you,
SAS does not use fully quallified names from the SQL server but you can direct SAS to a specific schema. The following is an example that uses a libname as connection to a 2008 SQL Server.
proc print data=myDBconn.v_Lots (SCHEMA=SAS);
WHERE Study_ID IS NOT NULL;
run;
proc print data=myDBconn.Drugs (SCHEMA=Pharmacy);
where _drug_id=1;
run;
proc sql;
create table myTest.drugs as ;
(SELECT * FROM myDbconn.drugs (SCHEMA=Pharmacy));
quit;