The following instruction aims at using a specific format to import DATEs
alter session set DATE_INPUT_FORMAT = 'YYYY-MM-DD HH24:MI:SS.FF';
However, it seems to have no effect on the following:
copy into schema.table
from s3://bucket/file.parquet
credentials=(aws_key_id='...' aws_secret_key='...')
match_by_column_name=case_insensitive
file_format=(type=parquet);
Which results in errors like below:
sqlalchemy.exc.ProgrammingError: (snowflake.connector.errors.ProgrammingError) 100071 (22000):
Failed to cast variant value "2020-06-16 00:00:00.000" to DATE
When a column in the imported Parquet file has a format as specified above for a date field.
This really sounds like a bug, as the above COPY INTO scenario should in theory be a typical use case for altering the DATE_INPUT_FORMAT.
Is there a way to address this?
The DATE_INPUT_FORMAT should affect the copy command. The documentation talks about not supporting a timestamp from a variant column on a date conversion.
Although TO_DATE accepts a TIMESTAMP value, it does not accept a TIMESTAMP inside a VARIANT.
Related
I have a String field in a Dataset in (mmddyyyy) format.
I am trying to convert it into a Date field in SSRS.
I already tried using the below command but I am getting error.
CDate(Fields!LocalTXNDate.Value)
Can anyone please suggest.
While Larnu is correct, the way to do it is to correct the database, sometimes we lowly report makers have no say in making these changes - much less getting a DBA to do it in a reasonable amount of time.
If you can't change the data to be correct, the easiest way to convert and use the field as a date is to add a Calculated Field to the Dataset. Open the dataset properties, click on the Fields tab, Add a Calculated field.
For the Expression, use string functions to parse the field into a generic date format and then CDATE to convert to a date type. Then use the new field for dates. You could also use this in your text box if it's not being reused but it's easier to manipulate the Calculated field.
=CDATE(
RIGHT(Fields!LocalTXNDate.Value, 4) & "-" &
LEFT(Fields!LocalTXNDate.Value, 2) & "-" &
MID(Fields!LocalTXNDate.Value, 3, 2)
)
The problem here isn't SSRS but your data, and that you are using a string based data type to store the data. You need to fix the problem at the source, not at the report level.
The string format you have chosen, MMddyyyy isn't a format that is recognised by default in any of the languages in SQL Server, nor if you explicitly use SET DATEFORMAT, nor does it appear as a style. SET DATEFORMAT MDY; SELECT CONVERT(date,'11172022'); will fail. Therefore you'll need to first do some string manipulation on the data first to be an unambiguous format (yyyyMMdd):
UPDATE YT
SET YourDateColumn = CONVERT(varchar(8),V.DateValue,112)
FROM dbo.YourTable YT
CROSS APPLY (VALUES(TRY_CONVERT(date,CONCAT(RIGHT(YT.YourDateColumn,4),LEFT(YT.YourDateColumn,4)))))V(DateValue);
For any bad values you have, such as '17112022' this will UPDATE the value to NULL; as such you may want to create a new column for the new value, or perhaps a new column to store the value of dates that couldn't be converted.
After you've changed the value to an unambiguous format, then you can ALTER the column:
ALTER TABLe dbo.YourTable ALTER COLUMN YourDateColumn date NULL;
Note that if you have any constraints, you will need to DROP those first, and then reCREATE them afterwards.
Now that the data type of the column is correct, you need not do anything in SSRS, as the data type is correct.
My local SQL Server 2016 setup at work decided not to accept the YMD date format after going through a reinstall. For example, the following query, that was and still is accepted in my coworkers' setups:
SELECT "id"
FROM test.dbo.tabEmp
WHERE "DateAdmission" <= '2021-12-31' AND "DateAdmission">= '2021-12-30' `
When I try to run it I see this:
The conversion of a varchar data type to a datetime data type resulted in an out-of-range value
however, if i rewrite the dates as 2021-31-12 and 2021-12-30, in the YYYY-DD-MM format, they are accepted.
I can't really convert or format it since the sql queries in our system are numerous and done so in a way that it would be nearly impossible to. Is there something that can be done? I tried changing windows' Date format but to no avail.
For the datetime and smalldatetime data types the format yyyy-MM-dd is not unambiguous (note that it is for the newer date and time data types). If you are not American, the date will very likely be interpreted as yyyy-dd-MM, and as there are not 31 months in the year you get an error.
For SQL Server, the formats that are unambiguous regardless of data type and language setting are yyyyMMdd and yyyy-MM-ddThh:mm:ss.nnnnnnn; ideally if you are using string literals use one of those formats as you can never get an error (unless you legitimately have an invalid date).
Otherwise you can explicitly CONVERT your value with a style code:
SELECT CONVERT(datetime, '2021-12-31', 126);
It seems that your new DB instance picked up a new language after the reinstallation.
The current language setting determines the language used on all system messages, as well as the date/time formats to use.
The date format setting affects the interpretation of character strings as they are converted
to date values for storage in the database. It does not affect the display of date data type values
that are stored in the database or the storage format.
You can run the following statement to return the language currently being used:
SELECT ##LANGUAGE;
This will tell us what the current language is and the date format (as well as a few other things):
DBCC USEROPTIONS;
Date format is modifiable via the following statements:
SET LANGUAGE us_english;
SET DATEFORMAT YMD;
Here is a good article on the subject: How to Change the Current Date Format in SQL Server (T-SQL)
It is also possible to modify SQL Server instance default language globally, once and for all: How to change default language for SQL Server?
I need to make the FORM_YEAR the year of a derived column. The derived column is called Invoice_Document_Date and its data type is DateTime so I only want to grab the year. For my expression I wrote YEAR(dc_Invoice_Document_Date) but that doesn't seem to be right. What am I missing that will make this expression successful?
Make sure that the column SSIS data type is not DT_DBTIMESTAMPOFFSET or DT_DBTIMESTAMP2. Since in the official documentation they mentioned that:
The expression fails to validate when a date literal is explicitly cast to one of these date data types: DT_DBTIMESTAMPOFFSET and DT_DBTIMESTAMP2.
You can try to convert the dc_Invoice_Document_Date to DT_DATE or DT_DBDATE or DT_DBTIMESTAMP data types. As example:
YEAR((DT_DBTIMESTAMP)[dc_Invoice_Document_Date]))
You can also use DATEPART() function as follows:
DATEPART("yy",[dc_Invoice_Document_Date]))
I'd like to change display format of all dates/datetimes among entire instance/database (whichever is possible).
I tried changing default language for the instance and for single users and it doesn't work. It always displays YYYY-MM-DD. Can this be changed without messing with the code to always include FORMAT function?
Use the following function and write the format you want
SELECT FORMAT(GetDate(), 'yyyy-MM-dd')
Is there any way to check Date(like isDate function in TSQL) column in SSIS package derived column expression after extraction from Sourcefile before loading to target dtabase?
Thanks
there is no built in function but you can run a script task and use vb.net code to check if the column is a date and operate on it as you wish...
I had a similar issue. I had a date/time in a text file, but the field had a dash between the date and the time. I created a derived column to do a replace on the dash:
REPLACE([TimeField], "- ", "")
I then added a convert column to convert the field to a date. I chose to ignore errors. I then added another Derived Column to check if the converted field was NULL, indicating that it could not convert the value. If it was, I had it set to the current date.
There is a a data conversion task you can drop in. Then redirect the rows as needed, either failing the import entirely or redircting the rows that don't work.
Or you could try a conditional split wher eyou cast the field to a date data type and then send the failures along another path (either deleting the records or nulling out the field would be the common action.)
See also http://www.sqlis.com/sqlis/post/Expression-Date-Functions.aspx > "IsDate workaround" for a technique that can be adapted
You can check whether your variable has a date or not using a conditional statement like this:
testDateVariable?true:false
For example, if date > 2 then it is true (and put the date, or format the date as you wish). If it is false, put null (you replace true with the date format and false with null).
All this is in a drived column in SSIS.