Snowflake has a TO_DATE(string, pattern) function, but currently has no TRY_TO_DATE version of the function with the same signature. While we've built a javascript parsing function as a workaround, it has abysmal performance due to its complexity and the fact that it's performance could never measure against a real native function.
Is there a roadmap for the availability of such native function? Is there a community voting board to prioritize such function?
Although there is not a parameter for this function, you can set DATE_INPUT_FORMAT to force a pattern for TRY_TO_DATE function:
ALTER SESSION SET DATE_INPUT_FORMAT = 'YYYY-MM-DD';
SELECT TRY_TO_DATE('2020-04-05'); -- will return a date value
ALTER SESSION SET DATE_INPUT_FORMAT = 'DD-MM-YYYY';
SELECT TRY_TO_DATE('2020-04-05'); -- will return NULL
https://docs.snowflake.com/en/sql-reference/parameters.html#date-input-format
You can use Snowflake Ideas to suggest and vote for new features:
https://community.snowflake.com/s/ideas
Release Notes - August 3-6, 2020
TRY_TO_DATE(), TRY_TO_TIME(), and TRY_TO_TIMESTAMP() Functions: Support Added for Optional Format Specifier
With this release, the TRY_TO_DATE(), TRY_TO_TIME(), and the TRY_TO_TIMESTAMP() family of functions support an optional format specifier, similar to the corresponding TO_DATE(), TO_TIME(), and TO_TIMESTAMP() functions.
TRY_TO_DATE
A special version of TO_DATE , DATE that performs the same operation (i.e. converts an input expression to a date), but with error-handling support (i.e. if the conversion cannot be performed, it returns a NULL value instead of raising an error).
Syntax
TRY_TO_DATE( <string_expr> [, <format> ] )
Related
The following instruction aims at using a specific format to import DATEs
alter session set DATE_INPUT_FORMAT = 'YYYY-MM-DD HH24:MI:SS.FF';
However, it seems to have no effect on the following:
copy into schema.table
from s3://bucket/file.parquet
credentials=(aws_key_id='...' aws_secret_key='...')
match_by_column_name=case_insensitive
file_format=(type=parquet);
Which results in errors like below:
sqlalchemy.exc.ProgrammingError: (snowflake.connector.errors.ProgrammingError) 100071 (22000):
Failed to cast variant value "2020-06-16 00:00:00.000" to DATE
When a column in the imported Parquet file has a format as specified above for a date field.
This really sounds like a bug, as the above COPY INTO scenario should in theory be a typical use case for altering the DATE_INPUT_FORMAT.
Is there a way to address this?
The DATE_INPUT_FORMAT should affect the copy command. The documentation talks about not supporting a timestamp from a variant column on a date conversion.
Although TO_DATE accepts a TIMESTAMP value, it does not accept a TIMESTAMP inside a VARIANT.
Is it possible in SQL Server to define a user defined function with fixed enumerable parameters?
Like many pre-defined functions in SQL Server like DATEDIFF that takes DAY, MONTH, etc as first parameter, but these are not char, or any other data types...
I think that it should be easy to find the answer on Internet, but I don't know what I should exactly search. 😅😅
SQL Server doesn't have constants or enums in that sense; parameters to functions or procedures require to pass in strings, numbers, or variables.
Yes, this is unlike the ability to use well-defined constants directly in built-in functions and other T-SQL structs. Maybe that's something we'll see in a future evolution of the language, but this is what we have today.
For things like DATEADD you are passing an identifier... note that these work:
SELECT DATEADD([DAY], 1, GETDATE());
SELECT DATEADD("DAY", 1, GETDATE());
But this will fail:
SELECT DATEADD('DAY', 1, GETDATE());
Interestingly this will also fail (just further evidence that this is being handled like an identifier):
SET QUOTED_IDENTIFIER OFF;
SELECT DATEADD("DAY", 1, GETDATE());
You can't write your own functions or procedures that take identifiers as input - they are always either interpreted as an implicitly converted string (as in EXEC sp_who active;) or they simply fail a parse check (as in the above). Input parameters to built-in and user-defined functions will take expressions, but procedures will not.
Does anyone know if there is a way to achieve the same affect as a datetime column with a default binding of getdate() with computed columns?
I have tried setting the formula to getdate() and persist to Yes but I get an error
Computed column 'InsertDateTime' in table 'Tmp_Table' cannot be persisted because the column is non-deterministic.
forget the "computed column" and make it a regular not null column, with a default to GETDATE(), or use an INSTEAD OF UPDATE/INSERT trigger to set it.
you can't make a computed column use a function that constantly returns a different value (based on the same parameter values), it must return the same value each time (based on the same parameter values). Read this: Deterministic and Nondeterministic Functions
All functions are deterministic or nondeterministic:
Deterministic functions always return the same result any time
they are called with a specific set of input values.
Nondeterministic functions may return different results each time
they are called with a specific set of input values.
Whether a function is deterministic or nondeterministic is called the
determinism of the function.
For example, the DATEADD built-in function is deterministic because it
always returns the same result for any given set of argument values
for its three parameters. GETDATE is not deterministic because it is
always invoked with the same argument, yet the value it returns
changes each time it is executed.
Let's say my database is called "data" and its second column is called "datetime". The format of my datetime is "7/22/2011 12:00:00". If I want to get the the information between 12:00 and 13:00 today, how do I do that?
SELECT * FROM data WHERE
strftime('%m/%d/%Y %H:%M:%S', datetime) > strftime('%m/%d/%Y %H:%M:%S', '7/22/2011 12:00:00')
AND
strftime('%m/%d/%Y %H:%M:%S', datetime) < strftime('%m/%d/%Y %H:%M:%S', '7/22/2011 13:00:00')
doesn't do the trick.
I got this wrong -- shame on me. strftime will only read in ISO 8601 (or timestamp/julian) values. The format specified doesn't apply to the the input -- it is only for the output value.
For this reason the data in the database must still have a natural ordering to be sorted. The suggested formats are ISO 8601 (string), timestamp (integer), and julian-date (float). Using a date like m/d/y in the database won't work for range compares because it fails to adhere to natural ordering.
strftime and friends simply allow generation of one of the values useful for the comparison -- e.g. can convert from ISO 8601/timestamp/julian input to custom/ISO 8601/timestamp/julian-date. They can not directly convert from a custom value though.
The easiest way to fix the database is probably to use a language like Python to write a script for the conversion of existing data. A small Java program could also be written taking advantage of SimpleDateFormat to parse the existing values and write them back using one of the recommended types.
Historical:
Try it in pieces from the console (this will enable must faster experimenting with the query) -- the first conditional then the second. What returns what and why? Once each piece is working then putting them to-gether will work.
There are multiple strftime format mistakes -- see Date and Time for the formats and check the strings again :) Also, if using a saner date format (ISO 8601), could just use datetime and not worry about specifying the format explicitly. ISO 8601 values are is also lexically sortable as strings -- an added advantage.
I am not sure how the Java SQLite adatper will handle a Date when passed as a parameter to a parametrized query (parametrized queries really should be used) ... but in any case this is a secondary issue to the current behavior observed.
Happy coding.
looks like you've got a bad format character
SELECT * FROM data WHERE
strftime('%m/%d/%Y %H:%H:%S', datetime) > strftime('%m/%d/%y %H:%M:%S', '7/22/2011 12:00:00')
-- ^
AND
strftime('%m/%d/%y %H:%M:%S', datetime) < strftime('%m/%d/%y %H:%M:%S', '7/22/2011 13:00:00')
I have a function that takes a number as an input and converts it to a date. This number isn't any standard form of date number, so I have to manually subdivide portions of the number to various date parts, cast the date parts to varchar strings and then, concatenate and cast the strings to a new datetime object.
My question is how can I catch a casting failure and return a null or low-range value from my function? I would prefer for my function to "passively" fail, returning a default value, instead of returning a fail code to my stored procedure. TRY/CATCH statements apparently don't work form within functions (unless there is some type of definition flag that I am unaware of) and trying the standard '##Error <> 0' method doesn't work either.
Incidentally this sounds like it could be a scalar UDF. This is a performance disaster, as Alex's blog points out. http://sqlblog.com/blogs/alexander_kuznetsov/archive/2008/05/23/reuse-your-code-with-cross-apply.aspx
SELECT CASE WHEN ISDATE(#yourParameter) = 1
THEN CAST(#yourParameter AS DATETIME)
ELSE YourDefaultValue
END
Since the format is nonstandard it sounds to me like you are stuck with doing all the validation yourself, prior to casting. Making sure that the individual pieces are numeric, checking that the month is between 1 and 12, making sure it's not Feb 30, etc. If anything fails you return nothing.