I receive a file monthly that for some reason has service_date as a char(8) data type. The image shows some of the things that are entered and the second column is the length of the service date field. Is there a way for me to clean up the bad dates? I was hoping if the length was shorter or longer I could eliminate them but it doesn't seem like it will work. Any ideas on what the best way to fix this field would be. Thanks.
You can use the try_convert function to attempt to convert your Service_Date values into a date and excluding any where a null is returned.
Related
From my understanding, and I am probably wrong here, but doesn't the ISNUMERIC() return a 1 if whatever we are looking at is a numeric data type? And since most zip codes are saved as varchar data types, shouldn't it then return a 0? I looked up the documentation and that's what it says there to me, what am I missing here? I get zip codes are numbers, but because they are saved as a string shouldn't that make a difference? Thanks in advance.
ISNUMERIC() is specifically to look at strings, not at things already stored as numbers.
However, I don't recommend using it. For instance, '3.1e5' is considered numeric.
Instead, use try_convert():
try_convert(int, zip)
This returns NULL if the column cannot be converted.
I am getting the below error while doing a count distinct on a varchar field
Function EXTRACT does not support VARCHAR(16777216) argument type
I am not able to figure out how to solve this
I am getting the below error while doing a count distinct on a varchar field
Please include a query that reproduces the issue for better answers. The limited description does not cover the context behind the error message.
Function EXTRACT does not support VARCHAR(16777216) argument type
Going by the error message alone, it appears you've used more than just COUNT over your VARCHAR typed column (perhaps there are predicates with other functions in use).
Specifically, Snowflake's EXTRACT function cannot be applied to a VARCHAR type, it only accepts a DATE or TIMESTAMP type.
This is just a example/guess, but replace your EXTRACT(year FROM column_name) portion with EXTRACT(year FROM TO_TIMESTAMP(column_name)) if the column can be parsed as a timestamp.
Thanks Harsh, you are right. I also had a where clause on a data field and I was doing a YEAR(). I assumed that the date field is of timestamp data type. But someone defined it as VARCHAR and that was causing the issue
I am trying to declare a session variable to a date value but the variable keeps reading in as string or numeric.
I've tried setting the date with varying formats, with and without time, in utc format, etc but nothing has worked. All are seen as text unless i don't use quotes or apostrophe's, in which case 2019-09-01 results in 2009 number type.
set(myDate)='2019-09-01'
set(myDate)="2019-09-01"
set(myDate as date)='2019-09-01'
set(myDate)='2019-09-01 18:25:53.820000000Z'
no matter what i try when i run show variables it doesn't show as date or timestamp data type. if i run set(myDate)=current_timestamp() that works fine but I do not want the current date.
Finally figured it out so maybe this will help someone else. When setting the variable, use to_date to cast the value at the same time. e.g:
set(myDate)=to_date('2019-09-01');
hopefully the title describes what I'm trying to do.
I have a varchar field in a SQL Server 2008 table that contains text dates in the format dd-mm-yyyy (e.g., 31-12-2009). I am trying to use CONVERT to convert it to a DATE field. I was successful in converting a similar varchar field in the same table using the following:
SELECT DISTINCT(CONVERT(DATE, MYDATEFIELD1, 103)) AS [CONV_MYDATEFIELD1] FROM MYTABLE;
But when I apply the same to MYDATEFIELD2, which appears to have the same type of data values as MYDATEFIELD1, it fails with the following error:
Msg 241, Level 16, State 1, Line 1
Conversion failed when converting date and/or time from character string.
I've tried sorting and using LIKE to try to find any characters that might prevent the conversion but I haven't been able to pinpoint anything.
Any help will be greatly appreciated. Thanks!
You may have some invalid dates (e.g. 30-02-2009), try to find them splitting the characters and validating the day and the months, assuring that the days correspond to the month and the month is in the range 01 - 12.
If you can't find which value is causing the conversion error then use a cursor to go through all the records individually and use TRY CATCH to find which record(s) cause the conversion error. You could use a PRINT statement in the CATCH block to identify the records that are erroring.
Find your bad dates with the following:
SET DATEFORMAT dmy;
select MYDATEFIELD1, isdate(MYDATEFIELD1)
from MYDATEFIELD1
I figured out the issue that was causing the CONVERT to fail but I'm not sure of the best way to select an answer (veritable stack noob) so, any help on that would be appreciated. Here are the major steps I took to find the issue:
I used MIN and MAX SUBSTRING to identify that the component parts of the
varchar field were correct (i.e., the 1st two digits min=01 max=31,
middle two min=01 max=12)
I used DISTINCT SUBSTRING to identify that all of the date separators were consistent (i.e., all dashes).
I used MAX(LEN) to determine that my varchar "date" field was 12 characters (vs. the 10 characters I was expecting).
I used CONVERT(VARBINARY, MYDATEFIELD2) to determine what was actually stored in the string.
The last step revealed that the field contained line feeds (00A). I opened the source text file in notepad++, clicked View -> Show Symbol -> Show All Characters and I could see the LF at the end of each line.
So now I'm modifying the DTSX package (fixed width text) to include an extra field for the linefeed that I can drop afterwards. Now that I know what the intended format of the date fields is, I'll try to import them as DT_DATE vs DT_STR. I'm not exactly sure how to specify the correct date style 105 at import (thanks #Panagiotis Kanavos) but I'll figure it out.
Whew! What a learning experience! :D
Thanks to everyone who helped - and if you can give advice on the best way to select the best answer it will be greatly appreciated.
I'm using LINQ-to-SQL and need to order by a date field. The date field is stored as text and could have anything in it since it is user entered data. I need to handle cases where an invalid date was placed in there.
For example, a date of "02/23/0000" returns:
The conversion of a varchar data type to a datetime data type resulted in an out-of-range value.
I need to avoid errors and I don't care where an invalid date like this gets sorted to. The parameters of the project mean I can't modify my source data, only read from it.
Here is an example LINQ statement.
from x in dbo_myTable
orderby Convert.ToDateTime(x.MyDateField)
select x
Do you definitely need to perform the ordering in the database? In this situation I would strongly consider pulling all of the data, then performing some conversion in .NET code where you have a great deal more control over what's going on, and then order it still in .NET code.
Ultimately, dealing with screwed up data in this sort of situation is tricky - the more control you have, the better.
You can do something like this (Based on your comment that they are all 10 characters)
from x in dbo_myTable
orderby x.MyDateField.Substring(6, 4), x.MyDateField.Substring(0, 2), x.MyDateField.Substring(3, 2)
select x
First is first; why cant you change the database to fix you obviously brokes table structure? Why cant you sanitize input when its going into the database? Why are you not validating user input?
Now for the answer; your only option is attempt to parse the date (with DateTime.TryParse or DateTime.TryParseExact) and set a default value of your choosing if it fails to parse:
from x in dbo_myTable
orderby TryConvertToDateTime(x.MyDateField)
select x
private DateTime TryConvertToDateTime(string dt)
{
DateTime rtn = DateTime.MinValue; // or whatever you want your default to be
DateTime.TryParse(dt,out rtn);
return rtn;
}