Validate Date Values in NVarchar field - sql-server

Question

Hi I have a reporting application written against some 3rd party software. Unfortunately it stores all values as nvarchar and does not validate data entry on the client side as a result I am getting the following error when
"Conversion failed when converting date and/or time from character string"
System.Data.SqlClient.SqlException was unhandled by user code
or if I try to execute the code in SSMS:
Msg 241, Level 16, State 1, Procedure settlement_list, Line 10
Conversion failed when converting date and/or time from character string.
I assume this is the result of someone entering a text value in the data field so I've tried this ISDATE code to find the bad value:
SELECT mat3_02_01, CONVERT(datetime, mat3_04_02), mat3_04_02 FROM lntmu11.matter3
WHERE ISDATE(mat3_04_02) <> 1
AND Coalesce(mat3_04_02, '') <> ''
order by mat3_04_02 desc
and I get zero row returned ... I also manually sifted through the data (its sveral 100 thousand rows so its kind of hard and see no bad values ???
Does anyone have any suggestions ?
EDIT ---
Here is the stored proc (I know where clause is ugly)
SELECT mat_no, 'index'=matter.mat1_01_06,
'insurance'=Replace(Replace(matter.mat1_03_01, 'INSURANCE COMPANY', ' '), 'COMPANY', ''),
matter.[status], 'casestage'=mat1_04_01, 'injured'=matter.MAT1_01_07, matter.client,
'terms'=mat3_04_06, 'ClmAmt'=matter.mat1_07_01,
'ClmBal'=matter.mat1_07_03, 'SetTot'=matter3.MAT3_04_09, 'By'=mat3_03_02,
'DtSttld'=mat3_04_02, 'SettlStg'=(MAT3_06_08 + ' / ' + MAT3_06_05)
FROM [lntmu11].matter3 inner join
[lntmu11].matter ON [lntmu11].matter.sysid = [lntmu11].matter3.sysid
WHERE
(DateDiff(month, convert(datetime, MAT3_04_02, 101), GETDATE()) = #range
and mat3_03_02 like #by)
or
(mat3_04_06 like #by2
and DateDiff(month, convert(datetime, MAT3_04_02, 101), GETDATE()) = #range)
ORDER BY MAT3_03_02

Answer 1

You can't force the order the query engine will try to process the statement without first dumping the ISDATE() = 1 rows into a #temp table. You can't guarantee the processing order or short circuiting, even though some will suggest using a CTE or subquery to filter out the bad rows first. So some might suggest:
;WITH x AS
(
SELECT mat3_02_01, mat3_04_02
FROM Intmu11.matter3
WHERE ISDATE(mat3_04_02) = 1
AND mat3_04_02 IS NOT NULL -- edited!
)
SELECT mat3_02_01, CONVERT(DATETIME, mat3_04_02), mat3_04_02
FROM x
ORDER BY mat3_04_02 DESC;
And this may even appear to work, today. But in the long term, really the only way to guarantee this processing order - in current versions of SQL Server - is:
SELECT mat3_02_01, mat3_04_02
INTO #x
FROM Intmu11.matter3
WHERE ISDATE(mat3_04_02) = 1
AND mat3_04_02 IS NOT NULL; -- edited!
SELECT mat3_02_01, CONVERT(DATETIME, mat3_04_02), mat3_04_02
FROM #x
ORDER BY mat3_04_02 DESC;
Have you thought about validating the values on input? For example, you can change where this error appears in the application by slapping them on the wrist when they enter an invalid date, instead of punishing the person who selects their bad data. If you are controlling the update/insert via a stored procedure, you can say:
IF ISDATE(#mat3_04_02) = 0
BEGIN
RAISERROR('Please enter a valid date.', 11, 1);
RETURN;
END
If you aren't controlling data manipulation via stored procedure(s), then you can add a check constraint to the table (after you've cleaned up the existing bad data).
UPDATE Intmu11.matter3 SET mat3_04_02 = NULL
WHERE ISDATE(mat3_04_02) = 0;
ALTER TABLE Intmu11 WITH NOCHECK
ADD CONSTRAINT mat3_04_02_valid_date CHECK (ISDATE(mat3_04_02)=1);
This way when the error message gets bubbled up to the user they will see the constraint name and hopefully will be able to map that to the data entry point on the front end that failed:
Msg 547, Level 16, State 0, Line 1 The INSERT statement conflicted
with the CHECK constraint "mat3_04_02_valid_date". The conflict
occurred in database "your_db", table "Intmu11.matter3", column
'mat3_04_02'. The statement has been terminated.
Or better yet, use the right data type in the first place! Again, after updating the existing bad data to be NULL, you can say:
ALTER TABLE Intmu11.matter3 ALTER COLUMN mat3_04_02 DATETIME;
Now when someone tries to enter a non-date, they'll get the same error that the users are currently getting when they try to select the bad data:
Msg 241, Level 16, State 1, Line 1 Conversion failed when
converting date and/or time from character string.
In SQL Server 2012, you'll be able to get around this with TRY_CONVERT() but you should still be trying to get the data type right from the beginning.

Answer 2

Examine the query where
ISDATE(mat3_04_02) = 1
AND
Coalesce(mat3_04_02, '') = ''
To be a date it must have a value.
But is only matches the second condition if it has not value.
The intersection (and) of those two conditions is always false.
If you are looking for null then "mat3_04_02 is null" but it still will return 0 rows.
Try
SELECT mat3_02_01, CONVERT(datetime, mat3_04_02), mat3_04_02
FROM lntmu11.matter3
WHERE ISDATE(mat3_04_02) = 1
order by CONVERT(datetime, mat3_04_02) desc
I think you would want date sorted and not string sorted
The question started as finding valid dates and it morphed into finding invalid dates
SELECT mat3_02_01, mat3_04_02
FROM lntmu11.matter3
WHERE ISDATE(mat3_04_02) = 0
AND mat3_04_02 is not null
order by mat3_04_02) desc