I am trying to (synthetically) populate an empty time dimension table in a star modeled data mart, after which it should look like this:
I am using the following T-SQL code for this:
/*
STEP 3
Populate DIM_TIJD table with date and its derrived values data
*/
USE Fuzzy_DM_Robin
--DECLARE DATE VARIABLES FOR DATE PERIOD
DECLARE #StartDate datetime = '01/01/1995'
DECLARE #EndDate datetime = '01/01/2026'
DECLARE #DateInProcess datetime
SET #DateInProcess = #StartDate
WHILE #DateInProcess < = #EndDate
BEGIN
SET NOCOUNT ON
--LOOP THROUGH INDIVIDUAL DATES DEFINED BY TIME PERIOD
INSERT INTO DIM_TIJD (
[DATUM_ID],
[DATUM],
[DAG_VD_WEEK],
[WEEKNR],
[MAAND],
[MAAND_OMSCHRIJVING],
[LAATSTE_DAG_MAAND],
[KWARTAAL],
[JAAR]
)
VALUES (
CAST ( #DateInProcess AS numeric (10) ),
#DateInProcess,
CONVERT(varchar(10), #DateInProcess, 110) + ', ' + DATENAME(WEEKDAY, #DateInProcess ),
DATEPART (wk, #DateInProcess),
MONTH( #DateInProcess),
CAST(YEAR(#DateInProcess) as varchar(4)) + ' - ' + DATENAME(MONTH, #DateInProcess ),
DATEPART (dd, EOMONTH ( #DateInProcess)),
DATENAME( QUARTER, #DateInProcess ),
YEAR(#DateInProcess))
END
Nevertheless, it won't load. Messages I receive are:
Message 1:
String or binary data would be truncated
and message 2:
[Execute SQL Task] Error: Executing the query "/*
STEP 3
Populate DIM_TIJD table with date and it..." failed with the following error: "The statement has been terminated.". Possible failure reasons: Problems with the query, "ResultSet" property not set correctly, parameters not set correctly, or connection not established correctly."
What am I doing wrong? There are no under linings in SSMS indicating that something is wrong with the code and I have checked all the varchar lengths to be the same for both the table and the code. I have tried fiddling around with result set settings, but that only gives me another message:
Message 3:
[Execute SQL Task] Error: There is an invalid number of result bindings returned for the ResultSetType: "ResultSetType_SingleRow".
Thank you in advance!
The #DateInProcess variable needs to be incremented as part of the loop.
Try adding this as the last line of your loop (before END):
SET #DateInProcess = DATEADD(DAY, 1, #DateInProcess)
Related
I have a database in which a table names are associated with month and year numbers ex: datavalues_7_2017 for july, datavalues_8_2017 for august and so on.
I am using a query to retrieve certain values from the table.
SELECT o_key ,MIN(dateadd(hour, datediff(hour, 0, time), 0)) as on_time
,MAX(dateadd(hour, datediff(hour, 0, dateadd(hour, 1, time)), 0)) as off_time
,cast(time as date) as RDNG_DT ,[repeated_hour],[value]
FROM [data_values_8_2017]
WHERE value = 1 and o_key in (X,X,X,X...) and cast(time as date) = cast(getdate() as date)
GROUP BY o_key,cast(time as date),repeated_hour,value
I am using SSIS package and using this query I am loading the result into another table.
Now this table datavalues_X_2017 is bound to change every month creating a new table, and my SSIS query should be pointing to the new table.
Can someone suggest me a way where I can automate this processes.
All you need to do is create a stored procedure as a wrapper and have a date parameter passed to it as shown below. Call this procedure as part of your source.
If you do not want it as stored procedure, use the same logic and put the sql as a variable and in the expressions pass the month and year variable
create procedure dbo.usp_GetData
(
#Date datetime = null
)
as
if #Date is null
set #Date = GETDATE()
declare #Month int
,#Year int
,#sql nvarchar(max)
select #Month = DATEPART(MONTH,#Date)
,#Year = DATEPART(YEAR,#Date)
set #sql ='
select o_key
,MIN(dateadd(hour, datediff(hour, 0, time), 0)) as on_time
,MAX(dateadd(hour, datediff(hour, 0, dateadd(hour, 1, time)), 0)) as off_time
,cast(time as date) as RDNG_DT
,[repeated_hour],[value]
from [data_values_'+CAST(#Month AS VARCHAR(2))+'_'+CAST(#Year AS VARCHAR(4))+']
where value = 1
and o_key in (
X,X,X,X...
)
and cast(time as date) = cast(getdate() as date)
group by o_key
,cast(time as date)
,repeated_hour
,value
';
exec sp_executesql #sql;
go
Get dynamic SQL with Variable Expressions the following way:
Create a String variable DateSuffux which will be filled with your datevalue like "7_2017"
Create a String variable SQLQuery with properties EvaluateAsExpression=true and define the following Expression for it
"SELECT o_key ,...,[repeated_hour],[value] FROM [data_values_" +
#[User::DateSuffix] + "] WHERE ... "
Specify variable SQLQuery as a source for your Data Source or SQL Query task
You are done.
This variable expression substitutes result of the expression when the SQLQuery variable is used. The expression itself injects contents of DateSuffix string variable into a query text.
I have a table with column ImportDate of datatype varchar(100).
I want to convert its values from varchar to Datetime and for that I have used this query:
select
convert(datetime, ImportDate)
from ImportHistory
But it throws an exception with message
Msg 241, Level 16, State 1, Line 1
Conversion failed when converting date and/or time from character string.
But when I individually select each value and run the statement it works fine. For example the below query works perfectly, and so do all the values in the table
select convert(Datetime, '1826-07-04 18:20:00')
There are no null values in that table and below are the values:
1826-07-04 18:20:00
1826-07-04 18:20:00
1917-11-08 11:11:00
2003-07-16 16:02:00
1984-06-08 00:00:00
2004-06-05 00:00:00
1826-07-04 18:20:00
1826-07-04 18:20:00
1917-11-08 11:11:00
2003-07-16 16:02:00
1984-06-08 00:00:00
2004-06-05 00:00:00
If you're using SQL Server 2012+, use TRY_PARSE or TRY_CONVERT in this kind of scenario:
DECLARE #ImportHistory TABLE (ImportDate VARCHAR(100))
INSERT #ImportHistory
VALUES
('1826-07-04 18:20:00'),
('1826-07-04 18:20:00'),
('1917-11-08 11:11:00'),
('2003-07-16 16:02:00'),
('1984-06-08 00:00:00'),
('2004-06-05 00:00:00'),
('1826-07-04 18:20:00'),
('1826-07-04 18:20:00'),
('1917-11-08 11:11:00'),
('Invalid!'),
('2003-07-16 16:02:00'),
('1984-06-08 00:00:00'),
('2004-06-05 00:00:00')
SELECT
ImportDate, TRY_CONVERT(datetime, ImportDate) as dt
FROM #ImportHistory
WHERE TRY_CONVERT(datetime, ImportDate) IS NULL
-- output: Invalid!, NULL
To find the invalid value. If you want invalid values to be converted to NULL, you can remove the WHERE clause and just use TRY_PARSE in place of CONVERT.
The dates you've listed are all valid, but it's very likely in your actual table you have at least one invalid date - or at least not one that can be parsed as is (extra space, month/day stored in different culture format, etc.).
If you must keep your column as a VARCHAR for some unknown reason and you want to make sure that applications don't insert unparsable dates into it, you could add a constraint
ALTER TABLE ImportHistory
ADD CONSTRAINT CK_ImportDate
CHECK(TRY_CONVERT(datetime, ImportDate) IS NOT NULL)
If you don't have SQL Server 2012+, you could try making a cursor to find the invalid data:
DECLARE #dt VARCHAR(100);
DECLARE #dt2 DATETIME;
BEGIN TRY
DECLARE test_cursor1 CURSOR FOR
SELECT Importdate FROM #ImportHistory
OPEN test_cursor1
WHILE ##FETCH_STATUS = 0
BEGIN
FETCH NEXT FROM test_cursor1 INTO #dt
SET #dt2 = CONVERT(datetime, #dt)
END
END TRY
BEGIN CATCH
SELECT #dt
END CATCH
-- output: Invalid!
This resolved the issue.
Select Convert(Datetime, LTRIM ( RTRIM ( REPLACE ( REPLACE ( REPLACE ( ImportDate, CHAR(10), ''), CHAR(13), ''), CHAR(9), '') ) )) from ImportHistory
Thank you All !!
I have below SQL.
UPDATE student_queues
SET Deleted=0,
last_accessed_by='raja',
last_accessed_on=CONVERT(VARCHAR(24),'23-07-2014 09:37:00',113)
WHERE std_id IN ('2144-384-11564')
AND reject_details='REJECT'
when I ran the above SQL the below exception has been throwed.
Conversion failed when converting date and/or time from character string.
If you're trying to insert in to last_accessed_on, which is a DateTime2, then your issue is with the fact that you are converting it to a varchar in a format that SQL doesn't understand.
If you modify your code to this, it should work, note the format of your date has been changed to: YYYY-MM-DD hh:mm:ss:
UPDATE student_queues
SET Deleted=0,
last_accessed_by='raja',
last_accessed_on=CONVERT(datetime2,'2014-07-23 09:37:00')
WHERE std_id IN ('2144-384-11564') AND reject_details='REJECT'
Or if you want to use CAST, replace with:
CAST('2014-07-23 09:37:00.000' AS datetime2)
This is using the SQL ISO Date Format.
Seems like last_accessed_on, is a date time, and you are converting '23-07-2014 09:37:00' to a varchar. This would not work, and give you conversion errors. Try
last_accessed_on= convert(datetime,'23-07-2014 09:37:00', 103)
I think you can avoid the cast though, and update with '23-07-2014 09:37:00'. It should work given that the format is correct.
Your query is not going to work because in last_accessed_on (which is DateTime2 type), you are trying to pass a Varchar value.
You query would be
UPDATE student_queues SET Deleted=0 , last_accessed_by='raja', last_accessed_on=convert(datetime,'23-07-2014 09:37:00', 103)
WHERE std_id IN ('2144-384-11564') AND reject_details='REJECT'
DECLARE #FromDate DATETIME
SET #FromDate = 'Jan 10 2016 12:00AM'
DECLARE #ToDate DATETIME
SET #ToDate = 'Jan 10 2017 12:00AM'
DECLARE #Dynamic_Qry nvarchar(Max) =''
SET #Dynamic_Qry='SELECT
(CONVERT(DATETIME,(SELECT
CASE WHEN ( ''IssueDate'' =''IssueDate'') THEN
EMP_DOCUMENT.ISSUE_DATE
WHEN (''IssueDate'' =''ExpiryDate'' ) THEN
EMP_DOCUMENT.EXPIRY_DATE ELSE EMP_DOCUMENT.APPROVED_ON END
CHEKDATE ), 101)
)FROM CR.EMP_DOCUMENT as EMP_DOCUMENT WHERE 1=1
AND (
CONVERT(DATETIME,(SELECT
CASE WHEN ( ''IssueDate'' =''IssueDate'') THEN
EMP_DOCUMENT.ISSUE_DATE
WHEN (''IssueDate'' =''ExpiryDate'' ) THEN EMP_DOCUMENT.EXPIRY_DATE
ELSE EMP_DOCUMENT.APPROVED_ON END
CHEKDATE ), 101)
) BETWEEN '''+ CONVERT(CHAR(10), #FromDate, 126) +''' AND '''+CONVERT(CHAR(10), #ToDate , 126
)
+'''
'
print #Dynamic_Qry
EXEC(#Dynamic_Qry)
I use this command to select all the specific dates if the given variable is date, if it is not it should return all of the fields.
The commands works when #query is in the form of a date, but it returns an error:
"Conversion failed when converting date and/or time from character string."
when it is any other arbitrary string.
Code:
select * from table where
format(dateofbirth,'dd/MMM/yyyy') = Case
when ISDATE(#query)=1 then
format(CONVERT(datetime,#query),'dd/MMM/yyyy')
else
format(dateofbirth,'dd/MMM/yyyy')
Edit:
#query can be any string for eg. "1/1/2013" , "random" , "3".
The command should return all fields if #query is not in form of a date.
You can work around this problem by re-formulating your query condition like this:
declare #query as varchar(20)='blah'
SELECT *
FROM testtable
WHERE ISDATE(#query) = 0
OR CONVERT(date, dateofbirth) = CASE ISDATE(#query)
WHEN 1 THEN CONVERT(date, #query) ELSE NULL
END
Demo on sqlfiddle.
The problem is that logic operators are not short-circuited in SQL, so the optimizer treats CONVERT(date, #query) as something that it can pre-compute to speed up the query. By expanding the condition to a CASE that depends entirely on #query you can eliminate the execution of the CONVERT branch when ISDATE(#query) returns "false".
Hi I have a reporting application written against some 3rd party software. Unfortunately it stores all values as nvarchar and does not validate data entry on the client side as a result I am getting the following error when
"Conversion failed when converting date and/or time from character string"
System.Data.SqlClient.SqlException was unhandled by user code
or if I try to execute the code in SSMS:
Msg 241, Level 16, State 1, Procedure settlement_list, Line 10
Conversion failed when converting date and/or time from character string.
I assume this is the result of someone entering a text value in the data field so I've tried this ISDATE code to find the bad value:
SELECT mat3_02_01, CONVERT(datetime, mat3_04_02), mat3_04_02 FROM lntmu11.matter3
WHERE ISDATE(mat3_04_02) <> 1
AND Coalesce(mat3_04_02, '') <> ''
order by mat3_04_02 desc
and I get zero row returned ... I also manually sifted through the data (its sveral 100 thousand rows so its kind of hard and see no bad values ???
Does anyone have any suggestions ?
EDIT ---
Here is the stored proc (I know where clause is ugly)
SELECT mat_no, 'index'=matter.mat1_01_06,
'insurance'=Replace(Replace(matter.mat1_03_01, 'INSURANCE COMPANY', ' '), 'COMPANY', ''),
matter.[status], 'casestage'=mat1_04_01, 'injured'=matter.MAT1_01_07, matter.client,
'terms'=mat3_04_06, 'ClmAmt'=matter.mat1_07_01,
'ClmBal'=matter.mat1_07_03, 'SetTot'=matter3.MAT3_04_09, 'By'=mat3_03_02,
'DtSttld'=mat3_04_02, 'SettlStg'=(MAT3_06_08 + ' / ' + MAT3_06_05)
FROM [lntmu11].matter3 inner join
[lntmu11].matter ON [lntmu11].matter.sysid = [lntmu11].matter3.sysid
WHERE
(DateDiff(month, convert(datetime, MAT3_04_02, 101), GETDATE()) = #range
and mat3_03_02 like #by)
or
(mat3_04_06 like #by2
and DateDiff(month, convert(datetime, MAT3_04_02, 101), GETDATE()) = #range)
ORDER BY MAT3_03_02
You can't force the order the query engine will try to process the statement without first dumping the ISDATE() = 1 rows into a #temp table. You can't guarantee the processing order or short circuiting, even though some will suggest using a CTE or subquery to filter out the bad rows first. So some might suggest:
;WITH x AS
(
SELECT mat3_02_01, mat3_04_02
FROM Intmu11.matter3
WHERE ISDATE(mat3_04_02) = 1
AND mat3_04_02 IS NOT NULL -- edited!
)
SELECT mat3_02_01, CONVERT(DATETIME, mat3_04_02), mat3_04_02
FROM x
ORDER BY mat3_04_02 DESC;
And this may even appear to work, today. But in the long term, really the only way to guarantee this processing order - in current versions of SQL Server - is:
SELECT mat3_02_01, mat3_04_02
INTO #x
FROM Intmu11.matter3
WHERE ISDATE(mat3_04_02) = 1
AND mat3_04_02 IS NOT NULL; -- edited!
SELECT mat3_02_01, CONVERT(DATETIME, mat3_04_02), mat3_04_02
FROM #x
ORDER BY mat3_04_02 DESC;
Have you thought about validating the values on input? For example, you can change where this error appears in the application by slapping them on the wrist when they enter an invalid date, instead of punishing the person who selects their bad data. If you are controlling the update/insert via a stored procedure, you can say:
IF ISDATE(#mat3_04_02) = 0
BEGIN
RAISERROR('Please enter a valid date.', 11, 1);
RETURN;
END
If you aren't controlling data manipulation via stored procedure(s), then you can add a check constraint to the table (after you've cleaned up the existing bad data).
UPDATE Intmu11.matter3 SET mat3_04_02 = NULL
WHERE ISDATE(mat3_04_02) = 0;
ALTER TABLE Intmu11 WITH NOCHECK
ADD CONSTRAINT mat3_04_02_valid_date CHECK (ISDATE(mat3_04_02)=1);
This way when the error message gets bubbled up to the user they will see the constraint name and hopefully will be able to map that to the data entry point on the front end that failed:
Msg 547, Level 16, State 0, Line 1 The INSERT statement conflicted
with the CHECK constraint "mat3_04_02_valid_date". The conflict
occurred in database "your_db", table "Intmu11.matter3", column
'mat3_04_02'. The statement has been terminated.
Or better yet, use the right data type in the first place! Again, after updating the existing bad data to be NULL, you can say:
ALTER TABLE Intmu11.matter3 ALTER COLUMN mat3_04_02 DATETIME;
Now when someone tries to enter a non-date, they'll get the same error that the users are currently getting when they try to select the bad data:
Msg 241, Level 16, State 1, Line 1 Conversion failed when
converting date and/or time from character string.
In SQL Server 2012, you'll be able to get around this with TRY_CONVERT() but you should still be trying to get the data type right from the beginning.
Examine the query where
ISDATE(mat3_04_02) = 1
AND
Coalesce(mat3_04_02, '') = ''
To be a date it must have a value.
But is only matches the second condition if it has not value.
The intersection (and) of those two conditions is always false.
If you are looking for null then "mat3_04_02 is null" but it still will return 0 rows.
Try
SELECT mat3_02_01, CONVERT(datetime, mat3_04_02), mat3_04_02
FROM lntmu11.matter3
WHERE ISDATE(mat3_04_02) = 1
order by CONVERT(datetime, mat3_04_02) desc
I think you would want date sorted and not string sorted
The question started as finding valid dates and it morphed into finding invalid dates
SELECT mat3_02_01, mat3_04_02
FROM lntmu11.matter3
WHERE ISDATE(mat3_04_02) = 0
AND mat3_04_02 is not null
order by mat3_04_02) desc