I have a fixed format .txt file which has a date field. This field may have a date or be blank. I have wracked my brains to get this field to convert from DT_STR to DT_DBDATE in the Derived Column transformation. I have tried multiple scenarios:
[CHNG_DT] == "" ? ISNULL((DT_DBDATE)[CHNG_DT] ) : (DT_DBDATE)SUBSTRING([CHNG_DT], 1, 4)) + "-" + (DB_DBDATE)(SUBSTRING(CHNG_DT], 6, 2)) + "-" + (DB_DBDATE)(SUBSTRING([CHNG_DT], 9, 2))
[CHNG_DT] == "" ? NULL(DB_DBDATE) : (DB_DBDATE)[CHNG_DT]
ISNULL([CHNG_DT]) ? NULL(DT_DBDATE) : (DT_DBDATE)((DT_STR, 10, 1252)[CHNG_DT])
ISNULL(CHNG_DT) ? NULL(DT_DBDATE) : (DT_DBDATE)((DT_STR,10,1252)CHNG_DT)
LEFT([CHNG_DT], 10) == " " ? ISNULL((DT_DBDATE)[CHNG_DT]) : (DT_DBDATE)[CHNG_DT]
(DT_DBDATE)CHNG_DT
First of all, when you are looking to convert values from DT_STR to DT_DBDATE, you should make sure that those values are stored in the yyyy-MM-dd format. You can refer to the following official documentation to learn more:
Converting Between Strings and Date/Time Data Types in SSIS
The second suggestion is to use the LTRIM() and RTRIM() functions to make sure that there are no white blanks in the beginning and end of the date values. As an example:
LTRIM(RTRIM(REPLACENULL([CHNG_DT],""))) == "" ? NULL(DT_DBDATE) : (DT_DBDATE)LTRIM(RTRIM([CHNG_DT]))
If date values are stored in a different format, you should write an expression that changes the format before converting it to DT_DBDATE. As an example:
How to convert string in format dd.mm.yyyy to date using SSIS expression?
Other helpful answers:
SSIS Source Format Implicit Conversion for Datetime
Convert DDMonYY and time to datetime column in SSIS package (Derived Column)
The year function doesn't support dt_wstr
Related
I'm using SSIS ETL (I'm a newbie) and trying to extract dates from a SQL server table in format "08/03/2013 00:00:00" to an Excel 2013 file. I need the dates in Excel to be in format "dd/mm/yyyy", they must remain as dates so that the hierarchy remains. ie when you do an autofilter on the column you can see the date groups. There are also some null entries in the column of data.
Things I have tried in ETL script:-
1)Direct conversion from SQL table to Excel file - Result "2013-03-08 00:00:00",yyyy/mm/dd ss:ss:ss no hierarchy. I have tried changing the Excel column format but no matter what I select the date stays the same.
2)I have then added a Date Conversion to the ETL, to change the SQL date to DT_DATE - Result "3/8/2013", m/d/yyyy, no hierarchy.
3)I have then tried a Date Conversion, to change the SQL date to DTW_STR - Result "2013/03/08 00:00:00", yyyy/mm/dd ss:ss:ss no hierarchy.
4)I have then tried various ETL conversions to strings but none are recognised by Excel as dates so no hierarchy.
I'm not in a position to "re-educate" users to a new date format of "yyyy/mm/dd"
I'm not a strong coder so would prefer to do this within ETL if possible.
So to summarise what I'm after I need the dates in Excel to be actual dates that can be grouped as such in format dd/mm/yyyy.
I believe what should do is try using the derived column transformation.
Derived column name: newDate
Derived column: Replace 'originalDate'
Expression: RIGHT("0" + (DT_WSTR, 2)DATEPART("dd", [originalDate]), 2) + "/" + RIGHT("0" + (DT_WSTR, 2) DATEPART("mm", [originalDate]), 2) + "/" + (DT_WSTR, 4)DATEPART("yyyy", [originalDate])
This will take each part and convert it into a string format so that we can concatenate them back together into the format we want. You may need to enclose everything in parenthesis and convert it back to date format if excel doesn't recognize it. (DT_DATE)(...)
There are 2 string columns on an inbound file, both formatted YYYYMMDD.
If column 1 is after 10/1/2015, add 90 days to the column 1 date and load in column 2. The data needs to be loaded in the system as YYYYMMDD string (so convert it back).
I'm trying to set up a Dervied Column module to do this, but I haven't been having much luck.
Here's what I have so far:
[Column 1] >= (DT_DATE)"2015-10-01" ? DATEADD("dd",90,[Column 1])
Please help.
There might be a simpler way of doing this, but since you are trying to do this with a derived column transform, this is how I did it for now.
Use a derived column transform to Convert column1 to date
Use a derived column transform to check condition and change value
Use a derived column transform to change it back to string
This is the code to change the YYYYMMDD string to date (I named it Column1AsDate)
(DT_DATE)(SUBSTRING(column1, 1, 4) + "-" + SUBSTRING(column1, 5, 2) + "-" + SUBSTRING(column1, 7, 2))
This is the code to check the condition (Now use Column1AsDate from the earlier transfor, I named this column as Column1AfterCondition)
Column1AsDate >= (DT_DATE)"2015-10-01" ? DATEADD("dd", 90, Column1AsDate) : Column1AsDate
This is the code to put it back to string (Using Column1AfterCondition from earlier transform)
(DT_WSTR, 4)DATEPART("yy", Column1AfterCondition) + RIGHT("0" + (DT_WSTR, 2)DATEPART("mm", Column1AfterCondition), 2) + RIGHT("0" + (DT_WSTR, 2)DATEPART("dd", Column1AfterCondition), 2)
Create a sample text file
This is the package
This is the data viewer after condition is checked
This is the data viewer after converting it back to string
So I suppose at the end, you can map the appropriate column to the destination. In this case the "Column1As String". Now that I think of it, I should have probably used better names.. but hopefully you get the picture.
I'm trying to convert the string column [mydate] to a date, and I would like to use the Derived Column Transformation. The problem is this that my dates are like '1/10/2015', '1/1/2015', '11/1/2015' and '11/9/2015'. So the format can change between D/MM/YYYY, D/M/YYYY, DD/M/YYYY, and DD/MM/YYYY.
Can you guide me in creating the expression for the Derived Column Transformation?
I tried to use something like this:
(DT_DATE)(ISNULL([mydate]) == FALSE ? (RIGHT([mydate],4) + "-" + "2" + "-" + LEFT([mydate],1)) : [mydate])
This task is quite tricky since we're not only dealing with unformatted String dates but they are also not in the region netural YYYY-MM-DD format.
To fix this, we can use the follwing expression for the Derived Column:
RIGHT([mydate],4) + "/" + SUBSTRING([mydate],FINDSTRING([mydate],"/",1) + 1,FINDSTRING([mydate],"/",2) - FINDSTRING([mydate],"/",1) - 1) + "/" + SUBSTRING([mydate],1,FINDSTRING([mydate],"/",1) - 1)
The expression is quite long but what it is doing is taking the values (month, day, year) between the forward slashes / and concatenating them into a format that resembles YYYY/MM/DD that can then be converted in SSIS using a Data Conversion transformation. This avoids the error of dealing with the change in length in dates like 1/1/2000 and 10/10/2000.
The output of the derived column was named YYYYMMDD and this value was then passed into a Data Conversion transformation that has output Date Converted YYYYMMDD as seen below.
The Data Conversion task is simply doing the follwing:
You can use Replace to solve this
Replace(ColumnName,"/","-")
I have the below data in an excel source:
Date: 1-Jun-15
Received: 10
Answered: 9
AvgWaitTime: 0:00:05
AvgHandleTime: 0:00:07
Abandoned: 1
The destination table is as below:
Date date,
Received float,
Answered float,
AvgWaitTime time(7),
AvgHandleTime time(7),
Abandoned float
When I am trying to import the data I am getting error in Input column date as sql is unable to convert it to date format. I surpassed this by creating a new table as per the suggestion given by sql. In the new table the data type for the date column is datetime. Then I got stuck in AvgWaitTime as is sql is unable to convert it to time format.
Is there a way to convert all the required column data from source and then put it to destination? Thanks in advance.
In the destination table you could change the type of the column date to string. It should work. Then you could use a derrived column placed in your task between the source and destination to change it back to date (type datetime, derrived column tab use replace). Untested code below:
> (DT_DATE)("20" +
> SUBSTRING([ReceivedDt], 1, 2) + "-" +
> SUBSTRING([ReceivedDt], 3, 2) + "-" +
> SUBSTRING([ReceivedDt], 5, 2))
I have 2 questions, I have a text file with all my data, in the text file I have for example Sply_DT and Imprt_DT.
For Sply_Dt I have to create getdate() and I have it formatted as 2012-10-25 12:04:16.09900000 using (DT_DBTIMESTAMP)(DT_DBDATE)GETDATE() but I want it formatted as MM-DD-YY.
And for Impt_DT, it's in the 5/16/2011 format in dataviewer but when I placed it into a table it looks like 2011-05-16 00:00:00.000 and I want it in MM-DD-YY format.
I think you have some confusion about the datetime data type. It does not care whether your locale is US (mm/dd/yyyy), Japan (yy/mm/dd) or the UK (dd/mm/yyyy), it will always be stored in the internal format.
If you don't like the default presentation, you can investigate SET DATEFORMAT and perhaps that makes sense for your query.
You can also apply the appropriate CONVERT format when you are querying the data to make it in your intended format.
DECLARE #datevar datetime = '2012-10-25'
SELECT CONVERT(char(10), #datevar, 10) AS YourFomat, #datevar AS defaultFormat
If I have misunderstood your question, please clarify.
An easier way to do it using the Derived Column component is simply like the following (for MM-DD-YY format):
LEN((DT_WSTR, 2)MONTH(GETDATE())) == 1 ? "0" + (DT_WSTR, 2)MONTH(GETDATE())) : (DT_WSTR, 2)MONTH(GETDATE())) + "-" + LEN((DT_WSTR, 2)DAY(GETDATE())) == 1 ? "0" + (DT_WSTR, 2)DAY(GETDATE())) : (DT_WSTR, 2)DAY(GETDATE())) + "-" + RIGHT((DT_WSTR, 2)YEAR(GETDATE()), 2)
As I understand it you're aiming to alter the datetime format of data coming from the text file. I recommend you use a derived column transform in a data flow task to either add a column or replace the existing column, then you can use more common .NET date operators and format strings within the derived column to first parse the date, then to convert it to a string with the given format. If that does not work, you can instead use a script component in the data flow task to do what I described, in which case you have access to .NET to perform your modifications.