I'm trying to convert the string column [mydate] to a date, and I would like to use the Derived Column Transformation. The problem is this that my dates are like '1/10/2015', '1/1/2015', '11/1/2015' and '11/9/2015'. So the format can change between D/MM/YYYY, D/M/YYYY, DD/M/YYYY, and DD/MM/YYYY.
Can you guide me in creating the expression for the Derived Column Transformation?
I tried to use something like this:
(DT_DATE)(ISNULL([mydate]) == FALSE ? (RIGHT([mydate],4) + "-" + "2" + "-" + LEFT([mydate],1)) : [mydate])
This task is quite tricky since we're not only dealing with unformatted String dates but they are also not in the region netural YYYY-MM-DD format.
To fix this, we can use the follwing expression for the Derived Column:
RIGHT([mydate],4) + "/" + SUBSTRING([mydate],FINDSTRING([mydate],"/",1) + 1,FINDSTRING([mydate],"/",2) - FINDSTRING([mydate],"/",1) - 1) + "/" + SUBSTRING([mydate],1,FINDSTRING([mydate],"/",1) - 1)
The expression is quite long but what it is doing is taking the values (month, day, year) between the forward slashes / and concatenating them into a format that resembles YYYY/MM/DD that can then be converted in SSIS using a Data Conversion transformation. This avoids the error of dealing with the change in length in dates like 1/1/2000 and 10/10/2000.
The output of the derived column was named YYYYMMDD and this value was then passed into a Data Conversion transformation that has output Date Converted YYYYMMDD as seen below.
The Data Conversion task is simply doing the follwing:
You can use Replace to solve this
Replace(ColumnName,"/","-")
Related
I'm having a FlatFile of Call detailed record data (CDR) There are two columns containing a string date MM/DD/YYYY and a time column with the format HH:MM:SS.s. I would like to merge these two columns into a datetime2 datatype, however, I'm not able to achieve my desired goal.
I have tried to stack two Derived Column ontop of each other with the first one converting the data format to YYYY-MM-DD using the following expression
((DT_WSTR,4)YEAR(((DT_DATE)[6]))) + "-" + RIGHT("0" + ((DT_WSTR,2)MONTH(((DT_DATE)[6]))),2) + "-" + RIGHT("0" + ((DT_WSTR,2)DAY(((DT_DATE)[6]))),2)*
* MM/DD/YYYY is stored in [6]
* Validated outputs YYYY-MM-DD
Witin the second Derived Column I'm creating a column called StartDateTime
Exp: (DT_DBTIMESTAMP2,1)((DT_WSTR,10)SDATE + (DT_WSTR,10)7)
* SDATE comes from the first derive, 7 is time HH:MM:SS.s
First try to concatenate these two columns using derived column (as you say both are string type) and then use data conversion task to convert the merged column to T_DBTIMESTAMP2
I was able to get a working pipeline, was mostly some CAST misinterpretations within my previous code.
Derived column one: ((DT_WSTR,4)YEAR(((DT_DATE)[6]))) + "-" + RIGHT("0" + ((DT_WSTR,2)MONTH(((DT_DATE)[6]))),2) + "-" + RIGHT("0" + ((DT_WSTR,2)DAY(((DT_DATE)[6]))),2)
Derived column two: (DT_DBTIMESTAMP2,1)(SDATE + [7])
I have a fixed format .txt file which has a date field. This field may have a date or be blank. I have wracked my brains to get this field to convert from DT_STR to DT_DBDATE in the Derived Column transformation. I have tried multiple scenarios:
[CHNG_DT] == "" ? ISNULL((DT_DBDATE)[CHNG_DT] ) : (DT_DBDATE)SUBSTRING([CHNG_DT], 1, 4)) + "-" + (DB_DBDATE)(SUBSTRING(CHNG_DT], 6, 2)) + "-" + (DB_DBDATE)(SUBSTRING([CHNG_DT], 9, 2))
[CHNG_DT] == "" ? NULL(DB_DBDATE) : (DB_DBDATE)[CHNG_DT]
ISNULL([CHNG_DT]) ? NULL(DT_DBDATE) : (DT_DBDATE)((DT_STR, 10, 1252)[CHNG_DT])
ISNULL(CHNG_DT) ? NULL(DT_DBDATE) : (DT_DBDATE)((DT_STR,10,1252)CHNG_DT)
LEFT([CHNG_DT], 10) == " " ? ISNULL((DT_DBDATE)[CHNG_DT]) : (DT_DBDATE)[CHNG_DT]
(DT_DBDATE)CHNG_DT
First of all, when you are looking to convert values from DT_STR to DT_DBDATE, you should make sure that those values are stored in the yyyy-MM-dd format. You can refer to the following official documentation to learn more:
Converting Between Strings and Date/Time Data Types in SSIS
The second suggestion is to use the LTRIM() and RTRIM() functions to make sure that there are no white blanks in the beginning and end of the date values. As an example:
LTRIM(RTRIM(REPLACENULL([CHNG_DT],""))) == "" ? NULL(DT_DBDATE) : (DT_DBDATE)LTRIM(RTRIM([CHNG_DT]))
If date values are stored in a different format, you should write an expression that changes the format before converting it to DT_DBDATE. As an example:
How to convert string in format dd.mm.yyyy to date using SSIS expression?
Other helpful answers:
SSIS Source Format Implicit Conversion for Datetime
Convert DDMonYY and time to datetime column in SSIS package (Derived Column)
The year function doesn't support dt_wstr
I'm using SSIS ETL (I'm a newbie) and trying to extract dates from a SQL server table in format "08/03/2013 00:00:00" to an Excel 2013 file. I need the dates in Excel to be in format "dd/mm/yyyy", they must remain as dates so that the hierarchy remains. ie when you do an autofilter on the column you can see the date groups. There are also some null entries in the column of data.
Things I have tried in ETL script:-
1)Direct conversion from SQL table to Excel file - Result "2013-03-08 00:00:00",yyyy/mm/dd ss:ss:ss no hierarchy. I have tried changing the Excel column format but no matter what I select the date stays the same.
2)I have then added a Date Conversion to the ETL, to change the SQL date to DT_DATE - Result "3/8/2013", m/d/yyyy, no hierarchy.
3)I have then tried a Date Conversion, to change the SQL date to DTW_STR - Result "2013/03/08 00:00:00", yyyy/mm/dd ss:ss:ss no hierarchy.
4)I have then tried various ETL conversions to strings but none are recognised by Excel as dates so no hierarchy.
I'm not in a position to "re-educate" users to a new date format of "yyyy/mm/dd"
I'm not a strong coder so would prefer to do this within ETL if possible.
So to summarise what I'm after I need the dates in Excel to be actual dates that can be grouped as such in format dd/mm/yyyy.
I believe what should do is try using the derived column transformation.
Derived column name: newDate
Derived column: Replace 'originalDate'
Expression: RIGHT("0" + (DT_WSTR, 2)DATEPART("dd", [originalDate]), 2) + "/" + RIGHT("0" + (DT_WSTR, 2) DATEPART("mm", [originalDate]), 2) + "/" + (DT_WSTR, 4)DATEPART("yyyy", [originalDate])
This will take each part and convert it into a string format so that we can concatenate them back together into the format we want. You may need to enclose everything in parenthesis and convert it back to date format if excel doesn't recognize it. (DT_DATE)(...)
I have an issue concerning conversion in SSIS.
I'm trying to convert StartDATE from DT_WSTR to Datetime2 (for SQL Server)
My date originaly looks like this 20140804 but I need to convert it to Datetime2 in such format 2014-08-04 00:00:00.0000000.
What I've done earlier with the StartDATE Column is:
RTRIM(DATSTHYRA)
Since I need to remove blank spaces...
I figured I can use the already Derived Column and add a new expression to convert it to Datetime2 but I'm running into issues and can't really find a topic online that covers my issue.
You can do it in a single step.
Add Derived Column transformation - transform your YYYYMMDD string to YYYY-MM-DD with SUBSTRING functions and then - cast to DT_DBTIMESTAMP2 with scale needed. This would yield an expression like
(DT_DBTIMESTAMP2, 7)(SUBSTRING([StartDATE],1,4) + "-" + SUBSTRING([StartDATE],5,2)
+ "-" + SUBSTRING([StartDATE],7,2))
Then configure Error Output on this Derived Column transformation to capture and handle conversion errors.
In SSIS, you can use data conversion transformation, the data type mapping is database timestamp with precisionin SSIS is for datetime2 in SQL Server.
I have 2 questions, I have a text file with all my data, in the text file I have for example Sply_DT and Imprt_DT.
For Sply_Dt I have to create getdate() and I have it formatted as 2012-10-25 12:04:16.09900000 using (DT_DBTIMESTAMP)(DT_DBDATE)GETDATE() but I want it formatted as MM-DD-YY.
And for Impt_DT, it's in the 5/16/2011 format in dataviewer but when I placed it into a table it looks like 2011-05-16 00:00:00.000 and I want it in MM-DD-YY format.
I think you have some confusion about the datetime data type. It does not care whether your locale is US (mm/dd/yyyy), Japan (yy/mm/dd) or the UK (dd/mm/yyyy), it will always be stored in the internal format.
If you don't like the default presentation, you can investigate SET DATEFORMAT and perhaps that makes sense for your query.
You can also apply the appropriate CONVERT format when you are querying the data to make it in your intended format.
DECLARE #datevar datetime = '2012-10-25'
SELECT CONVERT(char(10), #datevar, 10) AS YourFomat, #datevar AS defaultFormat
If I have misunderstood your question, please clarify.
An easier way to do it using the Derived Column component is simply like the following (for MM-DD-YY format):
LEN((DT_WSTR, 2)MONTH(GETDATE())) == 1 ? "0" + (DT_WSTR, 2)MONTH(GETDATE())) : (DT_WSTR, 2)MONTH(GETDATE())) + "-" + LEN((DT_WSTR, 2)DAY(GETDATE())) == 1 ? "0" + (DT_WSTR, 2)DAY(GETDATE())) : (DT_WSTR, 2)DAY(GETDATE())) + "-" + RIGHT((DT_WSTR, 2)YEAR(GETDATE()), 2)
As I understand it you're aiming to alter the datetime format of data coming from the text file. I recommend you use a derived column transform in a data flow task to either add a column or replace the existing column, then you can use more common .NET date operators and format strings within the derived column to first parse the date, then to convert it to a string with the given format. If that does not work, you can instead use a script component in the data flow task to do what I described, in which case you have access to .NET to perform your modifications.