I am working with ssis. I have a column that contains date in format integer YYYYMMDD: For example to day I get 20190214.I am using derived column to convert it in format date YYYY-MM-DD in my case for example 2019-02-14
But sometimes I receive wrong values for example 20188101. How could I test that I have good values to be converted to date?
There are 3 methods to achieve that
(1) Using Another Derived Column
Beside of the first Derived COlumn, you can add another derived column with the following expression:
(DT_DATE)(LEFT([DateColumn],4) + "-" + SUBSTRING([DateColumn],5,2) + "-" + RIGHT([DateColumn],2))
If the date value is incorrect redirect the bad from Error Output configuration , and you can handle these bad values from the Error Output.
Helpful Links
USING THE ERROR OUTPUT OF THE DERIVED COLUMN
How to prevent CAST errors on SSIS?
(2) Using a Script Component
Add a script component to check if the value is correct, and add a Output Column of type i.e. OutDateColumn, use a similar code (VB.NET)
IF Not Row.DateColumn_IsNULL Then
dim dtDate as DateTime
If DateTime.TryParseExact(Row.DateColumn,"yyyyMMdd",System.Globalization.InvariantCulture,System.Globalization.DateTimeStyles.None , dtDate ) Then
Row.outDateColumn = dtDate.ToString("yyyy-MM-dd")
Else
Row.outDateColumn_IsNull = True
End If
Else
Row.outDateColumn_IsNull = True
End If
(3) Add a Data Conversion Transformation
After the derived column Transformation, add a Data COnversion Transformation and try to convert the Date Derived Column you add to DT_DATE, if conversion failed, than handle the bad values from Error Output as explained in the first method.
Related
I have project parameter whose value will be like 'Group_AR_21Sep2021' and I want to fetch this date and put it into one of the column of the table which I am loading from Excel. So I put this in derived column (DT_DBDATE)(SUBSTRING(#[$Project::ARFileName],10,18)) but it is throwing me this error
[Derived Column [2]] Error: Casting expression "(SUBSTRING(#[$Project::ARFileName],10,18))" from data type "DT_WSTR" to data type "DT_DBDATE" failed with error code 0xC00470C2.
The second parameter to the substring function should be the length of the date: (DT_DBDATE)(SUBSTRING(#[$Project::ARFileName],10,9))
However, this would still fail because SSIS is super fragile when typecasting to date. I would suggest using a script component instead:
1 - Select the project parameter in the read only variables collection
2 - Add a column to the output columns and make it DT_DBDATE, i.e. NewDate
3 - Update the script to do the cast to the new column:
public override void Input0_ProcessInputRow(Input0Buffer Row)
{
Row.NewDate = Convert.ToDateTime(Variables.ARFileName.Substring(9,9));
}
Good Day,
I am trying to insert records from csv file into my database table. Problem is in inserting alphanumeric values.
My column datatype is set to NUMERIC(19,0), in this column I am expecting some numeric values to be inserted from. For some specific reasons I am getting alphanumeric values in my csv file. For example:
I am getting value: GBS1182000945008.
My goal here is to remove those three characters and cast the remaining string as Numeric and get it inserted inside my table.
So far I have tried:
CAST((select substring(?,4,30)) AS NUMERIC)
But, I am still getting that annoying error, I cannot just ignore those values by using TRY_CONVERT as I do need those records in my database. What am I missing here?
Edit: I have tested this code separately and it is working as expected, only problem is in using it while inserting values. What I have done is that, I checked whether the given parameter is numeric or not, if it is I am just inserting the param if not then I am converting that param into numeric.
So here is my whole scenario:
If (SELECT ISNUMERIC(?)) = 1 {
// Just insert the parameter as:
Insert INTO table (NUMERIC_FIELD) VALUE(?)
}
ELSE {
Insert INTO table (NUMERIC_FIELD) VALUE(CAST((select substring(?,4,30)) AS NUMERIC))
}
Here ? represents the value from CSV.
Try AS NUMERIC(19,0) instead of AS NUMERIC
Also, please note you can have 30 digits in the extracted substring (it will not fit your 19 digis of the column datatype.
I have an issue concerning conversion in SSIS.
I'm trying to convert StartDATE from DT_WSTR to Datetime2 (for SQL Server)
My date originaly looks like this 20140804 but I need to convert it to Datetime2 in such format 2014-08-04 00:00:00.0000000.
What I've done earlier with the StartDATE Column is:
RTRIM(DATSTHYRA)
Since I need to remove blank spaces...
I figured I can use the already Derived Column and add a new expression to convert it to Datetime2 but I'm running into issues and can't really find a topic online that covers my issue.
You can do it in a single step.
Add Derived Column transformation - transform your YYYYMMDD string to YYYY-MM-DD with SUBSTRING functions and then - cast to DT_DBTIMESTAMP2 with scale needed. This would yield an expression like
(DT_DBTIMESTAMP2, 7)(SUBSTRING([StartDATE],1,4) + "-" + SUBSTRING([StartDATE],5,2)
+ "-" + SUBSTRING([StartDATE],7,2))
Then configure Error Output on this Derived Column transformation to capture and handle conversion errors.
In SSIS, you can use data conversion transformation, the data type mapping is database timestamp with precisionin SSIS is for datetime2 in SQL Server.
I'm creating a table B from an exisitng table A. In the table A I have a column ValDate which is varchar and contains Date. When I try to create the table B I have a function used in the query as given below and I get a conversion error. Table A contains null values as well.
MS Access:
((DateDiff("d",Date(),format(Replace(Replace([Table A].ValDate,".","/"),"00/00/0000","00:00:00"),"dd/mm/yyyy")))>0)).
Tables were in MS Access and are being migrated to SQL Server 2012.
SQL Server:
((DATEDIFF(day,FORMAT( GETDATE(), 'dd-MM-yyyy', 'en-US' ),FORMAT( ValDate, 'dd-MM-yyyy', 'en-US' ))>0))
or
((DateDiff(day,GETDATE(),Format(Replace(Replace([TableA].[ValidFrom],'.','/'),'00/00/0000','00:00:00'),'dd/mm/yyyy')))
I tried converting the date using several approachs like Convert , Format and Cast but I end up getting error as below.
Msg 8116, Level 16, State 1, Line 1
Argument data type date is invalid for argument 1 of isdate function.
I would really appreciate someone telling me what I'm missing here.
since you have date data in a string field is very likely you have some value that is not valid against your expected date format.
copy the data in a sql server table and then perform check and validation of the content of the string field.
have a look to the function try_convert that can be helpful when checking the content of the string field containing the date values.
when bad data is ruled out you can apply again your formula with (hopefully) a different result.
a better solution would be to create a separate field with appropriate datatype to store date values converted from the string field and apply your logic to that field.
I'm working in ETL process where I have a column in a table varchar with data as 28JUN1952.
This will always be in this format i.e. 2 digits for date, 3 letters for months and 4 numbers for years.
I was able to convert this into a date column by I need this to come in yyyyMMdd format i.e. 28JUN1952 become 19520628. I was able split the years months and date, but unable to add a padding in the month if it's a 1 digit month for e.g. Jan or Feb.
Is there any other way to convert the data format or any other alternative to add a padding?
I need the conversion in SSIS only.
The easiest route will be to add a Script Transformation.
On the Input and Output tab, expand the Output 0, select Output Columns, click Add Column. 2. Rename Column to something like ccyymmdd and change the data type. I'm not sure what your target type is (int32, string?) but I'll assume string as I'm lazy
On the Input Columns tab, select the string date source column (assuming it's called srcDate)
Inside the script task, assuming C#, use code like this
Inside script task
public override void Input0_ProcessInputRow(Input0Buffer Row)
{
DateTime x = DateTime.MinDate;
if (DateTime.TryParse(Row.srcDate, out x))
{
Row.ccyymmdd = x.ToString("yyyyMMdd");
}
else
{
// Handle NULL values however you wish
// Assign "known bad value"
// Row.ccyymmdd = "99991231";
// Or set the null property to true (preferred)
Row.ccyymmdd_IsNull = true;
}
}
Try this:
declare #d varchar(20)='28JAN1952';
SELECT replace(CONVERT(date,RIGHT(#d,4)+SUBSTRING(#d,3,3)+LEFT(#d,2)),'-','')