I am trying to load some data from a .csv file into my SQL Server. There is a column for date which has datatype Unicode (WSTR) in the .csv file and the column for storing that date in SQL Server is of Datetime data type.
When I used DATA CONVERSION transformation to convert WSTR data to DBTIMESTAMP data, it got changed but with an error that it interchanged the month and date which gives me the wrong date.
The date should be like 2019-09-03 (for 3rd Sep 2019), but I get 2019-03-09.
Please suggest what the issue is that I am facing?
Problem
This may occurs when converting a string to a date value without specifying the date format. Reffering to the SSIS data conversion transformation official documentation:
If you are converting data to a date or a datetime data type, the date in the output column is in the ISO format, although the locale preference may specify a different format.
Assume that the data is stored in the csv file with the following date format MM/dd/yyyy and the default date format in the regional settings is dd/MM/yyyy then date values will not be converted properly.
Solution
In order to specify the date format you have to use a Script Component or a Derived column.
Derived Column
You have to reorder the date part manually, you can use TOKEN() and TOKENCOUNT() function to do that, the following expression convert dd/MM/yyyy format into yyyy-MM-dd universal format:
TOKEN([DateColumn],"/",3) + "-" + RIGHT("0" + TOKEN([DateColumn],"/",2),2) + "-" +RIGHT("0" + TOKEN([DateColumn],"/",1),2)
Script Component
You can use DateTime.ParseExact() function to parse a date based on a specific format:
DateTime.ParseExact(Row.DateColumn,"MM/dd/yyyy",System.Globalization.CultureInfo.InvariantCulture");
Related
I have a situation where I am getting dates in two separate formats, MM/dd/yyyy & yyyy-dd-MM, AND there might be even more different formats as well in csv which will be obviously in string.
Below are the data which currently come as String from CSV-
1/14/2022 0:00
2021-12-31 00:00:00
I am using a Dataflow task in ADF to load the data into Azure SQL where the default format it uses should be yyyy-MM-dd HH:mm:ss.
how can I do this?
ok, i managed to build a quick demo.
Main idea of my solution:
you need to differentiate between valid rows and rows that needs to be modified.
in order to do so, i used case condition.
the idea is to add a derived column with a name 'Date' and modify only needed rows.
Input Data:
i created a csv file and saved my data as a dataset in ADF.
ADF:
In source, i select my dataset as an input.
in a derived column activity:
added a new derived column with a name 'Date' , value :
case(contains(split(Date,''),#item=='/'), toString(toTimestamp(Date,'MM/dd/yyyy H:mm'),'yyyy-MM-dd HH:mm:SS'), Date)
in toTimestamp method, i added first the dateFormat of my input Date and in toString the desired format that i want to cast the date to it.
Output:
P.s
You can cast all possible date formats that will appear in your data in that way.
you can read more about it here:
https://learn.microsoft.com/en-us/azure/data-factory/data-flow-expressions-usage#toTimestamp
My date is stored in a type date column in dd/mm/yyyy format. I want to print the date in yyyymmdd format.
When i used the following formula
tonumber(totext(db.colname,'YYYYMMDD'))
It gave me a "the string is non numeric" error when previewing the report.
Secondly,
My time is stored in a string column in 12 hour format. I want to display it as hh24miss format.
How do i do that ?
First of all, you should NOT store dates/times as text in your database.
Use the apropriate datatype of your DBMS.
Otherwise you will very likely have further problems because of this.
When you've changed the datatype you could just drag&drop the database-field inside your report and use the formatting options of Crystal Reports to get the desired format.
If for any reason (I doubt there's a good one) you can't change the datatype, use the following formula.
ToText(Date({db.colname}), "yyyyMMdd")
This formula converts the string to a date and then formats the date with yyyyMMdd format.
Notice the uppercase M which is used for the month. Lowercase m is used for minutes.
I am loading a flat file to a database table, and need to change the format of the date from YYYY-MM-DD in the flat file, to MM/DD/YYYY in the database table. I tried using the following statement in Derived Columns as shown below, but not sure how to configure the statement, so I got an error message stating that SSIS could not parse the expression.
Derived Column Name: EFF_DATE
Derived Column: Replace EFF_DATE
Expression: TOKEN( MONTH([EFF_DATE]),"//|",DAY([EFF_DATE]),"//|",YEAR([Copy of EFF_DATE]) )
DATA TYPE: databasetimestamp[DT_DBTIMESTAMP]
Can anyone help me determine how to change the format of the column in Derived Column? Otherwise, please let me know if there is another way to do it. Thank you.
This question was different from the last one. In the last question, the date column was data type DateTime. But in this question, the date is a string, and when I used the Derived Column to change the date from YYYY-MM-DD to MM/DD/YYYY, it kept the leading zeroes in MM and DD. The issue then became, not just changing the date format, but also removing the leading zeroes from the Month and Day.
However, I researched and came up with a better solution in SSIS for changing the date value with data type string, as the database I am working with stores the date in that format.
I removed the Derived Column from my Data Source Task, and added an Execute SQL Task in the Control Flow, then added the following Update statement which not only changes the format from YYYY-MM-DD to MM/DD/YYYY, but also removes the leading zeroes from Month and Day. The CONCAT function I used the sample SQL below changes the format from YYYY-MM-DD to MM/DD/YYYY, while the Convert function changes the MM and DD values to data type INT which removed any leading Zeros. This solution allowed the date to remain a string, as that was the table format I had to work with.
UPDATE [StagingTable]
SET START_DATE =
CONCAT( CONVERT(INT, SUBSTRING(START_DATE, 6,2)), '/', CONVERT(INT, RIGHT(START_DATE, 2)),'/', LEFT(START_DATE,4) )]
Thanks to everyone for their comments, as it helped me to think outside the box and determine this solution.
I have a particular situation that I can't quite figure out.
I have a field in a SQL table that is saved as a DATETIME in the following format:
2016-11-26 00:00:00
This is all good-and-well, except that I need it in dd/MM/yyyy format.
I don't want to use the CONVERT(VARCHAR(10), #dateField, 103) function in SQL because I want a DATE format, not a STRING format.
I export a result DataTable to Excel using the EPPlus library and I need the date fields in the export in this exact format.
I tried:
FORMAT(#dateField, 'dd/MM/yyyy', 'en-gb') as well as casting this back to a DATE type. It still implicitly converts to VARCHAR or NVARCHAR
Casting #dateField to DATE and SMALLDATETIME
Using C# to try and convert this to a date in the correct format. DateTime.ParseExact(dataTable.Rows[i]["DateField"].ToString(), "dd/MM/yyyy", CultureInfo.InvariantCulture);
EPPlus column formatting: workSheet.Column(19).Style.Numberformat.Format = "dd/MM/yyyy";
I'm running out of options on this conversion because none of my attempts are working for a DateTime format. Every path I've taken converts my value(s) back to a string. Any assistance would be appreciated :)
If it's a datetime column it's saved as binary representation of the date; it's only when rendered in a client application such as management studio is it rendered in some format.
So if you need it in a particular format you mean rendered as a string in that format. You can always convert back to a datetime (and should!) when saving back to the database. For example, you may want to trim the time part from a datetime before saving.
In the crystals I have to convert the date into date Part as below format.
If date is 1) 23/09/2015 the convert in the format like Sept-2015
2) 21/3/2015- mar-2015
3) 19/2/2015 feb-2015
check it also possible in Sql server.
In Crystal Report Design, Right Click on date field->select Format field. Now you can see more date formats including your required format