When I use T-SQL to convert a datetime into dd.mm.yyyy for an csv output using SSIS, the file is produced with a dd-mm-yyyy hh:mm:ss which is not what i need.
I am using:
convert(varchar,dbo.[RE-TENANCY].[TNCY-START],104)
which appears correct in SSMS.
Which is the best way to handle the conversion to be output from SSIS?
Not as simple as i thought it would be.
It works for me.
Using your query as a framework for driving the package
SELECT
CONVERT(char(10),CURRENT_TIMESTAMP,104) AS DayMonthYearDate
I explicitly declared a length for our dd.mm.yyyy value and since it's always going to be 10 characters, let's use a data type that reflects that.
Run the query, you can see it correctly produces 13.02.2019
In SSIS, I added an OLE DB Source to the data flow and pasted in my query
I wired up a flat file destination and ran the package. As expected, the string that was generated by the query entered the data flow and landed in the output file as expected.
If you're experiencing otherwise, the first place I'd check is double clicking the line between your source and the next component and choose Metadata. Look at what is reported for the tenancy start column. If it doesn't indicate dt_str/dt_wstr then SSIS thinks the data type is date variant and is applying locale specific rules to the format. You might also need to check how the column is defined in the flat file connection manager.
The most precise control on output format of the date can be achieved by T-SQL FORMAT(). It is available since SQL Server 2012.
It is slightly slower than CONVERT() but gives desired flexibility
An example:
SELECT TOP 4
name,
FORMAT(create_date, 'dd.MM.yyyy') AS create_date
FROM sys.databases;
name create_date
--------------------
master 08.04.2003
tempdb 12.02.2019
model 08.04.2003
msdb 30.04.2016
p.s. take into account that FORMAT() produces NVARCHAR output, which is different from your initial conversation logic, therefore extra cast to VARCHAR(10)) perhaps will be necessary to apply
Related
I have a data factory in Azure that has a pipeline to execute an SSIS package. An extract script is included in this package that will convert a date value from 2019-01-13 00:00:00 to 2019-01-13. After this, a logic app picks up the extract.txt file and sends it to sftp to drop into a folder.
As you can see from my extract script in SSMS converting a varchar data type to date (I receive date in Unix), for Arrival date the following value is given.
Script
Data viewer
However when I open my extract.txt file, the date shows as: 2019-01-13 00:00:00.0000000
Can anyone tell me why they think this may be happening? Thanks
When you map a date column to flat file it will include time even if it doesn't appear in data viewer or ssms.
SSIS solution
Instead of using dateadd() in the source, leave the column as it is, in SSIS use a derived column with the following expression:
LEFT((DT_WSTR,50)DATEADD("s", [start], (DT_DATE)"1970-01-01",10)
Output
2019-01-13
Based on the Cast (SSIS Expression) official documentation:
When a string is cast to a DT_DATE, or vice versa, the locale of the transformation is used. However, the date is in the ISO format of YYYY-MM-DD, regardless of whether the locale preference uses the ISO format.
SQL Server solution
Use the following SQL command:
select convert(varchar(10), dateadd(S, [start], '1970-01-01'),120)
Our business would be providing us a .csv file. One of the columns in the file would be in date format. Now as we know there are many date formats in Excel. The problem is that we need to check whether the date provided is a correct date. It could be in any format like ddmmyyyy, yyyymmdd, dd-mon-yyyy etc basically any format that Excel supports.
We are planning to first load the data in a staging area and the date field would be defined as varchar so that it can accept any data.
Now either using SSIS or via T-SQL, I need to check whether the date provided is actually a date and if it is I need to load it into a different table in YYYYMMDD format.
How do I go about doing the above?
Considering you have your excel data already loaded into a SQL Server table as varchar (you can easily do this using SSIS), something like this would work:
SELECT
case when ISDATE(YOUR_DATE) = 1 then CONVERT(int,YOUR_DATE,112) else null end as MyDate
FROM
YOUR_TABLE
I don't have access to a SQL Server instance at the moment and can't test the code above, so you may need to adapt to your needs, but this is the general idea.
You can also do further research on ISDATE and CONVERT functions in SQL Server. You should be able to achieve what you need combining them together.
I have failed to load a text file's data to my db's table with the persistent error
invalid time format.
I have changed the time format to include a T between the date and time, to no avail. I also substituted the year, month, date delimiter from . to - but the error persisted.
This is how I am attempting to load from a txt file
bcp mydb.dbo.mytable in c:\data.txt -T -S myserver\instance
I then proceed to confirm data types of the fields, prefix-length (this is the length of the delimiter before the field... right?) and the field terminator. I create a format file which when used still yields the same error. The datetime field I am importing is one of four other fields in a typical row.
What am I missing?
EDIT
Here is a typical row
14,1999-01-04T08:08:24.000,1.36000,1.36000
I have also failed when using SSIS my prior work around of first using access is not applicable here. It is either bcp or SSIS
I believe you are using SQL Server 2008 (R2)? If so, please add appropriate tag to your question.
This problem has been reported here at dba.stackexchange.
Following hints were given:
You need to use a generated format file and use -f for both import and export.
Ensure your datetimes are formatted like yyyy-mm-dd hh:mm:ss, so with a space instead of a T in between date and time
Note that I read here the claim that this is a bug in BCP that ships with SQL Server 2008 (R2). Apparantly this was solved in SQL Server 2012+.
all
I have pretty simple pack to bring data from table to table (all MS SQL). However I'm stuck on something strange I have 2 sources and one destination table and it works for 1 and failing for 2.
Destination DDL:
DestTable (
MyDate date,
...
Source 1. Works OK directly in SSIS without any conversion e.g. 20151025 -> date format
MyDateOK varchar(10) -- 2015-10-25
in SSIS presented like: Unicode String [DT_WSTR]
Source 2. Failing with truncation error.
MyDateFail datetime ---2012-07-02 00:00:00.000
in SSIS presented like: DT_DBTIMESTAMP
I'm trying to fix this with Derived column transformation, but this all strange, + I can't use trim on DT_DBTIMESTAMP, what is the best way to handle this flow?
Should datetime(aka dbtimestamp) be accepted by date without any fixes?
Best
Mario
Source 1 should be mapped into DT_STR in SSIS, otherwise you can get implicit ANSI to Unicode conversion which can get messy.
Source 2 should be explicitly converted to DT_DBDATE to get date part only.
Check this link: Integration Services Data Types in order to get a general understanding of database-to-SSIS data types mapping scheme. Also, the Cast (SSIS Expression) might be of some help.
Using SQL Server 2014 SSIS to import an vendor supplied Excel file through the Excel Source Data Flow. Two issues I'm having related to data conversion to the SQL table.
In the file is a text column that has prices (numeric values) in it I can't not get it to transform into a numeric field (decimal(8,2)) in SQL. I have used the Data Conversion data flow task converting it to DT_NUMERIC and it fails to process the field. I have also tried to let it go through the Data Conversion task and converted through a Derived Column casting the field to Numeric. Both fail, I'm at a loss as to how to get this into the database in a Decimal/Numeric format.
In the same file are three date fields with dates that look like 07/18/2015 in Excel. I have tried similarly with the Data Conversion and Derived column to get the date into the database as SQL date formats. I have cast the dates at DT_DBDATE and DT_DBDATE and DT_DBTIMESTANP and neither has worked I have also tried taking the month day and year and rearranging them into the SQL date format with Substring/left/right functions to split the string. Also to no avail.
Here is what I tried:
Excel Source ---> Data Conversion ----> Derived Column -----> OLE DB Destination
In the excel source it recognized the date as text, I leave that be in the data conversion to deal with it in the Derived Column where I have tried.
a. (DT_DBDATE)("20" + RIGHT(TRIM(sale_start),2) + "-" + LEFT(TRIM(sale_start),2) + "-" + SUBSTRING(TRIM(sale_start),4,2)) - I have done this with and without the trim with same results. I have also used Right(sale_start,4).
b. (DT_DBDATE) sale_start
The SQL table is data type DATE. I have also changed it to DATETIME and used DT_DBTIMESTAMP in place of DT_DBDATE above.
I can't change the file I'm receiving it needs to process into the database the way it comes from the vendor. Looking at the data in excel there seems to be no reason it wouldn't be ok.
Any direction on bringing this data in would be much appreciated.
2.
I was able to figure this out although I don't completely understand what the connection setting is doing. Similarly with a XML file this connection setting wasn't necessary although some version of a derived column was, I the above I believe in my XML import.
For the EXCEL Solution:
1) In the Excel File connection I added IMEX=1 to the end of the connection under properties. So the connection string looked like this:
Provider=Microsoft.Jet.OLEDB.4.0;Data Source=C:\SSIS\Test.xls;Extended Properties="EXCEL 8.0;HDR=YES;IMEX=1";
2) Used the following script in the derived column:
ISNULL([Copy of expected_date]) ? NULL(DT_DATE) : (LEN(TRIM([Copy of expected_date])) == 0 ? NULL(DT_DATE) : (DT_DATE)((DT_DBDATE)TRIM([Copy of expected_date])))
Thanks for taking the time to respond.