I am using an OLE DB Command task to execute an INSERT statement. The INSERT statement accepts a number of parameters represented by ?. SSIS maps SQL Server DATETIME columns to parameters of type DT_DBTIMESTAMP which I think is fine.
The INSERT query fails as the DT_DBTIMESTAMP is passed to SQL Server as a string in format 'yyyy-MM-dd hh:mm:ss' but the database is expecting 'dd/MM/yyyy hh:nn:ss'. The error is due to SQL Server treating the 'day' and 'month' parts the wrong way around.
I've seen responses to questions around formatting dates in SSIS using derived columns etc. but I already have the DT_DBTIMESTAMP value (it has no format!) and the problem occurs when SSIS sets the parameter value as a string, and I can't see how to control the format so it outputs in 'dd/MM/yyyy hh:mm:ss'.
I've tried setting LocaleID and language but still seems to make no difference. An interesting observation is that this error does not occur when running through Visual Studio, only from a SQL Agent job.
Any help greatly appreciated.
What errors do you get, when running this?
The only way to solve this, I see in converting DT_DBTIMESTAMP to DATETIME between the reading of the source file and the writing in SQL table.
Related
When I use T-SQL to convert a datetime into dd.mm.yyyy for an csv output using SSIS, the file is produced with a dd-mm-yyyy hh:mm:ss which is not what i need.
I am using:
convert(varchar,dbo.[RE-TENANCY].[TNCY-START],104)
which appears correct in SSMS.
Which is the best way to handle the conversion to be output from SSIS?
Not as simple as i thought it would be.
It works for me.
Using your query as a framework for driving the package
SELECT
CONVERT(char(10),CURRENT_TIMESTAMP,104) AS DayMonthYearDate
I explicitly declared a length for our dd.mm.yyyy value and since it's always going to be 10 characters, let's use a data type that reflects that.
Run the query, you can see it correctly produces 13.02.2019
In SSIS, I added an OLE DB Source to the data flow and pasted in my query
I wired up a flat file destination and ran the package. As expected, the string that was generated by the query entered the data flow and landed in the output file as expected.
If you're experiencing otherwise, the first place I'd check is double clicking the line between your source and the next component and choose Metadata. Look at what is reported for the tenancy start column. If it doesn't indicate dt_str/dt_wstr then SSIS thinks the data type is date variant and is applying locale specific rules to the format. You might also need to check how the column is defined in the flat file connection manager.
The most precise control on output format of the date can be achieved by T-SQL FORMAT(). It is available since SQL Server 2012.
It is slightly slower than CONVERT() but gives desired flexibility
An example:
SELECT TOP 4
name,
FORMAT(create_date, 'dd.MM.yyyy') AS create_date
FROM sys.databases;
name create_date
--------------------
master 08.04.2003
tempdb 12.02.2019
model 08.04.2003
msdb 30.04.2016
p.s. take into account that FORMAT() produces NVARCHAR output, which is different from your initial conversation logic, therefore extra cast to VARCHAR(10)) perhaps will be necessary to apply
I am taking some simple data from an SQL table, making a small transformation and converting it to Unicode. Then I output it into an Oracle CHAR(1 byte) field on an Oracle server.
This works without any error on my local PC. I then deploy to the server and it says that
"Column "A" cannot convert between unicode and non-unicode string data types".
After trying several things I threw my hands up in the air and just took out the data conversion to unicode and now it is broken and wont run on my PC.
BUT - it now works on the server and is all happy. I've searched and found that others have had this problem, but none seem to find the cause and just work around it it other ways.
Why can I not have my PC and my Server work the same? All tables and data connection are the SAME for both. No change other than execution location.
I got the same issue in my SSIS package and found no solution.
It was a simple data, not containing any unicode character, and it doesn't throw any exception if converting it using an SQL query or a .net code ... But it throws an exception in SSIS when using Data Conversion Transformation
Workarounds
I made a simple workaround to achieve this (you can use it if this error occurs again)
I replaced the Data Conversion Component with a Script Component
I marked the columns i want to convert as input
For each column i want to convert i created an output column of type DT_WSTR
In the Script for each column i used the following code: (assuming the input is inColumn and the output is outColumn)
If Not Row.inColumn_IsNull AndAlso _
Not String.IsNullOrEmpty(Row.inColumn) Then
Row.outColumn = Row.inColumn
Else
Row.outColumn_IsNull = True
End If
OR
If the source is an OLEDB Source you can use a casting function in the OLEDB source command
ex:
SELECT CAST([COLUMN] as NVARCHAR(255)) AS [COLUMN]
FROM ....
I am exporting data from Excel to SQL Server through an API. Everything is supposed to save as nvarchar but date is stored as int value (automatically). How can I keep, in SQL table, date value as it is i.e. like 01/01/1990 instead of 32874.
Might be a basic question, Googled and looked into SO but couldn't find what I am after yet. Help appreciated!
In SQL Server, you can convert to a date by using:
select dateadd(day, 32874, '1899-12-31')
There is probably also a way to fix this when importing the data, by treating it as an actual date or string.
Using SQL Server 2014 SSIS to import an vendor supplied Excel file through the Excel Source Data Flow. Two issues I'm having related to data conversion to the SQL table.
In the file is a text column that has prices (numeric values) in it I can't not get it to transform into a numeric field (decimal(8,2)) in SQL. I have used the Data Conversion data flow task converting it to DT_NUMERIC and it fails to process the field. I have also tried to let it go through the Data Conversion task and converted through a Derived Column casting the field to Numeric. Both fail, I'm at a loss as to how to get this into the database in a Decimal/Numeric format.
In the same file are three date fields with dates that look like 07/18/2015 in Excel. I have tried similarly with the Data Conversion and Derived column to get the date into the database as SQL date formats. I have cast the dates at DT_DBDATE and DT_DBDATE and DT_DBTIMESTANP and neither has worked I have also tried taking the month day and year and rearranging them into the SQL date format with Substring/left/right functions to split the string. Also to no avail.
Here is what I tried:
Excel Source ---> Data Conversion ----> Derived Column -----> OLE DB Destination
In the excel source it recognized the date as text, I leave that be in the data conversion to deal with it in the Derived Column where I have tried.
a. (DT_DBDATE)("20" + RIGHT(TRIM(sale_start),2) + "-" + LEFT(TRIM(sale_start),2) + "-" + SUBSTRING(TRIM(sale_start),4,2)) - I have done this with and without the trim with same results. I have also used Right(sale_start,4).
b. (DT_DBDATE) sale_start
The SQL table is data type DATE. I have also changed it to DATETIME and used DT_DBTIMESTAMP in place of DT_DBDATE above.
I can't change the file I'm receiving it needs to process into the database the way it comes from the vendor. Looking at the data in excel there seems to be no reason it wouldn't be ok.
Any direction on bringing this data in would be much appreciated.
2.
I was able to figure this out although I don't completely understand what the connection setting is doing. Similarly with a XML file this connection setting wasn't necessary although some version of a derived column was, I the above I believe in my XML import.
For the EXCEL Solution:
1) In the Excel File connection I added IMEX=1 to the end of the connection under properties. So the connection string looked like this:
Provider=Microsoft.Jet.OLEDB.4.0;Data Source=C:\SSIS\Test.xls;Extended Properties="EXCEL 8.0;HDR=YES;IMEX=1";
2) Used the following script in the derived column:
ISNULL([Copy of expected_date]) ? NULL(DT_DATE) : (LEN(TRIM([Copy of expected_date])) == 0 ? NULL(DT_DATE) : (DT_DATE)((DT_DBDATE)TRIM([Copy of expected_date])))
Thanks for taking the time to respond.
I am connecting to an Oracle DB and the connection works, but I get the following error for some of the columns:
Description: Column "RESOURCE_NAME" cannot convert between unicode
and non-unicode string data types.
Value for RESOURCE_NAME:
For Oracle: VARCHAR2(200 BYTE)
For SQL Server: VARCHAR(200 BYTE)
I can connect to the Oracle DB via Oracle SQL Developer without any issues. Also, I have the SSIS package setting Run64BitRuntime = False.
The Oracle data type VARCHAR2 appears to be equivalent to NVARCHAR in SQL Server, or DT_WSTR in SSIS. Reference
You will have to convert using the Data Conversion Transformation, or CAST or CONVERT functions in SQL Server.
If the package works in one machine and doesn't in other; Try setting the NLS_LANG to right language, territory and character set and test the package.
[Command Prompt]> set NLS_LANG=AMERICAN_AMERICA.WE8ISO8859P1
The easiest way around this to to open the SSIS package in notepad (the dtsx file) and do a global find and replace of all instances of validateExternalMetadata="True" with validateExternalMetadata="False".
note: we encountered this issue when connecting to an Oracle 11g database on Linux through SSIS.
on oledb source ->advanced editor options->input/output columns->output columns->select RESOURCE_NAME column and change Data type as DT_WSTR and length also u can change as required
You can use SQL command in SSIS and use CONVERT or CAST. If SSIS still gives you an error its because of the metadata. Here is how you can fix it.
Open the Advanced Editor.
Under the Input and Output properties, Expand Source Output.
Expand Output columns
Select the column which is causing the issue.
Go to Data Type Properties and change the DataType to your desired type DT_STR, DT_Text etc.
You can just double-click on the "Data Conversion" block in the Data Flow and for every item change it to: "Unicode String [DT_WSTR]"
Works
If everything failed from above. Create a table variable and insert the data into it. Then select all records as source. use SET NOCOUNT ON in the script.
I encountered a very similar problem even using SQL Server rather than Oracle. In my case, I was using a Flat File as a data source, so I just went in to the Flat File Connection Manager and manually changed the column type to be a Unicode string:
I don't know if this would fix your problem or not, but it helped me - hopefully someone else will be helped too. (I was inspired to try that by this previous answer to this question BTW, just to give credit where credit's due).