I am using SSIS to convert some oracle data to sql server. I found for Oracle date timestamp6 like this
26-DEC-82 12.00.00.000000000 AM
will cause the conversion in SSIS to fail
Error: Year, Month, and Day parameters describe an un-representable
I think it's because ssis don't know whether it's 2082 or 1982, so don't know how to convert.How can I convert the oracle dates to something with yyyy for the year part?
update: tried to_char function mentioned by Hadi. I can see the year now is 2682 (most of them). I added a pic showing with to_char and original column for plate_date and sold_date columns. As you can see most of the years are 26xx, two exceptions are 18xx. Can someone explain?
In the Oracle Source use an SQL Command to convert TimeStamp to nvarchar using TO_CHAR() function and use the universal data format yyyy-MM-dd HH:mm:ss:
TO_CHAR(SOURCECOLUMN, 'YYYY-MM-DD HH24:MI:SS')
And in SSIS data flow add a derived column with the following expression:
(DT_DATE)[SOURCECOLUMN]
Or add a data conversion transformation and convert the column to date data type.
In SQL Server, the datatype "timestamp" is nothing to do with dates or times.
Microsoft have renamed their old "timestamp" datatype to "rowversion" because it is just an 8-byte number that is used to record a sequence of “row changed” events.
On the other hand Oracle's "timestamp" really is about time because Oracle's "timestamp" extends their "date" datatype. more here.
Unfortunately, SQL Server still recognises "timestamp" as a valid datatype name.
So, I suspect that your error message may have something to do with the timestamp-timestamp homonym.
Related
I have a data factory in Azure that has a pipeline to execute an SSIS package. An extract script is included in this package that will convert a date value from 2019-01-13 00:00:00 to 2019-01-13. After this, a logic app picks up the extract.txt file and sends it to sftp to drop into a folder.
As you can see from my extract script in SSMS converting a varchar data type to date (I receive date in Unix), for Arrival date the following value is given.
Script
Data viewer
However when I open my extract.txt file, the date shows as: 2019-01-13 00:00:00.0000000
Can anyone tell me why they think this may be happening? Thanks
When you map a date column to flat file it will include time even if it doesn't appear in data viewer or ssms.
SSIS solution
Instead of using dateadd() in the source, leave the column as it is, in SSIS use a derived column with the following expression:
LEFT((DT_WSTR,50)DATEADD("s", [start], (DT_DATE)"1970-01-01",10)
Output
2019-01-13
Based on the Cast (SSIS Expression) official documentation:
When a string is cast to a DT_DATE, or vice versa, the locale of the transformation is used. However, the date is in the ISO format of YYYY-MM-DD, regardless of whether the locale preference uses the ISO format.
SQL Server solution
Use the following SQL command:
select convert(varchar(10), dateadd(S, [start], '1970-01-01'),120)
As part of my job duties, I'm responsible for extracting data from our vendor's Oracle 11g database, and loading it into our SQL Server 2016 database. I've been doing this successfully with SSIS and the Attunity Oracle connectors.
Today I was informed that there was a new column added to the existing Invoices table on the Oracle side. There was already a DATE column called Order Date, which contains valid date values with zero'd times, like 2017-12-25 00:00:00.
The new column is called Order Date Time and is also a DATE column. When I opened up the SSIS package and pulled up the Oracle source in my DFT, I previewed the data and found the values in Order Date Time to be 2432-82-75 50:08:01. I tried converting the column with CAST and all the TO_* functions, but the conversions either failed outright, or returned a string of zeros.
TO_CHAR("Order Date Time", 'YYYYMMDDHH24MISS')
yields 00000000000000
After a bit of Googling for "Oracle date value invalid", I'm now thinking that these DATE values are actually corrupted. Am I missing anything here? Is there some sort of special Oracle-specific technique for storing time values in a DATE column that I may not be aware of?
(And yes, it does bother me quite a bit that our vendor added another DATE column instead of just using the time portion of the existing Order Date column.)
Unfortunately, Oracle database engine allows inserting invalid date values, which leads to many problems especially when importing data to others database engines such as SQL Server.
To handle this issue, you have to implement the logic that fits your needs, as example:
You can exclude these records from you queries by filtering on acceptable date ranges: (WHERE date between ...)
You can Update records with invalid values by replacing with NULL
You can use a CASE statement in your query to replace values with NULL
I faced this issue one time while importing data to SQL Server from an Oracle data source, there was unacceptable date values, i decided to update all records where date are invalid and replace with NULL values before starting the import process.
There are many links related to this issue:
Detecting invalid values in the DB
How to identify invalid (corrupted) values stored in Oracle DATE columns
Corrupt date fields causing query failure in Oracle
Invalid Date in DATE Column SQLPlus VS SQLDeveloper
Ask Tom - date validation in oracle
Dealing with invalid dates
Error: Invalid date format
DB Connect; Oracle DB date field data is corrupt
Using SQL Server 2014 SSIS to import an vendor supplied Excel file through the Excel Source Data Flow. Two issues I'm having related to data conversion to the SQL table.
In the file is a text column that has prices (numeric values) in it I can't not get it to transform into a numeric field (decimal(8,2)) in SQL. I have used the Data Conversion data flow task converting it to DT_NUMERIC and it fails to process the field. I have also tried to let it go through the Data Conversion task and converted through a Derived Column casting the field to Numeric. Both fail, I'm at a loss as to how to get this into the database in a Decimal/Numeric format.
In the same file are three date fields with dates that look like 07/18/2015 in Excel. I have tried similarly with the Data Conversion and Derived column to get the date into the database as SQL date formats. I have cast the dates at DT_DBDATE and DT_DBDATE and DT_DBTIMESTANP and neither has worked I have also tried taking the month day and year and rearranging them into the SQL date format with Substring/left/right functions to split the string. Also to no avail.
Here is what I tried:
Excel Source ---> Data Conversion ----> Derived Column -----> OLE DB Destination
In the excel source it recognized the date as text, I leave that be in the data conversion to deal with it in the Derived Column where I have tried.
a. (DT_DBDATE)("20" + RIGHT(TRIM(sale_start),2) + "-" + LEFT(TRIM(sale_start),2) + "-" + SUBSTRING(TRIM(sale_start),4,2)) - I have done this with and without the trim with same results. I have also used Right(sale_start,4).
b. (DT_DBDATE) sale_start
The SQL table is data type DATE. I have also changed it to DATETIME and used DT_DBTIMESTAMP in place of DT_DBDATE above.
I can't change the file I'm receiving it needs to process into the database the way it comes from the vendor. Looking at the data in excel there seems to be no reason it wouldn't be ok.
Any direction on bringing this data in would be much appreciated.
2.
I was able to figure this out although I don't completely understand what the connection setting is doing. Similarly with a XML file this connection setting wasn't necessary although some version of a derived column was, I the above I believe in my XML import.
For the EXCEL Solution:
1) In the Excel File connection I added IMEX=1 to the end of the connection under properties. So the connection string looked like this:
Provider=Microsoft.Jet.OLEDB.4.0;Data Source=C:\SSIS\Test.xls;Extended Properties="EXCEL 8.0;HDR=YES;IMEX=1";
2) Used the following script in the derived column:
ISNULL([Copy of expected_date]) ? NULL(DT_DATE) : (LEN(TRIM([Copy of expected_date])) == 0 ? NULL(DT_DATE) : (DT_DATE)((DT_DBDATE)TRIM([Copy of expected_date])))
Thanks for taking the time to respond.
I have migrated an Access database to SQL server. Many of my "dates" were stored in the Access database in the format "DD/MM/YYYY". However, I notice the SSMA has updated all date columns to the format "DD-MM-YYYY HH:MM:SS". What type should we choose in SQL Server to accomplish the same? The thing that is I want to keep it this way as else we need to change the underlying code.
Much appreciated for your help!
According to this http://social.msdn.microsoft.com/Forums/en-US/4920b4f5-6855-4855-96a9-43f9365d63a0/change-sql-server-date-format the format SQL Server stores datetime fields is generic. You can convert the datetime fields and convert it to varchar in order to show the formal you want.
For example this
convert(varchar, datimefield, 103)
will convert the datetime field using the format 'dd/mm/yyyy'
You should be able to use the date datatype to not store the time, as according to This Link, but only in 2008 or 2012
I checked if it works as such on my SQL Server 2008, and it does.
This will still show it with a dash rather then a slash and in the order YYYY-MM-DD though.
I want to copy elements to a remote MS-SQL database.
I got conversion error on it.
When I checked the ADOTable structure I saw the MS field
WHENCREATED DATE [NULL]
is converted to
ftWideString 10
Hmmmmm....
Is it normal? Or I can set something to Date fields are come as TDateTime?
The Provider is "SQLOLEDB.1"
Its a DATE (yyyy-mm-dd) type which was introduced in SQL Server 2008 as an alternative to the DATETIME type.
Because SQLOLEDB.1 precedes this there is a backward conversion to DBTYPE_WSTR, using an updated provider (SQLNCLI) would be preferable.