I am converting a Google Sheet to a SQL database table. When it loads the Excel sheet (I copy the Google Sheet data to Excel and import it) SQL changes some of the data types. I have a column in my Excel sheet that is a date that SQL converts to nvarchar. When I try to convert it back to a date data type, I get this error:
"Conversion failed when converting date and/or time from character
string."
My code seems to be correct...I'll show you.
USE Sample
ALTER TABLE ['RM Item List$']
ALTER COLUMN [Kosher Cert] date
Can someone give me an idea why this doesn't work? I know questions about data conversion have been asked ad nausea, but I didn't see an answer that helped me solve my problem. SQL converted my dates in the Kosher Cert column to strings of five numbers. Any help would be appreciated.
This related topic might be helpful:
T-SQL to convert Excel Date Serial Number to Regular Date
It would probably make most sense to add a column to your table with the date type, populate it and then drop the nvarchar column:
ALTER TABLE ['RM Item List$'] ADD [KosherCert] date;
UPDATE ['RM Item List$']
SET [KosherCert] = DateAdd(Day, Convert(int, [Kosher Cert]), '1899-12-30');
Related
I have a nvarchar(50) column in SQL which is of format mm/dd/yyyy. I am trying to import this data in excel using SSIS. In ssis package, I am creating a column Date_1 with datatype Date in Execute SQL task (Since .xlsx file needs to be created dynamically) and in my data flow task I am converting Date_1 to date(DT_DATE). But after data migration when I check my file column Date_1 is of General data type and not Date. Can someone help me how to convert General column to Date
The drop down list you are showing in the Microsoft Excel interface is not related to the data type it is the Number Format property which is used to change the way the value is shown in Excel.
Available number formats
To change this property you need to use Microsoft.Interop.Excel library within a Script Task and change the Excel.Range.NumberFormat property. As example:
Range rg = (Excel.Range)xlWorksheet.Cells[1,1];
rg.EntireColumn.NumberFormat = "MM/dd/yyyy";
References
How to make correct date format when writing data to Excel
How to: Specify Number or Date Format for Cell Content
As part of my job duties, I'm responsible for extracting data from our vendor's Oracle 11g database, and loading it into our SQL Server 2016 database. I've been doing this successfully with SSIS and the Attunity Oracle connectors.
Today I was informed that there was a new column added to the existing Invoices table on the Oracle side. There was already a DATE column called Order Date, which contains valid date values with zero'd times, like 2017-12-25 00:00:00.
The new column is called Order Date Time and is also a DATE column. When I opened up the SSIS package and pulled up the Oracle source in my DFT, I previewed the data and found the values in Order Date Time to be 2432-82-75 50:08:01. I tried converting the column with CAST and all the TO_* functions, but the conversions either failed outright, or returned a string of zeros.
TO_CHAR("Order Date Time", 'YYYYMMDDHH24MISS')
yields 00000000000000
After a bit of Googling for "Oracle date value invalid", I'm now thinking that these DATE values are actually corrupted. Am I missing anything here? Is there some sort of special Oracle-specific technique for storing time values in a DATE column that I may not be aware of?
(And yes, it does bother me quite a bit that our vendor added another DATE column instead of just using the time portion of the existing Order Date column.)
Unfortunately, Oracle database engine allows inserting invalid date values, which leads to many problems especially when importing data to others database engines such as SQL Server.
To handle this issue, you have to implement the logic that fits your needs, as example:
You can exclude these records from you queries by filtering on acceptable date ranges: (WHERE date between ...)
You can Update records with invalid values by replacing with NULL
You can use a CASE statement in your query to replace values with NULL
I faced this issue one time while importing data to SQL Server from an Oracle data source, there was unacceptable date values, i decided to update all records where date are invalid and replace with NULL values before starting the import process.
There are many links related to this issue:
Detecting invalid values in the DB
How to identify invalid (corrupted) values stored in Oracle DATE columns
Corrupt date fields causing query failure in Oracle
Invalid Date in DATE Column SQLPlus VS SQLDeveloper
Ask Tom - date validation in oracle
Dealing with invalid dates
Error: Invalid date format
DB Connect; Oracle DB date field data is corrupt
I am sorry to post what seems a very simple issue but I cannot find an answer and I am wasting days (not just hours at this point). I am fairly new to SSIS and it is just kicking my backside.
Background:
Pretty straightforward SSIS Package to import an Excel sheet into a Staging table in SQL Server. Since I do not want to mislead anyone by using the wrong nomenclature, I will refer to the Excel source as Excel and the SQL Server table as the Target Table.
This package HAS worked before. However, it is now failing because of data truncation for a Date Column. The Excel Column has been formatted as a DATE (and I have tried a few different format options within DATE). The target column is also a DATE column (NOT datetime). The data in Excel is predominantly empty cells with a few sporadic values. I think the errors started when the dates started appearing in the data (rather than just blanks).
I have tried using the Advanced Editor for both sides (Excel & Target) and tried numerous data type settings all around but I keep getting the same failure. I suspect that it is now pretty messed up with the various tests that I have done.
I have also tried adding a Data Conversion transform for the Date field “date[DT_DATE]” – that did not work. AND, I have tried creating a Derived Column - first based on the Excel column and then on the Transformed column. All of those attempts have failed.
Questions:
1) What is the best practice for importing Excel data into SQL Server for DATE Columns?
2) Since this is two very mature Microsoft Apps (Excel & SQL Server) working together, it seems like it should be simple. This leads me to believe that I must be missing some basic concepts here. Can anyone set me straight?
3) How do all of you get an Excel date into SQL Server?
4) What is the trick for synchronizing columns after making edits?
Thanks for any insights you can provide. Sorry to bother you all with what seems pretty simple.
David
Personally I don't think there is a best practice for excel dates, it is always a pain for me.
If you can format the excel file try changing it to 'Text'. it will import as Unicode and not a date. If not, try and convert the column in a "Data Conversion
" task to Unicode
after that is done, you would need to use a "Derived Column" task. Build the date in the format you want.
example for source MM/dd/yyyy hh:mm:ss
Build to be yyyy-MM-dd
SUBSTRING(datecolumn,7,4)+ "-" + SUBSTRING(datecolumn,1,2)+ "-" +SUBSTRING(datecolumn,4,2)
Might be crude, but saves my sanity.
If the date looks something like m/d/yyyy not including 2 values when Jan or something, you will add a few things like this for the month part.
RIGHT("0" + SUBSTRING(datecolumn,1,FINDSTRING(datecolumn,"/"1)-1),2)
Good luck
The main problem when importing Data from excel worksheets is that excel is that each column in excel can have multiple data types or formats, so the same column can contains Dates and Numbers and text or dates with differents formats (Some formats cannot be converted implicitly to dates in SSIS).
If all date value are stored as date (not Text), The best practice to Import dates from an excel worksheet is to convert DATE to Number format "0.000000000" (in excel it is called Serial DateTime) from excel or programmatically using a library like Microsoft.Office.Interop.Excel
You can refer to this Link but use the following:
xlCells.NumberFormat = "0.0000000"
Then in SSIS package use a script component to convert it again to Date using DateTime.FromOADate() Function
*Assuming that inColumn is the Date column with a numeric type, add an output column outColumn of type DT_DBTIMESTAMP or DT_DATE and use the following code:
If Not Row.inColumn_IsNull Then
Row.OutColumn = DateTime.FromOADate(CDbl(Row.inColumn))
Else
Row.OutColumn_IsNull = True
End If
Note: When converting column to Number Format, you ignored all formats but still have the date value
To read more about DateTimes in Excel you can refer to this Link
To read more about Date time formats that can be implicitly converted to date in SSIS follow SSIS Source Format Implicit Conversion for Datetime
I am exporting data from Excel to SQL Server through an API. Everything is supposed to save as nvarchar but date is stored as int value (automatically). How can I keep, in SQL table, date value as it is i.e. like 01/01/1990 instead of 32874.
Might be a basic question, Googled and looked into SO but couldn't find what I am after yet. Help appreciated!
In SQL Server, you can convert to a date by using:
select dateadd(day, 32874, '1899-12-31')
There is probably also a way to fix this when importing the data, by treating it as an actual date or string.
Using SQL Server 2014 SSIS to import an vendor supplied Excel file through the Excel Source Data Flow. Two issues I'm having related to data conversion to the SQL table.
In the file is a text column that has prices (numeric values) in it I can't not get it to transform into a numeric field (decimal(8,2)) in SQL. I have used the Data Conversion data flow task converting it to DT_NUMERIC and it fails to process the field. I have also tried to let it go through the Data Conversion task and converted through a Derived Column casting the field to Numeric. Both fail, I'm at a loss as to how to get this into the database in a Decimal/Numeric format.
In the same file are three date fields with dates that look like 07/18/2015 in Excel. I have tried similarly with the Data Conversion and Derived column to get the date into the database as SQL date formats. I have cast the dates at DT_DBDATE and DT_DBDATE and DT_DBTIMESTANP and neither has worked I have also tried taking the month day and year and rearranging them into the SQL date format with Substring/left/right functions to split the string. Also to no avail.
Here is what I tried:
Excel Source ---> Data Conversion ----> Derived Column -----> OLE DB Destination
In the excel source it recognized the date as text, I leave that be in the data conversion to deal with it in the Derived Column where I have tried.
a. (DT_DBDATE)("20" + RIGHT(TRIM(sale_start),2) + "-" + LEFT(TRIM(sale_start),2) + "-" + SUBSTRING(TRIM(sale_start),4,2)) - I have done this with and without the trim with same results. I have also used Right(sale_start,4).
b. (DT_DBDATE) sale_start
The SQL table is data type DATE. I have also changed it to DATETIME and used DT_DBTIMESTAMP in place of DT_DBDATE above.
I can't change the file I'm receiving it needs to process into the database the way it comes from the vendor. Looking at the data in excel there seems to be no reason it wouldn't be ok.
Any direction on bringing this data in would be much appreciated.
2.
I was able to figure this out although I don't completely understand what the connection setting is doing. Similarly with a XML file this connection setting wasn't necessary although some version of a derived column was, I the above I believe in my XML import.
For the EXCEL Solution:
1) In the Excel File connection I added IMEX=1 to the end of the connection under properties. So the connection string looked like this:
Provider=Microsoft.Jet.OLEDB.4.0;Data Source=C:\SSIS\Test.xls;Extended Properties="EXCEL 8.0;HDR=YES;IMEX=1";
2) Used the following script in the derived column:
ISNULL([Copy of expected_date]) ? NULL(DT_DATE) : (LEN(TRIM([Copy of expected_date])) == 0 ? NULL(DT_DATE) : (DT_DATE)((DT_DBDATE)TRIM([Copy of expected_date])))
Thanks for taking the time to respond.