SSIS - SQL Server datetimeoffset(0) destination column recognized as DT_WSTR - sql-server

We get data delivered to us in a flat file. A date column we want to store in a destination column called DWValidFrom has the following format:
2017-02-06T22:07:09Z
In SSIS using a Flat File Connection Manager, I set the datatype of said column to DT_DBTIMESTAMPOFFSET. It correctly shows us when checking the data in the Columns and Preview pages of the Connection Manager.
In SQL Server, I created the destination table, and defined the DWValidFrom column as datetimeoffset(0):
[DWValidFrom] [datetimeoffset](0) NOT NULL,
When I attempt to set the mappings in the OLE DB Destination object, which has been set to the SQL Server table in question, SSIS won't have it, and throws the following error:
The OLE DB provider used by the OLE DB adapter cannot convert between types "DT_DBTIMESTAMPOFFSET" and "DT_WSTR" for "DWValidFrom".
Suspecting something off with my regional settings, I issued the following query in Management Studio to ensure the format of the date wouldn't change:
SELECT CAST('2017-02-06T22:07:09Z' AS datetimeoffset(0))
This yielded the following result:
2017-02-06 22:07:09 +00:00
Why is SSIS not recognizing the column's proper data type? I do not have any other conversions or expressions set, so I'm confused as to why SSIS won't allow me to push a valid datetimeoffset.
We're using SQL Server 2014, Visual Studio 2015.
Thanks.

This sounds like the OLEDB source metadata is out of sync with the changes you made on the flat file connection manager. The quickest fix it would be to recreate the OLEDB source, but don't do that quite yet.
SSIS is not going to like that standard ISO format for the date. If you remove the "T" in the middle and the "Z" at the end it be ok. i.e.
2017-02-06 22:07:09
Because of this conversion issue in SSIS, the connection manager will probably fail in converting the string to datetimeoffset. So you will need to configure it as a string and then fix it's value in a derived column:
(DT_DBTIMESTAMPOFFSET, 0) REPLACE(REPLACE( [DWValidFrom] , "T", " " ), "Z", "")
Hope that helps,
m

The issue seemed to be that the OLEDB destination does not recognize datetimeoffset as a valid column format. Despite everything working in SQL Server and SSIS pushing a datetime that would be perfectly valid, the OLEDB destination wouldn't have any of it.
I considered using a SQL Server destination, but because the target server is a different server than the one we develop on, that wasn't an option either.
The fix for us was to instead format the columns using datetime as a datatype, which causes us to loose the timezone info, but because all of the dates were UTC, we really don't miss any data.

Quick Answer: Set DataTypeCompatibility to 0
I noticed in Connection Manager for my SQL Server Native Client 11.0 (OLEDB) connection, clicking on "All", then under the SQLNCLI11.1 section there's a value DataTypeCompatibility which was set to "80". 80 is code for SQL Server 2000 compatibility, well before they introduced TimeStampOffset (or in my case DT_DBDATE and DT_DBTIME2 types). I tried setting compatibility to 130, then 100, but "Test Connection" failed.
At https://learn.microsoft.com/en-us/sql/relational-databases/native-client/applications/using-connection-string-keywords-with-sql-server-native-client?view=sql-server-2017 there's a table, specifying information about this value
DataTypeCompatibility SSPROP_INIT_DATATYPECOMPATIBILITY Specifies the mode of data type handling to use. Recognized values are "0" for provider data types and "80" for SQL Server 2000 data types.
Changing the value to 0, then refreshing all of my connections using the OLEDB connection manager seems to have done the trick - now all my database's types are recognized rather than forcing it to nvarchar/DT_WSTR

Related

Error SSAS (v14.0.1.439) Error of data type convert during the processing of measure group

We migrate from Production SQL Server version 12.0.4100.1 to SQL server version 14.0.3238.1.
And when we process the cube we get this type of error :
Errors in the back-end database access module. OLE DB was unable to convert
a value to the data type requested for column '*****'.
We migrate from French server to English version.
The work around to process successfully the dimension was to cast all the numeric data type into float.
It works but the error stay for the measures group.
We try to change the data type in the properties of the measures but it doesn't work.
Is that a problem of version compatibility or a cast to add on any columns ?
you can change Datatype manually to Double it can works

Best way to handle time formats from SQL Server in Access front end

I've migrated a database formerly in Access to SQL Server and am now rebuilding my Access front end to work with that SQL Server back end using a DSN-less link. I'm running into issues with new data entry in my time field. The error I get is ODBC--update on a linked table...failed. [Microsoft][ODBC Driver 17 for SQL Server]Invalid character value for cast specification (#0). I'm assuming this has to do with the way Access converts the data into short text from SQL Server, where it is a time(0) data type.
My question is what is the best way to handle "time" data to work in both Access and SQL Server? Ideally users would enter data in Access simply as something like "0130" where this means "1 hour and 30 minutes" (we never record seconds). And ideally the data in SQL Server would be formatted in some sort of time or datetime/datetime2 format.
I'm in a position to modify the formatting or code of the Access front end or the SQL Server back end (or both)--what's the cleanest way to go about this?
The best way is to user data type DateTime in SQL Server. Any ODBC driver will read and write that from Access as native DateTime of VBA.
If you must use DateTime2 in SQL Server, you must install and use one of the never ODBC drivers, not the "SQL Server" ODBC driver that comes with Windows as it cannot read the microsecond resolution of DateTime2.
You should never use the other date/time data types of SQL Server: Time and Short Date

Oracle Date format exception in SQL Server Reporting Services

Earlier my client was using SSRS 2008R2 with Oracle as transaction database. Recently they have upgraded to SSRS 2017 and now many reports are throwing following error:
ERROR: Throwing
Microsoft.ReportingServices.ReportProcessing.ProcessingAbortedException:
[AbnormalTermination:ReportProcessing],
Microsoft.ReportingServices.ReportProcessing.ProcessingAbortedException:
An error has occurred during report processing. --->
Microsoft.ReportingServices.ReportProcessing.ReportProcessingException:
Query execution failed for dataset 'Ds_Main'. --->
Oracle.ManagedDataAccess.Client.OracleException: ORA-01830: date
format picture ends before converting entire input string
After closely looking into report query, I have noticed that this error is for all those reports where oracle function TO_DATE(<Date Value>) has been used without date format. For example:
To_date(:Date_Parameter) -> this syntax throws above mentioned error
To_Date(:Date_Parameter,’MM/DD/YYYY’) -> this syntax works perfectly
I am willing to know:
what has changed in SSRS 2017 vs SSRS 2008 R2 that is causing this issue because same reports are working as expected in SSRS 2008 R2 and it is throwing above error in SSRS 2017.
Is there is any suggestions to fix this issue without updating bunch of reports?
what has changed in SSRS2017 vs SSRS2008R2
SSRS 2008 used the old System.Data.OracleClient. In SSRS 2016 and later you have to install the Oracle ODP.NET provider, built and supported by Oracle. So probably just a difference in how the NLS_DATE_FORMAT session parameter is set by the two drivers.
You can see your setting if you add a dataset to your report with this query:
select parameter, value
from nls_session_parameters
where parameter like 'NLS%'
order by parameter
Unfortunately there doesn't appear to be a way to globally change the client date format in Oracle.ManagedDataAccess, so you'll have to make all the changes in the report dataset queries.
Alternatively you can try to ensure that you are passing Date parameters and not string parameters. If you pass a date to Oracle's to_date() function, you don't need to specify a format.
The docs for SSRS 2014
"This built-in data source type is based on the .NET Framework Managed Provider for Oracle and requires an Oracle client software component."
And for SSRS 2016 "This built-in data source type uses the Oracle Data Provider directly and requires an Oracle client software component."
Trying to figure out the issue
I don't think the issue is related to the Visual Studio upgrade. It is related to the date format passed as a parameter to TO_DATE() function.
Based on the official documentation of the Oracle / PLSQL: ORA-01830 Error Message:
Cause: You tried to enter a date value, but the date entered did not match the date format.
In Oracle, the default date format is generally DD-MON-YYYY. If you try to enter a date value that does not comply with this format.
You seem to have passed the date parameters in the dd-MMM-yyyy format and now they are passed as MM/dd/yyyy.
First of all, check that the regional setting or the applications Culture information didn't change.
Possible workarounds
You can fix the issue using several approaches:
(1) Handling the parameters date format
If you do not want to edit all code, then it is easier to force the parameter data string format, make sure that all parameters passed to TO_DATE() function are in following format (or try to change the default date format from the OS regional settings):
dd-MMM-yyyy example: 01-AUG-2019
(2) Adding the date format to TO_DATE function
If you are sure that the date parameters format is fixed and will not change then you can edit your code as you mentioned in the question:
To_Date(:Date_Parameter,’MM/DD/YYYY’)
(3) Pass the date and format as parameters
This also requires changing the code, but you will pass the date and format as parameters.
To_Date(:Date_Parameter,:DateFormat_Parameter)
SQLFiddle demo for parsing dates
You may find other methods at the following link:
ORA-01830: date format picture ends before converting entire input string / Select sum where date query
Update 1 - Making common change in multiple reports
While searching I found the following link providing a method to loop over reports and make changes. You have to only replace To_Date(:Date_Parameter) with To_Date(:Date_Parameter,’MM/DD/YYYY’):
SSRS - Make common change in multiple reports in one click
Update 2 - Other possible workarounds
Forcing Culture info by editing ReportViewer.aspx
You can edit the ReportViewer.aspx file is located in the SQL Server reporting services directory, and force the culture info used within reports. Check out the following question it will give you more details:
I want Datetime Parameter in DDMMYYYY Format in ssrs report
Changing the browser language settings
Check the following link (read Mike Honey and Nick St Mags answers):
SSRS Datetime Parameter value should display in DD/MM/YYYY format
Update 3 - Issue cause
In addition of what #DavidBrownie posted i found the SQL Server 2008 R2 documentation:
Oracle Connection Type (SSRS 2008 R2)
Where they mentioned:
This built-in data source type is based on the .NET Framework Managed Provider for Oracle and requires an Oracle client software component.
Also if you take a look at SQL Server 2017 documentation:
Oracle Connection Type (SSRS 2017)
This built-in data source type uses the Oracle Data Provider directly and requires an Oracle client software component.
In addition, referring to Microsoft OLE DB Provider for Oracle documentation (which is the old used provider). They mentioned that:
This feature will be removed in a future version of Windows. Avoid using this feature in new development work, and plan to modify applications that currently use this feature. Instead, use Oracle's OLE DB provider.
Which is the reason for changing the provider used to connect to Oracle in Reporting Services.

creating a SSIS package with a SQL query to Oracle

I have about had it with SQL Server 2012 64 bit!
I am creating a SSIS package with a SQL query to Oracle and trying to put the file into a flat file. I am using the Oracle OLEDB source and a Flat File Destination for the output. Everything works fine locally, but when put on the server and run through SQL Agent I keep getting the Unicode to Non-unicode errors!
The latest drivers are on the server and the 11g client is on my development machine. The types shown in each step show as DT-STR.
I have the exact same source writing to an OLEDB destination just fine. I don't want to have to write these to a table and then pull them back out just to get this to work. Any solutions? And please, no "just add this" responses.
I have tried a data conversion, but get same result. Please supply DETAILED answers as in go here and change this to this. Pictures never hurt. Thanks
Short answer is that you need to convert codepages, not datatypes.
Long answer follows:
Step 0: If you're not already using it, I highly recommend that you switch to using the Attunity Connectors instead of stock Oracle OLEDB connector. You can download it for SQL2012 from Microsoft at: https://www.microsoft.com/en-us/download/details.aspx?id=29283
Step 1: Using the Attunity Oracle Source, you can specify a SQL Command as the data access mode, instead of just pointing at a table.
Step 2: You need to determine the exact code page the Oracle server is using, and the exact code page your SQL server is using. For Oracle servers using the UTF-8 character set, this is most likely AL32UTF8 and for a Windows Server using the default ANSI-1252 char set, the code page is WE8MSWIN1252.
Step 3: Write your plsql query and CONVERT the codepage of all the columns on the Oracle side. Make sure you use double quotes around the Oracle column names. It should look something like this:
SELECT
CONVERT("Data Source Code",'AL32UTF8','WE8MSWIN1252') AS DataSourceCode
,CONVERT("Order#",'AL32UTF8','WE8MSWIN1252') AS OrderNumber
,CONVERT("Invoice#",'AL32UTF8','WE8MSWIN1252') AS InvoiceNumber
,CONVERT("Item#",'AL32UTF8','WE8MSWIN1252') AS ItemNumber
,CONVERT("Order Line Type",'AL32UTF8','WE8MSWIN1252') AS OrderLineType
,CONVERT("Order Status",'AL32UTF8','WE8MSWIN1252') AS OrderStatus
,CONVERT("Order Date",'AL32UTF8','WE8MSWIN1252') AS OrderDate
,CONVERT("Invoice Date",'AL32UTF8','WE8MSWIN1252') AS InvoiceDate
,CONVERT("Ship To Cust#",'AL32UTF8','WE8MSWIN1252') AS ShipToCustNumber
,CONVERT("Billing Account #",'AL32UTF8','WE8MSWIN1252') AS BillingAccountNumber
,CONVERT("Sold Qty",'AL32UTF8','WE8MSWIN1252') AS SoldQty
,CONVERT("Unit Price",'AL32UTF8','WE8MSWIN1252') AS UnitPrice
,CONVERT("Sales Amount",'AL32UTF8','WE8MSWIN1252') AS SalesAmount
,CONVERT("Handling Amount",'AL32UTF8','WE8MSWIN1252') AS HandlingAmount
,CONVERT("Freight Amount",'AL32UTF8','WE8MSWIN1252') AS FreightAmount
FROM MYORACLEDB.DIGITAL_SALES_FEED
WHERE "Invoice Date" >= TO_DATE('2018/08/01', 'yyyy/mm/dd')
Step 4: Use this query text as the SQL command text in the Oracle Source configuration window.
Fun Fact: Oracle will return the column names in ALL CAPS, regardless of your AS ColumnName format.
Step 5 (optional): All columns will be returned as strings. You might want to put a Data Conversion task in your Data Flow, but if you're just dumping your data into a flatfile, you might not care about the data conversion. I have decimal/numeric and dates in my data set, so I do a conversion before inserting into SQL Server.

cannot convert between unicode and non-unicode string data types

I have a number of packages in SSIS that moves data from an oracle data source to Sql Server.
A new package was failing and I could not figure out what was going on. It was giving the message "cannot convert between unicode and non-unicode string data types"
It was weird that the package worked on my local machine but when deployed to server it always failed.
I already do a data conversion of my string values from dt_Wstr to dt_str.
All of my string columns on SQL Server are varchar
At wits end I decided to try to change the nps_lang registry value for oracle on the server box.
These are the settings that I changed.
(old)AMERICAN_AMERICA.WE8MSWIN1252
(new)AMERICAN_AMERICA.WE8ISO8859P1
It worked! At least so I thought.
The next day all the other packages that had been running fine are now failing.
Now my questions are what are the difference between these two that would cause one to fail and the others not?
Is there a character set that I could use that would work with both?
Any other suggestions to get all of my packages to run successfully on the server?

Resources