I have a SQL Server SSIS package that inserts data in a table according to data in another table. The thing is this job breaks the accents of the varchar data in the process. I suppose it has something to do with encoding.
Simplified, my package does the following :
Select data with an OLE DB Source through SQL command data access mode
SELECT id, name, lastname
FROM Client
<WHERE id = 1>
Insert the selected data with an OLE DB Destination through mapping to ClientCopy table.
I noticed that previewing the data in the first step was returning me the broken accents already. Because of this, the data inserted in ClientCopy obviously has those broken accents.
id name lastname
1 Andr‚ BriŠre
This same query when executed in SQL Server returns the data correctly so I'm a bit lost right now.
id name lastname
1 André Brière
Thanks for helping me out!
Related
I am filling a table using a package and after preparing the data to be saved I present it to an OLEDB destination for SQL server and setting the data acces mode to Table or View - Fastload or just Table or view (no fast load).
There is no error message but it does not write to the table.
I switch over to the normal Table or View so that each record is inserted with a separate INSERT command instead of a BULK insert.
When nothing happens I stop the execution of the package and do a select * from the destination table. I saw that he inserted 20 records. After investigation the data which is send to the OLE DB destination, I saw that record 21 results in a duplicate key.
Instead of getting an error message the package does not continue its execution flow.
What am I doing wrong.
Go through all the elements of the package, including the package itself, and set:
FailPackageOnFailure = True
I have a package in my ETL that loops through 4 different databases, each with a copy of the same table. I found out that one of the four tables has one extra column named MTFvalue_Permit thus giving me the below error. My fact and staging tables, however, do include this column.
I was hoping ssis would just insert a null value if the column didn't exist. How can I add a null value into my Data Warehouse tables for the other three tables in which this column doesn't exist?
Error:
[Source DB IBS [1]] Error: SSIS Error Code DTS_E_OLEDBERROR. An OLE DB error has occurred. Error code: 0x80040E14.
An OLE DB record is available. Source: "Microsoft SQL Server Native Client 10.0" Hresult: 0x80040E14 Description: "Statement(s) could not be prepared.".
An OLE DB record is available. Source: "Microsoft SQL Server Native Client 10.0" Hresult: 0x80040E14 Description: "Invalid column name 'MTFvalue_Permit'.".
Edit: I'm thinking about using a case statement like this for the query but I'm getting a different error.
SELECT
[blahblahblah] ,
[blahblahblah] ,
[blahblahblah] ,
[blahblahblah] ,
[blahblahblah] ,
CASE WHEN COL_LENGTH('wmManifests', 'MTFvalue_Permit') IS NOT NULL
THEN [MTFvalue_Permit]
ELSE NULL
END AS 'MTFvalue_Permit'
FROM dbo.wmManifests
Invalid column name 'MTFvalue_Permit'.
I don't think that you can do it using expressions.
You can do this workaround.
Add a Script task that list columns from the table
Build the Select query dynamically:
If the column is not found pass NULL instead of the column name and give an alias (same as column name)
save this query string into a SSIS variable and use it as a source
You can also create a stored procedure that generate the query string. Execute it from a sql task and store the query string into ssis variable
Perhaps I am not understanding your question, but it seams like you are approaching this problem backwards. You cannot insert data into columns that do not exist. Is there a reason this column exists in only one out of four versions of this table? And does this column have a default value that you can leverage?
I would continue inserting to the set of columns that are common among all tables and allow the tables with columns that do not exist in every one of the tables in the set to use the default value for that column.
We have an Oracle table that contains archived data in the table format:
BLOB | BLOBID
Each BLOB is an XML file that contains all the business objects we need.
Every BLOB needs to be read, the XML parsed and put into 1 SQL Server table that will hold all data.
The table has 5 columns R | S | T | D | BLOBID
Sample XML derived from BLOB:
<N>
<E>
<R>33</R>
<S>1</S>
<T>2012-01-25T09:48:43.213</T>
<D>6.9534619e+003</D>
</E>
<E>
<R>33</R>
<S>1</S>
<T>2012-01-25T09:48:45.227</T>
<D>1.1085871e+004</D>
</E>
<E>
<R>33</R>
<S>1</S>
<T>2012-01-25T09:48:47.227</T>
<D>1.1561764e+004</D>
</E>
</N>
There are a few million BLOBs and we want to avoid copying all the data over as an XML column then to a table, instead we want to go BLOB to table in one step.
What is the best approach to doing this with SSIS/SQL Server?
The code below almost does what we are looking for but only in Oracle Developer and only for one BLOB:
ALTER SESSION SET NLS_TIMESTAMP_FORMAT='yyyy-mm-dd HH24:MI:SS.FF';
SELECT b.BLOBID, a.R as R, a.S as S, a.T as T, cast(a.D AS float(23)) as D
FROM XMLTABLE('/N/E' PASSING
(SELECT xmltype.createxml(BLOB, NLS_CHARSET_ID('UTF16'), null)
FROM CLOUD --Oracle BLOB Cloud
WHERE BLOBID = 23321835)
COLUMNS
R int PATH 'R',
S int PATH 'S',
T TIMESTAMP(3) PATH 'T',
D VARCHAR(23) PATH 'D'
) a
Removing WHERE BLOBID = 23321835 gives the following error ORA-01427: single-row subquery returns more than one row since there are millions of BLOBS. Even so is there a way to run this through SSIS? Adding the SQL to the OLE DB Source did not work for pulling the data from Oracle even for 1 BLOB and would result in errors.
Using SQL Server 2012 and Oracle 10g
To summarize, how would we go from a Oracle BLOB containing XML to SQL Server table with business objects derived from XML with SSIS?
I'm new to working with Oracle, any help would be greatly appreciated!
Update:
I was able to get some of my code to work in SSIS by modifying the Oracle Source in SSIS to use the SQL command code above minus the first line,
ALTER SESSION SET NLS_TIMESTAMP_FORMAT='yyyy-mm-dd HH24:MI:SS.FF';
SSIS doesn't like this line.
Error message with the ALTER SESSION line above included:
No column information was returned by the SQL Command
Would there be another way to format the date without losing data? I'll try experimenting more, possible using varchar(23) for the date instead of timestamp.
We are migrating from SQL Server to Teradata database. All the views and tables are migrated. But the issue is that we aren't able to take the comments from each table.
In SQL Server we have a function called extended property which if used will get the comments from the respective tables/views. Badhri helped with providing a query for fetching the comment in Teradata in the same way but it doesn’t work in the expected way.
I tried inserting sample comments in columns and the below query is not fetching exact results. Can you please help?
Sample Query:
COMMENT ON COLUMN UtilityApp_DB.SQL_Views_Columns.ColumnNAME 'A Columnname for SQL Server!';
select 'COMMENT ON COLUMN '||trim(b.databasename)||'.'||trim(b.tablename) ||'.'||trim(b.columnname)||' IS '||''''||trim(b.commentstring)||''';'
FROM DBC.Columns b
WHERE b.CommentString IS NOT NULL
AND DatabaseName='UtilityApp_DB'
AND TableName in ('UtilityApp_DB.SQL_Views_Columns');
I created a table and tried inserting the values for the comment string in the table by copying the DBC Columns structure but still I can't get the comment string.
insert into UtilityApp_DB.commentstable (commentstring) values ('UtilityApp_DB.SQL_Views_Columns');
TableName in DBC.Columns does not include the database name and IS in your string concatenation should be AS.
SELECT 'COMMENT ON COLUMN '||trim(b.databasename)||'.'||trim(b.tablename) ||'.'||trim(b.columnname)||' AS '||''''||trim(b.commentstring)||''';'
FROM DBC.Columns
WHERE DatabaseName = 'UtilityApp_DB'
AND TableName IN ('SQL_Views_Columns');
I have created a table called DimInternationalFunction.
IF EXISTS (SELECT * FROM sys.objects WHERE object_id = OBJECT_ID(N'[DimInternationalFunction]') AND type in (N'U'))
DROP TABLE [DimInternationalFunction]
Go
Create Table DimInternationalFunction
(IntFunctionKey int NOT NULL identity primary key,
SubSubFunctionString char(10),
FunctionCode char(3),
SubFunctionCode char(6),
SubSubFunctionCode char(10),
SubSubFunctionName nvarchar(60),
SubFunctionName nvarchar(60),
FunctionName nvarchar(60))
I have initially inserted records in this table in SSMS.
After inserting the initial records manually in SSMS, now my manager wants me to insert "new records only" using SSIS.
I have tried using this in SSMS and it worked. Either it gives me 0 records inserted or sometimes it gives me 5 records inserted as a result. My manager wants me to do this in SSIS.
I tried using this script inside the OLE DB Source under Data Access Mode: SQL Command and SQL Command text:
insert into DWResourceTask.dbo.DimInternationalFunction
select f.SubSubFunctionString,
f.FunctionCode,
f.SubFunctionCode,
f.SubSubFunctionCode,
f.SubSubFunctionName,
f.SubFunctionName,
f.FunctionName
from ODS_Function F
where FunctionViewCode = 'INT'
and not exists (select * from DWResourceTask.dbo.DimInternationalFunction I
where (f.SubSubFunctionString=i.SubSubFunctionString
and f.FunctionCode=i.FunctionCode
and f.SubFunctionCode=i.SubFunctionCode
and f.SubSubFunctionCode=i.SubSubFunctionCode
and f.SubSubFunctionName=i.SubSubFunctionName
and f.SubFunctionName=i.SubFunctionName
and f.FunctionName=i.FunctionName)
)
The error message that I got after clicking preview is
The component reported the following warnings:
Error at Int Function [International Function Table [33]]: No column information was returned by the SQL command.
Choose OK if you want to continue with the operation.
Choose Cancel if you want to stop the operation.
Is there another component in SSIS that can do this? or can I just use either exec sql task component or ole db source?
I am thinking of using exec sql task connected to a data flow task, inside the data flow task I will put ole db source containing a staging table and do a delete on that or is there any other way to do it. Please help. Thanks in advance.
You could do it with an Execute SQL task.
If you want to do it "the pure SSIS way", you could use a lookup component. Set the "rows with no matching" handler to "Redirect to no match output", and configure the target table as connection. Then use the "No Match Output" only, ignoring the "Match Output". And send the records from the "No Match Output" to the target.
In spite of its name, the "Lookup" component can be used to filter data in many cases.
But I would assume the Execute SQL task would be more efficient for large data sets, keeping all data within the database engine.