In SSIS, I have a package that deals with dumping data from one table to another. However after the package finishes executing I notice that my column has 
 in place of carriage returns.
Below is part of the query that handles this column.
(select cast((text) as varchar(max)) from [table]
where columna = x.columna for xml path (''), type)
Using the type keyword fixed this issue when I was testing this query on the SSMS.
I also encountered another error before then, where I got the message
Column "MyColumn" cannot convert between unicode and non-unicode string data types.
So I had to modify the affected column to output to Unicode Text Stream (DT_NTEXT) in order to avoid any errors (using Unicode String will cause truncation).
in SSIS package, i will assume that you are using OLEDB Source to read data from Sql server.
you can simply use the same sql query as datasource instead of using a Table name
Adding .value('.',nvarchar(max)') to the end of this statement removes all the 
 that appears in the result.
Final query should like the following:
(select cast((text) as varchar(max)) from [table1]
where columna = [table2].columna for xml path (''), type).value('.',nvarchar(max)')
This was to fix another issue I was having in SSIS when dealing with special characters and unicode.
The site containing the solution can be found here.
Related
I know it's a common issue, but I still didn't find a solution. I need to export data from SQL Server 2012 to an Excel destination using SSIS.
The value of one of the columns is going to be more that 255 characters long, so the cell in Excel has to accept more than 255 characters as well.
That's why I got an error.
I changed the registry key to 0, but it still doesn't work.
I tried to create a dummy column in Excel, but the data looks weird.
Are there any other solutions that I am not aware of?
This is what I usually do as a workaround in such cases.
I try to use SQL query as source, so this may work only in that case.
So assuming you are picking up data from a view or table (say tbl)
so instead of
select ID,Name,DetailDescription from tbl
do (assuming DetailDescription is the column which can contain huge data)
select
ID =NULL,
Name= N'',
DetailDescription =REPLACE(REPLACE(REPLACE(REPLACE(REPLACE(N'__________','_','_____'),'_','_____'),'_','_____'),'_','_____'),'_','_____')
union
select ID,Name,DetailDescription from tbl
I'm trying to use SSIS to extract XML representation of a query result set to a text file. My query is currently successfully extracting the exact XML output I need when I run it in SSMS. I've tried every trick I can find to use this result set in a SSIS package to create a file.
Using a dataflow to port a OLE Source to a Flat file doesn't work because the output of a XML query is treated as TEXT and SSIS can't push TEXT, NTEXT or IMAGE to a file destination.
I've tried to then Execute SQL Task to fill a user variable and then use a Script Task (written using C#) to write the contents of this user variable to a file output, but the user variable is always empty. I don't know, but I suspect this is, again, because the XML is treated as TEXT or IMAGE and the user variable doesn't handle this.
The query is in this form:
SELECT *
FROM dataTable
WHERE dataTable.FIELD = 'Value'
FOR XML AUTO, ROOT('RootVal')
The resulting dataset is well formed XML, but I can't figure out how to get it from result set to file.
It's a relatively easy task for me to write a console app to do this in C# 4.0, but restrictions require me to at least prove it CAN'T be done with SSIS before I write the console app and a scheduler.
Sorry to spoil, but there's an SSIS option for you: Export Column Transformation.
I defined an OLE DB query with
SELECT
*
FROM
(
SELECT * FROM dbo.spt_values FOR XML AUTO, ROOT('RootVal')
) D (xml_node)
CROSS APPLY
(
SELECT 'C:\ssisdata\so_xmlExtract.xml'
) F (fileName)
This results in 1 row and 2 columns in the dataflow. I then attached the Export Column Transformation and wired it up with xml_node as Extract Column and fileName as the File Path Column
Mostly truncated results follow
<RootVal>
<dbo.spt_values name="rpc" number="1" type="A " status="0"/>
<dbo.spt_values name="dist" number="8" type="A " status="0"/>
<dbo.spt_values name="deferred" number="8192" type="V " low="0" high="1" status="0"/>
</RootVal>
A more detailed answer, with pictures, is available on this Q&A Export Varbinary(max) column with ssis
BillInKC's answer is the best I've ever seen, but SQL can be simplified (no need for cross apply):
SELECT X.*, 'output.xml' AS filename
FROM (SELECT * FROM #t FOR XML PATH('item'), ROOT('itemList')) AS X (xml_node)
It will output the same structure:
xml_node filename
-------------------------------------------------- ----------
<itemList><item><num>1000</num></item></itemlist> output.xml
(1 row(s) affected)
I have an xml doc (size: 3.59 mb) with 3765815 total characters in it. My sql server 2008 database table has a column with xml data type. When I try to insert this xml into the column it seems to truncate it.
I thought xml data type can handle 2GB of data. Is this a correct understanding or am i missing something?
Thanks
Here is the query i am using
declare printxml nvarchar(max)
select printxml=cast(inputxml as varchar(max))
from TableA
where SomeKey='<some key>'
print printxml
Select the data directly instead of printing it to the messages window:
SELECT
inputxml
FROM TableA
WHERE SomeKey = '<somekey>'
The caveat is that you have to set up Management Studio to be able to return all the data to the window. You do that using the following option (the default setting is 2MB):
In Sql Server I am using an XML type column to store a message. I do not want to store duplicate messages.
I only will have a few messages per user. I am currently querying the table for these messages, converting the XML to string in my C# code. I then compare the strings with what I am about to insert.
Unfortunately, Sql Server pretty-prints the data in the XML typed fields. What you store into the database is not necessarily exactly the same string as what you get back out later. It is functionally equivalent, but may have white space removed, etc.
Is there an efficient way to compare an XML string that I am considering inserting with those that are already in the database? As an aside, if I detect a duplicate I need to delete the older message then insert the replacement.
0 - Add a hash column to your table
1 - when you receive a new message, convert the whole XML to uppercase, remove all blanks and returns/linefeed, then compute the hash value of the normalized string.
2 - check if you already have a row with the resulting hash code in it.
If yes, this is duplicated, treat it
accordingly
If not, store the original XML along with the hash in a new row
I'm not 100% sure on your exact implementation but here is something I played around with. The idea being a stored procedure would do the inserting. Inserting into the messages table does a basic check on existing messages (SQL 2008 syntax):
declare #messages table (msg xml)
insert into #messages values
('<message>You like oranges</message>')
,('<message>You like apples</message>')
declare #newMessage xml = '<message>You like apples</message>'
insert into #messages (msg)
select #newMessage
where #newMessage.value('(message)[1]', 'nvarchar(50)') not in (
select msg.value('(message)[1]', 'nvarchar(50)')
from #messages
)
One solution is to stop using the XML typed field. Store the XML string into a varchar typed field.
I don't really like this solution, but I don't really like p.marino's solution either. It doesn't seem right to store a hash of something that is already in the row in the table.
What if you use OPENXML on each row in the table and query the actual XML information for key nodes and/or key attributes? But then you need to do it row by row, I don't think OPENXML works with a whole set of table rows.
In a project using a MSSQL 2005 Database we are required to log all data manipulating actions in a logging table. One field in that table is supposed to contain the row before it was changed. We have a lot of tables so I was trying to write a stored procedure that would gather up all the fields in one row of a table that was given to it, concatenate them somehow and then write a new log entry with that information.
I already tried using FOR XML PATH and it worked, but the client doesn't like the XML notation, they want a csv field.
Here's what I had with FOR XML PATH:
DECLARE #foo varchar(max);
SET #foo = (SELECT * FROM table WHERE id = 5775 FOR XML PATH(''));
The values for "table", "id" and the actual id (here: 5775) would later be passed in via the call to the stored procedure.
Is there any way to do this without getting XML notation and without knowing in advance which fields are going to be returned by the SELECT statement?
We used XML path and as you've discovered, it works very well. Since one of SQL's features is to store XML properly, the CSV makes no sense.
What you could try is a stored proc that reads out the XML in CSV format (fake it). I would. Since you won't likely be reading the data that much compared to saving it, the overhead is negligible.
How about:
Set #Foo = Stuff(
( Select ',' + MyCol1 + ',' + MyCol2 ...
From Table
Where Id = 5775
For Xml Path('')
), 1, 1, '')
This will produce a CSV line (presuming the inner SQL returns a single row). Now, this solves the second part of your question. As to the first part of "without knowing in advance which fields", there is no means to do this without using dynamic SQL. I.e., you have to build the SQL statement as a string on the fly. If you are going to do that you might as well build the entire CSV result on the fly.