Is there a direct route that is pretty straight forward? (i.e. can SQL Server read XML)
Or, is it best to parse the XML and just transfer it in the usual way via ADO.Net either as individual rows or perhaps a batch update?
I realize there may be solutions that involve large complex stored procs--while I'm not entirely opposed to this, I tend to prefer to have most of my business logic in the C# code. I have seen a solution using SQLXMLBulkLoad, but it seemed to require fairly complex SQL code.
For reference, I'll be working with about 100 rows at a time with about 50 small pieces of data for each (strings and ints). This will eventually become a daily batch job.
Any code snippets you can provide would be very much appreciated.
SQL Server 2005 and up have a datatype called "XML" which you can store XML in - untyped or typed with a XSD schema.
You can basically fill columns of type XML from an XML literal string, so you can easily just use a normal INSERT statement and fill the XML contents into that field.
Marc
You can use the function OPENXML and stored procedure sp_xml_preparedocument to easily convert your XML into rowsets.
If you are using SQL Server 2008 (or 2005), it has an xml native datatype. You can associate an XSD schema with xml variables, and Insert directly into columns of type xml.
Yes, SQL Server 2005 and above can parse XML out of the box.
You use the nodes, value and query methods to break it down how you want, whether values or attributes
Some shameless plugging:
Importing XML into SQL Server
Search XML Column in SQL
Xml data and Xml document could have different meaning.
When xml type is good for data, it doesn't save formatting (white spaces removed), so in some cases (e.g. cofiguration files) the best option is nvarchar.
Related
I have a column that is a blob. Here is the screenshot:
As you can see that I have a TBDOCUMENTS table. In this table DOCUMENT column is BLOB. I want to read this column. Then I know that for this particular DOCUMENTURL this column contains xml. So I want to convert it into XML. And then I want to write this XML in a file.
How can I do it in SQL Server? I am using SQL Server 2014
To read the xml you stored inside a blob column (for example varbinary) you can use CONVERT:
select CONVERT(xml,(CONVERT(varbinary(max),DOCUMENT)))
from TBDOCUMENTS
where DOCUMENTURL='...'
Now you can write this string to an XML file. I think that SQL Server is probably not a good fit for this task; nonetheless there are many techniques to achieve this, for example with bcp (more info here and here)
As reported in the comments a DTD related error may occur when using CONVERT:
Parsing XML with internal subset DTDs not allowed. Use CONVERT with style option 2 to enable limited internal subset DTD support.
In this case using the xml-styles option of CONVERT command (more info here) fixes the problem:
select CONVERT(xml,(CONVERT(varbinary(max),DOCUMENT)), 2)
from TBDOCUMENTS
where DOCUMENTURL='...'
In my stored procedure, I am creating an XML file which has the potential to be very large, > 1GB in size. The data needs to be inserted into a varbinary column and I was wondering what the most efficient method of doing this is in SQL Server 2014?
I was storing it in an xml column but have been asked to move it to this new column as a result of a decision outside of my control
If you have the slightest chance to speak with these persons, you should do this!
You must be aware, that XML is not stored as the string representation you see, but as a hierarchically organized tree. Reading this data or manipulating it is astonishingly fast! If you store the XML as BLOB, you will keep it in its string format (hopefully this is unicode/UCS-2!). Reading this data will need a cast to NVARCHAR(MAX) and then to XML, which means a full parse of the whole document to get the hierarchy tree. When this is done, you can use XML data type methods like .value or .nodes). You will need this very expensive process over and over and over and ...
Especially in cases of huge XMLs (or - even worse - many of them) this is a really bad decision!! Why should one do this??? It will take roughly the same amount of storage space.
The only thing you will get is bad performance! And you will be the one who has to repair this later...
VARBINARY is the appropriate type for data, where you do not care what's inside (e.g. pictures). If these XMLs are just plain archive data and you do not want to read or manipulate them, this can be a choice. But there is no advantage at all!
I would look into using a File Table: https://learn.microsoft.com/en-us/sql/relational-databases/blob/filetables-sql-server
And Check out this for inserting blobs: How to insert a blob into a database using sql server management studio
Consider a column in an MS SQL database which will house either potentially large chunks or XML or pipe-delimited HL7v2 data.
Currently (due to not using forward-thinking) it's currently typed as XML because originally we were only ever accepting XML data. While technically this could work, it means that all the XML special characters in the HL7v2 messages are being encoded (& --> & etc.).
This is not ideal for what we are doing. If I were to convert this column to a different datatype, what would be recommended? I was thinking nvarchar(max) as it seems like it would handle it, but I'm not well-versed in SQL datatypes and the implications of using different types for such data.
There really isn't much of a choice other than nvarchar(max).
The other options are either varchar(max) or varbinary(max). You might need Unicode so you can't use varchar. It would work to store it as varbinary, but it would just be annoying to work with.
Use HAPI to transform the HL7 messages from ER7 (pipe delimited) to XML encoding. That way you can use a single SQL Server XML column for everything. And it will give you the added benefit of being able to query into HL7 message contents using XQuery.
As Nicks say, converting pipe delimited to XML and then persist in XML is the best option, trying to persist xml and pipe delimited values in a same column for me it make no sense, as on source they are different data types.
I have 20 record group that i need to batch insert them all in one connection so there is two solution (XML or stored procedure). this operation frequently executed so i need fast performance and least overhead
1) I think XML is performs slower but we can freely specify how many record we need to insert as a batch by producing the appropriate XML, I don't know the values of each field in a record, there maybe characters that malformed our XML like using " or filed tags in values so how should i prevent this behavior ?
2) using stored procedure is faster but i need to define all input parameters which is boring task and if i need to increase or decrease the number of records inserted in a batch then i need to change the SP
so which solution is better in my environment with respect to my constrains
XML is likely the better choice, however there are other options
If you're using SQL Server 2008 you can use Table Valued parameters instead.
Starting with .NET 2.0 you had the option to use the SQLBulkCopy
If you're using oracle you can pass a user defined type but I'm not sure what versions of ODP and Oracle that works with.
Note these are all .NET samples. I don't know that this will work for you. It would probably help if you include the database and version and client technology that you're using.
At the moment, if I save <element></element> to a SQL Server 2008 database in a field of type xml, it converts it to <element/>.
How can I preserve the xml empty text as is when saving?
In case this is a gotcha, I am utilising Linq to Sql as my ORM to communicate to the database in order to save it.
What you're asking for is not possible.
SQL Server stores data in xml columns as a binary representation, so any extraneous formatting is discarded, as you found out.
To preserve the formatting, you would have to store the content in a text field of type varchar(MAX) or nvarchar(MAX). Hopefully you don't have to run XML-based queries on the data.
http://msdn.microsoft.com/en-us/library/ms189887.aspx