Write specifically structured XML from t-sql query - sql-server

I would like to be able to write out an XML file in this specific format. I've been reading up on bcp and experimenting with FOR XML but can't seem to achieve what I need here. I would appreciate any help in the right direction. While I know I could refactor the code that would consume this XML output; I'd like to be adventurous here and stick to the task at hand without changing anything.
SQL code for recreating a temp var table dataset
declare #dataset table(Color nvarchar(10), Number int, Code nvarchar(10))
insert into #dataset
select 'Green', 12345, 'US1'
union
select 'Red', 56789, 'US2'
select * from #dataset
And from this query I would like to generate the following XML document.
<?xml version="1.0" encoding="utf-8" ?>
<test>
<collection>
<Case>
<input>
<attribute name="Color">Green</attribute>
<attribute name="Number">12345</attribute>
<attribute name="Code">US1</attribute>
</input>
</Case>
<Case>
<input>
<attribute name="Color">Red</attribute>
<attribute name="Number">56789</attribute>
<attribute name="Code">US2</attribute>
</input>
</Case>
</collection>
</test>
I will defer to you experts to tell me if this is too ridiculous to be accomplished, but I think it "is" possible as I've got a little close so far.
I've been tinkering with this and able to write out XML files.
exec master..xp_cmdshell 'bcp "Query Here" queryout "c:\filename.xml" -c -T'
Thanks SO members!

select (
select (
select 'Color' as [attribute/#name],
Color as [attribute],
null,
'Number' as [attribute/#name],
Number as [attribute],
null,
'Code' as [attribute/#name],
Code as [attribute]
for xml path('input'), type
)
from #dataset
for xml path('Case'), root('collection'), type
)
for xml path('test'), type

Related

t-sql to create XML file using FOR XML Path, multilevel issues

I am creating an XML file to Upload to a 3rd party product. The file must begin with specific file and source information level and then it is followed with the specific data requirements/levels of EVENT and CREW members for those events.
I can create the initial level with the file/source information, and I have the data requirements exactly as they should be, but I cannot get them together in the same file between the "ROOT" level without the initial level repeating between each EVENT level or the an extra EVENT level as if they're nested. I've also managed to get a result with a ROW level that I did not define and the "tags" modified to < and &gt: instead of < >. I've done a good bit of research and tried using a union method, sub-selects, nesting methods as well many combinations of FOR XML PATH, AUTO, EXPLICIT, with and without elements. I've learned a lot, but I'm just not finding the right combination for the results I need.
The first example is the layout that is required. The second is one of the examples that is most common for my efforts, followed by the SQL that created it.
what it should be (FILEINFO level only once, only one EVENT level for each EVENT)
<ROOT>
<FILEINFO>
<SOURCE_ID>P</SOURCE_ID>
</FILEINFO>
<EVENT>
<DATE>2019-09-24T08:00:00</DATE>
<NO>1</NO>
<DEL_FLAG>false</DEL_FLAG>
<DATE_TIME_STAMP>2019-09-24T14:14:00</DATE_TIME_STAMP>
<CREW>
<LAST_NAME>DOE</LAST_NAME>
<DEL_FLAG>false</DEL_FLAG>
<DATE_TIME_STAMP>2019-09-24T14:14:00</DATE_TIME_STAMP>
</CREW>
</EVENT>
<EVENT>
<DATE>2019-09-16T12:30:00</DATE>
<NO>1</NO>
<DEL_FLAG>false</DEL_FLAG>
<DATE_TIME_STAMP>2019-09-16T18:20:00</DATE_TIME_STAMP>
<CREW>
<LAST_NAME>DOE</LAST_NAME>
<DEL_FLAG>false</DEL_FLAG>
<DATE_TIME_STAMP>2019-09-16T18:20:00</DATE_TIME_STAMP>
</CREW>
</EVENT>
</ROOT>
what i'm getting:
<ROOT>
<EVENT>
<FILEINFO>
<SOURCE_ID>P</SOURCE_ID>
</FILEINFO>
<EVENT>
<DATE>2019-09-16T08:00:00</DATE>
<NO>1</NO>
<DEL_FLAG>false</DEL_FLAG>
<DATE_TIME_STAMP>2019-09-16T15:12:00</DATE_TIME_STAMP>
<CREW>
<LAST_NAME>DOE</LAST_NAME>
<DEL_FLAG>false</DEL_FLAG>
<DATE_TIME_STAMP>2019-09-16T15:12:00</DATE_TIME_STAMP>
</CREW>
</EVENT>
</EVENT>
<EVENT>
<FILEINFO>
<SOURCE_ID>P</SOURCE_ID>
</FILEINFO>
<EVENT>
<DATE>2019-09-16T08:00:00</DATE>
<NO>1</NO>
<DEL_FLAG>false</DEL_FLAG>
<DATE_TIME_STAMP>2019-09-16T15:12:00</DATE_TIME_STAMP>
<CREW>
<LAST_NAME>DOE</LAST_NAME>
<DEL_FLAG>false</DEL_FLAG>
<DATE_TIME_STAMP>2019-09-16T15:12:00</DATE_TIME_STAMP>
</CREW>
</EVENT>
</EVENT>
... ...
most recent/simplest attempt that creates the above:
SELECT
(SELECT SOURCE_ID FROM (select 'P' as SOURCE_ID) FILEINFO ) AS 'FILEINFO/SOURCE_ID'
,[DATE] AS 'EVENT/DATE'
,[NO] AS 'EVENT/NO'
,[DEL_FLAG] AS 'EVENT/DEL_FLAG'
,[DATE_TIME_STAMP] AS 'EVENT/DATE_TIME_STAMP'
,'DOE' as 'EVENT/CREW/LAST_NAME'
,[DEL_FLAG2] as 'EVENT/CREW/DEL_FLAG'
,[DATE_TIME_STAMP3] as 'EVENT/CREW/DATE_TIME_STAMP'
FROM [dbo].XMLForFILEExport x
FOR XML path('EVENT'), elements, ROOT('ROOT') ;
This is easy, just use a sub-select and deal with this like it was a *normal column:
This easy SELECT will return the single <FILEINFO>
SELECT 'P' AS [FILEINFO/SOURCE_ID]
FOR XML PATH(''),ROOT('ROOT');
You see, that I used an empty PATH(), but I set the ROOT().
This is the result
<ROOT>
<FILEINFO>
<SOURCE_ID>P</SOURCE_ID>
</FILEINFO>
</ROOT>
Now we can start to add your events. First I need a mockup table to simulate your issue
DECLARE #mockupEventTable TABLE(ID INT IDENTITY,[NO] INT, [DATE] DATETIME, EventText VARCHAR(100));
INSERT INTO #mockupEventTable VALUES(1,'20190916','Event 1')
,(2,'20190917','Event 2');
--The query
SELECT 'P' AS [FILEINFO/SOURCE_ID]
,(
SELECT e.[DATE]
,e.[NO]
,e.EventText
,'Doe' AS [CREW/LASTNAME]
FROM #mockupEventTable e
FOR XML PATH('EVENT'),TYPE
) AS [*]
FOR XML PATH(''),ROOT('ROOT');
The result
<ROOT>
<FILEINFO>
<SOURCE_ID>P</SOURCE_ID>
</FILEINFO>
<EVENT>
<DATE>2019-09-16T00:00:00</DATE>
<NO>1</NO>
<EventText>Event 1</EventText>
<CREW>
<LASTNAME>Doe</LASTNAME>
</CREW>
</EVENT>
<EVENT>
<DATE>2019-09-17T00:00:00</DATE>
<NO>2</NO>
<EventText>Event 2</EventText>
<CREW>
<LASTNAME>Doe</LASTNAME>
</CREW>
</EVENT>
</ROOT>
You can see, that the sub-select will create the inner XML just as you need it. We have to specify ,TYPE in order to get this as typed XML. Try the same without. You will get the XML escaped, as if it was simple text...
And I specify AS [*] (the same was AS [node()]) to indicate, that the XML "column" has no own name, but should be inserted as is. This is not mandatory (try it without), but it makes things more readable...
That's because you specified the PATH "EVENT" already. Also you can remove the EVENT in the field name, e.g. 'EVENT/CREW/DATE_TIME_STAMP' can just be 'CREW/DATE_TIME_STAMP'
TO achieve what required, you can generate the xml with EVENT elementsand then insert the FILEINFO.
DECLARE #x xml;
SET #x=(SELECT
[DATE] AS 'DATE'
,[NO] AS 'NO'
,[DEL_FLAG] AS 'DEL_FLAG'
,[DATE_TIME_STAMP] AS 'DATE_TIME_STAMP'
,'DOE' as 'CREW/LAST_NAME'
,[DEL_FLAG2] as 'CREW/DEL_FLAG'
,[DATE_TIME_STAMP3] as 'CREW/DATE_TIME_STAMP'
FROM [dbo].XMLForFILEExport x
FOR XML path('EVENT'), elements, ROOT('ROOT'))
SET #x.modify('
insert <FILEINFO><SOURCE_ID>P</SOURCE_ID></FILEINFO>
as first
into (/ROOT)[1]');

Combine and modify XML in TSQL

Using SQL Server 2005, is it possible to combine XML and add an attribute at same time?
Unfortunately, due to project restrictions, I need a SQL Server 2005 solution.
Consider the following, where I need to combine XML from multiple rows within a new <root> element...
; WITH [TestTable] AS (
SELECT 7 AS [PkId], CAST('<data><id>11</id><id>12</id></data>' AS XML) AS [Data]
UNION ALL
SELECT 12, CAST('<data><id>22</id></data>' AS XML)
UNION ALL
SELECT 43, CAST('<data><id>33</id></data>' AS XML)
)
SELECT (
SELECT XMLDATA as [*]
FROM (
SELECT [Data] AS [*]
FROM [TestTable]
FOR XML PATH(''), TYPE
) AS DATA(XMLDATA)
FOR XML PATH('root')
)
This produces the desired output of...
<root>
<data><id>11</id><id>12</id></data>
<data><id>22</id></data>
<data><id>33</id></data>
</root>
But what I need to do, if possible, is add an attribute to the existing data element in each of the rows with the PkId value. The desired output would then look like this...
<root>
<data pkid="7"><id>11</id><id>12</id></data>
<data pkid="12"><id>22</id></data>
<data pkid="43"><id>33</id></data>
</root>
My gut feeling is that this is going to be impossible without the use of a cursor, but if anybody knows a way of doing it I'd love to hear it.
At the request of #MattA, here is an example of some random data in the table...
[PkId] [UserId] [SubmittedDate] [Data]
1 1 2015-03-24 12:34:56 '<data><id>1</id><id>2</id></data>'
2 1 2015-03-23 09:15:52 '<data><id>3</id></data>'
3 2 2015-03-22 16:01:23 '<data><id>4</id><id>5</id></data>'
4 1 2015-03-21 13:45:34 '<data><id>6</id></data>'
Please note, that to make the question easier, I stated that I needed the PkId column as the attribute to the data. This is not actually the case - instead I need the [SubmittedDate] column to be used. I apologise if this caused confusion.
Using UserId=1 as a filter, the XML I would like from the above would be...
<root>
<data submitteddate="2015-03-24T12:34:56"><id>1</id><id>2</id></data>
<data submitteddate="2015-03-23T09:15:52"><id>3</id></data>
<data submitteddate="2015-03-21T13:45:34"><id>6</id></data>
</root>
The date would be formatted using the 126 date format available from CONVERT
Here's the quick answer for you. XML does support "modify", but shredding on a small data set like this works quite well too.
Code
--The existing XML
DECLARE #XML XML = '<root>
<data><id>11</id></data>
<data><id>22</id></data>
<data><id>33</id></data>
</root>'
--XML Shredded Back to a table
;WITH
ShreddedXML AS (
SELECT
ID = FieldAlias.value('(id)[1]','int')
FROM
#XML.nodes('/root/data') AS TableAlias(FieldAlias)
), ArbitraryPKGenerator AS (
SELECT CURRENT_TIMESTAMP AS PKid,
ID
FROM ShreddedXML
)
SELECT A.PKId AS "#PKid",
A.ID AS "id"
FROM ArbitraryPKGenerator AS A
FOR XML PATH('data'), ROOT('root')
And the XML
<root>
<data PKid="2015-03-24T09:44:55.770">
<id>11</id>
</data>
<data PKid="2015-03-24T09:44:55.770">
<id>22</id>
</data>
<data PKid="2015-03-24T09:44:55.770">
<id>33</id>
</data>
</root>

SQL Server 2012 create Xml with default values from xsd

I have imported an xsd, containing 258 elements, into my SQL Server 2012 instance. It is mandatory that all 258 elements are present in the final xml. The issue I am having is that 246 of them will contain default values that are identified in the xsd and I do not how to construct my SQL to populate the xml with the default values.
The following is an example I created that illustrates my issue using a much smaller xsd:
DROP XML SCHEMA COLLECTION TestSchema
GO
CREATE XML SCHEMA COLLECTION TestSchema AS
'<schema xmlns="http://www.w3.org/2001/XMLSchema">
<element name="document">
<complexType>
<sequence>
<element minOccurs="0" name="field1" type="string" default="1" />
<element name="field2" type="int" />
</sequence>
</complexType>
</element>
</schema>'
GO
declare #xml xml(TestSchema) = null
declare #reccount table(recordcount int not null)
insert into #reccount select 32
set #xml =
(
select
recordcount as field2
from
#reccount
for xml
PATH('document')
)
select #xml
The value of #xml is:
<document>
<field2>32</field2>
</document>
Whereas i was expecting
<document>
<field1>1</field1>
<field2>32</field2>
</document>
Any ideas how I can generate the default value of field1?
Thanks in advance.
Lucky for me I came across the answer fooling around with the select statement above. I just needed to add the field 'field1' with a '' for the data.
set #xml =
(
select
'' as field1,
recordcount as field2
from
#reccount
for xml
PATH('document')
)
That did it.

SQLXML - Search and Query node element?

I have an XML like this stored in an XML datatype column (will have multiple such rows in table)-
<Root xmlns="http://tempuri.org" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<Elem1 type="T1">
<Name type="string" display="First name">John</Name>
<TimeZone display="Time zone">
<DisplayName type="string" display="Display name">GMT Standard Time</DisplayName>
</TimeZone>
</Elem1>
</Root>
How can I filter based on a node element say (using SQL SERVER 2008 R2) - get all 'Elem1' nodes or get all 'Name' nodes or get all TimeZone nodes ? Something like using local-name() function ?
EDIT - Part Solution -
I got the solution partly (see John's reply below and then run this) -
SELECT C1.query('fn:local-name(.)') AS Nodes FROM [dbo].[MyXmlTable] AS MyXML CROSS APPLY MyXML.MyXmlCol.nodes('//*') AS T ( C1 )
The query above returns all the node elements across the TABLE. Now, I want to say filter upon specific elements and return the element and its value or its attribute value. How to achieve this (by using WHERE clause or any other filter mechanism)?
I'm not sure what result you are looking for but something like this perhaps.
declare #T table(XMLCol xml)
insert into #T values
('<Root xmlns="http://tempuri.org" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<Elem1 type="T1">
<Name type="string" display="First name">John</Name>
<TimeZone display="Time zone">
<DisplayName type="string" display="Display name">GMT Standard Time</DisplayName>
</TimeZone>
</Elem1>
</Root>')
declare #Node varchar(50)
set #Node = 'Elem1'
select N.query('.') as Value
from #T as T
cross apply T.XMLCol.nodes('//*[local-name()=sql:variable("#Node")]') as X(N)
Result:
<p1:Elem1 xmlns:p1="http://tempuri.org" type="T1">
<p1:Name type="string" display="First name">John</p1:Name>
<p1:TimeZone display="Time zone">
<p1:DisplayName type="string" display="Display name">GMT Standard Time</p1:DisplayName>
</p1:TimeZone>
</p1:Elem1>
Edit
If you want the actual value instead of the entire XML you can do like this instead.
declare #Node varchar(50)
set #Node = 'TimeZone'
select N.value('.', 'varchar(100)') as Value
from #T as T
cross apply T.XMLCol.nodes('//*[local-name()=sql:variable("#Node")]') as X(N)
Result:
Value
------------------
GMT Standard Time
You can transform XML into table like here:
declare #XML xml='<Root xmlns="http://tempuri.org" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<Elem1 type="T1">
<Name type="string" display="First name">John</Name>
<TimeZone display="Time zone">
<DisplayName type="string" display="Display name">GMT Standard Time</DisplayName>
</TimeZone>
</Elem1>
</Root> '
;WITH XMLNAMESPACES(DEFAULT 'http://tempuri.org'),
numbers as(
SELECT ROW_NUMBER() OVER(ORDER BY o1.object_id,o2.object_id) Num
FROM sys.objects o1 CROSS JOIN sys.objects o2),
c as(
SELECT
b.value('local-name(.)','nvarchar(1000)') Node_Name,
b.value('./text()[1]','nvarchar(1000)') Node_Value,
b.value('count(#*)','nvarchar(MAX)') AttributeCount,
Num Attribute_Number
FROM
#xml.nodes('Root//*') a(b)
CROSS APPLY Numbers
WHERE Num<=b.value('count(#*)','nvarchar(MAX)')
)
SELECT c.Node_Name,c.node_Value,Attribute_Number,
#XML.query('for $Attr in //*/.[local-name(.)=sql:column("Node_Name")]/#*[sql:column("Attribute_Number")] return local-name($Attr)').value('.','nvarchar(MAX)') Attribute_Name,
#XML.value('data(//*/.[local-name(.)=sql:column("Node_Name")]/#*[sql:column("Attribute_Number")])[1]','nvarchar(1000)') Attribute_Value
FROM c
Result:
Node_Name node_Value Attribute_Number Attribute_Name Attribute_Value
Elem1 NULL 1 type T1
Name John 1 type string
Name John 2 display First name
TimeZone NULL 1 display Time zone
DisplayName GMT Standard Time 1 type string
DisplayName GMT Standard Time 2 display Display name
Later you can query this result to get node/attribute value which do you need.
But it works only in your example, when you have only one node and all names are unique. In multinode XML you should use hierarchical numbering like '1-1-2' or something like this. It is much more complicated and i do not suggest to going this way.
It's not clear to me exactly what your output should look like. However, this should get you started:
create table MyXmlTable (MyXmlCol xml)
insert into MyXmlTable (MyXmlCol) values
(
'
<Root xmlns="http://tempuri.org" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<Elem1 type="T1">
<Name type="string" display="First name">John</Name>
<TimeZone display="Time zone">
<DisplayName type="string" display="Display name">GMT Standard Time</DisplayName>
</TimeZone>
</Elem1>
<Elem1 type="T2">
<Name type="string" display="First name">Fred</Name>
<TimeZone display="Time zone">
<DisplayName type="string" display="Display name">EST Standard Time</DisplayName>
</TimeZone>
</Elem1>
</Root>
');
;WITH XMLNAMESPACES(DEFAULT 'http://tempuri.org')
select MyXmlCol.query('/Root/Elem1/Name')
from MyXmlTable
This queries the XML for the "Name" elements -- you can modify the query depending on what kind of output you want exactly. It's a bit long, but the MSDN article on SQLXML is pretty informative:
http://msdn.microsoft.com/en-us/library/ms345117(v=sql.90).aspx
Hope this helps!
John
Update: you can add a where clause something like this. I'm still not clear on what you want the output to look like, but this will filter out the "Elem1" values:
SELECT C1.query('fn:local-name(.)') AS Nodes
FROM [dbo].[MyXmlTable] AS MyXML
CROSS APPLY MyXML.MyXmlCol.nodes('//*') AS T ( C1 )
WHERE CAST(C1.query('fn:local-name(.)') AS NVARCHAR(32)) <> 'Elem1'
One more update; hopefully this is the answer you are looking for!
Try using a wildcard in the query. I had to use dynamic SQL because the XML query() function will only take string literals for paths (you can use sql:variable("#filter") for values, but I wasn't able to get that working for a path.)
DECLARE #filter nvarchar(20)
SET #filter = '*/Elem1'
DECLARE #sqlCommand nvarchar(1000)
SET #sqlCommand =
';WITH XMLNAMESPACES(DEFAULT ''http://tempuri.org'')
select MyXmlCol.query(''' + #filter + ''')
from MyXmlTable'
print #sqlCommand
EXECUTE sp_executesql #sqlCommand, N'#filter nvarchar(20)', #filter = #filter
This will return the Elem1 XML (and all sub-nodes):
<p1:Elem1 xmlns:p1="http://tempuri.org" type="T1">
<p1:Name type="string" display="First name">John</p1:Name>
<p1:TimeZone display="Time zone">
<p1:DisplayName type="string" display="Display name">GMT Standard Time</p1:DisplayName>
</p1:TimeZone>
</p1:Elem1>
<p2:Elem1 xmlns:p2="http://tempuri.org" type="T2">
<p2:Name type="string" display="First name">Fred</p2:Name>
<p2:TimeZone display="Time zone">
<p2:DisplayName type="string" display="Display name">EST Standard Time</p2:DisplayName>
</p2:TimeZone>
</p2:Elem1>
And if you want to pick out "TimeZone" you would do this:
SET #filter = '*/*/TimeZone'

Microsoft SQL Server xml data

This site has a technique to pass xml data around in Microsoft SQL Server:
DECLARE #productIds xml
SET #productIds ='<Products><id>3</id><id>6</id><id>15</id></Products>'
SELECT
ParamValues.ID.value('.','VARCHAR(20)')
FROM #productIds.nodes('/Products/id') as ParamValues(ID)
But what is the syntax if I add another field?
The following does NOT work:
DECLARE #productIds xml
SET #productIds ='<Products><id>3</id><descr>Three</descr><id>6</id><descr>six</descr><id>15</id><descr>Fifteen</descr></Products>'
SELECT
ParamValues.ID.value('.','VARCHAR(20)')
,ParamValues.descr.value('.','VARCHAR(20)')
FROM #productIds.nodes('/Products/id') as ParamValues(ID)
Note: Maybe I've constructed my xml wrong.
You need to use something like:
SELECT
ParamValues.ID.value('(id)[1]','VARCHAR(20)'),
ParamValues.ID.value('(descr)[1]','VARCHAR(20)')
FROM
#productIds.nodes('/Products') as ParamValues(ID)
That FROM statement there defines something like a "virtual table" called ParamValues.ID - you need to select the <Products> node into that virtual table and then access the properties inside it.
Furthermore, your XML structure is very badly chosen:
<Products>
<id>3</id>
<descr>Three</descr>
<id>6</id>
<descr>six</descr>
<id>15</id>
<descr>Fifteen</descr>
</Products>
You won't be able to select the individual pairs of id/descr - you should use something more like:
<Products>
<Product>
<id>3</id>
<descr>Three</descr>
</Product>
<Product>
<id>6</id>
<descr>six</descr>
</Product>
<Product>
<id>15</id>
<descr>Fifteen</descr>
</Product>
</Products>
Then you could retrieve all items using this SQL XML query:
SELECT
ParamValues.ID.value('(id)[1]','VARCHAR(20)') AS 'ID',
ParamValues.ID.value('(descr)[1]','VARCHAR(20)') AS 'Description'
FROM
#productIds.nodes('/Products/Product') as ParamValues(ID)
ID Descrition
3 Three
6 six
15 Fifteen
You must wrap each set of id and descr into one parent node. Say Row. Now you can access each pair like this.
DECLARE #productIds xml
SET #productIds ='<Products><Row><id>3</id><descr>Three</descr></Row><Row><id>6</id><descr>six</descr></Row><Row><id>15</id><descr>Fifteen</descr></Row></Products>'
SELECT
ParamValues.Row.query('id').value('.','VARCHAR(20)'),
ParamValues.Row.query('descr').value('.','VARCHAR(20)')
FROM #productIds.nodes('/Products/Row') as ParamValues(Row)

Resources