How to import XML to SQL Server when base nodes repeat - sql-server

Question

I am importing generated XML files into a SQL Server database. It's been working fine, but now I need additional information from a node that repeats, and I can't figure it out. I am using VBScript with SQLXMLBulkLoad.SQLXMLBulkload.4.0 to import the files. There are many <file> nodes in each file, and here a simplified as an example:
<?xml version="1.0" encoding="UTF-8"?>
<samp_catalog version='1.0'>
<file>
<name>123.wav</name>
<xyz>
<a_samp>
<a_one>1</a_one>
<a_two>2</a_two>
<a_three>3</a_three>
</a_samp>
<b_samp version='2.0'>
<b_four>sample</b_four>
<b_five>sample</b_five>
<b_six>sample</b_six>
</b_samp>
<c_samp version='1.0'>
<c_entry>
<total_size>0</total_size>
<sig>sample</sig>
<type>SAMPLE</type>
<data_size>256</data_size>
<data encoding='string'></data>
</c_entry>
</c_samp>
<c_samp version='1.0'>
<c_entry>
<total_size>0</total_size>
<sig>sample</sig>
<type>PHONE</type>
<data_size>256</data_size>
<data encoding='string'>15555551212</data>
</c_entry>
</c_samp>
<c_samp version='1.0'>
<c_entry>
<total_size>0</total_size>
<sig>sample</sig>
<type>OTHER_SAMPLE</type>
<data_size>256</data_size>
<data encoding='string'></data>
</c_entry>
</c_samp>
</xyz>
</file>
</samp_catalog>
Without the repeating c_samp node, the schema was easy:
<xsd:schema xmlns:xsd = "http://www.w3.org/2001/XMLSchema" xmlns:sql = "urn:schemas-microsoft-com:mapping-schema">
<xsd:element name = "samp_catalog" sql:is-constant = "1">
<xsd:complexType>
<xsd:sequence>
<xsd:element name = "file" sql:relation = "files" maxOccurs = "unbounded">
<xsd:complexType>
<xsd:sequence>
<xsd:element name = "name" type = "xsd:string" sql:field = "name" />
<xsd:element name = "xyz" sql:is-constant = "1">
<xsd:complexType>
<xsd:sequence>
<xsd:element name = "a_samp" sql:is-constant = "1">
<xsd:complexType>
<xsd:sequence>
<xsd:element name = "a_one" type = "xsd:integer" sql:field = "a_one" />
<xsd:element name = "a_two" type = "xsd:integer" sql:field = "a_two" />
<xsd:element name = "a_three" type = "xsd:integer" sql:field = "a_three" />
</xsd:sequence>
</xsd:complexType>
</xsd:element>
<xsd:element name = "b_samp" sql:is-constant = "1">
<xsd:complexType>
<xsd:sequence>
<xsd:element name = "b_four" type = "xsd:string" sql:field = "b_four" />
<xsd:element name = "b_five" type = "xsd:string" sql:field = "b_five" />
<xsd:element name = "b_six" type = "xsd:string" sql:field = "b_six" />
</xsd:sequence>
</xsd:complexType>
</xsd:element>
</xsd:sequence>
</xsd:complexType>
</xsd:element>
</xsd:sequence>
</xsd:complexType>
</xsd:element>
</xsd:sequence>
</xsd:complexType>
</xsd:element>
</xsd:schema>
The important one is the second data element (of type PHONE), but importing them all is acceptable as well. I have tried defining all 3 c_samps in the schema and defining a sql:field for only the specified data element and defining it for all elements, however the import always results in NULL for everything under a c_samp element.
How can I get the repeating nodes to import into the database?

Answer 1

I would write a stored procedure in SQL-Server and pass over the XML as-is. Use SQL Server's abilities to read the XML without a schema.
The following code will extract all information from the XML given.
Just do something like
CREATE dbo.ImportMyXML(#xml XML)
AS
BEGIN
--Code here
END
GO
Within the BEGIN END you write this query (out-commented the INSERT INTO to test the query's result:
--INSERT INTO YourTargetTable(col1,col2,col3,...)
SELECT #xml.value(N'(/samp_catalog/file/name/text())[1]',N'nvarchar(max)') AS name
,a_samp.value(N'(*/text())[1]',N'nvarchar(max)') AS a_one
,a_samp.value(N'(*/text())[2]',N'nvarchar(max)') AS a_two
,a_samp.value(N'(*/text())[3]',N'nvarchar(max)') AS a_three
,b_samp.value(N'#version',N'nvarchar(max)') AS b_version
,b_samp.value(N'(*/text())[1]',N'nvarchar(max)') AS b_one
,b_samp.value(N'(*/text())[2]',N'nvarchar(max)') AS b_two
,b_samp.value(N'(*/text())[3]',N'nvarchar(max)') AS b_three
,NodesWith_c_entry.value(N'(../#version)[1]',N'nvarchar(max)') AS c_version
,NodesWith_c_entry.value(N'(total_size/text())[1]','int') AS c_total_size
,NodesWith_c_entry.value(N'(sig/text())[1]','nvarchar(max)') AS c_sig
,NodesWith_c_entry.value(N'(type/text())[1]','nvarchar(max)') AS c_type
,NodesWith_c_entry.value(N'(data_size/text())[1]','int') AS c_data_size
,NodesWith_c_entry.value(N'(data/#encoding)[1]','nvarchar(max)') AS c_data_encoding
,NodesWith_c_entry.value(N'(data/text())[1]','nvarchar(max)') AS c_data_text
FROM #xml.nodes(N'samp_catalog/file/xyz') AS A(xyz)
OUTER APPLY xyz.nodes('a_samp') AS B(a_samp)
OUTER APPLY xyz.nodes('b_samp') AS C(b_samp)
OUTER APPLY xyZ.nodes('*/c_entry') AS D(NodesWith_c_entry);
With the XML given the result is this:
+---------+-------+-------+---------+-----------+--------+--------+---------+-----------+--------------+--------+--------------+-------------+-----------------+-------------+
| name | a_one | a_two | a_three | b_version | b_one | b_two | b_three | c_version | c_total_size | c_sig | c_type | c_data_size | c_data_encoding | c_data_text |
+---------+-------+-------+---------+-----------+--------+--------+---------+-----------+--------------+--------+--------------+-------------+-----------------+-------------+
| 123.wav | 1 | 2 | 3 | 2.0 | sample | sample | sample | 1.0 | 0 | sample | SAMPLE | 256 | string | NULL |
+---------+-------+-------+---------+-----------+--------+--------+---------+-----------+--------------+--------+--------------+-------------+-----------------+-------------+
| 123.wav | 1 | 2 | 3 | 2.0 | sample | sample | sample | 1.0 | 0 | sample | PHONE | 256 | string | 15555551212 |
+---------+-------+-------+---------+-----------+--------+--------+---------+-----------+--------------+--------+--------------+-------------+-----------------+-------------+
| 123.wav | 1 | 2 | 3 | 2.0 | sample | sample | sample | 1.0 | 0 | sample | OTHER_SAMPLE | 256 | string | NULL |
+---------+-------+-------+---------+-----------+--------+--------+---------+-----------+--------------+--------+--------------+-------------+-----------------+-------------+

Answer 2

So far, I haven't found an answer for what I was looking to do, but here is a work around I'm using for now. In VBScript, after loading the file with Microsoft.XMLDOM I loop through the file and find all type nodes whose text value is PHONE and then create a new element and append it to b_samp with its sibling data's text value.
VBScript:
Set nodes = xmlFile.selectNodes("//type")
For Each node in nodes
If node.text = "PHONE" Then
Set phoneNode = xmlFile.CreateElement("phone")
For Each child in node.ParentNode.ChildNodes
If child.nodeName = "data" Then phoneNode.text = child.text
Next
For Each child in node.ParentNode.ParentNode.ParentNode.ChildNodes
If child.nodeName = "b_samp" Then child.appendChild phoneNode
Next
End If
Next
Then save the file with a new name to leave the original intact, and import it with a new schema adding this line right after the b_six element:
<xsd:element name = "phone" type = "xsd:string" sql:field = "phone" />