XML Data multiple nodes - sql-server

I have this xml file that all the rest of the nodes depends on the first one. I created a tree diagram to help illustrate it.
The problem I am having is that each node depends on the first one but they do not have any columns that link each other.
Here is the Link to the
XML Document
Here is my sql so far
SELECT
B.RD.query('racedata/todays_cls').value('.','varchar(max)') AS todays_cls,
B.RD.query('racedata/horsedata/horse_name').value('.', 'varchar(max)') AS horse_name,
B.RD.query('racedata/horsedata/jockey/jock_disp').value('.', 'varchar(max)') AS jockeyname
FROM #xmlData.nodes('data') AS B(RD)

Do the second CROSS APPLY on the result of the first one, so that you get <horsedata> elements that corresponds to current <racedata>. See a working demo example below.
sample data :
declare #xml XML = '
<data xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="http://www.trackmaster.com/xmlSchema/ppXMLData.xsd">
<racedata>
<race>1</race>
<todays_cls>59</todays_cls>
<distance>800.0</distance>
<country>USA</country>
<horsedata>
<horse_name>BROADWAY KATE</horse_name>
<owner_name>C. Steve Larue</owner_name>
<program>1</program>
<pp>1</pp>
<weight>121</weight>
</horsedata>
<horsedata>
<horse_name>UNFAITHFUL</horse_name>
<owner_name>Melson, James L., Thomson, Ramona and Connell</owner_name>
<program>2</program>
<pp>2</pp>
<weight>121</weight>
</horsedata>
</racedata>
<racedata>
<race>2</race>
<todays_cls>87</todays_cls>
<distance>800.0</distance>
<country>USA</country>
<horsedata>
<horse_name>MAGNETIC START</horse_name>
<owner_name>Vernon D. Coyle</owner_name>
<program>1</program>
<pp>1</pp>
<weight>121</weight>
</horsedata>
<horsedata>
<horse_name>SKI POLE</horse_name>
<owner_name>Downunder Cable, LLC</owner_name>
<program>2</program>
<pp>2</pp>
<weight>121</weight>
</horsedata>
</racedata>
</data>
'
query :
SELECT
rd.value('race[1]', 'int') AS race,
rd.value('todays_cls[1]','int') AS todays_cls,
hd.value('pp[1]','int') AS pp,
hd.value('weight[1]','int') AS weight
FROM #xml.nodes('data/racedata') AS B(RD)
CROSS APPLY RD.nodes('horsedata') AS C(HD)
output :
UPDATE
In response to the updated query, you should've shredded on horsedata element level as suggested in the above query, something like this :
SELECT
B.RD.query('todays_cls').value('.','varchar(max)') AS todays_cls,
C.HD.query('horse_name').value('.', 'varchar(max)') AS horse_name,
C.HD.query('jockey/jock_disp').value('.', 'varchar(max)') AS jockeyname
FROM #xml.nodes('data/racedata') AS B(RD)
CROSS APPLY RD.nodes('horsedata') AS C(HD)

Related

Can't figure out how to search XML column in my table

I have a table called v_EpisodeAudit, with a column called EventData that contains XML data. The XML data differs from row to row, so one record could have XML data in this column that looks like this:
<AddMDMDocument>
<EpisodeMDMId>282521</EpisodeMDMId>
<OncologyReferral>0</OncologyReferral>
<SpecialPalliativeReferral>0</SpecialPalliativeReferral>
<SurgeonReferral>0</SurgeonReferral>
<MDMReport>0</MDMReport>
<GPReferral>0</GPReferral>
<GPReferralApproval>0</GPReferralApproval>
<GeneralPalliativeCare>0</GeneralPalliativeCare>
<AuditLogin>mkell010</AuditLogin>
<AuditTrust>4</AuditTrust>
<Error />
</AddMDMDocument>
while another row might contain the following XML data:
<CloseEpisode>
<EpisodeId>652503</EpisodeId>
<TrackingStatusId>9</TrackingStatusId>
<TrackingClosureReason>100</TrackingClosureReason>
<DateOfTrackingClosure>Sep 25 2017 12:37PM</DateOfTrackingClosure>
<AuditLogin>ccass001</AuditLogin>
<AuditTrust>1</AuditTrust>
<Error />
</CloseEpisode>
And there are further differing types/configurations of XML data. I've read about 20 different sources this morning trying to work out how to search against the XML data in this column to get a specific EpisodeId in the CloseEpisode XMLs, and I can't for the life of me figure it out. Can anyone help me with a query that will find a specified EpisodeId in this column?
XML can be queried very generically. Some approaches:
DECLARE #v_EpisodeAudit TABLE(ID INT IDENTITY, [EventData] XML);
INSERT INTO #v_EpisodeAudit VALUES
(N'<AddMDMDocument>
<EpisodeMDMId>282521</EpisodeMDMId>
<OncologyReferral>0</OncologyReferral>
<SpecialPalliativeReferral>0</SpecialPalliativeReferral>
<SurgeonReferral>0</SurgeonReferral>
<MDMReport>0</MDMReport>
<GPReferral>0</GPReferral>
<GPReferralApproval>0</GPReferralApproval>
<GeneralPalliativeCare>0</GeneralPalliativeCare>
<AuditLogin>mkell010</AuditLogin>
<AuditTrust>4</AuditTrust>
<Error />
</AddMDMDocument>')
,(N'<CloseEpisode>
<EpisodeId>652503</EpisodeId>
<TrackingStatusId>9</TrackingStatusId>
<TrackingClosureReason>100</TrackingClosureReason>
<DateOfTrackingClosure>Sep 25 2017 12:37PM</DateOfTrackingClosure>
<AuditLogin>ccass001</AuditLogin>
<AuditTrust>1</AuditTrust>
<Error />
</CloseEpisode>');00
--This will return the very first node on the second level
SELECT ID
,vEA.[EventData].value(N'local-name(/*[1]/*[1])',N'nvarchar(max)') AS NodeName
,vEA.[EventData].value(N'/*[1]/*[1]/text()[1]',N'nvarchar(max)') AS NodeValue
FROM #v_EpisodeAudit AS vEA
--This will return all nodes of the sevond level and use WHERE with LIKE to find the Episode..Id elements
SELECT ID
,SecondLevelNode.Nd.value(N'local-name(.)',N'nvarchar(max)') AS NodeName
,SecondLevelNode.Nd.value(N'text()[1]',N'nvarchar(max)') AS NodeValue
FROM #v_EpisodeAudit AS vEA
OUTER APPLY vEA.[EventData].nodes(N'/*/*') AS SecondLevelNode(Nd)
WHERE SecondLevelNode.Nd.value(N'local-name(.)',N'nvarchar(max)') LIKE 'Episode%' --or LIKE 'Episode%Id'
--Similar but filtering on XQuery level
SELECT ID
,SecondLevelNode.Nd.value(N'local-name(.)',N'nvarchar(max)') AS NodeName
,SecondLevelNode.Nd.value(N'text()[1]',N'nvarchar(max)') AS NodeValue
FROM #v_EpisodeAudit AS vEA
OUTER APPLY vEA.[EventData].nodes(N'/*/*[substring(local-name(),1,7)="Episode"]') AS SecondLevelNode(Nd)
Use the xml querying functions
select EventData.value('(/CloseEpisode/EpisodeId)[1]','int')
from v_EpisodeAudit
where EventData.value('local-name(/*[1])','varchar(100)')='CloseEpisode'
or perhaps
select EventData
from #v_EpisodeAudit
where EventData.value('(/CloseEpisode/EpisodeId)[1]','int')=652503
depending on what you're trying to do.
If you don't know the root node name, you could use
select EventData.value('(//EpisodeId)[1]','int')
from v_EpisodeAudit
where EventData.exist('//EpisodeId')=1
See https://learn.microsoft.com/en-us/sql/t-sql/xml/value-method-xml-data-type

How to get the partial value of a XML Node value

I'm new to Xpath and This is my XML . I'm trying to the get the attribute value #name in the appl/*__job tag and the value 'TESTQUEUE 'in the node snmp_notify/message and I'm taking one step at a time. As of now I was able to get the child nodes of all _job, but I couldn't get the value in the node /snmp_notifylist/snmp_notify/message. This is the SQL and Could someone help me with identifying where I got stuck.
This is the Sample XML Document stored as DEFINITION in the table TAB_AR.
<appl xmlns="http://dto.wa.ca.com/application" name="TEST_NEW_AGENT">
<version>12.0</version>
<comment />
<unix_job name="TEST_JOB">
<dependencies><relcount>0</relcount></dependencies>
<snmp_notifylist>
<snmp_notify>
<returncode>4</returncode>
<monitor_states><monitor_state>FAILED</monitor_state></monitor_states>
<snmpagent />
<message>TICKET TESTQUEUE TSTMSG</message>
</snmp_notify>
</snmp_notifylist>
</unix_job>
<link name="HOLD_LINK">
<dependencies><relcount>0</relcount></dependencies>
<hold>true</hold>
<job_ancestor_wait_default_ignore>true</job_ancestor_wait_default_ignore>
</link>
<sftp_job name="TEST_SFTP1">
<dependencies><relcount>0</relcount></dependencies>
<snmp_notifylist>
<snmp_notify>
<returncode>4</returncode>
<monitor_states>
<monitor_state>FAILED</monitor_state>
</monitor_states>
<snmpagent />
<message>TICKET MFG1AWA TSTMSG</message>
</snmp_notify>
</snmp_notifylist>
</sftp_job>
</appl>
And this is the SQL I wrote,
SELECT
SFTP_Job_name = DEFT1.value('(#name)[1]','nvarchar(max)'),
Server_Address = DEFT1.query('local-name(/*:snmp_notifylist/*:snmp_notify/*:message)')
from (select CAST([DEFINITION] as XML) as DEFT from TAB_AR)TAB
CROSS APPLY TAB.DEFT.nodes('/*:appl/*[fn:contains(local-name(),"_job")]') as XMLTAB1(DEFT1)
You were close...
In this line I'm not sure, what you really wanted to get:
DEFT1.query('local-name(/*:snmp_notifylist/*:snmp_notify/*:message)')
With local-name() you can return the name of one specific node. As you are reading from several nodes ending on _job it perfectly makes sense to return the name of the element you are reading from.
But you are telling us, that you are trying to read the <message> too. Might be, that you are mixing two calls in one line?
I slightly modified your code:
SELECT
SFTP_Job_name = DEFT1.value('(#name)[1]','nvarchar(max)')
,NodeName = DEFT1.value('local-name(.)','nvarchar(max)')
,Server_Address = DEFT1.value('(*:snmp_notifylist/*:snmp_notify/*:message)[1]','nvarchar(max)')
from (select CAST([DEFINITION] as XML) as DEFT from TAB_AR)TAB
CROSS APPLY TAB.DEFT.nodes('/*:appl/*[fn:contains(local-name(.),"_job")]') as XMLTAB1(DEFT1);
This returns
SFTP_Job_name NodeName Server_Address
TEST_SFTP1 sftp_job TICKET MFG1AWA TSTMSG
TEST_JOB unix_job TICKET TESTQUEUE TSTMSG
Like Roger Wolf pointed out, it was better to read with a specified namespaces like this:
WITH XMLNAMESPACES (default 'http://dto.wa.ca.com/application')
SELECT
SFTP_Job_name = DEFT1.value('(#name)[1]','nvarchar(max)')
,NodeName = DEFT1.value('local-name(.)','nvarchar(max)')
,Server_Address = DEFT1.value('(snmp_notifylist/snmp_notify/message)[1]','nvarchar(max)')
from (select CAST([DEFINITION] as XML) as DEFT from TAB_AR)TAB
CROSS APPLY TAB.DEFT.nodes('/appl/*[fn:contains(local-name(.),"_job")]') as XMLTAB1(DEFT1);
The general rule is: Be as specific as possible!
Hint
If you can change this, you should store your XML in a column of type XML.
This construction from (select CAST([DEFINITION] as XML) as DEFT from TAB_AR)TAB should really not be necessary...
Might be, that your column is XML actually and you just did not know how to transfer the code you found somewhere to get the right syntax for the .nodes()? In this case just try this:
SELECT
SFTP_Job_name = DEFT1.value('(#name)[1]','nvarchar(max)')
,NodeName = DEFT1.value('local-name(.)','nvarchar(max)')
,Server_Address = DEFT1.value('(*:snmp_notifylist/*:snmp_notify/*:message)[1]','nvarchar(max)')
from TAB_AR
CROSS APPLY TAB_AR.[DEFINITION].nodes('/*:appl/*[fn:contains(local-name(.),"_job")]') as XMLTAB1(DEFT1);
This seems to be working:
with xmlnamespaces (default 'http://dto.wa.ca.com/application')
select j.c.value('./#name', 'sysname') as [JobName],
m.c.value('./text()[1]', 'varchar(max)') as [MessageText]
from (
select cast(t.[Definition] as xml) as [Deft] from tab_ar t
) sq
cross apply sq.Deft.nodes('/appl/*[fn:contains(local-name(),"_job")]') j(c)
cross apply j.c.nodes('./snmp_notifylist/snmp_notify/message') m(c);
After that, splitting the string by spaces and taking the middle part should be relatively trivial.

SQL Server FOR XML PATH: Set xml-declaration or processing instruction "xml-stylesheet" on top

I want to set a processing instruction to include a stylesheet on top of an XML:
The same issue was with the xml-declaration (e.g. <?xml version="1.0" encoding="utf-8"?>)
Desired result:
<?xml-stylesheet type="text/xsl" href="stylesheet.xsl"?>
<TestPath>
<Test>Test</Test>
<SomeMore>SomeMore</SomeMore>
</TestPath>
My research brought me to node test syntax and processing-instruction().
This
SELECT 'type="text/xsl" href="stylesheet.xsl"' AS [processing-instruction(xml-stylesheet)]
,'Test' AS Test
,'SomeMore' AS SomeMore
FOR XML PATH('TestPath')
produces this:
<TestPath>
<?xml-stylesheet type="text/xsl" href="stylesheet.xsl"?>
<Test>Test</Test>
<SomeMore>SomeMore</SomeMore>
</TestPath>
All hints I found tell me to convert the XML to VARCHAR, concatenate it "manually" and convert it back to XML. But this is - how to say - ugly?
This works obviously:
SELECT CAST(
'<?xml-stylesheet type="text/xsl" href="stylesheet.xsl"?>
<TestPath>
<Test>Test</Test>
<SomeMore>SomeMore</SomeMore>
</TestPath>' AS XML);
Is there a chance to solve this?
There is another way, which will need two steps but don't need you to treat the XML as string anywhere in the process :
declare #result XML =
(
SELECT
'Test' AS Test,
'SomeMore' AS SomeMore
FOR XML PATH('TestPath')
)
set #result.modify('
insert <?xml-stylesheet type="text/xsl" href="stylesheet.xsl"?>
before /*[1]
')
Sqlfiddle Demo
The XQuery expression passed to modify() function tells SQL Server to insert the processing instruction node before the root element of the XML.
UPDATE :
Found another alternative based on the following thread : Merge the two xml fragments into one? . I personally prefer this way :
SELECT CONVERT(XML, '<?xml-stylesheet type="text/xsl" href="stylesheet.xsl"?>'),
(
SELECT
'Test' AS Test,
'SomeMore' AS SomeMore
FOR XML PATH('TestPath')
)
FOR XML PATH('')
Sqlfiddle Demo
As it came out, har07's great answer does not work with an XML-declaration. The only way I could find was this:
DECLARE #ExistingXML XML=
(
SELECT
'Test' AS Test,
'SomeMore' AS SomeMore
FOR XML PATH('TestPath'),TYPE
);
DECLARE #XmlWithDeclaration NVARCHAR(MAX)=
(
SELECT N'<?xml version="1.0" encoding="UTF-8"?>'
+
CAST(#ExistingXml AS NVARCHAR(MAX))
);
SELECT #XmlWithDeclaration;
You must stay in the string line after this step, any conversion to real XML will either give an error (when the encoding is other then UTF-16) or will omit this xml-declaration.

Bulk Import of XML Into Existing Tables

I am new to XML and SQL Server and am trying import an XML file into SQL Server 2010. I have 14 tables that I would like to parse the data into. All 14 table names are listed in the XML as nodes (I think) I found some example code that worked with the simple example XML, but my XML seems a little more complicated and may not be structured optimally; unfortunately, I can't change that. As a basic attempt, I tried to insert the data into just one field of one existing table (SILVX_SN16000), but the Message pane shows "(0 rows(s) affected). Thanks in advance for looking at this.
USE TEST
Declare #xml XML
Select #xml =
CONVERT(XML,bulkcolumn,2) FROM OPENROWSET(BULK 'C:\Users\Kevin_S\Documents \SilvxInSightImport.xml',SINGLE_BLOB) AS X
SET ARITHABORT ON
Insert into [SILVX_SN16000]
(
md_group
)
Select
P.value('MD_GROUP[1]','NVARCHAR(255)') AS md_group
From #xml.nodes('/TableData/Row') PropertyFeed(P)
Here is a much-shortened (rows removed) version of my XML:
<?xml version="1.0" ?>
<SilvxInSightImport Version="1.0" Host="uslsss17" Date="14-09-14_20-40-02">
<Tables Count="14">
<Table Name="SN16000">
<TableSchema>
<Column><COLUMN_NAME>PARENT_HPKEY</COLUMN_NAME><DATA_TYPE>VARCHAR2</DATA_TYPE></Column>
<Column><COLUMN_NAME>MD_GROUP</COLUMN_NAME><DATA_TYPE>VARCHAR2</DATA_TYPE></Column>
<Column><COLUMN_NAME>PKEY</COLUMN_NAME><DATA_TYPE>NUMBER</DATA_TYPE></Column>
<Column><COLUMN_NAME>S_STATE</COLUMN_NAME><DATA_TYPE>VARCHAR2</DATA_TYPE></Column>
<Column><COLUMN_NAME>NAME</COLUMN_NAME><DATA_TYPE>VARCHAR2</DATA_TYPE></Column>
<Column><COLUMN_NAME>ROUTER_ID</COLUMN_NAME><DATA_TYPE>VARCHAR2</DATA_TYPE></Column>
<Column><COLUMN_NAME>IP_ADDR</COLUMN_NAME><DATA_TYPE>VARCHAR2</DATA_TYPE></Column>
</TableSchema>
<TableData>
<Row><MD_GROUP>100.120.25162</MD_GROUP><PARENT_HPKEY>100</PARENT_HPKEY> <PKEY>161888</PKEY><NAME>UODEDTM010</NAME><ROUTER_ID>10.41.32.129</ROUTER_ID> <IP_ADDR>10.41.32.129</IP_ADDR><S_STATE>IS-NR</S_STATE></Row>
<Row><MD_GROUP>100.120.25162</MD_GROUP><PARENT_HPKEY>100</PARENT_HPKEY> <PKEY>278599</PKEY><NAME>UODEETM010</NAME><ROUTER_ID>10.41.4.129</ROUTER_ID> <IP_ADDR>10.41.4.129</IP_ADDR><S_STATE>IS-NR</S_STATE></Row>
<Row><MD_GROUP>100.120.25162</MD_GROUP><PARENT_HPKEY>100</PARENT_HPKEY> <PKEY>183583</PKEY><NAME>UODEGRM010</NAME><ROUTER_ID>10.41.76.129</ROUTER_ID> <IP_ADDR>10.41.76.129</IP_ADDR><S_STATE>IS-NR</S_STATE></Row>
NT_HPKEY>100</PARENT_HPKEY><PKEY>811003</PKEY><NAME>UODWTIN010</NAME> <ROUTER_ID>10.27.36.130</ROUTER_ID><IP_ADDR>10.27.36.130</IP_ADDR><S_STATE>IS-NR</S_STATE> </Row>
</TableData>
</Table>
</Tables>
</SilvxInSightImport>
The xPath in .nodes() must specify the whole path to the Row nodes so you should start with SilvxInSightImport and work your way down to Row.
/SilvxInSightImport/Tables/Table/TableData/Row
In your case you have multiple table nodes, one for each table and I assume you only need one table at a time. You can use a predicate on the table name in the .nodes() xPath expression.
/SilvxInSightImport/Tables/Table[#Name = "SN16000"]/TableData/Row
Your whole query for SN16000 should look something like this.
select T.X.value('(MD_GROUP/text())[1]', 'varchar(20)') as MD_GROUP,
T.X.value('(PARENT_HPKEY/text())[1]', 'int') as PARENT_HPKEY,
T.X.value('(PKEY/text())[1]', 'int') as PKEY,
T.X.value('(NAME/text())[1]', 'varchar(20)') as NAME,
T.X.value('(ROUTER_ID/text())[1]', 'varchar(20)') as ROUTER_ID,
T.X.value('(IP_ADDR/text())[1]', 'varchar(20)') as IP_ADDR,
T.X.value('(S_STATE/text())[1]', 'varchar(20)') as S_STATE
from #XML.nodes('/SilvxInSightImport/Tables/Table[#Name = "SN16000"]/TableData/Row') as T(X)
You have to sort out the data types used for each column.
SQL Fiddle

Return all xml nodes import into SQL Server

I have imported an xml document into SQL Server, I am now trying to import various parts into different tables. When I use the below query it only returns one row of hotel_facilities, I need to return all the hotel_facilities with the hotel_ref.
DECLARE #Details xml
SET #Details = '<hotels>
<hotel>
<hotel_ref>105</hotel_ref>
<hotel_facilities>
<id>2</id>
<name>Disabled Facilities</name>
<id>4</id>
<name>24 Hour Reception</name>
<id>12</id>
<name>Restaurant</name>
</hotel_facilities>
</hotel>
</hotels>'
SELECT tab.col.value('../hotel_ref[1]','varchar(100)') AS 'hotel_ref',
tab.col.value('./id[1]','varchar(100)') AS 'HotelFacilityID',
tab.col.value('./name[1]','varchar(100)') AS 'HotelFacilityName'
FROM #Details.nodes('//hotels/hotel/hotel_facilities') AS tab(col)
Your XML is structured a bit funny - the <hotel_facilities> doesn't contain a proper "sub-entity" which you could enumerate over......
If your facilities would be wrapped into a <facility>....</facility> element, you could easily enumerate that.
<hotel_facilities>
<facility>
<id>2</id>
<name>Disabled Facilities</name>
</facility>
<facility>
<id>4</id>
<name>24 Hour Reception</name>
</facility>
<facility>
<id>12</id>
<name>Restaurant</name>
</facility>
</hotel_facilities>
But with your current setup, I think you'll be hard-pressed to find a good solution....
I agree with marc_s The XML does not have a good schema.
The closest I could get is:
SELECT tab.col.value('./hotel_ref[1]','varchar(100)') AS 'hotel_ref',
fac.value('(.)[1]','varchar(100)') AS 'HotelFacilityID',
ROWID=IDENTITY(int,1,1)
into #facilitiesid
FROM #Details.nodes('/hotels/hotel') AS tab(col)
cross apply col.nodes('.//id') a(fac)
SELECT tab.col.value('../hotel_ref[1]','varchar(100)') AS 'hotel_ref',
fac.value('(.)[1]','varchar(100)') AS 'HotelFacilityName',
ROWID=IDENTITY(int,1,1)
into #facilitiesnames
FROM #Details.nodes('//hotels/hotel/hotel_facilities') AS tab(col)
cross apply col.nodes('.//name') a(fac)
select i.hotel_ref, HotelFacilityID, HotelFacilityName
from #facilitiesid i
inner join #facilitiesnames n
on i.rowid = n.rowid

Resources