Selecting data from XML - sql-server

I'm trying to insert a row based on data extracted from a chunk of XML. Some columns need to initialized to node values a couple of nodes "deep" in the XML structure.
I can't seem to the query right. Here's what I got:
declare #xmlRaw xml = '
<LogEntry>
<SummaryMessage>Something bad happened</SummaryMessage>
<Exception>
<Type>System.ApplicationException</Type>
<Message>A test of the error handling</Message>
</Exception>
</LogEntry>'
select
LogEntryColumn.value('SummaryMessage[1]', 'varchar(10)') as SummaryMessage, -- works fine
LogEntryColumn.query('Exception[1]').value('Message[1]', 'varchar(10)') as ExMessage -- not working
from
#xmlRaw.nodes('LogEntry[1]') as LogEntryTable(LogEntryColumn)
This outputs:
SummaryMessage ExMessage
-------------- ----------
Something NULL
I've tried a raft of variations for the "ExMessage" column query but no joy.
Note that I'm using "LogEntryColumn.query(...).value(...)" because I want to check how that form performs versus something like:
select
LogEntryColumn.value('SummaryMessage[1]', 'varchar(10)') as SummaryMessage, -- works fine
ExceptionEntryColumn.value('Message[1]', 'varchar(10)') as ExMessage -- not working
from
#xmlRaw.nodes('LogEntry[1]') as LogEntryTable(LogEntryColumn)
outer apply #xmlData.nodes('LogEntry[1]/Exception') as ExceptionTable(ExceptionEntryColumn)
Basically I'm wondering if multiple "outer apply" from clauses is better/worse than multiple .query(...) invocations.

Here is what you need.
SQL
DECLARE #xmlRaw XML =
N'<LogEntry>
<SummaryMessage>Something bad happened</SummaryMessage>
<Exception>
<Type>System.ApplicationException</Type>
<Message>A test of the error handling</Message>
</Exception>
</LogEntry>';
SELECT c.value('(SummaryMessage/text())[1]', 'varchar(100)') AS SummaryMessage
, c.value('(Exception/Message/text())[1]', 'varchar(100)') AS ExMessage
FROM #xmlRaw.nodes('/LogEntry') AS t(c);
Output
+------------------------+------------------------------+
| SummaryMessage | ExMessage |
+------------------------+------------------------------+
| Something bad happened | A test of the error handling |
+------------------------+------------------------------+

Related

XPath 'contains()' requires a singleton (or empty sequence)

Given the XML:
<Dial>
<DialID>
24521
</DialID>
<DialName>
Base Price
</DialName>
</Dial>
<Dial>
<DialID>
24528
</DialID>
<DialName>
Rush Options
</DialName>
<DialValue>
1.5
</DialValue>
</Dial>
<Dial>
<DialID>
24530
</DialID>
<DialName>
Bill Rush Charges
</DialName>
<DialValue>
School
</DialValue>
</Dial>
I can use the contains() function in my xpath:
//Dial[DialName[contains(text(), 'Bill')]]/DialValue
To retrieve the values I'm after:
School
The above XML is stored in a field in my SQL database so I'm using the .value method to select from that field.
SELECT Dials.DialDetail.value('(//Dial[DialName[contains(text(), "Bill")]]/DialValue)[1]','VARCHAR(64)') AS BillTo
FROM CampaignDials Dials
I can't seem to get the syntax right though... the xpath works as expected (tested in Oxygen and elsewhere) but when I use it in the XQuery argument of the .value() method, I get an error:
Started executing query at Line 1
Msg 2389, Level 16, State 1, Line 36
XQuery [Dials.DialDetail.value()]: 'contains()' requires a singleton (or empty sequence), found operand of type 'xdt:untypedAtomic *'
Total execution time: 00:00:00.004
I've tried different variations of single and double quotes with no effect. The error refers to an XPath data type for attributes, but I'm not retrieving an attribute; I'm getting the text value. I receive the same error if I type the response with //Dial[DialName[contains(text(), 'Bill')]]/DialValue/text() instead.
What is the correct way to use contains() in an XQuery when it's used in the XML.value() method? Or is this the wrong approach to begin with?
You nearly have it right, you just need [1] on the text() function to guarantee a single value.
You should also use text() on the actual node you are pulling out, for performance reasons.
Also, // can be inefficient, so only use it if you really need recursive descent. You can instead use /*/ to get the first node of any name.
SELECT
Dials.DialDetail.value(
'(//Dial[DialName[contains(text()[1], "Bill")]]/DialValue/text())[1]',
'VARCHAR(64)') AS BillTo
FROM CampaignDials Dials
As Yitzhak Kabinsky notes, this only gets you one value per row of the table, you need .nodes if you want to shred the XML itself into rows.
The difference between your actual database case that fails and your reduced sample case that works is likely one of different data.
The error,
contains() requires a singleton (or empty sequence)
indicates that one of your DialName elements has multiple text node children rather than a single text node child as you're expecting.
You can abstract away such variations by testing the string-value of DialName rather than its text node children:
//Dial[contains(DialName, 'Bill')]/DialValue
See also
Testing text() nodes vs string values in XPath
Here is how to do XML shredding in MS SQL Server correctly.
You need to apply filter in the XQuery .nodes() method.
The .value() method is just for the actual value retrieval.
It is possible to pass SQL Server variable as a parameter instead of the hard-coding "Bill" value.
SQL
-- DDL and sample data population, start
DECLARE #tbl TABLE (ID INT IDENTITY PRIMARY KEY, DialDetail XML);
INSERT INTO #tbl (DialDetail) VALUES
(N'<Dial>
<DialID>24521</DialID>
<DialName>Base Price</DialName>
</Dial>
<Dial>
<DialID>24528</DialID>
<DialName>Rush Options</DialName>
<DialValue>1.5</DialValue>
</Dial>
<Dial>
<DialID>24530</DialID>
<DialName>Bill Rush Charges</DialName>
<DialValue>School</DialValue>
</Dial>');
-- DDL and sample data population, end
SELECT ID
, c.value('(DialID/text())[1]', 'INT') AS DialID
, c.value('(DialName/text())[1]', 'VARCHAR(30)') AS DialName
, c.value('(DialValue/text())[1]', 'VARCHAR(30)') AS DialValue
FROM #tbl CROSS APPLY DialDetail.nodes('/Dial[contains((DialName/text())[1], "Bill")]') AS t(c);
Output
+----+--------+-------------------+-----------+
| ID | DialID | DialName | DialValue |
+----+--------+-------------------+-----------+
| 1 | 24530 | Bill Rush Charges | School |
+----+--------+-------------------+-----------+

Sql Server - How do I get JSON nested value in my SQL Select statement

Environment: SQL Server 2014 and above
How do I access the email value in my JSON value with my SELECT statement?
select JSON_VALUE('[{"data":{"email":"test#email.com"}}]', '$.email') as test
Json support was only introduced in SQL Server 2016 - so with any prior version you would need to either use string manipulation code or simply parse the json outside of SQL Server (maybe using a CLR function)
For 2016 version or higher, you can use JSON_VALUE like this:
declare #json as varchar(100) = '[{"data":{"email":"test#email.com"}}]';
select JSON_VALUE(#json, '$[0].data.email') as test
For older versions - you might be able to get away with this, but if your json value does not contain an email property, you will get unexpected results:
select substring(string, start, charindex('"', string, start+1) - start) as test
from (
select #json as string, charindex('"email":"', #json) + 9 as start
) s
You can see a live demo on db<>fiddle
Another way. PatternSplitCM is great for stuff like this.
Extract a single Email value:
DECLARE #json as varchar(200) = '[{"data":{"email":"test#email.com"}}]';
SELECT f.Item
FROM dbo.patternsplitCM(#json,'[a-z0-9#.]') AS f
WHERE f.item LIKE '%[a-z]%#%.%[a-z]%'; -- Simple Email Check Pattern
Extracting all Email Addresses (if/when there are more):
DECLARE #json VARCHAR(200) = '[{"data":{"email":"test#email.com"},{"email2":"test2#email.net"}},{"data":{"MoreEmail":"test3#email.555whatever"}}]';
SELECT f.Item
FROM dbo.patternsplitCM(#json,'[a-z0-9#.]') AS f
WHERE f.item LIKE '%[a-z]%#%.%[a-z]%'; -- Simple Email Check Pattern
Returns:
Item
--------------------------
test#email.com
test2#email.net
test3#email.555whatever
Or... the get only the first Email address that appears:
SELECT TOP (1) f.Item
FROM dbo.patternsplitCM(#json,'[a-z0-9#.]') AS f
WHERE f.item LIKE '%[a-z]%#%.%[a-z]%' -- Simple Email Check Pattern
ORDER BY ROW_NUMBER() OVER (ORDER BY f.ItemNumber)
Nasty fast, super-simple. No cursors, loops or other bad stuff.
With v2014 there is no JSON support, but - if your real JSON is that simple - it is sometimes a good idea to use some replacements in order to transform the JSON to XML like here, which allows for the native XML methods:
DECLARE #YourJSON NVARCHAR(MAX)=N'[{"data":{"email":"test#email.com"}}]';
SELECT CAST(REPLACE(REPLACE(REPLACE(REPLACE(#YourJSON,'[{"','<'),'":{"',' '),'":"','="'),'}}]',' />') AS XML).value('(/data/#email)[1]','nvarchar(max)');
It can be done in two ways:
First, if your JSON data is between [ ] like in your question:
select JSON_VALUE('[{"data":{"email":"test#email.com"}}]','$[0].data.email' ) as test
And if your JSON data is not between [ ]:
select JSON_VALUE('{"data":{"email":"test#email.com"}}','$.data.email' ) as test
You can teste the code above here
Your query should be like this (SQL Server 2016):
DECLARE #json_string NVARCHAR(MAX) = 'your_json_value'
SELECT [key],value
FROM OPENJSON(#json_string, '$.email'))
UPDATE :
select JSON_VALUE(#json_string, '$[0].data.email') as test

How to get the partial value of a XML Node value

I'm new to Xpath and This is my XML . I'm trying to the get the attribute value #name in the appl/*__job tag and the value 'TESTQUEUE 'in the node snmp_notify/message and I'm taking one step at a time. As of now I was able to get the child nodes of all _job, but I couldn't get the value in the node /snmp_notifylist/snmp_notify/message. This is the SQL and Could someone help me with identifying where I got stuck.
This is the Sample XML Document stored as DEFINITION in the table TAB_AR.
<appl xmlns="http://dto.wa.ca.com/application" name="TEST_NEW_AGENT">
<version>12.0</version>
<comment />
<unix_job name="TEST_JOB">
<dependencies><relcount>0</relcount></dependencies>
<snmp_notifylist>
<snmp_notify>
<returncode>4</returncode>
<monitor_states><monitor_state>FAILED</monitor_state></monitor_states>
<snmpagent />
<message>TICKET TESTQUEUE TSTMSG</message>
</snmp_notify>
</snmp_notifylist>
</unix_job>
<link name="HOLD_LINK">
<dependencies><relcount>0</relcount></dependencies>
<hold>true</hold>
<job_ancestor_wait_default_ignore>true</job_ancestor_wait_default_ignore>
</link>
<sftp_job name="TEST_SFTP1">
<dependencies><relcount>0</relcount></dependencies>
<snmp_notifylist>
<snmp_notify>
<returncode>4</returncode>
<monitor_states>
<monitor_state>FAILED</monitor_state>
</monitor_states>
<snmpagent />
<message>TICKET MFG1AWA TSTMSG</message>
</snmp_notify>
</snmp_notifylist>
</sftp_job>
</appl>
And this is the SQL I wrote,
SELECT
SFTP_Job_name = DEFT1.value('(#name)[1]','nvarchar(max)'),
Server_Address = DEFT1.query('local-name(/*:snmp_notifylist/*:snmp_notify/*:message)')
from (select CAST([DEFINITION] as XML) as DEFT from TAB_AR)TAB
CROSS APPLY TAB.DEFT.nodes('/*:appl/*[fn:contains(local-name(),"_job")]') as XMLTAB1(DEFT1)
You were close...
In this line I'm not sure, what you really wanted to get:
DEFT1.query('local-name(/*:snmp_notifylist/*:snmp_notify/*:message)')
With local-name() you can return the name of one specific node. As you are reading from several nodes ending on _job it perfectly makes sense to return the name of the element you are reading from.
But you are telling us, that you are trying to read the <message> too. Might be, that you are mixing two calls in one line?
I slightly modified your code:
SELECT
SFTP_Job_name = DEFT1.value('(#name)[1]','nvarchar(max)')
,NodeName = DEFT1.value('local-name(.)','nvarchar(max)')
,Server_Address = DEFT1.value('(*:snmp_notifylist/*:snmp_notify/*:message)[1]','nvarchar(max)')
from (select CAST([DEFINITION] as XML) as DEFT from TAB_AR)TAB
CROSS APPLY TAB.DEFT.nodes('/*:appl/*[fn:contains(local-name(.),"_job")]') as XMLTAB1(DEFT1);
This returns
SFTP_Job_name NodeName Server_Address
TEST_SFTP1 sftp_job TICKET MFG1AWA TSTMSG
TEST_JOB unix_job TICKET TESTQUEUE TSTMSG
Like Roger Wolf pointed out, it was better to read with a specified namespaces like this:
WITH XMLNAMESPACES (default 'http://dto.wa.ca.com/application')
SELECT
SFTP_Job_name = DEFT1.value('(#name)[1]','nvarchar(max)')
,NodeName = DEFT1.value('local-name(.)','nvarchar(max)')
,Server_Address = DEFT1.value('(snmp_notifylist/snmp_notify/message)[1]','nvarchar(max)')
from (select CAST([DEFINITION] as XML) as DEFT from TAB_AR)TAB
CROSS APPLY TAB.DEFT.nodes('/appl/*[fn:contains(local-name(.),"_job")]') as XMLTAB1(DEFT1);
The general rule is: Be as specific as possible!
Hint
If you can change this, you should store your XML in a column of type XML.
This construction from (select CAST([DEFINITION] as XML) as DEFT from TAB_AR)TAB should really not be necessary...
Might be, that your column is XML actually and you just did not know how to transfer the code you found somewhere to get the right syntax for the .nodes()? In this case just try this:
SELECT
SFTP_Job_name = DEFT1.value('(#name)[1]','nvarchar(max)')
,NodeName = DEFT1.value('local-name(.)','nvarchar(max)')
,Server_Address = DEFT1.value('(*:snmp_notifylist/*:snmp_notify/*:message)[1]','nvarchar(max)')
from TAB_AR
CROSS APPLY TAB_AR.[DEFINITION].nodes('/*:appl/*[fn:contains(local-name(.),"_job")]') as XMLTAB1(DEFT1);
This seems to be working:
with xmlnamespaces (default 'http://dto.wa.ca.com/application')
select j.c.value('./#name', 'sysname') as [JobName],
m.c.value('./text()[1]', 'varchar(max)') as [MessageText]
from (
select cast(t.[Definition] as xml) as [Deft] from tab_ar t
) sq
cross apply sq.Deft.nodes('/appl/*[fn:contains(local-name(),"_job")]') j(c)
cross apply j.c.nodes('./snmp_notifylist/snmp_notify/message') m(c);
After that, splitting the string by spaces and taking the middle part should be relatively trivial.

Multiple values from XML column

I am trying to figure out how to get multiple values from multiple nodes of an XML field in a table (actually it's XML stored as text).
I've seen several methods that involve declaring the XML as a variable and using it as a table but I don't see how that would work for me. How to Extract data from xml column in sql 2008
I am currently using .value to get some fields but I don't see how to make it work since there can be multiple LX01_AssignedNumber and I need to get all of the ProcedureModifier from each.
SELECT CAST(xmldata as xml).value('declare namespace ns1="http://schemas.microsoft.com/BizTalk/EDI/EDIFACT/2006/EnrichedMessageXML";declare namespace ns0="http://schemas.microsoft.com/BizTalk/EDI/X12/2006";
(/ns1:X12EnrichedMessage/TransactionSet/ns0:X12_00501_837_P/ns0:TS837_2000A_Loop/ns0:TS837_2000B_Loop/ns0:TS837_2300_Loop/ns0:TS837_2400_Loop/ns0:SV1_ProfessionalService/ns0:C003_CompositeMedicalProcedureIdentifier/C00303_ProcedureModifier) [1]', 'varchar(20)') AS RendAttendNPI
FROM EDI_DATA
How do I get all the Line Numbers and all of the Procedure Modifiers from each record?
XML:
<ns1:X12EnrichedMessage xmlns:ns1="http://schemas.microsoft.com/BizTalk/EDI/EDIFACT/2006/EnrichedMessageXML">
...
<TransactionSet>
<!-- ProcessLogID=PLG0005169955 ;ProcessLogDetailID=PLG0005173285 ;EnvID=1;RetryCount=1 -->
<ns0:X12_00501_837_P xmlns:ns0="http://schemas.microsoft.com/BizTalk/EDI/X12/2006">
<ns0:TS837_2000A_Loop xmlns:ns0="http://schemas.microsoft.com/BizTalk/EDI/X12/2006">
<ns0:TS837_2000B_Loop xmlns:ns0="http://schemas.microsoft.com/BizTalk/EDI/X12/2006">
<ns0:TS837_2300_Loop xmlns:ns0="http://schemas.microsoft.com/BizTalk/EDI/X12/2006">
<ns0:TS837_2400_Loop>
<ns0:LX_ServiceLineNumber>
<LX01_AssignedNumber>1</LX01_AssignedNumber>
</ns0:LX_ServiceLineNumber>
<ns0:SV1_ProfessionalService>
<ns0:C003_CompositeMedicalProcedureIdentifier>
<C00301_ProductorServiceIDQualifier>HC</C00301_ProductorServiceIDQualifier>
<C00302_ProcedureCode>26340</C00302_ProcedureCode>
<C00303_ProcedureModifier>AG</C00303_ProcedureModifier>
<C00304_ProcedureModifier>58</C00304_ProcedureModifier>
<C00305_ProcedureModifier>51</C00305_ProcedureModifier>
<C00306_ProcedureModifier>XS</C00306_ProcedureModifier>
</ns0:C003_CompositeMedicalProcedureIdentifier>
<SV102_LineItemChargeAmount>8918</SV102_LineItemChargeAmount>
<SV103_UnitorBasisforMeasurementCode>UN</SV103_UnitorBasisforMeasurementCode>
<SV104_ServiceUnitCount>13</SV104_ServiceUnitCount>
<ns0:C004_CompositeDiagnosisCodePointer>
<C00401_DiagnosisCodePointer>1</C00401_DiagnosisCodePointer>
<C00402_DiagnosisCodePointer>2</C00402_DiagnosisCodePointer>
</ns0:C004_CompositeDiagnosisCodePointer>
</ns0:SV1_ProfessionalService>
<ns0:DTP_SubLoop_2>
<ns0:DTP_Date_ServiceDate>
<DTP01_DateTimeQualifier>472</DTP01_DateTimeQualifier>
<DTP02_DateTimePeriodFormatQualifier>D8</DTP02_DateTimePeriodFormatQualifier>
<DTP03_ServiceDate>20160104</DTP03_ServiceDate>
</ns0:DTP_Date_ServiceDate>
</ns0:DTP_SubLoop_2>
<ns0:REF_SubLoop_7>
<ns0:REF_LineItemControlNumber>
<REF01_ReferenceIdentificationQualifier>6R</REF01_ReferenceIdentificationQualifier>
<REF02_LineItemControlNumber>11453481</REF02_LineItemControlNumber>
</ns0:REF_LineItemControlNumber>
</ns0:REF_SubLoop_7>
</ns0:TS837_2400_Loop>
<ns0:TS837_2400_Loop>
<ns0:LX_ServiceLineNumber>
<LX01_AssignedNumber>2</LX01_AssignedNumber>
</ns0:LX_ServiceLineNumber>
<ns0:SV1_ProfessionalService>
<ns0:C003_CompositeMedicalProcedureIdentifier>
<C00301_ProductorServiceIDQualifier>HC</C00301_ProductorServiceIDQualifier>
<C00302_ProcedureCode>20680</C00302_ProcedureCode>
<C00303_ProcedureModifier>58</C00303_ProcedureModifier>
</ns0:C003_CompositeMedicalProcedureIdentifier>
<SV102_LineItemChargeAmount>1277</SV102_LineItemChargeAmount>
<SV103_UnitorBasisforMeasurementCode>UN</SV103_UnitorBasisforMeasurementCode>
<SV104_ServiceUnitCount>1</SV104_ServiceUnitCount>
<ns0:C004_CompositeDiagnosisCodePointer>
<C00401_DiagnosisCodePointer>3</C00401_DiagnosisCodePointer>
</ns0:C004_CompositeDiagnosisCodePointer>
</ns0:SV1_ProfessionalService>
</ns0:TS837_2400_Loop>
</ns0:TS837_2300_Loop>
</ns0:TS837_2000B_Loop>
</ns0:TS837_2000A_Loop>
</ns0:X12_00501_837_P>
</TransactionSet>
</ns1:X12EnrichedMessage>
Look into SQL Server CROSS APPLY which you can use to shred single XML data into multiple rows, for example :
;WITH XMLNAMESPACES ('http://schemas.microsoft.com/BizTalk/EDI/X12/2006' as ns0
,'http://schemas.microsoft.com/BizTalk/EDI/EDIFACT/2006/EnrichedMessageXML' as ns1)
SELECT
TS837_2400_Loop.value('(.//LX01_AssignedNumber)[1]', 'int') 'line_number'
,C00303_ProcedureModifier.value('.', 'varchar(100)') 'procedure_modifier'
FROM EDI_DATA
CROSS APPLY (select CONVERT(XML, xmldata)) as P(X)
CROSS APPLY X.nodes('.//ns0:TS837_2400_Loop') AS Q(TS837_2400_Loop)
CROSS APPLY TS837_2400_Loop.nodes('.//C00303_ProcedureModifier') AS R(C00303_ProcedureModifier)
sqlfiddle demo
output :
| line_number | procedure_modifier |
|-------------|--------------------|
| 1 | AG |
| 2 | 58 |

Extracting XML field in SQL Server

I have fldxml column in a MIWOD table that contains multiple type of data. How can I display [fldxml] column value as shown below?
Routing Drawing
1 C:\Users\XXX\Documents\LETTUCE_WHEEL.pdf
Here is how [fldxml] looks like
<fields>
<field1>1</field1>
<field2>C:\Users\XXX\Documents\LETTUCE_WHEEL.pdf</field2>
</fields>
I want get something like this
Routing Drawing:
1 C:\Users\XXX\Documents\LETTUCE_WHEEL.pdf
I tried using the following
SELECT
MIWOD.fldXml('(field1/text())[1]', 'varchar(50)') as Routing,
MIWOD.fldXml('(field1/text())[1]', 'varchar(50)') as Routing
FROM
[MISAMPCO].[dbo].[MIWOD]
But I get the following error
Cannot find either column "MIWOD" or the user-defined function or aggregate "MIWOD.fldXml", or the name is ambiguous.
DECLARE #stuff xml
SET #stuff = '<fields><field1>1</field1><field2>C:\Users\Kinfe\Documents\LETTUCE_WHEEL.pdf</field2></fields>'
SELECT
Child.value('field1[1]', 'int') [**Routing:**],
Child.value('field2[1]', 'nvarchar(max)') [**Drawing:**]
FROM
#stuff.nodes('fields[1]') as N(Child)

Resources