Hello I have a table called KNXRUNHISTORY and I am trying to pull data from a column called RUNSUMMARYTXT.
The data stored in this column is in xml format.
I am trying to pull the values for these two fields
(entry key="Link1.#OutputType#) and
(entry key="Link6.#SourceRecordsProcessed#)
Any help would be appreciated as I have banging my head against the wall on this one.
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE properties SYSTEM
http://java.sun.com/dtd/properties.dtd">
<properties>
<comment>Comment</comment>
<entry key="#InterfaceErrorCount#">0</entry>
<entry key="Link3.#LinkElapsedTime#">0:00:21</entry>
<entry key="Link1.#OutputType#">4</entry>
<entry key="Link6.#SourceRecordsProcessed#">148</entry>
The general idea is to use xpath query to xml data. But in your case there are some pitfalls.
1. Your data aren't xml data type. Most probably data type of the column in varchar(max) or nvarchar(max). It would be OK but
2. You have <!DOCTYPE declaration which requires conversion with with style equals to 2. Encoding must be utf-16 (yours is utf-8). So encoding in the xml declaration must be fixed or just removed.
Here is working example.
declare #tbl table (id int, prop nvarchar(max))
insert #tbl(id,prop) values(1,
N'<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE properties SYSTEM "http://java.sun.com/dtd/properties.dtd">
<properties>
<comment>Comment</comment>
<entry key="#InterfaceErrorCount#">0</entry>
<entry key="Link3.#LinkElapsedTime#">0:00:21</entry>
<entry key="Link1.#OutputType#">4</entry>
<entry key="Link6.#SourceRecordsProcessed#">148</entry></properties>')
;with cte as(
select id, convert(xml,
replace( prop,'<?xml version="1.0" encoding="UTF-8"?>',''),--sanitize data
2 --Enable limited internal DTD subset processing
) x --and then convert to xml data type
from #tbl
)
select t.v.value('entry[#key="Link1.#OutputType#"][1]','int') outputtype,
t.v.value('entry[#key="Link6.#SourceRecordsProcessed#"][1]','int') srcrecprocessed
from cte cross apply x.nodes('properties') t(v) -- use xpath query
Related
I have the following XML and you can see it contains three LevelA elements. I want to fetch /LevelA/Value (which is 9101) where /LevelA/LevelB2/LevelC == "Address". Now in PROD, the position of elements might change and so the address element might be first or last or in the middle. I am able to write two separate queries based on indexes but how to write one xpath query that contains the condition within it for matching with address and gives back 9101 independent of position of Level A elements.
DECLARE #x XML;
SET #x = '
<root>
<LevelA>
<LevelB>EqualTo</LevelB>
<LevelB2>
<LevelC>Item4</LevelC>
<LevelC2>Item5</LevelC2>
<LevelC3>
<anyType xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:type="xsd:string">123456</anyType>
</LevelC3>
</LevelB2>
<Value xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:type="xsd:boolean">true</Value>
</LevelA>
<LevelA>
<LevelB>EqualTo</LevelB>
<LevelB2>
<LevelC>Item4</LevelC>
<LevelC2>Item5</LevelC2>
<LevelC3>
<anyType xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:type="xsd:string">123456</anyType>
</LevelC3>
</LevelB2>
<Value xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:type="xsd:boolean">true</Value>
</LevelA>
<LevelA>
<LevelB>EqualTo</LevelB>
<LevelB2>
<LevelC>Address</LevelC>
<LevelC2>House</LevelC2>
<LevelC3/>
</LevelB2>
<Value xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:type="xsd:string">9101</Value>
</LevelA>
</root>
'
SELECT #x.value('/root[1]/LevelA[3][1]/LevelB2[1]/LevelC[1]', 'varchar(max)')
SELECT #x.value('/root[1]/LevelA[3][1]/Value[1]', 'int')
You can do
SELECT #x.value('(/root/LevelA[LevelB2/LevelC = "Address"]/Value)[1]', 'int')
To apply the predicate to LevelA that LevelB2/LevelC/text() is "Address"
DB Fiddle
I have a SQL Server table called t for example which has 2 columns and lots of rows
ID (PK, int, not null)
Data (XML(.), not null)
In the Data field i have this XML (this I can't change the format of)
<ArrayOfDataAttribute xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<DataAttribute>
<Name>field1</Name>
<Value>default-value-example</Value>
</DataAttribute>
<DataAttribute>
<Name>field2</Name>
</DataAttribute>
<DataAttribute>
<Name>field5</Name>
<Value>False</Value>
</DataAttribute>
<DataAttribute>
<Name>field4</Name>
<Value>example value</Value>
</DataAttribute>
<DataAttribute>
<Name>field5</Name>
<Value>another value</Value>
</DataAttribute>
</ArrayOfDataAttribute>
I need to return from t the ID and the contents of for the sibling xml item name where it equals field4 or null/empty string if its not there for that row.
So i would end up with this if there were only one row
ID | field4
1 | example value
From reading online it looks like using 'nodes' would be the way to do it but I'm not getting anywhere, the examples I've found online seem to be where people are looking for a specific value. This is the closest I've got:
SELECT T2.Loc.value('.', 'varchar(max)')
FROM t
CROSS APPLY t.Data.nodes('/*:ArrayOfDataAttribute/*:DataAttribute/Name') as T2(Loc)
any help is gratefully appreciated
Many thanks
Richard
What you are looking for is a XQuery predicate and how to stuff a value into your XQuery. The first needs brackets ([]), the second can be achieved with the function sql:variable():
Try it like this:
DECLARE #YourTable TABLE(ID INT IDENTITY,[Data] XML);
INSERT INTO #YourTable VALUES
(N'<ArrayOfDataAttribute xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<DataAttribute>
<Name>field1</Name>
<Value>default-value-example</Value>
</DataAttribute>
<DataAttribute>
<Name>field2</Name>
</DataAttribute>
<DataAttribute>
<Name>field5</Name>
<Value>False</Value>
</DataAttribute>
<DataAttribute>
<Name>field4</Name>
<Value>example value</Value>
</DataAttribute>
<DataAttribute>
<Name>field5</Name>
<Value>another value</Value>
</DataAttribute>
</ArrayOfDataAttribute>');
--A parameter for your search string
DECLARE #searchFor NVARCHAR(MAX)=N'field4'
--the query
SELECT t.ID
,t.[Data].value(N'(/ArrayOfDataAttribute
/DataAttribute[(Name/text())[1]=sql:variable("#searchFor")]
/Value
/text())[1]',N'nvarchar(max)') AS field4
FROM #YourTable t;
The XPath can be read as:
Dive into the array of attributes and look for the <DataAttribute> with a <Name> of a given value. There we need the text() within Value.
Hint: Although there are namespaces, the given sample makes no use of any of them. We can omit the declaration in this case...
I am storing an XML payload in a SQL Server table that has a datatype of 'XML'. The data I am receiving has a section that is enclosed in a CDATA block. Here is an example:
<event>
<projectId>123456</projectId>
<eventTs>2018-01-04T13:07:23</eventTs>
<eventData>
<![CDATA[
<company>
<companyId>849</companyId>
<companyName>My Company Name</companyName>
<activeFlag>Y</activeFlag>
<timestamp>27-JUL-17</timestamp>
</company>
]]>
</eventData>
</event>
when this data lands in my table in the field that has a data type of 'XML' the 'CDATA' block is stripped out but then all of the "<" and ">" characters are escaped. Since those characters are escaped, XPATH queries on that data field no longer work. Is there any way around this behavior short of having to strip out the CDATA block before it is inserted/converted to an XML data type?
This is what the data looks like after being inserted into the XML datatype field:
<event>
<projectId>123456</projectId>
<eventTs>2018-01-04T13:07:23</eventTs>
<eventData>
<company>
<companyId>849</companyId>
<companyName>My Company Name</companyName>
<activeFlag>Y</activeFlag>
<timestamp>27-JUL-17</timestamp>
</company>
</eventData>
</event>
It is a very bad approach to store XML within a CDATA section. Everything within this block is just text. This special text looks like an XML, but you cannot query this XML to return the <companyName>.
Try this:
DECLARE #xml XML=
N'<event>
<projectId>123456</projectId>
<eventTs>2018-01-04T13:07:23</eventTs>
<eventData>
<![CDATA[
<company>
<companyId>849</companyId>
<companyName>My Company Name</companyName>
<activeFlag>Y</activeFlag>
<timestamp>27-JUL-17</timestamp>
</company>
]]>
</eventData>
</event>';
SQL Server's developer decided not even to support CDATA anymore. It will implicitly be taken away, while its content remains properly escaped. But you can read the content without problems:
SELECT #xml.value('(/event/eventData)[1]','nvarchar(max)');
The point is: This result looks like an XML, but - in order to use it like an XML - it must be casted.
This you could do to solve this:
DECLARE #innerXML XML=(SELECT CAST('<eventData>' + #xml.value('(/event/eventData)[1]','nvarchar(max)') + '</eventData>' AS XML));
SET #xml.modify('delete /event/eventData[1]');
SET #xml.modify('insert sql:variable("#innerXML") as last into /event[1]');
SELECT #xml;
Easy going:
As long as the incoming XML is a string (before you try to cast it to XML) you can just throw away the CDATA markers:
DECLARE #xmlString NVARCHAR(MAX)=
N'<event>
<projectId>123456</projectId>
<eventTs>2018-01-04T13:07:23</eventTs>
<eventData>
<![CDATA[
<company>
<companyId>849</companyId>
<companyName>My Company Name</companyName>
<activeFlag>Y</activeFlag>
<timestamp>27-JUL-17</timestamp>
</company>
]]>
</eventData>
</event>';
SELECT CAST(REPLACE(REPLACE(#xmlString,' <![CDATA[',''),']]>','') AS XML)
I am trying to output SQL as XML to match the exact format as the following
<?xml version="1.0" encoding="utf-8"?>
<ProrateImport xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns="http://schema.aldi-
sued.com/Logistics/Shipping/ProrateImport/20151009">
<Prorates>
<Prorate>
<OrderTypeId>1</OrderTypeId>
<DeliveryDate>2015-10-12T00:00:00+02:00</DeliveryDate>
<DivNo>632</DivNo>
<ProrateUnit>1</ProrateUnit>
<ProrateProducts>
<ProrateProduct ProductCode="8467">
<ProrateItems>
<ProrateItem StoreNo="1">
<Quantity>5</Quantity>
</ProrateItem>
<ProrateItem StoreNo="2">
<Quantity>5</Quantity>
</ProrateItem>
<ProrateItem StoreNo="3">
<Quantity>5</Quantity>
</ProrateItem>
</ProrateItems>
</ProrateProduct>
</ProrateProducts>
</Prorate>
</Prorates>
</ProrateImport>
Here is my query:
SELECT
OrderTypeID,
DeliveryDate, DivNo,
ProrateUnit,
(SELECT
ProductOrder [#ProductCode],
(SELECT
ProrateItem [#StoreNo],
CAST(Quantity AS INT) [Quantity]
FROM
##Result2 T3
WHERE
T3.DivNo = T2.DivNo
AND T3.DivNo = T1.DivNo
AND T3.DeliveryDate = T2.DeliveryDate
AND T3.DeliveryDate = T1.DeliveryDate
AND T3.ProductOrder = t2.ProductOrder
FOR XML PATH('ProrateItem'), TYPE, ROOT('ProrateItems')
)
FROM
##Result2 T2
WHERE
T2.DivNo = T1.DivNo
AND T2.DeliveryDate = T1.DeliveryDate
FOR XML PATH('ProrateProduct'), TYPE, ROOT('ProrateProducts')
)
FROM
##Result2 T1
GROUP BY
OrderTypeID, DeliveryDate, DivNo, ProrateUnit
FOR XML PATH('Prorate'), TYPE, ROOT('Prorates')
How do I add in the Following and have the ProrateImport/20151009" change to the current date?
<?xml version="1.0" encoding="utf-8"?>
<ProrateImport xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns="http://schema.aldi-
sued.com/Logistics/Shipping/ProrateImport/20151009">
This is my first time I have used XML
Im not sure i understand. Did you create the first XML yourself and just need to add the last script?
DECLARE #XMLHEADER nvarchar(max)
SET #XMLHEADER = '<?xml version="1.0" encoding="utf-8"?>
<ProrateImport xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns="http://schema.aldi-sued.com/Logistics/Shipping/ProrateImport/'+convert(varchar(8),getdate(),112)+'"
>'
select #xmlheader
And then you just need to add the rest of your output from your select statement.
There are several problems:
How to introduce namespaces?
How to introduce namespaces dynamically
How to add a <?xml ?> directive
two-leveled root (<ProrateImport><Prorate>)
namespaces
You have to use WITH XMLNAMESSPACES to introduce a namespace to your query.
Hint: the naked xmlns is introduced by DEFAULT, the xsi namespace will be introduced automatically by using ELEMENTS XSINIL:
WITH XMLNAMESPACES('http://www.w3.org/2001/XMLSchema' AS xsd
,DEFAULT 'http://schema.aldi-sued.com/Logistics/Shipping/ProrateImport/20151009')
SELECT 1 AS Dummy
FOR XML PATH('rowElement'), ELEMENTS XSINIL, ROOT('root')
The result
<root xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns="http://schema.aldi-sued.com/Logistics/Shipping/ProrateImport/20151009"
xmlns:xsd="http://www.w3.org/2001/XMLSchema">
<rowElement>
<Dummy>1</Dummy>
</rowElement>
</root>
Note: The namespaces must be stated literally. No computations, no variables!
dynamic namespaces
This is - out of the box - impossible. But you might use dynamically created SQL and use EXEC to get your result. Just create exactly the statement as above
DECLARE #cmd VARCHAR(MAX)=
'
WITH XMLNAMESPACES(''http://www.w3.org/2001/XMLSchema'' AS xsd
,DEFAULT ''http://schema.aldi-sued.com/Logistics/Shipping/ProrateImport/' + CONVERT(VARCHAR(8),GETDATE(),112) + ''')
SELECT 1 AS Dummy
FOR XML PATH(''rowElement''), ELEMENTS XSINIL, ROOT(''root'')';
PRINT #cmd
EXEC(#cmd);
the result
<root xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns="http://schema.aldi-sued.com/Logistics/Shipping/ProrateImport/20171019"
xmlns:xsd="http://www.w3.org/2001/XMLSchema">
<rowElement>
<Dummy>1</Dummy>
</rowElement>
</root>
directive
The directive cannot be introduced into XML. SQL-Server will omit any <?xml ?> directive! This can only be done on string level:
DECLARE #cmd VARCHAR(MAX)=
'
WITH XMLNAMESPACES(''http://www.w3.org/2001/XMLSchema'' AS xsd
,DEFAULT ''http://schema.aldi-sued.com/Logistics/Shipping/ProrateImport/' + CONVERT(VARCHAR(8),GETDATE(),112) + ''')
SELECT(
SELECT 1 AS Dummy
FOR XML PATH(''rowElement''), ELEMENTS XSINIL, ROOT(''root'')) AS MyResult';
CREATE TABLE #resultTable(MyXmlAsString VARCHAR(MAX))
INSERT INTO #resultTable(MyXmlAsString)
EXEC(#cmd);
SELECT '<?xml version="1.0" encoding="utf-8"?>' + MyXmlAsString
FROM #resultTable;
The result
<?xml version="1.0" encoding="utf-8"?>
<root xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns="http://schema.aldi-sued.com/Logistics/Shipping/ProrateImport/20171019"
xmlns:xsd="http://www.w3.org/2001/XMLSchema">
<rowElement>
<Dummy>1</Dummy>
</rowElement>
</root>
two-leveled root
You can nest two FOR XML statements to achieve this:
WITH XMLNAMESPACES(DEFAULT 'blah')
SELECT
(
SELECT 1 AS Dummy
FOR XML PATH('rowElement'),ROOT('innerRoot'),TYPE
)
FOR XML PATH('outerRoot');
But the annoying part is, that namespaces are introduced by each sub-select over and over. Not wrong but very annoying! A well known Microsoft connect issue. Please sign in and vote for it! The result:
<outerRoot xmlns="blah">
<innerRoot xmlns="blah"> <!--Here's the second xmlns! -->
<rowElement>
<Dummy>1</Dummy>
</rowElement>
</innerRoot>
</outerRoot>
Your solution
After explained all this I'd suggest to create the XML without any namespace or declaration (what you are doing already!), then convert the result to NVARCHAR(MAX) and add the header and the closing footer on string level. This is ugly, but in your case the only way.
Hint: You will not be able to store the final result in a native XML type in SQL Server without loosing the directive.
My process involves getting a large XML file on a daily basis.
I have developed an SSIS package (2008 r2) which first gets rid of the multiple namespaces via a XSLT and then imports data into 40 tables (due to its complexity) by using the XML source object.
Here is the watered down version of a test xml file
<?xml version="1.0" encoding="UTF-8"?>
<s:Test xmlns:s="http://###.##.com/xml"
<sequence>62</sequence>
<generated>2015-04-28T00:59:38</generated>
<report_date>2015-04-27</report_date>
<orders>
<order>
</order>
</orders>
My question is: The XML source imports all the Orders with its nested attributes. How do I extract the 'report_date' and 'generated' from the header?
Any help would be much appreciated.
Thanks
SD
You can use XML method value() passing proper XPath/XQuery expression as parameter. For demo, consider the following table and data :
CREATE TABLE MyTable (id int, MyXmlColumn XML)
DECLARE #data XML = '<?xml version="1.0" encoding="UTF-8"?>
<s:Test xmlns:s="http://###.##.com/xml">
<sequence>62</sequence>
<generated>2015-04-28T00:59:38</generated>
<report_date>2015-04-27</report_date>
<orders>
<order>
</order>
</orders>
</s:Test>'
INSERT INTO Mytable VALUES(1,#data)
You can use the following query to get generated and report_date data :
SELECT
t.MyXmlColumn.value('(/*/generated)[1]','datetime') as generated
, t.MyXmlColumn.value('(/*/report_date)[1]','date') as report_date
FROM Mytable t
SQL Fiddle Demo
output :
generated report_date
----------------------- -----------
2015-04-28 00:59:38.000 2015-04-27