Convert XML from one format to another - sql-server

I have this below xml data which is stored in a table.
The XML Structure I have
<Response>
<Question ID="1">
<Value ID="1">I want a completely natural childbirth - no medical interventions for me</Value>
<Value ID="2">no medical interventions for me</Value>
</Question>
</Response>
I need to convert this XML to a slightly different format, like the below one.
The XML Structure I need
<Response>
<Question ID="1">
<SelectedChoices>
<Choice>
<ID>1</ID>
</Choice>
<Choice>
<ID>2</ID>
</Choice>
</SelectedChoices>
</Question>
</Response>
Here the "Value" is changed to "Choice" and "ID" attribute of "Value" element is changed to an element.
I know this can be done in other ways, like using an XSLT. But it will be much more helpful if can accomplish with SQL itself.
Can someone help me to convert this using SQL?

Use this variable to test the statements
DECLARE #xml XML=
N'<Response>
<Question ID="1">
<Value ID="1">I want a completely natural childbirth - no medical interventions for me</Value>
<Value ID="2">no medical interventions for me</Value>
</Question>
</Response>';
This can be done with FLWOR-XQuery:
The query will re-build the XML out of itself... Very similar to XSLT...
SELECT #xml.query(
N'
<Response>
{
for $q in /Response/Question
return
<Question ID="{$q/#ID}">
<SelectedChoices>
{
for $v in $q/Value
return <Choice><ID>{string($v/#ID)}</ID></Choice>
}
</SelectedChoices>
</Question>
}
</Response>
'
);
Another approach: Shredding and re-build
You'd reach the same with this, but I'd prefere the first...
WITH Shredded AS
(
SELECT q.value('#ID','int') AS qID
,v.value('#ID','int') AS vID
FROM #xml.nodes('/Response/Question') AS A(q)
OUTER APPLY q.nodes('Value') AS B(v)
)
SELECT t1.qID AS [#ID]
,(
SELECT t2.vID AS ID
FROM Shredded AS t2
WHERE t1.qID=t2.qID
FOR XML PATH('Choice'),ROOT('SelectedChoices'),TYPE
) AS [node()]
FROM Shredded AS t1
GROUP BY t1.qID
FOR XML PATH('Question'),ROOT('Response')

Related

XQuery - Fetch XML Value from SQL Server using XPath based on Condition

I have the following XML and you can see it contains three LevelA elements. I want to fetch /LevelA/Value (which is 9101) where /LevelA/LevelB2/LevelC == "Address". Now in PROD, the position of elements might change and so the address element might be first or last or in the middle. I am able to write two separate queries based on indexes but how to write one xpath query that contains the condition within it for matching with address and gives back 9101 independent of position of Level A elements.
DECLARE #x XML;
SET #x = '
<root>
<LevelA>
<LevelB>EqualTo</LevelB>
<LevelB2>
<LevelC>Item4</LevelC>
<LevelC2>Item5</LevelC2>
<LevelC3>
<anyType xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:type="xsd:string">123456</anyType>
</LevelC3>
</LevelB2>
<Value xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:type="xsd:boolean">true</Value>
</LevelA>
<LevelA>
<LevelB>EqualTo</LevelB>
<LevelB2>
<LevelC>Item4</LevelC>
<LevelC2>Item5</LevelC2>
<LevelC3>
<anyType xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:type="xsd:string">123456</anyType>
</LevelC3>
</LevelB2>
<Value xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:type="xsd:boolean">true</Value>
</LevelA>
<LevelA>
<LevelB>EqualTo</LevelB>
<LevelB2>
<LevelC>Address</LevelC>
<LevelC2>House</LevelC2>
<LevelC3/>
</LevelB2>
<Value xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:type="xsd:string">9101</Value>
</LevelA>
</root>
'
SELECT #x.value('/root[1]/LevelA[3][1]/LevelB2[1]/LevelC[1]', 'varchar(max)')
SELECT #x.value('/root[1]/LevelA[3][1]/Value[1]', 'int')
You can do
SELECT #x.value('(/root/LevelA[LevelB2/LevelC = "Address"]/Value)[1]', 'int')
To apply the predicate to LevelA that LevelB2/LevelC/text() is "Address"
DB Fiddle

Insert XML child node to SQL table

I've got an XML file like this and I'm working with SQL 2014 SP2
<?xml version='1.0' encoding='UTF-8'?>
<gwl>
<version>123456789</version>
<entities>
<entity id="1" version="123456789">
<name>xxxxx</name>
<listId>0</listId>
<listCode>Oxxx</listCode>
<entityType>08</entityType>
<createdDate>03/03/1993</createdDate>
<lastUpdateDate>05/06/2011</lastUpdateDate>
<source>src</source>
<OriginalSource>o_src</OriginalSource>
<aliases>
<alias category="STRONG" type="Alias">USCJSC</alias>
<alias category="WEAK" type="Alias">'OSKOAO'</alias>
</aliases>
<programs>
<program type="21">prog</program>
</programs>
<sdfs>
<sdf name="OriginalID">9876</sdf>
</sdfs>
<addresses>
<address>
<address1>1141, SYA-KAYA STR.</address1>
<country>RU</country>
<postalCode>1234</postalCode>
</address>
<address>
<address1>90, MARATA UL.</address1>
<country>RU</country>
<postalCode>1919</postalCode>
</address>
</addresses>
<otherIds>
<childId>737606</childId>
<childId>737607</childId>
</otherIds>
</entity>
</entities>
</gwl>
I made a script to insert data from the XML to a SQL table. How can I insert child node into a table? I think I should replicate the row for each new child node but i don't know the best way to proceed.
Here is my SQL code
DECLARE #InputXML XML
SELECT #InputXML = CAST(x AS XML)
FROM OPENROWSET(BULK 'C:\MyFiles\sample.XML', SINGLE_BLOB) AS T(x)
SELECT
product.value('(#id)[1]', 'NVARCHAR(10)') id,
product.value('(#version)[1]', 'NVARCHAR(14)') ID
product.value('(name[1])', 'NVARCHAR(255)') name,
product.value('(listId[1])', 'NVARCHAR(9)')listId,
product.value('(listCode[1])', 'NVARCHAR(10)')listCode,
product.value('(entityType[1])', 'NVARCHAR(2)')entityType,
product.value('(createdDate[1])', 'NVARCHAR(10)')createdDate,
product.value('(lastUpdateDate[1])', 'NVARCHAR(10)')lastUpdateDate,
product.value('(source[1])', 'NVARCHAR(15)')source,
product.value('(OriginalSource[1])', 'NVARCHAR(50)')OriginalSource,
product.value('(aliases[1])', 'NVARCHAR(50)')aliases,
product.value('(programs[1])', 'NVARCHAR(50)')programs,
product.value('(sdfs[1])', 'NVARCHAR(500)')sdfs,
product.value('(addresses[1])', 'NVARCHAR(50)')addresses,
product.value('(otherIDs[1])', 'NVARCHAR(50)')otherIDs
FROM #InputXML.nodes('gwl/entities/entity') AS X(product)
You have a lot of different children here...
Just to show the principles:
DECLARE #xml XML=
N'<gwl>
<version>123456789</version>
<entities>
<entity id="1" version="123456789">
<name>xxxxx</name>
<listId>0</listId>
<listCode>Oxxx</listCode>
<entityType>08</entityType>
<createdDate>03/03/1993</createdDate>
<lastUpdateDate>05/06/2011</lastUpdateDate>
<source>src</source>
<OriginalSource>o_src</OriginalSource>
<aliases>
<alias category="STRONG" type="Alias">USCJSC</alias>
<alias category="WEAK" type="Alias">''OSKOAO''</alias>
</aliases>
<programs>
<program type="21">prog</program>
</programs>
<sdfs>
<sdf name="OriginalID">9876</sdf>
</sdfs>
<addresses>
<address>
<address1>1141, SYA-KAYA STR.</address1>
<country>RU</country>
<postalCode>1234</postalCode>
</address>
<address>
<address1>90, MARATA UL.</address1>
<country>RU</country>
<postalCode>1919</postalCode>
</address>
</addresses>
<otherIds>
<childId>737606</childId>
<childId>737607</childId>
</otherIds>
</entity>
</entities>
</gwl>';
-The query will fetch some values from several places.
--It should be easy to get the rest yourself...
SELECT #xml.value('(/gwl/version/text())[1]','bigint') AS [version]
,A.ent.value('(name/text())[1]','nvarchar(max)') AS [Entity_Name]
,A.ent.value('(listId/text())[1]','int') AS Entity_ListId
--more columns taken from A.ent
,B.als.value('#category','nvarchar(max)') AS Alias_Category
,B.als.value('text()[1]','nvarchar(max)') AS Alias_Content
--similar for programs and sdfs
,E.addr.value('(address1/text())[1]','nvarchar(max)') AS Address_Address1
,E.addr.value('(country/text())[1]','nvarchar(max)') AS Address_Country
--and so on
FROM #xml.nodes('/gwl/entities/entity') A(ent)
OUTER APPLY A.ent.nodes('aliases/alias') B(als)
OUTER APPLY A.ent.nodes('programs/program') C(prg)
OUTER APPLY A.ent.nodes('sdfs/sdf') D(sdfs)
OUTER APPLY A.ent.nodes('addresses/address') E(addr)
OUTER APPLY A.ent.nodes('otherIds/childId') F(ids);
The idea in short:
We read non-repeating values (e.g. version) from the xml variable directly
We use .nodes() to return repeating elements as derived sets.
We can use a cascade of .nodes() to dive deeper into repeating child elements by using a relativ Xpath (no / at the beginning).
You have two approaches:
Read the XML like above into a staging table (simply by adding INTO #tmpTable before FROM) and proceed from there (will need one SELECT ... GROUP BY for each type of child).
Create one SELECT per type of child, using only one of the APPLY lines and shift the data into specific child tables.
I would tend to the first one.
This allows to do some cleaning, generate IDs, check for business rules, before you shift this into the target tables.

t-sql to create XML file using FOR XML Path, multilevel issues

I am creating an XML file to Upload to a 3rd party product. The file must begin with specific file and source information level and then it is followed with the specific data requirements/levels of EVENT and CREW members for those events.
I can create the initial level with the file/source information, and I have the data requirements exactly as they should be, but I cannot get them together in the same file between the "ROOT" level without the initial level repeating between each EVENT level or the an extra EVENT level as if they're nested. I've also managed to get a result with a ROW level that I did not define and the "tags" modified to < and &gt: instead of < >. I've done a good bit of research and tried using a union method, sub-selects, nesting methods as well many combinations of FOR XML PATH, AUTO, EXPLICIT, with and without elements. I've learned a lot, but I'm just not finding the right combination for the results I need.
The first example is the layout that is required. The second is one of the examples that is most common for my efforts, followed by the SQL that created it.
what it should be (FILEINFO level only once, only one EVENT level for each EVENT)
<ROOT>
<FILEINFO>
<SOURCE_ID>P</SOURCE_ID>
</FILEINFO>
<EVENT>
<DATE>2019-09-24T08:00:00</DATE>
<NO>1</NO>
<DEL_FLAG>false</DEL_FLAG>
<DATE_TIME_STAMP>2019-09-24T14:14:00</DATE_TIME_STAMP>
<CREW>
<LAST_NAME>DOE</LAST_NAME>
<DEL_FLAG>false</DEL_FLAG>
<DATE_TIME_STAMP>2019-09-24T14:14:00</DATE_TIME_STAMP>
</CREW>
</EVENT>
<EVENT>
<DATE>2019-09-16T12:30:00</DATE>
<NO>1</NO>
<DEL_FLAG>false</DEL_FLAG>
<DATE_TIME_STAMP>2019-09-16T18:20:00</DATE_TIME_STAMP>
<CREW>
<LAST_NAME>DOE</LAST_NAME>
<DEL_FLAG>false</DEL_FLAG>
<DATE_TIME_STAMP>2019-09-16T18:20:00</DATE_TIME_STAMP>
</CREW>
</EVENT>
</ROOT>
what i'm getting:
<ROOT>
<EVENT>
<FILEINFO>
<SOURCE_ID>P</SOURCE_ID>
</FILEINFO>
<EVENT>
<DATE>2019-09-16T08:00:00</DATE>
<NO>1</NO>
<DEL_FLAG>false</DEL_FLAG>
<DATE_TIME_STAMP>2019-09-16T15:12:00</DATE_TIME_STAMP>
<CREW>
<LAST_NAME>DOE</LAST_NAME>
<DEL_FLAG>false</DEL_FLAG>
<DATE_TIME_STAMP>2019-09-16T15:12:00</DATE_TIME_STAMP>
</CREW>
</EVENT>
</EVENT>
<EVENT>
<FILEINFO>
<SOURCE_ID>P</SOURCE_ID>
</FILEINFO>
<EVENT>
<DATE>2019-09-16T08:00:00</DATE>
<NO>1</NO>
<DEL_FLAG>false</DEL_FLAG>
<DATE_TIME_STAMP>2019-09-16T15:12:00</DATE_TIME_STAMP>
<CREW>
<LAST_NAME>DOE</LAST_NAME>
<DEL_FLAG>false</DEL_FLAG>
<DATE_TIME_STAMP>2019-09-16T15:12:00</DATE_TIME_STAMP>
</CREW>
</EVENT>
</EVENT>
... ...
most recent/simplest attempt that creates the above:
SELECT
(SELECT SOURCE_ID FROM (select 'P' as SOURCE_ID) FILEINFO ) AS 'FILEINFO/SOURCE_ID'
,[DATE] AS 'EVENT/DATE'
,[NO] AS 'EVENT/NO'
,[DEL_FLAG] AS 'EVENT/DEL_FLAG'
,[DATE_TIME_STAMP] AS 'EVENT/DATE_TIME_STAMP'
,'DOE' as 'EVENT/CREW/LAST_NAME'
,[DEL_FLAG2] as 'EVENT/CREW/DEL_FLAG'
,[DATE_TIME_STAMP3] as 'EVENT/CREW/DATE_TIME_STAMP'
FROM [dbo].XMLForFILEExport x
FOR XML path('EVENT'), elements, ROOT('ROOT') ;
This is easy, just use a sub-select and deal with this like it was a *normal column:
This easy SELECT will return the single <FILEINFO>
SELECT 'P' AS [FILEINFO/SOURCE_ID]
FOR XML PATH(''),ROOT('ROOT');
You see, that I used an empty PATH(), but I set the ROOT().
This is the result
<ROOT>
<FILEINFO>
<SOURCE_ID>P</SOURCE_ID>
</FILEINFO>
</ROOT>
Now we can start to add your events. First I need a mockup table to simulate your issue
DECLARE #mockupEventTable TABLE(ID INT IDENTITY,[NO] INT, [DATE] DATETIME, EventText VARCHAR(100));
INSERT INTO #mockupEventTable VALUES(1,'20190916','Event 1')
,(2,'20190917','Event 2');
--The query
SELECT 'P' AS [FILEINFO/SOURCE_ID]
,(
SELECT e.[DATE]
,e.[NO]
,e.EventText
,'Doe' AS [CREW/LASTNAME]
FROM #mockupEventTable e
FOR XML PATH('EVENT'),TYPE
) AS [*]
FOR XML PATH(''),ROOT('ROOT');
The result
<ROOT>
<FILEINFO>
<SOURCE_ID>P</SOURCE_ID>
</FILEINFO>
<EVENT>
<DATE>2019-09-16T00:00:00</DATE>
<NO>1</NO>
<EventText>Event 1</EventText>
<CREW>
<LASTNAME>Doe</LASTNAME>
</CREW>
</EVENT>
<EVENT>
<DATE>2019-09-17T00:00:00</DATE>
<NO>2</NO>
<EventText>Event 2</EventText>
<CREW>
<LASTNAME>Doe</LASTNAME>
</CREW>
</EVENT>
</ROOT>
You can see, that the sub-select will create the inner XML just as you need it. We have to specify ,TYPE in order to get this as typed XML. Try the same without. You will get the XML escaped, as if it was simple text...
And I specify AS [*] (the same was AS [node()]) to indicate, that the XML "column" has no own name, but should be inserted as is. This is not mandatory (try it without), but it makes things more readable...
That's because you specified the PATH "EVENT" already. Also you can remove the EVENT in the field name, e.g. 'EVENT/CREW/DATE_TIME_STAMP' can just be 'CREW/DATE_TIME_STAMP'
TO achieve what required, you can generate the xml with EVENT elementsand then insert the FILEINFO.
DECLARE #x xml;
SET #x=(SELECT
[DATE] AS 'DATE'
,[NO] AS 'NO'
,[DEL_FLAG] AS 'DEL_FLAG'
,[DATE_TIME_STAMP] AS 'DATE_TIME_STAMP'
,'DOE' as 'CREW/LAST_NAME'
,[DEL_FLAG2] as 'CREW/DEL_FLAG'
,[DATE_TIME_STAMP3] as 'CREW/DATE_TIME_STAMP'
FROM [dbo].XMLForFILEExport x
FOR XML path('EVENT'), elements, ROOT('ROOT'))
SET #x.modify('
insert <FILEINFO><SOURCE_ID>P</SOURCE_ID></FILEINFO>
as first
into (/ROOT)[1]');

nested element FOR XML in SQL Server

I'm trying to create XML from SQL Server and I'm stuck with nested elements. I try different FOR XML parameters but still cannot get the correct results.
query looks like this:
SELECT
Product.ID, Product.ProductName,
(SELECT
Images.ProductImage AS image
FROM Images
WHERE Images.ProductID = Product.ID
FOR XML PATH ('image_list'), ELEMENTS, TYPE
)
FROM (SELECT DISTINCT ID, ProductName FROM Product) Product
FOR XML PATH ('products'), ELEMENTS, root ('Root')
and I want get this XML like this:
<Root>
<products>
<ID>1</ID>
<ProductName>product 1</ProductName>
<image_list>
<image>picture1.jpg</image>
<image>picture2.jpg</image>
<image>picture3.jpg</image>
</image_list>
</products>
<products>
<ID>2</ID>
<ProductName>product 2</ProductName>
<image_list>
<image>picture1.jpg</image>
<image>picture2.jpg</image>
</image_list>
</products>
</Root>
First part is OK, the image_list is a problem. Any advice?
I solved :) ..probably is good to post a qustion, after that brain starts to work :)
I add AS 'image_list' under subquery, remove ELEMENTS and PATH name.
And that's it.
SELECT
Product.ID, Product.ProductName,
(SELECT
Images.ProductImage AS image
FROM Images
WHERE Images.ProductID = Product.ID
FOR XML PATH (''), TYPE
) 'image_list'
FROM (SELECT DISTINCT ID, ProductName FROM Product) Product
FOR XML PATH ('products'), ELEMENTS, root ('Root')
Results is:
<Root>
<products>
<ID>1</ID>
<ProductName>product 1 </ProductName>
<image_list>
<image>picture1.jpg</image>
<image>picture2.jpg</image>
<image>picture3.jpg</image>
</image_list>
</products>
<products>
<ID>2</ID>
<ProductName>product 2 </ProductName>
<image_list>
<image>picture1.jpg</image>
<image>picture2.jpg</image>
</image_list>
</products>
</Root>

Call a procedure or function in Oracle DB with return = array of UDT from WSO2 DSS

I follow this post[1] as a guide to build an example of query an array of UDT in WSO2 DSS. In the post just query an UDT, my config try to query an UDT array.
I created this in my DB, a dummy PROCEDURE to try this:
create or replace
TYPE "LIST_CUSTOMERS" IS TABLE OF customer_t
CREATE OR REPLACE
PROCEDURE getCustomer2(listcust OUT list_customers) IS
cust customer_t;
cust2 customer_t;
BEGIN
listcust := list_customers();
cust := customer_t(1, 'prabath');
cust2 := customer_t(2, 'jorge');
listcust.extend;
listcust(1) := cust;
listcust.extend;
listcust(2) := cust2;
END;
My DS is this:
<?xml version="1.0" encoding="UTF-8"?>
<data name="UDTSample2">
<config id="default">
<property name="org.wso2.ws.dataservice.driver">oracle.jdbc.driver.OracleDriver</property>
<property name="org.wso2.ws.dataservice.protocol">jdbc:oracle:thin:#localhost:1521:DBMB</property>
<property name="org.wso2.ws.dataservice.user">****</property>
<property name="org.wso2.ws.dataservice.password">****</property>
</config>
<query id="q3" useConfig="default">
<sql>call getCustomer2(?)</sql>
<result element="customers">
<element name="customer" arrayName="custArray" column="cust" optional="true"/>
</result>
<param name="cust" paramType="ARRAY" sqlType="ARRAY" type="OUT" structType="LIST_CUSTOMERS" />
</query>
<operation name="op3">
<call-query href="q3" />
</operation>
</data>
ant return:
<customers xmlns="http://ws.wso2.org/dataservice">
<customer>{1,prabath}</customer>
<customer>{2,jorge}</customer>
</customers>
but I want something like this:
<customers xmlns="http://ws.wso2.org/dataservice">
<customer>
<id>1</id>
<name>prabath<name>
</customer>
<customer>
<id>2</id>
<name>Jorge<name>
</customer>
</customers>
How can I accomplish this?
[1] http://prabathabey.blogspot.com/2012/05/query-udtsuser-defined-types-with-wso2.html
Not sure whether this kind of transformation can be done at DSS level because DSS gives back what it recieves from database. Better use WSO2 esb for this kind of transformation.
By the moment it's not possible to accomplish this scenario using just DSS. the DS response must be send to the WSO2 ESB to do the corresponding transformation before send the response to the client. A JIRA was created to do this in the future https://wso2.org/jira/browse/DS-1104
As a workaround, you can use a procedure returning a sys_refcursor.
It would look like this:
PROCEDURE getCustomer_CUR(cur_cust OUT SYS_REFCURSOR)
l_cust LIST_CUSTOMERS;
IS
-- Retrieve cust list:
getCustomer2(l_cust);
OPEN cur_cust for
select cast(multiset(select * from TABLE(l_cust)) as customer_t) from dual;
...
END;
Then you can do your DSS mapping something like:
<sql>call getCustomer_CUR(?)</sql>
<result element="customers">
<element arrayName="custArray" name="Customers">
<element column="custArray[0]" name="col0" xsdType=.../>
...
</element>
</result>
<param name="cust" sqlType="ORACLE_REF_CURSOR" type="OUT"/>
It is tedious but it works.

Resources