Converting XML data to SQL table - sql-server

I have the following XML data with me. I need to convert this to SQL table.
<SalesDetails>
<Customer Name="Johny" DateofBirth="1990-01-02T00:00:00">
<OrderInfo>
<OrderDate>1993-02-03T00:00:00</OrderDate>
<OrderAmount>1000</OrderAmount>
</OrderInfo>
</Customer>
</SalesDetails>
Can anyone help me with a SQL query that gives the above XML file as output?
In my initial attempt, I have created two tables #TI and #T2. I had then inserted different values into it. I had then queried it as :
SELECT
(SELECT * FROM #T1 FOR XML RAW('Sales') , TYPE),
(SELECT * FROM #T2 FOR XML PATH('OrderInfo') , TYPE)
FOR XML PATH('') , ROOT('SalesDetails')
But I need the output in the first XML format based on SQL tables and corresponding joins. That is, when the name of a customer is displayed, his corresponding order information needs to be displayed. I do not want it in a grouped format.

Sorry, in my first attempt I completely misread your question and thought you'd like to get the data out of your XML. This is the approach to create such an XML out of table's data:
DECLARE #cust TABLE(ID INT, CustomerName VARCHAR(100),DateOfBirth DATE);
INSERT INTO #cust VALUES(1,'Jonny','1990-01-02T00:00:00')
,(2,'Jimmy','1980-01-02T00:00:00');
DECLARE #ord TABLE(ID INT,CustomerID INT,OrderDate DATE, OrderAmount INT);
INSERT INTO #ord VALUES(1,1,'1993-02-03T00:00:00',1000)
,(2,1,'1994-02-03T00:00:00',500)
,(3,2,'1994-02-03T00:00:00',200);
SELECT c.CustomerName AS [#Name]
,c.DateOfBirth AS [#DateofBirth]
,(
SELECT o.OrderDate
,o.OrderAmount
FROM #ord AS o
WHERE o.CustomerID=c.ID
FOR XML PATH('OrderInfo'),TYPE
)
FROM #cust AS c
FOR XML PATH('Customer'),ROOT('SalesDetails')
And this is the created XML
<SalesDetails>
<Customer Name="Jonny" DateofBirth="1990-01-02">
<OrderInfo>
<OrderDate>1993-02-03</OrderDate>
<OrderAmount>1000</OrderAmount>
</OrderInfo>
<OrderInfo>
<OrderDate>1994-02-03</OrderDate>
<OrderAmount>500</OrderAmount>
</OrderInfo>
</Customer>
<Customer Name="Jimmy" DateofBirth="1980-01-02">
<OrderInfo>
<OrderDate>1994-02-03</OrderDate>
<OrderAmount>200</OrderAmount>
</OrderInfo>
</Customer>
</SalesDetails>
Just for the case you want to read your XML, I let this appended
You can retrieve all the information like this:
The generated Index columns are IDs you can use to insert this into relational tables. The problem with your XML is, that the information about your target tabels is missing. But the rest should be easy for you.
Btw: I declared some more similar nodes to make the relational structure visible
DECLARE #x XML=
'<SalesDetails>
<Customer Name="Johny" DateofBirth="1990-01-02T00:00:00">
<OrderInfo>
<OrderDate>1993-02-03T00:00:00</OrderDate>
<OrderAmount>1000</OrderAmount>
</OrderInfo>
<OrderInfo>
<OrderDate>1994-02-03T00:00:00</OrderDate>
<OrderAmount>500</OrderAmount>
</OrderInfo>
</Customer>
<Customer Name="Jimmy" DateofBirth="1980-01-02T00:00:00">
<OrderInfo>
<OrderDate>1994-02-03T00:00:00</OrderDate>
<OrderAmount>200</OrderAmount>
</OrderInfo>
<OrderInfo>
<OrderDate>1993-02-03T00:00:00</OrderDate>
<OrderAmount>100</OrderAmount>
</OrderInfo>
</Customer>
</SalesDetails>';
WITH CustomerNodes AS
(
SELECT ROW_NUMBER() OVER(ORDER BY (SELECT NULL)) AS CustomerIndex
,Customer.value('#Name','varchar(max)') AS CustomerName
,Customer.value('#DateofBirth','date') AS CustomerDateOfBirth
,One.Customer.query('.') AS CustomerNode
FROM #x.nodes('SalesDetails/Customer') AS One(Customer)
)
SELECT cn.*
,ROW_NUMBER() OVER(PARTITION BY cn.CustomerIndex ORDER BY (SELECT NULL)) AS OrderIndex
,OrderInfo.value('OrderDate[1]','date') AS OrderDate
,OrderInfo.value('OrderAmount[1]','int') AS OrderAmount
FROM CustomerNodes AS cn
CROSS APPLY cn.CustomerNode.nodes('Customer/OrderInfo') As The(OrderInfo)
The result:
Customer Order
ID Name DateOfBirth ID OrderDate OrderAmount
1 Johny 1990-01-02 1 1993-02-03 1000
1 Johny 1990-01-02 2 1994-02-03 500
2 Jimmy 1980-01-02 1 1994-02-03 200
2 Jimmy 1980-01-02 2 1993-02-03 100

Related

Split XML field into multiple delimited values - SQL

I have some XML content in a single field; I want to split each xml field in multiple rows.
The XML is something like that:
<env>
<id>id1<\id>
<DailyProperties>
<date>01/01/2022<\date>
<value>1<\value>
<\DailyProperties>
<DailyProperties>
<date>05/05/2022<\date>
<value>2<\value>
<\DailyProperties>
<\env>
I want to put everything in a table as:
ID DATE VALUE
id1 01/01/2022 1
id1 05/05/2022 2
For now I managed to parse the xml value, and I have found something online to get a string into multiple rows (like this), but my string should have some kind of delimiter. I did this:
SELECT
ID,
XMLDATA.X.query('/env/DailyProperties/date').value('.', 'varchar(100)') as r_date,
XMLDATA.X.query('/env/DailyProperties/value').value('.', 'varchar(100)') as r_value
from tableX
outer apply xmlData.nodes('.') as XMLDATA(X)
WHERE ID = 'id1'
but I get all values without a delimiter, as such:
01/10/202202/10/202203/10/202204/10/202205/10/202206/10/202207/10/202208/10/202209/10/202210/10/2022
Or, as in my example:
ID R_DATE R_VALUE
id01 01/01/202205/05/2022 12
I have found out that XQuery has a last() function that return the last value parsed; in my xml example it will return only 05/05/2022, so it should exists something for address the adding of a delimiter. The number of rows could vary, as it could vary the number of days of which I have a value.
Please try the following solution.
I had to fix your XML to make it well-formed.
SQL
DECLARE #tbl TABLE (id INT IDENTITY PRIMARY KEY, xmldata XML);
INSERT INTO #tbl (xmldata) VALUES
(N'<env>
<id>id1</id>
<DailyProperties>
<date>01/01/2022</date>
<value>1</value>
</DailyProperties>
<DailyProperties>
<date>05/05/2022</date>
<value>2</value>
</DailyProperties>
</env>');
SELECT p.value('(id/text())[1]','VARCHAR(20)') AS id
, c.value('(date/text())[1]','VARCHAR(10)') AS [date]
, c.value('(value/text())[1]','INT') AS [value]
FROM #tbl
CROSS APPLY xmldata.nodes('/env') AS t1(p)
OUTER APPLY t1.p.nodes('DailyProperties') AS t2(c);
Output
id
date
value
id1
01/01/2022
1
id1
05/05/2022
2
Yitzhak beat me to it by 2 min. Nonetheless, here's what I have:
--==== XML Data:
DECLARE #xml XML =
'<env>
<id>id1</id>
<DailyProperties>
<date>01/01/2022</date>
<value>1</value>
</DailyProperties>
<DailyProperties>
<date>05/05/2022</date>
<value>2</value>
</DailyProperties>
</env>';
--==== Solution:
SELECT
ID = ff2.xx.value('(text())[1]','varchar(20)'),
[Date] = ff.xx.value('(date/text())[1]', 'date'),
[Value] = ff.xx.value('(value/text())[1]', 'int')
FROM (VALUES(#xml)) AS f(X)
CROSS APPLY f.X.nodes('env/DailyProperties') AS ff(xx)
CROSS APPLY f.X.nodes('env/id') AS ff2(xx);
Returns:
ID Date Value
-------------------- ---------- -----------
id1 2022-01-01 1
id1 2022-05-05 2

Substitute for STRING_AGG pre SQL Server 2016

I need to group a table by a set of values together with all matching row numbers/id:s for each set. This operation must be done within the boundaries of SQL Server 2016.
Let's suppose I have the following table (Places):
ID
Country
City
1
Sweden
Stockholm
2
Norway
Oslo
3
Iceland
Reykjavik
4
Sweden
Stockholm
The result that I'm after (No curly-brackets because Stack Overflow thinks it's code, preventing me from posting):
ID
Json
1,4
"Country":"Sweden","City":"Stockholm"
2
"Country":"Norway ","City":"Oslo"
3
"Country":"Iceland ","City":"Reykjavik"
In SQL Server 2017 the above result can be achieved with:
SELECT STRING_AGG(ID) ID, (SELECT Country, City FOR JSON PATH) Json
FROM Places GROUP BY Country, City
I managed to get a similar result in SQL Server 2016 with the code below. (But with my actual amount of data and columns, this solution is too slow.)
SELECT DISTINCT Country, City INTO #temp FROM Places
SELECT (SELECT ID From Places WHERE Country = P.Country AND City = P.City FOR JSON PATH) ID,
(SELECT Country, City FOR JSON Path) Json FROM #temp P
Is there any more performance-effective way of achieving the result that I'm after?
EDIT: As people suggested me to try "FOR XML Path" I tried the code below. This gives the following error "Places.ID is invalid in the select list because it is not contained in either an aggregate function or the GROUP BY clause ":
SELECT stuff((select ',' + cast(ID as varchar(max)) for xml path ('')), 1, 1, '') ID,
(SELECT Country, City FOR JSON PATH) Json
FROM Places GROUP BY Country, City
Here's a solution you can try with for xml path
Basically select and group the json columns needed and using an apply, use the for xml path solution to aggregate the correlated ID values; because the outer query needs to refer to the output of the apply it needs to be aggregated also, I chose to use max
select max(x.Ids), (select country,city for json path) as [Json]
from t
outer apply (
select Stuff((select ',' + Convert(varchar(10),t2.Id)
from t t2
where t2.city=t.city and t2.country=t.country
for xml path(''),type).value('(./text())[1]','varchar(10)'),1,1,'') as Ids
)x
group by country,city
Working Fiddle
Here is another possible solution:
Declare #testTable Table (ID int, Country varchar(30), City varchar(30));
Insert Into #testTable (ID, Country, City)
Values (1, 'Sweden', 'Stockholm')
, (2, 'Normway', 'Oslo')
, (3, 'Iceland', 'Reykjavik')
, (4, 'Sweden', 'Stockholm');
Select Distinct
ID = stuff((Select concat(',', tt2.ID)
From #testTable tt2
Where tt2.City = tt.City
And tt2.Country = tt.Country
For xml path (''), Type).value('.', 'varchar(10)'), 1, 1, '')
, json = (Select Country, City For JSON PATH)
From #testTable tt;
No idea if this will perform any better though. It is essentially the same - just using DISTINCT instead of GROUP BY.

Executing a SELECT statement on a cast(text) column that has XML

I am trying to retrieve all values in the XML that contains defined values in the WHERE clause but I am only retrieving the first record and not the subsequent records in the IN operator. I am needing to the CAST a text column to XML and then retrieve the records but I am not able to make this work. Any help/direction would be appreciated.
Here is the XML:
<Payment>
<CoverageCd>COLL</CoverageCd>
<LossTypeCd>COLL</LossTypeCd>
<ClaimStatusCd>C</ClaimStatusCd>
<LossPaymentAmt>14596</LossPaymentAmt>
</Payment>
<Payment>
<CoverageCd>LIAB</CoverageCd>
<LossTypeCd>PD</LossTypeCd>
<ClaimStatusCd>C</ClaimStatusCd>
<LossPaymentAmt>3480</LossPaymentAmt>
</Payment>
Here is my SQL code:
SELECT
ad.AplusDataSysID,
CAST(ad.xmlAplus AS XML).value('(/ISO/PassportSvcRs/Reports/Report/ReportData/ISO/PassportSvcRs/PassportInqRs/Match/Claim/Payment/LossTypeCd)[1]','varchar(max)') AS LossTypeCode
FROM
[dbo].[AUT_Policy] p
INNER JOIN
[dbo].[IP_Policy] ip ON p.PolicySysID = ip.Aut_PolicyID
INNER JOIN
[dbo].[AUT_AplusData] ad ON ip.PolicySysID = ad.PolicySysID
WHERE
CAST(ad.xmlAplus AS XML).value('(/ISO/PassportSvcRs/Reports/Report/ReportData/ISO/PassportSvcRs/PassportInqRs/Match/Claim/Payment/LossTypeCd)[1]', 'VARCHAR(MAX)') IN ('BI','PD','COLL','COMP','PIP','UM','MEDPY','TOWL','RENT','OTHR');
Here is my SQL result:
Here is what the SQL result should look like:
It would appear that the XML nodes method is what you need.
-- Sample data
DECLARE #AUT_AplusData TABLE (AplusDataSysID INT, xmlAplus TEXT);
INSERT #AUT_AplusData VALUES (1,
'<Payment>
<CoverageCd>COLL</CoverageCd>
<LossTypeCd>COLL</LossTypeCd>
<ClaimStatusCd>C</ClaimStatusCd>
<LossPaymentAmt>14596</LossPaymentAmt>
</Payment>
<Payment>
<CoverageCd>LIAB</CoverageCd>
<LossTypeCd>PD</LossTypeCd>
<ClaimStatusCd>C</ClaimStatusCd>
<LossPaymentAmt>3480</LossPaymentAmt>
</Payment>');
-- Solution
SELECT
AplusDataSysID = ad.AplusDataSysID,
LossTypeCd = pay.loss.value('(LossTypeCd/text())[1]', 'varchar(8000)')
FROM #AUT_AplusData AS ad
CROSS APPLY (VALUES(CAST(ad.xmlAplus AS XML))) AS x(xmlAplus)
CROSS APPLY x.xmlAplus.nodes('/Payment') AS pay(loss);
Returns:
AplusDataSysID LossTypeCd
---------------- ---------------
1 COLL
1 PD

Reading xml data in SQL

I am reading data from xml in Sql.
Here is my xml :
Declare #MainXml XML =
'<?xml version="1.0" encoding="utf-8"?>
<result>
<details>
<admin>
<code>555</code>
</admin>
<claimhistory>
<claim id="1" number="100">
<account>Closed</account>
</claim>
<claim id="2" number="200">
<account>Closed</account>
</claim>
</claimhistory>
</details>
</result>'
Reading data like this:
select
C.X.value('(admin/code)[1]', 'varchar(max)') as Code,
A.X.value('#id', 'varchar(max)') as Id,
A.X.value('#number', 'varchar(max)') as No,
A.X.value('(account)[1]', 'varchar(max)') as Status
from
#MainXml.nodes('result/details') as C(X)
cross apply
C.X.nodes('claimhistory/claim') as A(X)
This is returning:
Code Id No Status
---------------------
555 1 100 Closed
555 2 200 Closed
Stored procedure contains above code.
Here datatable variable is used as an input for Stored Procedure. It contains id and name.
Declare #dtValue As [dbo].[DataTableDetails]
Insert Into #dtValue(Requested_Id, Name) Values(1, 'Tim');
Insert Into #dtValue(Requested_Id, Name) Values(2, 'Joe');
I want to add these names to select query based on matching Id of an xml to input.
Expected output -
Code Id No Status Name
----------------------------
555 1 100 Closed Tim
555 2 200 Closed Joe
Currently - after inserting the selected records from xml, I am using update query But table contains over a million records so it is effecting performance now.
Please suggest me.
Edited:
Tried with Join - [added below line in select query]
Select
C.X.value('(admin/code)[1]', 'varchar(max)') as Code,
A.X.value('#id', 'varchar(max)') as Id,
A.X.value('#number', 'varchar(max)') as No,
A.X.value('(account)[1]', 'varchar(max)') as Status,
CA.Name
from
#MainXml.nodes('result/details') as C(X)
cross apply
C.X.nodes('claimhistory/claim') as A(X)
join
#dtValue CA on CA.Requested_Id = A.X.value('#id', 'varchar(max)')
I'd recommend refactoring the way you're selecting from the XML like so:
select
C.X.value('(../../admin/code)[1]', 'varchar(max)') as Code,
C.X.value('#id', 'varchar(max)') as Id,
C.X.value('#number', 'varchar(max)') as No,
C.X.value('(account)[1]', 'varchar(max)') as Status,
dt.Name
from
#MainXml.nodes('result/details/claimhistory/claim') as C(X)
INNER JOIN #dtValue dt
ON dt.Requested_Id = C.X.value('(#id)[1]', 'int')
You don't actually want to CROSS APPLY the child nodes, you want them to be the primary part you're selecting from (i.e. one row per claim element) - it's then easy enough to select based on the grandparent node to get the Code value, and then you can properly INNER JOIN your table variable.
Full sample:
Declare #MainXml XML =
'<?xml version="1.0" encoding="utf-8"?>
<result>
<details>
<admin>
<code>555</code>
</admin>
<claimhistory>
<claim id="1" number="100">
<account>Closed</account>
</claim>
<claim id="2" number="200">
<account>Closed</account>
</claim>
</claimhistory>
</details>
</result>'
DECLARE #dtValue TABLE (Requested_Id int, Name varchar(10))
Insert Into #dtValue(Requested_Id, Name) Values(1, 'Tim'), (2, 'Joe');
select
C.X.value('(../../admin/code)[1]', 'varchar(max)') as Code,
C.X.value('#id', 'varchar(max)') as Id,
C.X.value('#number', 'varchar(max)') as No,
C.X.value('(account)[1]', 'varchar(max)') as Status,
dt.Name
from
#MainXml.nodes('result/details/claimhistory/claim') as C(X)
INNER JOIN #dtValue dt
ON dt.Requested_Id = C.X.value('(#id)[1]', 'int')

Parsing XML content with absent elements in SQL Server 2012

I need to parse XML below into table of transactions for the customers for each day. The XML file is coming from external service which is not controlled by me.
The problem is when a customer doesn't have a transaction for the day, I am not able to view it in my table. How do I get to see that customer had zero transactions?
declare #xml xml =
'<root>
<customers>
<customer id="777">
<orders>
<order currency="USD" id="888" date="2014-06-18">
<transactions>
<transaction id="998">
<date>2014-08-01</date>
<itemid>10001</itemid>
<amount>745.96</amount>
</transaction>
</transactions>
</order>
</orders>
</customer>
<customer id="778">
<orders>
<order id="999" />
</orders>
</customer>
</customers>
</root>'
My transformation query is like this:
select
newid() ID,
ltrim(rtrim(B.C.value('#id', 'nvarchar(50)'))) CUSTOMER_ID,
ltrim(rtrim(K.C.value('#id', 'nvarchar(450)'))) ACCOUNT_ID,
ltrim(rtrim(K.C.value('#date', 'datetime'))) DATE_PLACED,
ltrim(rtrim(K.C.value('#currency', 'nvarchar(50)'))) CURRENCY,
ltrim(rtrim(T.C.value('#id', 'nvarchar(50)'))) TRANSACTION_ID,
ltrim(rtrim(T.C.value('date[1]', 'datetime'))) TRANSACTION_DATE,
ltrim(rtrim(T.C.value('itemid[1]', 'nvarchar(50)'))) TRANSACTION_ITEMID,
ltrim(rtrim(T.C.value('amount[1]', 'money'))) TRANSACTION_BANK_CODE
from
#xml.nodes('/root/customers/customer') as B(C)
outer apply B.C.nodes('/root/customers/customer/orders/order') as K(C)
outer apply K.C.nodes('/root/customers/customer/orders/order/transactions/transaction') as T(C)
where
ltrim(rtrim(b.c.value('#id', 'nvarchar(50)'))) = ltrim(rtrim(k.c.value('../../#id', 'nvarchar(50)')))
and
(
(
t.c.value('../../#id','nvarchar(50)') is not null
and
ltrim(rtrim(k.c.value('#id','nvarchar(50)'))) = ltrim(rtrim(t.c.value('../../#id','nvarchar(50)')))
)
or (ltrim(rtrim(t.c.value('../../#id','nvarchar(50)'))) is null)
)
Thank you in advance!
You should not do the cross apply against the full xpath from root. Begin with where you are instead and remove the where clause.
select newid() as ID,
B.C.value('#id', 'nvarchar(50)') as CUSTOMER_ID,
K.C.value('#id', 'varchar(50)') as ACCOUNT_ID,
T.C.value('#id', 'nvarchar(50)') as TRANSACTION_ID
from #xml.nodes('/root/customers/customer') as B(C)
outer apply B.C.nodes('orders/order') as K(C)
outer apply K.C.nodes('transactions/transaction') as T(C)
Result
ID CUSTOMER_ID ACCOUNT_ID TRANSACTION_ID
------------------------------------ ----------- ---------- --------------
767FCA17-578A-495E-9EFA-75E3509B2BD2 777 888 998
59965290-EB7C-429B-AA5F-97EED0EB35BD 778 999 NULL

Resources