I have here a small extract of the XML files I will have to handle:
<garRoot fileMaster="9371034.0582.30582">
<garTransactions>
<garTransaction InnerTransId="89274503">
<garSection>
<garSectionCounterFName />
<garColumns />
<garSection>
<garSectionName>Header Section</ChapterName>
<garSectionCounterFName />
<garColumns />
<garSection>
<garSectionName>Startup</ChapterName>
<garSectionCounterFName />
<garColumns>
<garColumn>
<garColText>Idea Date:</garColText>
<garColVal>2017-03-22</garColVal>
</garColumn>
<garColumn>
<garColText>Idea Name:</garColText>
<garColVal>The Invisible Cloak</garColVal>
</garColumn>
</garColumns>
</garSection>
I have tried some code to attempt to:
Start with getting the InnerTransId values for each garTransaction:
SELECT
T.value('./#InnerTransID','varchar(50)') As InnerTransID
FROM #XML.nodes('//garTransaction') AS GarT(T)
Because in reality there have been nested garSection in other garSection I've tried to get all garColText and garColVal via garColumn:
SELECT
C.query('./garColText') As cText
, C.query('./garColVal') As cVal
FROM #XML.nodes('//garColumn') as garC(C)
Where I'm having trouble is, as an example I know I have 145 columns for each transaction id but i cannot seem to link the data together, as I would need to have returned:
InnerTransId cText cVal
--------------- ----------- -------------------
89274503 Idea Date: 2017-03-22
89274503 Idea Name: The Invisible Cloak
You can use CROSS APPLY or OUTER APPLY for that purpose :
SELECT
T.value('#InnerTransID','varchar(50)') As InnerTransID
, C.value('garColText[1]','varchar(max)') As cText
, C.value('garColVal[1]','varchar(max)') As cVal
FROM #XML.nodes('//garTransaction') AS GarT(T)
OUTER APPLY T.nodes('.//garColumn') as garC(C)
Notice how nodes('.//garColumn') was called on T, so that the result, garC(C), only contains garColumn that related to current garTransaction.
Related
I have XML data that comes in from a third party (so I cannot change the XML format) and gets stored in a table.
An example XML message format
<MedicalAidMessage xmlns="test.co.za/messaging" version="6.0">
<BenefitCheckResponseMessage>
<FinancialResponseDetails>
<FinancialResponseLines>
<FinancialResponseLine>
<LineIdentifier>1</LineIdentifier>
<RequestedAmount>1000</RequestedAmount>
<AmountToProvider>800</AmountToProvider>
<AmountToMember>0</AmountToMember>
<MemberLiableAmount>200</MemberLiableAmount>
<MemberNotLiableAmount>0</MemberNotLiableAmount>
<TariffCode>12345</TariffCode>
<LineResponseCodes>
<LineResponseCode>
<Sequence>1</Sequence>
<Code>274</Code>
<Description>We have not paid the amount claimed because the funds in the Medical Savings are used up.</Description>
<Type>Info</Type>
</LineResponseCode>
<LineResponseCode>
<Sequence>2</Sequence>
<Code>1239</Code>
<Description>We have applied a co-payment on this claim, in line with the member’s plan benefit for MRI and CT scans.</Description>
<Type>Info</Type>
</LineResponseCode>
</LineResponseCodes>
</FinancialResponseLine>
</FinancialResponseLines>
</FinancialResponseDetails>
</BenefitCheckResponseMessage>
</MedicalAidMessage>
I have tried the following code to get the Tariffcode, RequestedAmount, AmountToProvider and AmountToMember
DECLARE #XMLData XML = (select top 1 replace(br.xmlmessage,'<?xml version="1.0" encoding="UTF-8"?>','')
from benefitcheckresponse br
inner join benefitcheck bc on bc.id = br.BenefitCheckId
where bc.id =1562
order by bc.id desc)
SELECT
[TariffCode] = Node.Data.value('TariffCode', 'vacrhar(50)'),
[RequestedAmount] = Node.Data.value('RequestedAmount', 'float)'),
[AmountToProvider] = Node.Data.value('AmountToProvider', 'float)'),
[AmountToMember] = Node.Data.value('AmountToMember', 'float)')
FROM #XMLData.nodes('/*/FinancialResponseLine/') Node(Data)
The problem I am getting is that it is giving me the following error message
Msg 9341, Level 16, State 1, Line 12 XQuery [nodes()]: Syntax error
near '', expected a step expression.
How do I resolve that error?
How would I include the results for multiple lines when there multiple line responses?
How would I include the values from the sub nodes to the line response nodes?
You would be better off with syntax like this, if I am honest. I wasn't able to complete the ORDER BY clause for you, as I don't know what column to order by, but as you have a TOP (1) you need one. You also need to define your XML namespace, which you have not, which would have resulted in no rows being returned:
WITH XMLNAMESPACES(DEFAULT 'test.co.za/messaging')
SELECT TOP (1)
FRL.FRL.value('(TariffCode/text())[1]','int') AS TariffCode,
FRL.FRL.value('(RequestedAmount/text())[1]','int') AS RequestedAmount, --I doubt float is the correct data type here
FRL.FRL.value('(AmountToProvider/text())[1]','int') AS AmountToProvider, --I doubt float is the correct data type here
FRL.FRL.value('(AmountToMember/text())[1]','int') AS AmountToMember --I doubt float is the correct data type here
FROM dbo.benefitcheckresponse br
INNER JOIN benefitcheck bc ON bc.id = br.BenefitCheckId
CROSS APPLY br.xmlmessage.nodes('MedicalAidMessage/BenefitCheckResponseMessage/FinancialResponseDetails/FinancialResponseLines/FinancialResponseLine') FRL(FRL)
WHERE bc.id = 1562
ORDER BY {Column Name(s)};
I have the following XML:
<Envelope format="ProceedoOrderTransaction2.1">
<Sender>SENDER</Sender>
<Receiver>RECEIVER</Receiver>
<EnvelopeID>xxxxx</EnvelopeID>
<Date>2021-05-06</Date>
<Time>11:59:46</Time>
<NumberOfOrder>1</NumberOfOrder>
<Order>
<Header>
<OrderNumber>POXXXXX</OrderNumber>
</Header>
<Lines>
<Line>
<LineNumber>1</LineNumber>
<ItemName>Ipsum Lorum</ItemName>
<SupplierArticleNumber>999999</SupplierArticleNumber>
<UnitPrice vatRate="25.0">50</UnitPrice>
<UnitPriceBasis>1</UnitPriceBasis>
<OrderedQuantity unit="Styck">200</OrderedQuantity>
<AdditionalItemProperty Key="ARTIKELNUMMER" Description="Unik ordermärkning (artikelnummer):" />
<Value>999999</Value>
<AdditionalItemProperty Key="BESKRIVNING" Description="Kort beskrivning:" />
<Value>Ipsum Lorum</Value>
<AdditionalItemProperty Key="BSKRIVNING" Description="Beskrivning:" />
<Value>Ipsum Lorum</Value>
<AdditionalItemProperty Key="ENHET" Description="Enhet:" />
<Value>Styck</Value>
<AdditionalItemProperty Key="KVANTITET" Description="Kvantitet:" />
<Value>200</Value>
<AdditionalItemProperty Key="PRIS" Description="Pris/Enhet (ex. moms):" />
<Value>50</Value>
<AdditionalItemProperty Key="VALUTA" Description="Valuta:" />
<Value>SEK</Value>
<Accounting>
<AccountingLine amount="10000">
<AccountingValue dimensionPosition="001" dimensionExternalID="ACCOUNT">xxx</AccountingValue>
<AccountingValue dimensionPosition="002" dimensionExternalID="F1">Ipsum Lorum</AccountingValue>
<AccountingValue dimensionPosition="005" dimensionExternalID="F3">1</AccountingValue>
<AccountingValue dimensionPosition="010" dimensionExternalID="F2">9999</AccountingValue>
</AccountingLine>
</Accounting>
</Line>
</Lines>
</Order>
</Envelope>
I am able to parse out all values correctly to table structure except for 1 value in a way that ensures its it associated with its tag. So where I stumble is that I am correctly getting 1 row per AdditionalItemProperty and I am able to get the Key and Description tag values, for example BESKRIVNING and Kort beskrivning:, but I can't (in a reasonable way) get the value between <Value> </Value> brackets that is also associated with each tag value. So for tag key value BESKRIVNING the associated value is 99999 which seem to be on same hierarchy level (insane I know) as the AdditionalItemProperty it is associated with. Seems like they use logic that value for a AdditionalItemProperty will be following the AdditionalItemProperty tag.
I am using SQL Server 2019. I have gotten this far:
-- Purchaseorderrowattributes
select top(10)
i.value(N'(./Header/OrderNumber/text())[1]', 'nvarchar(30)') as OrderNumber,
ap.value(N'(../LineNumber/text())[1]', 'nvarchar(30)') as LineNumber,
ap.value(N'(#Description)', 'nvarchar(50)') property_description
from
load.proceedo_orders t
outer apply
t.xml_data.nodes('Envelope/Order') AS i(i)
outer apply
i.nodes('Lines/Line/AdditionalItemProperty') as ap(ap)
where
file_name = #filename
Which produces the following output:
OrderNumber LineNumber property_description
--------------------------------------------
PO170006416 1 Antal timmar
PO170006416 1 Beskrivning
PO170006416 1 Kompetensområde
PO170006416 1 Roll
PO170006416 1 Ordernummer
PO170006416 1 Timpris
I can't find a way to add the value for each property in a correct way. Since the ordering of the values will always be same as the ordering of the AdditionalItemProperty i found solution to get ordering of the AdditionalItemProperty and then i could use rownumber and i was then hoping to input the rownumber value into the bracket in
ap.value(N'(../Value/text())[1]', 'nvarchar(50)') property_description
but SQL Server throws exception that it has to be string literal.
So to be clear what I tried doing with something like:
ap.value(CONCAT( N'(../Value/text())[', CAST(ROWNUMBER as varchar) ,']'), 'nvarchar(50)') property_description
SQL Server uses XQuery 1.0.
You can make use of the >> Node Comparison operator to find the Value sibling node with code similar to the following:
-- Purchaseorderrowattributes
select top(10)
i.value(N'(./Header/OrderNumber/text())[1]', 'nvarchar(30)') as OrderNumber
,ap.value(N'(../LineNumber/text())[1]', 'nvarchar(30)') as LineNumber
,ap.value(N'(#Description)', 'nvarchar(50)') property_description
,ap.query('
let $property := .
return (../Value[. >> $property][1])/text()
').value('.[1]', 'varchar(20)') as property_value
from load.proceedo_orders t
outer apply t.xml_data.nodes('Envelope/Order') AS i(i)
outer apply i.nodes('Lines/Line/AdditionalItemProperty') as ap(ap)
where file_name = #filename
So what's going on here?
let $property := . is creating a reference to the current AdditionalItemProperty node.
../Value[. >> $property] is ascending to the parent node, Line, and descending again to find Value nodes after the AditionalItemProperty node reference in document order, with [1] selecting the first one of those nodes.
See the 3.5.3 Node Comparisons section for a little more detail.
I have a column (text) in my Postgres DB (v.10) with a JSON format.
As far as i now it's has an array format.
Here is an fiddle example: Fiddle
If table1 = persons and change_type = create then i only want to return the name and firstname concatenated as one field and clear the rest of the text.
Output should be like this:
id table1 did execution_date change_type attr context_data
1 Persons 1 2021-01-01 Create Name [["+","name","Leon Bill"]]
1 Persons 2 2021-01-01 Update Firt_name [["+","cur_nr","12345"],["+","art_cd","1"],["+","name","Leon"],["+","versand_art",null],["+","email",null],["+","firstname","Bill"],["+","code_cd",null]]
1 Users 3 2021-01-01 Create Street [["+","cur_nr","12345"],["+","art_cd","1"],["+","name","Leon"],["+","versand_art",null],["+","email",null],["+","firstname","Bill"],["+","code_cd",null]]
Disassemble json array into SETOF using json_array_elements function, then assemble it back into structure you want.
select m.*
, case
when m.table1 = 'Persons' and m.change_type = 'Create'
then (
select '[["+","name",' || to_json(string_agg(a.value->>2,' ' order by a.value->>1 desc))::text || ']]'
from json_array_elements(m.context_data::json) a
where a.value->>1 in ('name','firstname')
)
else m.context_data
end as context_data
from mutations m
modified fiddle
(Note:
utilization of alphabetical ordering of names of required fields is little bit dirty, explicit order by case could improve readability
resulting json is assembled from string literals as much as possible since you didn't specified if "+" should be taken from any of original array elements
the to_json()::text is just for safety against injection
)
I'm trying to retrieve the health value from Snowflake semi structured data in a variant column called extra from table X.
An example of the code can be seen below:
[
{
"party":
"[{\"class\":\"Farmer\",\"gender\":\"Female\",\"ethnicity\":\"NativeAmerican\",\"health\":2},
{\"class\":\"Adventurer\",\"gender\":\"Male\",\"ethnicity\":\"White\",\"health\":3},
{\"class\":\"Farmer\",\"gender\":\"Male\",\"ethnicity\":\"White\",\"health\":0},
{\"class\":\"Banker\",\"gender\":\"Female\",\"ethnicity\":\"White\",\"health\":0}
}
]
I have tried reading the Snowflake documentation from https://community.snowflake.com/s/article/querying-semi-structured-data
I have also tried the following queries to flatten the query:
SELECT result.value:health AS PartyHealth
FROM X
WHERE value = 'Trail'
AND name = 'Completed'
AND PartyHealth > 0,
TABLE(FLATTEN(X, 'party')) result
AND
SELECT [0]['party'][0]['health'] AS Health
FROM X
WHERE value = 'Trail'
AND name = 'Completed'
AND PH > 0;
I am trying to retrieve the health value from table X from column extra which contains the the variant party, which has 4 repeating values [0-3]. Im not sure how to do this is someone able to tell me how to query semi structured data in Snowflake, considering the documentation doesn't make much sense?
First, the JSON value you posted seems wrong formatted (might be a copy paste issue).
Here's an example that works:
first your JSON formatted:
[{ "party": [ {"class":"Farmer","gender":"Female","ethnicity":"NativeAmerican","health":2}, {"class":"Adventurer","gender":"Male","ethnicity":"White","health":3}, {"class":"Farmer","gender":"Male","ethnicity":"White","health":0}, {"class":"Banker","gender":"Female","ethnicity":"White","health":0} ] }]
create a table to test:
CREATE OR REPLACE TABLE myvariant (v variant);
insert the JSON value into this table:
INSERT INTO myvariant SELECT PARSE_JSON('[{ "party": [ {"class":"Farmer","gender":"Female","ethnicity":"NativeAmerican","health":2}, {"class":"Adventurer","gender":"Male","ethnicity":"White","health":3}, {"class":"Farmer","gender":"Male","ethnicity":"White","health":0}, {"class":"Banker","gender":"Female","ethnicity":"White","health":0} ] }]');
now, to select a value you start from column name, in my case v, and as your JSON is an array inside, I specify first value [0], and from there expand, so something like this:
SELECT v[0]:party[0].health FROM myvariant;
Above gives me:
For the other rows you can simply do:
SELECT v[0]:party[1].health FROM myvariant;
SELECT v[0]:party[2].health FROM myvariant;
SELECT v[0]:party[3].health FROM myvariant;
Another option might be to make the data more like a table ... I find it easier to work with than the JSON :-)
Code at bottom - just copy/paste and it runs in Snowflake returning screenshot below.
Key Doco is Lateral Flatten
SELECT d4.path, d4.value
from
lateral flatten(input=>PARSE_JSON('[{ "party": [ {"class":"Farmer","gender":"Female","ethnicity":"NativeAmerican","health":2}, {"class":"Adventurer","gender":"Male","ethnicity":"White","health":3}, {"class":"Farmer","gender":"Male","ethnicity":"White","health":0}, {"class":"Banker","gender":"Female","ethnicity":"White","health":0} ] }]') ) as d ,
lateral flatten(input=> value) as d2 ,
lateral flatten(input=> d2.value) as d3 ,
lateral flatten(input=> d3.value) as d4
I need to generate the following XML structure from my SQL server view for four inspection type. The XML structure is as given below.
==
<Form FormIdentifier="Major Approvals (EPIC:EPIC Test)" CompanyCode="EPIC:EPIC Test" CompanyName="EPIC Test" VesselCode="EPBARN" SubmittedDate="2019-05-22T08:00:00" Status="Submitted" ApprovedDate="" ImoNumber="9251121">
<VesselName>EPIC BARNES</VesselName>
<VoyageNo></VoyageNo>
<IMO_Number>9251121</IMO_Number>
<SIRE>
<SIRERow>
<Company__Terminal2>SHELL</Company__Terminal2>
<Last_Inspected>2019-05-22T12:00:00+00:00</Last_Inspected>
<No_of_Obs>1</No_of_Obs>
<Risk_Rating>2</Risk_Rating>
<ve_Screening_Yes_No>3</ve_Screening_Yes_No>
<Comments>4</Comments>
<Expiry_Date>2019-05-23T12:00:00+00:00</Expiry_Date>
<Planned_Date>2019-05-31T12:00:00+00:00</Planned_Date>
<Planned_Port>SINGAPORE</Planned_Port>
<Plannng_Comments>5</Plannng_Comments>
<Observations_closed_out2>6</Observations_closed_out2>
</SIRERow>
</SIRE>
<Non_SIRE>
<Non_SIRERow>
<Company__Terminal1>BP</Company__Terminal1>
<Last_Inspected1>2019-05-01T12:00:00+00:00</Last_Inspected1>
<No_of_Obs1>1</No_of_Obs1>
<Risk_Rating1>2</Risk_Rating1>
<ve_Screening_Yes_No1>3</ve_Screening_Yes_No1>
<Comments1>4</Comments1>
<Expiry_Date1>2019-05-22T12:00:00+00:00</Expiry_Date1>
<Planned_Date1>2019-05-31T12:00:00+00:00</Planned_Date1>
<Planned_Port1>KERTEH</Planned_Port1>
<Planning_Comments>5</Planning_Comments>
<Observations_closed_out1>6</Observations_closed_out1>
</Non_SIRERow>
</Non_SIRE>
<Additional_Screening>
<Additional_ScreeningRow>
<Company__Terminal>EXXON</Company__Terminal>
<Last_Inspected2>2019-05-01T12:00:00+00:00</Last_Inspected2>
<No_of_Obs2>1</No_of_Obs2>
<Risk_Rating2>2</Risk_Rating2>
<ve_Screening_Yes_No2>3</ve_Screening_Yes_No2>
<Comments2>4</Comments2>
<Expiry_Date2>2019-05-22T12:00:00+00:00</Expiry_Date2>
<Planned_Date_>2019-05-31T12:00:00+00:00</Planned_Date_>
<Planned_Port2>OSAKA</Planned_Port2>
<Planning_Comments1>5</Planning_Comments1>
<Observations_closed_out>6</Observations_closed_out>
</Additional_ScreeningRow>
</Additional_Screening>
</Form>
=====
I have the written following query using for XML path and TYPE to create the above XML file.
it works fine as far as the syntax is concerned but never returns exact no of records. for one or two records it works fine and for more records, it either not generated fully or shows fewer records
DECLARE #vsl AS varchar(50) = 'Sea Fortune 1';
DECLARE #imo AS varchar(50) = '9293741';
SELECT 'Major Approvals (EPIC:EPIC Test)' AS "#FormIdentifier",
'EPIC:EPIC Test' AS "#CompanyCode",
'EPIC Test' AS "#CompanyName",
'' AS "#VesselCode",
GETDATE() AS "#SubmittedDate",
'Submitted' AS "#Status",
'' AS "#ApprovedDate",
#imo AS "#ImoNumber",
#vsl AS VesselName,
'' AS VoyageNo,
#imo AS IMONO,
(SELECT otmajorname AS "SIRERow/Company__Terminal2",
inspectedOn AS "SIRERow/Last_Inspected",
tobs AS "SIRERow/No_of_Obs2",
riskrating AS "SIRERow/Risk_Rating2",
pscreen AS "SIRERow/ve_Screening_Yes_No2",
comment AS "SIRERow/Comments2",
ApprovalTo AS "SIRERow/Expiry_Date",
plplandate AS "SIRERow/Planned_Date",
plport AS "SIRERow/Planned_Port",
remark AS "SIRERow/Plannng_Comments",
openobs AS "SIRERow/Observations_closed_out2"
FROM RptXMLepic
WHERE vtIMONo = #imo
AND InspType = 'SIRE'
FOR XML PATH('SIRE'), TYPE),
(SELECT otmajorname AS "Non_SIRERow/Company__Terminal2",
inspectedOn AS "Non_SIRERow/Last_Inspected",
tobs AS "Non_SIRERow/No_of_Obs2",
riskrating AS "Non_SIRERow/Risk_Rating2",
pscreen AS "Non_SIRERow/ve_Screening_Yes_No2",
comment AS "Non_SIRERow/Comments2",
ApprovalTo AS "Non_SIRERow/Expiry_Date",
plplandate AS "Non_SIRERow/Planned_Date",
plport AS "Non_SIRERow/Planned_Port",
remark AS "Non_SIRERow/Plannng_Comments",
openobs AS "Non_SIRERow/Observations_closed_out2"
FROM RptXMLepic
WHERE vtIMONo = #imo
AND InspType = 'NON- SIRE'
FOR XML PATH('Non_SIRE'), TYPE),
(SELECT otmajorname AS "CDIRow/Company__Terminal2",
inspectedOn AS "CDIRow/Last_Inspected",
tobs AS "CDIRow/No_of_Obs2",
riskrating AS "CDIRow/Risk_Rating2",
pscreen AS "CDIRow/ve_Screening_Yes_No2",
comment AS "CDIRow/Comments2",
ApprovalTo AS "CDIRow/Expiry_Date",
plplandate AS "CDIRow/Planned_Date",
plport AS "CDIRow/Planned_Port",
remark AS "CDIRow/Plannng_Comments",
openobs AS "CDIRow/Observations_closed_out2"
FROM RptXMLepic
WHERE vtIMONo = #imo
AND InspType = 'CDI'
FOR XML PATH('CDI'), TYPE),
(SELECT otmajorname AS "Addional_ScreeningRow/Company__Terminal2",
inspectedOn AS "Addional_ScreeningRow/Last_Inspected",
tobs AS "Addional_ScreeningRow/No_of_Obs2",
riskrating AS "Addional_ScreeningRow/Risk_Rating2",
pscreen AS "Addional_ScreeningRow/ve_Screening_Yes_No2",
comment AS "Addional_ScreeningRow/Comments2",
ApprovalTo AS "Addional_ScreeningRow/Expiry_Date",
plplandate AS "Addional_ScreeningRow/Planned_Date",
plport AS "Addional_ScreeningRow/Planned_Port",
remark AS "Addional_ScreeningRow/Plannng_Comments",
openobs AS "Addional_ScreeningRow/Observations_closed_out2"
FROM RptXMLepic
WHERE vtIMONo = #imo
AND InspType = 'Screen'
FOR XML PATH('Addional_Screening'), TYPE)
FOR XML PATH('Form');
GO
my data lies in the view RptXMLepic.
would you please help me where exactly the problem lies.
It was my mistake only . the query is perfect after a lot of searching on the net I found nothing. actually what I was doing just copying the XML output from the query window and pasting on some noteapd++ editor. But whole XML text was not getting copied. so finally I have clicked on on the output and it was blah blah! I mean it opened pretty well in SQL management studio only with all the data. sorry guys I am pretty relaxed now ..thanks