SQL Server XML Filter - sql-server

I have a XML Data like:
<EmployeeDetails>
<BusinessEntityID>3</BusinessEntityID>
<NationalIDNumber>509647174</NationalIDNumber>
<JobTitle>Engineering Manager</JobTitle>
<BirthDate>1974-11-12</BirthDate>
<MaritalStatus>M</MaritalStatus>
<Gender>M</Gender>
<StoreDetail>
<Store>
<AnnualSales>800000</AnnualSales>
<AnnualRevenue>80000</AnnualRevenue>
<BankName>Guardian Bank</BankName>
<BusinessType>BM</BusinessType>
<YearOpened>1987</YearOpened>
<Specialty>Touring</Specialty>
<SquareFeet>21000</SquareFeet>
</Store>
<Store>
<AnnualSales>300000</AnnualSales>
<AnnualRevenue>30000</AnnualRevenue>
<BankName>International Bank</BankName>
<BusinessType>BM</BusinessType>
<YearOpened>1982</YearOpened>
<Specialty>Road</Specialty>
<SquareFeet>9000</SquareFeet>
</Store>
</StoreDetail>
</EmployeeDetails>
as an example I want to filter as follows 'SquareFeet>10000' and the result I want to get
<EmployeeDetails>
<BusinessEntityID>3</BusinessEntityID>
<NationalIDNumber>509647174</NationalIDNumber>
<JobTitle>Engineering Manager</JobTitle>
<BirthDate>1974-11-12</BirthDate>
<MaritalStatus>M</MaritalStatus>
<Gender>M</Gender>
<StoreDetail>
<Store>
<AnnualSales>800000</AnnualSales>
<AnnualRevenue>80000</AnnualRevenue>
<BankName>Guardian Bank</BankName>
<BusinessType>BM</BusinessType>
<YearOpened>1987</YearOpened>
<Specialty>Touring</Specialty>
<SquareFeet>21000</SquareFeet>
</Store>
</StoreDetail>
</EmployeeDetails>
can I do this with SQL server? The result I want to get is still an XML data.

SQL Server XQuery and its FLWOR expression allow to achieve what you need with ease.
T-SQL
DECLARE #xml XML =
N'<EmployeeDetails>
<BusinessEntityID>3</BusinessEntityID>
<NationalIDNumber>509647174</NationalIDNumber>
<JobTitle>Engineering Manager</JobTitle>
<BirthDate>1974-11-12</BirthDate>
<MaritalStatus>M</MaritalStatus>
<Gender>M</Gender>
<StoreDetail>
<Store>
<AnnualSales>800000</AnnualSales>
<AnnualRevenue>80000</AnnualRevenue>
<BankName>Guardian Bank</BankName>
<BusinessType>BM</BusinessType>
<YearOpened>1987</YearOpened>
<Specialty>Touring</Specialty>
<SquareFeet>21000</SquareFeet>
</Store>
<Store>
<AnnualSales>300000</AnnualSales>
<AnnualRevenue>30000</AnnualRevenue>
<BankName>International Bank</BankName>
<BusinessType>BM</BusinessType>
<YearOpened>1982</YearOpened>
<Specialty>Road</Specialty>
<SquareFeet>9000</SquareFeet>
</Store>
</StoreDetail>
</EmployeeDetails>';
SELECT #xml.query('<EmployeeDetails>
{
for $x in /EmployeeDetails/*
return if (local-name($x) ne "StoreDetail") then $x
else
<StoreDetail>
{
for $y in $x/Store
where $y/SquareFeet/text() > 10000
return $y
}
</StoreDetail>
}
</EmployeeDetails>');
Output
<EmployeeDetails>
<BusinessEntityID>3</BusinessEntityID>
<NationalIDNumber>509647174</NationalIDNumber>
<JobTitle>Engineering Manager</JobTitle>
<BirthDate>1974-11-12</BirthDate>
<MaritalStatus>M</MaritalStatus>
<Gender>M</Gender>
<StoreDetail>
<Store>
<AnnualSales>800000</AnnualSales>
<AnnualRevenue>80000</AnnualRevenue>
<BankName>Guardian Bank</BankName>
<BusinessType>BM</BusinessType>
<YearOpened>1987</YearOpened>
<Specialty>Touring</Specialty>
<SquareFeet>21000</SquareFeet>
</Store>
</StoreDetail>
</EmployeeDetails>

This can be done as follows
select * into #tmp from YourTable
update #tmp
set xmldata.modify('
delete (/EmployeeDetails/StoreDetail/Store[SquareFeet <= 10000])[1]
')
select * from #tmp
drop table #tmp

Related

Read XML on SQL Server : Select custom attribute

I'm trying to read an XML, it works pretty well on the whole except for the attribute named :
<custom-attribute attribute-id="loyaltyNumber">1234567890</custom-attribute>
which I am trying to get the value "1234567890"
Here is the test code:
DECLARE #XML XML = '<customers>
<customer customer-no="00000001">
<credentials>
<login>test#email.com</login>
</credentials>
<profile>
<preferred-locale>fr_BE</preferred-locale>
<custom-attributes>
<custom-attribute attribute-id="ServiceId">1</custom-attribute>
<custom-attribute attribute-id="loyaltyNumber">1234567890</custom-attribute>
</custom-attributes>
</profile>
<note/>
</customer>
<customer customer-no="00000002">
<credentials>
<login>test2#email.com</login>
</credentials>
<profile>
<preferred-locale>fr_FR</preferred-locale>
<custom-attributes>
<custom-attribute attribute-id="loyaltyNumber">1234567890</custom-attribute>
</custom-attributes>
</profile>
<note/>
</customer>
</customers>'
SELECT
CustomerNo = Events.value('#customer-no', 'int'),
--EventType = Events.value('#Type', 'varchar(20)'),
CustomerLogin =Events.value('(credentials/login)[1]', 'varchar(60)'),
CustomerLocal =Events.value('(profile/preferred-locale)[1]', 'varchar(60)'),
EventKind =Events.value('(profile/custom-attributes/custom-attribute)[2]', 'varchar(60)')
FROM
#XML.nodes('/customers/customer') AS XTbl(Events)
The current result is:
CustomerNo
CustomerLogin
CustomerLocal
EventKind
1
test#email.com
fr_BE
1234567890
2
test2#email.com
fr_FR
NULL
And it's logical since custom-attributes are optional, so you can't access them with the index.
So I'm looking for a way to access them by name: attribute-id="loyaltyNumber"
Thanks,
..filter the path to the attribute which has attribute-id = “loyaltyNumber”
EventKind =Events.value('(profile/custom-attributes/custom-attribute[#attribute-id="loyaltyNumber"])[1]', 'varchar(60)')
You can just add the attribute-id of the custom attribute to your query:
DECLARE #XML XML = '<customers>
<customer customer-no="00000001">
<credentials>
<login>test#email.com</login>
</credentials>
<profile>
<preferred-locale>fr_BE</preferred-locale>
<custom-attributes>
<custom-attribute attribute-id="ServiceId">1</custom-attribute>
<custom-attribute attribute-id="loyaltyNumber">1234567890</custom-attribute>
</custom-attributes>
</profile>
<note/>
</customer>
<customer customer-no="00000002">
<credentials>
<login>test2#email.com</login>
</credentials>
<profile>
<preferred-locale>fr_FR</preferred-locale>
<custom-attributes>
<custom-attribute attribute-id="loyaltyNumber">1234567777</custom-attribute>
</custom-attributes>
</profile>
<note/>
</customer>
</customers>'
SELECT
CustomerNo = Events.value('#customer-no', 'int'),
--EventType = Events.value('#Type', 'varchar(20)'),
CustomerLogin =Events.value('(credentials/login)[1]', 'varchar(60)'),
CustomerLocal =Events.value('(profile/preferred-locale)[1]', 'varchar(60)'),
EventKind =Events.value('(profile/custom-attributes/custom-attribute[#attribute-id = "loyaltyNumber"])[1]', 'varchar(60)')
FROM
#XML.nodes('/customers/customer') AS XTbl(Events)

Update multiple substrings within a string

I have a single text string stored in a SQL table which contains all of the text below. The format is XML but the field definition is varchar.
I am using SQL Server 2012 to query this data:
<?xml version="1.0" encoding="utf-16"?>
<SaveFileContext xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
<Mappings>
<SaveFileModel Header="Model1" FullPath="Model1" ViewType="ModelView" />
<SaveFileModel Header="xyz" FullPath="\\server\directory\" ViewType="TradeView" />
<SaveFileModel Header="Model2" FullPath="Model2" ViewType="ModelView" />
<SaveFileModel Header="Model3" FullPath="Model3" ViewType="ModelView" />
<SaveFileModel Header="abc" FullPath="\\server\directory\" ViewType="TradeView" />
<SaveFileModel Header="def" FullPath="\\server\directory\" ViewType="TradeView" />
<SaveFileModel Header="ghi" FullPath="\\server\directory\" ViewType="TradeView"/>
</Mappings>
</SaveFileContext>
How can I update or remove the entire lines of text where viewtype="ModelView"?
I want to remove any lines where viewtype="ModelView" within this single string and replace it with a blank space. In the example above I want to remove 3 lines total and leave the rest.
In the end I want the string to look like below (keep in mind all lines are contained in 1 single string. I just separated them to make viewing them easier.
<?xml version="1.0" encoding="utf-16"?>
<SaveFileContext xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
<Mappings>
<SaveFileModel Header="xyz" FullPath="\\server\directory\" ViewType="TradeView" />
<SaveFileModel Header="abc" FullPath="\\server\directory\" ViewType="TradeView" />
<SaveFileModel Header="def" FullPath="\\server\directory\" ViewType="TradeView" />
<SaveFileModel Header="ghi" FullPath="\\server\directory\" ViewType="TradeView"/>
</Mappings>
</SaveFileContext>
This is my current query that I am using to replace the lines but the values to replace are very specific. I'm basically entering the string to look for to replace.
IF OBJECT_ID('tempdb..#replacement') IS NOT NULL
DROP TABLE #replacement
CREATE TABLE #replacement (
string_pattern VARCHAR(100),
string_replacement VARCHAR(5)
);
INSERT INTO #replacement (
string_pattern,
string_replacement
)
VALUES
('<SaveFileModel Header="Model1" FullPath="Model1" ViewType="ModelView" />', ''),
('<SaveFileModel Header="Model2" FullPath="Model2" ViewType="ModelView" />', ''),
('<SaveFileModel Header="Model3" FullPath="Model3" ViewType="ModelView" />', ''),
('<SaveFileModel Header="Model4" FullPath="Model4" ViewType="ModelView" />', ''),
('<SaveFileModel Header="Model5" FullPath="Model5" ViewType="ModelView" />', ''),
('<SaveFileModel Header="Model6" FullPath="Model6" ViewType="ModelView" />', ''),
('<SaveFileModel Header="Model7" FullPath="Model7" ViewType="ModelView" />', ''),
('<SaveFileModel Header="Model8" FullPath="Model8" ViewType="ModelView" />', ''),
('<SaveFileModel Header="Model9" FullPath="Model9" ViewType="ModelView" />', ''),
('<SaveFileModel Header="Model10" FullPath="Model10" ViewType="ModelView" />', ''),
('<SaveFileModel Header="Model11" FullPath="Model11" ViewType="ModelView" />', ''),
('<SaveFileModel Header="Model12" FullPath="Model12" ViewType="ModelView" />', ''),
('<SaveFileModel Header="Model13" FullPath="Model13" ViewType="ModelView" />', ''),
('<SaveFileModel Header="Model14" FullPath="Model14" ViewType="ModelView" />', ''),
('<SaveFileModel Header="Model15" FullPath="Model15" ViewType="ModelView" />', ''),
('<SaveFileModel Header="Model16" FullPath="Model16" ViewType="ModelView" />', ''),
('<SaveFileModel Header="Model17" FullPath="Model17" ViewType="ModelView" />', ''),
('<SaveFileModel Header="Model18" FullPath="Model18" ViewType="ModelView" />', ''),
('<SaveFileModel Header="Model19" FullPath="Model19" ViewType="ModelView" />', ''),
('<SaveFileModel Header="Model20" FullPath="Model20" ViewType="ModelView" />', '');
DECLARE #string AS VARCHAR(MAX),
#userident varchar(20);
DECLARE userviewdef CURSOR FOR
SELECT
UserID
FROM
TableSettings where section = 'user view session'
OPEN userviewdef;
FETCH NEXT FROM userviewdef INTO
#userident;
WHILE ##FETCH_STATUS = 0
BEGIN
select #string = varcharvalue from tablesettings where section = 'user view session' and userid = #userident
-- Perform all replacements
SELECT #string = REPLACE(#string, string_pattern, string_replacement) FROM #replacement;
-- Return new string
--PRINT #string;
update tablesettings
set varcharvalue = #string
where section = 'user view session' and userid = #userident
FETCH NEXT FROM userviewdef INTO
#userident;
END;
CLOSE userviewdef;
DEALLOCATE userviewdef;
There are instances where a string could be what is seen below and it would not get removed since it doesn't fit any of the strings I coded to be removed. I want to find an easy way to remove any lines where ViewType="ModelView" since this is my criteria.
<SaveFileModel Header="ThisIsSomethingElse" FullPath="Model1" ViewType="ModelView" />
4/21/21:
After following the advice of #FrankPl to use XML here is what I came up with. The result returns a blank space. I might be coding the XML query piece incorrectly and need a bit of help:
DECLARE #stringXML XML,
#finalString varchar(max),
#userident varchar(20);
DECLARE userviewdef CURSOR FOR
SELECT
UserID
FROM
TableSettings where section = 'user view session';
OPEN userviewdef;
FETCH NEXT FROM userviewdef INTO
#userident;
WHILE ##FETCH_STATUS = 0
BEGIN
select #stringXML = varcharvalue from tablesettings where section = 'user view session' and userid = #userident
select #stringXML.query('
for $item in (/SaveFileContext/Mappings/SaveFileModel)
return if ($item[#ViewType = "ModelView"]) then string($item) else ""
')
select #finalString = convert(varchar(max), #stringXML)
-- Return new string
PRINT #finalstring;
FETCH NEXT FROM userviewdef INTO
#userident;
END;
CLOSE userviewdef;
DEALLOCATE userviewdef;
If you declare your #step variable as data type XML instead of varchar, you can use XQuery to process the data, in this case a FLOWR expression in curly braces within literal XML elements, the curly braces switch from literal mode to XPath mode:
SELECT #string.query('
<SaveFileContext>
<Mappings> {
for $item in (/SaveFileContext/Mappings/SaveFileModel)
return if ($item[#ViewType = "ModelView"]) then $item else ()
} </Mappings>
</SaveFileContext>
'
The XML declaration and the namespaces are swallowed by the SQLServer XML processor.
This solution assumes the data are valid XML fragments.
Please note that - as the whole query needs to be a SQL string (i. e. in single quotes), and XQuery accepts strings in double as well as in single quotes, for simplicity, I used double quotes here. Alternatively, I could have used doubled single quotes to escape them within the SQL string.

SQL XML - Create a XML file from SQL Server for an invoice including invoice positions in one XML file

I create my xml file in this way (I do not show all output fields because there are very many fields):
DECLARE #ID_Rechnung int = 8;
WITH XMLNAMESPACES (
'urn:oasis:names:specification:ubl:schema:xsd:CommonExtensionComponents-2' as ext,
'urn:oasis:names:specification:ubl:schema:xsd:CommonBasicComponents-2' as cbc,
'urn:oasis:names:specification:ubl:schema:xsd:CommonAggregateComponents-2' as cac,
'http://uri.etsi.org/01903/v1.3.2#' as xades,
'http://www.w3.org/2001/XMLSchema-instance' as xsi,
'http://www.w3.org/2000/09/xmldsig#' as ds
)
SELECT
#XMLData = xmldat.xmldataCol
FROM
(
SELECT
(
SELECT
-- HIER XML Daten generieren
'' AS 'ext:UBLExtensions',
'' AS 'ext:UBLExtensions/ext:UBLExtension',
'' AS 'ext:UBLExtensions/ext:UBLExtension/ext:ExtensionContent',
'2.1' AS 'cbc:UBLVersionID',
'TR1.2' AS 'cbc:CustomizationID',
'' AS 'cbc:ProfileID',
Rechnungen.Nummer AS 'cbc:ID',
'false' AS 'cbc:CopyIndicator',
'' AS 'cbc:UUID',
CAST(Rechnungen.Datum AS Date) AS 'cbc:IssueDate'
FROM
rechnungen
WHERE
rechnungen.id = #ID_Rechnung
FOR XML PATH(''), ROOT('Invoice')
) AS xmldataCol
This works fine - i get the following XML:
<Invoice xmlns:ds="http://www.w3.org/2000/09/xmldsig#" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xades="http://uri.etsi.org/01903/v1.3.2#" xmlns:cac="urn:oasis:names:specification:ubl:schema:xsd:CommonAggregateComponents-2" xmlns:cbc="urn:oasis:names:specification:ubl:schema:xsd:CommonBasicComponents-2" xmlns:ext="urn:oasis:names:specification:ubl:schema:xsd:CommonExtensionComponents-2">
<ext:UBLExtensions>
<ext:UBLExtension>
<ext:ExtensionContent />
</ext:UBLExtension>
</ext:UBLExtensions>
<cbc:UBLVersionID>2.1</cbc:UBLVersionID>
<cbc:CustomizationID>TR1.2</cbc:CustomizationID>
<cbc:ProfileID />
<cbc:ID>R200001</cbc:ID>
<cbc:CopyIndicator>false</cbc:CopyIndicator>
<cbc:UUID />
<cbc:IssueDate>2020-06-29</cbc:IssueDate>
</Invoice>
But now i need the invoice positions in the same file.
This SQL should be included in the first one and the date should be as invoice line in the xml file:
SELECT
Rechnungpos.ID AS 'cac:InvoiceLine/cbc:ID',
Rechnungpos.Anzahl AS 'cac:InvoiceLine/cbc:InvoicedQuantity'
FROM
RechnungPos
WHERE
RechnungPos.id_Rechnung = #ID_Rechnung
The output should be this:
<Invoice xmlns:ds="http://www.w3.org/2000/09/xmldsig#" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xades="http://uri.etsi.org/01903/v1.3.2#" xmlns:cac="urn:oasis:names:specification:ubl:schema:xsd:CommonAggregateComponents-2" xmlns:cbc="urn:oasis:names:specification:ubl:schema:xsd:CommonBasicComponents-2" xmlns:ext="urn:oasis:names:specification:ubl:schema:xsd:CommonExtensionComponents-2">
<ext:UBLExtensions>
<ext:UBLExtension>
<ext:ExtensionContent />
</ext:UBLExtension>
</ext:UBLExtensions>
<cbc:UBLVersionID>2.1</cbc:UBLVersionID>
<cbc:CustomizationID>TR1.2</cbc:CustomizationID>
<cbc:ProfileID />
<cbc:ID>R200001</cbc:ID>
<cbc:CopyIndicator>false</cbc:CopyIndicator>
<cbc:UUID />
<cbc:IssueDate>2020-06-29</cbc:IssueDate>
<cac:InvoiceLine>
<cbc:ID>1<(cbc:>
<cbc:InvoicedQuantity>3</cbc:InvoicedQuantity>
</cac:InvoiceLine>
<cac:InvoiceLine>
<cbc:ID>5<(cbc:>
<cbc:InvoicedQuantity>1</cbc:InvoicedQuantity>
</cac:InvoiceLine>
<cac:InvoiceLine>
<cbc:ID>9<(cbc:>
<cbc:InvoicedQuantity>2</cbc:InvoicedQuantity>
</cac:InvoiceLine>
</Invoice>
Here is the Code to generate Test Data:
CREATE TABLE [dbo].[Rechnungen](
[id] [int] NOT NULL,
[Nummer] [nvarchar](20) NOT NULL,
[Datum] [datetime] NOT NULL
)
INSERT INTO Rechnungen (id, Nummer, Datum) VALUES (8, 'R200001', '29.06.2020')
CREATE TABLE [dbo].Rechnungpos(
[id] [int] NOT NULL,
[id_Rechnung] [int] NOT NULL,
[Anzahl] [float] NOT NULL
)
INSERT INTO RechnungPos (id, id_Rechnung, Anzahl) VALUES (1, 8, 3)
INSERT INTO RechnungPos (id, id_Rechnung, Anzahl) VALUES (5, 8, 1)
INSERT INTO RechnungPos (id, id_Rechnung, Anzahl) VALUES (9, 8, 2)
it has to run on different versions - my version is SQL Server 2019
How can i do that?
Thanks for help, Thomas.
Here is how to do it.
The only complexity is how to handle a child table and specially namespaces in the output XML. That's why XQuery and its FLWOR expression are in use.
SQL
-- DDL and sample data population, start
DECLARE #Rechnungen TABLE (id int , Nummer nvarchar(20), Datum datetime);
INSERT INTO #Rechnungen (id, Nummer, Datum) VALUES
(8, 'R200001', '2020-06-29');
DECLARE #Rechnungpos TABLE (id int, id_Rechnung int, Anzahl float);
INSERT INTO #RechnungPos (id, id_Rechnung, Anzahl) VALUES
(1, 8, 3),
(5, 8, 1),
(9, 8, 2);
-- DDL and sample data population, end
DECLARE #ID_Rechnung int = 8;
;WITH XMLNAMESPACES ('urn:oasis:names:specification:ubl:schema:xsd:CommonExtensionComponents-2' as ext
, 'urn:oasis:names:specification:ubl:schema:xsd:CommonBasicComponents-2' as cbc
, 'urn:oasis:names:specification:ubl:schema:xsd:CommonAggregateComponents-2' as cac
, 'http://uri.etsi.org/01903/v1.3.2#' as xades
, 'http://www.w3.org/2001/XMLSchema-instance' as xsi
, 'http://www.w3.org/2000/09/xmldsig#' as ds)
SELECT (
SELECT '2.1' AS [cbc:UBLVersionID],
'TR1.2' AS [cbc:CustomizationID],
'' AS [cbc:ProfileID],
p.Nummer AS [cbc:ID],
'false' AS [cbc:CopyIndicator],
'' AS [cbc:UUID],
CAST(p.Datum AS Date) AS [cbc:IssueDate],
(
SELECT c.id AS [cbc:ID]
, CAST(c.Anzahl AS INT) AS [cbc:InvoicedQuantity]
FROM #Rechnungpos AS c INNER JOIN
#Rechnungen AS p ON p.id = c.id_Rechnung
FOR XML PATH('r'), TYPE, ROOT('root')
)
FROM #Rechnungen AS p
WHERE p.id = #ID_Rechnung
FOR XML PATH(''), TYPE, ROOT('Invoice')
).query('<Invoice xmlns:ds="http://www.w3.org/2000/09/xmldsig#"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:xades="http://uri.etsi.org/01903/v1.3.2#"
xmlns:cac="urn:oasis:names:specification:ubl:schema:xsd:CommonAggregateComponents-2"
xmlns:cbc="urn:oasis:names:specification:ubl:schema:xsd:CommonBasicComponents-2"
xmlns:ext="urn:oasis:names:specification:ubl:schema:xsd:CommonExtensionComponents-2">
<ext:UBLExtensions>
<ext:UBLExtension>
<ext:ExtensionContent/>
</ext:UBLExtension>
</ext:UBLExtensions>
{
for $x in /Invoice/*[local-name()!="root"]
return $x,
for $x in /Invoice/root/r
return <cac:InvoiceLine>{$x/*}</cac:InvoiceLine>
}
</Invoice>');
Output
<Invoice xmlns:ds="http://www.w3.org/2000/09/xmldsig#" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xades="http://uri.etsi.org/01903/v1.3.2#" xmlns:cac="urn:oasis:names:specification:ubl:schema:xsd:CommonAggregateComponents-2" xmlns:cbc="urn:oasis:names:specification:ubl:schema:xsd:CommonBasicComponents-2" xmlns:ext="urn:oasis:names:specification:ubl:schema:xsd:CommonExtensionComponents-2">
<ext:UBLExtensions>
<ext:UBLExtension>
<ext:ExtensionContent />
</ext:UBLExtension>
</ext:UBLExtensions>
<cbc:UBLVersionID>2.1</cbc:UBLVersionID>
<cbc:CustomizationID>TR1.2</cbc:CustomizationID>
<cbc:ProfileID />
<cbc:ID>R200001</cbc:ID>
<cbc:CopyIndicator>false</cbc:CopyIndicator>
<cbc:UUID />
<cbc:IssueDate>2020-06-29</cbc:IssueDate>
<cac:InvoiceLine>
<cbc:ID>1</cbc:ID>
<cbc:InvoicedQuantity>3</cbc:InvoicedQuantity>
</cac:InvoiceLine>
<cac:InvoiceLine>
<cbc:ID>5</cbc:ID>
<cbc:InvoicedQuantity>1</cbc:InvoicedQuantity>
</cac:InvoiceLine>
<cac:InvoiceLine>
<cbc:ID>9</cbc:ID>
<cbc:InvoicedQuantity>2</cbc:InvoicedQuantity>
</cac:InvoiceLine>
</Invoice>
Thanks, Yitzhak Khabinsky for help - heres the complete solution:
DECLARE #ID_Rechnung int = 8,
#XMLData xml;
WITH XMLNAMESPACES ('urn:oasis:names:specification:ubl:schema:xsd:CommonExtensionComponents-2' as ext
, 'urn:oasis:names:specification:ubl:schema:xsd:CommonBasicComponents-2' as cbc
, 'urn:oasis:names:specification:ubl:schema:xsd:CommonAggregateComponents-2' as cac
, 'http://uri.etsi.org/01903/v1.3.2#' as xades
, 'http://www.w3.org/2001/XMLSchema-instance' as xsi
, 'http://www.w3.org/2000/09/xmldsig#' as ds)
SELECT
#XMLData = xmldat.xmldataCol
FROM
(
SELECT (
SELECT
-- HIER XML Daten generieren
'' AS 'ext:UBLExtensions',
'' AS 'ext:UBLExtensions/ext:UBLExtension',
'' AS 'ext:UBLExtensions/ext:UBLExtension/ext:ExtensionContent',
'2.1' AS 'cbc:UBLVersionID',
'TR1.2' AS 'cbc:CustomizationID',
'' AS 'cbc:ProfileID',
Rechnungen.Nummer AS 'cbc:ID',
'false' AS 'cbc:CopyIndicator',
'' AS 'cbc:UUID',
CAST(Rechnungen.Datum AS Date) AS 'cbc:IssueDate',
'' AS 'cbc:InvoiceTypeCode',
Rechnungen.Bemerkung1 AS 'cbc:Note',
#Waehrung AS 'cbc:DocumentCurrencyCode',
#Waehrung AS 'cbc:TaxCurrencyCode',
Rechnungen.Auftrag AS 'cac:OrderReference/cbc:ID',
-- Verkaüfer
'' AS 'cac:AccountingSupplierParty/cac:Party/cbc:EndpointID',
'' AS 'cac:AccountingSupplierParty/cac:Party/cac:PartyIdentification/cbc:ID',
'' AS 'cac:AccountingSupplierParty/cac:Party/cac:PartyName/cbc:Name',
'' AS 'cac:AccountingSupplierParty/cac:Party/cac:PostalAddress/cbc:StreetName',
'' AS 'cac:AccountingSupplierParty/cac:Party/cac:PostalAddress/cbc:CityName',
'' AS 'cac:AccountingSupplierParty/cac:Party/cac:PostalAddress/cbc:PostalZone',
'' AS 'cac:AccountingSupplierParty/cac:Party/cac:PostalAddress/cac:Country/cbc:IdentificationCode',
'' AS 'cac:AccountingSupplierParty/cac:Party/cac:PartyTaxScheme/cbc:CompanyId',
'VAT' AS 'cac:AccountingSupplierParty/cac:Party/cac:PartyTaxScheme/cac:TaxScheme/cbc:ID',
'' AS 'cac:AccountingSupplierParty/cac:Party/cac:PartyLegalEntity/cbc:RegistrationName',
'' AS 'cac:AccountingSupplierParty/cac:Party/cac:PartyLegalEntity/cbc:CompanyID',
'' AS 'cac:AccountingSupplierParty/cac:Party/cac:PartyLegalEntity/cbc:CompanyLegalForm',
'' AS 'cac:AccountingSupplierParty/cac:Party/cac:Contact/cbc:Name',
'' AS 'cac:AccountingSupplierParty/cac:Party/cac:Contact/cbc:Telephone',
'' AS 'cac:AccountingSupplierParty/cac:Party/cac:Contact/cbc:ElectronicMail',
-- Käufer
'' AS 'cac:AccountingCustomerParty/cac:Party/cbc:EndpointID',
Rechnungen.DebKreNr AS 'cac:AccountingCustomerParty/cac:Party/cac:PartyIdentification/cbc:ID',
Rechnungen.DebBez01 + ' ' + DebBez02 AS 'cac:AccountingCustomerParty/cac:Party/cac:PartyName/cbc:Name',
Rechnungen.DebStrasse AS 'cac:AccountingCustomerParty/cac:Party/cac:PostalAddress/cbc:StreetName',
Rechnungen.DebOrt AS 'cac:AccountingCustomerParty/cac:Party/cac:PostalAddress/cbc:CityName',
Rechnungen.DebPLZ AS 'cac:AccountingCustomerParty/cac:Party/cac:PostalAddress/cbc:PostalZone',
Rechnungen.DebLandKFZ AS 'cac:AccountingCustomerParty/cac:Party/cac:PostalAddress/cac:Country/cbc:IdentificationCode',
Rechnungen.DebUMSTID AS 'cac:AccountingCustomerParty/cac:Party/cac:PartyTaxScheme/cbc:CompanyID',
'VAT' AS 'cac:AccountingCustomerParty/cac:Party/cac:PartyTaxScheme/cac:TaxScheme/cbc:ID',
Rechnungen.DebBez01 + ' ' + DebBez02 AS 'cac:AccountingCustomerParty/cac:Party/cac:PartyLegalEntity/cbc:RegistrationName',
'' AS 'cac:AccountingCustomerParty/cac:Party/cac:Contact/cbc:Name',
'' AS 'cac:AccountingCustomerParty/cac:Party/cac:Contact/cbc:Telephone',
'' AS 'cac:AccountingCustomerParty/cac:Party/cac:Contact/cbc:ElectronicMail',
-- Kontoverbindung Verkäufer
'' AS 'cac:PaymentMeans/cbc:PaymentMeansCode',
'' AS 'cac:PaymentMeans/cac:PayeeFinancialAccount/cbc:ID',
'' AS 'cac:PaymentMeans/cac:PayeeFinancialAccount/cbc:Name',
'' AS 'cac:PaymentMeans/cac:PayeeFinancialAccount/cac:FinancialInstitutionBranch/cbc:ID',
--'' AS 'cac:PaymentTerms/cbc:Note',
-- Steuern
#Waehrung AS 'cac:TaxTotal/cbc_TaxAmount/#currencyID',
CAST(Rechnungen.BetragMWST AS nvarchar(15)) AS 'cac:TaxTotal/cbc_TaxAmount',
#Waehrung AS 'cac:TaxTotal/cac:Taxubtotal/cbc:TaxableAmount/#currencyID',
CAST(Rechnungen.BetragNetto AS nvarchar(15))AS 'cac:TaxTotal/cac:Taxubtotal/cbc:TaxableAmount',
#Waehrung AS 'cac:TaxTotal/cac:Taxubtotal/cbc:TaxAmount/#currencyID',
CAST(Rechnungen.BetragMWST AS nvarchar(15))AS 'cac:TaxTotal/cac:Taxubtotal/cbc:TaxAmount',
'' AS 'cac:TaxTotal/cac:Taxubtotal/cac:TaxCategory/cbc:ID',
CAST(Rechnungen.MWST AS nvarchar(2)) AS 'cac:TaxTotal/cac:Taxubtotal/cac:TaxCategory/cbc:Percent',
'VAT' AS 'cac:TaxTotal/cac:Taxubtotal/cac:TaxCategory/cac:TaxScheme/cbc:ID',
#Waehrung AS 'cac:LegalMonetaryTotal/cbc:LineExtensionAmount/#currencyID',
CAST(Rechnungen.BetragNetto AS nvarchar(15))AS 'cac:LegalMonetaryTotal/cbc:LineExtensionAmount',
#Waehrung AS 'cac:LegalMonetaryTotal/cbc:TaxExclusiveAmount/#currencyID',
CAST(Rechnungen.BetragNetto AS nvarchar(15))AS 'cac:LegalMonetaryTotal/cbc:TaxExclusiveAmount',
#Waehrung AS 'cac:LegalMonetaryTotal/cbc:TaxInclusiveAmount/#currencyID',
CAST(Rechnungen.BetragBrutto AS nvarchar(15))AS 'cac:LegalMonetaryTotal/cbc:TaxInclusiveAmount',
#Waehrung AS 'cac:LegalMonetaryTotal/cbc:PayableAmount/#currencyID',
CAST(Rechnungen.BetragBrutto AS nvarchar(15))AS 'cac:LegalMonetaryTotal/cbc:PayableAmount',
(
SELECT Rechnungpos.id AS [cbc:ID]
, CAST(Rechnungpos.Anzahl AS INT) AS [cbc:InvoicedQuantity]
FROM Rechnungpos WHERE RechnungPos.id_Rechnung = #id_Rechnung
FOR XML PATH('r'), TYPE, ROOT('root')
)
FROM Rechnungen
WHERE Rechnungen.id = #ID_Rechnung
FOR XML PATH(''), TYPE, ROOT('Invoice')
) AS xmldataCol
) AS xmldat;
SELECT #XMLData
.query('<Invoice xmlns:ds="http://www.w3.org/2000/09/xmldsig#"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:xades="http://uri.etsi.org/01903/v1.3.2#"
xmlns:cac="urn:oasis:names:specification:ubl:schema:xsd:CommonAggregateComponents-2"
xmlns:cbc="urn:oasis:names:specification:ubl:schema:xsd:CommonBasicComponents-2"
xmlns:ext="urn:oasis:names:specification:ubl:schema:xsd:CommonExtensionComponents-2">
{
for $x in /Invoice/*[local-name()!="root"]
return $x,
for $x in /Invoice/root/r
return <cac:InvoiceLine>{$x/*}</cac:InvoiceLine>
}
</Invoice>');

Iterating through CSV and creating an XML file

I am trying to parse a CSV file in Perl and paste the information of some columns into an XML-file. I've never done anything in Perl, and my idea was to store the data into an array and then pull the information out of the array as I build it.
I'm sure I am doing several things wrong, since I am not getting the value I am expecting but instead what looks like the array addresses in the memory (here is an example: ARRAY(0x35e9360).
Could somebody help me out and point me to a better solution?
Here is the code in question:
use Text::CSV;
use utf8;
use XML::Simple qw(XMLout);
use XML::Twig;
use File::Slurp;
use Encode;
&buildXML();
my $csv = Text::CSV->new( { binary => 1 } ) # should set binary attribute.
or die "Cannot use CSV: " . Text::CSV->error_diag();
$csv = Text::CSV->new( { sep_char => '|' } );
$csv = Text::CSV_XS->new( { allow_loose_quotes => 1 } );
my $t = XML::Twig->new( pretty_print => indented );
$t->parsefile('output.xml');
$out_file = "output.xml";
open( my $fh_out, '>>', $out_file ) or die "unable to open $out_file for writing: $!";
my $root = $t->root; #get the root
open my $fh, "<:encoding(utf8)", "b.txt" or die "text.txt: $!";
while ( my $row = $csv->getline($fh) ) {
my #rows = $row;
$builds = $root->first_child(); # get the builds node
$xcr = $builds->first_child(); #get the xcr node
my $xcrCopy = $xcr->copy(); #copy the xcr node
$xcrCopy->paste( after, $xcr ); #paste the xcr node
$xcr->set_att( id => "#rows[0]" );
print {$fh_out} $t->sprint();
}
$csv->eof or $csv->error_diag();
Here is a testfile:
ID|Name|Pos
1|a|265
2|b|950
3|c|23
4|d|798
5|e|826
6|f|935
7|g|852
8|h|236
9|i|642
Here is the XML that is build by the buildXML() sub.
<?xml version='1.0' standalone='yes'?>
<project>
<builds>
<xcr id="" name="" pos="" />
</builds>
</project>
This program appears to do as you require
Links:
Text::CSV
XML::Twig
After reverse-engineering your code to discover your what you're aiming for, I find that it's really a fairly simply problem. It would have helped a lot if you had explained your intention in terms of adding a new xcr element for each line in the CSV file, with attributes corresponding to the columns
It's likely that you don't need the XML template file at all, or perhaps just the template xcr element with empty attributes is superfluous? I also wonder if you want to skip the header line from the CSV file? These changes are trivial, but I have left the code in the simplest state possible
use utf8;
use strict;
use warnings 'all';
use autodie;
use Text::CSV;
use XML::Twig;
use Encode;
use constant XML_FILE => 'output.xml';
use constant CSV_FILE => 'b.txt';
build_xml(XML_FILE);
my $csv = Text::CSV->new( {
sep_char => '|',
binary => 1,
allow_loose_quotes => 1, # This is brought forward. Probably unnecessary
} );
my $t = XML::Twig->new(
pretty_print => 'indented',
);
$t->parsefile(XML_FILE);
my ($xcr) = $t->findnodes('/project/builds/xcr');
open my $fh, '<:encoding(utf8)', CSV_FILE;
while ( my $row = $csv->getline($fh) ) {
my ($id, $name, $pos) = #$row;
my $xcr_copy = $xcr->copy;
$xcr_copy->set_att( id => $id, name => $name, pos => $pos );
$xcr_copy->paste( last_child => $xcr->parent );
}
$t->print;
sub build_xml {
open my $fh, '>', shift;
print $fh <<__END_XML__;
<?xml version='1.0' standalone='yes'?>
<project>
<builds>
<xcr id="" name="" pos="" />
</builds>
</project>
__END_XML__
}
output
<?xml version="1.0" standalone="yes"?>
<project>
<builds>
<xcr id="" name="" pos=""/>
<xcr id="ID" name="Name" pos="Pos"/>
<xcr id="1" name="a" pos="265"/>
<xcr id="2" name="b" pos="950"/>
<xcr id="3" name="c" pos="23"/>
<xcr id="4" name="d" pos="798"/>
<xcr id="5" name="e" pos="826"/>
<xcr id="6" name="f" pos="935"/>
<xcr id="7" name="g" pos="852"/>
<xcr id="8" name="h" pos="236"/>
<xcr id="9" name="i" pos="642"/>
</builds>
</project>
After reading your comment (stuff like this should be edited into the question) saying "I am building [the XML data] from scratch. There is a sub buildXML" I think this is more likely to be what you require. With XML::Twig it is simplest to parse some XML text instead of creating and linking individual XML::Twig::Elt objects
The $t object starts with no xcr objects at all. They are all created through XML::Twig::Elt->new and pasted as the last_child of the builds element
require v5.14.1; # For autodie
use utf8;
use strict;
use warnings 'all';
use autodie;
use Text::CSV;
use XML::Twig;
use Encode;
use constant XML_FILE => 'output.xml';
use constant CSV_FILE => 'b.txt';
my $t = XML::Twig->new(
pretty_print => 'indented',
);
$t->parse(<<END_XML);
<project>
<builds/>
</project>
END_XML
my ($builds) = $t->findnodes('/project/builds');
my $csv = Text::CSV->new( {
sep_char => '|',
binary => 1,
allow_loose_quotes => 1,
} );
{
open my $fh, '<:encoding(utf8)', CSV_FILE;
<$fh>; # Drop the header line
while ( my $row = $csv->getline($fh) ) {
my ($id, $name, $pos) = #$row;
my $xcr = XML::Twig::Elt->new(xcr => {
id => $id,
name => $name,
pos => $pos
});
$xcr->paste( last_child => $builds );
}
}
open my $fh, '>encoding(utf-8)', XML_FILE;
$t->set_output_encoding('UTF-8');
$t->print($fh, 'indented');
output
<?xml version="1.0" encoding="UTF-8"?><project>
<builds>
<xcr id="1" name="a" pos="265"/>
<xcr id="2" name="b" pos="950"/>
<xcr id="3" name="c" pos="23"/>
<xcr id="4" name="d" pos="798"/>
<xcr id="5" name="e" pos="826"/>
<xcr id="6" name="f" pos="935"/>
<xcr id="7" name="g" pos="852"/>
<xcr id="8" name="h" pos="236"/>
<xcr id="9" name="i" pos="642"/>
</builds>
</project>
The getline method of Text::CSV returns an arrayref
It reads a row from the IO object $io using $io->getline () and parses this row into an array ref.
The ARRAY(0x35e9360) is indeed what you get when you print out array reference. This is usual, many parsers normally return a reference to an array for a row. So you need to dereference that, generally by #{$arrayref}, but in this case there is no ambiguity and one can drop the curlies, #$arrayref.
use warnings;
use strict;
use Text::CSV_XS;
use XML::Twig;
my $csv = Text::CSV_XS->new (
{ binary => 1, sep_char => '|', allow_loose_quotes => 1 }
) or die "Cannot use CSV: " . Text::CSV->error_diag();
my $t = XML::Twig->new(pretty_print => 'indented');
$t->parsefile('output.xml');
my $out_file = 'output.xml';
open my $fh_out, '>>', $out_file or die "Can't open $out_file for append: $!";
my $root = $t->root;
my $file = 'b.txt';
open my $fh, "<:encoding(UTF-8)", $file or die "Can't open $file: $!";
while (my $rowref = $csv->getline($fh)) {
#my #cols = #$rowref;
#print "#cols\n";
my $builds = $root->first_child(); # get the builds node
my $xcr = $builds->first_child(); # get the xcr node
my $xcrCopy = $xcr->copy(); # copy the xcr node
$xcrCopy->paste('after', $xcr); # paste the xcr node
$xcr->set_att(id => $rowref->[0]); # or $cols[0];
print $fh_out $t->sprint();
}
This prints (when #cols and its print are uncommented) for the CSV file
ID Name Pos
1 a 265
2 b 950
...
So we've read the file OK.
The XML processing is copied from the question, except for the part that uses the CSV value. We take the first element of the current row, which is $rowref->[0] since $rowref is a reference. (Or use an element from the dereferenced array, $cols[0].)
I don't know what output is expected but it is built out of the template and seems OK for this code.
Note. A single element of an array is a scalar, thus it bears a $ -- so,
$cols[0]. If you were to extract multiple columns you could use an array slice, in which case the result is an array so it needs the #, for example #cols[0,2] is an array with the first and third element. This can then be assigned to a list, for example my ($c1, $c3) = #cols[0,2];.

Import parent-child XML data to Sql Server Tables

I have a XML like this:
<StateTree>
<State ID="01">
<Name>State1</Name>
<CityList>
<City ID="01" Order="1" CityGroup="1" CityBuild="1" GeoLocation="X">
<Name>City1</Name>
<Group>1</Group>
<AreaList>
<Area ID="01" GeoLocation="6">
<Name>Area1</Name>
</Area>
<Area ID="02" GeoLocation="6">
<Name>Area2</Name>
</Area>
</AreaList>
</City>
<City ID="02" Order="3" CityGroup="2" CityBuild="4" GeoLocation="5">
<Name>City2</Name>
<Group>2</Group>
<AreaList />
</City>
</CityList>
</State>
</StateTree>
and I want to convert it to tables like this:
State:
ID Name
01 State1
---------------------------------------------------
City:
ID Order CityGroup CityBuild GeoLocation Name State1
01 1 1 1 X City1 01
02 3 2 4 5 City2 01
---------------------------------------------------
AreaList:
ID GeoLocation Name CityID
01 6 Area1 01
02 6 Area2 01
How I can do this ?
thanks
I'm not going to write this for every field and all the inserts, but the following SQL should point you in the right direction:
declare #xml xml
set #xml =
'
<StateTree>
<State ID="01">
<Name>State1</Name>
<CityList>
<City ID="01" Order="1" CityGroup="1" CityBuild="1" GeoLocation="X">
<Name>City1</Name>
<Group>1</Group>
<AreaList>
<Area ID="01" GeoLocation="6">
<Name>Area1</Name>
</Area>
<Area ID="02" GeoLocation="6">
<Name>Area2</Name>
</Area>
</AreaList>
</City>
<City ID="02" Order="3" CityGroup="2" CityBuild="4" GeoLocation="5">
<Name>City2</Name>
<Group>2</Group>
<AreaList />
</City>
</CityList>
</State>
</StateTree>
'
--Select States
select
ID = s.value('#ID','varchar(10)'),
Name = s.value('Name[1]','varchar(100)')
from
#xml.nodes('/StateTree/State') x(s)
--Select Cities
select
ID = c.value('#ID','varchar(10)'),
Name = c.value('Name[1]','varchar(100)'),
StateID = c.value('../../#ID','varchar(10)')
from
#xml.nodes('/StateTree/State/CityList/City') x(c)
--Select Areas
select
ID = a.value('#ID','varchar(10)'),
Name = a.value('Name[1]','varchar(100)'),
CityID = a.value('../../#ID','varchar(10)')
from
#xml.nodes('/StateTree/State/CityList/City/AreaList/Area') x(a)

Resources