How to use XQuery to simulate STRING_AGG() (grouped string concatenation)? - sql-server

I have XML data which I need to convert to relational form. I use XQuery cause I don't know the number of address nodes. I'd like to get the whole address/adresses separeted by a comma. I guess I need to use LET clause but I'm still receiving an error.
Here's my code:
declare #xml as xml = '<root>
<Row>
<proceeding>
<signatures>V GU 86/18</signatures>
<signatures>V GUp 9/19</signatures>
<signatures>V GUp 8/19</signatures>
</proceeding>
<entity>
<info>
<cleaned_name>Kate Smith</cleaned_name>
</info>
<address>
<town>London </town>
<house_number>1 </house_number>
<flat_number>1</flat_number>
<street>Downing Street</street>
<zip_code>00-001</zip_code>
</address>
</entity>
<entity>
<info>
<cleaned_name>John Smith</cleaned_name>
</info>
<address>
<town>Washington </town>
<house_number>1</house_number>
<flat_number>1</flat_number>
<street>Pennsylvania Avenue</street>
<zip_code>00-001</zip_code>
</address>
</entity>
</Row>
</root>'
select
isnull(STUFF(a.x.query('for $s in entity/info/cleaned_name return <x>{concat(",",$s)}</x>').value('.','varchar(max)'),1,1,''),'') as 'Nazwa podmiotu'
,isnull(STUFF(a.x.query('for $s in proceeding/signatures return <x>{concat(",",$s)}</x>').value('.','varchar(max)'),1,1,''),'') as 'Sygnatura'
--,isnull(STUFF(a.x.query('for $s in entity let $L := $s/entity/address return <x>{concat(",",Address="{$s/Address}")}</x>').value('.','varchar(max)'),1,1,''),'')
from #xml.nodes('/root/Row') as a(x)
My desired outcome

Are you looking for this?
select
isnull(STUFF(a.x.query('for $s in entity/info/cleaned_name return <x>{concat(",",$s)}</x>').value('.','varchar(max)'),1,1,''),'') as 'Nazwa podmiotu'
,isnull(STUFF(a.x.query('for $s in proceeding/signatures return <x>{concat(",",$s)}</x>').value('.','varchar(max)'),1,1,''),'') as 'Sygnatura'
,isnull(STUFF(a.x.query('for $s in entity
return <x>
{
concat(", ",($s/address/zip_code/text())[1]," "
,($s/address/town/text())[1]," "
,($s/address/street/text())[1]," "
,($s/address/house_number/text())[1],"/"
,($s/address/flat_number/text())[1]
)
}
</x>').value('.','varchar(max)'),1,2,''),'')
from #xml.nodes('/root/Row') as a(x);
The result
Nazwa podmiotu Sygnatura AllAdresses
Kate Smith,John Smith V GU 86/18,V GUp 9/19,V GUp 8/19 00-001 London Downing Street 1 /1, 00-001 Washington Pennsylvania Avenue 1/1
UPDATE Multiple addresses and identical data
You can try this (according to your comment)
Your test data with one second address and one copied address:
declare #xml as xml = '<root>
<Row>
<proceeding>
<signatures>V GU 86/18</signatures>
<signatures>V GUp 9/19</signatures>
<signatures>V GUp 8/19</signatures>
</proceeding>
<entity>
<info>
<cleaned_name>Kate Smith</cleaned_name>
</info>
<address>
<town>London </town>
<house_number>1 </house_number>
<flat_number>1</flat_number>
<street>Downing Street</street>
<zip_code>00-001</zip_code>
</address>
<address>
<town>Yorkshire </town>
<house_number>1 </house_number>
<flat_number>1</flat_number>
<street>Morning Street</street>
<zip_code>00-999</zip_code>
</address>
</entity>
<entity>
<info>
<cleaned_name>John Smith</cleaned_name>
</info>
<address>
<town>Washington </town>
<house_number>1</house_number>
<flat_number>1</flat_number>
<street>Pennsylvania Avenue</street>
<zip_code>00-001</zip_code>
</address>
<address>
<town>Washington </town>
<house_number>1</house_number>
<flat_number>1</flat_number>
<street>Pennsylvania Avenue</street>
<zip_code>00-001</zip_code>
</address>
</entity>
</Row>
</root>'
--The query
select
isnull(STUFF(a.x.query('for $s in entity/info/cleaned_name return <x>{concat(",",$s)}</x>').value('.','varchar(max)'),1,1,''),'') as 'Nazwa podmiotu'
,isnull(STUFF(a.x.query('for $s in proceeding/signatures return <x>{concat(",",$s)}</x>').value('.','varchar(max)'),1,1,''),'') as 'Sygnatura'
,isnull(STUFF(a.x.query('for $s in entity/address
return
<x>{concat(", ",($s/zip_code/text())[1]," "
,($s/town/text())[1]," "
,($s/street/text())[1]," "
,($s/house_number/text())[1],"/"
,($s/flat_number/text())[1]
)}</x>')
.query('for $a in distinct-values(/x/text()) return $a').value('.','varchar(max)'),1,2,''),'')
from #xml.nodes('/root/Row') as a(x);
The idea in short:
We use the first XQuery to create a simple XML fragment like this
<x>, 00-001 London Downing Street 1 /1</x>
<x>, 00-999 Yorkshire Morning Street 1 /1</x>
<x>, 00-001 Washington Pennsylvania Avenue 1/1</x>
<x>, 00-001 Washington Pennsylvania Avenue 1/1</x>
With this we can use a second XQuery and place distinct-values() there.

Related

How to pass 'as' argument to gdbus?

This is a part of xml.
<method name="Mount">
<annotation name="org.chromium.DBus.Method.Kind" value="simple" />
<arg name="path" type="s" direction="in" />
<arg name="filesystem_type" type="s" direction="in" />
<arg name="options" type="as" direction="in" />
And this is what I want to implement exactly. (captured method call by dbus-monitor)
method call time=1675906347.661521 sender=:1.31 -> destination=org.chromium.CrosDisks serial=4557 path=/org/chromium/CrosDisks; interface=org.chromium.CrosDisks; member=Mount
string "/sys/devices/pci0000:00/0000:00:14.0/usb3/3-2/3-2:1.0/host1/target1:0:0/1:0:0:0/block/sdb/sdb1"
string ""
array [
string "rw"
string "mountlabel=MY-STORAGE"
]
And This is what I tried by gdbus.
gdbus call --system --dest org.chromium.CrosDisks --object-path /org/chromium/CrosDisks --method org.chromium.CrosDisks.Mount "/sys/devices/pci0000:00/0000:00:14.0/usb3/3-2/3-2:1.0/host1/target1:0:0/1:0:0:0/block/sdb/sdb1" “” [“rw” "mountlabel=MY-STORAGE"]
The I got errors.
(gdbus call:12863): GLib-CRITICAL **: 23:37:56.454: g_variant_new_string: assertion 'g_utf8_validate (string, -1, NULL)' failed
[Invalid UTF-8] Error parsing parameter 2: expected value:
\xe2\x80
^
(gdbus call:12863): GLib-CRITICAL **: 23:37:56.455: g_error_free: assertion 'error != NULL' failed
I think my expression for the type="as" is wrong.
Can someone help me?
I found the answer.
"['rw', 'mountlabel=MY-STORAGE']"

What is a better way to write this comparative PowerShell script?

I'm new to PowerShell and I'm sure I'm not using best practices here. I've been working on this PowerShell script to compare two XML files. I start out by looping through each of the XML files and throwing the data into PS objects:
Here are some samples of the XML data:
XML file 1
<RESULTS>
<ROW>
<COLUMN NAME="ATTR1"><![CDATA[123456ABCDEF]]></COLUMN>
<COLUMN NAME="ATTR2"><![CDATA[1.0.4.0]]></COLUMN>
<COLUMN NAME="ATTR3"><![CDATA[Google.com]]></COLUMN>
<COLUMN NAME="ATTR4"><![CDATA[Lorem ipsum]]></COLUMN>
<COLUMN NAME="ATTR5"><![CDATA[This is some text]]></COLUMN>
</ROW>
<ROW>
<COLUMN NAME="ATTR1"><![CDATA[123456ABCDEF]]></COLUMN>
<COLUMN NAME="ATTR2"><![CDATA[2.0.0.1]]></COLUMN>
<COLUMN NAME="ATTR3"><![CDATA[HelloWorld.com]]></COLUMN>
<COLUMN NAME="ATTR4"><![CDATA[Lorem ipsum]]></COLUMN>
<COLUMN NAME="ATTR5"><![CDATA[This is some text]]></COLUMN>
</ROW>
<ROW>
<COLUMN NAME="ATTR1"><![CDATA[123456ABCDEF]]></COLUMN>
<COLUMN NAME="ATTR2"><![CDATA[5.6.7.0]]></COLUMN>
<COLUMN NAME="ATTR3"><![CDATA[foo_foo_6 (2).org]]></COLUMN>
<COLUMN NAME="ATTR4"><![CDATA[Lorem ipsum]]></COLUMN>
<COLUMN NAME="ATTR5"><![CDATA[This is some text]]></COLUMN>
</ROW>
</RESULTS>
XML File 2
<applications xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<application>
<name>Google.com</name>
<version>1.2.0.0</version>
</application>
<application>
<name>HelloWorld.com</name>
<version>2.0.0.1</version>
</application>
<application>
<name>FOO_FOO.org</name>
<version>6.2.0.1</version>
</application>
</applications>
Creating arrays/objects filled with XML data
# assign all output from `foreach` loop to `$array1` - XML file 1
$array1 = foreach($row in $xmldata1.RESULTS.ROW){
# create new object with the pertinent details as property values
[pscustomobject]#{
Name = $row.COLUMN.Where{ $_.NAME -eq "ATTR3"}.'#cdata-section'
Version = $row.COLUMN.Where{ $_.NAME -eq "ATTR2"}.'#cdata-section'
}
}
# assign all output from `foreach` loop to `$array2` - XML file 2
$array2 = foreach($row in $xmldata2.applications.application){
# create new object with the pertinent details as property values
[pscustomobject]#{
Name = $row.name
Version = $row.version
}
}
This is the script I'm wondering how to write more effectively. It simply loops through $array1 and compares it with the data in $array2. If there is a match in the name, and a mismatch in the version, then it will store those values in a PS object.
Script I want to improve
#loop through array 1
for($i = 0; $i -le $array1.Length; $i++)
{
#loop through array 2
for($j = 0; $j -le $array2.Length; $j++)
{
#if file name in array 1 matches a name in array 2...
if (($array1.name[$i] -eq $array2.name[$j]) -or ($array1.name[$i].Substring(0, [Math]::Min($array1.name[$i].Length, 7)) -eq $array2.name[$j].Substring(0, [Math]::Min($array2.name[$i].Length, 7))))
{
#then, if that file names version does not match the version found in array 2...
if($array1.version[$i] -ne $array2.version[$j])
{
#create new object
[pscustomobject]#{
Name = $array1.name[$i]
Name2 = $array2.name[$j]
Version = $array1.version[$i]
Version2 = $array2.version[$j]
}
}
}
}
}
However, there are some names that don't match perfectly. So I use the -or operator and throw this line in my first if-statement to compare the first 7 characters of the file name in each array to see if there's some kind of match (which, I know there are):
($array1.name[$i].Substring(0, [Math]::Min($array1.name[$i].Length, 7)) -eq $array2.name[$j].Substring(0, [Math]::Min($array2.name[$i].Length, 7)))
Whenever I add that line though I get the following error for only some of the data objects in the arrays. The script will return some objects, but most of the time my console pane will be filled with the following error:
Error
You cannot call a method on a null-valued expression.
At line:8 char:13
+ if (($array1.name[$i] -eq $array2.name[$j]) -or ($array1 ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : InvalidOperation: (:) [], RuntimeException
+ FullyQualifiedErrorId : InvokeMethodOnNull
I don't even know what it's talking about. Cause when I extract that line and put actual indices in it, it works fine.
Example
if($array1.name[1020].Substring(0, [Math]::Min($array1.name[1020].Length, 7)) -eq $array2.name[2500].Substring(0, [Math]::Min($array2.name[2500].Length, 7))){
So, I'm stumped. Is there a better way to compare these two arrays and get a similar output?
I believe this could work and might be a more direct way to do it, this method would not require you to do the object construction of the first XML. Hopefully the inline comments explains the logic.
:outer foreach($i in $xml1.results.row) {
$name = $i.Column.Where{ $_.NAME -eq 'ATTR3' }.'#cdata-section'
$version = $i.Column.Where{ $_.NAME -eq 'ATTR2' }.'#cdata-section'
foreach($z in $xml2.applications.application) {
# check if they have the same version
$sameVersion = $version -eq $z.Version
# check if they have the same name
$sameName = $name -eq $z.Name
# if both conditions are `$true` we can skip this and continue with
# next item of outer loop
if($sameVersion -and $sameName) {
continue outer
}
# if their first 7 characters are the same but they're NOT the same version
if([string]::new($name[0..6]) -eq [string]::new($z.Name[0..6]) -and -not $sameVersion) {
[pscustomobject]#{
Name = $name
Name2 = $z.Name
Version = $version
Version2 = $z.Version
}
}
}
}
The result of this would be:
Name Name2 Version Version2
---- ----- ------- --------
Google.com Google.com 1.0.4.0 1.2.0.0
foo_foo_6 (2).org FOO_FOO.org 5.6.7.0 6.2.0.1
See Using a labeled continue in a loop which describes and explains the use of continue outer in this example.

bash to parse XML into multidimensional array

I have a XML file which I want to parse into bash variables/arrays.
I have limited linux commands (busybox) available since I am working on a NAS box.
My XML file looks like this:
<?xml version="1.0" encoding="UTF-8"?> .
<WEBCAMS>
<CAM>
<DESCRIPTION>description for cam 1</DESCRIPTION>
<URL>http://myURLtoWebcam1/cam1/pic.jpg</URL>
<FILENAME>filename1</FILENAME>
</CAM>
<CAM>
<DESCRIPTION>description for cam 2</DESCRIPTION>
<URL>http://myURLtoWebcam2/cam2/pic.jpg</URL>
<FILENAME>filename2</FILENAME>
</CAM>
</WEBCAMS>
my bash script so far:
#!/bin/sh
rdom () { local IFS=\> ; read -d \< E C ;}
while rdom; do
if [[ $E = DESCRIPTION ]]; then
counter=$((counter+1))
declare cam$counter="$C"
fi
done < webcams.xml
I want to get the XML content like the following:
echo "Cam1 description: ${cam1[0]}"; ## should show: description for cam 1
echo "Cam1 URL: ${cam1[1]}"; ## should show: http://myURLtoWebcam1/cam1/pic.jpg
echo "Cam1 filename: ${cam1[2]}"; should show: filename1
echo "Cam2 description: ${cam2[0]}"; ## should show: description for cam 2
echo "Cam2 URL: ${cam2[1]}"; ## should show: http://myURLtoWebcam1/cam2/pic.jpg
echo "Cam2 filename: ${cam2[2]}"; ## should show: filename2
So far I am only able to read the "DESCRIPTION" fields into bash variables.
Any idea how to get the other fields "URL" and "FILENAME" into my arrays/variables? The so far found solutions did not fit or could not be modified to my needs due to the limited Linux commands on my NAS.
If XSLTPROC is available you can use it - bonus its a real XML parser.
> xsltproc transform.xsl webcams.xml
Cam1 description: description for cam 1
Cam1 URL: http://myURLtoWebcam1/cam1/pic.jpg
Cam1 filename: filename1
Cam2 description: description for cam 2
Cam2 URL: http://myURLtoWebcam2/cam2/pic.jpg
Cam2 filename: filename2
Where transform.xsl is
<?xml version="1.0"?>
<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" version="1.0">
<xsl:output method="text" omit-xml-declaration="yes" indent="no"/>
<xsl:strip-space elements="*"/>
<xsl:template match="CAM">
<xsl:variable name="i" select="position()" />
Cam<xsl:value-of select="$i"/> description: <xsl:value-of select="DESCRIPTION"/>
Cam<xsl:value-of select="$i"/> URL: <xsl:value-of select="URL"/>
Cam<xsl:value-of select="$i"/> filename: <xsl:value-of select="FILENAME"/>
</xsl:template>
<xsl:template match="/WEBCAMS"><xsl:apply-templates select="*"/>
<xsl:text>
</xsl:text>
</xsl:template>
</xsl:stylesheet>
This seems to work ... but still the 1,5D dimensional array or the way to escape variables makes me headache - see below problems within a FOR loop:
#!/bin/sh
rdom () { local IFS=\> ; read -d \< E C ;}
while rdom; do
if [[ $E = DESCRIPTION ]]; then
counter0=$((counter0+1))
declare -a cam$((counter0))[0]="$C"
fi
if [[ $E = URL ]]; then
counter1=$((counter1+1))
declare -a cam$((counter1))[1]="$C"
fi
if [[ $E = FILENAME ]]; then
counter2=$((counter2+1))
declare -a cam$((counter2))[2]="$C"
fi
done < webcams.xml
echo "Cam1 description: ${cam1[0]}";
echo "Cam1 URL: ${cam1[1]}";
echo "Cam1 filename: ${cam1[2]}";
echo "Cam2 description: ${cam2[0]}";
echo "Cam2 URL: ${cam2[1]}";
echo "Cam2 filename: ${cam2[2]}";
But it is still impossible to get the values within a FOR loop like:
for (( c=1; c<=$counter0; c++ ))
do
var=cam$c;
echo "Cam$c description: ${!var[0]}";
echo "Cam$c URL: ${!var[1]}";
echo "Cam$c filename: ${!var[2]}";
done

Get xmllint xpath nodes to arrays

I have NMAP-output xml-file like this:
<host><status state="down" reason="no-response" reason_ttl="0"/>
<address addr="192.168.1.1" addrtype="ipv4"/>
</host>
<taskbegin task="SYN Stealth Scan" time="1457545799"/>
<taskend task="SYN Stealth Scan" time="1457545841" extrainfo="1600 total ports"/>
<host starttime="1457545794" endtime="1457545839"><status state="up" reason="echo-reply" reason_ttl="243"/>
<address addr="192.168.1.2" addrtype="ipv4"/>
<hostnames>
<hostname name="192-168-1-2.liibalaaba.com" type="PTR"/>
</hostnames>
<ports><extraports state="closed" count="100">
<extrareasons reason="resets" count="100"/>
</extraports>
</ports>
<times srtt="6454" rttvar="163" to="100000"/>
</host>
<host starttime="1457545794" endtime="1457545837"><status state="up" reason="echo-reply" reason_ttl="51"/>
<address addr="192.168.1.3" addrtype="ipv4"/>
<hostnames>
<hostname name="192-168-1-3.liibalaaba.com" type="PTR"/>
</hostnames>
<ports><extraports state="filtered" count="98">
<extrareasons reason="no-responses" count="98"/>
</extraports>
<port protocol="tcp" portid="80"></port>
<port protocol="tcp" portid="443"></port>
</ports>
<times srtt="6378" rttvar="191" to="100000"/>
</host>
<host starttime="1457545794" endtime="1457545841"><status state="up" reason="echo-reply" reason_ttl="115"/>
<address addr="192.168.1.4" addrtype="ipv4"/>
<hostnames>
<hostname name="192-168-1-4.liibalaaba.com" type="PTR"/>
</hostnames>
<ports><extraports state="filtered" count="97">
<extrareasons reason="no-responses" count="97"/>
</extraports>
<port protocol="tcp" portid="80"></port>
<port protocol="tcp" portid="81"></port>
<port protocol="tcp" portid="443"></port>
</ports>
<times srtt="6417" rttvar="113" to="100000"/>
</host>
How I can put only the hosts that have open ports to an array and then to readable format? (Do I even need an array?)
Something like this:
address:192.168.1.3 [0,0]
open ports:80 [0,1], 443 [0,2]
address:192.168.1.4 [1,0]
open ports:80[1,1], 81 [1,2], 443 [1,3]
Now i'm using xmllint xpath combo, but it just splits out every address that has open ports to same line:
`xmllint --xpath "//*[local-name()='host']/*[local-name()='address']/#addr | //*[local-name()='ports']/*[local-name()='port']/#portid" nmap-scan.xml > result.txt`
Thanks for help!

How i set variable in XSL in for

Sry, when i don't i find answare, but i don't make correct question. I have APEX 4 and trying create report from multiarray select. For example data is:
Name1 Value1
Name1 Value2
Name1 Value3
Name1 Value4
Name2 Value5
Name2 Value6
Name2 Value7
and I need output in report:
-Name1
--Value1
--Value2
--Value3
--Value4
-Name2
--Value5
--Value6
--Value7
I need do it in XSL similiar as php:
$x_key= '';
foreach($arr as $key=>$value)
{
if ($x_key != $key) {
$x_key = $key;
echo "-$key\n";
}
echo "--$value\n";
}
My code possible is (but don't do it what I want):
<xsl:variable name="NAMEOLD" select="X" />
<xsl:for-each select="//ROWSET1_ROW[position()]">
<xsl:variable name="NAME" select="NAME_NUMBER" />
<xsl:choose>
<xsl:when test="NAME!=NAMEOLD">
<xsl:variable name="NAMEOLD" select="NAME" />
<fo:table-row>
<fo:table-cell xsl:use-attribute-sets="cell-transparent" >
<fo:block xsl:use-attribute-sets="align-left txt" keep-together.within-page="always">
<xsl:value-of select="NAME_NUMBER"/>
</fo:block>
</fo:table-cell>
</fo:table-row>
</xsl:when>
</xsl:choose>
<fo:table-row>
<fo:table-cell xsl:use-attribute-sets="cell-transparent" >
<fo:block xsl:use-attribute-sets="align-left txt" keep-together.within-page="always">
<xsl:value-of select="VALUE"/>
</fo:block>
</fo:table-cell>
</fo:table-row>
</xsl:for-each>
Thanks for any clue or answer
Stoupa101
You haven't shown your input XML. The simplest solution is along the following lines (note this requires XSLT 2.0):
<xsl:for-each-group select="ROW" group-adjacent="name">
<h1><xsl:value-of select="current-grouping-key"></h1>
<xsl:for-each select="current-group()"/>
<p><xsl:value-of select="value"/></p>
</xsl:for-each-group>
Don't try to do it the PHP way - there's no point writing low-level code when you have a high-level language.

Resources