Issue:
I have a single .xml file containing 13.5k of data sets and need to import it into SSMS. Unfortunatly, it contain a style I never met before; first a column declaration part, then the data part without any specific column names. With thus, I have issues to catch the needed fields. On top of that the .xml may even be corrupt (incorrect hierarchical structure).
Xml:
<?xml version="1.0" encoding="UTF-8"?>
<Root>
<DMSContent format="LOL"/>
<Archive name="Adressdossier" id="52" osguid="43AAEC21AC6C40F1BEDB34D92512ED84"/>
<ObjectType name="Dokument" internal_name="CitizenFileDocument" id="262216" osguid="F287C984EB9E48BEA280BA46C305567C" type="DOCUMENT" modul="MULTIDOC"/>
<Rowset>
<Columns>
<Column name="Salutation" type="TEXT" ostype="X" size="50" otype="FOLDER"/>
<Column name="Name" type="TEXT" ostype="X" size="200" otype="FOLDER"/>
<Column name="FirstName" type="TEXT" ostype="X" size="100" otype="FOLDER"/>
<Column name="StreetNo" type="TEXT" ostype="X" size="100" otype="FOLDER"/>
<Column name="City" type="TEXT" ostype="X" size="150" otype="FOLDER"/>
<Column name="ZIP" type="TEXT" ostype="X" size="50" otype="FOLDER"/>
<Column name="Country" type="TEXT" ostype="X" size="50" otype="FOLDER"/>
<Column name="Birthday" type="DATE" ostype="D" size="50" otype="FOLDER"/>
<Column name="Filename" type="INTEGER" ostype="9" size="100" otype="FOLDER"/>
</Columns>
</Rowset>
<Rows>
<Row id="2538">
<Value>Mrs</Value>
<Value>Doe</Value>
<Value>Jane</Value>
<Value>Main Street 5</Value>
<Value>Ghost Town</Value>
<Value>5315</Value>
<Value>Switzerland</Value>
<Value>12.12.2017</Value>
<Value>jp4_B025DF7DBAFC49879103ECB8AE59C3A2.docx</Value>
</Row>
<Row id="2579">
<Value>Mr</Value>
<Value>Ding</Value>
<Value>Chavez</Value>
<Value>Sun Boulevard 3a</Value>
<Value>Alien City</Value>
<Value>4586</Value>
<Value>Germany</Value>
<Value>01.01.1980</Value>
<Value>jp4_DCA9345C93E84F1697668E6ACDC596C9.docx</Value>
</Row>
<Row id="2580">
<Value>Mr</Value>
<Value>Dale</Value>
<Value>Dick</Value>
<Value>Beach Avenue 13</Value>
<Value>Zombie Village</Value>
<Value>9513</Value>
<Value>Italy</Value>
<Value>09.11.1911</Value>
<Value>jp4_5DDBF2A05BD0421A8C53B0CC4EB64232.doc</Value>
</Row>
</Rows>
</Root>
The usually used MS-Sql code snippet, of course not working for this type of .xml-structure:
set ansi_nulls on;
declare #xmlfile xml;
select #xmlfile = bulkcolumn
from openrowset(bulk 'C:\Meta.xml', single_blob) x;
select
id = c.value('#id', 'int'),
Salutation = c.value('(Column[#k="Salutation"]/#v)[1]', 'varchar(60)'),
[Name] = c.value('(Column[#k="name"]/#v)[1]', 'varchar(100)'),
Birthday = c.value('(Column[#k="Birthday"]/#v)[1]', 'date'),
[Filename] = c.value('(Column[#k="Filename"]/#v)[1]', 'varchar(100)')
into #Meta --
from #xmlfile.nodes('/root/rows') as T(c);
set ansi_nulls off;
Thank you in advance for any help!
SQL Server doesn't support fn::position() or preceding-sibling:: syntaxes. But you can use a hack involving << to get the position of each node.
So we calculate the position of each Column node, then push those values into the Value lookups
SELECT
id = x2.Row.value('#id', 'int'),
Salutation = x2.Row.value('(Value[sql:column("ColIndex.Salutation")]/text())[1]', 'varchar(60)'),
[Name] = x2.Row.value('(Value[sql:column("ColIndex.Name" )]/text())[1]', 'varchar(100)'),
Birthday = x2.Row.value('(Value[sql:column("ColIndex.Birthday" )]/text())[1]', 'date'),
[Filename] = x2.Row.value('(Value[sql:column("ColIndex.Filename" )]/text())[1]', 'varchar(100)')
FROM #xml.nodes('/Root/Rowset/Columns') x1(Col)
CROSS APPLY (
SELECT
Salutation = x1.Col.value('let $c:= Column[#name="Salutation"][1] return count(Column[. << $c]) + 1', 'int'),
[Name] = x1.Col.value('let $c:= Column[#name="Name"] [1] return count(Column[. << $c]) + 1', 'int'),
Birthday = x1.Col.value('let $c:= Column[#name="Birthday"] [1] return count(Column[. << $c]) + 1', 'int'),
[Filename] = x1.Col.value('let $c:= Column[#name="Filename"] [1] return count(Column[. << $c]) + 1', 'int')
) ColIndex
CROSS APPLY #xml.nodes('/Root/Rows/Row') x2(Row);
db<>fiddle
If you want to keep your current approach of importing the file, you can, with the following changes:
set ansi_nulls on;
declare #xmlfile xml;
select #xmlfile = bulkcolumn
from openrowset(bulk 'C:\Meta.xml', single_blob) x;
select
id = c.value('#id', 'int'),
Salutation = c.value('(Value[count(/Root/Rowset/Columns/Column[#name="Salutation"]/preceding-sibling::*) + 1]/text())[1]', 'varchar(60)'),
[Name] = c.value('(Value[count(/Root/Rowset/Columns/Column[#name="Name"]/preceding-sibling::*) + 1]/text())[1]', 'varchar(100)'),
Birthday = c.value('(Value[count(/Root/Rowset/Columns/Column[#name="Birthday"]/preceding-sibling::*) + 1]/text())[1]', 'date'),
[Filename] = c.value('(Value[count(/Root/Rowset/Columns/Column[#name="Filename"]/preceding-sibling::*) + 1]/text())[1]', 'varchar(100)')
into #Meta --
from #xmlfile.nodes('/Root/Rows/Row') as T(c);
set ansi_nulls off;
This finds the right <Value> position by looking up the <Column> of the given name and figuring out how many columns precede it. Not pretty, but effective.
If this is a one-off and/or you're certain of the column order, you can of course access the values directly.
Birthday = c.value('(Value[8]/text())[1]', 'varchar(60)'),
Related
I am working on a table in sql server which stores xml file in a column. In that xml file I am doing some changes. The XML file looks like:
<Report version=1>
<Title>
<Student>
<InputNumber type="int" min="0" max="100" name="age" description="Age
of student">
<Value>20</Value>
</InputNumber>
<InputNumber type="int" min="0" max="100" name="height"
description="height of student">
<Value>170</Value>
</InputNumber>
</Student>
</Title>
</Report>
I understand the usage of modify function for updating attributes or text present between tags as:
UPDATE student
SET dataxml.modify('replace value of (/Report/#version)[1] with "2"')
WHERE id=10
or
UPDATE student
SET dataxml.modify('replace value of (/Report/Title/Student/InputNumber[1]/Value[1]/text())[1] with "21"')
WHERE id=10
But now I want to replace entire tag with another tag i.e.
<InputNumber type="int" min="0" max="100" name="height"
description="height of student">
<Value>170</Value>
</InputNumber>
with
<InputText name="height"
description="height of student">
<Value>170 cm</Value>
</InputText>
I found something on internet like this and tried.
Update Student
set dataxml = replace(cast(dataxml as nvarchar(max)),'/Report/Title/Student/InputNumber[2]>','InputText>')
WHERE id=10
It says updated successfully. But I don't see the change in XML.
How can I do that?
First of all: Your XML is not valid. The attribute version=1 must be version="1".
Second: The verb tag is just one markup like <Student> or </Student>, but the whole node with attributes and nested sub-nodes is called node or - as a special type of node - element.
Now to your issue:
We need a declared table to simulate your issue:
DECLARE #student TABLE(ID INT IDENTITY, dataxml XML);
INSERT INTO #student VALUES
(N'<Report version="1">
<Title>
<Student>
<InputNumber type="int" min="0" max="100" name="age" description="Age of student">
<Value>20</Value>
</InputNumber>
<InputNumber type="int" min="0" max="100" name="height" description="height of student">
<Value>170</Value>
</InputNumber>
</Student>
</Title>
</Report>');
--This is the new element we want to insert (better: want to use to replace another)
DECLARE #newElement XML=
N'<InputText name="height" description="height of student">
<Value>170 cm</Value>
</InputText>';
--approach one calls `.modify()` twice:
UPDATE #student SET dataxml.modify('insert sql:variable("#newElement") after (/Report/Title/Student/InputNumber[#name="height"])[1]');
UPDATE #student SET dataxml.modify('delete (/Report/Title/Student/InputNumber[#name="height"])[1]');
SELECT * FROM #student;
--approach two uses FLWOR-XQuery
UPDATE #student SET dataxml=dataxml.query('<Report version="{/Report/#version}">
{<Title>
<Student>
{
for $elmt in /Report/Title/Student/*
return
if(local-name($elmt)="InputNumber" and $elmt[#name="height"]) then
<InputText name="height" description="height of student">
<Value>{$elmt/Value/text()} cm</Value>
</InputText>
else
$elmt
}
</Student>
</Title>}
</Report>');
Both ideas in short:
1) We insert the new element right after the one which should be replaced and remove it in a separate step.
2) We re-create the XML via XQuery by running through the inner list of nodes within <Student> and insert the new content instead of the existing node.
I want to have a gridview which contains Members.firstName, Members.LastName, Team.TeamName. Team.TeamName should be a dropdown containing the possible TeamName's.
My Tables:
Members
--------
MemberID BIGINT
FirstName NVARCHAR(50)
LastName NVARCHAR(50)
TeamID BIGINT
Team
---------
TeamID BIGINT
TeamName NVARCHAR(50)
My Select:
SELECT Members.FirstName, Members.LastName, Team.TeamName
FROM Members
INNER JOIN Team ON Members.TeamId = Team.TeamId
My Update:
CREATE PROCEDURE updateTeamMembers
(
#TeamId BIGINT,
#FirstName NVARCHAR(50),
#LastName NVARCHAR(50),
#TeamName NVARCHAR(50)
)
AS
BEGIN
UPDATE Members SET FirstName=#FirstName, LastName=#LastName WHERE TeamId=#TeamId
UPDATE Team SET TeamName=#TeamName WHERE TeamId=#TeamId
END
RETURN
I believe the error to be with the update procedure, I get this error:
Procedure or function updateTeamMembers has too many arguments specified.
My HTML5:
<asp:SqlDataSource ID="SqlDataSource1" runat="server" ConflictDetection="CompareAllValues" ConnectionString="<%$ ConnectionStrings:RegistrationConnectionString %>" DeleteCommand="DELETE FROM [Members] WHERE [MemberId] = #original_MemberId AND (([FirstName] = #original_FirstName) OR ([FirstName] IS NULL AND #original_FirstName IS NULL)) AND (([LastName] = #original_LastName) OR ([LastName] IS NULL AND #original_LastName IS NULL)) AND (([TeamId] = #original_TeamId) OR ([TeamId] IS NULL AND #original_TeamId IS NULL))" InsertCommand="INSERT INTO [Members] ([FirstName], [LastName], [TeamId]) VALUES (#FirstName, #LastName, #TeamId)" OldValuesParameterFormatString="original_{0}" SelectCommand="SELECT Members.FirstName, Members.LastName, Team.TeamName FROM Members INNER JOIN Team ON Members.TeamId = Team.TeamId" UpdateCommand="updateTeamMembers" UpdateCommandType="StoredProcedure">
<DeleteParameters>
<asp:Parameter Name="original_MemberId" Type="Int32" />
<asp:Parameter Name="original_FirstName" Type="String" />
<asp:Parameter Name="original_LastName" Type="String" />
<asp:Parameter Name="original_TeamId" Type="Int64" />
</DeleteParameters>
<InsertParameters>
<asp:Parameter Name="FirstName" Type="String" />
<asp:Parameter Name="LastName" Type="String" />
<asp:Parameter Name="TeamId" Type="Int64" />
</InsertParameters>
<UpdateParameters>
<asp:Parameter Name="TeamId" Type="Int64" />
<asp:Parameter Name="FirstName" Type="String" />
<asp:Parameter Name="LastName" Type="String" />
<asp:Parameter Name="TeamName" Type="String" />
</UpdateParameters>
</asp:SqlDataSource>
<asp:GridView ID="GridView1" runat="server" AutoGenerateColumns="False" BackColor="White" BorderColor="#CCCCCC" BorderStyle="None" BorderWidth="1px" CellPadding="3" DataSourceID="SqlDataSource1" Height="217px" Width="596px">
<Columns>
<asp:BoundField DataField="FirstName" HeaderText="First Name" SortExpression="FirstName" />
<asp:BoundField DataField="LastName" HeaderText="Last Name" SortExpression="LastName" />
<asp:BoundField DataField="TeamName" HeaderText="Team Name" SortExpression="TeamName" />
<asp:CommandField ShowEditButton="True" />
</Columns>
Update command when I configure through datasource (Doesn't update):
UPDATE [Members] SET [FirstName] = #FirstName, [LastName] = #LastName, [TeamId] = #TeamId WHERE [MemberId] = #original_MemberId AND (([FirstName] = #original_FirstName) OR ([FirstName] IS NULL AND #original_FirstName IS NULL)) AND (([LastName] = #original_LastName) OR ([LastName] IS NULL AND #original_LastName IS NULL)) AND (([TeamId] = #original_TeamId) OR ([TeamId] IS NULL AND #original_TeamId IS NULL))
When you execute the stored procedure, you specify the arguments comma separated, something like this:
EXEC updateTeamMembers #ParamValue1, #ParamValue2, #ParamValue3, #ParamValue4, #ParamValue5...
First of all, it's a good practice to specify the arguments as key value pair, where the key is the stored procedure parameter name and the value is the value you want to pass to that parameter. So, the above query will look like this:
EXEC updateTeamMembers
#TeamId = #ParamValue1,
#FirstName = #ParamValue2,
#LastName = #ParamValue3,
#TeamName = #ParamValue4
If you do this, will observe that you don't have a fifth parameter.
To help you more, you should paste the code from VS.
declare #x xml =
'<Detials xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<Tests>
<Test Name="Test1" TotalMarks="100">95</Test>
<Test Name="Test2" TotalMarks="200">65</Test>
<Test Name="Test3" TotalMarks="150">95</Test>
<Test Name="Test4" TotalMarks="150"></Test>
</Tests>
<Tests>
<Test Name="Test1" TotalMarks="100">95</Test>
<Test Name="Test2" TotalMarks="200">65</Test>
<Test Name="Test3" TotalMarks="150">95</Test>
<Test Name="Test4" TotalMarks="150"></Test>
</Tests>
</Detials>'
When i queried like this
SELECT STUFF(
#x.query('for $a in (*:Detials/Tests/Test/#Name)
return <a>{concat(",", $a)}</a>')
.value('.', 'NVARCHAR(MAX)'),
1, 1, '') AS ListOfName
I get like this
ListofName
Test1,Test2,Test3,Test4,Test1,Test2,Test3,Test4
But Want to shred the xml based on the <Test> Which can give the result like this below
ListofName
Test1,Test2,Test3,Test4
Test1,Test2,Test3,Test4
Please help me here to shred original XML into separate rows before string concatenation with query() and value()
Thanks in Advance ,Jayendran
Try this solution:
declare #x xml =
'<Detials xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<Tests>
<Test Name="Test1" TotalMarks="100">95</Test>
<Test Name="Test2" TotalMarks="200">65</Test>
<Test Name="Test3" TotalMarks="150">95</Test>
<Test Name="Test4" TotalMarks="150"></Test>
</Tests>
<Tests>
<Test Name="Test1" TotalMarks="100">95</Test>
<Test Name="Test2" TotalMarks="200">65</Test>
<Test Name="Test3" TotalMarks="150">95</Test>
<Test Name="Test4" TotalMarks="150"></Test>
</Tests>
</Detials>'
-- Edit 1
SELECT XmlTable.OriginalID, ROW_NUMBER() OVER(ORDER BY x.XmlCol) AS RowNum, y.ListOfName
FROM (SELECT 1 AS OriginalID, #x AS XmlCol) AS XmlTable -- Edit 2
CROSS APPLY XMLTable.XmlCol.nodes('*:Detials/Tests') AS x(XmlCol)
CROSS APPLY(
SELECT STUFF(x.XmlCol.query('for $a in (Test/#Name) return <a>{concat(",", $a)}</a>').value('.', 'NVARCHAR(MAX)'), 1, 1, '')
) AS y(ListOfName)
-- End of Edit 1
Demo
Using XPath in T-SQL I am trying to get an attribute value from the options list where the Id to choose the right list item is found in the Value element.
Any help would be appreciated.
declare #myTable table (pk int primary key identity(1,1), myXML xml)
insert into #myTable values ('
<Fields>
<Field ID="1111">
<Description>How Now Brown Cow</Description>
<Value>3</Value>
<Options>
<Options>
<Option OptionContent="Select one" OptionID="-1" />
<Option OptionContent="Mars" OptionID="1" />
<Option OptionContent="Pluto" OptionID="2" />
<Option OptionContent="Saturn" OptionID="3" />
</Options>
</Options>
</Field>
<Field ID="2222">
<Description>Foo Bar</Description>
<Value>2</Value>
<Options>
<Options>
<Option OptionContent="Select one" OptionID="-1" />
<Option OptionContent="Coffee" OptionID="1" />
<Option OptionContent="Tea" OptionID="2" />
<Option OptionContent="Water" OptionID="3" />
<Option OptionContent="Juice" OptionID="4" />
<Option OptionContent="Water" OptionID="5" />
</Options>
</Options>
</Field>
</Fields>
')
select
myField.ref.value('#ID', 'smallint') as [ID]
,myField.ref.value('(./Description)[1]', 'nvarchar(10)') as [Description]
,myField.ref.value('(./Value)[1]', 'int') as [Value]
,myField.ref.value('(./Options/Options/Option[#OptionID="-1"]/#OptionContent)[1]', 'nvarchar(10)') as [SelectedDescription]
from #myTable c
cross apply c.myXML.nodes('/Fields/Field') myField(ref)
ID Description Value Actual Expected
------ ----------- ----------- ---------- --------
1111 How Now Br 3 NULL Saturn
2222 Foo Bar 2 NULL Tea
Change
(./Options/Options/Option[#OptionID="-1"]/#OptionContent)[1]
to
let $id := ./Value[1] return (./Options/Options/Option[#OptionID=$id]/#OptionContent)[1]
So your query should be
select
myField.ref.value('#ID', 'smallint') as [ID]
,myField.ref.value('(./Description)[1]', 'nvarchar(10)') as [Description]
,myField.ref.value('(./Value)[1]', 'int') as [Value]
,myField.ref.value('let $id := ./Value[1] return (./Options/Options/Option[#OptionID=$id]/#OptionContent)[1]', 'nvarchar(10)') as [SelectedDescription]
from #myTable c
cross apply c.myXML.nodes('/Fields/Field') myField(ref)
I need to query Xml data that has multiple REPORT tag elements. It needs to be filtered to return only the rows where the REPORTID is equal to a given name. I've tried to perform this filter with no luck. Could someone point me in the right direction here using the Sql Server Xml functions?
Basically, I'm looking for my result set to return as a table and look like:
ID
------
1
2
3
Given the following, how would I select out on the rows for the REPORT where the REPORTID (/TEST/REPORT/TITLE[#ReportId = "Report One"]) is equal to 'Report One'?
DECLARE #Xml XML, #ReportId VARCHAR(200);
SET #ReportId = 'Report One';
SET #Xml = '
<TEST>
<REPORT ReportType="Type One">
<TITLE ReportId="Report One">
<TITLE1>Title One</TITLE1>
</TITLE>
<HEADER>
<Run_Date OrigName="Run Date">4/10/2012</Run_Date>
</HEADER>
<BODY>
<TABLE1>
<DATA />
<ROW>
<ID>1</ID>
</ROW>
<ROW>
<ID>2</ID>
</ROW>
<ROW>
<ID>3</ID>
</ROW>
</TABLE1>
</BODY>
</REPORT>
<REPORT ReportType="Type Two">
<TITLE ReportId="Report Two">
<TITLE1>Title Two</TITLE1>
</TITLE>
<HEADER>
<Run_Date OrigName="Run Date">4/10/2012</Run_Date>
</HEADER>
<BODY>
<TABLE1>
<DATA />
<ROW>
<ID>4</ID>
</ROW>
<ROW>
<ID>5</ID>
</ROW>
<ROW>
<ID>6</ID>
</ROW>
</TABLE1>
</BODY>
</REPORT>
</TEST>';
select I.N.value('.', 'int') as ID
from #Xml.nodes('TEST/REPORT') as R(N)
cross apply R.N.nodes('BODY/TABLE1/ROW/ID') as I(N)
where R.N.exist('TITLE[#ReportId = sql:variable("#ReportId")]') = 1
I realize this is almost three years old, but I couldn't resist. The CROSS APPLY is unnecessary if you expand the XPATH expression in nodes().
SELECT ID = c.value('.', 'int')
FROM #Xml.nodes('/TEST/REPORT[TITLE/#ReportId=sql:variable("#ReportId")]/BODY/TABLE1/ROW/ID') x(c)