Import xml key value pair to sql server? - sql-server

I have the XML data below that represents one row but can't seem to get this imported into sql server without completely rewriting the xml file to look like the second block of code at the very bottom. I have thousands of small xml files in the first format I need to start processing and importing into a sql table. Ideally if I don't use some custom scripting to rewrite the xml I'm thinking I could import into a temp table and use pivot or transposing to get the attributes into cells in a row.
<ping>
<feed Scale="4.0" resolution="67.58859099656746">
<beta name="my_misc_beta" totalRecords="1">
<row>
<column>
<key>CUSTOMER ID</key>
<value>123456</value>
</column>
<column>
<key>CUSTOMER NAME</key>
<value>Johnys Bike Shop</value>
</column>
<column>
<key>REGION NAME</key>
<value>Cool Area</value>
</column>
<column>
<key>CUSTOMER CATEGORY</key>
<value>Bike Shop</value>
</column>
<column>
<key>CUSTOMER DESCRIPTION</key>
<value>coolest bike shop</value>
</column>
<column>
<key>CUSTOMER STATUS</key>
<value>Current</value>
</column>
<column>
<key>CUSTOMER CONTACT</key>
<value>johnny#bikeshop.net</value>
</column>
</row>
</beta>
</feed>
</ping>
This xml below easily imports into sql server using ssis and looping through the directory. But I had to hand rewrite this by hand for testing. Is there a way with maybe c# or another language to take the inside text and write them to element tags etc in ssis. Compare the two xml docs and you'll see that the above code will not import diectly to a row.
<ping>
<feed Scale="4.0" resolution="67.58859099656746">
<beta name="my_misc_beta" totalRecords="1">
<row>
<column>
<CUSTOMER_ID>123456</CUSTOMER_ID>
<CUSTOMER_NAME>Johnys Pedal Shop</CUSTOMER_NAME>
<REGION_NAME>Cool Area</REGION_NAME>
<CUSTOMER_CATEGORY>Bike Shop</CUSTOMER_CATEGORY>
<CUSTOMER_DESCRIPTION>coolest bike shop</CUSTOMER_DESCRIPTION>
<CUSTOMER_STATUS>Current</CUSTOMER_STATUS>
<CUSTOMER_CONTACT>johnny#bikeshop.net</CUSTOMER_CONTACT>
</column>
</row>
</beta>
</feed>
</ping>

Don't know anything about what XML capabilities there are in SSIS but you can do what you want using a regular query.
First you need to shred the XML on the row node using nodes() Method (xml Data Type) to get one row in the resul for each row in the XML.
Then you use value() Method (xml Data Type) to extract the column value you want. You could fetch the values by column number but it is safer to check the key value in a predicate before fetching the value.
select T.X.value('(column[key/text() = "CUSTOMER ID"]/value/text())[1]', 'int') as CustomerID,
T.X.value('(column[key/text() = "CUSTOMER NAME"]/value/text())[1]', 'nvarchar(100)') as CustomerName,
T.X.value('(column[key/text() = "REGION NAME"]/value/text())[1]', 'nvarchar(100)') as RegionName,
T.X.value('(column[key/text() = "CUSTOMER CATEGORY"]/value/text())[1]', 'nvarchar(100)') as CustomerCategory,
T.X.value('(column[key/text() = "CUSTOMER DESCRIPTION"]/value/text())[1]', 'nvarchar(100)') as CustomerDescription,
T.X.value('(column[key/text() = "CUSTOMER STATUS"]/value/text())[1]', 'nvarchar(100)') as CustomerStatus,
T.X.value('(column[key/text() = "CUSTOMER CONTACT"]/value/text())[1]', 'nvarchar(100)') as CustomerContact
from #XML.nodes('/ping/feed/beta/row') as T(X)
SQL Fiddle

Related

SQL Server Update Column Name XML Node

I stored my data in XML format in SQL server, in this way
**<Column Name="GROSS" DataType="float" Value="939760" />**
but somehow one column name (GROSS) in my XML data is stored twice, now I want to remove/rename one of them.
Below are screenshots of my database view:
Table view
XML view
This it what I tried, but it only changed the value, it did not rename the column name.
update Aquara7bc772839.EmpTransaction set TransactionFieldDetails.modify('replace value of (/PayDetails/Column[#Name="LEV_ENCASHRATE"]/#Value)[1] with "796.00"') WHERE Id = 276620;
I have highlighted my column names in above image link please check
I want to remove or rename one column.
You can use the XML.modify() method with a delete node instruction...
declare #EmpTransaction table (
Id int not null,
TransactionFieldDetails xml
);
insert #EmpTransaction values (
276620,
N'<PayDetails>
<Column Name="GROSS" DataType="float" Value="939760" />
<Column Name="GROSS" DataType="float" Value="939760" />
</PayDetails>'
);
update #EmpTransaction
set TransactionFieldDetails.modify('delete /PayDetails/Column[#Name="GROSS"][2]')
where Id = 276620;
select * from #EmpTransaction;
Which gives you...
<PayDetails>
<Column Name="GROSS" DataType="float" Value="939760" />
</PayDetails>
Note that the node index is 1-based, i.e.: Column[#Name="GROSS"][1] would remove the first GROSS node, Column[#Name="GROSS"][2] removes the second GROSS node.
As to how you got two GROSS values in the first place ... whatever created the XML for you probably lists the GROSS column twice.
For your next question: Please do not poste pictures. Best was, to provide a MCVE (a stand-alone sample to reproduce your issue).
You can try something along this:
This is such a stand-alone sample
DECLARE #mockupTable TABLE(ID INT IDENTITY, YourXml XML);
INSERT INTO #mockupTable VALUES
(N'<PayDetails>
<Column Name="blah" DataType="string" Value="Some blah" />
<Column Name="GROSS" DataType="float" Value="1.1" />
<Column Name="Another" DataType="string" Value="One more" />
<Column Name="GROSS" DataType="float" Value="1.2" />
</PayDetails>')
,(N'<PayDetails>
<Column Name="blah" DataType="string" Value="Some blah" />
<Column Name="GROSS" DataType="float" Value="2.0" />
<Column Name="Another" DataType="string" Value="One more" />
<Column Name="GROSS" DataType="float" Value="2.0" />
</PayDetails>');
--This query will first check, if there are different values for GROSS within one XML. This might be a reason to look a bit closer before deleting them.
SELECT t.ID
,t.YourXml
FROM #mockupTable t
WHERE t.YourXml.value('count(distinct-values(/PayDetails/Column[#Name="GROSS"]/#Value))','int')>1;
--And this statement will return each <Column> just once per name (always the first of its kind)
UPDATE #mockupTable SET YourXml=YourXml.query(N'
<PayDetails>
{
for $elmt in distinct-values(/PayDetails/Column/#Name)
return /PayDetails/Column[#Name=$elmt][1]
}
</PayDetails>
');
--Check the output
SELECT * FROM #mockupTable
The idea in short:
With the Xpath /PayDetails/Column[#Name="GROSS"] we are reducing the observed set to columns, where the attribute Name equals GROSS. distinct-values() is a XQuery-function returning each value in a list just once. So we can use count() to check, if there are differing values for GROSS within one XML.
The UPDATE uses XQuery's FLWOR abilities. We use again distinct-values to get all values for Name within <Column>. Then we return just the first (see the [1]) for each name.
UPDATE: Check for doubled elements
With this query you can run through your whole table to search for any non-unique column name per XML:
SELECT t.YourXml.query(N'
for $elmt in distinct-values(/PayDetails/Column/#Name)
return <NameCount Name="{$elmt}" Count="{count(/PayDetails/Column[#Name=$elmt])}"/>
').query('/NameCount[#Count>1]')
FROM #mockupTable t;

SQL Query values out of XML using

I can't figure out what I'm doing wrong. I have XML stored in a table column as text. I'm taking the ID, and the XML text and querying it into a temp table that stores the XML as XML type.
Each Order in the XML has multiple licenses in it that I need to pull out and create a new table with OrderID and License ID. But I can't tell what I'm doing wrong.
So, I'm trying to start basic but I can't seem to even just get the Account Info from the first Node.
The XML looks like this:
<ns1:OrderFromCRM xmlns:ns1="http://company.com/licensing/neworder/v2">
<ns1:AccountInfo Name="Company Name" AccountId="A012345" />
<ns1:OrderInfo CRMOrderId="S147360" Date="2/23/2017 12:00:00 AM" ffEmail="emailaddress.#gmail.com" >
<ns1:Licensing>
<ns1:Foundations>
<ns1:Foundation LicenseId="L012345678" Action="Create" Environment="Production" Type="Enterprise">
<Metadata>
<AllowedInstances>999</AllowedInstances>
</Metadata>
</ns1:Foundation>
<ns1:Foundation LicenseId="L012345698" Action="Create" Environment="Production" Type="Enterprise">
<Metadata>
<AllowedInstances>999</AllowedInstances>
</Metadata>
</ns1:Foundation>
</ns1:Foundations>
<ns1:Licenses Type="Create">
<ns1:License LicenseId="L0123451234" ProductFamily="Fam1" Product="EStudio" LicenseType="Perpetual" StartDate="2017-02-23" ExpiryDate="2017-12-18" MaintenanceExpiryDate="2017-12-18">
<ns1:Capabilities>
<ns1:Capability Name="T1" />
<ns1:Capability Name="Q1" />
<ns1:Capability Name="B1" />
</ns1:Capabilities>
</ns1:License>
<ns1:License LicenseId="L333356675" ProductFamily="Fam1" Product="EStudio" LicenseType="Perpetual" StartDate="2017-02-23" ExpiryDate="2017-12-18" MaintenanceExpiryDate="2017-12-18">
<ns1:Capabilities>
<ns1:Capability Name="T1" />
<ns1:Capability Name="Q1" />
<ns1:Capability Name="B1" />
</ns1:Capabilities>
</ns1:License>
The SQL I wrote is:
CREATE TABLE #demoData
(
ActivationCode NVARCHAR(100) NOT NULL,
OrderXMLText XML
)
SELECT OrderId, OrderXMLText.value('(/OrderFromCRM/AccountInfo)[1]', 'varchar(30)')
FROM #DEMODATA
I mentioned I need the OrderID and the LicenseId but even with this, I can't get anything. Am I on right track? First, what am I missing? Second, once this is formatted correctly, how do I get the nested LicenseIds in the XML?
Thanks so much for any help. I've been trying to make this work for a couple days
You’re missing the namespace so the query isn’t matching the xml, therefore it doesn’t find the elements you’re querying.
Add
;WITH XMLNAMESPACES
https://learn.microsoft.com/en-us/sql/t-sql/xml/with-xmlnamespaces
Assuming you have a complete, valid XML document in your table (the one you're showing is lacking several closing tags), then you could try something like this:
;WITH XMLNAMESPACES('http://company.com/licensing/neworder/v2' AS ns1)
SELECT
ActivationCode,
CRMOrderId = XC.value('#CRMOrderId', 'varchar(100)'),
FoundationsLicenseId = xcf.value('#LicenseId', 'varchar(50)'),
LicensesLicenseId = xcl.value('#LicenseId', 'varchar(50)')
FROM
#demoData
CROSS APPLY
OrderXMLText.nodes('/ns1:OrderFromCRM/ns1:OrderInfo') AS XT(XC)
CROSS APPLY
XC.nodes('ns1:Licensing/ns1:Foundations/ns1:Foundation') AS XTF(XCF)
CROSS APPLY
XC.nodes('ns1:Licensing/ns1:Licenses/ns1:License') AS XTL(XCL)
First of all, you need to include and respect the XML namespace in your XQuery - that's what I do with the WITH XMLNAMESPACES() directive.
Next, you need to use .nodes() to get a list of XML fragments for each <OrderInfo> node, which is located below the root node (<OrderFromCRM>). This is the first CROSS APPLY. This returns a list of XML fragments, one for each <OrderInfo> node.
Then you need to reach into these XML fragments, and again use CROSS APPLY with the .nodes() function to get a list of the <Foundation> elements (contained in the <Licensing>/<Foundations> subtree to get the license Id's from those nodes. In addition, you need a second CROSS APPLY to get all the <License> subnodes under <Licensing>/<Licenses> to get those LicenseId attributes.
That should return an output something like:
Hope this helps you some!

SSIS Process XML document in Excel Workbook format

I'm trying to import an XML document using SSIS that is exported using a Microsoft Office Excel format with 17 columns. I have an XML task that is removing the multiple namespaces, but now I have a document that is formatted like the sample code below. I can load each Cell into it's own record in the database, but since there are no tags inside the row or cell sections, I have a database table with one column for each Cell. I don't have any row numbers so not sure if I can do some sort of pre-sort, or if I'm going to have to do a bunch of SQL based on the row number being a multiple of 17 and STUFF FOR XML PATH the rows via temp tables which seems messy.
<Worksheet>
<Table>
<Column/>
<Row>
<Cell StyleID="s62">
<Data Type="String">City</Data>
</Cell>
<Cell StyleID="s62">
<Data Type="String">State</Data>
</Cell>
<Cell StyleID="s62">
<Data Type="String">Zip</Data>
</Cell>
</Row>
</Table>
</Worksheet>
I do not know, if I understood this correctly...
Assuming your XML is in a variable DECLARE #xml XML you might get the Cells by calling them with their position within the tree
SELECT R.value('Cell[1]/Data[1]','varchar(max)') AS City
,R.value('Cell[2]/Data[1]','varchar(max)') AS State
,R.value('Cell[3]/Data[1]','varchar(max)') AS Zip
--add more
FROM #xml.nodes('/Worksheet/Table/Row') AS A(R)
or you might think about pivot like this
SELECT p.*
FROM
(
SELECT 'Cell_' + CAST(ROW_NUMBER() OVER(ORDER BY (SELECT NULL)) AS VARCHAR(10)) AS ColumnName
,Cell.value('Data[1]','varchar(max)') AS Data
FROM #xml.nodes('/Worksheet/Table/Row/Cell') AS A(Cell)
) AS tbl
PIVOT
(
MIN(Data) FOR ColumnName IN(Cell_1,Cell_2,Cell_3,Cell_4 /*add as many as you need*/)
) AS p;
The second could be transfered into dynamic SQL to analyse the existing Cell nodes, get their names and number and return a resultset fitting to the XML's data...
The last you would need, if the XML's data is not the same in all calls...
The only solution I found was to open the file with script task using interop and save it in excel format.

Save XML nested child nodes

I have a Problem with saving the XML nodes in different Tables.
I want to save the Listnodes in one Table(ListsSet) and the Columnsnodes in another Table(ColumnsSet) which is referencing the ListTable. But in the Columnsnode is also a DependentLookupField childnode. I want to save the value from the DependentLookupField in a third Table(DependentLookupFieldsSet), which is referencing the ColumnTable.
My XML:
<Lists>
<List title="test">
<ListUrl>fsasa</ListUrl>
<ListTitle>gsdfgsg</ListTitle>
<ListDesc>jasdh</ListDesc>
<Columns>
<Column action="modify">
<InternalName>Title</InternalName>
<DisplayNameOrigin>Titel</DisplayNameOrigin>
<DisplayName>Name</DisplayName>
</Column>
<Column action="new">
<Required>true</Required>
<FieldType>Choice</FieldType>
<InternalName>anrede</InternalName>
<DisplayName>Anrede</DisplayName>
</Column>
<Column action="add" type="sitecolumn">
<Required>true</Required>
<InternalName>Bank</InternalName>
<DisplayName>Bank</DisplayName>
<MultipleValues>0</MultipleValues>
<DependentLookupFields>
<DependentLookupField internalName="Title">My Value</DependentLookupField>
</DependentLookupFields>
</Column>
</Columns>
</List>
<List>
.
.
.
etc
</List>
</Lists>
My Code of the Stored Procedure:
INSERT INTO ListsSet
SELECT
List.value('ListUrl[1]','NVARCHAR(100)') AS ListURL,
List.value('ListTitle[1]','NVARCHAR(100)') AS ListTitle,
List.value('ListDesc[1]','NVARCHAR(100)') AS ListDesc,
FROM
#xml.nodes('/Lists/List')AS TEMPTABLE(List)
INSERT INTO ColumnsSet
SELECT
l.ListsID,
c.value('FieldType[1]','NVARCHAR(100)') AS FieldType,
c.value('InternalName[1]','NVARCHAR(100)') AS InternalName,
c.value('DisplayName[1]','NVARCHAR(100)') AS DisplayName,
c.value('Required[1]','NVARCHAR(100)') AS Required,
c.value('DisplayNameOrigin[1]','NVARCHAR(100)') AS DisplayNameOrigin,
c.value('MultipleValues[1]','NVARCHAR(100)') AS MultipleValues,
FROM
ListsSet AS l
cross apply #xml.nodes('/Lists/List/Columns/Column[../../ListUrl=sql:column("l.ListUrl")]') AS TEMPTABLE(c)
INSERT INTO DependentLookupFieldsSet
SELECT
c.ColumnsID,
d.value('DependentLookupField[1]','NVARCHAR(100)') AS DependetLookupField
FROM
ColumnsSet AS c
cross apply #xml.nodes('/Lists/List/Columns/Column/DependentLookupFields/DependentLookupField[../../InternalName=sql:column("c.InternalName")]') AS TEMPTABLE(d)
The Code for saving the Listsnodes and Columnsnode is working fine. But the Code for the DependentLookupField don't save the value. In the Table "DependentLookupFieldsSet" I get the Rows which are referencing the ColumsTable(that is working), but the saved value is always null in each row.
Any help is appreciated!

For XML returning All column names as same element

I have a requirement to return xml as follow
<row id="1">
<cell>1</cell>
<cell>setup/pagemst</cell>
<cell>Page Master</cell>
</row>
<row id="2">
<cell>2</cell>
<cell>setup/modules</cell>
<cell>Module Master</cell>
</row>
I used the following Query but it does not work
select
pageid id,pgurl cell,pgname cell
from m_pages
for xml raw
The same column names for all columns works fine in oracle but not in SQL Server 2005. Any Ideas ?
Thanks in advance
deb
Use the FOR XML PATH syntax available in SQL Server 2005 and newer - try something like this:
DECLARE #table TABLE (PageID INT, PageUrl VARCHAR(50), PageName VARCHAR(50))
INSERT INTO #table VALUES(1, 'setup/pagemst', 'Page Master'),(2, 'setup/modules', 'Module Master')
select
PageID AS '#id',
PageID AS 'cell',
'',
PageUrl AS 'cell',
'',
PageName AS 'cell'
from #table
FOR XML PATH('row')
Gives me the output in the end:
<row id="1">
<cell>1</cell>
<cell>setup/pagemst</cell>
<cell>Page Master</cell>
</row>
<row id="2">
<cell>2</cell>
<cell>setup/modules</cell>
<cell>Module Master</cell>
</row>
The FOR XML PATH syntax allows you to define what values to return as XML attributes (... AS '#id') or XML elements. By adding "empty" lines between the elements, you prevent the output from being merged / concatenated
into a single <cell>....</cell> XML element

Resources