Read XML file into SQL Server database - sql-server

I'm trying to read an XML file into a database table that already exists.
The problem is that the XML tags and the database columns don't have the same name although they have the same datatype. Therefore I'd like to "translate" the XML tags into the database columns so that the the input to the database becomes possible.
I'm not sure how to do that however.
Here is what I've done so far.
static void writeToDatabase()
{
XmlDocument doc= new XmlDocument();
try {
// Reading the xml
doc.Load("C:\\Temp\navetout.xml");
DataTable dt = new DataTable();
// Code here to read the xml into an already existing database table?
}
catch (Exception e)
{
Console.WriteLine(e.Message);
}
}
The database is located on another server, I've included this in the app.config
<connectionStrings>
<add name="CS"
connectionString="Data Source=tsrv2062;Initial Catalog=BUMS;Integrated Security=True"/>
</connectionStrings>
Let's say for an example that the XML file has the tags "Name" while the database table column has the column "Firstname".
XML example:
<?xml version="1.0" encoding="utf-8"?>
<ArrayOfFolkbokforingspostTYPE xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
<FolkbokforingspostTYPE>
<Sekretessmarkering xsi:nil="true" />
<Personpost>
<PersonId>
<PersonNr>7527245452542</PersonNr>
</PersonId>
<HanvisningsPersonNr xsi:nil="true" />
<Namn>
<Tilltalsnamnsmarkering>20</Tilltalsnamnsmarkering>
<Fornamn>skjdgnsdng</Fornamn>
<Mellannamn xsi:nil="true" />
<Efternamn>sdsdgsdgs</Efternamn>
<Aviseringsnamn xsi:nil="true" />
</Namn>
<Folkbokforing>
<Folkbokforingsdatum>20060512</Folkbokforingsdatum>
<LanKod>56</LanKod>
<KommunKod>77</KommunKod>
<ForsamlingKod xsi:nil="true" />
<Fastighetsbeteckning>PÅLNGE 6:38</Fastighetsbeteckning>
<FiktivtNr>0</FiktivtNr>
</Folkbokforing>
<Adresser>
<Folkbokforingsadress>
<CareOf xsi:nil="true" />
<Utdelningsadress1 xsi:nil="true" />
<Utdelningsadress2>sgdsdgsdgs</Utdelningsadress2>
<PostNr>78965</PostNr>
<Postort>PÅLÄNG</Postort>
</Folkbokforingsadress>
<Riksnycklar>
<FastighetsId>46464545</FastighetsId>
<AdressplatsId>764846846</AdressplatsId>
<LagenhetsId>45465654645</LagenhetsId>
</Riksnycklar>
</Adresser>
<Fodelse>
<HemortSverige>
<FodelselanKod>00</FodelselanKod>
<Fodelseforsamling>NEDERKALIX</Fodelseforsamling>
</HemortSverige>
</Fodelse>
<Medborgarskap>
<MedborgarskapslandKod>SE</MedborgarskapslandKod>
<Medborgarskapsdatum>0</Medborgarskapsdatum>
</Medborgarskap>
</Personpost>
</FolkbokforingspostTYPE>
</ArrayOfFolkbokforingspostTYPE>
These are the columns of the database table:
PersonalIdentityNumber
ProtectedIdentity
ReferedCivicRegistrationNumber
UnregistrationReason
UnregistrationDate
MessageComputerComputer
GivenNameNumber
FirstName
MiddleName
LastName
NotifyName
NationalRegistrationDate
NationalRegistrationCountyCode
NationalRegistrationMunicipalityCode
NationalRegistrationCoAddress
NationalRegistrationDistributionAddress1
NationalRegistrationDistributionAddress2
NationalRegistrationPostCode
NationalRegistrationCity
NationalRegistrationNotifyDistributionAddress
NationalRegistrationNotifyPostCode
NationalRegistrationNotifyCity
ForeignDistrubtionAddress1
ForeignDistrubtionAddress2
ForeignDistrubtionAddress3
ForeignDistrubtionCountry
ForeignDate
BirthCountyCode
BirthParish
ForeignBirthCity
CitizenshipCode
CitizenshipDate
Email
Telephone
Mobiletelephone
Gender
NotNewsPaper
Note
StatusCode
NationalRegistrationCode
RegistrationDate
LastUpdatedFromNavet
TemporaryDistrubtionAddress1
TemporaryDistrubtionAddress2
TemporaryDistrubtionAddress3
TemporaryDistrubtionCountry
Password
VisibilityLevel
LastChangedBy
LastChangedDate
SeamanIdentity
Category
Here for an example, the <PersonNr> tagg and the databse column PersonalIdentityNumber are the same.
The column that doesn't match with the XML-tags are supposed to returning null.
Before reading the the XML data into the database table, I suppose the XML-tags has to be translated into the Database table column. In this case "Firstname".
Can anyone help me out with this "translation" and the reading into the database table.

DECLARE #xml XML
SELECT #xml = BulkColumn
FROM OPENROWSET(BULK 'D:\sample.xml', SINGLE_BLOB) x
SELECT
t.c.value('(PersonId/PersonNr/text())[1]', 'VARCHAR(100)'),
t.c.value('(Namn/Tilltalsnamnsmarkering/text())[1]', 'INT')
FROM #xml.nodes('*:ArrayOfFolkbokforingspostTYPE/*:FolkbokforingspostTYPE/*:Personpost') t(c)

This is how I used to do. This is not a solutionm but you may refer this.
EXEC sp_xml_preparedocument #XML_OUT OUTPUT, #XML_DATA;
Here, #XML_DATA is the XML data you pass. #XML_OUT is just an INT type.
SELECT * INTO #TEMP
FROM OPENXML(#XML_OUT,'DATA/INNER_TAG', 1)
WITH
(
a VARCHAR(500),
b VARCHAR(500),
c INT,
d VARCHAR(20)
)
But these name1, name2 etc need to be the same as in XML file.
So I use INSERT INTO SELECT Query to insert this data into specified table. Like,
INSERT INTO OriginalTable
(
name1,
name2,
name3,
name4
)
SELECT
a
b,
c,
d
FROM #TEMP

Related

How to use xml type-valued functions in SQL Server FOR XML

Is there a way to make function work like an object?
e.g.: I have an xml typed function like this:
CREATE FUNCTION dbo.CompanyXml(#id int) RETURNS XML
AS
BEGIN
RETURN (SELECT id AS [#id], name AS [#name] FROM Companies WHERE id = #id
FOR XML PATH('CompanyType'),TYPE)
END
When I use the function:
SELECT dbo.CompanyXml(1) AS Supplier,
dbo.CompanyXml(2) AS Client
FOR XML PATH('Document'), TYPE
I get:
<Document>
<Supplier>
<CompanyType id="1" name="Company 1" />
</Supplier>
<Client>
<CompanyType id="2" name="Company 2" />
</Client>
</Document>
but i need:
<Document>
<Supplier id="1" name="Company 1" />
<Client id="2" name="Company 2" />
</Document>
Is there a way to achieve this?
[UPDATE] My solution
(inspired by #Shnugo)
I have used table valued function with FOR XML AUTO
CREATE FUNCTION dbo.CompanyTbl(#id int) RETURNS TABLE
AS SELECT id, name FROM Companies WHERE id = #id
used like this
SELECT (SELECT * FROM dbo.CompanyTbl(1) AS Supplier FOR XML AUTO, TYPE),
(SELECT * FROM dbo.CompanyTbl(2) AS Client FOR XML AUTO, TYPE)
FOR XML PATH('Document'), TYPE
The alias AS Supplier or AS Client is the caption for a specific column in your result set. The function you are calling does not know (and can not know), that its result will be displayed as Supplier or as Client...
Further more the element's name must be a literal using FOR XML PATH().
There are three approaches, I'd pick the last:
You can go with a modified function like Kannan Kandasamys' suggestion, but you will need one section per role and you must hand in the "type" as parameter. More roles will need modifications of the function. Might be difficult in deployed databases...
You could create the xml on string level (something like '<' + #element + id="' + ...) and then use a CAST(... AS XML). Be careful how you deal with special characters in this case!
(My choice): Introduce one separate function for each document role. New roles are new functions, which is easier in most cases
For Nr 3 your code would look like
SELECT dbo.SupplierXml(1) AS [*],
dbo.ClientXml(2) AS [*]
FOR XML PATH('Document'), TYPE;
UPDATE One more approach: FOR XML AUTO
Try this:
CREATE DATABASE TestDB;
GO
USE TestDB;
GO
CREATE TABLE TestTable(id INT,SomeOther VARCHAR(100));
INSERT INTO TestTable VALUES(1,'Some 1'),(2,'Some 2');
SELECT * FROM TestTable FOR XML AUTO;
--The result: You see, that the table name is the element's name:
<TestTable id="1" SomeOther="Some 1" />
<TestTable id="2" SomeOther="Some 2" />
--Nice is, that you can force this name using a table alias:
SELECT * FROM TestTable AS OtherName FOR XML AUTO;
--returns
<OtherName id="1" SomeOther="Some 1" />
<OtherName id="2" SomeOther="Some 2" />
GO
USE master;
GO
DROP DATABASE TestDB;
Now the bad thing is, that - again - the alias must be a literal and cannot be passed in as parameter. It is not inlineable, but you might do something like
DECLARE #cmd VARCHAR(1000)='SELECT * FROM YourTable AS ' + #alias + ' FOR XML AUTO';
EXEC (#cmd);
When it comes to dynamically set column names (same applies to element names), you must use some ugly tricks...
You can change the function as below:
CREATE FUNCTION dbo.CompanyXml1(#id int, #type varchar(15)) RETURNS XML
AS
BEGIN
if #type = 'Supplier'
begin
return(
SELECT id AS [#id], name AS [#name] FROM Companies WHERE id = #id
FOR XML PATH('Supplier'),TYPE )
end
else
begin
return(
SELECT id AS [#id], name AS [#name] FROM Companies WHERE id = #id
FOR XML PATH('Client'),TYPE
)
end
return(null);
END
your query as below:
SELECT dbo.CompanyXml1(1, 'Supplier'),
dbo.CompanyXml1(2, 'Client')
FOR XML PATH('Document'), TYPE
Output ...:
<Document>
<Supplier id="1" name="Company 1" />
<Client id="2" name="Company 2" />
</Document>

Extract XML header information into SQL server

My process involves getting a large XML file on a daily basis.
I have developed an SSIS package (2008 r2) which first gets rid of the multiple namespaces via a XSLT and then imports data into 40 tables (due to its complexity) by using the XML source object.
Here is the watered down version of a test xml file
<?xml version="1.0" encoding="UTF-8"?>
<s:Test xmlns:s="http://###.##.com/xml"
<sequence>62</sequence>
<generated>2015-04-28T00:59:38</generated>
<report_date>2015-04-27</report_date>
<orders>
<order>
</order>
</orders>
My question is: The XML source imports all the Orders with its nested attributes. How do I extract the 'report_date' and 'generated' from the header?
Any help would be much appreciated.
Thanks
SD
You can use XML method value() passing proper XPath/XQuery expression as parameter. For demo, consider the following table and data :
CREATE TABLE MyTable (id int, MyXmlColumn XML)
DECLARE #data XML = '<?xml version="1.0" encoding="UTF-8"?>
<s:Test xmlns:s="http://###.##.com/xml">
<sequence>62</sequence>
<generated>2015-04-28T00:59:38</generated>
<report_date>2015-04-27</report_date>
<orders>
<order>
</order>
</orders>
</s:Test>'
INSERT INTO Mytable VALUES(1,#data)
You can use the following query to get generated and report_date data :
SELECT
t.MyXmlColumn.value('(/*/generated)[1]','datetime') as generated
, t.MyXmlColumn.value('(/*/report_date)[1]','date') as report_date
FROM Mytable t
SQL Fiddle Demo
output :
generated report_date
----------------------- -----------
2015-04-28 00:59:38.000 2015-04-27

Combine and modify XML in TSQL

Using SQL Server 2005, is it possible to combine XML and add an attribute at same time?
Unfortunately, due to project restrictions, I need a SQL Server 2005 solution.
Consider the following, where I need to combine XML from multiple rows within a new <root> element...
; WITH [TestTable] AS (
SELECT 7 AS [PkId], CAST('<data><id>11</id><id>12</id></data>' AS XML) AS [Data]
UNION ALL
SELECT 12, CAST('<data><id>22</id></data>' AS XML)
UNION ALL
SELECT 43, CAST('<data><id>33</id></data>' AS XML)
)
SELECT (
SELECT XMLDATA as [*]
FROM (
SELECT [Data] AS [*]
FROM [TestTable]
FOR XML PATH(''), TYPE
) AS DATA(XMLDATA)
FOR XML PATH('root')
)
This produces the desired output of...
<root>
<data><id>11</id><id>12</id></data>
<data><id>22</id></data>
<data><id>33</id></data>
</root>
But what I need to do, if possible, is add an attribute to the existing data element in each of the rows with the PkId value. The desired output would then look like this...
<root>
<data pkid="7"><id>11</id><id>12</id></data>
<data pkid="12"><id>22</id></data>
<data pkid="43"><id>33</id></data>
</root>
My gut feeling is that this is going to be impossible without the use of a cursor, but if anybody knows a way of doing it I'd love to hear it.
At the request of #MattA, here is an example of some random data in the table...
[PkId] [UserId] [SubmittedDate] [Data]
1 1 2015-03-24 12:34:56 '<data><id>1</id><id>2</id></data>'
2 1 2015-03-23 09:15:52 '<data><id>3</id></data>'
3 2 2015-03-22 16:01:23 '<data><id>4</id><id>5</id></data>'
4 1 2015-03-21 13:45:34 '<data><id>6</id></data>'
Please note, that to make the question easier, I stated that I needed the PkId column as the attribute to the data. This is not actually the case - instead I need the [SubmittedDate] column to be used. I apologise if this caused confusion.
Using UserId=1 as a filter, the XML I would like from the above would be...
<root>
<data submitteddate="2015-03-24T12:34:56"><id>1</id><id>2</id></data>
<data submitteddate="2015-03-23T09:15:52"><id>3</id></data>
<data submitteddate="2015-03-21T13:45:34"><id>6</id></data>
</root>
The date would be formatted using the 126 date format available from CONVERT
Here's the quick answer for you. XML does support "modify", but shredding on a small data set like this works quite well too.
Code
--The existing XML
DECLARE #XML XML = '<root>
<data><id>11</id></data>
<data><id>22</id></data>
<data><id>33</id></data>
</root>'
--XML Shredded Back to a table
;WITH
ShreddedXML AS (
SELECT
ID = FieldAlias.value('(id)[1]','int')
FROM
#XML.nodes('/root/data') AS TableAlias(FieldAlias)
), ArbitraryPKGenerator AS (
SELECT CURRENT_TIMESTAMP AS PKid,
ID
FROM ShreddedXML
)
SELECT A.PKId AS "#PKid",
A.ID AS "id"
FROM ArbitraryPKGenerator AS A
FOR XML PATH('data'), ROOT('root')
And the XML
<root>
<data PKid="2015-03-24T09:44:55.770">
<id>11</id>
</data>
<data PKid="2015-03-24T09:44:55.770">
<id>22</id>
</data>
<data PKid="2015-03-24T09:44:55.770">
<id>33</id>
</data>
</root>

Return XML cell data in XML query with SQL Server 2012

I have a table in SQL Server 2012 that contains some customer data. One of the columns in the table contains license data that is stored as XML. The type of the cell is nvarchar(MAX).
Is it possible to use FOR XML (or some other method) so that when the data is returned the XML from the license data is included as XML rather than a formatted string?
If I simply use FOR XML RAW, then the result is:
<Customers id="1" CustomerName="FirstCustomer"
LicenseData="<license customerid="1">...More data here...</license>" />
What I would liket to get is:
<Customers id="1" CustomerName="FirstCustomer">
<license customerid="1">
...More data here...
</license>
</Customers>
Is there any way to make that happen?
If the XML is a valid fragment then you can simply CAST it to XML.
SELECT CAST(MyColumn as XML) as MyXml
declare #temp table (id int, customername nvarchar(128), data nvarchar(max))
insert into #temp
select 1, 'FirstCustomer', '<license customerid="1"><element id="2">data1</element><element id="3"/></license>'
select id, customername, cast(data as xml)
from #temp
for xml raw
And you'll get results like this:
<row id="1" customername="FirstCustomer">
<license customerid="1">
<element id="2">data1</element>
<element id="3" />
</license>
</row>

SQL - Read an XML node from a table field

I am using SQL Server 2008. I have a field called RequestParameters in one of my SQL table called Requests with XML data. An example would be:
<RequestParameters xmlns="http://schemas.datacontract.org/2004/07/My.Name.Space" xmlns:i="http://www.w3.org/2001/XMLSchema-instance" xmlns:z="http://schemas.microsoft.com/2003/10/Serialization/" z:Id="1">
<Data z:Id="2" i:type="CheckoutRequest">
<UserGuid>7ec38c44-5aa6-49e6-9fc7-25e9028f2148</UserGuid>
<DefaultData i:nil="true" />
</Data>
</RequestParameters>
I ultimately want to retrieve the value of UserGuid. For that, I am doing this:
SELECT RequestParameters.value('(/RequestParameters/Data/UserGuid)[0]', 'uniqueidentifier') as UserGuid
FROM Requests
However, the results I am seeing are all NULL. What am I doing wrong?
You have to specify the default namespace and use [1] instead of [0].
WITH XMLNAMESPACES(default 'http://schemas.datacontract.org/2004/07/My.Name.Space')
SELECT RequestParameters.value('(/RequestParameters/Data/UserGuid)[1]', 'uniqueidentifier') as UserGuid
FROM Requests;
SQL Fiddle
declare #XML xml
set #XML = "<RequestParameters xmlns="http://schemas.datacontract.org/2004/07/My.Name.Space" xmlns:i="http://www.w3.org/2001/XMLSchema-instance" xmlns:z="http://schemas.microsoft.com/2003/10/Serialization/" z:Id="1">
<Data z:Id="2" i:type="CheckoutRequest">
<UserGuid>7ec38c44-5aa6-49e6-9fc7-25e9028f2148</UserGuid>
<DefaultData i:nil="true" />
</Data>
</RequestParameters>"
select #XML.value('(/RequestParameters/Data /UserGuid)[1]', 'varchar')
'

Resources