Using SQLXML Bulk Import with complicated schema - sql-server

I'm trying to figure out how to use SQLXML bulk load for a rather complicated XML schema I have no control over (It's nominally a "standard"). A small example would be:
<RootElement>
<id root="e5404218-250a-4454-98d5-f9feee8a3589" />
<code code="93000" />
<time>
<low value="20140221095400.000" />
<high value="20140221095410.000" />
</time>
</RootElement>
With an AXSD schema of:
<?xml version="1.0"?>
<xs:schema xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:xs="http://www.w3.org/2001/XMLSchema"
xmlns:sql="urn:schemas-microsoft-com:mapping-schema">
<xs:element name="RootElement" sql:relation="re" >
<xs:complexType>
<xs:sequence>
<xs:element name="id" maxOccurs="1" minOccurs="1">
<xs:complexType>
<xs:attribute name="root" type="xs:ID" sql:field="id" sql:identity="useValue"/>
</xs:complexType>
</xs:element>
<xs:element name="code" minOccurs="1" maxOccurs="1">
<xs:complexType>
<xs:attribute name="code" type="xs:long"/>
</xs:complexType>
</xs:element>
<xs:element name="time" type="timeRange"/>
</xs:sequence>
</xs:complexType>
</xs:element>
<xs:complexType name="timeRange">
<xs:choice>
<xs:sequence>
<xs:element name="low">
<xs:complexType>
<xs:attribute name="value" type="xs:string"/>
</xs:complexType>
</xs:element>
</xs:sequence>
<xs:sequence>
<xs:element name="high">
<xs:complexType>
<xs:attribute name="value" type="xs:string"/>
</xs:complexType>
</xs:element>
</xs:sequence>
</xs:choice>
</xs:complexType>
</xs:schema>
My sticking point seems to come from the ID being attached to an attribute on an element, instead of an element value, as no matter what tweaks I use, I get the error Schema: relationship expected on 'id'.. There really is no relationship there, it's just a container, since the committee that designed this schema seems to have really taken a liking to attributes.
All examples I find deal with the values being element content instead of attributes. This includes decorating the element with sql:is-constant (Constant element cannot have attributes), and various permutations of the sql:relation element.
The only option I could think of is to have a pre-processing step that makes attributes child elements, and base the schema on that, but that seems kind of hacky.
Any suggestions on how to convince SQLXML to do my bidding?

Related

Generating XML output using XSD structure file in SQL Server

I have a quite complex XSD structure file that is supposed to help us generate a report in XML, however the XSD is prone to change and one has to keep updating the base code. Wondering if there's any way to generate XML outputs using the xsd file itself?
The XSD has a lot of nested fields
<?xml version="1.0" encoding="UTF-8"?>
<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema" xmlns:ns="http://-" " elementFormDefault="qualified" attributeFormDefault="unqualified" version="6-2">
<xs:include schemaLocation="--"/>
<xs:complexType name="Address_Structure">
<xs:choice>
<xs:element name="UnstructuredAddress" type="ns:UnstructuredAddress_Type"/>
<xs:element name="StructuredAddress" type="ns:StructuredAddress_Type" minOccurs="2" maxOccurs="5"/>
</xs:choice>
</xs:complexType>
<xs:complexType name="ClinicalActivity_AE_Structure">
<xs:sequence>
<xs:element name="ClinicalActivityGroupOPCS" type="ns:ClinicalActivityGroupOPCS_Type" minOccurs="0"/>
<xs:element name="ClinicalActivityGroupREAD" type="ns:ClinicalActivityGroupREAD_Type" minOccurs="0"/>
<xs:element name="ClinicalTreatmentGroupAandE" type="ns:ClinicalTreatmentGroupAAndE_Type" minOccurs="0"/>
</xs:sequence>
</xs:complexType>
<xs:complexType name="ClinicalActivity_Structure">
<xs:sequence>
<xs:element name="ClinicalActivityGroupOPCS" type="ns:ClinicalActivityGroupOPCS_Type" minOccurs="0"/>
<xs:element name="ClinicalActivityGroupREAD" type="ns:ClinicalActivityGroupREAD_Type" minOccurs="0"/>
</xs:sequence>
</xs:complexType>
<xs:complexType name="ClinicalActivityGroupOPCS_Type">
<xs:sequence>
<xs:element name="ProcedureSchemeInUse" type="ns:ProcedureSchemeInUse_Type"/>
<xs:element name="PrimaryProcedureGroupOPCS">
<xs:complexType mixed="false">
<xs:sequence>
<xs:element name="PrimaryProcedure_OPCS" type="ns:PrimaryProcedure_OPCS_Type"/>
<xs:element name="ProcedureDate" type="ns:ProcedureDate_Type" minOccurs="0"/>
<xs:element name="MainOperatingHCP" type="ns:ProfessionalsInvolved_Structure" minOccurs="0"/>
<xs:element name="ResponsibleAnaesthetist" type="ns:ProfessionalsInvolved_Structure" minOccurs="0"/>
</xs:sequence>
</xs:complexType>
</xs:element>
<xs:element name="SecondaryProcedureGroupOPCS" minOccurs="0" maxOccurs="unbounded">
<xs:complexType mixed="false">
<xs:sequence>
<xs:element name="SecondaryProcedure_OPCS" type="ns:Procedure_OPCS_Type"/>
<xs:element name="ProcedureDate" type="ns:ProcedureDate_Type" minOccurs="0"/>
<xs:element name="MainOperatingHCP" type="ns:ProfessionalsInvolved_Structure" minOccurs="0"/>
<xs:element name="ResponsibleAnaesthetist" type="ns:ProfessionalsInvolved_Structure" minOccurs="0"/>
</xs:sequence>
</xs:complexType>
</xs:element>
</xs:sequence>
</xs:complexType>

xml /XSD file source ssis 2010 - data at root level is invalid to SQL server table

I am trying to learn how to load an XML (XML source) into SSIS 2010 , the data will then go to an SQL Server 2012 Table.
The issue I am having, Is I am trying to follow a tutorial, and I am getting the same error:
The XML Source was unable to read the XML data. Data at the root level is invalid. Line 1, position 1.
This when downloading any of the tables - the one I have chosen is region.
Here is the script - this is in an XSD file
<?xml version="1.0" encoding="UTF-8" ?>
<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema">
<xs:element name="countries">
<xs:complexType>
<xs:sequence>
<xs:element minOccurs="0" maxOccurs="unbounded" name="country">
<xs:complexType>
<xs:sequence>
<xs:element minOccurs="0" name="iso2Code" type="xs:string" />
<xs:element minOccurs="0" name="name" type="xs:string" />
<xs:element minOccurs="0" name="region">
<xs:complexType>
<xs:simpleContent>
<xs:extension base="xs:string">
<xs:attribute name="id" type="xs:string" use="optional" />
</xs:extension>
</xs:simpleContent>
</xs:complexType>
</xs:element>
<xs:element minOccurs="0" name="adminregion">
<xs:complexType>
<xs:simpleContent>
<xs:extension base="xs:string">
<xs:attribute name="id" type="xs:string" use="optional" />
</xs:extension>
</xs:simpleContent>
</xs:complexType>
</xs:element>
<xs:element minOccurs="0" name="incomeLevel">
<xs:complexType>
<xs:simpleContent>
<xs:extension base="xs:string">
<xs:attribute name="id" type="xs:string" use="optional" />
</xs:extension>
</xs:simpleContent>
</xs:complexType>
</xs:element>
<xs:element minOccurs="0" name="lendingType">
<xs:complexType>
<xs:simpleContent>
<xs:extension base="xs:string">
<xs:attribute name="id" type="xs:string" use="optional" />
</xs:extension>
</xs:simpleContent>
</xs:complexType>
</xs:element>
<xs:element minOccurs="0" name="capitalCity" type="xs:string" />
<xs:element minOccurs="0" name="longitude" type="xs:string" />
<xs:element minOccurs="0" name="latitude" type="xs:string" />
</xs:sequence>
<xs:attribute name="id" type="xs:string" use="optional" />
</xs:complexType>
</xs:element>
</xs:sequence>
<xs:attribute name="page" type="xs:unsignedByte" use="optional" />
<xs:attribute name="pages" type="xs:unsignedByte" use="optional" />
<xs:attribute name="per_page" type="xs:unsignedByte" use="optional" />
<xs:attribute name="total" type="xs:unsignedByte" use="optional" />
</xs:complexType>
</xs:element>
</xs:schema>
The trail site is http://api.worldbank.org/countries
I have tried changing the top line
<?xml version="1.0" encoding="UTF-8" ?>
with no success. Please help

Want to create XML schema to validate two XMLS

I have 2 Xml files. 1st file we can call master as its having all nodes and in 2nd files only few information is there.
How can i creat a single xsd supporting both files.
File1:
<Information ID="08" >
<description>office information</description>
<officeName>SBV</officeName>
<A id="234">
<name>xyz</name>
< age>23</age>
<dept>SE</dept>
</A>
<B id="244">
<name>lkj</name>
< age>27</age>
<dept>Sales</dept>
</B>
<C id="264">
<name>opi</name>
< age>34</age>
<dept>Manufacture</dept>
</C>
<D id="774">
<name>yui</name>
< age>25</age>
<dept>Production</dept>
</D>
</Information>
File2:
<Information >
<description>Worker information</description>
<A id="234">
<name>xyz</name>
< age>23</age>
<dept>WP</dept>
</A>
<B id="244">
<name>lkj</name>
< age>27</age>
<dept>WP</dept>
</B>
</Information>
This is a possible schema - which should be considered as a starting point.
<?xml version="1.0" encoding="utf-8"?>
<!--XML Schema generated by QTAssistant/XML Schema Refactoring (XSR) Module (http://www.paschidev.com)-->
<xs:schema attributeFormDefault="unqualified" elementFormDefault="qualified" xmlns:xs="http://www.w3.org/2001/XMLSchema">
<xs:element name="Information">
<xs:complexType>
<xs:sequence>
<xs:element name="description" type="xs:string" />
<xs:element minOccurs="0" name="officeName" type="xs:string" />
<xs:element name="A">
<xs:complexType>
<xs:sequence>
<xs:element name="name" type="xs:string" />
<xs:element name="age" type="xs:unsignedByte" />
<xs:element name="dept" type="xs:string" />
</xs:sequence>
<xs:attribute name="id" type="xs:unsignedByte" use="required" />
</xs:complexType>
</xs:element>
<xs:element name="B">
<xs:complexType>
<xs:sequence>
<xs:element name="name" type="xs:string" />
<xs:element name="age" type="xs:unsignedByte" />
<xs:element name="dept" type="xs:string" />
</xs:sequence>
<xs:attribute name="id" type="xs:unsignedByte" use="required" />
</xs:complexType>
</xs:element>
<xs:element minOccurs="0" name="C">
<xs:complexType>
<xs:sequence>
<xs:element name="name" type="xs:string" />
<xs:element name="age" type="xs:unsignedByte" />
<xs:element name="dept" type="xs:string" />
</xs:sequence>
<xs:attribute name="id" type="xs:unsignedShort" use="required" />
</xs:complexType>
</xs:element>
<xs:element minOccurs="0" name="D">
<xs:complexType>
<xs:sequence>
<xs:element name="name" type="xs:string" />
<xs:element name="age" type="xs:unsignedByte" />
<xs:element name="dept" type="xs:string" />
</xs:sequence>
<xs:attribute name="id" type="xs:unsignedShort" use="required" />
</xs:complexType>
</xs:element>
</xs:sequence>
<xs:attribute name="ID" type="xs:unsignedByte" use="optional" />
</xs:complexType>
</xs:element>
</xs:schema>
In my case, I've used QTAssistant (I am associated with it) to create the XSD automatically, by pointing it to the two XML documents you've posted.
This is what I've got (it validates both):
The next steps should then be to "rationalize" the generated XSD. For example, if you look at the content models of elements A, B, C and D, all share the same; if indeed this is the case, then you could have a global type declaration, basically reused. Or, it could be that C and D must appear in a "both or none" combination, in which case additional tweaking may achieve exactly that. Below is a manually amended schema, based on the above comments, to give you an idea on what it would look like. Keep in mind that both XSDs validate your XMLs.
<?xml version="1.0" encoding="utf-8"?>
<!--XML Schema generated by QTAssistant/XML Schema Refactoring (XSR) Module (http://www.paschidev.com)-->
<xs:schema attributeFormDefault="unqualified" elementFormDefault="qualified" xmlns:xs="http://www.w3.org/2001/XMLSchema">
<xs:element name="Information">
<xs:complexType>
<xs:sequence>
<xs:element name="description" type="xs:string"/>
<xs:element minOccurs="0" name="officeName" type="xs:string"/>
<xs:element name="A" type="atype"/>
<xs:element name="B" type="atype"/>
<xs:sequence minOccurs="0">
<xs:element name="C" type="atype"/>
<xs:element name="D" type="atype"/>
</xs:sequence>
</xs:sequence>
<xs:attribute name="ID" type="xs:unsignedByte" use="optional"/>
</xs:complexType>
</xs:element>
<xs:complexType name="atype">
<xs:sequence>
<xs:element name="name" type="xs:string"/>
<xs:element name="age" type="xs:unsignedByte"/>
<xs:element name="dept" type="xs:string"/>
</xs:sequence>
<xs:attribute name="id" type="xs:int" use="required"/>
</xs:complexType>
</xs:schema>
By now, one can see that the latter is more compact in terms of "source". The content models though are quite similar.

Camel Validator return entire message

I have a route,
<from uri="a">
<to uri="validator:schema.xsd">
<to uri="b">
Say the XML file being validated is missing two elements, the validator seems to stop once it finds the first missing element and returns a message saying it was missing.
Is it possible to carry on validating the XML file to look for any other missing elements and return that in the error message so the sender doesn't have to keep sending to find out which elements are missing or invalid?
Validator component throws SchemaValidationException, if validation fails. This exception contains method getError() returning List<org.xml.sax.SAXParseException>.
You can use this to transform message to List<org.xml.sax.SAXParseException> in onException block.
Following code catches SchemaValidationException, transforms body to SchemaValidationException.getErrors() and marks exception as continued, to continue routing and return this list on output route.
from("timer:simple?period=1000")
.setBody(constant(XML_INVALID))
.to("direct:a");
from("direct:a")
.onException(SchemaValidationException.class)
.to("direct:validationErrors")
.continued(true)
.end()
.to("validator:test.xsd")
.to("log:result");
from("direct:validationErrors")
.setBody(simple("${property.CamelExceptionCaught.errors}"))
.end();
Note: This example was tested with following resources
XML_INVALID
<?xml version="1.0" encoding="utf-8"?>
<shiporder orderid="str1234">
<orderperson>str1234</orderperson>
<shipto>
<name>str1234</name>
<address>str1234</address>
</shipto>
<item>
<quantity>745</quantity>
<price>123.45</price>
</item>
</shiporder>
test.xsd copied from w3schools.com
<?xml version="1.0" encoding="UTF-8" ?>
<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema">
<!-- definition of simple elements -->
<xs:element name="orderperson" type="xs:string"/>
<xs:element name="name" type="xs:string"/>
<xs:element name="address" type="xs:string"/>
<xs:element name="city" type="xs:string"/>
<xs:element name="country" type="xs:string"/>
<xs:element name="title" type="xs:string"/>
<xs:element name="note" type="xs:string"/>
<xs:element name="quantity" type="xs:positiveInteger"/>
<xs:element name="price" type="xs:decimal"/>
<!-- definition of attributes -->
<xs:attribute name="orderid" type="xs:string"/>
<!-- definition of complex elements -->
<xs:element name="shipto">
<xs:complexType>
<xs:sequence>
<xs:element ref="name"/>
<xs:element ref="address"/>
<xs:element ref="city"/>
<xs:element ref="country"/>
</xs:sequence>
</xs:complexType>
</xs:element>
<xs:element name="item">
<xs:complexType>
<xs:sequence>
<xs:element ref="title"/>
<xs:element ref="note" minOccurs="0"/>
<xs:element ref="quantity"/>
<xs:element ref="price"/>
</xs:sequence>
</xs:complexType>
</xs:element>
<xs:element name="shiporder">
<xs:complexType>
<xs:sequence>
<xs:element ref="orderperson"/>
<xs:element ref="shipto"/>
<xs:element ref="item" maxOccurs="unbounded"/>
</xs:sequence>
<xs:attribute ref="orderid" use="required"/>
</xs:complexType>
</xs:element>
</xs:schema>

Having trouble with an XML file and SQL Server 2008 SSIS

I'm trying to use SQL Server 2008 SSIS to read an XML file and put it into a database, but I'm having problems. Everything is green when I run debugging, but no data is actually transferred. Even when a make a simple XML Source -> Flat File output, the flat file is blank after being created.
I'm using the inline schema, but even if I remove the schema SSIS can't generate an XSD file.
Here's the XML:
<?xml version="1.0" encoding="utf-16"?>
<DataSet>
<xs:schema id="Schedule_x0020_Set" xmlns="" xmlns:xs="http://www.w3.org/2001/XMLSchema" xmlns:msdata="urn:schemas-microsoft-com:xml-msdata">
<xs:element name="Schedule_x0020_Set" msdata:IsDataSet="true" msdata:UseCurrentLocale="true">
<xs:complexType>
<xs:choice minOccurs="0" maxOccurs="unbounded">
<xs:element name="Schedule_x0020_Table">
<xs:complexType>
<xs:sequence>
<xs:element name="ScheduleDate" type="xs:string" minOccurs="0" />
<xs:element name="StaffLName" type="xs:string" minOccurs="0" />
<xs:element name="StaffFName" type="xs:string" minOccurs="0" />
<xs:element name="TaskName" type="xs:string" minOccurs="0" />
</xs:sequence>
</xs:complexType>
</xs:element>
</xs:choice>
</xs:complexType>
</xs:element>
</xs:schema>
<diffgr:diffgram xmlns:msdata="urn:schemas-microsoft-com:xml-msdata" xmlns:diffgr="urn:schemas-microsoft-com:xml-diffgram-v1">
<Schedule_x0020_Set>
<Schedule_x0020_Table diffgr:id="Schedule Table1" msdata:rowOrder="0" diffgr:hasChanges="inserted">
<ScheduleDate>1/24/2011</ScheduleDate>
<StaffLName>Smith</StaffLName>
<StaffFName>Josh</StaffFName>
<TaskName>Day Shift</TaskName>
</Schedule_x0020_Table>
<Schedule_x0020_Table diffgr:id="Schedule Table2" msdata:rowOrder="1" diffgr:hasChanges="inserted">
<ScheduleDate>1/24/2011</ScheduleDate>
<StaffLName>Doe</StaffLName>
<StaffFName>Jane</StaffFName>
<TaskName>Night Shift</TaskName>
</Schedule_x0020_Table>
<Schedule_x0020_Table diffgr:id="Schedule Table3" msdata:rowOrder="2" diffgr:hasChanges="inserted">
<ScheduleDate>1/24/2011</ScheduleDate>
<StaffLName>Johnson</StaffLName>
<StaffFName>Barbara</StaffFName>
<TaskName>Off</TaskName>
</Schedule_x0020_Table>
</Schedule_x0020_Set>
</diffgr:diffgram>
</DataSet>
I had to use the following file to get it to work. It only began to work when I changed the schema AND removed the diffgr stuff. Hope this helps.
<DataSet>
<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema" elementFormDefault="qualified">
<xs:element name="DataSet">
<xs:complexType>
<xs:sequence>
<xs:element ref="Schedule_x0020_Set"/>
</xs:sequence>
</xs:complexType>
</xs:element>
<xs:element name="Schedule_x0020_Set">
<xs:complexType>
<xs:sequence>
<xs:element maxOccurs="unbounded" ref="Schedule_x0020_Table"/>
</xs:sequence>
</xs:complexType>
</xs:element>
<xs:element name="Schedule_x0020_Table">
<xs:complexType>
<xs:sequence>
<xs:element ref="ScheduleDate"/>
<xs:element ref="StaffLName"/>
<xs:element ref="StaffFName"/>
<xs:element ref="TaskName"/>
</xs:sequence>
</xs:complexType>
</xs:element>
<xs:element name="ScheduleDate" type="xs:string"/>
<xs:element name="StaffLName" type="xs:NCName"/>
<xs:element name="StaffFName" type="xs:NCName"/>
<xs:element name="TaskName" type="xs:string"/>
</xs:schema>
<Schedule_x0020_Set>
<Schedule_x0020_Table>
<ScheduleDate>1/24/2011</ScheduleDate>
<StaffLName>Smith</StaffLName>
<StaffFName>Josh</StaffFName>
<TaskName>Day Shift</TaskName>
</Schedule_x0020_Table>
<Schedule_x0020_Table>
<ScheduleDate>1/24/2011</ScheduleDate>
<StaffLName>Doe</StaffLName>
<StaffFName>Jane</StaffFName>
<TaskName>Night Shift</TaskName>
</Schedule_x0020_Table>
<Schedule_x0020_Table>
<ScheduleDate>1/24/2011</ScheduleDate>
<StaffLName>Johnson</StaffLName>
<StaffFName>Barbara</StaffFName>
<TaskName>Off</TaskName>
</Schedule_x0020_Table>
</Schedule_x0020_Set>
</DataSet>

Resources