I am pretty new to working with XML/XSD schemas etc. I am trying to use SQLXMLBulkLoad to load an XML file into SQL Server (the file to upload is greater than 2gb) and am having a problem as the source data has element names that contain hyphens, e.g. "first-name" or "start-date".
I have managed to get a few sample records imported into the database excluding those fields, but as soon as I add in any of the elements with a hyphen in the name, the import fails with an error message saying:
Schema: Invalid Value for 'column', Code: 80004005 Source: Schema Mapping
This only happens when there is a hyphen in an element name and I have no idea how to get around it. I have tried searching for the answer but can't find any reference to the issue I'm having.
The schema code is shown below:
<?xml version="1.0" encoding="utf-8"?>
<xsd:schema xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:sql="urn:schemas-microsoft-com:mapping-schema">
<xsd:element name="jobs" sql:is-constant="1">
<xsd:complexType>
<xsd:sequence>
<xsd:element name="job" sql:relation="mytemptable" maxOccurs="unbounded">
<xsd:complexType>
<xsd:sequence>
<xsd:element name="id" type="xsd:string" />
<xsd:element name="title" type="xsd:string" />
<xsd:element name="description" type="xsd:string" />
<xsd:element name="town" type="xsd:string" />
<xsd:element name="start-date" type="xsd:string" />
</xsd:sequence>
</xsd:complexType>
</xsd:element>
</xsd:sequence>
</xsd:complexType>
</xsd:element>
</xsd:schema>
I am unable to get the source data file amended to exclude the hyphens so I have to find a way to handle it in the import routine, but I am still not able to get it to work.
Related
I'm attempting to import a tcx file from Garmin connect into SQL Server with SSIS. Unfortunately, I'm at a complete loss as to how I would create an xsd in order to import the datapoints I want. The Garmin XSD is http://www8.garmin.com/xmlschemas/TrainingCenterDatabasev2.xsd. And the parts I'm concerned about are only the elements here:
<xsd:complexType name="Trackpoint_t">
<xsd:sequence>
<xsd:element type="xsd:dateTime" name="Time"/>
<xsd:element type="Position_t" name="Position" minOccurs="0"/>
<xsd:element type="xsd:double" name="AltitudeMeters" minOccurs="0"/>
<xsd:element type="xsd:double" name="DistanceMeters" minOccurs="0"/>
<xsd:element type="HeartRateInBeatsPerMinute_t" name="HeartRateBpm" minOccurs="0"/>
<xsd:element type="CadenceValue_t" name="Cadence" minOccurs="0"/>
<xsd:element type="SensorState_t" name="SensorState" minOccurs="0"/>
<xsd:element type="Extensions_t" name="Extensions" minOccurs="0">
<xsd:annotation>
</xsd:element>
</xsd:sequence>
Any pointers on how to create the xsd would be helpful. Thanks
If you would like to import part of XML file without modifications, you do not have to design your own xsd. All you need to do is:
Take XSD file from Garmit (presuming it is valid and aligns with your data) and your XML file
In SSIS package, add dataflow task, XML source in it, select your XML file as source and Garmin XSD file as schema file
Alternatively, you can try this, XML Source can design (guess) XSD from XML provided. I would recommend this only as a starting point, since guessed XSD is very approximate and based only on the sample provided; it can misinterpret node and attribute types based on the sample.
XML Source usually produce multitude of outputs, so you have to check with data viewer which one contains data you need.
After playing with this prototype, you can transform package to your needs.
You did not provide your XML-file... But I found one sample of a TCX-file here and another one here. Reading this does not need a schema. I hope, that you can solve your problems, when you see the following example:
DECLARE #xml XML=
N'<TrainingCenterDatabase xmlns="http://www.garmin.com/xmlschemas/TrainingCenterDatabase/v2" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.garmin.com/xmlschemas/TrainingCenterDatabase/v2 http://www.garmin.com/xmlschemas/TrainingCenterDatabasev2.xsd">
<Activities>
<Activity Sport="Running">
<Id>2015-01-25T12:14:34Z</Id>
<Lap StartTime="2015-01-25T12:14:34Z">
<TotalTimeSeconds>507.0989990</TotalTimeSeconds>
<DistanceMeters>1000.0000000</DistanceMeters>
<MaximumSpeed>2.5790000</MaximumSpeed>
<Calories>95</Calories>
<AverageHeartRateBpm xsi:type="HeartRateInBeatsPerMinute_t">
<Value>155</Value>
</AverageHeartRateBpm>
<MaximumHeartRateBpm xsi:type="HeartRateInBeatsPerMinute_t">
<Value>173</Value>
</MaximumHeartRateBpm>
<Intensity>Resting</Intensity>
<TriggerMethod>Distance</TriggerMethod>
<Track>
<Trackpoint>
<Time>2015-01-25T12:14:34Z</Time>
<Position>
<LatitudeDegrees>50.8918607</LatitudeDegrees>
<LongitudeDegrees>16.7403161</LongitudeDegrees>
</Position>
<AltitudeMeters>233.1999969</AltitudeMeters>
<DistanceMeters>0.0000000</DistanceMeters>
<HeartRateBpm xsi:type="HeartRateInBeatsPerMinute_t">
<Value>88</Value>
</HeartRateBpm>
<Extensions>
<TPX xmlns="http://www.garmin.com/xmlschemas/ActivityExtension/v2" CadenceSensor="Footpod">
<Speed>0.0000000</Speed>
</TPX>
</Extensions>
</Trackpoint>
</Track>
<Track />
<Extensions>
<FatCalories xmlns="http://www.garmin.com/xmlschemas/FatCalories/v1">
<Value>0</Value>
</FatCalories>
<LX xmlns="http://www.garmin.com/xmlschemas/ActivityExtension/v2">
<AvgSpeed>1.9720000</AvgSpeed>
</LX>
</Extensions>
</Lap>
</Activity>
</Activities>
</TrainingCenterDatabase>';
WITH XMLNAMESPACES(DEFAULT 'http://www.garmin.com/xmlschemas/TrainingCenterDatabase/v2')
select Act.value(N'#Sport','nvarchar(max)') AS Activity_Sport
,Act.value(N'(Id)[1]','datetime') AS Activity_Id
,Lap.value(N'#StartTime','datetime') AS Lap_StartTime
,Lap.value(N'(TotalTimeSeconds)[1]','decimal(20,8)') AS Lap_TotalTimeSeconds
--more fields
,Lap.value(N'(Calories)[1]','int') AS Lap_TotalTimeSeconds
,Lap.value(N'(AverageHeartRateBpm/Value)[1]','int') AS Lap_AverageHeartRateBpm
,Lap.value(N'(MaximumHeartRateBpm/Value)[1]','int') AS Lap_MaximumHeartRateBpm
,Lap.value(N'(Intensity)[1]','nvarchar(max)') AS Lap_Intensity
--more fields
,TrP.value(N'(Time)[1]','datetime') AS Track_Trackpoint_Time
,TrP.value(N'(Position/LatitudeDegrees)[1]','decimal(20,8)') AS Track_Trackpoint_Position_LatitudeDegrees
--more fields
FROM #xml.nodes(N'/TrainingCenterDatabase/Activities/Activity') AS A(Act)
OUTER APPLY Act.nodes(N'Lap') AS B(Lap)
OUTER APPLY Lap.nodes(N'Track/Trackpoint') AS C(TrP)
The result
Activity_Sport Activity_Id Lap_StartTime Lap_TotalTimeSeconds ....
Running 2015-01-25 12:14:34.000 2015-01-25 12:14:34.000 507.09899900 ....
I have the below json , how can it be represented in XSD. It is a json tuple i could not find a valid construct in XSD to represent this type of json structure.
{
"type": "array",
"items": [
{
"type": "number"
},
{
"type": "string"
},
{
"type": "string",
"enum": ["Street", "Avenue", "Boulevard"]
},
{
"type": "string",
"enum": ["NW", "NE", "SW", "SE"]
}
]
}
It would help others if you would provide more context to your question. First, your example is from page 30 (print version) of the Understanding JSON Schema guide. Your JSON is a JSON schema (draft #4).
Because you're talking about schemas, I take your question as being about models. To be even clearer, I consider your question similar to one asked many years back: can one use UML to represent an XSD structure?
I actually do use XSD to describe JSON structures; we have an automatic conversion between XSD and JSON schemas (draft #4), since I use myself XSD as a data modeling language.
Your particular example has no natural XSD equivalent. By natural, I mean one that would make sense as XML as well, and which a "generic" XML-to-JSON transformation would yield an expected result.
If we take away simple things, such as a simple type modeled as a list:
<xsd:simpleType name="array">
<xsd:list itemType="xsd:float"/>
</xsd:simpleType>
which basically is the same as (except for the comma separator):
{
"type": "array",
"items": {
"type": "number"
}
}
then my answer is really about a "profile" that will be used by a conversion tool to take the semantics of a model represented in XSD (in other words, we're not hung up on XML documents here), into the same represented using JSON schema.
Our XSD-to-JSON schema profile uses a complex type to represent objects and arrays. When the "stereotype" applied to a complex type is "array", then the following model perfectly matches your JSON schema.
<xsd:complexType name="address">
<xsd:annotation>
<xsd:appinfo>
<xsd2json:type>array</xsd2json:type>
</xsd:appinfo>
</xsd:annotation>
<xsd:sequence>
<xsd:element name="StreetNumber" type="xsd:float"/>
<xsd:element name="StreetName" type="xsd:string"/>
<xsd:element name="StreetType">
<xsd:simpleType>
<xsd:restriction base="xsd:string">
<xsd:enumeration value="Street"/>
<xsd:enumeration value="Avenue"/>
<xsd:enumeration value="Boulevard"/>
</xsd:restriction>
</xsd:simpleType>
</xsd:element>
<xsd:element name="StreetDirection">
<xsd:simpleType>
<xsd:restriction base="xsd:string">
<xsd:enumeration value="NW"/>
<xsd:enumeration value="NE"/>
<xsd:enumeration value="SW"/>
<xsd:enumeration value="SE"/>
</xsd:restriction>
</xsd:simpleType>
</xsd:element>
</xsd:sequence>
</xsd:complexType>
I am still experimenting with the SimpleXML framework. As a first step I am parsing a larger XML file and then immediately emitting it as XML again, i.e. my test-program so far looks like:
Persister serializer = new Persister(new TypeMatcher());
File source = new File(filenameIn);
List myList = serializer.read(List.class, source);
serializer.write(myList, outfile);
This works all fine, however, what I noticed is, that the fields of the List are emitted in the wrong order!
The schema that my List has to follow reads:
<xs:element name="list">
<xs:complexType>
<xs:sequence>
<xs:element name="properties" type="Properties" maxOccurs="1" minOccurs="1" />
<xs:element name="columns" type="Columns" maxOccurs="unbounded" minOccurs="1" />
<xs:element name="items" type="Items" maxOccurs="unbounded" minOccurs="0" />
</xs:sequence>
</xs:complexType>
</xs:element>
When the list is written, however, the sequence that I get is:
<list>
<items>
...
</items>
<properties ...>
...
</properties>
<columns>
...
</columns>
</list>
i.e. the items, which should come last, are emitted first. That causes error on later reading again, since the items refer to data that are define in the columns, which are then still undefined, if that order is not maintained.
M.
For maintaining order on serialization the #Order annotation can be used (see this tutorial).
Also, in the specific context of serializing with simple-framework, Java List does not usually guarantee order, but you can use LinkedList for that matter.
there are some strange brain racked occurring to me.
In a given xsd file,we find a xml element that says as below:
<xsd:element name="getAllOperationsSystemsRequest">
<xsd:annotation>
<xsd:documentation>
</xsd:documentation>
</xsd:annotation>
</xsd:element>
and a wsdl operation is linked to the element
<wsdl:operation name="getAllOperationsSystems">
<wsdl:input message="tns:getAllOperationsSystemsRequest"/>
<wsdl:output message="tns:getAllOperationsSystemsResponse"/>
<wsdl:fault name="getAllOperationsSystemsException" message="tns:getAllOperationsSystemsException"/>
</wsdl:operation>
Apparently , getAllOperationsSystemsRequest is not bound to any known type ( the attribute "type" is missing.
As the result,after we exeucte wsdl2java tool,we finally get a method definition :
public org.tmforum.mtop.mri.xsd.osr.v1.MultipleObjectsResponseType getAllOperationsSystems(
javax.xml.ws.Holder<org.tmforum.mtop.fmw.xsd.hdr.v1.Header> mtopHeader,
java.lang.**Object** mtopBody
)throws GetAllOperationsSystemsException
An Object instead of OperationsSystemsRequest is generated as the input parameter type,(actually the OperationsSystemsRequest never come into existence.
Finally ,we get a bold runtime error in marshall/unmarshall
Caused by: com.sun.istack.SAXException2: Instance of "javax.xml.bind.JAXBElement" is substituting "java.lang.Object", but "javax.xml.bind.JAXBElement" is bound to an anonymous type.
I am really appreciated anyone for information to solve it. Tnanks in advance.
Per XML Schema spec, the "type" for the getAllOperationsSystemsRequest element is an xsd:anyType. Basically, anything. That is why Object is generated in the code. The code generators generate for types, not elements (mostly). If the type is only represented in an element, then it would get an #XmlRootElement annotation, but for the most part, you have to think about the types.
I would recommend changing the schema to:
<xsd:element name="getAllOperationsSystemsRequest">
<xsd:annotation>
<xsd:documentation>
</xsd:documentation>
</xsd:annotation>
<xsd:complexType>
<xsd:sequence/>
</xsd:complexType>
</xsd:element>
That should generate a type and restrict the element to being an empty element.
I'm new to ADF mobile application development. I'm currently blocked in a scenario. I have a web service(from third party) which needs to retrieve data. In soapUI, the web service will retrieve the response only when there is this security tag. Otherwise it will give response as invalid security. The web service looks like this:
<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:xx="http://xmlns.oracle.com/apps/csf/soaprovider/plsql/xx_fs_mob_login/" xmlns:get="http://xmlns.oracle.com/apps/csf/soaprovider/plsql/xx_fs_mob_login/get_login/">
<soapenv:Header>
<xx:SOAHeader>
<!--Optional:-->
<xx:Responsibility>XXX</xx:Responsibility>
<!--Optional:-->
<xx:RespApplication>XXX</xx:RespApplication>
<!--Optional:-->
<xx:SecurityGroup>XXX</xx:SecurityGroup>
<!--Optional:-->
<xx:NLSLanguage>XXX</xx:NLSLanguage>
<!--Optional:-->
<xx:Org_Id>XXX</xx:Org_Id>
</xx:SOAHeader>
<The portion which is excluded from the soap, but which is required for getting response>
<wsse:Security soapenv:mustUnderstand="1" xmlns:wsse="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd" xmlns:wsu="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd">
<wsse:UsernameToken wsu:Id="UsernameToken-1">
<wsse:Username>XXX</wsse:Username>
<wsse:Password Type="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-username-token-profile-1.0#PasswordText">XXX</wsse:Password>
<wsse:Nonce EncodingType="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-soap-message-security-1.0#Base64Binary">XXX</wsse:Nonce>
<wsu:Created>2013-02-13T08:58:50.649Z</wsu:Created>
</wsse:UsernameToken>
</wsse:Security>
<The portion which is excluded from the soap, but which is required for getting response>
</soapenv:Header>
<soapenv:Body>
<get:InputParameters>
<!--Optional:-->
<get:P_USERNAME>XXX</get:P_USERNAME>
</get:InputParameters>
</soapenv:Body>
</soapenv:Envelope>
I tried the steps described in ADF mobile tutorial to get web service data. But I'm getting response code 500 from the server. I tried the steps for secured web service also. But I'm not sure which security policy has been implemented in the web service. I tried with oracle/wss_username_client_token_policy and some others but didn't succeed. Later I tried by creating web service client/proxy. But as Oracle ADF only supports java 1.4, I got errors in the generated code(errors on generics and annotation).
The WSDL for the web service is as follows:
<definitions xmlns:tns="http://xmlns.oracle.com/apps/csf/soaprovider/plsql/xx_fs_mob_login/" xmlns="http://schemas.xmlsoap.org/wsdl/" xmlns:soap="http://schemas.xmlsoap.org/wsdl/soap/" xmlns:tns1="http://xmlns.oracle.com/apps/csf/soaprovider/plsql/xx_fs_mob_login/get_login/" name="XX_FS_MOB_LOGIN" targetNamespace="http://xmlns.oracle.com/apps/csf/soaprovider/plsql/xx_fs_mob_login/">
<types>
<schema xmlns="http://www.w3.org/2001/XMLSchema" elementFormDefault="qualified" targetNamespace="http://xmlns.oracle.com/apps/csf/soaprovider/plsql/xx_fs_mob_login/get_login/">
<include schemaLocation="http://27.251.157.211:8000/webservices/SOAProvider/plsql/xx_fs_mob_login/APPS_XX_FS_MOB_LOGIN_GET_LOGIN.xsd"/>
</schema>
<schema xmlns="http://www.w3.org/2001/XMLSchema" elementFormDefault="qualified" targetNamespace="http://xmlns.oracle.com/apps/csf/soaprovider/plsql/xx_fs_mob_login/">
<element name="SOAHeader">
<complexType>
<sequence>
<element name="Responsibility" minOccurs="0" type="string"/>
<element name="RespApplication" minOccurs="0" type="string"/>
<element name="SecurityGroup" minOccurs="0" type="string"/>
<element name="NLSLanguage" minOccurs="0" type="string"/>
<element name="Org_Id" minOccurs="0" type="string"/>
</sequence>
</complexType>
</element>
</schema>
</types>
<message name="GET_LOGIN_Input_Msg">
<part name="header" element="tns:SOAHeader"/>
<part name="body" element="tns1:InputParameters"/>
</message>
<message name="GET_LOGIN_Output_Msg">
<part name="body" element="tns1:OutputParameters"/>
</message>
<portType name="XX_FS_MOB_LOGIN_PortType">
<operation name="GET_LOGIN">
<input message="tns:GET_LOGIN_Input_Msg"/>
<output message="tns:GET_LOGIN_Output_Msg"/>
</operation>
</portType>
<binding name="XX_FS_MOB_LOGIN_Binding" type="tns:XX_FS_MOB_LOGIN_PortType">
<soap:binding style="document" transport="http://schemas.xmlsoap.org/soap/http"/>
<operation name="GET_LOGIN">
<soap:operation soapAction="http://XXX:8000/webservices/SOAProvider/plsql/xx_fs_mob_login/"/>
<input>
<soap:header message="tns:GET_LOGIN_Input_Msg" part="header" use="literal"/>
<soap:body parts="body" use="literal"/>
</input>
<output>
<soap:body use="literal"/>
</output>
</operation>
</binding>
<service name="XX_FS_MOB_LOGIN_Service">
<port name="XX_FS_MOB_LOGIN_Port" binding="tns:XX_FS_MOB_LOGIN_Binding">
<soap:address location="http://XXX:8000/webservices/SOAProvider/plsql/xx_fs_mob_login/"/>
</port>
</service>
</definitions>
Please help me to figure out a solution for this.
Thanks in advance
Rino
You won't be able to get far in accessing a secure Web service directly from ADF Mobile if you don't know the type of security it uses.
If you can create a proxy client that invokes the web service, then one solution is to run that on the server, and expose that client as a web service that the ADF Mobile layer can access.