Is that possible to parse and import an arbitrary XML file to SQL Server tables using C#?
The XML file can be highly hierarchical.
I have 1 Gb XML-file, I do not know anything about it.
How can I determine what tables it contains, create them at SQL Server and bulk load the data to SQL Server?
Is that possible?
No.
XML and SQL are very, very different things. It is possible to take SQL data an save it into XML, and such SQL-like XML can be converted back to an SQL database, but arbitrary XML that did not originate in an SQL-like system is not likely to be easily convertible to an SQL database schema in any meaningful way.
You could work out an SQL schema that represents generic XML in a key-value type design but it would not be an SQL schema in the traditional meaning of the word.
Related
I have to move data from existing database oracle to which I don't have direct access. The data is about 11 tables, 5GB each. The database admin can export the tables to some .csv or xml. The problem with csv is that some data is textual with lots of special characters. The problem with xml is that the markup is an overhead which will increase significantly the size of the files. The DBA admin is not competent enough to provide a working and neat solution. He uses toad as the database tool. Can you provide some ideas how to perform such a migration in the best possible way?
Please refer the below steps to migrate the data from Oracle to SQL server.
Recommended Migration Process
To successfully migrate objects and data from Oracle databases to SQL Server, Azure SQL DB, or Azure SQL Data Warehouse, use the following process:
1.Create a new SSMA project.
2.After you create the project, you can set project conversion, migration, and type mapping options. For information about project settings, see Setting Project Options (OracleToSQL). For information about how to customize data type mappings, see Mapping Oracle and SQL Server Data Types (OracleToSQL).
3.Connect to the Oracle database server.
4.Connect to an instance of SQL Server.
5.Map Oracle database schemas to SQL Server database schemas.
6.Optionally, Create assessment reports to assess database objects for conversion and estimate the conversion time.
7.Convert Oracle database schemas into SQL Server schemas.
8.Load the converted database objects into SQL Server.
You can do this in one of the following ways:
* Save a script and run it in SQL Server.
* Synchronize the database objects.
9. Migrate data to SQL Server.
10.If necessary, update database applications.
For more details :
[https://learn.microsoft.com/en-us/sql/ssma/oracle/migrating-oracle-databases-to-sql-server-oracletosql?view=sql-server-2017]
After the admin export data into CSV, try to convert it into a character set which will recognize all special characters.
Then, try to follow the steps from this link: link, it might work.
If after the import, there are still special characters, thy to manually convert them.
Get the DBA to export the tables using the ASCII delimiters which were designed for this purpose:
Row delimiter: Decimal 30 / 0x1E
Column delimiter: Decimal 31 / 0x1F
Then you can use BCP (or any other similar product) to upload the data to SQL Server.
Is it even possible to import .csv flat file data into a SQL Server 2014 table using only the SSMS or SSIS Import/Export Wizards, if the table contains a varbinary(max) column?
I have searched hours and hours, and tried numerous configurations and different data types (e.g. DT_IMAGE) in both the SSMS and SSIS Import/Export Wizards to perform a simple, quick-n-dirty import of .csv file data into a four column table containing a varbinary(max) column in SQL Server.
I realize there are various other means to accomplish this by writing Trans SQL, using bulk-copy, adding column-import tasks, etc., but I find it hard to believe that I can't use the simple point-n-click configuration of the Import/Export Wizard, simply because the data happens to contain a varbinary(max) field, so I assume I must be doing something wrong.
Below are screen shots from the SSMS Import/Export Wizard...I get the same error in both SSMS and SSIS:
You can use DT_TEXT An ANSI/MBCS character string with a maximum length of 2^31-1 (2,147,483,647) characters.
Integration Service Data types.
It Will be varbinary(max) in database.
I have used the sample data provide by you and imported it in the database.
I'm trying to move data from Oracle to SQL Server. We have a large number of NUMBER fields in the original Oracle 11g schema. In SSIS, we are using an OLE destination control. Most of the NUMBER fields are mapping to float fields.
We are using the attunity connectors. I understand that there are mapping xml files in SSIS that will map numeric data types to SSIS data types; however, I need something that will take a sample of the data and infer the correct numeric, sql server destination data type.
Is there a solution for this problem? There are a lot of fields, we are worried about manually mapping them.
Thanks!
I want to transfer data from an SQL Server 2008 R2 database to a Oracle 11g Database, A very straightforward ETL operation. But in the SQL Server a Database is using the Filestream functionality to store certain videos and images, but the Oracle database does not have this functionality to my knowledge.
Is there anyone out there who has come across this kind of situation earlier? What was the solution you applied? Simply stored the binary files in a separate server or simply used the BLOB type to store the files?
Thanks in advance....
It sounds like you are looking for the BFILE type. This is an Oracle data type that allows you to work with binary data that is stored on the file system outside the database.
What are the benefits of storing XML in SQL Server over storing JSON in a varchar field?
Any tutorial available for how to use the XML data type effectively?
Do I need to provide the dtd / xml schema somehow? I've heard it is optional, right?
Thank you.
UPDATE: here's the answer to the last part of the Q.
XML schema information is used in
storage and query optimizations. Typed
XML instances contain typed values in
the internal, binary representation as
well as in XML indexes. This provides
efficient processing of typed XML
data.
quoted from: http://msdn.microsoft.com/en-us/library/ms345117(SQL.90).aspx
XML in SQL Server 2005 and up allows you to directly manipulate the XML stored inside the database table using XQuery - you can't really do that with JSON in a VARCHAR field.
Check out XML Support in Microsoft SQL Server 2005 and Fundamentals of the XML Datatype in SQL Server 2005 for more info and more background.
Also, the XML stored in a XML column in SQL Server is "tokenized" and pre-parsed - it's not just stored as plain text. And you can even put indices on your XML - on its nodes, its values - whatever you need.
Storing it as XML allows you to leverage the SQL XML support: XPATH, XQUERY, XML Indexes and such. These allow for efficient search and manipulation of the content. I recommend you read XML Best Practices for Microsoft SQL Server 2005
JSON content in VARCHAR would be opaque to searches and manipulation.
XML can be indexed for more performant querying.
Data can be extracted from XML data usng XPath.
A schema can be provided to constrain the XML to a secifiation but this is optional.
Client libraries understan the XML data type and can send/receive it more carefully/easily.
None of the above is available for JSON stored in varchar.
you can query for individual xml property values if you use the xml data type. I don't believe the same functionality is available for json:
http://msdn.microsoft.com/en-us/library/ms191474.aspx
I don't believe that the xsd is required as I've used the query feature without having that defined before
Storing XML in SQL 2005 is great. You can put the whole XML file in a single field, then run SELECT commands to pull out certain attributes and elements. Putting a XSD is not neccesary. I don't believe it has any JSON support even in 2008 although I could be wrong. Here's a good starter article on it.