Can a Linq query retrieve BLOBs from a Sql Database?
And how do they come out?
LINQ-To-SQL classes create properties of type System.Data.Linq.Binary for all binary and varbinary fields in SQL-Server. The Binary type has a .ToArray() method that returns a byte[], and its constructor can take a byte[].
Older versions of SQLMetal generated properties of type byte[], but the problem with these was that they failed in any joins. I think that's the main reason they replaced it with the IEquatable Binary type.
If I'm not mistaken LINQ to SQL teats BLOB as System.Byte[]
I recall that there was some problem with SqlMetal, it generated wrong type for BLOB, but MSVS dmbl designer should work.
Related
I am learning/doing my first ETL with Apache Nifi, but I have a problem, the data that I am migrating comes to a JSON file and goes to a Postgres database, but it fails on the Insert part
One of the columns in the database is defined as an Enum:
CREATE TYPE "Insurer" AS ENUM ('INSURER-A', 'INSURER-B');
The table has the column name as insurer of type Insurer
And the JSON data has the field insurer as string 'INSURER-A'
Of course, the hint says that I need to do some kind of casting... but I don't have any idea about how to do it, any suggestions?
I assume you're using a Record Reader with Infer Schema set. If so, there's no way for NiFi to know that a string/text field is an enum on the database side. To make matters worse, JDBC doesn't support Enum types per se, they are treated as strings. So in this case both NiFi and the Postgres driver think the field is a string and treat it as such, but then the Postgres DB complains that it is not the right type.
You can get by this by specifying the schema yourself. For example, using the "mood" enum specified here, you can use the following schema:
and then NiFi will know the incoming data is of enum type. However, that uncovers a bug in NiFi where we don't convert the enum type into the JDBC String type and instead report an error. I have written up NIFI-10635 to cover this fix, I expect it to be in the upcoming NiFi 1.19.0 release.
As far as I know you can't do this with putdatabaserecord.
I had similar issue and I wanted to put content of flowfile as a json field to SQL Server (I had more other attribute as well).
What I did was create Insert statement with replace text and then use PutSQL processor.
The content of Replacement value was:
INSERT INTO rbcapi.Archive(TracingNo,Type,Content,ShamsiYearMonth,CreatedAt)
VALUES('${tracingNumber}','${messageType}','$1','${ShamsiYearMonth}','${CreatedAt}');
I wanted to put json value in Content field. $1 means entire content in flowfile (which was json).
Can anyone tell me the difference between typed and untyped XML in SQL Server?
Thanks in advance.
Beside the fact, that this is really easy to find with short research some points which come to my mind quickly:
The typed XML is like a contract: I promise, that you'll find XML-data of a given structure.
This is the base for XML indices, query optimizations and other fancy things.
The more SQL Server knows in advance, the better it will create its execution plans.
And very important: You will not be able to store different XML-structures in the same column.
So: If you know the structure in advance and you know that it won't change to often and you want to store the same kind of XML data in all rows of your table in this certain column, than type it.
One example here: http://www.dotnetfunda.com/interviews/show/583/what-is-typed-vs-untyped-xml-in-sql-server-explain-with-an-example
Typed XML type have XML schemas to validate the Data where as Untyped XML type does not have schema.
Typed XML type has advantage of storage and query optimizations where as Untyped XML type does not have this advantage.
Typed XML types can be validated at both client side and Server side where as Untyped XML types does not have option to validate on server side as they do not have any schemas.
Typed XML types can take advantage of type information during compilations of queries.
Refer these:
(It will give you an idea
)
https://msdn.microsoft.com/en-us/library/ms184277.aspx
http://www.nullskull.com/faq/253/typed-xml-type-vs-untyped-xml-type-in-sql-server.aspx
Also, you can refer quora. I have seen this in quora.
I'm using EF with SQL Server and when I'm fetching a DateTime field that is DateTime on the MSSQL and on EF, I would get something like this:
"UpdateTime":"\/Date(1358716865533+0200)\/"
I have no idea what format is this (only that it shows some time with GMT+2 offset).
Any suggestions?
Second, I've tried to use ToString("") on that fields inside the LINQ that fetches the records and got an error that SQL doesn't support formatting this way - so I have no easy way to format that date as I want.
Is there any way to apply formatting?
Thanks!
As said in the comment, this is a Unix format. You'll get a similar string when you serialize a DateTime with JavaScriptSerializer (JSON).
As for your second question. Yes it can be a nuisance that EF does not allow ToString in linq queries. On the other hand, EF is for data access, formatting is for UI. So you better do the formatting after you've fetched data from the database into memory display them somewhere.
With a XML type column in SQL server, what is the most efficient way to read this back into an XmlDocument in ADO.Net?
For this particular use, an XmlDocument is needed for random-access to the loaded document. Using .Net 4.0 (C#) and SQL Server 2008 R2.
Originally, we had a stored procedure that was returning a result set. When calling SqlDataAdapter.Fill(DataTable) to obtain the results, the XML is returned only as a string.
I then changed this to have the T-SQL return an output parameter of type 'XML', and registered this in .Net as an output parameter of type SqlDbType.Xml. After execution, the resulting output parameter has both a .DbType and .SqlDbType of DbType.Xml - but calling .Value returns a type of System.String.
I can call .SqlValue on this, which returns a type of SqlTypes.SqlXml. From here, I can call its .Value - which just returns a string, or CreateReader - which returns an XmlReader. So I could use this to populate the XmlDocument using XmlDocument.Load(XmlReader).
The main question is - is there any benefit to doing this, or should I just revert back to retrieving the string - then passing this to new XmlDocument().LoadXml(string)? Will using the SqlXml approach allow the XML to be passed between the SQL and the .Net layer more efficiently - or will the XML just be transparently transmitted as a string anyway "on the wire"? Even if the later, I'm guessing that the SqlXml.CreateReader approach may be more efficient, as a string containing the entire XML structure won't ever need to be created in-memory at once - and will plan on using this for now.
(Alternatively, I was looking at using a SqlDataReader and its GetSqlXml method. Interestingly, there are currently only 3 other results on SO for "GetSqlXml", and while How to get XML data from a column in an SQL table? seems to be the most relevant result, it is looking to address a different issue.)
The inverse of this was rather straight-forward: Create the XmlDocument, add an input parameter of type SqlDbType.Xml, pass-in the XmlDocument instance as the value - and done.
I realize that I can profile the differences between these methods myself - and plan to do so once the time is available. Just looking for anyone who may already have the answers, or can provide some alternatives that I haven't yet considered or that I'm not familiar with.
Whatever choice you make should use an XmlReader. Don't force SQL Server to serialize the XML into text form.
Update:
SQL Server stores XML in an efficient binary format. SqlXml.CreateReader creates XmlNode objects from that binary format. Otherwise, the efficient XML has to be serialized to a string, and on top of it has to be parsed again.
Is there a direct route that is pretty straight forward? (i.e. can SQL Server read XML)
Or, is it best to parse the XML and just transfer it in the usual way via ADO.Net either as individual rows or perhaps a batch update?
I realize there may be solutions that involve large complex stored procs--while I'm not entirely opposed to this, I tend to prefer to have most of my business logic in the C# code. I have seen a solution using SQLXMLBulkLoad, but it seemed to require fairly complex SQL code.
For reference, I'll be working with about 100 rows at a time with about 50 small pieces of data for each (strings and ints). This will eventually become a daily batch job.
Any code snippets you can provide would be very much appreciated.
SQL Server 2005 and up have a datatype called "XML" which you can store XML in - untyped or typed with a XSD schema.
You can basically fill columns of type XML from an XML literal string, so you can easily just use a normal INSERT statement and fill the XML contents into that field.
Marc
You can use the function OPENXML and stored procedure sp_xml_preparedocument to easily convert your XML into rowsets.
If you are using SQL Server 2008 (or 2005), it has an xml native datatype. You can associate an XSD schema with xml variables, and Insert directly into columns of type xml.
Yes, SQL Server 2005 and above can parse XML out of the box.
You use the nodes, value and query methods to break it down how you want, whether values or attributes
Some shameless plugging:
Importing XML into SQL Server
Search XML Column in SQL
Xml data and Xml document could have different meaning.
When xml type is good for data, it doesn't save formatting (white spaces removed), so in some cases (e.g. cofiguration files) the best option is nvarchar.