We have a varbinary field storing some XML data
There is a proc which returns this field to C# calling code which decodes the byte array
Encoding.UTF8.GetString((byte[])rdr.GetValue(1))
I don't think this is decoding the byte array correctly, and I'm not sure why.
If I copy the varbinary and convert it manually, it doesn't decode correctly:
select convert(varchar(max),
0x3C4150495061796D656E7473526......)
Ä•–ÖVçG5&WVW7CàТĆVFW#àТÅ5fW#ã"ããÂõ5fW#àТÄ6Æ–VçCå
If however I simply convert the column in the proc
select convert(varchar(max), filedata), filedata from TABLE
It decodes to valid XML.
I can't unfortunately include the actual data. This behaviour is recent and not consistent.
Related
I have old data that also contains an ntext column.
I want to convert it to nvarchar. When I try something like
Select Charindex(Cast(old_column as nvarchar(max))) it say it cuts off data.
Researching I found that nvarchar(max) should be able to store more than ntext
How can I search information inside of ntext? Or how can I convert the information?
We currently have a function in SQL which I simply do not understand.
Currently we convert a nvarchar to XML, and then select the XML value, and convert that to a varbinary.
When I try to simplify this to convert the nvarchar directly to varbinary, the output is different... Why?
--- Current situation:
Declare #inputString nvarchar(max) = '4d95605d1b8f3bca5ea3e0d2af26027004d17218152e726da0622d669a71f85c'
--1: input to XML
declare #inputXML XML = convert(varchar(max), #inputString)
--2: input XML to binary
declare #inputBinray varbinary(max) = #inputXML.value('(/)[1]', 'varbinary(max)')
select #inputString -- 4d95605d1b8f3bca5ea3e0d2af26027004d17218152e726da0622d669a71f85c
select #inputXML -- 4d95605d1b8f3bca5ea3e0d2af26027004d17218152e726da0622d669a71f85c
select #inputBinray -- 0xE1DF79EB4E5DD5BF1FDDB71AE5E6B77B477669FDBAD36EF4D38775EF6D7CD79D9EEF6E9D6B4EB6D9DEBAF5AEF57FCE5C
--- New situation
--1: Input to binary
declare #inputString2 varbinary(max) = CAST(#inputString as varbinary(max));
select #inputString2 -- 0x3400640039003500360030003500640031006200380066003300620063006100350065006100330065003000640032006100660032003600300032003700300030003400640031003700320031003800310035003200650037003200360064006100300036003200320064003600360039006100370031006600380035006300
Using the value() function to get a XML value specified as varbinary(max) will read the data as if it was Base64 encoded. Casting a string to varbinary(max) does not, it treats it as just any string.
If you use the input string QQA= which is the letter A in UTF-16 LE encoded to Base64 you will see more clearly what is happening.
XML gives you 0x4100, the varbinary of the letter A, and direct cast on the string gives you 0x5100510041003D00 where you have two 5100 = "Q" and of course one 4100 = "A" followed by a 3D00 = "="
Might be I get something wrong, but - if I understand you correctly - I think you simply want to get a real binary from a HEX-string, which just looks like a binary. Correct?
Above I wrote "simply", but this was not simple at all a while ago.
I'm not sure at the moment, but I think it was version v2012, which enhanced CONVERT() (read about binary values and how the third parameter works) and try this:
DECLARE #hexString VARCHAR(max)='4d95605d1b8f3bca5ea3e0d2af26027004d17218152e726da0622d669a71f85c';
SELECT CONVERT(varbinary(max),#hexString,2);
The result is a real binary
0x4D95605D1B8F3BCA5EA3E0D2AF26027004D17218152E726DA0622D669A71F85C
What might be the reason for your issue:
Very long ago, I think it was until v2005, the default encoding of varbinaries in XML was a HEX string. Later this was changed to base64. Might be, that you code was used in a very old environment and was upgraded to a higher version?
Today we use XML in a smiliar way to create and to read base64, which is not supported otherwise. Maybe your code did something similar with HEX strings...?
One more hint for this: The many 00 in your New Situation example show clearly, that this is a two-byte encoded NVARCHAR string. Contrary, your Current Situation shows a simple HEX string.
Your final result is just the binary pattern of your input as string:
After referring to the question Base64 encoding in SQL Server 2005 T-SQL, I tried to get the base64 values for some data from sql table but it's not giving proper values when compared to direct text values.
Using Direct text:
SELECT CAST('?' as varbinary) FOR XML PATH(''), BINARY BASE64
gives value as Pw== which is correct and it decodes to ?
Using Database entry:
SELECT CAST([Col] as varbinary) from tblTable FOR XML PATH(''), BINARY BASE64
with [Col] value = ?, gives output as PwA= which when decoded gives ? and an extra non-printable character.
Screenshot when checked using len function.
The reason for this is that I want to convert data for few columns from plain text to base64 using update statement, this is just sample value and actual values are bigger text which is also having this problem but with every character.
Edit: This when decoded from ASP.Net, if it's on label then it displays properly but when used in textbox shows extra junk characters.
Two things:
First, the "Direct Text" example:
SELECT CAST('?' as varbinary) FOR XML PATH(''), BINARY BASE64
----
Pw==
Is encoding the single byte (VARCHAR) character to base 64.
For an NVARCHAR, a 2 bytes per character type, it should be this:
SELECT CAST(N'?' as varbinary) FOR XML PATH(''), BINARY BASE64
----
PwA=
In the sceond part of your question, asking why there is an extra character produced during decoding of your previously encoded NVARCHAR type column. You're actually taking the 2 bytes encoded as base 64 and converting them to 2 single byte (VARCHAR) characters.
In order to decode to NVARCHAR you need to do this:
SELECT CAST(CAST( 'PwA=' as XML ).value('.','varbinary(max)') AS NVARCHAR(250) )
---
?
I'm trying to insert a large string into a nvarchar(max) column, but after inserting I found, that string was cut. There was only 43601 characters stored.
What is wrong?
It stores complete string but when you use select statement SQL interface display only certain number of characters...you can try reading thru .NET/java or try getting data thru SubString().. you should see complete data...
Usually you'll take warning text like 'string or binary will be truncated' in a case of truncation during insert. If you didn't have such warning, then likely You've inserted data successfully.
To check it, you can cast your field with large data to XML type and then open it to ensure what it's been stored well.
Example:
SELECT TOP 1000 [id]
,CAST([text] AS XML)
FROM [AnomalyDetection].[dbo].[TableA]
Result:
And finally, click on data to open it in the editor
This question already has an answer here:
For XML length limitation
(1 answer)
Closed 10 years ago.
I have a query that returns XML which I want to convert to varchar. My query returns 93,643 characters of XML. When I try to cast my xml result as varchar, I only get 43,679 characters when I copy the result set to a text editor. When I do len(xmlString), I get 93,643 characters.
I know from this post that varchar(max) can have up to 2^31 characters and 1 byte = 1 character, but it seems to be cutting off my data.
Do XML characters count as more than 1 byte? Why am I not able to select all the data from my xml result?
CAST((SELECT COLUMNS FROM TABLE FOR XML PATH('Name'), TYPE) AS VARCHAR(MAX)
This is just a limitation of the Managementstudio.
With a testquery on a bigger table I get described 43,679 characters.
The same Query deliveres 267089 characters in a application via ADO.
Not sure why you need to cast you xml data to varchar(max) but if you just want to copy all data don't cast it at all. In this case in the result window you will see one cell with a clickable value (just like a web link). Click it and all your data will be opened in a new window, then you will be able to save it like a file or just copy it. Hope it helps.