How to convert varbinary(max) to base64 SQL Server 2014 - sql-server

I have an Image saved as varbinary(max) in SQL Server 2014:
0xFFD8FFE115064578696600004D4D002A0000000800070...........
I want to convert it to Base64 To use it in Flutter.
I tried
SELECT CAST('' as varbinary(max)) FOR XML PATH(''), BINARY BASE64
and get :
MHhGRkQ4RkZFMTE1MDY0NTc4Njk2NjAwMDA0RDREMDAyQTAwMDAwMDA4MDAwN..........
But according to this site I should get:
/9j/4RUGRXhpZgAATU0AKgAAAAgABwESAAMAAAABAAEAAAEaAAUAAAABAAAAYgEbAA........
So how to convert varbinary(max) to base64?

Why are you attempting to CAST() the varbinary data? You just need to select it as an element or an attribute for the varbinary value to get base64 encoded...
/*
* Data setup...
*/
if object_id('tempdb..#demo') is not null
drop table #demo;
create table #demo (
fancyImage varbinary(max)
);
insert #demo (fancyImage) values (0xFFD8FFE115064578696600004D4D002A000000080007);
/*
* Select as an element containing base64 data
*/
select fancyImage as [base64DemoElement]
from #demo
for xml path(''), binary base64;
/*
* Select as an attribute containing base64 data
*/
select fancyImage as [#base64Attribute]
from #demo
for xml path('demoElement'), binary base64;
The first select outputs the base data in an element:
<base64DemoElement>/9j/4RUGRXhpZgAATU0AKgAAAAgABw==</base64DemoElement>
The second select outputs the base64 data in an attribute:
<demoElement base64Attribute="/9j/4RUGRXhpZgAATU0AKgAAAAgABw==" />
Following comments discussion with #DaleK, a third alternative to return the bare base64 characters without any XML tags:
select (
select top 1 cast(fancyImage as varbinary(max)) as [base64DemoElement]
from #demo
for xml path(''), type, binary base64
).value('.', 'varchar(max)') as [Base64 characters];
Which outputs:
Base64 characters
/9j/4RUGRXhpZgAATU0AKgAAAAgABw==

To select a bare Bas64 value in SQL Server, without any XML node, you just need an unnamed column in FOR XML
SELECT CAST(fancyImage AS varbinary(max))
FROM #demo
FOR XML PATH(''), BINARY BASE64;
Or as a correlated subquery
SELECT
myBase64 = (
SELECT CAST(fancyImage AS varbinary(max))
FOR XML PATH(''), BINARY BASE64
)
FROM #demo;
db<>fiddle

I think it was v2008 of SQL-Server, when base64 was made the default in XML for binaries (before it was a hex string). No need to specify this explicitly.
(The option BINARY BASE64 is needed with mode AUTO...)
Just to demonstrate the back and forth I declare some text (a chain of characters) and cast it to binary (the same chain of bytes, but not a string any more):
DECLARE #someText VARCHAR(100) = 'This is just some text...';
DECLARE #binary VARBINARY(MAX) = CAST(#someText AS VARBINARY(MAX));
--In this case it's okay to rely on implicit casting: easy approach
DECLARE #base64_easy VARCHAR(100) = (SELECT #binary FOR XML PATH(''));
--Just to demonstrate that the base64 we found (VGhpcyBpcyBqdXN0IHNvbWUgdGV4dC4uLg==) is correct we reconvert it simply by casting it to XML and using .value() to retrieve it as binary:
DECLARE #reConverted VARBINARY(MAX) = (SELECT CAST(#base64_easy AS XML).value('.','varbinary(max)'));
--Casting this chain of bytes into a varchar again will show its (unchanged) content:
SELECT CAST(#reConverted AS VARCHAR(100));
All of this can be used within ad-hoc queries.
Hint:
The more explicit statement SELECTs the value into XML and reads this into text via .value()
(The ,type is needed to allow for XML methods)
DECLARE #base64 VARCHAR(100) = (SELECT #binary FOR XML PATH(''), type).value('.','nvarchar(max)'); --VGhpcyBpcyBqdXN0IHNvbWUgdGV4dC4uLg==

Related

Read Xml Field From Image field Sql Server

I am trying to change the code below and read it from the image field
DECLARE #xml XML
SELECT #xml= x FROM OPENROWSET (BULK ''' + #XMLFILE + ''', SINGLE_BLOB) AS T(x)
How Can I Convert To Xml Type?
SELECT #xml= FileField FROM FileListTable (FileField is İmage Field)
Try:
SELECT #xml = CAST(CAST(FileField AS VARBINARY(MAX)) AS XML)
FROM FileListTable
Whether this works or not, may depend on how the XML was converted to the IMAGE data type before being inserted into your table. This db<>fiddle exercises various ways of inserting XML into an IMAGE column and several ways (both good and bad) to try to get it out. Almost surprisingly, the cast to VARBINARY(MAX) and then to XML seems to handle the cases demonstrated.

Convert UTF-8 varbinary(max) to varchar(max)

I have a varbinary(max) column with UTF-8-encoded text that has been compressed. I would like to decompress this data and work with it in T-SQL as a varchar(max) using the UTF-8 capabilities of SQL Server.
I'm looking for a way of specifying the encoding when converting from varbinary(max) to varchar(max). The only way I've managed to do that is by creating a table variable with a column with a UTF-8 collation and inserting the varbinary data into it.
DECLARE #rv TABLE(
Res varchar(max) COLLATE Latin1_General_100_CI_AS_SC_UTF8
)
INSERT INTO #rv
SELECT SUBSTRING(Decompressed, 4, DATALENGTH(Decompressed) - 3) WithoutBOM
FROM
(SELECT DECOMPRESS(RawResource) AS Decompressed FROM Resource) t
I'm wondering if there is a more elegant and efficient approach that does not involve inserting into a table variable.
UPDATE:
Boiling this down to a simple example that doesn't deal with byte order marks or compression:
I have the string "Hello 😊" UTF-8 encoded without a BOM stored in variable #utf8Binary
DECLARE #utf8Binary varbinary(max) = 0x48656C6C6F20F09F988A
Now I try to assign that into various char-based variables and print the result:
DECLARE #brokenVarChar varchar(max) = CONVERT(varchar(max), #utf8Binary)
print '#brokenVarChar = ' + #brokenVarChar
DECLARE #brokenNVarChar nvarchar(max) = CONVERT(varchar(max), #utf8Binary)
print '#brokenNVarChar = ' + #brokenNVarChar
DECLARE #rv TABLE(
Res varchar(max) COLLATE Latin1_General_100_CI_AS_SC_UTF8
)
INSERT INTO #rv
select #utf8Binary
DECLARE #working nvarchar(max)
Select TOP 1 #working = Res from #rv
print '#working = ' + #working
The results of this are:
#brokenVarChar = Hello 😊
#brokenNVarChar = Hello 😊
#working = Hello 😊
So I am able to get the binary result properly decoded using this indirect method, but I am wondering if there is a more straightforward (and likely efficient) approach.
I don't like this solution, but it's one I got to (I initially thought it wasn't working, due to what appears to be a bug in ADS). One method would be to create a new database in a UTF8 collation, and then pass the value to a function in that database. As the database is in a UTF8 collation, the default collation will be different to the local one, and the correct result will be returned:
CREATE DATABASE UTF8 COLLATE Latin1_General_100_CI_AS_SC_UTF8;
GO
USE UTF8;
GO
CREATE OR ALTER FUNCTION dbo.Bin2UTF8 (#utfbinary varbinary(MAX))
RETURNS varchar(MAX) AS
BEGIN
RETURN CAST(#utfbinary AS varchar(MAX));
END
GO
USE YourDatabase;
GO
SELECT UTF8.dbo.Bin2UTF8(0x48656C6C6F20F09F988A);
This, however, isn't particularly "pretty".
There is an undocumented hack:
DECLARE #utf8 VARBINARY(MAX)=0x48656C6C6F20F09F988A;
SELECT CAST(CONCAT('<?xml version="1.0" encoding="UTF-8" ?><![CDATA[',#utf8,']]>') AS XML)
.value('.','nvarchar(max)');
The result
Hello 😊
This works even in versions without the new UTF8 collations...
UPDATE: calling this as a function
This can easily be wrapped in a scalar function
CREATE FUNCTION dbo.Convert_UTF8_Binary_To_NVarchar(#utfBinary VARBINARY(MAX))
RETURNS NVARCHAR(MAX)
AS
BEGIN
RETURN
(
SELECT CAST(CONCAT('<?xml version="1.0" encoding="UTF-8" ?><![CDATA[',#utfBinary,']]>') AS XML)
.value('.','nvarchar(max)')
);
END
GO
Or like this as an inlined table valued function
CREATE FUNCTION dbo.Convert_UTF8_Binary_To_NVarchar(#utfBinary VARBINARY(MAX))
RETURNS TABLE
AS
RETURN
SELECT CAST(CONCAT('<?xml version="1.0" encoding="UTF-8" ?><![CDATA[',#utfBinary,']]>') AS XML)
.value('.','nvarchar(max)') AS ConvertedString
GO
This can be used after FROM or - more appropriate - with APPLY
DECLARE #utf8Binary varbinary(max) = 0x48656C6C6F20F09F988A;
DECLARE #brokenNVarChar nvarchar(max) = concat(#utf8Binary, '' COLLATE Latin1_General_100_CI_AS_SC_UTF8);
print '#brokenNVarChar = ' + #brokenNVarChar;
You didn't say how your data is compressed or what compression algorithm was used. But if you are using the COMPRESS function in SQL Server 2016 or later, you can use the DECOMPRESS function and then cast it to a VARCHAR(MAX). Both COMPRESS and DECOMPRESS use the GZip compression algorithm.
This function will decompress an input expression value, using the GZIP algorithm. DECOMPRESS will return a byte array (VARBINARY(MAX) type).
CAST(DECOMPRESS([compressed content here]) AS VARCHAR(MAX))
See: COMPRESS (Transact-SQL) and DECOMPRESS (Transact-SQL)

TSQL "Illegal XML Character" When Converting Varbinary to XML

I'm trying to create a stored procedure in SQL Server 2016 that converts XML that was previously converted into Varbinary back into XML, but getting an "Illegal XML character" error when converting. I've found a workaround that seems to work, but I can't actually figure out why it works, which makes me uncomfortable.
The stored procedure takes data that was converted to binary in SSIS and inserted into a varbinary(MAX) column in a table and performs a simple
CAST(Column AS XML)
It worked fine for a long time, and I only began seeing an issue when the initial XML started containing an ® (registered trademark) symbol.
Now, when I attempt to convert the binary to XML I get this error
Msg 9420, Level 16, State 1, Line 23
XML parsing: line 1, character 7, illegal xml character
However, if I first convert the binary to varchar(MAX), then convert that to XML, it seems to work fine. I don't understand what is happening when I perform that intermediate CAST that is different than casting directly to XML. My main concern is that I don't want to add it in to account for this scenario and end up with unintended consequences.
Test code:
DECLARE #foo VARBINARY(MAX)
DECLARE #bar VARCHAR(MAX)
DECLARE #Nbar NVARCHAR(MAX)
--SELECT Varbinary
SET #foo = CAST( '<Test>®</Test>' AS VARBINARY(MAX))
SELECT #foo AsBinary
--select as binary as varchar
SET #bar = CAST(#foo AS VARCHAR(MAX))
SELECT #bar BinaryAsVarchar -- Correct string output
--select binary as nvarchar
SET #nbar = CAST(#foo AS NVARCHAR(MAX))
SELECT #nbar BinaryAsNvarchar -- Chinese characters
--select binary as XML
SELECT TRY_CAST(#foo AS XML) BinaryAsXML -- ILLEGAL XML character
-- SELECT CONVERT(xml, #obfoo) BinaryAsXML --ILLEGAL XML Character
--select BinaryAsVarcharAsXML
SELECT TRY_CAST(#bar AS XML) BinaryAsVarcharAsXML -- Correct Output
--select BinaryAsNVarcharAsXML
SELECT TRY_CAST(#nbar AS XML) BinaryAsNvarcharAsXML -- Chinese Characters
There are several things to know:
SQL-Server is rather limited with character encodings. There is VARCHAR, which is 1-byte-encoded extended ASCII and NVARCHAR, which is UCS-2 (almost the same as utf-16).
VARCHAR uses plain latin for the first set of characters and a codepage-mapping provided by the collation in use for the second set.
VARCHAR is not utf-8. utf-8 works with VARCHAR, as long as all characters are 1-byte-enocded. But utf-8 knows a lot of 2-byte-enocded (up to 4-byte-enocded) characters, which would break the internal storage of a VARCHAR string.
NVARCHAR will work with almost any 2-byte encoded character natively (that means with almost any existing character). But it is not exactly utf-16 (there are 3-byte encoded characters, which would break SQL-Servers internal storage).
XML is not stored as the XML-string you see, but as an hierarchically organised physical table, based on NVARCHAR values.
The natively stored XML is really fast, while any text-based storage will need a very expensive parse-operation in advance (over and over...).
Storing XML as string is bad, storing XML as VARCHAR string is even worse.
Storing a VARCHAR-string-XML as VARBINARY is a cummulation of things you should not do.
Try this:
DECLARE #text1Byte VARCHAR(100)='<test>blah</test>';
DECLARE #text2Byte NVARCHAR(100)=N'<test>blah</test>';
SELECT CAST(#text1Byte AS VARBINARY(MAX)) AS text1Byte_Binary
,CAST(#text2Byte AS VARBINARY(MAX)) AS text2Byte_Binary
,CAST(#text1Byte AS XML) AS text1Byte_XML
,CAST(#text2Byte AS XML) AS text2Byte_XML
,CAST(CAST(#text1Byte AS VARBINARY(MAX)) AS XML) AS text1Byte_XML_via_Binary
,CAST(CAST(#text2Byte AS VARBINARY(MAX)) AS XML) AS text2Byte_XML_via_Binary
The only difference you'll see are the many zeros in 0x3C0074006500730074003E0062006C00610068003C002F0074006500730074003E00. This is due to the 2-byte-encoding of nvarchar, each second byte is not needed in this sample. But if you'd need far-east-characters the picture would be completely different.
The reason why it works: SQL-Server is very smart. The cast from the variable to XML is rather easy, as the engine knows, that the underlying variable is varchar or nvarchar. But the last two casts are different. The engine has to examine the binary, whether it is a valid nvarchar and will give it a second try with varchar if it fails.
Now try to add your registered trademark to the given example. Add it first to the second variable DECLARE #text2Byte NVARCHAR(100)=N'<test>blah®</test>'; and try to run this. Then add it to the first variable and try it again.
What you can try:
Cast your binary to varchar(max), then to nvarchar(max) and finally to xml.
,CAST(CAST(CAST(CAST(#text1Byte AS VARBINARY(MAX)) AS VARCHAR(MAX)) AS NVARCHAR(MAX)) AS XML) AS text1Byte_XML_via_Binary
This will work, but it won't be fast...

Convert SQL Server varbinary field to string?

I have a column in a sql server 2005 database that stores xml as varbinary,
the value lookes something like:
0x3C3F54657374696E672075706C6F616 // but much longer
Is there an online conversion tool for converting this to a readable string?
select CAST(0x3C3F54657374696E672075706C6F61643F3E3C666F6F3E3C2F666F6F3E as XML)
appears to work.
Online Conversion Link
And as it seems your datatype is image...
;with t(c) as
(
select CAST(0x3C3F54657374696E672075706C6F61643F3E3C666F6F3E3C2F666F6F3E as IMAGE)
)
select CAST(CAST(c as VARBINARY(MAX)) as XML)
from t
You can use the undocumented stored procedure.
SELECT fn_varbintohexstr(#YourVarbar)
You should also be able to just cast it to a varchar(max) as well
Select CAST(#YourVarbar as Varchar(max))
Third alternative is to create an XML document out of it and use forXML
Example at the following location http://beyondrelational.com/blogs/jacob/archive/2009/06/13/converting-varbinary-to-varchar-using-for-xml.aspx
DECLARE #x VARBINARY(10)
SELECT #x = CAST('10' as VARBINARY(10))
SELECT #x AS VarBinaryValue
SELECT (
SELECT
CHAR(SUBSTRING(#x,number,1)) AS 'text()'
FROM master..spt_values
WHERE type = 'P'
AND Number BETWEEN 1 AND LEN(#x)
FOR XML PATH('')
) AS TextValue

ms sql xml data type convert to text

in MS Sql there are data types that are not supported by delphi 7, the xml datatype is one example.
I wish to convert the XML datatype to Text datatype, so that i could handle it in delphi.
Is there a way to convert from xml to text?
A simple cast will suffice:
select cast(XMLCol as nvarchar(max)) as XMLCol
Or for non-unicode:
select cast(XMLCol as varchar(max)) as XMLCol
You can't convert explicitly to a 'text' data type.
I've added the as XMLCol to ensure that the converted data has the the same name as the column. You needn't have this, of course.
EDIT:
A few links. You are encouraged to use nvarchar(max) instead of text regardless. Microsoft have said they will be deprecating these types in future releases. nvarchar(max) ought to offer you 2GB:
http://www.petefreitag.com/item/734.cfm
http://www.teratrax.com/articles/varchar_max.html
http://msdn.microsoft.com/en-us/library/ms187752(v=SQL.90).aspx
SELECT CAST(YourXMLColumn as nvarchar(max))
FROM YourTable
I just tried the follwing solution and yes, you do need the as XMLCol
select cast(XMLCol as nvarchar(max)) as XMLCol

Resources