I am updating a CGI application that accesses an MSSQL 2008 database containing customer data. The database is managed by a third-party application, so I cannot change the data structure.
One of the tables ('guests') contains a column 'mug_shot' of type 'Image'. The column contains a JPEG image of each guest. When I retrieve data from this column, it always appears to be in text format. For example, when I perform the following query:
my $mugshotQuery = "SELECT TOP 1 mug_shot FROM guests where guest_no = ?";
my $mugshotStatementHandle = $dbh->prepare($mugshotQuery);
$mugshotStatementHandle->execute($guest_number);
and fetch the data:
my $mugshotHash = $mugshotStatementHandle->fetchrow_hashref();
$mugshotHash->{mug_shot} contains a hexadecimal representation of the JPEG binary data. Here is a shortened example:
ffd8ffe000104a46494600010101004c004c0000ffe1004245786966000049492a000800000002001a01050001000000260000001b0105000100000030000000000000007d3b8c0440420f000000ed168e0440420f000000ffdb0043000302020302020303030304030304050805050404050a070706080c0a0c0c0b0a0b0b0d0e12100d0e110e0b0b1016101113141515150c0f171816141812141514ffdb00430103040405040509050509140d0b0d1414141414141414141414141414141414141414141414141414141414141414141414141414141414141414141414141414ffc00011080156010003012200021101031101ffc4001f0000010501010101010100000000000000000102030405060708090a0bffc400b5100002010303020403050504040000017d01020300041105122131410613516107227114328191a1082342b1c11552d1f02433627282090a161718191a25262728292a3435363738393a434445464748494a535455565758595a636465666768696a737475767778797a838485868788898a92939495969798999aa2a3a4a5a6a7a8a9aab2b3b4b5b6b7b8b9bac2c3c4c5c6c7c8c9cad2d3d4d5d6d7d8d9dae1e2e3e4e5e6e7e8e9eaf1f2f3f4f5f6f7f8f9faffc4001f0100030101010101010101010000000000000102030405060708090a0bffc400b51100020102040403040705040400010277000102031104052131061241510761711322328108144291a1b1c109233352f0156272d10a162434e125f11718191a262728292a35363738393a434445464748494a535455565758595a636465666768696a737475767778797a82838485868788898a92939495969798999aa2a3a4a5a6a7a8a9aab2b3b4b5b6b7b8b9bac2c3c4c5c6c7c8c9cad2d3d4d5d6d7d8d9dae2e3e4e5e6e7e8e9eaf2f3f4f5f6f7f8f9faffda000c03010002110311003f00bd3dc482790798ff0078ff0011f5a68b9931feb1ff0033493f133f7f98ff003a6923d39aed48f997e63fed1267fd63
Therefore, my attempt to display the image fails:
print STDOUT "Content-type: image/jpeg\n";
print STDOUT "Content-length: \n\n";
binmode STDOUT;
print STDOUT $mugshotHash->{mug_shot};
The browser reports that the image is invalid. Why is the data returned as text/hexadecimal instead of binary data, and what can I do to fetch the binary data?
There is a flag to return image data in binary format:
$dbh->{syb_binary_images} = 1;
After I set this flag, the images are returned in the correct format. For good measure, I also used the following to make sure that the images are not truncated:
$dbh->{LongTruncOK} = 0;
$dbh->do("set textsize 1000000");
Related
A multi user program, written in VB6, saves scanned pictures and shows/displays them on Image control. Users can scan pictures and same and/or other users can view them. Pictures are saved in a Text column (datatype = Text) of SQL Server. Saving and showing pictures works fine as long as Control Panel -> Region -> Administrative -> Language for non-Unicode programs (say non-uni-lang) is English (United States) or probably the same on both scanners' and viewers' PCs.
When viewers' non-uni-lang is different then they can't view pictures, pictures get corrupted.
I have noticed that the Recordset receives different data/text on machines with different non-uni-lang. Base64 equivalent of the saved Text is same on PCs with different non-uni-lang but decoding them back to binary differs.
GetACP API can be used to get the value but i couldn't find any API to change the value. There is a registry key but it requires reboot to have effect.
Questions:
Is there any way to change non-uni-lang?
Is there any way, independent of non-uni-lang, to decode base64 back to
binary?
Edit 1
Base64 Encoding
select cast('' as xml).value('xs:base64Binary(sql:column("mytext"))', 'nvarchar(max)') from (
select CAST(mytext AS varbinary(MAX)) mytext from (
select CAST(ImageData AS nvarchar(MAX)) mytext from ImageTable where ImageID = 123
) t2
) t1
Base64 Decoding (Code by CVMichael)
https://www.vbforums.com/showthread.php?197891-CryptBinaryToString-(new-BASE-64-function-so-RESOLVED)&p=1166849#post1166849
Edit 2
Eliminated SQL Server. I copied base64 string in a text file, read it as byte array in vb6, decoded and made multiple conversions and it showed the image on English PC but failed on Arabic PC. Please download the working code from https://drive.google.com/file/d/1Jv0ELXvxhd9E0V6RhiqrexVyiunG9UE1/view?usp=sharing
I have created simple table in Snowflake:
Column name Type
S VARCHAR
N NUMBER
Both columns are nullable.
Now I want to load partially bad result into the table from CSV and JSON files.
CSV
s, n
hello, 1
bye, 2
nothing, zero
Third line is "bad": its second element it not a number.
Command that I use to load this file:
"COPY INTO "DEMO_DB"."PUBLIC"."TEST5" FROM #my_s3_stage1 files=('2good-1bad.csv') file_format = (type = csv field_delimiter = ',' skip_header = 1) ON_ERROR = CONTINUE;
Thrown SnowflakeSQLException:
errorCode = 200038
SQLState = 0A000
message: Cannot convert value in the driver from type:12 to type:int, value=PARTIALLY_LOADED.
Two "good" lines are written into table; the "bad" one is ignored. This result is expected.
However when I am using the following JSON lines file:
{"s":"hello", "n":1}
{"s":"bye", "n":2}
{"s":"nothing", "n":"zero"}
with this command:
COPY INTO "DEMO_DB"."PUBLIC"."TEST5" FROM #my_s3_stage1 files=('2good-1bad.json') file_format = (type = json)
MATCH_BY_COLUMN_NAME=CASE_INSENSITIVE
ON_ERROR = CONTINUE
I get the following SnowflakeSQLException:
errorCode = 100071
SQLState = 22000
message: Failed to cast variant value "zero" to FIXED
and nothing is written to the DB.
The question is "What's wrong?"
Why ON_ERROR = CONTINUE does not work with my JSON file?
PS:
wrapping CONTINUE with single quotes does not help
using lower case does not help
actually I do not need CONTINUE, I need SKIP_FILE_<num>, however this does not work with JSON as well.
actually we are using avro in production environment, so it is more relevant. I am using JSON for tests because it is easier.
You are correct that on_error is not supported with non-CSV file formats. I've seen folks with files who can workaround specifying CSV file type with FIELD_DELIMITER = 'none'.
I have seen a couple folks request that this option work for semi-structured files and you are welcome to submit a feature request as well to create more demand for it:
https://community.snowflake.com/s/ideas
The documentation doesn't really spell it out that it's not supported (feel free to submit docs feedback using the button at the bottom), but you can see it hinted at:
https://docs.snowflake.com/en/sql-reference/sql/copy-into-table.html
"You can use the corresponding file format (e.g. JSON), but any error in the transformation will stop the COPY operation, even if you set the ON_ERROR option to continue or skip the file."
Table
id - int
file - varbinary(max)
Query
SELECT file
FROM Table
WHERE id = 1
Data
The id 1 data has a file. And it's binary length is 836,412. But when I run the query, I only see a 43,680 bytes of binary data.
I tried to download the data as a CSV through "Save Result as..." button in the pop up panel. But still, I couldn't get a full length of the binary data.
In the CSV, there's only 16 bit (unsigned, 65534 length) data available. Unfortunately, I cannot request the data from my application at this moment. I have to pull it out from SSMS and convert it manually in my test code to see the file.
How can I get the full binary data from SSMS? Is there an option for displaying full length of binary data?
You can try casting the "file" field to XML using a query such as the following...
SELECT CAST(CONVERT(VARCHAR(MAX), file, 1) AS XML)
FROM Table
WHERE id = 1
However, you'll need to confirm that your SSMS Query options are set appropriately:
Query | Query Options... | Results | Grid | XML Data: unlimited
This will provide your results in hex, like so:
0x47494638396150003200F70 ...
How to convert binary data to text?
I have column called File names in test table with image datatype so when i am selecting the data from test table its showing the data for file names column is binary data i.e ('0x433A5C55736535').
Regards
Anji
I can't beleive nobody answered this. If it's image data you won't get anything readable. but in case it is text data in a binary field, you can do this:
select cast(DataColumn as varchar(MAX)) as DataAsText from [TableWithData];
'0x433A5C55736535' is not binary data, binary data is composed only with 0 and 1.
I think you need to use the defined functions of your test table to get the data you want, if you are in MySQL it's "Select * from file names"
I've got a SQL Server database which has a table which contains a varbinary column.
This table has tens of thousands of rows.
This varbinary column contains documents: 85% in MS Word .doc format, 10% in .docx format and the rest in .pdf and .rtf.
There is a particular string which appears in all of these documents (an email address). I'd like to replace this string in all of these documents with a new string (an updated email address). (To be clear: The string to find and the string to replace it with is the same in all cases).
Ideally I'd like to be able to do this for all the file types but if it is only possible for .doc and .docx that would at least be the bulk of the problem solved.
I'd also like not to have to install MS Word if possible but appreciate this may be necessary.
Thanks!
You can Convert the Value to VarBinary and then replace the value. use below link to replace the varbinary value:
http://www.sqlteam.com/forums/topic.asp?TOPIC_ID=76304
You can be Managing FILESTREAM Data by Using Win32 API to get BLOB into a variable.
This way you get contents of your BLOB in a variable and as if it was opened in Notepad. Use Replace to update .DOC, .DOCX and .RTF files. I do not know how to update PDFs.
This link contains C# code that loads BLOB into a variable in you C# code. Then you can save it with path, file name and extention derrived from DB as well. Here is a a small quote of code:
//Read the data from the FILESTREAM
//BLOB.
sqlFileStream.Seek(0L, SeekOrigin.Begin);
numBytes = sqlFileStream.Read(buffer, 0, buffer.Length);
string readData = unicode.GetString(buffer);
if (numBytes != 0)
Console.WriteLine(readData);
//Here you have contents of your BLOB as if opened in Notepad. Use Replace to update .doc, .docx and .rtf files.
//Write the string, "EKG data." to the FILESTREAM BLOB.
//In your application this string would be replaced with
//the binary data that you want to write.
string someData = "EKG data.";
Encoding unicode = Encoding.GetEncoding(0);
sqlFileStream.Write(unicode.GetBytes(someData.ToCharArray()),
0,
someData.Length);
See also Using FILESTREAM Storage in Client Applications.