In a data importing script:
client = TinyTds.Client.new(...)
insert_str = "INSERT INTO [...] (...) VALUE (...)"
client.execute(insert_str).do
So far so good.
However, how can I attach a .pdf file into the varbinary field (SQL Server 2000)?
I've recently had the same issue and using activerecord was not really adapted for what I wanted to do...
So, without using activerecord:
client = TinyTds.Client.new(...)
data = "0x" + File.open(file, 'rb').read.unpack('H*').first
insert_str = "INSERT INTO [...] (...) VALUE (... #{data})"
client.execute(insert_str).do
To send proper varbinary data, you need to read the file, convert it to hexadecimal string with unpack('H*').first and prepend '0x' to the result.
Here is PHP-MSSQL code to save binary data:
mssql_query("SET TEXTSIZE 2147483647",$link);
$sql = "UPDATE UploadTable SET UploadTable_Data = ".varbinary_encode($data)." WHERE Person_ID = '".intval($p_id)."'";
mssql_query($sql,$link) or
die('cannot upload_resume() in '.__FILE__.' on line '.__LINE__.'.<br/>'.mssql_get_last_message());
function varbinary_encode($data=null) {
$encoded = null;
if (!is_null($data)) {
$a = unpack("H*hex", $data);
$encoded = "0x";
$encoded .= $a['hex'];
}
return $encoded;
}
Here is PHP-MSSQL code to get binary data:
mssql_query("SET TEXTSIZE 2147483647",$link);
$sql = "SELECT * FROM UploadTable WHERE ID = 123";
$db_result = mssql_query($sql,$link);
// work with result like normal
I ended up using activerecord:
require 'rubygems'
require 'tiny_tds'
require 'activerecord-sqlserver-adapter'
..
my_table.create(:file_name => "abc.pdf", :file_data => File.open("abc.pdf", "rb").read)
For SQLServer 2000 support, go for 2.3.x version activerecord-sqlserver-adapter gem.
Related
I am trying to upload a binary.zip to SQL Server as varbinary type column content.
Target Table:
CREATE TABLE myTable ( zipFile varbinary(MAX) );
My NIFI Flow is very simple:
-> GetFile:
filter:binary.zip
-> UpdateAttribute:<br>
sql.args.1.type = -3 # as varbinary according to JDBC types enumeration
sql.args.1.value = ??? # I don't know what to put here ! (I've triying everything!)
sql.args.1.format= ??? # Is It required? I triyed 'hex'
-> PutSQL:<br>
SQLstatement= INSERT INTO myTable (zip_file) VALUES (?);
What should I put in sql.args.1.value?
I think it should be the flowfile payload, but it would work as part of the INSERT in the PutSQL? Not by the moment!
Thanks!
SOLUTION UPDATE:
Based on https://issues.apache.org/jira/browse/NIFI-8052
(Consider I'm sending some data as attribute parameter)
import java.nio.charset.StandardCharsets
import org.apache.nifi.controller.ControllerService
import groovy.sql.Sql
def flowFile = session.get()
def lookup = context.controllerServiceLookup
def dbServiceName = flowFile.getAttribute('DatabaseConnectionPoolName')
def tableName = flowFile.getAttribute('table_name')
def fieldName = flowFile.getAttribute('field_name')
def dbcpServiceId = lookup.getControllerServiceIdentifiers(ControllerService).find
{ cs -> lookup.getControllerServiceName(cs) == dbServiceName }
def conn = lookup.getControllerService(dbcpServiceId)?.getConnection()
def sql = new Sql(conn)
flowFile.read{ rawIn->
def parms = [rawIn ]
sql.executeInsert "INSERT INTO " + tableName + " (date, "+ fieldName + ") VALUES (CAST( GETDATE() AS Date ) , ?) ", parms
}
conn?.close()
if(!flowFile) return
session.transfer(flowFile, REL_SUCCESS)
session.commit()
maybe there is a nifi native way to insert blob however you could use ExecuteGroovyScript instead of UpdateAttribute and PutSQL
add SQL.mydb parameter on the level of processor and link it to required DBCP pool.
use following script body:
def ff=session.get()
if(!ff)return
def statement = "INSERT INTO myTable (zip_file) VALUES (:p_zip_file)"
def params = [
p_zip_file: SQL.mydb.BLOB(ff.read()) //cast flow file content as BLOB sql type
]
SQL.mydb.executeInsert(params, statement) //committed automatically on flow file success
//transfer to success without changes
REL_SUCCESS << ff
inside the script SQL.mydb is a reference to groovy.sql.Sql oblject
I am following the Snowflake Python Connector docs for variable binding to avoid SQL injection. I successfully set up a db connection with the following dict of credentials:
import snowflake.connector
CONN = snowflake.connector.connect(
user=snowflake_creds['user'],
password=snowflake_creds['password'],
account=snowflake_creds['account'],
warehouse=snowflake_creds["warehouse"],
database=snowflake_creds['database'],
schema=snowflake_creds['schema'],
)
cur = CONN.cursor(snowflake.connector.DictCursor)
The following block works fine and I get back query results, hard-coding the table name and using the standard format binding:
command = ("SELECT * FROM TEST_INPUT_TABLE WHERE batch_id = %s")
bind_params = (2)
results = cur.execute(command % bind_params).fetchall()
Similarly, this block works fine, using the pyformat binding:
command = ("SELECT * FROM TEST_INPUT_TABLE WHERE batch_id = %(id)s")
bind_params = {"id": 2}
results = cur.execute(command, bind_params).fetchall()
But the following two blocks both result in a ProgrammingError (pasted below the second block):
command = ("SELECT * FROM %s WHERE batch_id = %s")
bind_params = ("TEST_INPUT_TABLE", 2)
results = cur.execute(command, bind_params).fetchall()
command = ("SELECT * FROM %(tablename)s WHERE batch_id = %(id)s")
bind_params = {
"tablename": "TEST_INPUT_TABLE",
"id": 2
}
results = cur.execute(command, bind_params).fetchall()
ProgrammingError: 001011 (42601): SQL compilation error:
invalid URL prefix found in: 'TEST_INPUT_TABLE'
Is there some difference between how strings and ints get interpolated? I would
not think it would make a difference but that is all I can think of. Am I
missing something simple here? I don't want to have to choose between hard-coding the table name and putting the system at risk of SQL injection. Thanks for any guidance.
You should be wrapping your bind variables with an INDENTIFER() function when they reference an object, rather than a string literal. For example:
command = ("SELECT * FROM IDENTIFIER(%(tablename)s) WHERE batch_id = %(id)s")
https://docs.snowflake.com/en/sql-reference/identifier-literal.html
Give that a try.
Using Nodejs and mssql package for SQL Server, I understand normal SELECT query with parameters:
aVar = "somestring"; // used as example, the real input is from a web form
request = new sql.Request(conn)
.input('aParam', sql.NVarChar(200), aVar);
q = "SELECT * FROM myTable WHERE theColumn = #aParam";
request.query(q, function(err, record) { ...
Those work just fine. Changing this to use LIKE, and I cannot get it to work. I've tried single quote before the % character, after, around it, just around the string, the whole thing, etc. Doing this:
q = "SELECT * FROM myTable WHERE theColumn LIKE " + "'%#aParam%'";
does not result in any errors, but returns no results. All other attempts with those single quotes results in query syntax errors. I cant seem to locate any documentation for parameterizing queries with wildcards. Where am I going wrong with this?
The syntax to do this is mentioned here - https://github.com/tediousjs/node-mssql/issues/332.
But in short:
q = `SELECT * FROM myTable WHERE theColumn LIKE '%' + #aParam + '%';`;
Unless you writing low level driver/worker, I would recommend to use any ORM:
const Sequelize = require('sequelize');
const sequelize = new Sequelize('database', 'username', 'password');
const MyTable = sequelize.define('myTable', {
theColumn: Sequelize.STRING
});
const aVar = 'somestring';
const results = await MyTable.findAll({
where: {
theColumn: { $like: `%${aVar}%` }
}
});
})
console.dir(results);
await can be used only inside async functions. For Framework supporting this out of the box take a look at Koa.js https://github.com/koajs/koa
I have DB on MS SQL 2008 R2
I install follow software:
Ubuntu 16
Apache 2
PHP 7
Symfony 3.0 (with leaseweb/doctrine-pdo-dblib)
Windows Server 2008 R2 SP1
MS SQL Server 2008 R2 SP3 (with Cyrillic_General_CI_AS - I can't change the collation)
In my project, there is controller which has follow code
$em = $this->getDoctrine()->getManager();
$connection = $em->getConnection();
$sql="
select
isnull(ID,'') as 'id'
,isnull(SNAME,'') as 'sname'
,isnull(FNAME,'') as 'fname'
from TBL1
where SNAMElike '%'+:sname+'%' ";
$sql = iconv('UTF-8','Windows-1251', $sql);
$statement = $connection->prepare($sql);
$statement->bindValue(':sname', iconv('UTF-8', 'Windows-1251', $request->request->get('sname')), 'text');
$statement->execute();
In SQL Server Profiler I watch the string 0xc8e2e0edeee2 instead of Агеев. As you can see the codes is come from UTF8.
if I don't use the code page conversion,
$statement->bindValue(':sname', $request->request->get('sname'), 'text');
I see the string 0xd098d0b2d0b0d0bdd0bed0b2 in SQL Server Profiler.
If I don't use BindValue and put the variable into SQL like it's shown below
$sname = $request->request->get('sname');
$em = $this->getDoctrine()->getManager();
$connection = $em->getConnection();
$sql="
select
isnull(ID,'') as 'id'
,isnull(SNAME,'') as 'sname'
,isnull(FNAME,'') as 'fname'
from TBL1
where SNAMElike '%'+'".$sname."'+'%' ";
$sql = iconv('UTF-8','Windows-1251', $sql);
$statement = $connection->prepare($sql);
$statement->execute();
I watch correct string Иванов in SQL Server Profiler.
If the variable containts a string which has only latin charasters then there areb't any problem.
I send the string Ivanov and I watch the string Ivanov.
What is the problem?
The problem is in quoter function that called for each string parameter dblib_handle_quoter in ext\pdo_dblib\dblib_driver.c. It will convert any string containing character with codes outside of 32 > char > 127 range into binary string.
So if Агеев is not converted to 'Windows-1251' it becomes 0xd098d0b2d0b0d0bdd0bed0b2 - a UTF-8 representation converted to binary string (something like Рванов).
SQL Server then implicitly converts this binary value to string using:
single-byte conversion (with CP1251 code page in your case i guess), if expected data type is char or varchar. Here you need to convert parameters to Windows-1251.
unicode conversion, if expected data type is nchar or nvarchar. Here you need to convert parameters to UCS-2LE.
That's why there is no other choice (at least for now) than to use iconv for parameters.
PS If you set connection charset right, you shouldn't need to convert $sql as it should be done by dblib.
I use freetds-1.00.13 with PHP7 as pdo-dblib like this (data types is varchar):
function pdo_params(...$params){
foreach ($params as &$v){
$v = iconv('UTF-8','Windows-1251', $v);
}
unset($v);
return $params;
}
$dsn = 'dblib:dbname=DataBase;host=sql.server.host.name';
<...>
$dbh = new PDO($dsn, $user, $password,array(PDO::ATTR_ERRMODE => PDO::ERRMODE_EXCEPTION));
$stmt = $dbh->prepare('INSERT INTO table (a,b) VALUES (?,?)');
$stmt->execute(pdo_params("Test",$msg));
I am working on a process in Codeigniter to take a user-uploaded image (managed using the CI upload library) and insert it into a varbinary(max) field in a SQLServer database. My controller and model code are as follows.
if($this->upload->do_upload($upload_name)) {
//get temp image
$tmpName = $config['upload_path'] . $config['file_name'];
// Read it into $data variable
$fp = fopen($tmpName, 'rb');
$data = fread($fp, filesize($tmpName));
fclose($fp);
//insert into DB
$this->the_model->storeImage($data, $user_id);
//delete temp image
unlink($config['upload_path'] . $config['file_name']);
}
/***** Function from the_model ************/
function storePropertyImage($image_data, $user_id) {
$my_db = $this->load->database('admin');
$stmt = "INSERT INTO my_table (UserID, ImageData) VALUES (" . $my_db->escape($user_id) . ", " . $my_db->escape($image_data) . ")";
$insert = $my_db->query($stmt);
return $insert;
}
This all seems like it should be OK but when I run the code, I get the error:
Fatal error: Uncaught exception 'PDOException' with message
'SQLSTATE[HY093]: Invalid parameter number: mixed named and positional parameters'
in {my app path}\helpers\mssql_helper.php on line 213
I've done some googling on this error message and the results seem to indicate this is the result of there being a colon character in the $data value being sent to the model, making the DB think that I am trying to pass a named parameter when I am not. However I haven't been able to find any reports that match my specific use case or that have much info on how to correct the error.
I'd appreciate any insight on where I might be tripping up.
$image_data is a binary string. ->escape may not work on it, since it may escape random bytes in it, thus leaving you with a corrupted image. Also the binary string may contain quote characters (or other characters) that is making your query invalid.
Try to encode the binary string as hex before inserting into MySQL. You can use PHP's bin2hex for this.
$escaped_user_id = $my_db->escape($user_id);
$hex_image = bin2hex($image_data);
$stmt = "INSERT INTO my_table (UserID, ImageData) VALUES ({$escaped_user_id}, X'{$hex_image}')";
The X in X{$hex_image} is how MySQL handles literal hex strings: http://dev.mysql.com/doc/refman/5.1/en/hexadecimal-literals.html
If that doesn't work, you can also try UNHEX().
$escaped_user_id = $my_db->escape($user_id);
$hex_image = bin2hex($image_data);
$stmt = "INSERT INTO my_table (UserID, ImageData) VALUES ({$escaped_user_id}, UNHEX('{$hex_image}'))";
EDIT: I didn't notice you were using MSSQL and not MySQL. My bad. In MSSQL, you can insert literal hex strings with 0x.
$escaped_user_id = $my_db->escape($user_id);
$hex_image = bin2hex($image_data);
$stmt = "INSERT INTO my_table (UserID, ImageData) VALUES ({$escaped_user_id}, 0x{$hex_image})";