I have table like
create table test(payload varbinary(max))
I am trying to store text lines in compressed format in the database using the following code
String sql = "insert into test(payload) values (compress(:payload))
MapSqlParametersource msps = new MapSqlParameterSource();
msps.addValue("payload", "some text", Types.VARBINARY)
NamedParameterJdbcTemplate npjt = //;
npjt.update(sql, msps);
This gives the following error -
String is not in a valid hex format
If I provide the datatype in MapSqlParameterSource as VARCHAR, it doesn't give any error, but then using MSSQL's decompress function returns garbage value
select decompress(payload) from test
Related
i want match string in json string that like:
"ids":[44,53,1,3,12,45]
i want run query in sqlite send only one digit as id and match one of the above id in sql statement
i write this regex "ids":[\[] for matching start of key
but i don't have any idea to match middle id and escape starting id
example:
i have calc_method table like this:
CREATE TABLE "calc_method" (
"calc_method_id" INTEGER NOT NULL PRIMARY KEY AUTOINCREMENT,
"calc_method_name" TEXT NOT NULL,
"calc_method_value" TEXT NOT NULL
);
in calc_method_value column i store calcMethod class which convert to json using Gson
class calcMethod{
var memberCafeIds:ArrayList<Long>,
var memberBarIds:ArrayList<Long>
}
after i convert calcMethod to json i have output like below and this value store in calc_method_value column:
{"memberCafeIds":[1,2,14,5,44],"memberBarIds":[23,1,5,78]}
now i want select row that match to my regex pattern like if calc_method_value column have memberBarIds with id 1
SELECT * FROM calc_method WHERE calc_method_value REGEXP '"memberCafeIds":\[[:paramId]'
:paramId is method parameter
Regards, a programmer struggle with regex
In Sqlite, use JSON1 functions to work with JSON, not regular expressions. In particular, json_each() to turn the JSON array into a table you can query:
sqlite> CREATE TABLE ex(json);
sqlite> INSERT INTO ex VALUES ('{"ids":[44,53,1,3,12,45]}');
sqlite> SELECT * FROM ex WHERE 1 IN (SELECT value FROM json_each(ex.json, '$.ids'));
json
-------------------------
{"ids":[44,53,1,3,12,45]}
sqlite> SELECT * FROM ex WHERE 50 IN (SELECT value FROM json_each(ex.json, '$.ids'));
sqlite>
|id | language
|123456| GE
I can't update data in SQL Server.
I'm beginner.I am writing telegram bot, and i want add new languages.Therefore, I use the database.
I have a table with 2 columns, one declared as nvarchar(id), and the other as nchar(language) and i'm trying change language.
import pypyodbc
...
#for example
id = 123456
lang = 'EN'
#
data = ("""
UPDATE bottest
SET language = ?
WHERE id = ?;
""")
k = (id, lang)
cursor.execute(data, k).rowcount
cursor.commit()
Getting this error:
File "C:\Users\MyPC\PycharmProjects\bottest_flask_i\venv\lib\site-packages\pypyodbc.py", line 1007, in check_success
ctrl_err(SQL_HANDLE_STMT, ODBC_obj.stmt_h, ret, ODBC_obj.ansi)
File "C:\Users\MyPC\PycharmProjects\bottest_flask_i\venv\lib\site-packages\pypyodbc.py", line 977, in ctrl_err
raise DataError(state,err_text)
pypyodbc.DataError: ('22003', '[22003] [Microsoft][ODBC SQL Server Driver][SQL Server]Arithmetic overflow error converting expression to data type nvarchar.')
Your parameter values are specified in the wrong order. The parameter for language (in the SET clause) appears before the parameter for id (in the WHERE clause) so you need to specify the language parameter value first. That is, instead of
k = (id, lang)
you need to use
k = (lang, id) # same order as the parameter placeholders appear in the command text
I am new to Dapper, want to know why below is suggested, when my code runs without it?
Ansi Strings and varchar
Dapper supports varchar params, if you are executing a where clause on
a varchar column using a param be sure to pass it in this way:
Query<Thing>("select * from Thing where Name = #Name", new {Name = new
DbString { Value = "abcde", IsFixedLength = true, Length = 10, IsAnsi
= true });
On SQL Server it is crucial to use the unicode when querying unicode
and ansi when querying non unicode.
Below is my code, which runs against SQL server 2012 without using DbString etc.
create table Author (
Id int identity(1,1),
FirstName varchar(50),
LastName varchar(50)
);
go
insert into Author (FirstName, LastName) values ('Tom', 'John');
public Author FindByVarchar(string firstName)
{
using (IDbConnection db = DBHelper.NewSqlConnection())
{
return db.Query<Author>("Select * From Author WHERE FirstName = #firstName", new { firstName }).SingleOrDefault();
}
}
Questions:
1 Why is DbString type used in this case?
2 Why the length is set to 10 (e.g.Length = 10) when "abcde" is 5?
3 Do I still need to use DbString when my current code works?
4 Is it correct to set IsAnsi = false for unicode column?
5 For varchar column, is it correct to set IsFixedLength = false, and ignore setting Length?
The purpose of the example is that it is describing a scenario where the data type is char(10). If we just used "abcde", Dapper might think that nvarchar(5) was appropriate. This would be very inefficient in some cases - especially in a where clause, since the RDBMS can decide that it can't use the index, and instead needs to table scan doing a string conversion for every row in the table from char(10) to the nvarchar version. It is for this reason that DbString exists - to help you control exactly how Dapper configures the parameter for text data.
I think this answers your 1 and 2.
3: are you using ANSI (non-unicode text) or fixed-width text? Note that the ANSI default can also be set globally if you always avoid unicode
4: yes
5: yes
4+5 combined: if you're using nvarchar: just use string
I am trying to learn hive, this may be stupid a question but
I created a table in hive as follows
create table if not exists tweets_table(
tweetdata STRING,
followerscount INT,
friendscount INT,
statuscount INT,
retweetcount INT,
favouritescount INT,
lang STRING,
placefullname STRING,
placename STRING,
countryname STRING,
countrycode STRING,
hashtags STRING)
ROW FORMAT DELIMITED
FIELDS TERMINATED BY '^**^'
LINES TERMINATED BY '\n'
STORED AS TEXTFILE;
LOAD DATA LOCAL INPATH '/home/cloudera/Desktop/TestDB.txt' INTO TABLE tweets_table5 ;
I have used '^**^' to delimit the text because the tweets has a lot of "\n \r ," (please suggest a standard practice if any)
So I have a text file which I am trying to load
09-09-2016 10:51:33|^**^|#ArvindKejriwal #abpnewstv तुम्हारे दावों का क्या हुआ केजरीवाल।|^**^|74|^**^|30|^**^|0|^**^|98|^**^|0|^**^|49|^**^|en|^**^|Ambikapur, India|^**^|Ambikapur|^**^|India|^**^|IN|^**^|[]
09-09-2016 10:51:37|^**^|#LiveLawIndia It is shocking a judge arrested. I am sure Higher Judiciary will come their rescue , Judges per se cannot be wrong|^**^|0|^**^|14|^**^|0|^**^|32|^**^|0|^**^|2|^**^|en|^**^|Rajasthan, India|^**^|Rajasthan|^**^|India|^**^|IN|^**^|[]
After successfully loading it and querying it,
I get the following output
09-09-2016 10:51:33| NULL NULL NULL NULL NULL |30| ** |0| ** |98| **
09-09-2016 10:51:37| NULL NULL NULL NULL NULL |14| ** |0| ** |32| **
I fail to understand where am I going wrong is it in my tex tfile or the hive table. Please help
Several issues with what you're trying to do:
Using FIELDS TERMINATED BY You can't have a delimiter that is more than 1 char.
Even if that worked, that doesn't solve the problem that your tweets have line delimiters in them - each \n in a tweet starts a new row.
The way you describe your table - it's impossible to parse correctly - you can't have \n as line delimiters and in the tweet data. If you are the one generating this input file, I would suggest replacing all \n and \r in the tweets with spaces.
Create table with regex serde instead of default hive serde.
Modify below regex based on number of columnns.:
^(.+)\|\^\*\*\^\|(.+)\|\^\*\*\^\|(.+)\|\^\*\*\^\|(.+)\|\^\*\*\^\|(.+)\|\^\*\*\^\|(.+)\|\^\*\*\^\|(.+)\|\^\*\*\^\|(.+)\|\^\*\*\^\|(.+)\|\^\*\*\^\|(.+)\|\^\*\*\^\|(.+)\|\^\*\*\^\|(.+)\|\^\*\*\^\|(.+)\|\^\*\*\^\|(.+)$
Table:
create external table if not exists tweets_table(
tweetdata STRING,
followerscount INT,
friendscount INT,
statuscount INT,
retweetcount INT,
favouritescount INT,
lang STRING,
placefullname STRING,
placename STRING,
countryname STRING,
countrycode STRING,
hashtags STRING)
ROW FORMAT SERDE 'org.apache.hadoop.hive.contrib.serde2.RegexSerDe'
WITH SERDEPROPERTIES (
"input.regex" = " ^(.+)\|\^\*\*\^\|(.+)\|\^\*\*\^\|(.+)\|\^\*\*\^\|(.+)\|\^\*\*\^\|(.+)\|\^\*\*\^\|(.+)\|\^\*\*\^\|(.+)\|\^\*\*\^\|(.+)\|\^\*\*\^\|(.+)\|\^\*\*\^\|(.+)\|\^\*\*\^\|(.+)\|\^\*\*\^\|(.+)\|\^\*\*\^\|(.+)\|\^\*\*\^\|(.+)$",
"output.format.string" = "%1$s %2$s %3$s %4$s %5$s %6$s %7$s %8$s %9$s %10$s %11$s %12$s %13$s"
)
STORED AS TEXTFILE;
Load data:
LOAD DATA LOCAL INPATH '/home/cloudera/Desktop/TestDB.txt' INTO TABLE tweets_table ;
If you get RegexSerDe classNotFoundException then add regex serde jar:
ADD JAR hive-contrib=x.x.x.jar
I am working on a process in Codeigniter to take a user-uploaded image (managed using the CI upload library) and insert it into a varbinary(max) field in a SQLServer database. My controller and model code are as follows.
if($this->upload->do_upload($upload_name)) {
//get temp image
$tmpName = $config['upload_path'] . $config['file_name'];
// Read it into $data variable
$fp = fopen($tmpName, 'rb');
$data = fread($fp, filesize($tmpName));
fclose($fp);
//insert into DB
$this->the_model->storeImage($data, $user_id);
//delete temp image
unlink($config['upload_path'] . $config['file_name']);
}
/***** Function from the_model ************/
function storePropertyImage($image_data, $user_id) {
$my_db = $this->load->database('admin');
$stmt = "INSERT INTO my_table (UserID, ImageData) VALUES (" . $my_db->escape($user_id) . ", " . $my_db->escape($image_data) . ")";
$insert = $my_db->query($stmt);
return $insert;
}
This all seems like it should be OK but when I run the code, I get the error:
Fatal error: Uncaught exception 'PDOException' with message
'SQLSTATE[HY093]: Invalid parameter number: mixed named and positional parameters'
in {my app path}\helpers\mssql_helper.php on line 213
I've done some googling on this error message and the results seem to indicate this is the result of there being a colon character in the $data value being sent to the model, making the DB think that I am trying to pass a named parameter when I am not. However I haven't been able to find any reports that match my specific use case or that have much info on how to correct the error.
I'd appreciate any insight on where I might be tripping up.
$image_data is a binary string. ->escape may not work on it, since it may escape random bytes in it, thus leaving you with a corrupted image. Also the binary string may contain quote characters (or other characters) that is making your query invalid.
Try to encode the binary string as hex before inserting into MySQL. You can use PHP's bin2hex for this.
$escaped_user_id = $my_db->escape($user_id);
$hex_image = bin2hex($image_data);
$stmt = "INSERT INTO my_table (UserID, ImageData) VALUES ({$escaped_user_id}, X'{$hex_image}')";
The X in X{$hex_image} is how MySQL handles literal hex strings: http://dev.mysql.com/doc/refman/5.1/en/hexadecimal-literals.html
If that doesn't work, you can also try UNHEX().
$escaped_user_id = $my_db->escape($user_id);
$hex_image = bin2hex($image_data);
$stmt = "INSERT INTO my_table (UserID, ImageData) VALUES ({$escaped_user_id}, UNHEX('{$hex_image}'))";
EDIT: I didn't notice you were using MSSQL and not MySQL. My bad. In MSSQL, you can insert literal hex strings with 0x.
$escaped_user_id = $my_db->escape($user_id);
$hex_image = bin2hex($image_data);
$stmt = "INSERT INTO my_table (UserID, ImageData) VALUES ({$escaped_user_id}, 0x{$hex_image})";