I am using Camel to send a named parameter to a query to a Postgres database. The named parameter in question is named members and is a list of stringified mac addresses, e.g. ['40:61:86:05:fd:69', '40:61:86:05:fd:69'], which I then want to use to filter my result set.
Simplified, where mac_column IN list_of_varchar_mac.
Actual snippet:
SELECT
*
FROM
table
WHERE
mac IN(:#in:members::macaddr);
However, the above works only when there is ONE item in the list of mac addresses. If there are two, I get the following error:
operator does not exist: macaddr = character varying
Seems simple enough, it's just an array, so I try the following instead:
WHERE
mac IN(:#in:members::macaddr[]);
New error, Caused by: org.postgresql.util.PSQLException: ERROR: operator does not exist: macaddr = macaddr[].
Reading the Postgres documentation on the macaddr type, it seems that using IN is not possible:
The macaddr type also supports the standard relational operators (>, <=, etc.) for lexicographical ordering, and the bitwise arithmetic operators (~, & and |) for NOT, AND and OR.
I have tried the following without Camel as an intermediary and it works:
mac = ANY('{40:61:86:05:fd:69, 40:61:86:05:fd:69}'::macaddr[]);
However, that would require me to create a string of the list of mac addresses and I'd rather not do that.
How do I solve this using only SQL? I can't touch the database, and I'd rather not touch the code.
Related
Consider this simple program:
from clickhouse_driver import Client
c = Client(host="localhost")
params = {"database": "test", "table": "t"}
query = c.substitute_params( query='SELECT * from %(database)s.%(table)s', params= params, context=c.connection.context)
print(query)
Clickhouse will put single quotes around the parameters, so the query result will be:
SELECT * from 'test'.'t'
I could also use f-string and the problem will be solved but that's vulnerable to SQLI. If I understand correctly, this is how parameterized queries are used in clickhouse to prevent SQLI.
How can we prevent the quotes from being put around the parameters?
As I understand it, substitute_params is not intended for database object identifiers like database and table, since those have to be quoted "differently" in ClickHouse (generally with backticks) than actual literal string values (with single quotes). https://clickhouse.com/docs/en/sql-reference/syntax/#identifiers
In general you can do your own bit of "SQL Injection defense" by validating the inputs for database and table, like ensuring they match a simple regex like "are all lower case letters or underscore" that applies to your ClickHouse schema. In that case using an f-string should be safe.
ClickHouse also support "server side substitution" where you can use an Identifier type for this use case, but I don't believe that feature is available in clickhouse-driver.
I am computing HASH value of each row in a table (for diffing purpose), after implementing the algorithm I am testing the results.
Results are consistent and algorithm somewhat seems to work, but testing it step by step I found a strange result.
The script:
SELECT HASHBYTES('SHA1', (SELECT INNERTBL.VALUT FOR XML RAW)) as KHASH
FROM ACLING AS INNERTBL
Should perform the SHA1 calculation on the table key, but when I perform the same calculation with external tool I get different results:
In fact when I perform SHA1('<row VALUT="A"/>') with external tool (tool here: https://emn178.github.io/online-tools/sha1.html) I get a different result:
So my question is, there is something wrong with my logic or simply SQL Server use some non standard SHA1 "parametrization"? (I have suspect about the use of a, may be standard but particular, padding scheme)
Example in fiddler: https://dbfiddle.uk/?rdbms=sqlserver_2019&fiddle=efa4e0ba11c112f54e36afb5d54d2cce
SELECT HASHBYTES('SHA1','<row VALUT="A"/>'), --- you are testing this
HASHBYTES('SHA1',N'<row VALUT="A"/>') -- ..but for xml returns Nvarchar
It is important to remember that you get the same result only if the string are binary the same. For example if the two strings uses different characterset they will have different HASH value. For more details please check thsi out https://security.stackexchange.com/questions/18290/is-sha-1-hash-always-the-same
I ran into strange situation working with jsonb type.
Expected behavior
Using short jsonb structure:
{"price": 99.99}
I wrote query like this:
SELECT * FROM table t WHERE t.data->price > 90.90
And it fail with error operator does not exist: jsonb > numeric the same as text (->>) operator does not exist: text > numeric
Then I wrote comparison as mentioned in many resources:
SELECT * FROM table t WHERE (t.data->>price)::NUMERIC > 90.90
And it's works as expected.
What's strange:
SELECT * FROM table t WHERE t.data->price > '90.90';
a little weird but query above works right.
EXPLAIN: Filter: ((data -> 'price'::text) > '90.90'::jsonb)
But if I change jsonb value to text as: {"price": "99.99"}
there is no result any more - empty.
Question: How actually PostgreSQL compare numeric data and what preferable way to do this kind of comparison.
But you aren't comparing numeric data, are you.
I can see that you think price contains a number, but it doesn't. It contains a JSON value. That might be a number, or it might be text, or an array, or an object, or an object containing arrays of objects containing...
You might say "but the key is called 'price' of course it is a number" but that's no use to PostgreSQL, particularly if I come along and sneakily insert an object containing arrays of objects containing...1
So - if you want a number to compare to you need convert it to a number (t.data->>price)::NUMERIC or convert your target value to JSON and let PostgreSQL do a JSON-based comparison (which might do what you want, it might not - I don't know what the exact rules are for JSON).
1 And that's exactly the sort of thing I would do, even though it is Christmas. I'm a bad person.
Is there any way/use of putting pipe symbol || in select clause.
I have come across following query in one of the article(probably to concatenate two values), but when I try to use the same in my query I am getting syntax error.
select FirstName ||''|| LastName As CustomerName from Customer
Please correct if I am using wrong syntax.
You can use CONCAT() function, which works in SQL Server 2012 and above, or just a plain + sign to do concatenation.
https://msdn.microsoft.com/en-us/library/hh231515(v=sql.110).aspx
Returns a string that is the result of concatenating two or more
string values.
you need to use '+' to perform Concat() instead of pipe if you are using SQL-Server. Pipe operator is not used in SQL-Server
It is used to concatenate you columns and output a single result i.e in one column.
For example, if i want to see first name and last name together as in one column then i could use pipes:
SELECT Fname||Lname FROM my_table;
If you are asking whether you can use pipes || for concatenation in Microsoft SQL, then the short answer is no.
If you’re asking about the concatenation operator itself, then read on.
|| is the standard ANSI concatenation operator. This is apparent in PostgreSQL, SQLite and Oracle, among others.
Microsoft, however uses +, because, why not. Except Microsoft Access uses &, because, why not.
MariaDB/MySQL have two modes. In traditional mode, || is interpreted as “or”, and there is no concatenation operator. In ANSI mode, || is interpreted as the concatenation operator.
Most DBMS (not SQLite) have the non-standard concat() function which will also concatenate. They also coalesce any NULLs to empty strings, so they’re a bit more forgiving if you don’t care about NULLs.
I have to get the records from a table field where Length of record/data/string is greater then 8 characters. I cannot use any string function as the query has to be used on (MySQL, MSSQL, Oracle).
I don't want to do the below EXAMPLE:
List<String> names = new ArrayList<String>();
String st = select 'name' from table;
rs = executeSQL(st);
if ( rs != null )
{
rs.next();
names.add(rs.getString(1));
}
for(String name : names)
{
if(name.length() > 8)
result.add(name);
}
Any idea other then the one coded above? A query that can get the required result instead of processing on the retrieved data and then getting the required result.
Thank you for any help / clue.
JDBC Drivers may implement a JDBC escapes for the functions listed in appendix D (Scalar Functions) of the JDBC specification. A driver should convert the scalar functions it supports to the appropriate function on the database side. A list of the supported functions can be queried using 'DatabaseMetaData.getStringFunctions()'
To use this in a query you would then either use CHAR_LENGTH(string) or LENGTH(string) like :
SELECT * FROM table WHERE {fn CHAR_LENGTH(field)} > 8
You can replace CHAR_LENGTH with LENGTH. The driver (if it supports this function) will then convert it to the appropriate function in the underlying database.
From section 13.4.1 Scalar Functions of the JDBC 4.1 specification:
Appendix D “Scalar Functions" provides a list of the scalar functions
a driver is expected to support. A driver is required to implement
these functions only if the data source supports them, however.
The escape syntax for scalar functions must only be used to invoke the
scalar functions defined in Appendix D “Scalar Functions". The escape
syntax is not intended to be used to invoke user-defined or vendor
specific scalar functions.
I think you may be better off leveraging the power of the database and implementing a factory for your SQL statements (or perhaps for objects encapsulating your SQL functionality).
That way you can configure your factory with the name/type of the database, and it'll give you the appropriate SQL statements for that database. It gives you a clean means of parameterising this info, whilst allowing you to leverage the functionality of your databases and not having to replicate the database functionality in a suboptimal fashion in your code.
e.g.
DabaseStatementFactory fac = DatabaseStatementFactory.for(NAME_OF_DATABASE);
String statement = fac.getLongNames();
// then use this statement. It'll be configured for each db type
It's probably wise to encapsulate further and use something like:
DabaseStatementFactory fac = DatabaseStatementFactory.for(NAME_OF_DATABASE);
List<String> names = fac.getLongNames();
such that you're not making assumptions re. common schema and means of queries etc.
Another solution that I found is:
Select name from table where name like '________';
SQL counts the underscore (_) characters and return a name of length equal to number of underscore characters.