The package docs https://magicstack.github.io/asyncpg/current/api/index.html#prepared-statements.
The only coroutines seem to be dealing with SELECT queries with prepared statements. The docs list cursor, explain, fetch, fetchrow, fetchval. It doesn't appear obvious whether asyncpg supports prepared INSERT statements. I tried stmt.execute(...) as a guess and indeed the PreparedStatement object has no attribute execute.
Related
one simple query written in store procedure and same written as inline query which will execute fast in SQL server.
someone from interview panel asked this question from me i said store procedure reason being procedure is compiled but he said i am wrong.
please explain.?
I suppose, that a simple query is some read-only code.
A VIEW is fully inlineable and precompiled. The biggest advantage is, that you can bind this into bigger statement with joins and filters and this will - in most cases - be able to use indexes and statistics.
A table valued function (TVF) in the "new" syntax (without BEGIN and END is very similar to a VIEW but the parameters and there handling are precompiled too. The advantages of the VIEW are the same here.
An UDF returning a table ("old" syntax") is in most cases something one should not do. The biggest disadvantage is the fact, that the optimizer cannot pre-estimate the result and will handle this as one-row-table which is - in most cases - really bad...
A StoredProcedure which does nothing more than a VIEW or a TVF could do as well, is a pain in the neck - at least in my eyes. I know, that there are other opinions... The biggest draw back is: Whenever you want to continue with the returned result set, you have to insert it into a table (or declared table variable). Further joins or filters against this new table will miss indexes and statistics. Might be that a simple Hey SP give me your result! is fast, but everything after this call is absolutely down.
So my facit: Use SPs when there is something to do and use VIEW or TVF when there is something to read
Does cfquery becomes a prepared statement as long as there's 1 cfqueryparam? Or are there other conditions?
What happen when the ORDER BY clause or FROM clause is dynamic? Would every unique combination becomes a prepared statement?
And what happen when we're doing cfloop with INSERT, with every value cfqueryparam'ed, and invoke the cfquery with different number of iterations?
Any potential problems with too many prepared statements?
How does DB handle prepared statement? Will they be converted into something similar to store procedure?
Under what circumstances should we Not use prepared statement?
Thank you!
I can answer some parts of your question:
a query will become a preparedStatement as long as there is one <queryparam. I have in the past added a
where 1 = <cfqueryparam value="1" to queries which didn't have any dynamic parameters, in order to get them run as preparedStatements
Most DBs handle preparedStarements similarly to Stored Procedures, just held temporarily, rather than long-term, however the details are likely to be DB-specific.
Assuming you are using the drivers supplied with ColdFusion, if you turn on the 'Log Activity' checkbox in the advanced panel of the DataSource setup, then you'll get very detailed information about how CF is interacting with he DB and when it is creating a new preparedStatement and when it is re-using them. I'd recommend trying this out for yourself, as so many factors are involved (DB setup, Driver, CF version etc). If you do use the DB logging, re-start CF before running your test code, so you can see it creating the prepared statements, otherwise you'll just see it re-using statements by ID, without seeing what those statements are.
In addition, if you are asking about execution plans then there is more involved than just the number PreparedStatement's generated. It is a huge topic and very database dependent. I do not have a DBA's grasp on it, but I can answer a few of the questions about MS SQL.
What happen when the ORDER BY clause or FROM clause is dynamic? Would
every unique combination becomes a prepared statement?
The base sql is different. So you will end up with separate execution plans for each unique ORDER BY clause.
And what happen when we're doing cfloop with INSERT, with every value
cfqueryparam'ed, and invoke the cfquery with different number of
iterations?
MS SQL should reuse the same plan for all iterations because only the parameters change.
The sys.dm_exec_cached_plans view is very useful for seeing what plans are cached and how often they are reused.
SELECT p.usecounts, p.cacheobjtype, p.objtype, t.text
FROM sys.dm_exec_cached_plans p
CROSS APPLY sys.dm_exec_sql_text( p.plan_handle) t
ORDER BY p.usecounts DESC
To clear the cache first, use DBCC FLUSHPROCINDB. Obviously do not use it on a production server.
DECLARE #ID int
SET #ID = DB_ID(N'YourTestDatabaseName')
DBCC FLUSHPROCINDB( #ID )
I don't use Stored procedures very often and was wondering if it made sense to wrap my select queries in a transaction.
My procedure has three simple select queries, two of which use the returned value of the first.
In a highly concurrent application it could (theoretically) happen that data you've read in the first select is modified before the other selects are executed.
If that is a situation that could occur in your application you should use a transaction to wrap your selects. Make sure you pick the correct isolation level though, not all transaction types guarantee consistent reads.
Update :
You may also find this article on concurrent update/insert solutions (aka upsert) interesting. It puts several common methods of upsert to the test to see what method actually guarantees data is not modified between a select and the next statement. The results are, well, shocking I'd say.
Transactions are usually used when you have CREATE, UPDATE or DELETE statements and you want to have the atomic behavior, that is, Either commit everything or commit nothing.
However, you could use a transaction for READ select statements to:
Make sure nobody else could update the table of interest while the bunch of your select query is executing.
Have a look at this msdn post.
Most databases run every single query in a transaction even if not specified it is implicitly wrapped. This includes select statements.
PostgreSQL actually treats every SQL statement as being executed within a transaction. If you do not issue a BEGIN command, then each individual statement has an implicit BEGIN and (if successful) COMMIT wrapped around it. A group of statements surrounded by BEGIN and COMMIT is sometimes called a transaction block.
https://www.postgresql.org/docs/current/tutorial-transactions.html
The problem
We need to defend against a 'WAITFOR DELAY' sql injection attack in our java application.
Background
[This is long. Skip to 'Solution?' section below if you're in a rush ]
Our application mostly uses prepared statements and callable statements (stored procedures) in accessing the database.
In a few places we dynamically build-and-execute queries for selection. In this paradigm we use a criteria object to build the query depending on the user-input criteria. For example, if the user specified values for first_name and last_name, the result querying always looks something like this:
SELECT first_name,last_name FROM MEMBER WHERE first_name ='joe' AND last_name='frazier'
(In this example the user would have specified "joe" and "frazier" as his/her input values. If the user had more or less critieria we would have longer or shorter queries. We have found that this approach is easier than using prepared statements and quicker/more performant than stored procedures).
The attack
A vulnerability audit reported an sql injection failure. The attacker injected the value 'frazier WAITFOR DELAY '00:00:20' for the 'last_name' parameter, resulting in this sql:
SELECT first_name,last_name FROM MEMBER WHERE first_name ='joe' AND last_name='frazier' WAITFOR DELAY '00:00:20'
The result: the query executes successfully, but takes 20 seconds to execute. An attacker could tie up all your database connections in the db pool and effectively shut down your site.
Some observations about this 'WAITFOR DELAY' attack
I had thought that because we used Statement executeQuery(String) we would be safe from sql injection. executeQuery(String) will not execute DML or DDL (deletes or drops). And executeQuery(String) chokes on semi-colons, thus the 'Bobby Tables' paradigm will fail (i.e. user enters 'frazier; DROP TABLE member' for a parameter. See. http://xkcd.com/327/)
The 'WAITFOR' attack differs in one important respect: WAITFOR modifies the existing 'SELECT' command, and is not a separate command.
The attack only works on the 'last parameter' in the resulting query. i.e. 'WAITFOR' must occur at the very end of the sql statement
Solution, Cheap Hack, or Both?
The most obvious solution entails simply tacking "AND 1=1" onto the where clause.
The resulting sql fails immediately and foils the attacker:
SELECT first_name,last_name FROM MEMBER WHERE first_name ='joe' AND last_name='frazier' WAITFOR DELAY '00:00:20' AND 1=1
The Questions
Is this a viable solution for the WAITFOR attack?
Does it defend against other similar vulnerabilities?
I think the best option would entail using prepared statements. More work, but less vulnerable.
The correct way to handle SQL injection is to use parameterized queries. Everything else is just pissing in the wind. It might work one time, even twice, but eventually you'll get hit by that warm feeling that says "you screwed up, badly!"
Whatever you do, except parameterized queries, is going to be sub-optimal, and it will be up to you to ensure your solution doesn't have other holes that you need to patch.
Parameterized queries, on the other hand, works out of the box, and prevents all of these attacks.
SQL injection is SQL injection - there's nothing special about a WAITFOR DELAY.
There is absolutely no excuse for not using prepared statements for such a simple query in this day and age.
(Edit: Okay, not "absolutely" - but there's almost never an excuse)
I think you suggested the solution yourself: Parameterized Queries.
How did you find that your dynamically built query is quicker than using a stored procedure? In general it is often the opposite.
To answer all your questions:
Is this a viable solution for the WAITFOR attack?
No. Just add -- to the attack string and it will ignore your fix.
Does it defend against other similar vulnerabilities?
No. See above.
I think the best option would entail using prepared statements. More work, but less vulnerable.
Yes. You don't fix SQL injection yourself. You use what's already existing and you use it right, that is, by parametrizing any dynamic part of your query.
Another lesser solution is to escape any string that is going to get inserted in your query, however, you will forget one one day and you only need one to get attacked.
Everyone else has nailed this (parameterize!) but just to touch on a couple of points here:
Is this a viable solution for the WAITFOR attack?
Does it defend against other similar vulnerabilities?
No, it doesn't. The WAITFOR trick is most likely just being used for 'sniffing' for the vulnerability; once they've found the vulnerable page, there's a lot more they can do without DDL or (the non-SELECT parts of) DML. For example, think about if they passed the following as last_name
' UNION ALL SELECT username, password FROM adminusers WHERE 'A'='A
Even after you add the AND 1 = 1, you're still hosed. In most databases, there's a lot of malicious things you can do with just SELECT access...
How about follow xkcd and sanitize the input. You could check for reserved words in general and for WAITFOR in particular.
I currently have a 'Filter' object which corresponds to a business object. This object has properties that relate to the different ways that I want to be able to filter/search a list of such business objects. Currently these Filter objects have a method that builds the contents of a where-clause that is then passed to a SQL Server 2000 stored procedure where it is concatendated with the rest of the select query. The final string is then executed using Exec.
Currently this works fine except I worry about the performance issue with the lack of execution plan caching. In some research I have seen the use of calling sp_executesql; is this a better solution or are there better conventions for what I am doing?
Update: I think part of the issue with using sp_executesql is that based on a collection in my filter I need to generate a list of OR statements. I am not sure that the 'parameterized' query would be my solution.
example
var whereClause = new StringBuilder();
if (Status.Count > 0)
{
whereClause.Append("(");
foreach (OrderStatus item in Status)
{
whereClause.AppendFormat("Orders.Status = {0} OR ", (int)item);
}
whereClause.Remove(whereClause.Length - 4, 3);
whereClause.Append(") AND ");
}
Yes, sp_executesql will "cache" the execution plan of the query it executes.
Alternatively, instead of passing part of the query to the stored procedure, building the full query there, and executing dynamic SQL, you could build entire query on .NET side and execute it using ADO.NET command object. All queries executed through ADO.NET are getting "cached" by default.
sp_executesql is better than exec because of plan reuse, and you can use parameters which help against sql injection. sp_executesql also won't cause procedure cache bloat if used correctly
take a look at these two articles
Avoid Conversions In Execution Plans By Using sp_executesql Instead of Exec
Changing exec to sp_executesql doesn't provide any benefit if you are not using parameters correctly
You should be using sp_executesql, simply because, as you say, the query plan is stored and future executions will be optimized. It also generally seems to handle dynamic sql better than execute.
Modern RDBMS'es (can't really say whether to consider SQL Server 2000 a "modern" one) are optimized for ad-hoc queries, so there's a negligible performance hit (if any). What's bothering me is that you're using sproc to construct dynamic SQL: this is a huge debugging/support PITA.
sp_executesql is the better option. Have you considered not using a stored procedure for this or at least taking out some of the dynamics? I think it would be much safer from any kind of injection. I write filters much like you are talking about but i try to take care of the input in my code as opposed to in a stored procedure. I really like dynamic sql but maybe it's safer to go the extra mile sometimes.