I have seen sql statements using nolock and with(nolock)
e.g -
select * from table1 nolock where column1 > 10
AND
select * from table1 with(nolock) where column1 > 10
Which of the above statements is correct and why?
The first statement doesn't lock anything, whereas the second one does. When I tested this out just now on SQL Server 2005, in
select * from table1 nolock where column1 > 10 --INCORRECT
"nolock" became the alias, within that query, of table1.
select * from table1 with(nolock) where column1 > 10
performs the desired nolock functionality. Skeptical? In a separate window, run
BEGIN TRANSACTION
UPDATE tabl1
set SomeColumn = 'x' + SomeColumn
to lock the table, and then try each locking statement in its own window. The first will hang, waiting for the lock to be released, and the second will run immediately (and show the "dirty data"). Don't forget to issue
ROLLBACK
when you're done.
The list of deprecated features is at Deprecated Database Engine Features in SQL Server 2008:
Specifying NOLOCK or READUNCOMMITTED
in the FROM clause of an UPDATE or
DELETE statement.
Specifying table
hints without using the WITH keyword.
HOLDLOCK table hint without
parenthesis
Use of a space as a separator between table hints.
The indirect application of table hints to an invocation of a multi-statement table-valued function (TVF) through a view.
They are all in the list of features that will be removed sometimes after the next release of SQL, meaning they'll likely be supported in the enxt release only under a lower database compatibility level.
That being said my 2c on the issue are as such:
Both from table nolock and from table with(nolock) are wrong. If you need dirty reads, you should use appropiate transaction isolation levels: set transaction isolation level read uncommitted. This way the islation level used is explictily stated and controlled from one 'knob', as opposed to being spread out trough the source and subject to all the quirks of table hints (indirect application through views and TVFs etc).
Dirty reads are an abonimation. What is needed, in 99.99% of the cases, is reduction of contention, not read uncommitted data. Contention is reduced by writing proper queries against a well designed schema and, if necessary, by deploying snapshot isolation. The best solution, that solves works almost always save a few extreme cases, is to enable read commited snapshot in the database and let the engine work its magic:
ALTER DATABASE MyDatabase SET ALLOW_SNAPSHOT_ISOLATION ON
ALTER DATABASE MyDatabase SET READ_COMMITTED_SNAPSHOT ON
Then remove ALL hints from the selects.
They are both technically correct, however not using the WITH keyword has been deprecated as of SQL 2005, so get used to using the WITH keyword - short answer, use the WITH keyword.
Use "WITH (NOLOCK)".
Both are syntactically correct.
NOLOCK will become the alias for table1.
WITH (NOLOCK) is often exploited as a magic way to speed up database reads, but I try to avoid using it whever possible.
The result set can contain rows that have not yet been committed, that are often later rolled back.
An error or Result set can be empty, be missing rows or display the same row multiple times.
This is because other transactions are moving data at the same time you're reading it.
READ COMMITTED adds an additional issue where data is corrupted within a single column where multiple users change the same cell simultaneously.
There are other side-effects too, which result in sacrificing the speed increase you were hoping to gain in the first place.
Now you know, never use it again.
Related
I am using NHibernate to query data from a reporting database and would like to optimize READ performance. "Dirty reads" are not a concern at all. NHibernate provides Lock Modes. If I use the LockMode.UpgradeNoWait mode I can see that in the generated SQL a 'with (updlock, rowlock)' appears. I've been reading up on this, but my SQL is unfortunately not sophisticated enough to really understand the ramifications. I believe what I really want is a 'with (nolock)' statement to appear, but NHibernate does not see to provide any way of doing that.
My question then is this: Let's say my table is being written to in other transactions. Will the 'with (updlock, rowlock)' hint in a query against this table cause it to return any faster? Can it hurt performance? The query is a very simple select against a single table. No joins. There is a non-clustered index on the table that covers the query.
The updlock will place update locks on every row being touched (selected) - so this means until the end of the transaction (explicit or implicit), the row(s) touched by the SELECT will have an update lock on them which allows for other transactions to read, but not update or delete the row.
The rowlock just indicates that you want row-level locks instead of page or table locks.
That lock makes sense if you need to select first, then update a row within the same explicit transaction.
It doesn't make it run any faster, and can cause other transactions to be blocked
Apparently I was mistaken about the use of Lock modes in NHibernate. They do not have anything to do with adding the nolock hint to a query. This is the way to do it.
When I prefer to use WITH (NOLOCK) in all the SQL queries inside a specific large stored procedure, is there a generic way to use it for all the specific stored procedure statements, or I should use WITH (NOLOCK) for every individual query?
You could set the Transaction Isolation Level
SET TRANSACTION ISOLATION LEVEL READ UNCOMMITED
However, don't forget that NOLOCK means your queries can potentially return dirty or duplicated data, or miss out data altogether. If it's an option for you, I would suggest investigating the READ_COMMITTED_SNAPSHOT database option to allow you to avoid locking issues while returning queries with consistent results.
You want to use the following syntax:
SET TRANSACTION ISOLATION LEVEL READ UNCOMMITTED
I found this by looking at the NOLOCK table hint located here : http://msdn.microsoft.com/en-us/library/ms187373.aspx. The WITH(NOLOCK) table hint is equivalent to setting the isolation level to be READ UNCOMMITTED. Here's the snippet from MSDN (http://msdn.microsoft.com/en-us/library/ms187373.aspx):
NOLOCK Is equivalent to READUNCOMMITTED. For more information, see READUNCOMMITTED later in this topic.
In my SQL tempOrder table has millions of records and with 10 trigger to update tempOrder table with another table's update.
So I want to apply apply with(NOLOCK) on table.
I know with
SELECT * FROM temporder with(NOLOCK)
This statement I can do. But is there any way to apply with(NOLOCK) directly to the table from SQL Server 2008.
The direct answer to your question is NO -- there is no option to to tell SQL to never lock tableX. With that said, your question opens up a whole series of things that should be brought up.
Isolation Level
First, the most direct way you can accomplish what you want is to use with (nolock) option or SET TRANSACTION ISLOATION LEVEL READ UNCOMMITTED (aka chaos). These options are good for the query or the duration of the connection respectively. If I chose this route I would combine it with a long running SQL Profiler trace to identify any queries taking locks on TableX.
Lock Escalation
Second, SQL Server does have a table wide LOCK_ESCALATION threshold (executed as ALTER TABLE SET LOCK_ESCALATION x where X is the number of locks or AUTO). This controls when SQL attempts to consolidate many fine grained locks into fewer coarse grained locks. Said another way, it is a numeric threshold for converting how many locks are taken out on a single database object (think index).
Overriding SQL's lock escaltion generally isn't a good idea. As the documentation states:
In most cases, the Database Engine delivers the best performance when
operating with its default settings for locking and lock escalation.
As counter intuitive as it may seem, from the scenario you described you might have some luck with fewer broad locks instead of NOLOCK. You'll need to test this theory out with a real workload to determine if its worthwhile.
Snapshot Isolation
You might also check out the SNAPSHOT isolation level. There isn't enough information in your question to know, but I suspect it would help.
Dangers of NOLOCK
With that said, as you might have picked up from #GSerg's comment, NOLOCK can be evil. No-Lock is colloquially referred to as Chaos--and for good reason. When developers first encounter NOLOCK it seems like allowing dirty reads is the only implication. There are more...
dirty data is read for inconsistent results (the common impression)
wrong data -- meaning neither consistent with the pre-write or post-write state of your data.
Hard exceptions (like error 601 due to data movement) that terminate your query
Blank data is returned
previously committed rows are missed
Malformed bytes are returned
But don't take my word for it :
Actual Email: "NoLOCK is the epitome of evil?"
SQL Sever NOLOCK hint & other poor ideas
Is the nolock hint a bad practice
this is not a table's configuration.
If you add (nolock) to the query (it is called a query hint) you are saying that when executing this (and only this) query, it wont create lock on the affected tables.
Of course, you can make this configuration permanent for the current connection by setting a transaction isolation level to read uncommitted for example: set transaction isolation level read uncommitted. But again, it is valid only until that connection is open.
Perhaps if you explain in more details what you are trying to achieve, we can better help you.
You cannot change the default isolation level (except for snapshot) for a table or a database, however you can change it for all read queries in one transaction:
set transaction isolation level read uncommitted
See msdn for more information.
We're using a SQL Server 2005 database (no row versioning) with a huge select statement, and we're seeing it block other statements from running (seen using sp_who2). I didn't realise SELECT statements could cause blocking - is there anything I can do to mitigate this?
SELECT can block updates. A properly designed data model and query will only cause minimal blocking and not be an issue. The 'usual' WITH NOLOCK hint is almost always the wrong answer. The proper answer is to tune your query so it does not scan huge tables.
If the query is untunable then you should first consider SNAPSHOT ISOLATION level, second you should consider using DATABASE SNAPSHOTS and last option should be DIRTY READS (and is better to change the isolation level rather than using the NOLOCK HINT). Note that dirty reads, as the name clearly states, will return inconsistent data (eg. your total sheet may be unbalanced).
From documentation:
Shared (S) locks allow concurrent transactions to read (SELECT) a resource under pessimistic concurrency control. For more information, see Types of Concurrency Control. No other transactions can modify the data while shared (S) locks exist on the resource. Shared (S) locks on a resource are released as soon as the read operation completes, unless the transaction isolation level is set to repeatable read or higher, or a locking hint is used to retain the shared (S) locks for the duration of the transaction.
A shared lock is compatible with another shared lock or an update lock, but not with an exlusive lock.
That means that your SELECT queries will block UPDATE and INSERT queries and vice versa.
A SELECT query will place a temporary shared lock when it reads a block of values from the table, and remove it when it done reading.
For the time the lock exists, you will not be able to do anything with the data in the locked area.
Two SELECT queries will never block each other (unless they are SELECT FOR UPDATE)
You can enable SNAPSHOT isolation level on your database and use it, but note that it will not prevent UPDATE queries from being locked by SELECT queries (which seems to be your case).
It, though, will prevent SELECT queries from being locked by UPDATE.
Also note that SQL Server, unlike Oracle, uses lock manager and keeps it locks in an in-memory linked list.
That means that under heavy load, the mere fact of placing and removing a lock may be slow, since the linked list should itself be locked by the transaction thread.
To perform dirty reads you can either:
using (new TransactionScope(TransactionScopeOption.Required,
new TransactionOptions {
IsolationLevel = System.Transactions.IsolationLevel.ReadUncommitted }))
{
//Your code here
}
or
SelectCommand = "SELECT * FROM Table1 WITH (NOLOCK) INNER JOIN Table2 WITH (NOLOCK) ..."
remember that you have to write WITH (NOLOCK) after every table you want to dirty read
You could set the transaction level to Read Uncommitted
You might also get deadlocks:
"deadlocks involving only one table"
http://sqlblog.com/blogs/alexander_kuznetsov/archive/2009/01/01/reproducing-deadlocks-involving-only-one-table.aspx
and or incorrect results:
"Selects under READ COMMITTED and REPEATABLE READ may return incorrect results."
http://www2.sqlblog.com/blogs/alexander_kuznetsov/archive/2009/04/10/selects-under-read-committed-and-repeatable-read-may-return-incorrect-results.aspx
You can use WITH(READPAST) table hint. It's different than the WITH(NOLOCK). It will get the data before the transaction was started and will not block anyone. Imagine that, you ran the statement before the transaction was started.
SELECT * FROM table1 WITH (READPAST)
I have a query that runs each night on a table with a bunch of records (200,000+). This application simply iterates over the results (using a DbDataReader in a C# app if that's relevant) and processes each one. The processing is done outside of the database altogether. During the time that the application is iterating over the results I am unable to insert any records into the table that I am querying for. The insert statements just hang and eventually timeout. The inserts are done in completely separate applications.
Does SQL Server lock the table down while a query is being done? This seems like an overly aggressive locking policy. I could understand how there could be a conflict between the query and newly inserted records, but I would be perfectly ok if records inserted after the query started were simply not included in the results.
Any ways to avoid this?
Update:
The WITH (NOLOCK) definitely did the trick. As some of you pointed out, this isn't the cleanest approach. I can't really query everything into memory given the amount of records and some of the columns in this table are binary (some records are actually about 1MB of total data).
The other suggestion, was to query for batches of records at a time. This isn't a bad idea either, but it does bring up a new issue: database independent queries. Right now the application can work with a variety of different databases (Oracle, MySQL, Access, etc). Each database has their own way of limiting the rows returned in a query. But maybe this is better saved for another question?
Back on topic, the "WITH (NOLOCK)" clause is certainly SQL Server specific, is there any way to keep this out of my query (and thus preventing it from working with other databases)? Maybe I could somehow specify a parameter on the DbCommand object? Or can I specify the locking policy at the database level? That is, change some properties in SQL Server itself that will prevent the table from locking like this by default?
If you're using SQL Server 2005+, then how about giving the new MVCC snapshot isolation a try. I've had good results with it:
ALTER DATABASE SET SINGLE_USER WITH ROLLBACK IMMEDIATE;
ALTER DATABASE SET READ_COMMITTED_SNAPSHOT ON;
ALTER DATABASE SET MULTI_USER;
It will stop readers blocking writers and vice-versa. It eliminates many deadlocks, at very little cost.
It depends what Isolation Level you are using. You might try doing your selects using the With (NoLock) hint, that will prevent the read locks, but will also mean the data being read might change before the selecting transaction completes.
The first thing you could do is try to add the "WITH (NOLOCK)" to any tables you have in your query. This will "Tame down" the locking that SQL Server does. An example of using "NOLOCK" on a join is as follows...
SELECT COUNT(Users.UserID)
FROM Users WITH (NOLOCK)
JOIN UsersInUserGroups WITH (NOLOCK) ON
Users.UserID = UsersInUserGroups.UserID
Another option is to use a dataset instead of a datareader. A datareader is a "fire hose" technique that stays connected to the tables while your program is processing and basically handling the table row by row through the hose. A dataset uses a "disconnected" methodology where all the data is loaded into memory and then the connection is closed. Your program can then loop the data in memory without having to worry about locking. However, if this is a really large amount of data, there maybe memory issues.
Hope this helps.
If you add the WITH (NOLOCK) hint after a table name in the FROM clause it should make sure it doesn't lock, and it doesn't care about reading data that is locked. You might get "out of date" results if you are writing at the same time, but if you don't care about that then you should be fine.
I reckon your best way of avoiding this is to do it in SQL rather than in the application.
You can add a
WAITFOR DELAY '000:00:01'
at the end of each loop iteration to provide time for other processes to run - just make sure that you haven't initiated a TRANSACTION such that all other processes are locked out anyway
The query is performing a table lock, thus the inserts are failing.
It sounds to me like you're keeping a lock on the table while processing the results.
You should instead load them into an array or collection of some sort, and close the database connection.
Then process the array.
In addition, while you're doing your select use either:
WITH(NOLOCK) or WITH(READPAST)
I'm not a big fan of using lock hints as you could end up with dirty reads or other weirdness. A couple of other ideas:
Can you break the number of rows down so you don't grab 200k at a time? Is there a way to tell whether you've processed a row - a flag, a timestamp - you could use to make the query? Your query could be 'SELECT TOP 5000 ...' getting a differnet 5k each time. Shorter queries mean shorter-lived locks.
If you can use smaller sets of rows I like the DataSet vs. IDataReader idea. You will be loading data into memory and not consuming any SQL locks, but the amount of memory can cause other problems.
-Brian
You should be able to set the isolation level at the .NET level so that you don't have to include the WITH (NOLOCK) hint.
If you want to go with the batching option, you should be able to specify the Rowcount setting from the .NET level which would tell the database to only return n number of records. By setting these settings at the .NET level they should become database independent and work across all the platforms.