What happens if while a transaction is executing, another client computer does an insert?
I have two client computers and one DB. These computers have the same program on them. Let's say we have order table and salesorder# column, this column is UNIQUE.
If both computers execute at the same time I know SQL server will select one of the transactions and make the other one wait. So this transactions does following
#ordernumber= SELECT TOP Salesorder# +1 .
INSERT INTO order (salesorder#,dateship, user) VALUES (#ordernumber,GETDATE(),1,)
I Believe that if both happened at the same time, it would just choose one of then , run completely, and then do the same for the other one. Is that correct?
What happens in a different scenario. If the transaction Begins, and another INSERT ( not a TRANSACTION Just INSERT statement) is requested after after the SELECT Statement but before the INSERT happens.
What will SQL Server do in that situation? Is this even possible?
One word: DON'T DO THIS!! This WILL fail - for sure.
Assuming you have a table with the highest number of 100 - now you have two (or several) concurrent user requests to do an insert.
Both requests will first read the highest number (100) and each will increment it by +1 - so both have a value of 101 internally. The SELECT statement will only do a quick shared lock - but both SELECT will work and will read the value of 100. Whether or not they're inside a transaction makes no difference here - just because there's a transaction doesn't stop a second SELECT from happening.
The INSERT will in fact create an exclusive lock - but only on the row being inserted (NOT the whole table!). There's nothing from stopping both concurrent requests from inserting their new row, and both will attempt to insert a row with salesorder# value of 101. If you do have a unique constraint on this, one of the requests will fail since the other request has already inserted a value of 101.
This is one of the many reason you should NOT handle giving out sequential ID's manually - let the database handle it, by using an INT IDENTITY column, or a SEQUENCE object.
#marc_s thanks for you help , I believe found a way to follow your recommendation (since you crearly know what you are talking about ) and the same time have salesnumbers in a way I wanted.
Our software architecture centers in a gateway . So we have clients , sending command to a gateway and then the gateway communicates with SQL SERVER. What I will do, is to prevent the gateway from sending an INSERT command to the same table at the same time by giving priority to clients differently.
For example: if we have clients compuerta let's call it 1,2,3 . Do the same action at the same time. (I know is technically impossible that this happens ) I can give the priority to 1 then once 1 is finished 2 will execute and then 3 . I know this is not a solution thru SQL because but this way I will prevent a fail insert, and I will be able to keep IDENTITY for salesorder#. Limiting the times that a fail insert would happen to almost 0.
Once again thanks for all the help and explainations a per why not to INSERT the way I wanted to do before. Any books recommended for reading on DB information?
Related
I have an application that, when a new user is added to a location, it is assigned a sequential number. So, for example, the first user at Location 01 would be assigned 01-0001, the second user 01-0002, etc.
While it is simple enough for me to find the max user number at any time for that location and +1, my issue is that I need this to be thread/collision safe.
While its not super common, I don't want one query to find the max() number while another query is in the process of adding that number at that same moment. (it has happened before in my original app, though only twice in 5 years.)
What would be the best way to go about this? I would prefer not to rely on a unique constraint as that would just throw an error and force the process to try it all again.
You can use
BEGIN TRAN
SELECT #UserId = MAX(UserId)
FROM YourTable WITH(UPDLOCK, HOLDLOCK, ROWLOCK)
WHERE LocationId = #LocationId
--TODO: Increment it or initialise it if no users yet.
INSERT INTO YourTable (UserId, Name)
VALUES (#UserId, #Name)
COMMIT
Only one session at a time can hold an update lock on a resource so if two concurrent sessions try to insert a row for the same location one of them will be blocked until the other one commits. The HOLDLOCK is to give serializable semantics and block the range containing the max.
This is a potential bottleneck but this is by design because of the requirement that the numbers be sequential. Better performing alternatives such as IDENTITY do not guarantee this. In reality though it sounds as though your application is fairly low usage so this may not be a big problem.
Another possible issue is that the ID will be recycled if the user that is the current max for a location gets deleted but this will be the same for your existing application by the sounds of it so I assume this is either not a problem or just doesn't happen.
You can use a sequence object described here.
Create a new sequence is very simple, for example you can use this code
create sequence dbo.UserId as int
start with 1
increment by 1;
With sequence object you don't need to be aware about any collision. Sequence will return always next value in every time you get it with NEXT VALUE FOR statement, like in this code
select next value for dbo.UserId;
The next value will be return correctly even if your rollback transaction or even if you get next value in separate, paralel transactions.
I am running SQL Server 2012 and this one query is killing my database performance.
My text message provider does not support scheduled text messages so I have a text message engine that picks up messages from the database and sends them at the scheduled time. I put this query together that gets the messages from the database and also changes their status so that they do not get picked up again.
The query works fine, it is just causing wait times on the CPU especially since it runs every other second. I installed a database performance software and it said this query accounts for 92% of instance execution time. The software also said that every single execution is doing 347,267 Logical Reads.
Any ideas on how to make this perform better?
Should I maybe select into a temporary table and update those results before returning them?
Here is the current query:
UPDATE TOP (30) dbo.Outgoing
SET Status = 2
OUTPUT INSERTED.OutgoingID, INSERTED.[Message], n.PhoneNumber, c.OptInStatus
FROM dbo.Outgoing o
JOIN Numbers n on n.NumberID = o.NumberID
LEFT JOIN Contacts c on c.ContactID = o.ContactID
WHERE Scheduled <= GETUTCDATE() AND SmsId IS NULL AND Status = 1
Here is the execution plan
There are three tables involved in this query: Outgoing, Numbers, & Contacts
Outgoing is the main table that this query deals with. There are only two indexes right now, a clustered primary key index on OutgoingID [PK, bigint, not null] and a non-clustered, non-unique index on SmsId [varchar(255), null] which is an identifier sent back from our text message provider once the messages are successfully received in their system. The Status column is just an integer column that relates to a few different statuses (Scheduled, Queued, Sent, Failed, Etc)
Numbers is just a simple table where we store unique cell phone numbers, some different formats of that number, and some basic information identifying the customer such as First name, carrier, etc. It just has a clustered primary key index on NumberID [bigint]. The PhoneNumber column is just a varchar(15).
The Contacts table just connects the individual person (phone number) to one of our merchants and keeps up with the number's opt in status, and other information related to the customer/merchant relationship. The only columns related to this query are OptInStatus [bit, not null] and ContactID [PK, bigint, not null]
--UPDATE--
Added a non-clustered index on the the Outgoing table with columns (Scheduled, SmsId, Status) and that seems to have brought down the execution time from 2+ second to milliseconds. I will check in with my performance monitoring software tomorrow to see how it has improved. Thank you everyone for the help so far!
As several commenters have already pointed out you need a new index on the dbo.Outgoing table. The server is struggling with finding the rows to update/output. This is most probably where the problem is:
WHERE Scheduled <= GETUTCDATE() AND SmsId IS NULL AND Status = 1
To improve performance you should create an index on dbo.Outgoing where you include these columns. This will make is easier for Sql Server to find the correct rows. It will on the other hand create some more work for the actual update though since there will be a new index that needs attention when updating.
While you're working on this, it would likely be a good idea to shorten the SmsId column unless you actually need it to be 255 chars long. Preferably before you create the index.
As an alternate solution you might think about having separate tables for the messages that are outgoing and those that are outgone. Then you can:
insert all records from Outgoing to Outgone
delete all records from Outgoing, with output clause like you are currently doing.
Make sure though that the insert and the delete operations are done in one transaction or you will soon have weird inconsistencies in the database.
it is just causing wait times on the CPU especially since it runs every other second.
Get rid of the TOP 30 and run it much less often than once every other second... maybe every two or three minutes.
You can enable max degree of parallelism of your sql server for faster processing
Requirement:
To count the number of times a procedure has executed
From what I understand so far, sys.dm_exec_procedure_stats can be used for approximate count but that's only since the last service restart. I found this link on this website relevant but I need count to be precise and that should not flush off after the service restart.
Can I have some pointers on this, please?
Hack: The procedure I need to keep track of has a select statement so returns some rows that are stored in a permanent table called Results. The simplest solution I can think of is to create a column in Results table to keep track of the procedure execution, select the maximum value from this column before the insert and add one to it to increment the count. This solution seems quite stupid to me as well but the best I could think of.
What I thought is you could create a sequence object, assuming you're on SQL Server 2012 or newer.
CREATE SEQUENCE ProcXXXCounter
AS int
START WITH 1
INCREMENT BY 1 ;
And then in the procedure fetch a value from it:
declare #CallCount int
select #CallCount = NEXT VALUE FOR ProcXXXCounter
There is of course a small overhead with this, but doesn't cause similar blocking issue that could happen with using a table because sequences are handled outside transaction.
Sequence parameters: https://msdn.microsoft.com/en-us/library/ff878091.aspx
The only way I can think of keeping track of number of executions even when the service has restarted , is to have a table in your database and insert a row to that table inside your procedure everytime it is executed.
Maybe add a datetime column as well to collect more info about the execution. And a column for user who executed etc..
This can be done, easily and without Enterprise Edition, by using extended events. The sqlserver.module_end event will fire, set your predicates correctly and use a histogram target.
http://sqlperformance.com/2014/06/extended-events/predicate-order-matters
https://technet.microsoft.com/en-us/library/ff878023(v=sql.110).aspx
To consume the value, query the histogram target (under the reviewing target output examples).
I have a table called ticket, and it has a field called number and a foreign key called client that needs to work much like an auto-field (incrementing by 1 for each new record), except that the client chain needs to be able specify the starting number. This isn't a unique field because multiple clients will undoubtedly use the same numbers (e.g. start at 1001). In my application I'm fetching the row with the highest number, and using that number + 1 to make the next record's number. This all takes place inside of a single transaction (the fetching and the saving of the new record). Is it true that I won't have to worry about a ticket ever getting an incorrect (duplicate) number under a high load situation, or will the transaction protect from that possibility? (note: I'm using PostgreSQL 9.x)
without locking the whole table on every insert/update, no. The way transactions work on PostgreSQL means that new rows that appear as a result of concurrent transactions never conflict with each other; and thats exactly what would be happening.
You need to make sure that updates actually cause the same rows to conflict. You would basically need to implement something similar to the mechanic used by PostgreSQL's native sequences.
What I would do is add another column to the table referenced by your client column to represent the last_val of the sequence's you'll be using. So each transaction would look sort of like this:
BEGIN;
SET TRANSACTION SERIALIZABLE;
UPDATE clients
SET footable_last_val = footable_last_val + 1
WHERE clients.id = :client_id;
INSERT INTO footable(somecol, client_id, number)
VALUES (:somevalue,
:client_id,
(SELECT footable_last_val
FROM clients
WHERE clients.id = :client_id));
COMMIT;
So that the first update into the clients table fails due to a version conflict before reaching the insert.
You do have to worry about duplicate numbers.
The typical problematic scenario is: transaction T1 reads N, and creates a new row with N+1. But before T1 commits, another transaction T2 sees N as the max for this client and creates another new row with N+1 => conflict.
There are many ways to avoid this; here is a simple piece of plpgsql code that implements one method, assuming a unique index on (client,number). The solution is to let the inserts run concurrently but in the event of a unique index violation, retry with refreshed values until it's accepted (it's not a busy loop, though, since concurrent inserts are blocked until other transactions commit)
do
$$
begin
loop
BEGIN
-- client number is assumed to be 1234 for the sake of simplicity
insert into the_table(client,number)
select 1234, 1+coalesce(max(number),0) from the_table where client=1234;
exit;
EXCEPTION
when unique_violation then -- nothing (keep looping)
END;
end loop;
end$$;
This example is a bit similar to the UPSERT implementation from the PG documentation.
It's easily transferable into a plpgsql function taking the client id as input.
In SQL Server, if a transaction involving the inserting of a new row gets rolled back, a number is skipped in the identity field.
For example, if the highest ID in the Foos table is 99, then we try to insert a new Foo record but roll back, then ID 100 gets 'used up' and the next Foo row will be numbered 101.
Is there any way this behaviour can be changed so that identity fields are guaranteed to be sequential?
What you are after will never work with identity columns.
They are designed to "give out" and forget, by-design so that they don't cause waits or deadlocks etc. The property allows IDENTITY columns to be used as a sequence within a highly transactional system with no delay or bottlenecks.
To make sure that there are no gaps means that there is NO WAY to implement a 100-insert per second system because there would be a very long queue to figure out if the 1st insert was going to be rolled back.
For the same reason, you normally do not want this behaviour, nor such a number sequence for a high volume table. However, for very infrequent, single-process tables (such as invoice number by a single process monthly), it is acceptable to put a transaction around a MAX(number)+1 or similar query, e.g.
declare #next int
update sequence_for_tbl set #next=next=next+1
.. use #next
SQL Identity (autonumber) is Incremented Even with a Transaction Rollback