How to share a single sequence between multiple SQL Server databases? - sql-server

I need to share a single SQL Server sequence between tables in two different SQL Server databases. This example works just fine for two existing tables and a sequence defined on the same database:
USE DatabaseA ;
GO
CREATE SEQUENCE dbo.TransactionSequence START WITH 1 INCREMENT BY 1 ;
GO
ALTER TABLE Table1 ADD TransactionSequenceID int NOT NULL
CONSTRAINT DF_Table1_TransactionSequence DEFAULT (NEXT VALUE FOR dbo.TransactionSequence) ;
ALTER TABLE Table2 ADD TransactionSequenceID int NOT NULL
CONSTRAINT DF_Table2_TransactionSequence DEFAULT (NEXT VALUE FOR dbo.TransactionSequence) ;
GO
However, this command fails for a table in a different database:
USE DatabaseB ;
GO
ALTER TABLE Table3 ADD TransactionSequenceID int NOT NULL
CONSTRAINT DF_Table3_TransactionSequence DEFAULT (NEXT VALUE FOR DatabaseA.dbo.TransactionSequence) ;
GO
It fails because SQL Server does not allow a DEFAULT constraint to reference a sequence in another database.
A sequence cannot be referenced indirectly through a synonym in this context, and NEXT VALUE FOR <sequence> is not allowed inside of a function or several other possibly-useful constructs. Is it possible for tables in different databases to share a single sequence? If so, then how?

I've implemented something like this in the past and depending on your need/use case it might work for you. Let's say that you have need for a sequence that generates WidgetID values in multiple databases. You'd do it something like this:
use [Widgets_001];
create sequence SQ_WidgetID
as int
start with 1
minvalue 1
maxvalue 1000000
increment by 1;
use [Widgets_002];
create sequence SQ_WidgetID
as int
start with 1000001
minvalue 1000001
maxvalue 2000000
increment by 1;
Once you have the sequences created, you're allowed to use them in whatever default constraints you want.
The idea here being that they allocate from disjoint ranges. Because the sequences allocate independently, the databases can even be on different servers. This does come with the cost of needing to monitor for range exhaustion (e.g. when the sequence in Widget_001 is over 950,000 it's probably time to start thinking about allocating a new range) and care in re-seeding the sequence to allocate from a new range (which isn't too bad if you have a way to look at all of the sequences across databases and always allocate new ranges to have minvalue strictly larger than the max of all of the existing maxvalues).
I'd personally resist the temptation to attribute any meaning to the values generated by these respective sequences. That is, it's easy to say "Well... that's WidgetID 1234. Therefore it's in Widgets_001.". That comes with its own problems.

Related

Is there a way to ensure SQL Server identity column values are always consistent in an order even after many server restarts?

I have been having challenge with identity columns jumping after some server restart, an example is it would start counting from say 1,2,3,4 then later it jumps to 108,109,110 then later jumps to 10001, 10002, 10003.
I am now currently managing the IDs manually through triggers but this is an expensive exercise overtime.
An option to avoid the Identity Cache on a table level is to use sequence (with no cache) instead of identity.
create sequence dbo.YourTableId as int minvalue 1 start with 1 increment by 1 no cache;
GO
create table dbo.YourTable
(
Id int not null constraint DF_YourTable_Id default (next value for dbo.YourTableId),
[Name] varchar(50) not null,
...
CREATE SEQUENCE (Transact-SQL)
Warning: setting the sequence to no cache may impact the insertion performance.
To find a compromise you can set the cache size to a smaller number than the default value. For example, use cache 10 instead of no cache.
Here is more.

Identity(1,1) in Microsoft SQL Server - Is there a practical reason to ever use values other than 1,1 for seed/increment?

I'm teaching a Microsoft SQL Server course and a student asked if there was a practical reason why we would ever set the auto increment as something other than 1,1 (seed starting at 1, increment by 1). Is there a practical reason why we would ever set the seed to something higher or increment by a value other than 1? Shouldn't it not matter as long as that value is unique?
If there is no practical reason, why do we have the option to set those values for Identity in Microsoft SQL Server?
There are a lot of practical reasons for having a configurable start value :
You may want to insert a few predefined records with well-known IDs, eg Missing, Unknown and Not Applicable records in a dimension or lookup table should probably have predefined IDs. New rows should get IDs outside the range of the predefined numbers.
After loading or replicating data with existing ID values, new records should get IDs that don't conflict with the imported data. The easiest way to do this is by setting the starting ID somewhere above the maximum imported ID.
TRUNCATE TABLE resets the IDENTITY value. To avoid generating duplicate IDs you need to reseed the table with DBCC CHECKIDENT and set the current value to something other than 1
There are certainly dozens of other reasons.
If you are using a signed integer and start at 1, you're only using half the available range. Default should really be to start at minimum for the data type, for example -2 billion for 32 bit int.
In some cases, you may want to combine data from multiple tables. In that case each table should keep a separate range of ids. You could have each table start at a different number to prevent overlaps. For example start one at 1 and another one at 1 billion. Or you could use odd numbers for one (1,2) and even numbers for the other (2,2).

Postgres Dynamically Create Sequences

I'm writing an application where there are multiple users are users can upload reports within the application.
Currently, I have a 'reports' table that holds all of the submitted reports which has an 'id' field that is a serial primary key on the table.
The requirements I have specify that users need to be able to specify a prefix and a number for their reports to start counting from. For example, a user should be able to say that their reports start at ABC-100, then the next one is ABC-101, ABC-102, and so on and so forth.
The way I'm thinking of achieving this is that when a user creates an account, he can specify the prefix and start number, and I will create a postgres sequence with the name of the prefix that is specified and a minValue of the number the user wants reports to start at.
Then when a user submits a new report, I can mark the report_number as nextval(prefix_sequence). In theory this will work, but I am pretty new to postgres, and I want some advice and feedback on if this is a good use of sequences or if there is a better way.
This is an area where you probably don't need the key benefit of sequences - that they can be used concurrently by multiple transactions. You may also not want the corresponding downside, that gaps in sequences are normal. It's quite normal to get output like 1, 2, 4, 7, 8, 12, ... if you have concurrent transactions, rollbacks, etc.
In this case you are much better off with a counter. When a user creates an account, create a row in an account_sequences table like (account_id, counter). Do not store it in the main table of accounts, because you'll be locking and updating it a lot, and you want to minimise VACUUM workload.
e.g.
CREATE TABLE account_sequences
(
account_id integer PRIMARY KEY REFERENCES account(id),
counter integer NOT NULL DEFAULT 1,
);
Now write a simple LANGUAGE SQL function like
CREATE OR REPLACE FUNCTION account_get_next_id(integer)
RETURNS integer VOLATILE LANGUAGE sql AS
$$
UPDATE account_sequences
SET counter = counter + 1
WHERE account_id = $1
RETURNING counter
$$;
You can then use this in place of nextval.
This will work because each transaction that UPDATEs the relevant account_sequences row takes a lock on the row that it holds until it commits or rolls back. Other transactions that try to get IDs for the same account will wait for it to finish.
For more info search for "postgresql gapless sequence".
If you want, you can make your SQL function fetch the prefix too, concatenate it with the generated value using format, and return a text result. This will be easier if you put the prefix text NOT NULL column into your account_sequences table, so you can do something like:
CREATE OR REPLACE FUNCTION account_get_next_id(integer)
RETURNS text VOLATILE LANGUAGE sql AS
$$
UPDATE account_sequences
SET counter = counter + 1
WHERE account_id = $1
RETURNING format('%s%s', prefix, counter)
$$;
By the way, do not take the naïve approach of using a subquery with SELECT max(id) .... It's completely concurrency-unsafe, it'll produce wrong results or errors if multiple transactions run at once. Plus it's slow.

Fetch a Unique Number from a file containing list of numbers concurrently

I have a list of unique numbers in a file. Whenever user of my sites makes a request i need to fetch a value from that list , such that no other user gets the same value. There could be many concurrent requests, and hence the major issue is that of guaranteeing that no two users get the same value. I am not that concerned with performance in the sense that user doesn't expect any reply after making a request and so everything can happen in the background. From what i have read, this could be implemented using file locks or i could store data in the db and use db locks. I just want to get an ideas what is the best way of doing this thing. I am using postresql as my database and was wondering if this could be done using sequences where i store the current row number in the sequence so that my program knows which row to read from the db. But again i am not sure how to prevent multiple processes reading the same row before any one of them has a chance to update it.
Databases don't actually do many things, but what they do, they do very well. In particular, they handle concurrent access to data. You could use this to your advantage:
Create a table: create table special_number (id serial, num varchar(100) not null);
Ensure uniqueness: create unique index special_number_num on special_number(num);
Load your numbers into the table using COPY, letting id count up from 1 automatically
Create a sequence: create sequence num_seq;
Use the fact that postgres's nextval function is guaranteed concurrent-safe to safely pick a number from the list: select num from special_number where id = (select nextval('num_seq'));. This will return null if you run out of numbers.
This approach is 100% safe and easy - postgres's internal strengths as a database does all the heavy lifting.
Here's the SQL extracted from above:
create table special_number (id serial, num varchar(100) not null);
create unique index special_number_num on special_number(num);
copy special_number(num) from '/tmp/data.txt'; -- 1 special number per file line
create sequence num_seq;
select num from special_number where id = (select nextval('num_seq'));
This SQL has been tested on postgres and works as expected.

Oracle sequence but then in MS SQL Server

In Oracle there is a mechanism to generate sequence numbers e.g.;
CREATE SEQUENCE supplier_seq
MINVALUE 1
MAXVALUE 999999999999999999999999999
START WITH 1
INCREMENT BY 1
CACHE 20;
And then execute the statement
supplier_seq.nextval
to retrieve the next sequence number.
How would you create the same functionality in MS SQL Server ?
Edit: I'm not looking for ways to automaticly generate keys for table records. I need to generate a unique value that I can use as an (logical) ID for a process. So I need the exact functionality that Oracle provides.
There is no exact match.
The equivalent is IDENTITY that you can set as a datatype while creating a table. SQLSERVER will automatically create a running sequence number during insert.
The last inserted value can be obtained by calling SCOPE_IDENTITY() or by consulting the system variable ##IDENTITY (as pointed out by Frans)
If you need the exact equivalent, you would need to create a table and then write a procedure to retun the next value and other operations. See Marks response on pitfalls on this.
Edit:
SQL Server has implemented the Sequence similar to the Oracle. Please refer to this question for more details.
How would you implement sequences in Microsoft SQL Server?
Identity is the best and most scalable solution, BUT, if you need a sequence that is not an incrementing int, like 00A, 00B, 00C, or some special sequence, there is a second-best method. If implemented correctly, it scales OK, but if implemented badly, it scales badly. I hesitate to recommend it, but what you do is:
You have to store the "next value" in a table. The table can be a simple, one row, one column table with just that value. If you have several sequences, they can share the table, but you might get less contention by having separate tables for each.
You need to write a single update statement that will increment that value by 1 interval. You can put the update in a stored proc to make it simple to use and prevent repeating it in code in different places.
Using the sequence correctly, so that it will scale reasonably (no, not as well as Identitiy :-) requires two things: a. the update statement has a special syntax made for this exact problem that will both increment and return the value in a single statement; b. you have to fetch the value from the custom sequence BEFORE the start of a transaction and outside the transaction scope. That is one reason Identity scales -- it returns a new value irrespective of transaction scope, for any attempted insert, but does not roll back on failure. That means that it won't block, and also means you'll have gaps for failed transactions.
The special update syntax varies a little by version, but the gist is that you do an assignment to a variable and the update in the same statement. For 2008, Itzik Ben-Gan has this neat solution: http://www.sqlmag.com/Articles/ArticleID/101339/101339.html?Ad=1
The old-school 2000 and later method looks like this:
UPDATE SequenceTable SET #localVar = value = value + 5
-- change the tail end to your increment logic
This will both increment and return you the next value.
If you absolutely cannot have gaps (resist that requirement :-) then it is technically possible to put that update or proc in side the rest of your trnsaction, but you take a BIG concurrency hit as every insert waits for the prior one to commit.
I can't take credit on this; I learned it all from Itzik.
make the field an Identity field. The field will get its value automatically. You can obtain the last inserted value by calling SCOPE_IDENTITY() or by consulting the system variable ##IDENTITY
The SCOPE_IDENTITY() function is preferred.
As DHeer said there is absolutely no exact match. If you try to build your own procedure to do this you will invariably stop your application from scaling.
Oracle's sequences are highly scalable.
OK, I take it back slightly. If you're really willing to focus on concurrency and you're willing to take numbers out of order as is possible with a sequence, you have a chance. But since you seem rather unfamiliar with t-sql to begin with, I would start to look for some other options when (porting an Oracle app to MSSS - is that what you're doing)
For instance, just generate a GUID in the "nextval" function. That would scale.
Oh and DO NOT use a table for all the values, just to persist your max value in the cache. You'd have to lock it to ensure you give unique values and this is where you'll stop scaling. You'll have to figure out if there's a way to cache values in memory and programmatic access to some sort of lightweight locks- memory locks, not table locks.
I wish that SQL Server had this feature. It would make so many things easier.
Here is how I have gotten around this.
Create a table called tblIdentities. In this table put a row with your min and max values and how often the Sequence number should be reset. Also put the name of a new table (call it tblMySeqNum). Doing this makes adding more Sequence Number generators later fairly easy.
tblMySeqNum has two columns. ID (which is an int identity) and InsertDate (which is a date time column with a default value of GetDate()).
When you need a new seq num, call a sproc that inserts into this table and use SCOPE_IDENTITY() to get the identity created. Make sure you have not exceeded the max in tblIdentities. If you have then return an error. If not return your Sequence Number.
Now, to reset and clean up. Have a job that runs as regularly as needed that checks all the tables listed in tblIdentites (just one for now) to see if they need to be reset. If they have hit the reset value or time, then call DBCC IDENT RESEED on the name of the table listed in the row (tblMySeqNum in this example). This is also a good time to clear our the extra rows that you don't really need in that table.
DON'T do the cleanup or reseeding in your sproc that gets the identity. If you do then your sequence number generator will not scale well at all.
As I said, it would make so many things easier of this feature was in SQL Server, but I have found that this work around functions fairly well.
Vaccano
If you are able to update to SQL Server 2012 you can use SEQUENCE objects. Even SQL Server 2012 Express has support for sequences.
CREATE SEQUENCE supplier_seq
AS DECIMAL(38)
MINVALUE 1
MAXVALUE 999999999999999999999999999
START WITH 1
INCREMENT BY 1
CACHE 20;
SELECT NEXT VALUE FOR supplier_seq
SELECT NEXT VALUE FOR supplier_seq
SELECT NEXT VALUE FOR supplier_seq
SELECT NEXT VALUE FOR supplier_seq
SELECT NEXT VALUE FOR supplier_seq
Results in:
---------------------------------------
1
(1 row(s) affected)
---------------------------------------
2
(1 row(s) affected)
---------------------------------------
3
(1 row(s) affected)
---------------------------------------
4
(1 row(s) affected)
---------------------------------------
5
(1 row(s) affected)
Just take care to specify the right data type. If I hadn't specified it the MAXVALUE you've provided wouldn't be accepted, that's why I've used DECIMAL with the highest precision possible.
More on SEQUENCES here: http://msdn.microsoft.com/en-us/library/ff878091.aspx
This might have already been answered a long time ago... but from SQL 2005 onwards you can use the ROW_NUMBER function... an example would be:
select ROW_NUMBER() OVER (ORDER BY productID) as DynamicRowNumber, xxxxxx,xxxxx
The OVER statement uses the ORDER BY for the unique primary key in my case...
Hope this helps... no more temp tables, or strange joins!!
Not really an answer, but it looks like sequences are coming to SQLServer in 2012.
http://www.sql-server-performance.com/2011/sequence-sql-server-2011/
Not an exact answer but addition to some existing answers
SCOPE_IDENTITY (Transact-SQL)
SCOPE_IDENTITY, IDENT_CURRENT, and ##IDENTITY are similar functions
because they return values that are inserted into identity columns.
IDENT_CURRENT is not limited by scope and session; it is limited to a
specified table. IDENT_CURRENT returns the value generated for a
specific table in any session and any scope. For more information, see
IDENT_CURRENT (Transact-SQL).
It means two different sessions can have a same identity value or sequence number so to avoid this and get unique number for all sessions use IDENT_CURRENT
Exactly because of this
IDENT_CURRENT is not limited by scope and session; it is limited to a specified table.
we need to use SCOPE_IDENTITY() because scope identity will give us unique number generated in our session, and uniqueness is provided by identity itself.

Resources