What is "better" generating Primary Keys on the database or generating them in application code, specifically when using GUID/UniqueIdentifier datatype for the keys.
I have read up on the difference between using Guid's and int data types, and it sounds like Guids are feasable for so called "generating offline".
E.g.
instead of having a NEWID() contstraint in the database, In one project (where we are using Entity Framework) we use in the application code Guid.NewGuid() to generate the PK when inserting data.
Is this a bad approach ?
My concerns are:
Database indexes: Database performance because Id's might not be sequential
The one in 64 billion chance that the key is already used. (considering that the application will not be enormous but may need room to grow)
Perhaps there are other disadvantages ?
well actually, GUID could be sequential from SQL Server 2005. There is a function in named NEWSEQUENTIALID() , link here
Creates a GUID that is greater than any GUID previously generated by
this function on a specified computer since Windows was started. After
restarting Windows, the GUID can start again from a lower range, but
is still globally unique. When a GUID column is used as a row
identifier, using NEWSEQUENTIALID can be faster than using the NEWID
function. This is because the NEWID function causes random activity
and uses fewer cached data pages. Using NEWSEQUENTIALID also helps to
completely fill the data and index pages.
Related
In a simple Data Base design, entity tables have IDs (mostly auto increment)
But there are some system e.g. vtiger CRM that use a master table to store all newly created IDS.
My question is:
What is the benefits of described approach.
What is the name of described approach, if any. I mean what do designers call this
method?
moodle is another example of this method too. An example in Moodle:
mdl_context has all IDs of other modules:
mdl_context - id - contextlevel - instanceid - path - depth
values - 115 - 50 - 17 - /1/84/90/115 - 4
instanceid is the ID of other entity and contextlevel shows the table, for example 50 is a code for course table.
Without having mdl_context, mdl_course had it's own ID, so why does `mdl_course exists?
You may simply think about this when your database doesn't support auto increment columns and you would have to implement auto incremental values yourself.
Or due to limitations of specific implementation of auto increment in a database, based on you business rules, you need to customize auto increment module.
for example
When gaps in the column values, are important to NOT Happens.
Consider the selling scenario in witch you need to have exact sequence of numbers for billing_ number column. Using an auto increment approach will cause some problems:
1- If any bill, would be rejected you would lose a number (Rollback scenario)
2- In case of DELETE operation on Billing table (if happens) you will lose a number(Delete scenario)
3- In some distributed(clustered) DB environments like Oracle RAC (having multiple RDBMS nodes) and using oracle sequences as auto increment strategy, we must use a CACHE interval to maintain integrity, so again some numbers will be lost.
In these cases you may use a metadata table like crm_entity holding last used value per table on it(or any other information if needed). locking the metadata table will be inevitable, so in heavy TPS, there will be performance issue.
SQL DBMSs typically provide a key generator feature that can be directly associated with a column in a table, variously known as Identity or auto-incrementing columns. These suffer certain disadvantages however. The syntax is often highly proprietary and awkward to work with and the key generator usually comes with inbuilt limitations, such as not permitting updates or inserts or only allowing one such column per table. Table-based generator functions normally only work on insert, which means the value can't be accessed and used until after the row has been inserted, and they are associated with one table only, making it impossible to generate key values that are shared and distributed between tables.
To overcome those and other limitations, table-independent key generators are often used instead. Some DBMSs (Oracle, SQL Server) support this directly with special Sequence-generator objects that are independent of tables but other DBMSs do not. So keeping a sequence-generating table separate from other tables is a useful general way to create sequences without relying on DBMS-specific features.
I've been learning ASP.net, and been using the membership system. When it auto generated the tables, I was quite suprised to see it uses a field type called 'uniqueIdentifier' as a primary key, when for many years I have been using an integer field set to be an identity that auto increments.
What is the difference (if any at all) between these two methods, and why does .NET appear to favour the unique identifier field?
Thanks for any info!
Tom
The uniqueidentifier type is SQL's Guid type (the corresponding BCL type is System.Guid). In concept, Guids represent a random 128-bit number that is supposed to be unique.
While Guid's have their detractors (comparing guids is, strictly speaking, slightly slower than comparing ints), their random nature makes them helpful in environments like replication, where using an incrementing key can be difficult.
I'd say that .NET doesn't favour the uniqueidentifier or guid as an id, but this particular implementation (the ASP.NET SQL Server membership provider) does. I suspect that those who developed the database were working with the assumption that the db usage wasn't to be for high traffic sites, or where heavy reporting was likely to be done.
Perhaps they were trying to avoid any problems with integrating in an existing application, or a future scenario whereby your application had a key for a user. This could be any kind of key for any entity (PK, UserNumber, etc). In the ASP.NET SQL Server implementation, the likelihood of having a collision is very low/approaching zero.
The one drawback that I've learned is that having a clustered index on a guid doesn't scale to large volume databases.
I'm largely in the integer-as-PK camp. They're small, use few bytes, and work very well when your database needs to scale.
What is the difference (if any at all)
between these two method
for one a uniqueidentifier is 16 bytes while an int is 4 bytes. IF you have a URL like
http://bla.com?UserID=1
you can easily guess what someone else's userid is so you can try 2 or 4 etc etc
when you have this as UserID C7478034-BB60-4F5A-BE51-72AAE5A96640 it is not as easily and also uniqueidentifiers are supposed to be unique accross all computers
if they use NEWID() instead of NEWSEQUENTIALID() then they will get fragmentation and page splits, take a look at Best Practice: Do not cluster on UniqueIdentifier when you use NewId
I am thinking to use a GUID in my .net app which uses SQL Server. Should I be writing a stored procedure which generates the GUID on each record entered or should I be directly generating it from the application.
Reasons for asking the question (If am wrong correct me in this):
I (as/pre)sume:
When generating the GUID from the database, you can assume that the DB remembers the previous generated GUID where as the application remembering it is difficult.
SQL Server has the creation of GUID's built in. There is no need to write a separate stored procedure for this.
You can use
NEWID()
NEWSEQUENTIALID()
The key difference between both procedures would be that the sequential GUID should be used if it is for a primary clustered key.
I'm not sure why you would want the database engine to remember the previous generated GUID.
No, your assumption is wrong: the database won't be remembering anything - so there's no benefit from that point of view.
If you're using the GUID as your primary key / clustering key in SQL Server, which is a bad idea to begin with (see here, here or here why that's the case), you should at least use the newsequentialid() function as default constraint on that column.
CREATE TABLE YourTable(ColumnA uniqueidentifier DEFAULT NEWSEQUENTIALID())
That way, the database would generate pseudo-sequential GUID's for your PK and thus would make the negative effects of using a GUID as PK/CK at least bearable....
If you're not using the GUID as your primary key, then I don't see any benefit in creating that GUID on the server, really.
My preference is to create GUID in the application not the db.
Simplifies the retrieval of rows after insertion.
Easier domain/business layer unit testing.
Faster. At least for entity framework. Link
RFC41221: «Do not assume that UUIDs are hard to guess; they should not be used as security capabilities (identifiers whose mere possession grants access), for example. A predictable random number source will exacerbate the situation».
In simple task incremental uint64 better.
NOT USE GUID! If need security.
http://social.msdn.microsoft.com/Forums/en/netfxbcl/thread/b37b3438-90f4-41fb-adb9-3ddba16fe07c
In recent years I was using MSSQL databases, and all unique records in tables has the ID column type of bigint (long). It is autoincrementing and generally - works fine.
Currently I am observing people prefer to use GUIDs for record's identity.
Does it make sense to swap bigint to guid for unique record id?
I think it doesn't make sense as generating bigint as well as sorting would be always faster than guid, but... some troubles come when using two (or more) separated instances of application and database and keep them in sync, so you have to manage id pools between sql servers (for example: sql1 uses id's from 100 to 200, sql2 uses id's from 201 to 300) - this is a thin ice.
With guid id, you don't care about id pools.
What is your advice for my mirrored application (and db): stay with traditional ID's or move to GUIDs?
Thanks in advance for your reply!
guids have the
Advantages:
Being able to create them offline from the database without worrying about collisions.
You're never going to run out of them
Disadvantages:
Sequential inserts can perform poorly (especially on clustered indexes).
Sequential Guids fix this
Take up more space per row
creating one cleanly isn't cheap
but if the clients are generating them this is actually no problem
The column should still have a unique constraint (either as the PK or as a separate constraint if it is part of some other relationship) since there is nothing stopping someone supplying the GUID by hand and accidentally/deliberately violating uniqueness.
If the space doesn't bother you and your performance if not significantly impacted they make a lot of problems go away. The decision is inevitably specific to the individual needs of the application.
I use GUIDs in any scenario that involves either replication or client-side ID generation. It's just so much easier to manage identity in either of those situations with a GUID.
For two-tier scenarios like a web application talking directly to the database, or for servers that don't need to replicate (or perhaps only need to replicate in one direction, pub/sub style) then I think an auto-incrementing ID column is just fine.
As for whether to stay with autoincs or move to GUIDs ... it's one thing to advocate GUIDs in a green-field application where you can make all these decisions up front. It's another to advise someone to migrate an existing database. It might be more pain than it's worth.
GUIDs have issues with performance and concurrency when page splits occur. INTs can run page fill at 100% - only added at one end, GUIDS add everywhere so you probably have to run a lower fill - which wastes space throughout the index.
GUIDS can be allocated in the application, so the App can know the ID of the record it will have created, which can be handy; but, technically, it is possible for duplicate GUIDs to be generated (long odds, but at least put a Unique Index on GUID columns)
I agree for merging databases its easier. But for me a straight INT is better, and then live with the hassle of sorting out how to merge DBs when/if it is actually needed.
If your data move around often, then GUID is the best one for the Key of the table.
If you really care about the performance, just stick to int or bigint
If you want to leverage both of above, use int or bigint as the key of the table and each row can have a rowguid column so that the data can also be moved around easily without losing integrity.
If the ids are going to be displayed in the querystring, use Guids, otherwise use long as a rule.
I'm currently working on someone else's database where the primary keys are generated via a lookup table which contains a list of table names and the last primary key used. A stored procedure increments this value and checks it is unique before returning it to the calling 'insert' SP.
What are the benefits for using a method like this (or just generating a GUID) instead of just using the Identity/Auto-number?
I'm not talking about primary keys that actually 'mean' something like ISBNs or product codes, just the unique identifiers.
Thanks.
An auto generated ID can cause problems in situations where you are using replication (as I'm sure the techniques you've found can!). In these cases, I generally opt for a GUID.
If you are not likely to use replication, then an auto-incrementing PK will most likely work just fine.
There's nothing inherently wrong with using AutoNumber, but there are a few reasons not to do it. Still, rolling your own solution isn't the best idea, as dacracot mentioned. Let me explain.
The first reason not to use AutoNumber on each table is you may end up merging records from multiple tables. Say you have a Sales Order table and some other kind of order table, and you decide to pull out some common data and use multiple table inheritance. It's nice to have primary keys that are globally unique. This is similar to what bobwienholt said about merging databases, but it can happen within a database.
Second, other databases don't use this paradigm, and other paradigms such as Oracle's sequences are way better. Fortunately, it's possible to mimic Oracle sequences using SQL Server. One way to do this is to create a single AutoNumber table for your entire database, called MainSequence, or whatever. No other table in the database will use autonumber, but anyone that needs a primary key generated automatically will use MainSequence to get it. This way, you get all of the built in performance, locking, thread-safety, etc. that dacracot was talking about without having to build it yourself.
Another option is using GUIDs for primary keys, but I don't recommend that because even if you are sure a human (even a developer) is never going to read them, someone probably will, and it's hard. And more importantly, things implicitly cast to ints very easily in T-SQL but can have a lot of trouble implicitly casting to a GUID. Basically, they are inconvenient.
In building a new system, I'd recommend using a dedicated table for primary key generation (just like Oracle sequences). For an existing database, I wouldn't go out of my way to change it.
from CodingHorror:
GUID Pros
Unique across every table, every database, every server
Allows easy merging of records from different databases
Allows easy distribution of databases across multiple servers
You can generate IDs anywhere, instead of having to roundtrip to the database
Most replication scenarios require GUID columns anyway
GUID Cons
It is a whopping 4 times larger than the traditional 4-byte index value; this can have serious performance and storage implications if you're not careful
Cumbersome to debug (where userid='{BAE7DF4-DDF-3RG-5TY3E3RF456AS10}')
The generated GUIDs should be partially sequential for best performance (eg, newsequentialid() on SQL 2005) and to enable use of clustered indexes
The article provides a lot of good external links on making the decision on GUID vs. Auto Increment. If I can, I go with GUID.
It's useful for clients to be able to pre-allocate a whole bunch of IDs to do a bulk insert without having to then update their local objects with the inserted IDs. Then there's the whole replication issue, as mentioned by Galwegian.
The procedure method of incrementing must be thread safe. If not, you may not get unique numbers. Also, it must be fast, otherwise it will be an application bottleneck. The built in functions have already taken these two factors into account.
My main issue with auto-incrementing keys is that they lack any meaning
That's a requirement of a primary key, in my mind -- to have no other reason to exist other than identifying a record. If it has no real-world meaning, then it has no real-world reason to change. You don't want primary keys to change, generally speaking, because you have to search-replace your whole database or worse. I have been surprised at the sorts of things I have assumed would be unique and unchanging that have not turned out to be years later.
Here's the thing with auto incrementing integers as keys:
You HAVE to have posted the record before you get access to it. That means that until you have posted the record, you cannot, for example, prepare related records that will be stored in another table, or any one of a lot of other possible reasons why it might be useful to have access to the new record's unique id, before posting it.
The above is my deciding factor, whether to go with one method, or the other.
Using a unique identifiers would allow you to merge data from two different databases.
Maybe you have an application that collects data in multiple database and then "syncs" with a master database at various times in the day. You wouldn't have to worry about primary key collisions in this scenario.
Or, possibly, you might want to know what a record's ID will be before you actually create it.
One benefit is that it can allow the database/SQL to be more cross-platform. The SQL can be exactly the same on SQL Server, Oracle, etc...
The only reason I can think of is that the code was written before sequences were invented and the code forgot to catch up ;)
I would prefer to use a GUID for most of the scenarios in which the post's current method makes any sense to me (replication being a possible one). If replication was the issue, such a stored procedure would have to be aware of the other server which would have to be linked to ensure key uniqueness, which would make it very brittle and probably a poor way of doing this.
One situation where I use integer primary keys that are NOT auto-incrementing identities is the case of rarely-changed lookup tables that enforce foreign key constraints, that will have a corresponding enum in the data-consuming application. In that scenario, I want to ensure the enum mapping will be correct between development and deployment, especially if there will be multiple prod servers.
Another potential reason is that you deliberately want random keys. This can be desirable if, say, you don't want nosey browsers leafing through every item you have in the database, but it's not critical enough to warrant actual authentication security measures.
My main issue with auto-incrementing keys is that they lack any meaning.
For tables where certain fields provide uniqueness (whether alone or in combination with another), I'd opt for using that instead.
A useful side benefit of using a GUID primary key instead of an auto-incrementing one is that you can assign the PK value for a new row on the client side (in fact you have to do this in a replication scenario), sparing you the hassle of retrieving the PK of the row you just added on the server.
One of the downsides of a GUID PK is that joins on a GUID field are slower (unless this has changed recently). Another upside of using GUIDs is that it's fun to try and explain to a non-technical manager why a GUID collision is rather unlikely.
Galwegian's answer is not necessarily true.
With MySQL you can set a key offset for each database instance. If you combine this with a large enough increment it will for fine. I'm sure other vendors would have some sort of similar settings.
Lets say we have 2 databases we want to replicate. We can set it up in the following way.
increment = 2
db1 - offset = 1
db2 - offset = 2
This means that
db1 will have keys 1, 3, 5, 7....
db2 will have keys 2, 4, 6, 8....
Therefore we will not have key clashes on inserts.
The only real reason to do this is to be database agnostic (if different db versions use different auto-numbering techniques).
The other issue mentioned here is the ability to create records in multiple places (like in the central office as well as on traveling users' laptops). In that case, though, you would probably need something like a "sitecode" that was unique to each install that was prefixed to each ID.