Is it possible to create a non clustered index which is not unique? What data structure is used to implement non clustered indexes.
Assuming you are talking about SQL Server then simply don't specify UNIQUE when creating the index.
CREATE /*UNIQUE*/ NONCLUSTERED INDEX IX ON T(C)
As UNIQUE is commented out above this does not enforce uniqueness on the C column. But in fact it will still be made unique behind the scenes by adding the (unique) row locator in to the non clustered index key.
Regarding data structure both clustered and non clustered indexes are B+ trees.
As stated by Martin Smith, the indexes don't need to be logically unique but in practice, SQL Server adds a 4 byte 'uniquifier' column to guarantee physical uniqueness.
In terms of structural difference, the non-clustered indexes include pointers to the clustered index or the heap pointer (if you haven't created a clustered index).
You should note that while they are both B-Trees, there are other differences - non-clustered indexes have their leaf nodes 1 level higher, which can mean reading from non-clustered indexes can be faster than reading from a clustered index providing the data required is available in the leaf nodes (the columns required are in the key of the index).
Here's the clustered index structure from Books Online:
http://technet.microsoft.com/en-us/library/ms177443(v=sql.105).aspx
Here's the non-clustered index structure:
http://technet.microsoft.com/en-gb/library/ms177484(v=sql.105).aspx
So, reading from a 'covered' non-clustered index can be faster as each level incurs 1 page read so because the non-clustered index has fewer levels to get to the data then you will incur fewer logical reads which in turn will mean fewer physical disk reads and less work for the CPU.
You should also consider that covering indexes with only the specific columns required for a specific query will mean fewer total pages need to be read to grab all the data resulting in faster performance but also be aware that the more indexes you have, the more cost your writes will incur.
Related
Here is one of the definitions I found for clustered Index:
When is a file is organized so that the ordering of data records is
the same as or close to the ordering of data entries in some index, we
say that the index is clustered.
I'm having trouble understanding the above sentence regarding the clustered Indexes. The things I know about clustered index are:
Clustered indexes reorders the way the records are physically stored in the table, so only one clustered index is possible
Clustered index is created on non key attribute
Well for clustered index we have many view to look into
A clustered index is a type of index where the table records are physically re-ordered to match the index.
Clustered indexes are efficient on columns that are searched for a range of values. After the row with first value is found using a clustered index, rows with subsequent index values are guaranteed to be physically adjacent, thus providing faster access for a user query or an application
You also have to understand the Non-Clustered Index
In other words, a clustered index stores the actual data, where a non-clustered index is a pointer to the data. In most DBMSs, you can only have one clustered index per table, though there are systems that support multiple clusters (DB2 being an example).
Like a regular index that is stored unsorted in a database table, a clustered index can be a composite index, such as a concatenation of first name and last name in a table of personal information.
There are several example and explanations. And this is What do Clustered and Non clustered index actually mean? one of them.
I want to create an index on a column such that the select statement require min time to retrieve the data. The column is unique and if data is physically sorted, query will be faster. Which index would you use?.
Clustered
Non-Clustered
Unique
Composite
I think the clustered index is the right answer here.
It sorts the data physically to get better performance.
Wikipedia says:
Clustered indices can greatly increase overall speed of retrieval, but usually only where the data is accessed sequentially in the same or reverse order of the clustered index, or when a range of items is selected.
Since the physical records are in this sort order on disk, the next row item in the sequence is immediately before or after the last one, and so fewer data block reads are required.
Some SO questions that might help to understand the topic:
When should I use a composite index?
Primary key or Unique index?
What do Clustered and Non clustered index actually mean?
In the documentation for SQL server 2008 R2 is stated:
Wide keys are a composite of several columns or several large-size columns. The key values from the clustered index are used by all nonclustered indexes as lookup keys. Any nonclustered indexes defined on the same table will be significantly larger because the nonclustered index entries contain the clustering key and also the key columns defined for that nonclustered index.
Does this mean, that when there is a search using non-clustered index, than the clustered indes is search also? I originally thought that the non-clustered index contains ditrectly the address of the page (block) with the row it references. From the text above it seems that it contains just the key from the non-clustered index instead of the address.
Could somebody explain please?
Yes, that's exactly what happens:
SQL Server searches for your search value in the non-clustered index
if a match is found, in that index entry, there's also the clustering key (the column or columns that make up the clustered index)
with that clustered key, a key lookup (often also called bookmark lookup) is now performed - the clustered index is searched for that value given
when the item is found, the entire data record at the leaf level of the clustered index navigation structure is present and can be returned
SQL Server does this, because using a physical address would be really really bad:
if a page split occurs, all the entries that are moved to a new page would be updated
for all those entries, all nonclustered indices would also have to be updated
and this is really really bad for performance.
This is one of the reasons why it is beneficial to use limited column lists in SELECT (instead of always SELECT *) and possibly even include a few extra columns in the nonclustered index (to make it a covering index). That way, you can avoid unnecessary and expensive bookmark lookups.
And because the clustering key is included in each and every nonclustered index, it's highly important that this be a small and narrow key - optimally an INT IDENTITY or something like that - and not a huge structure; the clustering key is the most replicated data structure in SQL Server and should be a small as possible.
The fact that these bookmark lookups are relatively expensive is also one of the reasons why the query optimizer might opt for an index scan as soon as you select a larger number of rows - at at time, just scanning the clustered index might be cheaper than doing a lot of key lookups.
Recently I found a couple of tables in a Database with no Clustered Indexes defined.
But there are non-clustered indexes defined, so they are on HEAP.
On analysis I found that select statements were using filter on the columns defined in non-clustered indexes.
Not having a clustered index on these tables affect performance?
It's hard to state this more succinctly than SQL Server MVP Brad McGehee:
As a rule of thumb, every table should have a clustered index. Generally, but not always, the clustered index should be on a column that monotonically increases–such as an identity column, or some other column where the value is increasing–and is unique. In many cases, the primary key is the ideal column for a clustered index.
BOL echoes this sentiment:
With few exceptions, every table should have a clustered index.
The reasons for doing this are many and are primarily based upon the fact that a clustered index physically orders your data in storage.
If your clustered index is on a single column monotonically increases, inserts occur in order on your storage device and page splits will not happen.
Clustered indexes are efficient for finding a specific row when the indexed value is unique, such as the common pattern of selecting a row based upon the primary key.
A clustered index often allows for efficient queries on columns that are often searched for ranges of values (between, >, etc.).
Clustering can speed up queries where data is commonly sorted by a specific column or columns.
A clustered index can be rebuilt or reorganized on demand to control table fragmentation.
These benefits can even be applied to views.
You may not want to have a clustered index on:
Columns that have frequent data changes, as SQL Server must then physically re-order the data in storage.
Columns that are already covered by other indexes.
Wide keys, as the clustered index is also used in non-clustered index lookups.
GUID columns, which are larger than identities and also effectively random values (not likely to be sorted upon), though newsequentialid() could be used to help mitigate physical reordering during inserts.
A rare reason to use a heap (table without a clustered index) is if the data is always accessed through nonclustered indexes and the RID (SQL Server internal row identifier) is known to be smaller than a clustered index key.
Because of these and other considerations, such as your particular application workloads, you should carefully select your clustered indexes to get maximum benefit for your queries.
Also note that when you create a primary key on a table in SQL Server, it will by default create a unique clustered index (if it doesn't already have one). This means that if you find a table that doesn't have a clustered index, but does have a primary key (as all tables should), a developer had previously made the decision to create it that way. You may want to have a compelling reason to change that (of which there are many, as we've seen). Adding, changing or dropping the clustered index requires rewriting the entire table and any non-clustered indexes, so this can take some time on a large table.
I would not say "Every table should have a clustered index", I would say "Look carefully at every table and how they are accessed and try to define a clustered index on it if it makes sense". It's a plus, like a Joker, you have only one Joker per table, but you don't have to use it. Other database systems don't have this, at least in this form, BTW.
Putting clustered indices everywhere without understanding what you're doing can also kill your performance (in general, the INSERT performance because a clustered index means physical re-ordering on the disk, or at least it's a good way to understand it), for example with GUID primary keys as we see more and more.
So, read Tim Lehner's exceptions and reason.
Performance is a big hairy problem. Make sure you are optimizing for the right thing.
Free advice is always worth it's price, and there is no substitute for actual experimentation.
The purpose of an index is to find matching rows and help retrieve the data when found.
A non-clustered index on your search criteria will help to find rows, but there needs to be additional operation to get at the row's data.
If there is no clustered index, SQL uses an internal rowId to point to the location of the data.
However, If there is a clustered index on the table, that rowId is replaced by the data values in the clustered index.
So the step of reading the rows data would not be needed, and would be covered by the values in the index.
Even if a clustered index isn't very good at being selective, if those keys are frequently most or all of the results requested - it may be helpful to have them as the leaf of the non-clustered index.
Yes you should have clustered index on a table.So that all nonclustered indexes perform in better way.
Consider using a clustered index when Columns that contain a large number of distinct values so to avoid the need for SQL Server to add a "uniqueifier" to duplicate key values
Disadvantage : It takes longer to update records if only when the fields in the clustering index are changed.
Avoid clustering index constructions where there is a risk that many concurrent inserts will happen on almost the same clustering index value
Searches against a nonclustered index will appear slower is the clustered index isn't build correctly, or it does not include all the columns needed to return the data back to the calling application. In the event that the non-clustered index doesn't contain all the needed data then the SQL Server will go to the clustered index to get the missing data (via a lookup) which will make the query run slower as the lookup is done row by row.
Yes, every table should have a clustered index. The clustered index sets the physical order of data in a table. You can compare this to the ordering of music at a store, by bands name and or Yellow pages ordered by a last name. Since this deals with the physical order you can have only one it can be comprised by many columns but you can only have one.
It’s best to place the clustered index on columns often searched for a range of values. Example would be a date range. Clustered indexes are also efficient for finding a specific row when the indexed value is unique. Microsoft SQL will place clustered indexes on a PRIMARY KEY constraint automatically if no clustered indexes are defined.
Clustered indexes are not a good choice for:
Columns that undergo frequent changes
This results in the entire row moving (because SQL Server must keep
the data values of a row in physical order). This is an important
consideration in high-volume transaction processing systems where
data tends to be volatile.
Wide keys
The key values from the clustered index are used by all
nonclustered indexes as lookup keys and therefore are stored in each
nonclustered index leaf entry.
What are the differences between a clustered and a non-clustered index?
Clustered Index
Only one per table
Faster to read than non clustered as data is physically stored in index order
Non Clustered Index
Can be used many times per table
Quicker for insert and update operations than a clustered index
Both types of index will improve performance when select data with fields that use the index but will slow down update and insert operations.
Because of the slower insert and update clustered indexes should be set on a field that is normally incremental ie Id or Timestamp.
SQL Server will normally only use an index if its selectivity is above 95%.
Clustered indexes physically order the data on the disk. This means no extra data is needed for the index, but there can be only one clustered index (obviously). Accessing data using a clustered index is fastest.
All other indexes must be non-clustered. A non-clustered index has a duplicate of the data from the indexed columns kept ordered together with pointers to the actual data rows (pointers to the clustered index if there is one). This means that accessing data through a non-clustered index has to go through an extra layer of indirection. However if you select only the data that's available in the indexed columns you can get the data back directly from the duplicated index data (that's why it's a good idea to SELECT only the columns that you need and not use *)
Clustered indexes are stored physically on the table. This means they are the fastest and you can only have one clustered index per table.
Non-clustered indexes are stored separately, and you can have as many as you want.
The best option is to set your clustered index on the most used unique column, usually the PK. You should always have a well selected clustered index in your tables, unless a very compelling reason--can't think of a single one, but hey, it may be out there--for not doing so comes up.
Clustered Index
There can be only one clustered index for a table.
Usually made on the primary key.
The leaf nodes of a clustered index contain the data pages.
Non-Clustered Index
There can be only 249 non-clustered indexes for a table(till sql version 2005 later versions support upto 999 non-clustered indexes).
Usually made on the any key.
The leaf node of a nonclustered index does not consist of the data pages. Instead, the leaf nodes contain index rows.
Clustered Index
Only one clustered index can be there in a table
Sort the records and store them physically according to the order
Data retrieval is faster than non-clustered indexes
Do not need extra space to store logical structure
Non Clustered Index
There can be any number of non-clustered indexes in a table
Do not affect the physical order. Create a logical order for data rows and use pointers to physical data files
Data insertion/update is faster than clustered index
Use extra space to store logical structure
Apart from these differences you have to know that when table is non-clustered (when the table doesn't have a clustered index) data files are unordered and it uses Heap data structure as the data structure.
Pros:
Clustered indexes work great for ranges (e.g. select * from my_table where my_key between #min and #max)
In some conditions, the DBMS will not have to do work to sort if you use an orderby statement.
Cons:
Clustered indexes are can slow down inserts because the physical layouts of the records have to be modified as records are put in if the new keys are not in sequential order.
Clustered basically means that the data is in that physical order in the table. This is why you can have only one per table.
Unclustered means it's "only" a logical order.
A clustered index actually describes the order in which records are physically stored on the disk, hence the reason you can only have one.
A Non-Clustered Index defines a logical order that does not match the physical order on disk.
An indexed database has two parts: a set of physical records, which are arranged in some arbitrary order, and a set of indexes which identify the sequence in which records should be read to yield a result sorted by some criterion. If there is no correlation between the physical arrangement and the index, then reading out all the records in order may require making lots of independent single-record read operations. Because a database may be able to read dozens of consecutive records in less time than it would take to read two non-consecutive records, performance may be improved if records which are consecutive in the index are also stored consecutively on disk. Specifying that an index is clustered will cause the database to make some effort (different databases differ as to how much) to arrange things so that groups of records which are consecutive in the index will be consecutive on disk.
For example, if one were to start with an empty non-clustered database and add 10,000 records in random sequence, the records would likely be added at the end in the order they were added. Reading out the database in order by the index would require 10,000 one-record reads. If one were to use a clustered database, however, the system might check when adding each record whether the previous record was stored by itself; if it found that to be the case, it might write that record with the new one at the end of the database. It could then look at the physical record before the slots where the moved records used to reside and see if the record that followed that was stored by itself. If it found that to be the case, it could move that record to that spot. Using this sort of approach would cause many records to be grouped together in pairs, thus potentially nearly doubling sequential read speed.
In reality, clustered databases use more sophisticated algorithms than this. A key thing to note, though, is that there is a tradeoff between the time required to update the database and the time required to read it sequentially. Maintaining a clustered database will significantly increase the amount of work required to add, remove, or update records in any way that would affect the sorting sequence. If the database will be read sequentially much more often than it will be updated, clustering can be a big win. If it will be updated often but seldom read out in sequence, clustering can be a big performance drain, especially if the sequence in which items are added to the database is independent of their sort order with regard to the clustered index.
A clustered index is essentially a sorted copy of the data in the indexed columns.
The main advantage of a clustered index is that when your query (seek) locates the data in the index then no additional IO is needed to retrieve that data.
The overhead of maintaining a clustered index, especially in a frequently updated table, can lead to poor performance and for that reason it may be preferable to create a non-clustered index.
You might have gone through theory part from the above posts:
-The clustered Index as we can see points directly to record i.e. its direct so it takes less time for a search. Additionally it will not take any extra memory/space to store the index
-While, in non-clustered Index, it indirectly points to the clustered Index then it will access the actual record, due to its indirect nature it will take some what more time to access.Also it needs its own memory/space to store the index
// Copied from MSDN, the second point of non-clustered index is not clearly mentioned in the other answers.
Clustered
Clustered indexes sort and store the data rows in the table or view
based on their key values. These are the columns included in the
index definition. There can be only one clustered index per table,
because the data rows themselves can be stored in only one order.
The only time the data rows in a table are stored in sorted order is
when the table contains a clustered index. When a table has a
clustered index, the table is called a clustered table. If a table
has no clustered index, its data rows are stored in an unordered
structure called a heap.
Nonclustered
Nonclustered indexes have a structure separate from the data rows. A
nonclustered index contains the nonclustered index key values and
each key value entry has a pointer to the data row that contains the
key value.
The pointer from an index row in a nonclustered index to a data row
is called a row locator. The structure of the row locator depends on
whether the data pages are stored in a heap or a clustered table.
For a heap, a row locator is a pointer to the row. For a clustered
table, the row locator is the clustered index key.
Clustered Indexes
Clustered Indexes are faster for retrieval and slower for insertion
and update.
A table can have only one clustered index.
Don't require extra space to store logical structure.
Determines the order of storing the data on the disk.
Non-Clustered Indexes
Non-clustered indexes are slower in retrieving data and faster in
insertion and update.
A table can have multiple non-clustered indexes.
Require extra space to store logical structure.
Has no effect of order of storing data on the disk.