I have 2 questions about SQL Server statistics, please help me. I am using SQL Server 2016.
My table TBL1 has only one column COL1. When I used COL1 in joins with other tables, statistics are automatically created on COL1.
Next I create a non-clustered index on COL1 of TBL1, then another set of statistics are created on COL1. Now I have 2 sets of statistics on COL1.
Out of the above 2 statistics, which statistics are used by SQL Server for further queries? I am assuming that the statistics created by the non-clustered index will be used, am I right?
If I use the Update Statics TBL1 command, all the statistics for TBL1 are updated. In the MSDN documentation, I see that updating statistics causes queries to recompile, what do they mean by re-compiling of queries? The MSDN link is
https://learn.microsoft.com/en-us/sql/relational-databases/statistics/update-statistics?view=sql-server-ver15
Please explain.
If there's only 1 column in your table, there's no reason to have a non-clustered index. This creates a separate copy of that data. Just create the clustered index on that column.
Yes - Since your table only has the one column and an index was created on that column, it's almost certain that SQL Server will use that index whenever joining to that table and thus the statistics for that index will be used.
In this context, it means that the execution plan in cache will be invalidated due to stale statistics and the next time a query executes the optimizer will recreate an execution plan. In other words, it will be assumed there may be a better set of steps to execute the query and the optimizer will try to assemble a better set of steps (execution plan) to execute.
Recommended Reading:
SQL Server Statistics
Understanding Execution Plans
Execution Plan Caching & Reuse
Cluster Columnstore Index maintenance
We are using clustered columnstore index in SqlServer 2016 for 400M rows. We check the time for sample select query - it was 2s just after index creation. After month the same query took up to 20-30 s. Total fragmentation is 96% page fullness 66%, avg row size is 20, depth 3. Reorganizing index gives 1% less fragmentation. Rebuilding is not available as we need to have all data online. We insert 1M rows daily. Any ideas?
How to get query performance similar to initial?
Any ideas?
Problem likely is due to delta store overhead. We encountered similar issue on SQL Server 2016 and full rebuild solves for some time. Suggestions:
Use partitions, perhaps on monthly or bi-weekly basis
Rebuild index on partition level, this can reduce maintenance duration significantly. Run rebuilds at night.
Wait for SQL Server 2019, MS ships ONLINE rebuild of columnstored indexes in this version
Reorganizing index gives 1% less fragmentation
Was it executed it with COMPRESS_ALL_ROW_GROUPS?
ALTER INDEX idx_cci ON table REORGANIZE WITH (COMPRESS_ALL_ROW_GROUPS = ON);
ALTER INDEX idx_cci ON table REORGANIZE;
References:
https://blogs.msdn.microsoft.com/sqlserverstorageengine/2016/03/07/columnstore-index-defragmentation-using-reorganize-command/
http://www.nikoport.com/2014/07/04/clustered-columnstore-indexes-part-33-tuple-mover-that-closes-open-delta-stores/
I got some non-clustered indexes (unique) with uniqueidentifier (GUID) as column. The index gets a lot of fragmentation all the time.
How should I solve this with Ola Hallengren´s maintenance script?
Skip reorg/rebuild of these index?
The problem is described here:
https://blogs.msdn.microsoft.com/sqlserverfaq/2011/08/30/another-reason-of-index-logical-fragmentation/
here you have two options:
Very basic information.
DBCC DBReindex: locks up the tables and users may not be able to access the data until the reindex is done. Bottom line - this drops
the indexes and creates them from scratch. You have brand new indexes
when this is done, so they are in the 'best state' possible. Again, it
ties up the database tables. This is an all or nothing action. If you
stop the process, everything has to rollback.
DBCC INDEXDEFRAG: Does not lock up the tables as much. Users can still access the data. The indexes still exist, they are just being
'fixed'. If this is stopped, it doesn't rollback everything. So the
index will be less defragged than when you started.
If you run DBReindex, you don't need to run INDEXDEFRAG. There's
nothing to defrag when you have brand new indexes.
hope this help!
I think in this instance you should exclude these from Ola Hallengren's maintenance script. Also Guids should not be part of any clustered index.
Using SQL Server 2012 Entreprise.
I have a table of 12 billion rows that takes 700GB on disk, in 30 partitions.
It has only one index, clustered.
I have 500 GB free disk space.
I disabled the index (please don't ask why. If you have to know, I targeted the wrong database).
I now want to enable the index. If I do
alter index x1 on t1 rebuild
I eventually get an error because there is not enough free disk space. That was a painful lesson about disk space requirements for rebuilding a clustered index.
Ideally, I want to rebuild the index one partition at a time. If I do
alter index x1 on t1 rebuild partition = 1
I get the error: Cannot perform the specified operation on disabled index.
Any solution, besides buying more physical disks? The table has not changed since disabling the index (can't be accessed anyway), so I am really looking for a hack that can fool SQL into thinking the index is enabled. Any suggestions?
Thanks
If its a clustered index that you have disabled you have effectively disabled the table the only operations you can execute on this table is "drop" or "rebuild" as far as i am aware.
you could try the deprecated dbbc dbreindex command, maybe you are lucky and it rebuilds more disk space efficiently. Also you might squeeze some more space out if you set a fill factor to 100 when you rebuild. assuming you database table is now only being read.
DBCC DBREINDEX ('Person.Address', 'PK_Address_AddressID', 100)
allows you to reindex just the clustered index.
Currently our database has size 10 GB and is growing by around 3 GB per month. Often I hear that one should from time to time rebuild the indexes, to improve the query execution time. So how often should I rebuild the indexes in the given scenario?
There's a general consensus that you should reorganize ("defragment") your indices as soon as index fragmentation reaches more than 5 (sometimes 10%), and you should rebuild them completely when it goes beyond 30% (at least that's the numbers I've heard advocated in a lot of places).
Michelle Ufford (a.k.a. "SQL Fool") has an automated index defrag script, which uses those exact limits for deciding when to reorganize or rebuild an index.
Also see Brad McGehee's tips on rebuild indexes with some good thoughts and tips on how to deal with index rebuilding.
I use this script here (can't remember when I got this from - whoever it was: many thanks! Really helpful stuff) to display the index fragmentation on all your indices in a given database:
SELECT
t.NAME 'Table name',
i.NAME 'Index name',
ips.index_type_desc,
ips.alloc_unit_type_desc,
ips.index_depth,
ips.index_level,
ips.avg_fragmentation_in_percent,
ips.fragment_count,
ips.avg_fragment_size_in_pages,
ips.page_count,
ips.avg_page_space_used_in_percent,
ips.record_count,
ips.ghost_record_count,
ips.Version_ghost_record_count,
ips.min_record_size_in_bytes,
ips.max_record_size_in_bytes,
ips.avg_record_size_in_bytes,
ips.forwarded_record_count
FROM
sys.dm_db_index_physical_stats(DB_ID(), NULL, NULL, NULL, 'DETAILED') ips
INNER JOIN
sys.tables t ON ips.OBJECT_ID = t.Object_ID
INNER JOIN
sys.indexes i ON ips.index_id = i.index_id AND ips.OBJECT_ID = i.object_id
WHERE
AVG_FRAGMENTATION_IN_PERCENT > 0.0
ORDER BY
AVG_FRAGMENTATION_IN_PERCENT, fragment_count
"When you need to" and "When you can"!
For example...
Test for fragmentation first and decide whether to do nothing, reorg or rebuild.
SQL Fool's script does this, for example, has #minFragmentation and #rebuildThreshold parameters
Do statistics daily, say, but indexes at weekends. What is your maintenance window?
You should rebuild indexes often enough so that production is not detrimentally affected by index degradation. I understand that this seems vague, but all databases are different and are used in different ways. You only need to regularly rebuild/defrag indexes that incur write operations (inserts/updates) – your static or mostly read only tables will not need much reindexing.
You will need to use dbcc showcontig([Table]) to check the fragmentation level of your indexes, determine how often they become fragmented and as to what level the fragmentation actually is.
Use dbcc dbreindex([Table]) to totally rebuild the indexes when they become too fragmented (above 20%-30% or so) but if you cannot find a large enough downtime window and the fragmentation level is relatively low (1%-25%), you should use dbcc indexdefrag([Database], [Table], [Index]) to defrag the index in an "online" fassion. Also keep in mind, that you can stop the index defrag operation and start it again at a later time without losing any work.
Keeping a database and its indexes "in tune" takes a bit of monitoring to really get a feel for when and what to reindex.
Given the size of your database, you can easily rebuild the indexes once per month. But as the size increases, say to around 500 GB, you could do it twice per month.
Mentioned in Bacon Bits comment Ola Hallengren's SQL Server Maintenance Solution IndexOptimize is supported on SQL Server 2008, SQL Server 2008 R2, SQL Server 2012, SQL Server 2014, SQL Server 2016, SQL Server 2017, SQL Server 2019, Azure SQL Database, and Azure SQL Database Managed Instance.
It has 2K stars on https://github.com/olahallengren/sql-server-maintenance-solution.
Michelle Ufford (a.k.a. "SQL Fool") 's automated index defrag script, suggested in the accepted answer, conceptually seems does the same as Ola Hallengren's SQL Server Maintenance Solution, but the latest version is in 2011.