I want to defragmantation subpartitions table.
My question is should i rebuild all subpartition index and all partition index after compress subpartitions? Should i rebuild all normal index? Or is enough rebuild just subpartition index?
Related
I am running following query in SSMS and execution plan suggesting to add index on columns which are not part of where clause. I was planning to add index on two columns which are being used in where clause (OID and TransactionDate).
SELECT
[OID] , //this is not a PK. Primary key column is not a part of sql script
[CustomerNum] ,
[Amount] ,
[TransactionDate] ,
[CreatedDate]
FROM [dbo].[Transaction]
WHERE OID = 489
AND TransactionDate > '01/01/2018 06:13:06.46';
Index suggestion
CREATE NONCLUSTERED INDEX [<Name of Missing Index, sysname,>]
ON [dbo].[Transaction] ([OID],[TransactionDate])
INCLUDE ([CustomerNum],[Amount],[CreatedDate])
Updated
Do i need to include other columns? Data is being imported to that table through a back end process using SQLBulkCopy class in .net. I am wondering if having non cluster index on all columns would reduce the performance. (In my table is Pk column called TransactionID which is not needed but i have this in the table in case its needed in the future otherwise SQLBulkCopy works better with heap. Other option is to drop and recreate indexes before and after SQLBulkCopy operation)
the INCLUDE keyword specifies the non-key columns to be added to the leaf level of the nonclustered index.
This means that if you will add this index and run the query again, SQL Server can get all the information needed from the index, thus eliminating the need to perform a lookup in the table as well.
As a general rule of thumb - when SSMS suggest an index, create it. You can always drop it later if it doesn't help.
You don't need to add all table columns in your non-clustered index, suggested index is good for the query provided. SQL Server database engine suggestions are usually really good.
INCLUDE keyword is required to avoid KEY LOOKUP and use NONCLUSTERED INDEX SEEK.
All in all: No NONCLUSTERED INDEX results in Clustered index scan
Created NONCLUSTERED INDEX with no included columns results in NONCLUSTERED INDEX scan plus key lookup.
Created NONCLUSTERED INDEX with included columns results in NONCLUSTERED INDEX SEEK.
I have a big test table in my local host database (16 CPU, 124 GB RAM), which has nonclustered column store index. Every day I insert 10 million rows into this table. I found that my system runs very slowly without ending.
I see 2 queries which run parallel without ending and they make system extremely slow:
Query 1:
INSERT INTO TABLE ABC
Query 2:
ALTER NONCLUSTERED COLUMN STORED INDEX TABE ABC.
My questions:
Inserting into nonclustered column store index is very slow because it inserts new records and change the index at the same time. is that correct?
Do I need to disable the INDEX before INSERT and enable INDEX after INSERT to improve the performance?
i use SQL Server 2016 and this Version allows us to INSERT, UPDATE table with nonclustered column stored index.
Thank you
Those can't be running in parallel since according to the documentation:
Once you create a nonclustered columnstore index on a table, you cannot directly modify the data in that table. A query with INSERT, UPDATE, DELETE, or MERGE will fail and return an error message. To add or modify the data in the table, you can do one of the following:
Disable the columnstore index. You can then update the data in the table. If you disable the columnstore index, you can rebuild the columnstore index when you finish updating the data. For example:
Drop the columnstore index, update the table, and then re-create the columnstore index with CREATE COLUMNSTORE INDEX. For example:
EXAMPLE
ALTER INDEX mycolumnstoreindex ON mytable DISABLE;
-- update mytable --
ALTER INDEX mycolumnstoreindex on mytable REBUILD
DROP INDEX mycolumnstoreindex ON mytable
-- update mytable --
CREATE NONCLUSTERED COLUMNSTORE INDEX mycolumnstoreindex ON mytable;
So yes, you need to either DISABLE before the INSERT and REBUILD after the INSERT, or DROP then index before the INSERT and CREATE it again after the INSERT. I'm guessing the runs slow and never finishes is a blocking issue seperate from this index.
If the question was more generic, for a regular NONCLUSTERED INDEX, it could be beneficial to drop and recreate the index when you are trying to insert a large number of records, like 10 million in your case, since
indexes slow down inserts, updates and deletes (page splits, inserting / updating multiple indexes, etc)
inserting that many records will likely cause a lot of of fragmentation
Disclaimer: I am not an MSSQL dba....
I have used two sql queries presented elsewhere on stackoverflow [1]: https://stackoverflow.com/a/7892349/2195559 [2]: https://stackoverflow.com/a/17600911/2195559
[1] claims to show the total space used by a table and its indexes
[2] claims to show the total space used by an index
So looking at my largest table:
[1] gives UsedSpaceKB = 58757504 . So the table and indexes are 56GB
[2] gives Indexsize(KB) = 55166168 for the largest index on that same table (there are some others on the same table - but they are tiny in comparison). So the largest index is 52GB
[1] - [2] (for indexes on the same table) = 4GB.
So IF [1] and [2] are correct and I have understood the output then I have a 4GB table with a 52GB primary key index on it.
Questions:
have I understood [1] and [2] correctly?
if so then how can I have a pk index that is 52GB on a table that is itself just 4GB? We use nightly batch jobs to remove old rows in this table (its an audit log), is it possible that the index is not being reduced when rows are removed from the table?
what's the simplest & quickest way to clear this index - drop and build or is there some more efficient mechanism?
The ddl for the table and pk index are
CREATE TABLE "YYY"."XXX_audit"
(
id numeric(19,0) PRIMARY KEY,
version numeric(19,0),
compressed_response image,
date_created datetime,
duration numeric(19,0),
request text,
response text,
session_id varchar(255),
uid varchar(255),
webservice_uri varchar(255),
event_id varchar(36)
)
GO
CREATE UNIQUE INDEX PK__XXX_audit__6CC31A31 ON "YYY"."XXX_audit"(id)
have I understood 1 and [2] correctly?
No. You have a table with a clustered index of size 52 GB and a non-clustered index of size 4GB. Total size of indexes: 56 GB. You do not have anything other than indexes. There is no 'table', there are only indexes.
I suggest you read Table and Index Organization first. Tables can be organized as heaps, as clustered index B-Trees (and as clustered columnstores or as Hekaton indexes in SQL Server 2014 and later). What you are searching for as the 'table' would be a base heap, but since you declared an primary key that had become the clustered index of the table you do not have a base heap.
Now I am looking for your help to create index on these.
Now this is my table structure
This the query I need index for maximum performance.
select PageId
from tblPages
where UrlChecksumCode = #UrlChecksumCode
and PageUrl = #PageUrl
Now I am very bad with indexes. I plan like that when the query is executing it will find first that UrlChecksumCode rows then look pageurl columns. If you also explain me why to make such index I really appreciate that. Thank you.
one way, since your pageURL is long(nvarchar(1000) and an index can only be 900 bytes if you don't use included columns, I have created this index with included columns
create index ix_IndexName on tblPages (UrlChecksumCode) INCLUDE(PageUrl)
or
create index ix_IndexName on tblPages (UrlChecksumCode) INCLUDE(PageUrl, PageID)
See also SQL Server covering indexes
why is URL nvarchar instead of varchar...can they be unicode? if not make it varchar and save 1/2 the size
Create Nonclustered Indexes
create nonclustered index indexname on tablename(columnname)
Drop Index
Drop Index index name on tablename
See all Indexes in a database
select * from sys.indexes
You probably want an index over (UrlChecksumCode and PageUrl), because that's what it's selecting against.
Though you might, under some data patterns, have better performance if you just index over (UrlChecksumCode) because the PageUrl would require a larger text index
If you want to create cluster then , you can used this,
CREATE CLUSTERED INDEX IX_IndexName ON tblPages (UrlChecksumCode) INCLUDE(PageUrl)
The following query retrieves the indexes created on your table.
EXECUTE sp_helpindex tablename
Brief introduction for cluster indexing:-
A clustered index defines the order in which data is physically stored in a table. Table data can be sorted in only way, therefore, there can be only one clustered index per table. In SQL Server, the primary key constraint automatically creates a clustered index on that particular column. or you can create custom cluster index removing default cluster index which created by primary key.
i think even we can use clusterd index as for fast search like
create clusteredindex someName
on tableName(columnName1,columnName2,......)
but only one clustered index we can create for a tab
What is Clustered and non clustered indexing? How to index a table using sql server 2000 Enterprise manager?
In a clustered index on ID, the table rows are ordered by ID.
In a non-clustered index on ID, the references to table rows are ordered by ID.
We can compare a database to a CSV file:
ID,Value
-------
1,ReallyReallyLongValue1
3,ReallyReallyLongValue2
In a clustered table, when we insert a new row, we need to squeeze it between the existing rows:
ID,Value
-------
1,ReallyReallyLongValue1
2,ReallyReallyLongValue2
3,ReallyReallyLongValue3
, which is slow on insert but fast on retrieve.
In a non-clustered table, we keep a separate file index file which orders our rows:
Id,RowNumber
------------
1, 1
3, 2
When we insert the new row, we just append it to our main file and update the short index file:
ID,Value
-------
1,ReallyReallyLongValue1
3,ReallyReallyLongValue3
2,ReallyReallyLongValue2
Id,RowNumber
------------
1, 1
2, 3
3, 2
, which is fast on insert but less efficient on retrieve.
In real databases indexes use more efficient binary trees, but the principle remains the same.
Clustered indexes are faster on SELECT, non-clustered indexes are faster on INSERT / UPDATE / DELETE
A clustered index means that the rows are physically ordered by the values in that index. A non-clustered index means that an index table is kept up to date that allows for quick seeking and sorting based upon value, but does not physically order the rows.
Only one clustered index can exist for a table, and if a primary key exists then that is the clustered index (in SQL Server).
A clustered index defines how the actual table is stored. The rows are stored in a way to make searches on the fields in the clustered index fast. (They're not physically stored in the sort order of the index fields, but in a binary tree or something similiar.)
You can have only one clustered index per table. The clustered index contains all fields in the table, for example:
indexfield1 - indexfield2 - field2 - field3 - ....
A non-clustered index is like a separate table. It contains the fields in the index, and a reference to the fields in the table. For example:
secondindexfield1 - secondindexfield2 - reference to table row
When searching a non-clustered index, SQL server will find the value in the index, do a "bookmark lookup" to the table, and retrieve the other row fields from there. This is why non-clustered indexes perform slightly less wel then clustered indexes.
To add an index in SQL Server Management Studio, expand the table node in object view. Right click on "Indexes" and select "New Index".
Clustered Index: Only one clustered index per table is allowed. If an index is clustered, it means that the table on which the clustered index is based is physically sorted according to that index. Think of the page numbers in an encyclopedia.
Non-clustered Index: Can have many non-clustered indexes per table. Think of the keyword index at the back of the book.