Let's say I have a table like this:
create table test_partitions
(
pk_id int not null,
col1 nvarchar(20),
col2 nvarchar(100),
constraint pk_test_partitions primary key (pk_id, col1)
);
I want to partition this table to improve query performance so that I don't have to look through the whole table every time I need something. So I added a calculated column:
create table test_partitions
(
pk_id int not null,
partition_id as pk_id % 10 persisted not null,
col1 nvarchar(20),
col2 nvarchar(100),
constraint pk_test_partitions primary key (pk_id, col1)
);
So ever time I do select * from test_partitions where pk_id = 123 I want SQL Server to look in only 1/10th of the entire table. I don't want to add partition_id column to the primary key because it will never be part of the where clause. How do I partition my table on partition_id?
Just right now i did test and found a solution
SELECT TOP ((SELECT COUNT(*) [TABLE_NAME])/10)
* FROM [TABLE_NAME]
it returns 1/10th of your records
To improve your query performance you can apply pagination. That is you can use fetch next query to achieve your output.
Please go through this for more details.
SELECT column-names FROM table-name
ORDER BY column-names
OFFSET n ROWS
FETCH NEXT m ROWS ONLY
Related
How can I improve my select query in a table with more 150 million records in SQL Server? I need to run a simple select and retrieve the result in the minimum time as possible. Should I create some index? Table partition? What do you guys recommend for that?
Here is my current scenario:
Table:
CREATE TABLE [dbo].[table_name]
(
[id] [BIGINT] IDENTITY NOT NULL,
[key] [VARCHAR](20) NOT NULL,
[text_value] [TEXT] NOT NULL,
CONSTRAINT [PK_table_name]
PRIMARY KEY CLUSTERED ([id] ASC)
)
GO
Select:
SELECT TOP 1
text_value
FROM
table_name (NOLOCK)
WHERE
key = #Key
Additional info:
That table won't have updates or deletes
The column text_value has a Json that will be retrieved on the Select and an application will handle this info
No other queries will run on that table, just the query above to retrieve always the text_value based on key column
Every 2 or 3 months about 15 millions are added to the table
For that query:
SELECT top 1 text_value FROM table_name (NOLOCK) where key = #Key
I would add the following index:
CREATE INDEX idx ON table_name (key)
INCLUDE (text_value);
The lookup will always be on the key column so that will form the index structure, and you want to include the text_value but not have it in the non-leaf pages. This should always result in an index seek without a key lookup (a covering index).
Also, do not use the TEXT data type as it will be removed in a future version, use VARCHAR(MAX) instead. Ref: https://learn.microsoft.com/en-us/sql/t-sql/data-types/ntext-text-and-image-transact-sql?view=sql-server-2017
In my head this sounds improbable, but I'd like to know if I can do it:
INSERT INTO MyTable (Name)
VALUES ('First'),
('Second'),
('Third'),
('Fourth'),
('Fifth');
SELECT INSERTED Name, ID FROM TheAboveQuery
Where ID is an auto-indexed column?
Just to clarify, I want to select ONLY the newly inserted rows.
Starting with SQL Server 2008 you can use OUTPUT clause with INSERT statement
DECLARE #T TABLE (ID INT, Name NVARCHAR(100))
INSERT INTO MyTable (Name)
OUTPUT INSERTED.ID, INSERTED.Name INTO #T
VALUES
('First'),
('Second'),
('Third'),
('Fourth'),
('Fifth');
SELECT Name, ID FROM #T;
UPDATE: if table have no triggers
INSERT INTO MyTable (Name)
OUTPUT INSERTED.ID, INSERTED.Name
VALUES
('First'),
('Second'),
('Third'),
('Fourth'),
('Fifth');
Sure, you can use an IDENTITY property on your ID field, and create the CLUSTERED INDEX on it
ONLINE DEMO
create table MyTable ( ID int identity(1,1),
[Name] varchar(64),
constraint [PK_MyTable] primary key clustered (ID asc) on [Primary]
)
--suppose this data already existed...
INSERT INTO MyTable (Name)
VALUES
('First'),
('Second'),
('Third'),
('Fourth'),
('Fifth');
--now we insert some more... and then only return these rows
INSERT INTO MyTable (Name)
VALUES
('Sixth'),
('Seventh')
select top (##ROWCOUNT)
ID,
Name
from MyTable
order by ID desc
##ROWCOUNT returns the number of rows affected by the last statement executed. You can always see this in the messages tab of SQL Server Management Studio. Thus, we are getting the number of rows inserted and combining it with TOP which limits the rows returned in a query to the specified number of rows (or percentage if you use [PERCENT]). It is important that you use ORDER BY when using TOP otherwise your results aren't guaranteed to be the same
From my previous edited answer...
If you are trying to see what values were inserted, then I assume you are inserting them a different way and this is usually handled with an OUTPUT clause, TRIGGER if you are trying to do something with these records after the insert, etc... more information would be needed.
I have a table that can be simplified to the below:
Create Table Data (
DATAID bigint identity(1,1) NOT NULL,
VALUE1 varchar(200) NOT NULL,
VALUE2 varchar(200) NOT NULL,
CONSTRAINT PK_DATA PRIMARY KEY CLUSTERED (DATAID ASC)
)
Among others, this index exists:
CREATE NONCLUSTERED INDEX VALUEIDX ON dbo.DATA
(VALUE1 ASC) INCLUDE (VALUE2)
The table has about 9 million rows with mostly sparse data in VALUE1 and VALUE2.
The query Select Count(*) from DATA takes about 30 seconds. And the following query takes 1 minute and 30 seconds:
Select Count(*) from DATA Where VALUE1<>VALUE2
Is there any way I can make this faster? I basically need to find (and update) all rows where VALUE1 is different from VALUE2. I considered adding a bit field called ISDIFF and update that via a Trigger whenever the value fields are updated is updated. But then I need to create an index on the bit field and select WHERE ISDIFF=1.
Any help will be appreciated.
PS: Using MS SQL Server 2008
Yes i am very well aware the consequences. But i just want to reorder them. Start from 1 to end.
How do I go about reordering the keys using a single query ?
It is clustered primary key index
Reordering like
First record Id 1
second record Id 2
The primary key is Int
Drop PK constraint
Drop Identity column
Re-create Identity Column
Re-Create PK
USE Test
go
if(object_id('IdentityTest') Is not null)
drop table IdentityTest
create table IdentityTest
(
Id int identity not null,
Name varchar(5),
constraint pk primary key (Id)
)
set identity_insert dbo.IdentityTest ON
insert into dbo.IdentityTest (Id,Name) Values(23,'A'),(26,'B'),(34,'C'),(35,'D'),(40,'E')
set identity_insert dbo.IdentityTest OFF
select * from IdentityTest
------------------1. Drop PK constraint ------------------------------------
ALTER TABLE [dbo].[IdentityTest] DROP CONSTRAINT [pk]
GO
------------------2. Drop Identity column -----------------------------------
ALTER table dbo.IdentityTest
drop column Id
------------------3. Re-create Identity Column -----------------------------------
ALTER table dbo.IdentityTest
add Id int identity(1,1)
-------------------4. Re-Create PK-----------------------
ALTER TABLE [dbo].[IdentityTest] ADD CONSTRAINT [pk] PRIMARY KEY CLUSTERED
(
[Id] ASC
)
--------------------------------------------------------------
insert into dbo.IdentityTest (Name) Values('F')
select * from IdentityTest
IDENTITY columns are not updatable irrespective of SET IDENTITY_INSERT options.
You could create a shadow table with the same definition as the original except for the IDENTITY property. Switch into that (this is a metadata only change with no movement of rows that just affects the table's definition) then update the rows and switch back though.
A full worked example going from a situation with gaps to no gaps is shown below (error handling and transactions are omitted below for brevity).
Demo Scenario
/*Your original table*/
CREATE TABLE YourTable
(
Id INT IDENTITY PRIMARY KEY,
OtherColumns CHAR(100) NULL
)
/*Some dummy data*/
INSERT INTO YourTable (OtherColumns) VALUES ('A'),('B'),('C')
/*Delete a row leaving a gap*/
DELETE FROM YourTable WHERE Id =2
/*Verify there is a gap*/
SELECT *
FROM YourTable
Remove Gaps
/*Create table with same definition as original but no `IDENTITY`*/
CREATE TABLE ShadowTable
(
Id INT PRIMARY KEY,
OtherColumns CHAR(100)
)
/*1st metadata switch*/
ALTER TABLE YourTable SWITCH TO ShadowTable;
/*Do the update*/
WITH CTE AS
(
SELECT *,
ROW_NUMBER() OVER (ORDER BY Id) AS RN
FROM ShadowTable
)
UPDATE CTE SET Id = RN
/*Metadata switch back to restore IDENTITY property*/
ALTER TABLE ShadowTable SWITCH TO YourTable;
/*Remove unneeded table*/
DROP TABLE ShadowTable;
/*No Gaps*/
SELECT *
FROM YourTable
I don't think there is any way to do this in a single query. Your best bet is to copy the data to a new table, drop and recreate the original table (or delete the data and reseed the identity) and reinsert the data in the original order using the previous identity as the ordering (but not re-inserting it).
CREATE TABLE Table1_Stg (bla bla bla)
INSERT INTO Table1_Stg (Column2, Column3,...) SELECT Column2, Column3,... FROM Table1 ORDER BY Id
Here the Id column is excluded from the SELECT column list.
Or, you can do:
SELECT * INTO Table1_Stg FROM Table1 ORDER BY Id
DROP Table1
sp_rename Table1_stg Table1
Please lookup the usage for sp_rename as I am doing this from memory.
Hope this helps.
EDIT: Please save a script with all your indexes and constraints if any on Table1.
EDIT2: Added second method of creating table and inserting into table.
UPDATE tbl SET id = (SELECT COUNT(*) FROM tbl t WHERE t.id <= tbl.id);
This last statement is genius. Just had to remove the primary key from the table design first and make sure under the design option Identity Specifications is set to no. Once you run the query set these options back.
Just as the title says, if I have a table has a struct like this
create table myTable
(
ID int identity(1,1) not null,
col1 varchar(20) not null,
col2 varchar(20) not null
constraint pk_myTable primary key (ID)
)
If I insert some value in this table it would automatically take value in col ID. I wanna know is a way I could get SQL Server to store hex equivalent in the column. Say, for 1, it should store 0x01 2 as 0x02... 10 as 0xA and so on?
I don't wanna know any tricks or something, I know I can create this this col as varchar and then create a procedure for insert, or create a trigger for insert, that would do the required transformation and would produce the desired result, that's not a big issue.
But what I wanna know is there a inbuilt function/procedure/trigger that would help me achieve what I am trying to achieve? Not necessarily in SQL Server 2008, but may be in 2012, does there exists anything like this?
I agree with Joe Stefanelli and Habo above that you don't need do do this. That said the technique is usefull in other scenarios.
Define your table like this (note the computed column)
CREATE TABLE mytable(
ID int identity(1,1) not null,
col1 varchar(20) not null,
col2 varchar(20) not null,
ComputedHexColumn AS CONVERT(VARBINARY(8), ID),
CONSTRAINT pk_myTable PRIMARY KEY CLUSTERED(ID)
)
Then populate it
INSERT INTO mytable(col1,col2)
SELECT 'col1 - 1','col2 - 1'
UNION SELECT 'col1 - 2','col2 - 2'
UNION SELECT 'col1 - 3','col2 - 3'
UNION SELECT 'col1 - 4','col2 - 4'
then anytime you select from the table it will include the hex value
SELECT * FROM mytable
In cases where the computed value was much more expensive to calculate then you can mark the column as PERSISTED. The database will then physically store the value and update it when appropriate.