Query sql statistics without SET parameter - sql-server

I try to capture some statistic parameters for logging purpose. "SET parameters" are no option (i.e. set statistics time on).
So I tried to query some DMV:
select '3AAAAAAAAAAA';
--no GO-statement here
select
total_worker_time/execution_count AS [Avg CPU Time],
total_elapsed_time as [Elapsed Time],
total_rows as [Total rows],
st.text,
(select cast(text as varchar(4000)) from ::fn_get_sql((select sql_handle from sys.sysprocesses where spid = ##spid)))
from sys.dm_exec_query_stats AS qs
cross apply sys.dm_exec_sql_text(qs.sql_handle) AS st
--where ???
order by creation_time desc
The information captured here is almost what I need - but:
The query is only listed in the result of the DMV when it run in last executed GO-Block (not in the actual one). This is not what I need. I need something like ##error or ##rowcount what is available within the same GO-block and holds the elapsed time and CPU time. Any ideas how to query this information of the last statment?
If this can be solved: I would like to query the "last" statement execution within the session (##spid) without writing the statement twice.
Update on question:
This query is working "per session" and would list the requested values (although tirvial querys are missing). Top 1 would always bring back the values of the last Statement (not true if fired via exec #SQL what produces anonther session):
print 'hello';
select top 10 'my personal identifier: 1', * FROM sys.messages;
select top 20 'my personal identifier: 2', * FROM sys.messages;
print 'hello';
select 'hello';
select top 30 'my personal identifier: 3', * FROM sys.tables;
select top 1
total_worker_time/execution_count AS [Avg CPU Time],
total_elapsed_time as [Elapsed Time],
total_rows as [Total rows],
substring(st.text, (qs.statement_start_offset / 2) + 1, (case when qs.statement_end_offset = -1 then datalength(st.text) else qs.statement_end_offset end - qs.statement_start_offset ) / 2 + 5) as [executing statement]
from sys.dm_exec_query_stats AS qs
cross apply sys.dm_exec_sql_text(qs.sql_handle) AS st
where st.text = (select cast(text as varchar(4000)) from ::fn_get_sql((select sql_handle from sys.sysprocesses where spid = ##spid)))
order by qs.statement_start_offset desc;
The filter (where-clause) seems to be crude and not very robust. Is there any way to improve this?

I try to answer myself (Jeroen Mostert - Thank you very much for your help!) - the question is unanswered (see below):
The follwing function should give you CPU, execution time, I/O, number or rows of the last statement that was executed in the actual session, if the statement is complex enough to invoke a SQL plan generation. That is, after simple print commands the resultset would be enpty. Even so after the execution of stored procedures if they open a new session (i.e. after exec sp_executesql the resultset will be empty).
For the "average" SQL-Statement the following query should result in a rowset holding the information that you would otherwise get via set statistice time on and set statistice io on.
drop function if exists dbo.ufn_lastsql_resources ;
go
CREATE FUNCTION dbo.ufn_lastsql_resources (#session_id int)
RETURNS TABLE
AS
return
select
top 1
convert(char(10), getdate(), 121) + ' ' + substring(convert(char(40), getdate(), 121), 12,12) + ',' as [Time stamp],
cast(cast((last_worker_time / execution_count / 1000. ) as numeric(9,2)) as varchar(100)) + ',' as [Avg CPU Time in ms],
cast(cast((last_elapsed_time / 1000. ) as numeric(9,2)) as varchar(100)) + ',' as [Elapsed Time in ms],
cast(last_rows as varchar(100)) + ',' as [Total rows],
cast(substring(st.text, (statement_start_offset / 2) + 1, (case when statement_end_offset = -1 then datalength(st.text) else statement_end_offset end - statement_start_offset ) / 2 + 2) as varchar(4000)) + ','
as [executing statement],
last_physical_reads + last_logical_reads as [Reads],
last_logical_writes as [Writes],
--last_grant_kb,
--last_used_grant_kb,
--last_ideal_grant_kb,
--last_reserved_threads,
--last_used_threads
#session_id as spid
from
(
select qs.*
from sys.dm_exec_query_stats as qs
inner join sys.dm_exec_requests as eq
on qs.sql_handle = eq.sql_handle
and qs.plan_handle = eq.plan_handle
and eq.session_id = #session_id
) a
cross apply sys.dm_exec_sql_text(a.sql_handle) AS st
where
substring(st.text, (statement_start_offset / 2) + 1, (case when statement_end_offset = -1 then datalength(st.text) else statement_end_offset end - statement_start_offset ) / 2 + 2) not like '%ufn_lastsql_resources%'
order by
last_execution_time desc, statement_start_offset desc
go
Most probably there are more elegant ways to do so. Maybe it is possible to write something that will work properly even with statements that use an option (recompile) or on exec (#sql) Anyway: I seems to work on SQL Server 2016 and 2012. You need VIEW SERVER STATE permission on the Server. To invoke the function, try:
drop table if exists #t1
select top 10 'statement 1' a, * into #t1 from sys.messages
select 1, * from dbo.ufn_lastsql_resources(##spid) option (recompile)
drop table if exists #t2
select top 20 'statement 2' a, * into #t2 from sys.messages
--select 2, * from dbo.ufn_lastsql_resources(##spid)
select top 3 'statement 3' a, * from sys.messages
select 3, * from dbo.ufn_lastsql_resources(##spid) option (recompile)
The question remins unanswered, as the way is not working properly. It is not sure to catch the right statement out of the batch (top 1 within the session ordered by last_execution time and last in batch. This seems to be the wrong order. As the plans are reused this is the only way, I figured out to work.)

Related

Identify how many times a string exists on a stored procedure in SQL Server

I have to search the occurrence of a particular string inside all available stored procedures in SQL Server. I know that we can get this by using the below query.
SELECT OBJECT_NAME(OBJECT_ID) PrcName
FROM sys.sql_modules
WHERE DEFINITION LIKE '%SearchStr%'
But is there a way we can find out how many times the particular string is available in each stored procedure? This is for estimating the effort modifying the stored procedures.
Any help will be much appreciated.
This will work as tested:
;WITH cte as
(
SELECT OBJECT_NAME(OBJECT_ID) PrcName, OBJECT_ID
FROM sys.sql_modules
WHERE DEFINITION LIKE '%tblNDT%')
select t1.PrcName, (LEN(Definition) - LEN(replace(Definition,'tblNDT',''))) / LEN('tblNDT') Cnt
from cte t1
INNER JOIN sys.sql_modules t2 on t1.object_id = t2.object_id
An easy way of checking how many times something occurs is to take the initial length, replace your string with blanks, recheck the length, and divide by the length of your string:
DECLARE #sentence VARCHAR(100)
DECLARE #word VARCHAR(100)
SET #word = 'Cool'
SET #sentence = 'This cool sentence is really cool. Cool!'
DECLARE #wordlen INT = (SELECT LEN(#word))
--Original sentence and length
SELECT #sentence AS setencelen
SELECT LEN(#sentence) AS origsentence
--With word removed
SELECT REPLACE(#sentence, 'cool', '') AS shortenedsentence
SELECT LEN(REPLACE(#sentence, 'cool', '')) AS shortenedlen
SELECT LEN(#sentence) - LEN(REPLACE(#sentence, 'cool', '')) AS diffinlength
SELECT (LEN(#sentence) - LEN(REPLACE(#sentence, 'cool', ''))) / #wordlen AS occurrences
I have seen this work in some cases and not in others. If you have a bunch of comments that contain the same string, it will count incorrectly.
I have found a solution for this.
DECLARE #cnt AS INT= 1
DECLARE #SearchStr VARCHAR(MAX) = 'SearchText'
;WITH CTE_SearchStr1
AS
(
SELECT #cnt Cnt, #SearchStr SearchStr UNION ALL
SELECT Cnt + 1, #SearchStr+'%'+SearchStr FROM CTE_SearchStr1
),
CTE_SearchStr2
AS
(
SELECT TOP 100 * FROM CTE_SearchStr1
)
SELECT OBJECT_NAME(OBJECT_ID) ObjectName, MAX(cnt) cnt FROM sys.sql_modules a INNER JOIN CTE_SearchStr2 b ON
a.definition LIKE '%'+b.SearchStr+'%'
GROUP BY OBJECT_NAME(OBJECT_ID) ORDER BY 2 DESC
Only problem with the above query is that I can not search for more that 100 times. It will throw the below exception
Msg 530, Level 16, State 1, Line 3 The statement terminated. The
maximum recursion 100 has been exhausted before statement completion.
In my scenario, the number of occurrences are less than 100, but is there a way to overcome this error?

SQL Server : SELECT from a string

I am struggling with this: I have two SQL statements that have 2 different sets of keywords. These are stored in temporary tables since I cannot update, delete or insert into a table.
How do I write a third SQL statement (limited on SQL characters in each statement) that says: "If 'pingu' and 'noot' is correct then true, otherwise if 'sponge' and 'bob' are true display results" (this works)? But then how do I say: "if 'pingu' and 'sponge' is selected then true, or 'bob' and 'noot' are selected then true", but keeping the 'pingu' and 'noot' as true if selected?
Example of keyword list 1: 'Pingu' and 'Noot'
DECLARE #teststring varchar(512) = '{KEYWORD}'
SELECT TOP 1 k.type
FROM (VALUES
('pingu', '66'), ('noot', '66'))
k(word,type) WHERE #teststring LIKE '%' + k.word + '%'
GROUP BY k.type
HAVING COUNT(1) >=2
ORDER BY COUNT(1) DESC;
Example of keyword list 2: 'Sponge' and 'Bob'
DECLARE #teststring varchar(512) = '{KEYWORD}'
SELECT TOP 1 k.type
FROM (VALUES ('sponge', '66'), ('bob', '66')) k (word, type)
WHERE #teststring LIKE '%' + k.word + '%'
GROUP BY k.type
HAVING COUNT(1) >= 2
ORDER BY COUNT(1) DESC;
What about combining the two source queries with a UNION ALL?
For example (adapting your original queries):
DECLARE #teststring varchar(512) = '{KEYWORD}';
WITH Keywords AS (
SELECT *
FROM (VALUES ('pingu', '66'), ('noot', '66')) k(word, type)
UNION ALL
SELECT *
FROM (VALUES ('sponge', '66'), ('bob', '66')) k(word, type)
)
SELECT TOP 1 k.type
FROM Keywords k
WHERE #teststring LIKE '%' + k.word + '%'
GROUP BY k.type
HAVING COUNT(1) >=2
ORDER BY COUNT(1) DESC;
This returns a result row if at least two keywords, regardless of which source query the keywords come from, are found in #teststring.
Note: If your keywords lists are large, it may be worth reworking the query so that an index can be used to make processing the WHERE clause more efficient.

Concatenate the result of an ordered String_Split in a variable

In a SqlServer database I use, the database name is something like StackExchange.Audio.Meta, or StackExchange.Audio or StackOverflow . By sheer luck this is also the url for a website. I only need split it on the dots and reverse it: meta.audio.stackexchange. Adding http:// and .com and I'm done. Obviously Stackoverflow doesn't need any reversing.
Using the SqlServer 2016 string_split function I can easy split and reorder its result:
select value
from string_split(db_name(),'.')
order by row_number() over( order by (select 1)) desc
This gives me
| Value |
-----------------
| Meta |
| Audio |
| StackExchange |
As I need to have the url in a variable I hoped to concatenate it using this answer so my attempt looks like this:
declare #revname nvarchar(150)
select #revname = coalesce(#revname +'.','') + value
from string_split(db_name(),'.')
order by row_number() over( order by (select 1)) desc
However this only returns me the last value, StackExchange. I already noticed the warnings on that answer that this trick only works for certain execution plans as explained here.
The problem seems to be caused by the order by clause. Without that I get all values, but then in the wrong order. I tried to a add ltrimand rtrim function as suggested in the Microsoft article as well as a subquery but so far without luck.
Is there a way I can nudge the Sql Server 2016 Query Engine to concatenate the ordered result from that string_split in a variable?
I do know I can use for XML or even a plain cursor to get the result I need but I don't want to give up this elegant solution yet.
As I'm running this on the Stack Exchange Data Explorer I can't use functions, as we lack the permission to create those. I can do Stored procedures but I hoped I could evade those.
I prepared a SEDE Query to experiment with. The database names to expect are either without dots, aka StackOverflow, with 1 dot: StackOverflow.Meta or 2 dots, `StackExchange.Audio.Meta, the full list of databases is here
I think you are over-complicating things. You could use PARSENAME:
SELECT 'http://' + PARSENAME(db_name(),1) +
ISNULL('.' + PARSENAME(db_name(),2),'') + ISNULL('.'+PARSENAME(db_name(),3),'')
+ '.com'
This is exactly why I have the Presentation Sequence (PS) in my split function. People often scoff at using a UDF for such items, but it is generally a one-time hit to parse something for later consumption.
Select * from [dbo].[udf-Str-Parse]('meta.audio.stackexchange','.')
Returns
Key_PS Key_Value
1 meta
2 audio
3 stackexchange
The UDF
CREATE FUNCTION [dbo].[udf-Str-Parse] (#String varchar(max),#delimeter varchar(10))
--Usage: Select * from [dbo].[udf-Str-Parse]('meta.audio.stackexchange','.')
-- Select * from [dbo].[udf-Str-Parse]('John Cappelletti was here',' ')
-- Select * from [dbo].[udf-Str-Parse]('id26,id46|id658,id967','|')
Returns #ReturnTable Table (Key_PS int IDENTITY(1,1) NOT NULL , Key_Value varchar(max))
As
Begin
Declare #intPos int,#SubStr varchar(max)
Set #IntPos = CharIndex(#delimeter, #String)
Set #String = Replace(#String,#delimeter+#delimeter,#delimeter)
While #IntPos > 0
Begin
Set #SubStr = Substring(#String, 0, #IntPos)
Insert into #ReturnTable (Key_Value) values (#SubStr)
Set #String = Replace(#String, #SubStr + #delimeter, '')
Set #IntPos = CharIndex(#delimeter, #String)
End
Insert into #ReturnTable (Key_Value) values (#String)
Return
End
Probably less elegant solution but it takes only a few lines and works with any number of dots.
;with cte as (--build xml
select 1 num, cast('<str><s>'+replace(db_name(),'.','</s><s>')+'</s></str>' as xml) str
)
,x as (--make table from xml
select row_number() over(order by num) rn, --add numbers to sort later
t.v.value('.[1]','varchar(50)') s
from cte cross apply cte.str.nodes('str/s') t(v)
)
--combine into string
select STUFF((SELECT '.' + s AS [text()]
FROM x
order by rn desc --in reverse order
FOR XML PATH('')
), 1, 1, '' ) name
Is there a way I can nudge the Sql Server 2016 Query Engine to concatenate the ordered result from that string_split in a variable?
You can just use CONCAT:
DECLARE #URL NVARCHAR(MAX)
SELECT #URL = CONCAT(value, '.', #URL) FROM STRING_SPLIT(DB_NAME(), '.')
SET #URL = CONCAT('http://', LOWER(#URL), 'com');
The reversal is accomplished by the order of parameters to CONCAT. Here's an example.
It changes StackExchange.Garage.Meta to http://meta.garage.stackexchange.com.
This can be used to split and reverse strings in general, but note that it does leave a trailing delimiter. I'm sure you could add some logic or a COALESCE in there to make that not happen.
Also note that vNext will be adding STRING_AGG.
To answer the 'X' of this XY problem, and to address the HTTPS switch (especially for Meta sites) and some other site name changes, I've written the following SEDE query which outputs all site names in the format used on the network site list.
SELECT name,
LOWER('https://' +
IIF(PATINDEX('%.Mathoverflow%', name) > 0,
IIF(PATINDEX('%.Meta', name) > 0, 'meta.mathoverflow.net', 'mathoverflow.net'),
IIF(PATINDEX('%.Ubuntu%', name) > 0,
IIF(PATINDEX('%.Meta', name) > 0, 'meta.askubuntu.com', 'askubuntu.com'),
IIF(PATINDEX('StackExchange.%', name) > 0,
CASE SUBSTRING(name, 15, 200)
WHEN 'Audio' THEN 'video'
WHEN 'Audio.Meta' THEN 'video.meta'
WHEN 'Beer' THEN 'alcohol'
WHEN 'Beer.Meta' THEN 'alcohol.meta'
WHEN 'CogSci' THEN 'psychology'
WHEN 'CogSci.Meta' THEN 'psychology.meta'
WHEN 'Garage' THEN 'mechanics'
WHEN 'Garage.Meta' THEN 'mechanics.meta'
WHEN 'Health' THEN 'medicalsciences'
WHEN 'Health.Meta' THEN 'medicalsciences.meta'
WHEN 'Moderators' THEN 'communitybuilding'
WHEN 'Moderators.Meta' THEN 'communitybuilding.meta'
WHEN 'Photography' THEN 'photo'
WHEN 'Photography.Meta' THEN 'photo.meta'
WHEN 'Programmers' THEN 'softwareengineering'
WHEN 'Programmers.Meta' THEN 'softwareengineering.meta'
WHEN 'Vegetarian' THEN 'vegetarianism'
WHEN 'Vegetarian.Meta' THEN 'vegetarianism.meta'
WHEN 'Writers' THEN 'writing'
WHEN 'Writers.Meta' THEN 'writing.meta'
ELSE SUBSTRING(name, 15, 200)
END + '.stackexchange.com',
IIF(PATINDEX('StackOverflow.%', name) > 0,
CASE SUBSTRING(name, 15, 200)
WHEN 'Br' THEN 'pt'
WHEN 'Br.Meta' THEN 'pt.meta'
ELSE SUBSTRING(name, 15, 200)
END + '.stackoverflow.com',
IIF(PATINDEX('%.Meta', name) > 0,
'meta.' + SUBSTRING(name, 0, PATINDEX('%.Meta', name)) + '.com',
name + '.com'
)
)
)
)
) + '/'
)
FROM sys.databases WHERE database_id > 5

Paging, sorting and filtering in a stored procedure (SQL Server)

I was looking at different ways of writing a stored procedure to return a "page" of data. This was for use with the ASP ObjectDataSource, but it could be considered a more general problem.
The requirement is to return a subset of the data based on the usual paging parameters; startPageIndex and maximumRows, but also a sortBy parameter to allow the data to be sorted. Also there are some parameters passed in to filter the data on various conditions.
One common way to do this seems to be something like this:
[Method 1]
;WITH stuff AS (
SELECT
CASE
WHEN #SortBy = 'Name' THEN ROW_NUMBER() OVER (ORDER BY Name)
WHEN #SortBy = 'Name DESC' THEN ROW_NUMBER() OVER (ORDER BY Name DESC)
WHEN #SortBy = ...
ELSE ROW_NUMBER() OVER (ORDER BY whatever)
END AS Row,
.,
.,
.,
FROM Table1
INNER JOIN Table2 ...
LEFT JOIN Table3 ...
WHERE ... (lots of things to check)
)
SELECT *
FROM stuff
WHERE (Row > #startRowIndex)
AND (Row <= #startRowIndex + #maximumRows OR #maximumRows <= 0)
ORDER BY Row
One problem with this is that it doesn't give the total count and generally we need another stored procedure for that. This second stored procedure has to replicate the parameter list and the complex WHERE clause. Not nice.
One solution is to append an extra column to the final select list, (SELECT COUNT(*) FROM stuff) AS TotalRows. This gives us the total but repeats it for every row in the result set, which is not ideal.
[Method 2]
An interesting alternative is given here (https://web.archive.org/web/20211020111700/https://www.4guysfromrolla.com/articles/032206-1.aspx) using dynamic SQL. He reckons that the performance is better because the CASE statement in the first solution drags things down. Fair enough, and this solution makes it easy to get the totalRows and slap it into an output parameter. But I hate coding dynamic SQL. All that 'bit of SQL ' + STR(#parm1) +' bit more SQL' gubbins.
[Method 3]
The only way I can find to get what I want, without repeating code which would have to be synchronized, and keeping things reasonably readable is to go back to the "old way" of using a table variable:
DECLARE #stuff TABLE (Row INT, ...)
INSERT INTO #stuff
SELECT
CASE
WHEN #SortBy = 'Name' THEN ROW_NUMBER() OVER (ORDER BY Name)
WHEN #SortBy = 'Name DESC' THEN ROW_NUMBER() OVER (ORDER BY Name DESC)
WHEN #SortBy = ...
ELSE ROW_NUMBER() OVER (ORDER BY whatever)
END AS Row,
.,
.,
.,
FROM Table1
INNER JOIN Table2 ...
LEFT JOIN Table3 ...
WHERE ... (lots of things to check)
SELECT *
FROM stuff
WHERE (Row > #startRowIndex)
AND (Row <= #startRowIndex + #maximumRows OR #maximumRows <= 0)
ORDER BY Row
(Or a similar method using an IDENTITY column on the table variable).
Here I can just add a SELECT COUNT on the table variable to get the totalRows and put it into an output parameter.
I did some tests and with a fairly simple version of the query (no sortBy and no filter), method 1 seems to come up on top (almost twice as quick as the other 2). Then I decided to test probably I needed the complexity and I needed the SQL to be in stored procedures. With this I get method 1 taking nearly twice as long as the other 2 methods. Which seems strange.
Is there any good reason why I shouldn't spurn CTEs and stick with method 3?
UPDATE - 15 March 2012
I tried adapting Method 1 to dump the page from the CTE into a temporary table so that I could extract the TotalRows and then select just the relevant columns for the resultset. This seemed to add significantly to the time (more than I expected). I should add that I'm running this on a laptop with SQL Server Express 2008 (all that I have available) but still the comparison should be valid.
I looked again at the dynamic SQL method. It turns out I wasn't really doing it properly (just concatenating strings together). I set it up as in the documentation for sp_executesql (with a parameter description string and parameter list) and it's much more readable. Also this method runs fastest in my environment. Why that should be still baffles me, but I guess the answer is hinted at in Hogan's comment.
I would most likely split the #SortBy argument into two, #SortColumn and #SortDirection, and use them like this:
…
ROW_NUMBER() OVER (
ORDER BY CASE #SortColumn
WHEN 'Name' THEN Name
WHEN 'OtherName' THEN OtherName
…
END *
CASE #SortDirection
WHEN 'DESC' THEN -1
ELSE 1
END
) AS Row
…
And this is how the TotalRows column could be defined (in the main select):
…
COUNT(*) OVER () AS TotalRows
…
I would definitely want to do a combination of a temp table and NTILE for this sort of approach.
The temp table will allow you to do your complicated series of conditions just once. Because you're only storing the pieces you care about, it also means that when you start doing selects against it further in the procedure, it should have a smaller overall memory usage than if you ran the condition multiple times.
I like NTILE() for this better than ROW_NUMBER() because it's doing the work you're trying to accomplish for you, rather than having additional where conditions to worry about.
The example below is one based off a similar query I'm using as part of a research query; I have an ID I can use that I know will be unique in the results. Using an ID that was an identity column would also be appropriate here, though.
--DECLARES here would be stored procedure parameters
declare #pagesize int, #sortby varchar(25), #page int = 1;
--Create temp with all relevant columns; ID here could be an identity PK to help with paging query below
create table #temp (id int not null primary key clustered, status varchar(50), lastname varchar(100), startdate datetime);
--Insert into #temp based off of your complex conditions, but with no attempt at paging
insert into #temp
(id, status, lastname, startdate)
select id, status, lastname, startdate
from Table1 ...etc.
where ...complicated conditions
SET #pagesize = 50;
SET #page = 5;--OR CAST(#startRowIndex/#pagesize as int)+1
SET #sortby = 'name';
--Only use the id and count to use NTILE
;with paging(id, pagenum, totalrows) as
(
select id,
NTILE((SELECT COUNT(*) cnt FROM #temp)/#pagesize) OVER(ORDER BY CASE WHEN #sortby = 'NAME' THEN lastname ELSE convert(varchar(10), startdate, 112) END),
cnt
FROM #temp
cross apply (SELECT COUNT(*) cnt FROM #temp) total
)
--Use the id to join back to main select
SELECT *
FROM paging
JOIN #temp ON paging.id = #temp.id
WHERE paging.pagenum = #page
--Don't need the drop in the procedure, included here for rerunnability
drop table #temp;
I generally prefer temp tables over table variables in this scenario, largely so that there are definite statistics on the result set you have. (Search for temp table vs table variable and you'll find plenty of examples as to why)
Dynamic SQL would be most useful for handling the sorting method. Using my example, you could do the main query in dynamic SQL and only pull the sort method you want to pull into the OVER().
The example above also does the total in each row of the return set, which as you mentioned was not ideal. You could, instead, have a #totalrows output variable in your procedure and pull it as well as the result set. That would save you the CROSS APPLY that I'm doing above in the paging CTE.
I would create one procedure to stage, sort, and paginate (using NTILE()) a staging table; and a second procedure to retrieve by page. This way you don't have to run the entire main query for each page.
This example queries AdventureWorks.HumanResources.Employee:
--------------------------------------------------------------------------
create procedure dbo.EmployeesByMartialStatus
#MaritalStatus nchar(1)
, #sort varchar(20)
as
-- Init staging table
if exists(
select 1 from sys.objects o
inner join sys.schemas s on s.schema_id=o.schema_id
and s.name='Staging'
and o.name='EmployeesByMartialStatus'
where type='U'
)
drop table Staging.EmployeesByMartialStatus;
-- Populate staging table with sort value
with s as (
select *
, sr=ROW_NUMBER()over(order by case #sort
when 'NationalIDNumber' then NationalIDNumber
when 'ManagerID' then ManagerID
-- plus any other sort conditions
else EmployeeID end)
from AdventureWorks.HumanResources.Employee
where MaritalStatus=#MaritalStatus
)
select *
into #temp
from s;
-- And now pages
declare #RowCount int; select #rowCount=COUNT(*) from #temp;
declare #PageCount int=ceiling(#rowCount/20); --assuming 20 lines/page
select *
, Page=NTILE(#PageCount)over(order by sr)
into Staging.EmployeesByMartialStatus
from #temp;
go
--------------------------------------------------------------------------
-- procedure to retrieve selected pages
create procedure EmployeesByMartialStatus_GetPage
#page int
as
declare #MaxPage int;
select #MaxPage=MAX(Page) from Staging.EmployeesByMartialStatus;
set #page=case when #page not between 1 and #MaxPage then 1 else #page end;
select EmployeeID,NationalIDNumber,ContactID,LoginID,ManagerID
, Title,BirthDate,MaritalStatus,Gender,HireDate,SalariedFlag,VacationHours,SickLeaveHours
, CurrentFlag,rowguid,ModifiedDate
from Staging.EmployeesByMartialStatus
where Page=#page
GO
--------------------------------------------------------------------------
-- Usage
-- Load staging
exec dbo.EmployeesByMartialStatus 'M','NationalIDNumber';
-- Get pages 1 through n
exec dbo.EmployeesByMartialStatus_GetPage 1;
exec dbo.EmployeesByMartialStatus_GetPage 2;
-- ...etc (this would actually be a foreach loop, but that detail is omitted for brevity)
GO
I use this method of using EXEC():
-- SP parameters:
-- #query: Your query as an input parameter
-- #maximumRows: As number of rows per page
-- #startPageIndex: As number of page to filter
-- #sortBy: As a field name or field names with supporting DESC keyword
DECLARE #query nvarchar(max) = 'SELECT * FROM sys.Objects',
#maximumRows int = 8,
#startPageIndex int = 3,
#sortBy as nvarchar(100) = 'name Desc'
SET #query = ';WITH CTE AS (' + #query + ')' +
'SELECT *, (dt.pagingRowNo - 1) / ' + CAST(#maximumRows as nvarchar(10)) + ' + 1 As pagingPageNo' +
', pagingCountRow / ' + CAST(#maximumRows as nvarchar(10)) + ' As pagingCountPage ' +
', (dt.pagingRowNo - 1) % ' + CAST(#maximumRows as nvarchar(10)) + ' + 1 As pagingRowInPage ' +
'FROM ( SELECT *, ROW_NUMBER() OVER (ORDER BY ' + #sortBy + ') As pagingRowNo, COUNT(*) OVER () AS pagingCountRow ' +
'FROM CTE) dt ' +
'WHERE (dt.pagingRowNo - 1) / ' + CAST(#maximumRows as nvarchar(10)) + ' + 1 = ' + CAST(#startPageIndex as nvarchar(10))
EXEC(#query)
At result-set after query result columns:
Note:
I add some extra columns that you can remove them:
pagingRowNo : The row number
pagingCountRow : The total number of rows
pagingPageNo : The current page number
pagingCountPage : The total number of pages
pagingRowInPage : The row number that started with 1 in this page

How do I easily find IDENTITY columns in danger of overflowing?

My database is getting old, and one of my biggest INT IDENTITY columns has a value around 1.3 billion. This will overflow around 2.1 billion. I plan on increasing it's size, but I don't want to do it too soon because of the number of records in the database. I may replace my database hardware before I increase the column size, which could offset any performance problems this could cause. I also want to keep an eye on all the other columns in my databases that are more than 50% full. It's a lot of tables, and checking each one manually is not practical.
This is how I am getting the value now (I know the value returned may be slightly out of date, but it's good enough for my purposes):
PRINT IDENT_CURRENT('MyDatabase.dbo.MyTable')
Can I use the INFORMATION_SCHEMA to get this information?
You can consult the sys.identity_columns system catalog view:
SELECT
name,
seed_value, increment_value, last_value
FROM sys.identity_columns
This gives you the name, seed, increment and last value for each column. The view also contains the data type, so you can easily figure out which identity columns might be running out of numbers soonish...
I created a stored procedure to solve this problem. It uses the INFORMATION_SCHEMA to find the IDENTITY columns, and then uses IDENT_CURRENT and the column's DATA_TYPE to calculate the percent full. Specify the database as the first parameter, and then optionally the minimum percent and data type.
EXEC master.dbo.CheckIdentityColumns 'MyDatabase' --all
EXEC master.dbo.CheckIdentityColumns 'MyDatabase', 50 --columns 50% full or greater
EXEC master.dbo.CheckIdentityColumns 'MyDatabase', 50, 'int' --only int columns
Example output:
Table Column Type Percent Full Remaining
------------------------- ------------------ ------- ------------ ---------------
MyDatabase.dbo.Table1 Table1ID int 9 1,937,868,393
MyDatabase.dbo.Table2 Table2ID int 5 2,019,944,894
MyDatabase.dbo.Table3 Table3ID int 9 1,943,793,775
I created a reminder to check all my databases once per month, and I log this information in a spreadsheet.
CheckIdentityColumns Procedure
USE master
GO
CREATE PROCEDURE dbo.CheckIdentityColumns
(
#Database AS NVARCHAR(128),
#PercentFull AS TINYINT = 0,
#Type AS VARCHAR(8) = NULL
)
AS
--this procedure assumes you are not using negative numbers in your identity columns
DECLARE #Sql NVARCHAR(3000)
SET #Sql =
'USE ' + #Database + '
SELECT
[Column].TABLE_CATALOG + ''.'' +
[Column].TABLE_SCHEMA + ''.'' +
[Table].TABLE_NAME AS [Table],
[Column].COLUMN_NAME AS [Column],
[Column].DATA_TYPE AS [Type],
CAST((
CASE LOWER([Column].DATA_TYPE)
WHEN ''tinyint''
THEN (IDENT_CURRENT([Table].TABLE_NAME) / 255)
WHEN ''smallint''
THEN (IDENT_CURRENT([Table].TABLE_NAME) / 32767)
WHEN ''int''
THEN (IDENT_CURRENT([Table].TABLE_NAME) / 2147483647)
WHEN ''bigint''
THEN (IDENT_CURRENT([Table].TABLE_NAME) / 9223372036854775807)
WHEN ''decimal''
THEN (IDENT_CURRENT([Table].TABLE_NAME) / (([Column].NUMERIC_PRECISION * 10) - 1))
END * 100) AS INT) AS [Percent Full],
REPLACE(CONVERT(VARCHAR(19), CAST(
CASE LOWER([Column].DATA_TYPE)
WHEN ''tinyint''
THEN (255 - IDENT_CURRENT([Table].TABLE_NAME))
WHEN ''smallint''
THEN (32767 - IDENT_CURRENT([Table].TABLE_NAME))
WHEN ''int''
THEN (2147483647 - IDENT_CURRENT([Table].TABLE_NAME))
WHEN ''bigint''
THEN (9223372036854775807 - IDENT_CURRENT([Table].TABLE_NAME))
WHEN ''decimal''
THEN ((([Column].NUMERIC_PRECISION * 10) - 1) - IDENT_CURRENT([Table].TABLE_NAME))
END
AS MONEY) , 1), ''.00'', '''') AS Remaining
FROM
INFORMATION_SCHEMA.COLUMNS AS [Column]
INNER JOIN
INFORMATION_SCHEMA.TABLES AS [Table]
ON [Table].TABLE_NAME = [Column].TABLE_NAME
WHERE
COLUMNPROPERTY(
OBJECT_ID([Column].TABLE_NAME),
[Column].COLUMN_NAME, ''IsIdentity'') = 1 --true
AND [Table].TABLE_TYPE = ''Base Table''
AND [Table].TABLE_NAME NOT LIKE ''dt%''
AND [Table].TABLE_NAME NOT LIKE ''MS%''
AND [Table].TABLE_NAME NOT LIKE ''syncobj_%''
AND CAST(
(
CASE LOWER([Column].DATA_TYPE)
WHEN ''tinyint''
THEN (IDENT_CURRENT([Table].TABLE_NAME) / 255)
WHEN ''smallint''
THEN (IDENT_CURRENT([Table].TABLE_NAME) / 32767)
WHEN ''int''
THEN (IDENT_CURRENT([Table].TABLE_NAME) / 2147483647)
WHEN ''bigint''
THEN (IDENT_CURRENT([Table].TABLE_NAME) / 9223372036854775807)
WHEN ''decimal''
THEN (IDENT_CURRENT([Table].TABLE_NAME) / (([Column].NUMERIC_PRECISION * 10) - 1))
END * 100
) AS INT) >= ' + CAST(#PercentFull AS VARCHAR(4))
IF (#Type IS NOT NULL)
SET #Sql = #Sql + 'AND LOWER([Column].DATA_TYPE) = ''' + LOWER(#Type) + ''''
SET #Sql = #Sql + '
ORDER BY
[Column].TABLE_CATALOG + ''.'' +
[Column].TABLE_SCHEMA + ''.'' +
[Table].TABLE_NAME,
[Column].COLUMN_NAME'
EXECUTE sp_executesql #Sql
GO
Keith Walton has a very comprehensive query that is very good. Here's a little simpler one that is based on the assumption that the identity columns are all integers:
SELECT sys.tables.name AS [Table Name],
last_value AS [Last Value],
MAX_LENGTH,
CAST(cast(last_value as int) / 2147483647.0 * 100.0 AS DECIMAL(5,2))
AS [Percentage of ID's Used],
2147483647 - cast(last_value as int) AS Remaining
FROM sys.identity_columns
INNER JOIN sys.tables
ON sys.identity_columns.object_id = sys.tables.object_id
ORDER BY last_value DESC
The results will look like this:
Table Name Last Value MAX_LENGTH Percentage of ID's Used Remaining
My_Table 49181800 4 2.29 2098301847
Checking Integer Identity Columns
While crafting a solution for this problem, we found this thread both informative and interesting (we also wrote a detailed post about this and described how our tool works).
In our solution we're querying the information_schema to acquire a list of
all columns. Then we wrote a program that would go through each of them and compute the maximum and minimum (we account for both overflow and underflow).
SELECT
b.COLUMN_NAME,
b.COLUMN_TYPE,
b.DATA_TYPE,
b.signed,
a.TABLE_NAME,
a.TABLE_SCHEMA
FROM (
-- get all tables
SELECT
TABLE_NAME, TABLE_SCHEMA
FROM information_schema.tables
WHERE
TABLE_TYPE IN ('BASE TABLE', 'VIEW') AND
TABLE_SCHEMA NOT IN ('mysql', 'performance_schema')
) a
JOIN (
-- get information about columns types
SELECT
TABLE_NAME,
COLUMN_NAME,
COLUMN_TYPE,
TABLE_SCHEMA,
DATA_TYPE,
(!(LOWER(COLUMN_TYPE) REGEXP '.*unsigned.*')) AS signed
FROM information_schema.columns
) b ON a.TABLE_NAME = b.TABLE_NAME AND a.TABLE_SCHEMA = b.TABLE_SCHEMA
ORDER BY a.TABLE_SCHEMA DESC;

Resources