Is there any utility availble to count the total lines of user created Stored Procedure, Function, Views in a Database?
For SQL Server 2005 and 2008.
This includes all code including blank lines and trailing blank lines, but not the last line (no CRLF). So it's averages out... but it would always be an approximation anyway.
WITH CRLF AS
(
SELECT
CHARINDEX('
', definition) AS CRLF,
SM.[object_ID]
FROM
sys.sql_modules SM
WHERE
OBJECT_NAME([object_ID]) not in ('fn_diagramobjects', 'sp_alterdiagram', 'sp_creatediagram', 'sp_dropdiagram', 'sp_helpdiagramdefinition', 'sp_helpdiagrams', 'sp_renamediagram', 'sp_upgraddiagrams', 'sysdiagrams')
UNION ALL
SELECT
CHARINDEX('
', definition, C.CRLF + 2),
SM.[object_ID]
FROM
sys.sql_modules SM
JOIN
CRLF C ON SM.[object_ID] = C.[object_ID]
WHERE
CHARINDEX('
', definition, C.CRLF + 2) > C.CRLF
)
SELECT
COUNT(*)
FROM
CRLF
OPTION
(MAXRECURSION 0)
Edit:
You may need OBJECTPROPERTY(SM.[object_ID], 'IsMSShipped') = 0 or explicitly exclusions for diagram code etc
Edit 2:
From other solution in otehr answer, corrected to not give "-1" for check constraints and apply same filters/types
select t.sp_name, sum(t.lines_of_code) as lines_ofcode, t.type_desc
from
(
select o.name as sp_name,
(len(c.text) - len(replace(c.text, char(13), ''))) as lines_of_code,
case when o.xtype = 'P' then 'Stored Procedure'
when o.xtype in ('FN', 'IF', 'TF') then 'Function'
end as type_desc
from sysobjects o
inner join syscomments c
on c.id = o.id
where --o.xtype in ('V', 'P', 'FN', 'IF', 'TF', 'TR')
--and
o.category = 0
AND
o.name not in ('fn_diagramobjects', 'sp_alterdiagram', 'sp_creatediagram', 'sp_dropdiagram', 'sp_helpdiagramdefinition', 'sp_helpdiagrams', 'sp_renamediagram', 'sp_upgraddiagrams', 'sysdiagrams')
) t
group by t.sp_name, t.type_desc
order by 1
COMPUTE SUM (sum(t.lines_of_code))
They all give the same results here on several databases. eg 4607 for a SQL Server 2005 SP2 ReportServer database...
Not that I know of, but you could look through the stuff in sysobjects and execute sp_helptext on each proc and view and count the newlines.
If you want a non CTE based solution you could do something like this:
select sum(newlines) from
(
select newlines = (datalength(definition) - datalength(replace(definition, '
', ' '))) / 2 from sys.sql_modules
) as a
Related
I have to search the occurrence of a particular string inside all available stored procedures in SQL Server. I know that we can get this by using the below query.
SELECT OBJECT_NAME(OBJECT_ID) PrcName
FROM sys.sql_modules
WHERE DEFINITION LIKE '%SearchStr%'
But is there a way we can find out how many times the particular string is available in each stored procedure? This is for estimating the effort modifying the stored procedures.
Any help will be much appreciated.
This will work as tested:
;WITH cte as
(
SELECT OBJECT_NAME(OBJECT_ID) PrcName, OBJECT_ID
FROM sys.sql_modules
WHERE DEFINITION LIKE '%tblNDT%')
select t1.PrcName, (LEN(Definition) - LEN(replace(Definition,'tblNDT',''))) / LEN('tblNDT') Cnt
from cte t1
INNER JOIN sys.sql_modules t2 on t1.object_id = t2.object_id
An easy way of checking how many times something occurs is to take the initial length, replace your string with blanks, recheck the length, and divide by the length of your string:
DECLARE #sentence VARCHAR(100)
DECLARE #word VARCHAR(100)
SET #word = 'Cool'
SET #sentence = 'This cool sentence is really cool. Cool!'
DECLARE #wordlen INT = (SELECT LEN(#word))
--Original sentence and length
SELECT #sentence AS setencelen
SELECT LEN(#sentence) AS origsentence
--With word removed
SELECT REPLACE(#sentence, 'cool', '') AS shortenedsentence
SELECT LEN(REPLACE(#sentence, 'cool', '')) AS shortenedlen
SELECT LEN(#sentence) - LEN(REPLACE(#sentence, 'cool', '')) AS diffinlength
SELECT (LEN(#sentence) - LEN(REPLACE(#sentence, 'cool', ''))) / #wordlen AS occurrences
I have seen this work in some cases and not in others. If you have a bunch of comments that contain the same string, it will count incorrectly.
I have found a solution for this.
DECLARE #cnt AS INT= 1
DECLARE #SearchStr VARCHAR(MAX) = 'SearchText'
;WITH CTE_SearchStr1
AS
(
SELECT #cnt Cnt, #SearchStr SearchStr UNION ALL
SELECT Cnt + 1, #SearchStr+'%'+SearchStr FROM CTE_SearchStr1
),
CTE_SearchStr2
AS
(
SELECT TOP 100 * FROM CTE_SearchStr1
)
SELECT OBJECT_NAME(OBJECT_ID) ObjectName, MAX(cnt) cnt FROM sys.sql_modules a INNER JOIN CTE_SearchStr2 b ON
a.definition LIKE '%'+b.SearchStr+'%'
GROUP BY OBJECT_NAME(OBJECT_ID) ORDER BY 2 DESC
Only problem with the above query is that I can not search for more that 100 times. It will throw the below exception
Msg 530, Level 16, State 1, Line 3 The statement terminated. The
maximum recursion 100 has been exhausted before statement completion.
In my scenario, the number of occurrences are less than 100, but is there a way to overcome this error?
I'm using SQL Server 2014 and I need to find all stored procedures that use a Left(Column, 5) function
For example, it should find
left(jobno, 5)
but not
left(jobno, 50)
it should find
left(ZIP, 5)
I initially thought I could use
Like '%Left(%,5)%'
but it returns no records
Thanks
mark
This is a bit costly approach, but is serves the purpose. It will work when Length to LEFT function is specified as Number, but fails when Length is specified using variable or some other expressions. Hope this helps.
DECLARE #Search VARCHAR(10) = '5'
;WITH CTE AS
(
SELECT 1 AS Lvl, o.name, o.type, o.Object_ID
,NULLIF(CHARINDEX('left(',definition),0) AS LeftPos
,CHARINDEX(')', definition, (NULLIF(CHARINDEX('left(',definition),0))) AS LastIndex
,LTRIM(RTRIM(SUBSTRING(definition, CHARINDEX(',', definition, (NULLIF(CHARINDEX('left(',definition),0))) + 1, CHARINDEX(')', definition, (NULLIF(CHARINDEX('left(',definition),0))) - CHARINDEX(',', definition, (NULLIF(CHARINDEX('left(',definition),0))) -1 ))) AS LeftLen
FROM sys.all_sql_modules m
INNER JOIN sys.objects o on o.Object_ID = m.Object_ID
WHERE CHARINDEX(')', definition, (NULLIF(CHARINDEX('left(',definition),0))) - CHARINDEX(',', definition, (NULLIF(CHARINDEX('left(',definition),0))) -1 > 0
UNION ALL -- get all occurunces or Left
SELECT 2 AS Lvl, o.name, o.type, o.Object_ID
,NULLIF(CHARINDEX('left(',definition,LastIndex),0) AS LeftPos
,CHARINDEX(')', definition, (NULLIF(CHARINDEX('left(',definition,LastIndex),0))) AS LastIndex
,LTRIM(RTRIM(SUBSTRING(definition, CHARINDEX(',', definition, (NULLIF(CHARINDEX('left(',definition,LastIndex),0))) + 1, CHARINDEX(')', definition, (NULLIF(CHARINDEX('left(',definition,LastIndex),0))) - CHARINDEX(',', definition, (NULLIF(CHARINDEX('left(',definition,LastIndex),0))) -1 ))) AS LeftLen
FROM sys.all_sql_modules m
INNER JOIN sys.objects o ON o.Object_ID = m.Object_ID
INNER JOIN CTE c ON c.Object_ID = m.Object_ID
AND c.LastIndex IS NOT NULL AND ISNULL(c.LeftLen,#Search) <> #Search
WHERE CHARINDEX(')', definition, (NULLIF(CHARINDEX('left(',definition,LastIndex),0))) - CHARINDEX(',', definition, (NULLIF(CHARINDEX('left(',definition,LastIndex),0))) -1 > 0
)
SELECT * FROM CTE
WHERE LeftLen = #Search
OPTION (MAXRECURSION 10000);
SQL unfortunately cannot have wild cards in the middle of a string. Try this
(column name) like '%left(%' and (column name) like '%,5)%'
That should give you the desired result
This is one of the reasons why we've implemented regex using the SQL CLR. However, you can do a couple other things to solve your issue.
You could do something like this ...
select *
from sys.objects
where object_definition(object_id) like '%left(%,5)%';
Or something like this, which seems a bit cleaner, however, sometimes I use the above method, because I need some other stuff that spans from sys.objects ...
select *
from sys.sql_modules
where definition like '%left(%,5)%';
Another alternative is to come up with a list of possible column names. If you don't have any cross database worries, then you may want to use sys.columns, sys.all_columns or even sys.parameters as your pick list. To do this, you'd do something like this ...
select *
from sys.sql_modules m
cross join (
select distinct name
from sys.all_columns) c
where definition like '%left(' + c.name + ',5)%';
One other thing to consider is whitespace, which, again, is another reason to implement regex using SQL CLR. For instance, in the above case, somebody may put a space between the "," after the column name and the number "5". But, with that being said, the above queries should get you really close.
Hope this helps.
You could use something like
LIKE '%LEFT([a-z],5)%'
Can you give me some pointers (or point in the right direction on what search terms for google)? In a stored procedure I have a parameter #TAG (string). I receive '(14038314,14040071)' (for example) from another application that cannot be altered. In the stored procedure, I need to split apart '(14038314,14040071)' to put quotes around each string value, rebuild it, strip out the outer quotes,strip out the parens and pass it to #TAG in the query below so that it looks like the line commented out below?
SELECT
V.NAME AS VARIETY, TAGID
FROM
mfinv.dbo.onhand h
INNER JOIN
mfinv.dbo.onhand_tags t on h.onhand_id = t.onhand_id
INNER JOIN
mfinv.dbo.onhand_tag_details d on t.onhand_tag_id = d.onhand_tag_id
INNER JOIN
mfinv.dbo.FM_IC_PS_VARIETY V ON V.VARIETYIDX = d.VARIETYIDX
LEFT JOIN
mfinv.dbo.FM_IC_TAG TG ON TG.TAGIDX = t.TAGIDX
WHERE
h.onhand_id = (SELECT onhand_id FROM mfinv.dbo.onhand
WHERE onhand_id = IDENT_CURRENT('mfinv.dbo.onhand'))
AND TG.ID IN (#TAG)
--AND TG.ID IN ('14038314','14040071')
You can Use Dynamic SQL Like This
DECLARE #TAG Nvarchar(MAX)='14038314,14040071'
set #TAG=''''+REPLACE(#TAG,',',''',''')+''''
--select #TAG
DECLARE #SQL NVARCHAR(MAX)=N'
Select V.NAME AS VARIETY, TAGID
FROM mfinv.dbo.onhand h
INNER JOIN mfinv.dbo.onhand_tags t on h.onhand_id = t.onhand_id
INNER JOIN mfinv.dbo.onhand_tag_details d on t.onhand_tag_id = d.onhand_tag_id
INNER JOIN mfinv.dbo.FM_IC_PS_VARIETY V ON V.VARIETYIDX = d.VARIETYIDX
LEFT JOIN mfinv.dbo.FM_IC_TAG TG ON TG.TAGIDX = t.TAGIDX
WHERE h.onhand_id = (SELECT onhand_id FROM mfinv.dbo.onhand
WHERE onhand_id = IDENT_CURRENT (''mfinv.dbo.onhand''))
AND TG.ID IN ('+#TAG+')'
PRINT #SQL
EXEC (#SQL)
Here's what I did. Thank you all for responding. Thanks to dasblinkenlight for answering "How to replace first and last character of column in sql server?". Thanks to SQLMenace for answering "How Do I Split a Delimited String in SQL Server Without Creating a Function?".
Here's how I removed parenthesis:
#Tag nvarchar(256)
SET #Tag = SUBSTRING(#Tag, 2, LEN(#Tag)-2)
Here's how I split and rebuilt #Tag:
AND TG.ID in
(
SELECT SUBSTRING(',' + #Tag + ',', Number + 1,
CHARINDEX(',', ',' + #Tag + ',', Number + 1) - Number -1)AS VALUE
FROM master..spt_values
WHERE Type = 'P'
AND Number <= LEN(',' + #Tag + ',') - 1
AND SUBSTRING(',' + #Tag + ',', Number, 1) = ','
)
We have a client site with a 50Gb SQL 2012 database on a server with 100+ Gb of RAM.
As the application is used, SQL server does a great job of caching the db into memory but the performance increase from the caching occurs the SECOND time a query is run, not the first.
To try to maximize cache hits the first time queries are run, we wrote a proc that iterates through every index of every table within the entire DB, running this:
SELECT * INTO #Cache
FROM ' + #tablename + ' WITH (INDEX (' + #indexname + '))'
In an attempt to force a big, ugly, contrived read for as much data as possible.
We have it scheduled to run every 15 minutes, and it does a great job in general.
Without debating other bottlenecks, hardware specs, query plans, or query optimization, does anybody have any better ideas about how to accomplish this same task?
UPDATE
Thanks for the suggestions. Removed the "INTO #Cache". Tested & it didn't make a difference on filling the buffer.
Added: Instead of Select *, I'm selecting ONLY the keys from the Index. This (obviously) is more to-the-point and is much faster.
Added: Read & Cache Constraint Indexes also.
Here's the current code: (hope it's useful for somebody else)
CREATE VIEW _IndexView
as
-- Easy way to access sysobject and sysindex data
SELECT
so.name as tablename,
si.name as indexname,
CASE si.indid WHEN 1 THEN 1 ELSE 0 END as isClustered,
CASE WHEN (si.status & 2)<>0 then 1 else 0 end as isUnique,
dbo._GetIndexKeys(so.name, si.indid) as Keys,
CONVERT(bit,CASE WHEN EXISTS (SELECT * FROM sysconstraints sc WHERE object_name(sc.constid) = si.name) THEN 1 ELSE 0 END) as IsConstraintIndex
FROM sysobjects so
INNER JOIN sysindexes si ON so.id = si.id
WHERE (so.xtype = 'U')--User Table
AND ((si.status & 64) = 0) --Not statistics index
AND ( (si.indid = 0) AND (so.name <> si.name) --not a default clustered index
OR
(si.indid > 0)
)
AND si.indid <> 255 --is not a system index placeholder
UNION
SELECT
so.name as tablename,
si.name as indexname,
CASE si.indid WHEN 1 THEN 1 ELSE 0 END as isClustered,
CASE WHEN (si.status & 2)<>0 then 1 else 0 end as isUnique,
dbo._GetIndexKeys(so.name, si.indid) as Keys,
CONVERT(bit,0) as IsConstraintIndex
FROM sysobjects so
INNER JOIN sysindexes si ON so.id = si.id
WHERE (so.xtype = 'V')--View
AND ((si.status & 64) = 0) --Not statistics index
GO
CREATE PROCEDURE _CacheTableToSQLMemory
#tablename varchar(100)
AS
BEGIN
DECLARE #indexname varchar(100)
DECLARE #xtype varchar(10)
DECLARE #SQL varchar(MAX)
DECLARE #keys varchar(1000)
DECLARE #cur CURSOR
SET #cur = CURSOR FOR
SELECT v.IndexName, so.xtype, v.keys
FROM _IndexView v
INNER JOIN sysobjects so ON so.name = v.tablename
WHERE tablename = #tablename
PRINT 'Caching Table ' + #Tablename
OPEN #cur
FETCH NEXT FROM #cur INTO #indexname, #xtype, #keys
WHILE (##FETCH_STATUS = 0)
BEGIN
PRINT ' Index ' + #indexname
--BEGIN TRAN
IF #xtype = 'V'
SET #SQL = 'SELECT ' + #keys + ' FROM ' + #tablename + ' WITH (noexpand, INDEX (' + #indexname + '))' --
ELSE
SET #SQL = 'SELECT ' + #keys + ' FROM ' + #tablename + ' WITH (INDEX (' + #indexname + '))' --
EXEC(#SQL)
--ROLLBACK TRAN
FETCH NEXT FROM #cur INTO #indexname, #xtype, #keys
END
CLOSE #cur
DEALLOCATE #cur
END
GO
First of all, there is a setting called "Minumum Server Memory" that looks tempting. Ignore it. From MSDN:
The amount of memory acquired by the Database Engine is entirely dependent on the workload placed on the instance. A SQL Server instance that is not processing many requests may never reach min server memory.
This tells us that setting a larger minimum memory won't force or encourage any pre-caching. You may have other reasons to set this, but pre-filling the buffer pool isn't one of them.
So what can you do to pre-load data? It's easy. Just set up an agent job to do a select * from every table. You can schedule it to "Start automatically when Sql Agent Starts". In other words, what you're already doing is pretty close to the standard way to handle this.
However, I do need to suggest three changes:
Don't try to use a temporary table. Just select from the table. You don't need to do anything with the results to get Sql Server to load your buffer pool: all you need to do is the select. A temporary table could force sql server to copy the data from the buffer pool after loading... you'd end up (briefly) storing things twice.
Don't run this every 15 minutes. Just run it once at startup, and then leave it alone. Once allocated, it takes a lot to get Sql Server to release memory. It's just not needed to re-run this over and over.
Don't try to hint an index. Hints are just that: hints. Sql Server is free to ignore those hints, and it will do so for queries that have no clear use for the index. The best way to make sure the index is pre-loaded is to construct a query that obviously uses that index. One specific suggestion here is to order the results in the same order as the index. This will often help Sql Server use that index, because then it can "walk the index" to produce the results.
This is not an answer, but to supplement Joel Coehoorn's answer, you can look at the table data in the cache using this statement. Use this to determine whether all the pages are staying in the cache as you'd expect:
USE DBMaint
GO
SELECT COUNT(1) AS cached_pages_count, SUM(s.used_page_count)/COUNT(1) AS total_page_count,
name AS BaseTableName, IndexName,
IndexTypeDesc
FROM sys.dm_os_buffer_descriptors AS bd
INNER JOIN
(
SELECT s_obj.name, s_obj.index_id,
s_obj.allocation_unit_id, s_obj.OBJECT_ID,
i.name IndexName, i.type_desc IndexTypeDesc
FROM
(
SELECT OBJECT_NAME(OBJECT_ID) AS name,
index_id ,allocation_unit_id, OBJECT_ID
FROM sys.allocation_units AS au
INNER JOIN sys.partitions AS p
ON au.container_id = p.hobt_id
AND (au.type = 1 OR au.type = 3)
UNION ALL
SELECT OBJECT_NAME(OBJECT_ID) AS name,
index_id, allocation_unit_id, OBJECT_ID
FROM sys.allocation_units AS au
INNER JOIN sys.partitions AS p
ON au.container_id = p.partition_id
AND au.type = 2
) AS s_obj
LEFT JOIN sys.indexes i ON i.index_id = s_obj.index_id
AND i.OBJECT_ID = s_obj.OBJECT_ID ) AS obj
ON bd.allocation_unit_id = obj.allocation_unit_id
INNER JOIN sys.dm_db_partition_stats s ON s.index_id = obj.index_id AND s.object_id = obj.object_ID
WHERE database_id = DB_ID()
GROUP BY name, obj.index_id, IndexName, IndexTypeDesc
ORDER BY obj.name;
GO
Use this to replace function dbo._GetIndexKeys
(SELECT STRING_AGG(COL_NAME(ic.object_id,ic.column_id), ',') FROM sys.index_columns ic WHERE ic.object_id = so.id AND ic.index_id = si.indid) AS Keys,
--dbo._GetIndexKeys(so.name, si.indid) as Keys,
Here the code from my function:
-- Get Country Names
DECLARE getCountriesName CURSOR FOR
SELECT c.Name
FROM VocabCountry c
RIGHT OUTER JOIN ProjectCountryRelations cr
ON cr.CountryId = c.Id
WHERE
c.Id = #project_id;
-- Finally Create list to return
OPEN getCountriesName;
FETCH NEXT FROM getCountriesName INTO #Name;
WHILE ##FETCH_STATUS = 0
BEGIN
SET #listOfItems = #listOfItems + #Name + ', '
FETCH NEXT FROM getCountriesName INTO #Name;
END;
CLOSE getCountriesName;
DEALLOCATE getCountriesName;
I am expecting a list of comma separated values, for example, like so:
Canada, United States of America,
I verified that the SELECT returns the countries expected.
Thanks for any help!
Eric
If you're on SQL Server 2008 or newer, you could easily do this with a single SELECT and some FOR XML PATH magic :-)
SELECT
STUFF(
(SELECT ',' + c.Name
FROM VocabCountry c
RIGHT OUTER JOIN ProjectCountryRelations cr
ON cr.CountryId = c.Id
FOR XML PATH('')
), 1, 1, '')
That should return a comma-separated list of those country names for you. No ugly and awful cursor needed!
Using the FOR XML PATH will concat the values with commas.
Using the STUFF Function will remove the first comma you don't want.
So use this:
SELECT
STUFF(
(SELECT ',' + c.Name
FROM VocabCountry c
RIGHT OUTER JOIN ProjectCountryRelations cr
ON cr.CountryId = c.Id
WHERE cr.Id = #project_id
FOR XML PATH('')
), 1, 1, '');
See this fiddle: http://sqlfiddle.com/#!3/fd648/21