Related
sometimes I have completed the development of a solution on a development instance of a SQL database, after several test cycles, reworks, adjustments, etc.
When is time to move everything in production, I have to be sure that nothing is left behind by mistake (a trigger, a stored procedure).
One helper is to get a simple list of all the objects in the database.
A simple query to get the list of all the objects in your db is the following one:
select * from Sys.objects as o left join sys.schemas as s on o.schema_id = s.schema_id
where is_ms_shipped = 0
order by s.name, o.type, o.name
You can adjust it at your needs, remove fields, and so on.
Few notes:
I have excluded items delivered by Microsoft (is_ms_shipped)
I have joined the main system table with a catalogue of all the objects with the description of the schema. This is helpful to order all the elements
Using SSMS (SQL Server Management Studio), you can easily export the results in Excel, and work on it, add it to the documentation.
I am deleting a column from one of the frequently used tables in my database.
Last time I did this errors started to crawl up from all sorts of stored procs that I had long forgotten existed; complaining about the missing column. (only when that stored proc was called)
so, I dont want to get winded up with these things again, I want to make sure all stored procs are free of that column before I actually delete it.
What is the best way to search through all stored procs (and I have quite a lot of them) and remove the reference to that column?
I tried to find an option in the menu to do this but I did not find anything very obvious to do this.
any help, (other than telling me to go through them all one by one) is appreciated.
ps: of course, doesnt mean that I will depreciate your comment if you do tell me. I will only downvote :P
(nah, just kidding!)
To add to the various TSQL solutions, there is a free tool from Red Gate that integrates into SSMS: SQL Search
Use this script. It will also return triggers. If many tables has column with the same name you can add tale name to the where too. This script works on MSSQL 2000, 2005. I haven't tested it on 2008, but it should work fine too.
SELECT o.name
FROM sysobjects o
INNER JOIN syscomments c ON o.id = c.id
WHERE c.text like '%column_name%'
Edit: If you want to filter it only to store procedures add AND type ='P' to the where clause
Red Gate Software's SQL Prompt 5 has a couple of new features that might be useful in this situation:
Column Dependencies: hover over a column name in a script and up pops a window containing a list of all the objects that use that column
Find Invalid Objects: show objects across the database that can't be used, often because they use columns that have been deleted
You can download a 14-day free trial to see if the tool would be useful for you.
Paul Stephenson
SQL Prompt Project Manager
Red Gate Software
You can use Dependence option for that table to find the Dependent object or list of Procedure or function which are depend on this table.
Use below script
sp_depends 'TableName'
another option is create script for that column containing but that will filter all the text in the procedure or function.
EDIT: sorry, my bad. here's the code for searching within the stored procedure's code
The following stored procedure should be able to list all the stored procedures whose text contain the desired string (so, place your column name in it and fire away):
CREATE PROCEDURE Find_Text_In_SP
#StringToSearch varchar(100)
AS
SET #StringToSearch = '%' +#StringToSearch + '%'
SELECT Distinct SO.Name
FROM sysobjects SO (NOLOCK)
INNER JOIN syscomments SC (NOLOCK) on SO.Id = SC.ID
AND SO.Type = 'P'
AND SC.Text LIKE #stringtosearch
ORDER BY SO.Name
GO
Usage:
exec Find_Text_In_SP 'desired_column_name'
Source here
If you use MS SQL later than version 2000, it's better to search sys.sql_modules rather than sys.syscomments, since syscomments only hold records of nvarchar(4000), and the text you are looking for may be split into two records.
So while you can use a query like this from MSDN
SELECT sm.object_id, OBJECT_NAME(sm.object_id) AS object_name, o.type, o.type_desc, sm.definition
FROM sys.sql_modules AS sm
JOIN sys.objects AS o ON sm.object_id = o.object_id
WHERE sm.definition like '%' + #ColumnName + '%'
ORDER BY o.type;
you should be aware that this search finds any procedure containing that text, regardless of whether it is an actual column name and which table the column belongs to.
Is there an easier way of cleaning up a database that has a ton of stored procedures that I'm sure there are some that aren't used anymore other than one by one search.
I'd like to search my visual studio solution for stored procedures and see which ones from the database aren't used any longer.
You could create a list of the stored procedures in the database. Store them into a file.
SELECT *
FROM sys.procedures;
Then read that file into memory and run a large regex search across the source files for each entry. Once you hit the first match for a given stored procedure, you can move on.
If there are no matches for a given stored procedure, you probably can look more closely at that stored procedure and may even be able to remove it.
I'd be careful removing stored procedures - you also need to check that no other stored procedures depend on your candidate for removal!
I would use the profiler and set up a trace. The big problem is tracking SPs which are only used monthly or annually.
Anything not showing up in the trace can be investigated. I sometimes instrument individual SPs to log their invocations to a table and then review the table for activity. I've even had individual SPs instrumented to send me email when they are called.
It is relatively easy to ensure that an SP is not called from anywhere else in the server by searching the INFORAMTION_SCHEMA.ROUTINES or in source code with GREP. It's a little harder to check in SSIS packages and jobs.
But none of this eliminates the possibility that there might be the occasional SP which someone calls manually each month from SSMS to correct a data anomaly or something.
Hmmm... you could search your solution for code which calls a stored proc, like this (from the DAAB)
using (DbCommand cmd = DB.GetStoredProcCommand("sp_blog_posts_get_by_title"))
{
DB.AddInParameter(cmd, "#title", DbType.String,title);
using (IDataReader rdr = DB.ExecuteReader(cmd))
result.Load(rdr);
}
Search for the relevant part of the first line:
DB.GetStoredProcCommand("
Copy the search results from the "find results" pane, and compare to your stored proc list in the database (which you can generate with a select from the sysObjects table if you're using SQL Server).
If you really want to get fancy, you could write a small app (or use GREP or similar) to perform a regex match against your .cs files to extract a list of stored procedures, sort the list, generate a list of stored procs from your database via select from sysobjects, and do a diff. That might be easier to automate.
UPDATE Alternatively, see this link. The author suggest setting up a trace for a period of a week or so and comparing your list of procs against those found in the trace. Another author suggested: (copied)
-- Unused tables & indexes. Tables have index_id’s of either 0 = Heap table or 1 = Clustered Index
SELECT OBJECTNAME = OBJECT_NAME(I.OBJECT_ID), INDEXNAME = I.NAME, I.INDEX_ID
FROM SYS.INDEXES AS I
INNER JOIN SYS.OBJECTS AS O
ON I.OBJECT_ID = O.OBJECT_ID
WHERE OBJECTPROPERTY(O.OBJECT_ID,'IsUserTable') = 1
AND I.INDEX_ID
NOT IN (SELECT S.INDEX_ID
FROM SYS.DM_DB_INDEX_USAGE_STATS AS S
WHERE S.OBJECT_ID = I.OBJECT_ID
AND I.INDEX_ID = S.INDEX_ID
AND DATABASE_ID = DB_ID(db_name()))
ORDER BY OBJECTNAME, I.INDEX_ID, INDEXNAME ASC
which should find objects that haven't been used since a specified date. Note that I haven't tried either of these approaches, but they seem reasonable.
We starting to get a lot of stored procedures in our application. Many of them are for custom reports many of which are no longer used. Does anyone know of a query we could run on the system views in SQL Server 2005 that would tell us the last date a stored procedure was executed?
The below code should do the trick (>= 2008)
SELECT o.name,
ps.last_execution_time
FROM sys.dm_exec_procedure_stats ps
INNER JOIN
sys.objects o
ON ps.object_id = o.object_id
WHERE DB_NAME(ps.database_id) = ''
ORDER BY
ps.last_execution_time DESC
Edit 1 : Please take note of Jeff Modens advice below. If you find a procedure here, you can be sure that it is accurate. If you do not then you just don't know - you cannot conclude it is not running.
In a nutshell, no.
However, there are "nice" things you can do.
Run a profiler trace with, say, the stored proc name
Add a line each proc (create a tabel of course)
"INSERT dbo.SPCall (What, When) VALUES (OBJECT_NAME(##PROCID), GETDATE()"
Extend 2 with duration too
There are "fun" things you can do:
Remove it, see who calls
Remove rights, see who calls
Add RAISERROR ('Warning: pwn3d: call admin', 16, 1), see who calls
Add WAITFOR DELAY '00:01:00', see who calls
You get the idea. The tried-and-tested "see who calls" method of IT support.
If the reports are Reporting Services, then you can mine the RS database for the report runs if you can match code to report DataSet.
You couldn't rely on DMVs anyway because they are reset om SQL Server restart.
Query cache/locks are transient and don't persist for any length of time.
Oh, be careful now! All that glitters is NOT gold! All of the “stats” dm views and functions have a problem for this type of thing. They only work against what is in cache and the lifetime of what is in cache can be measured in minutes. If you were to use such a thing to determine which SPs are candidates for being dropped, you could be in for a world of hurt when you delete SPs that were used just minutes ago.
The following excerpts are from Books Online for the given dm views…
sys.dm_exec_procedure_stats
Returns aggregate performance statistics for cached stored procedures. The view contains one row per stored procedure, and the lifetime of the row is as long as the stored procedure remains cached. When a stored procedure is removed from the cache, the corresponding row is eliminated from this view.
sys.dm_exec_query_stats
The view contains one row per query statement within the cached plan, and the lifetime of the rows are tied to the plan itself. When a plan is removed from the cache, the corresponding rows are eliminated from this view.
sys.dm_exec_procedure_stats contains the information about the execution functions, constraints and Procedures etc. But the life time of the row has a limit, The moment the execution plan is removed from the cache the entry will disappear.
Use [yourDatabaseName]
GO
SELECT
SCHEMA_NAME(sysobject.schema_id),
OBJECT_NAME(stats.object_id),
stats.last_execution_time
FROM
sys.dm_exec_procedure_stats stats
INNER JOIN sys.objects sysobject ON sysobject.object_id = stats.object_id
WHERE
sysobject.type = 'P'
ORDER BY
stats.last_execution_time DESC
This will give you the list of the procedures recently executed.
If you want to check if a perticular stored procedure executed recently
SELECT
SCHEMA_NAME(sysobject.schema_id),
OBJECT_NAME(stats.object_id),
stats.last_execution_time
FROM
sys.dm_exec_procedure_stats stats
INNER JOIN sys.objects sysobject ON sysobject.object_id = stats.object_id
WHERE
sysobject.type = 'P'
and (sysobject.object_id = object_id('schemaname.procedurename')
OR sysobject.name = 'procedurename')
ORDER BY
stats.last_execution_time DESC
If you enable Query Store on SQL Server 2016 or newer you can use the following query to get last SP execution. The history depends on the Query Store Configuration.
SELECT
ObjectName = '[' + s.name + '].[' + o.Name + ']'
, LastModificationDate = MAX(o.modify_date)
, LastExecutionTime = MAX(q.last_execution_time)
FROM sys.query_store_query q
INNER JOIN sys.objects o
ON q.object_id = o.object_id
INNER JOIN sys.schemas s
ON o.schema_id = s.schema_id
WHERE o.type IN ('P')
GROUP BY o.name , + s.name
This works fine on 2005 (if the plan is in the cache)
USE YourDb;
SELECT qt.[text] AS [SP Name],
qs.last_execution_time,
qs.execution_count AS [Execution Count]
FROM sys.dm_exec_query_stats AS qs
CROSS APPLY sys.dm_exec_sql_text(qs.sql_handle) AS qt
WHERE qt.dbid = DB_ID()
AND objectid = OBJECT_ID('YourProc')
I use this:
use YourDB;
SELECT
object_name(object_id),
last_execution_time,
last_elapsed_time,
execution_count
FROM
sys.dm_exec_procedure_stats ps
where
lower(object_name(object_id)) like 'Appl-Name%'
order by 1
We're having problem with a huge number of legacy stored procedures at work. Do you guys recommend any tool the can help better understand those procedures? Some kind of reverse engineering that indentifies inter-procedure dependencies and/or procedure vs. tables dependencies. Can be a free or commercial tool.
Thanks!
The cheaper solution than 'dependency tracker' is the data dictionary table sys.sql_dependencies which from which this data can be queried from the data dictionary. Oracle has a data dictionary view with similar functionality called DBA_DEPENDENCIES (plus equivalent USER_ and ALL_ views) . Using the other data dictionary tables (sys.tables/DBA_TABLES) etc. you can generate object dependency reports.
If you're feeling particularly keen you can use a recursive query (Oracle CONNECT BY or SQL Server Common Table Expressions) to build a complete object dependency graph.
Here's an example of a recursive CTE on sys.sql_dependencies. It will return an entry for every dependency with its depth. Items can occur more than once, possibly at different depths, for every dependency relationship. I don't have a working Oracle instance to hand to build a CONNECT BY query on DBA_DEPENDENCIES so anyone with edit privileges and the time and expertise is welcome to annotate or edit this answer.
Note also with sys.sql_dependencies that you can get column references from referenced_minor_id. This could be used (for example) to determine which columns were actually used in the ETL sprocs from a staging area with copies of the DB tables from the source with more columns than are actually used.
with dep_cte as (
select o2.object_id as parent_id
,o2.name as parent_name
,o1.object_id as child_id
,o1.name as child_name
,d.referenced_minor_id
,1 as hierarchy_level
from sys.sql_dependencies d
join sys.objects o1
on o1.object_id = d.referenced_major_id
join sys.objects o2
on o2.object_id = d.object_id
where d.referenced_minor_id in (0,1)
and not exists
(select 1
from sys.sql_dependencies d2
where d2.referenced_major_id = d.object_id)
union all
select o2.object_id as parent_id
,o2.name as parent_name
,o1.object_id as child_id
,o1.name as child_name
,d.referenced_minor_id
,d2.hierarchy_level + 1 as hierarchy_level
from sys.sql_dependencies d
join sys.objects o1
on o1.object_id = d.referenced_major_id
join sys.objects o2
on o2.object_id = d.object_id
join dep_cte d2
on d.object_id = d2.child_id
where d.referenced_minor_id in (0,1)
)
select *
from dep_cte
order by hierarchy_level
I've got this to open-up to the community now. Could someone with convenient access to a running Oracle instance post a CONNECT BY recursive query here? Note that this is SQL-server specific and the question owner has since made it clear that he's using Oracle. I don't have a running Oracle instance to hand to develop and test anything.
Redgate has a rather expensive product called SQL Dependency Tracker that seems to fulfill the requirements.
I think the Red Gate Dependency Tracker mentioned by rpetrich is a decent solution, it works well and Red Gate has 30 day trial (ideally long enough for you do do your forensics).
I would also consider isolating the system and running the SQL Profiler which will show you all the SQL action on the tables. This is often a good starting point for building a sequence diagram or however you choose to document these codes. Good luck!
Redgate SQL Doc. the generated documentation included cross-referenced dependency information. For example, for each table, it lists views, stored procedures, triggers etc that reference that table.
What database are the stored procedures in? Oracle, SQL Server, something else?
Edit based on comment: Given you're using Oracle then, have a look at TOAD. I use a feature in it called the Code Roadmap, which allows you to graphically display PL/SQL interdependancies within the database. It can run in Code Only mode, showing runtime call stack dependancies, or Code Plus Data mode, where it also shows you database objects (tables, views, triggers) that are touched by your code.
(Note - I am a TOAD user, and gain no benefit from referring it)
This isn't real deep or thorough, but I think that if you're using MS SQL Server or Oracle (Perhaps Nigel can help with a PL-SQL sample)...Nigel is on to something . This only goes 3 dependencies deep, but could be modified to go however deep you need. It's not the prettiest thing...but it's functional...
select
so.name + case when so.xtype='P' then ' (Stored Proc)' when so.xtype='U' then ' (Table)' when so.xtype='V' then ' (View)' else ' (Unknown)' end as EntityName,
so2.name + case when so2.xtype='P' then ' (Stored Proc)' when so2.xtype='U' then ' (Table)' when so2.xtype='V' then ' (View)' else ' (Unknown)' end as FirstDependancy,
so3.name + case when so3.xtype='P' then ' (Stored Proc)' when so3.xtype='U' then ' (Table)' when so3.xtype='V' then ' (View)' else ' (Unknown)' end as SecondDependancy,
so4.name + case when so4.xtype='P' then ' (Stored Proc)' when so4.xtype='U' then ' (Table)' when so4.xtype='V' then ' (View)' else ' (Unknown)' end as ThirdDependancy
from
sysdepends sd
inner join sysobjects as so on sd.id=so.id
left join sysobjects as so2 on sd.depid=so2.id
left join sysdepends as sd2 on so2.id=sd2.id and so2.xtype not in ('S','PK','D')
left join sysobjects as so3 on sd2.depid=so3.id and so3.xtype not in ('S','PK','D')
left join sysdepends as sd3 on so3.id=sd3.id and so3.xtype not in ('S','PK','D')
left join sysobjects as so4 on sd3.depid=so4.id and so4.xtype not in ('S','PK','D')
where so.xtype = 'P' and left(so.name,2)<>'dt'
group by so.name, so2.name, so3.name, so4.name, so.xtype, so2.xtype, so3.xtype, so4.xtype
How to find the dependency chain of a database object (MS SQL Server 2000(?)+)
by Jacob Sebastian
Every time he needs to deploy a new report or modify an existing
report, he needs to know what are the database objects that depend on
the given report stored procedure. Some times the reports are very
complex and each stored procedure might have dozens of dependent
objects and each dependent object may be depending on other dozens of
objects.
He needed a way to recursively find all the depending objects of a
given stored procedure. I wrote a recursive query using CTE to achieve
this.
The single best tool for reverse engineering is by APEX. Its amazing. It can even trace into .NET assemblies and tell you where the procs are used. Its by far the deepest product of its kind. RedGate has great other tools but not in this case.