Is there a way to query the current status (executing, idle, etc) and the last result (successfull, failed, etc), and the last run time for a specific job name? The end result I am looking for is being able to display this information in an internal web application for various SSIS packages.
You should be able to find this information inMSDB - there are tables sysjobs, sysjobhistory and sysjobsteps which give the information that you are looking for
exec msdb.dbo.sp_help_job #job_name = 'TheJobName'
gives the information I want. So then I can just use a SqlDataReader to get the information. Note that this stored procedure returns multiple result sets.
The micrsoft documentation on this store procedure is
http://msdn.microsoft.com/en-us/library/ms186722(SQL.90).aspx
Another solution I have used is to update a reference table with the current status. It's quick and easy and usually very easy to retrieve the values you need.
For example, as soon as a package kicks off, insert a record with date and time, package name, etc.
Related
I constantly run into this problem. I am working in a data warehouse and I cannot find out what is populating a table. Typically the table is being populated on a daily basis from either other table in the warehouse or from an Oracle database. I have tried the below query and can confirm the updates, but i cannot see what is doing it. I searched to the known SSIS package and stored procedure with similar names and SQL jobs but I can find nothing.
select object_name(object_id) as DatabaseName, last_user_update, *
from sys.dm_db_index_usage_stats
where database_id = DB_ID('Warehouse')
and object_id=object_id('PAYMENTS_DAILY')
I only have the most basic SQL Server tools available so no fancy search tools :(
There is no way to tell, after data has been inserted into a data, where the data came from without having some sort of logging.
SSIS has logging, you can use triggers on the tables, change data capture, audit columns, etc. are the many ways to do this.
Frequently, if you know when the row was added, that can help you figure out what process is adding it. Add a new "InsertedDatetime" column to your warehouse table and give it a default value of getdate(). If you know that the rows always come in at 11:15 AM, you can use that to narrow your search.
That will probably be enough information, but if that doesn't help you track down the process, then you can add additional columns that contain everything from a source IP address to a calling object name.
As a last resort, you could rename your table and create a view named the same and then use an Instead Of Insert trigger on it that just holds open the connection so you can examine the currently executing processes to figure out where it's coming from.
I bet you can figure it out from the time alone though.
I am trying to hunt down a certain stored procedure which writes to certain table (it needs to be changed) however going through every single stored procedure is not a route I really want to take. So I was hoping there might be a way to find out which stored procedures INSERT or UPDATE certain table.
I have tried using this method (pinal_daves_blog), but it is not giving me any results.
NOTICE: The stored procedure might not be in the same DB!
Is there another way or can I somehow check what procedure/function has made the last insert or update to table.
One brute-force method would be to download an add-in from RedGate called SQL Search (free), then do a stored procedure search for the table name. I'm not affiliated at all with RedGate or anything, this is just a method that I have used to find similar things and has served me well.
http://www.red-gate.com/products/sql-development/sql-search/
If you go this route, you just type in the table name, change the 'object types' ddl selection to 'Procedures' and select 'All databases' in the DB ddl.
Hope this helps! I know it isn't the most technical solution, but it should work.
There is no built-in way to tell what function, procedure, or executed batch has made the last change to a table. There just isn't. Some databases have this as part of their transaction logging but SQL Server isn't one of them.
I have wondered in the past whether transactional replication might provide that information, if you already have that set up, but I don't know whether that's true.
If you know the change has to be taking place in a stored procedure (as opposed to someone using SSMS or executing lines of SQL via ADO.NET), then #koppinjo's suggestion is a good one, as is this one from Pinal Dave's blog:
USE AdventureWorks
GO
--Searching for Empoloyee table
SELECT Name
FROM sys.procedures
WHERE OBJECT_DEFINITION(OBJECT_ID) LIKE '%Employee%'
There are also dependency functions, though they can be outdated or incomplete:
select * from sys.dm_sql_referencing_entities( 'dbo.Employee', 'object' )
You could run a trace in Profiler. The procedure would have to write to the table while the trace is running for you to catch it.
I have set up a stored procedure tracker table on our databases with the hope of using it to flush out procedures that we no longer use. I set this up a few months ago, and am now ready to be able to start the cleansing. The tables utilises the sys.procedures and sys.dm_exec_procedure_stats DMVs in SQL Server 2008 R2, and a job updates the static table every 10 minutes, 24hours a day
I have been checking through my list of procedures, and have come across a couple that I know for a fact have run very recently. The particular one I have found runs as step 2 of a job, but the sys.dm_exec_procedure_stats doesn't seem to contain any record of it having been run, but the procedure in step 1 has appeared at the correct time. I have checked the job history, and both steps 1 and 2 ran without any problems.
The only difference I can see is that the procedure in step 2 comes up with a "Warning: Null value is eliminated by an aggregate or other SET operation" whereas step one doesn't. Does this make a difference as to whether or not the procedure will appear in the sys.dm_exec_procedure_stats?
Hope someone can help!
While the reason that it doesn't show up in the DMV is likely the reason specified in the linked/related answer mentioned by #bastos.sergio in a comment on the question, that still leaves the issue of "what can be done to find procs not being used?".
The accepted answer in that linked question (this is the question referenced by #bastos.sergio: Last Run Date on a Stored Procedure in SQL Server ) is missing something so I will add to it here:
The ONLY way to know what is calling it is:
scan all code (app code, other Stored Procs, Job Steps [in msdb.dbo.sysjobsteps], SSRS report definition files, etc.) for references
IF you allow ad hoc access (e.g. someone referenced a Stored Proc in an Access app [or any Microsoft Office "app"]) then you need to do some of the additional steps mentioned in the accepted answer of that linked question, namely:
Add a RAISERROR(N'Deprecated! Please contact YourName.', 16, 1); RETURN; at the top of the proc and leave it there for a month or two.
Add a table to log proc calls and an INSERT into that log table at the top of any of the supposed obsolete code and check once a week to see if anything shows up. If also doing the RAISERROR, put the INSERT prior to the RAISERROR(...); RETURN;.
With regards to ad hoc access (i.e. access outside of the code that you control), be careful to always keep in mind that infrequent access can be just that: infrequent. If there is a code path that is executed monthly, quarterly, bi-annually, annually, when some manager remembers to ask for such and such report, etc., then you could potentially remove valid code if you do not allow a long-enough time frame to capture "highly" infrequent usage (and this is why, even if the DMV data was more reliable, you would still need to be just as cautious).
Again, if all access is within code that you control, just scan your code (most likely using Regular Expressions).
EDIT:
To answer the specific question of:
Does the "Warning: Null value is eliminated by an aggregate or other SET operation" warning that the query, running in the Stored Proc that does not show up in the DMV, gets have something to do with why it is not showing up in the DMV?
do the following test:
CREATE PROCEDURE #NoWarning
AS
SELECT AVG(tmp.col)
FROM (
SELECT 1.0
UNION ALL
SELECT 2
) tmp(col);
GO
EXEC #NoWarning;
GO
CREATE PROCEDURE #Warning
AS
SELECT AVG(tmp.col)
FROM (
SELECT 1.0
UNION ALL
SELECT null
) tmp(col);
GO
EXEC #Warning;
And then run the following query and you should see both proc names appearing in "tempdb":
SELECT DB_NAME(ps.database_id) AS [DatabaseName],
OBJECT_NAME(ps.[object_id], ps.database_id) AS [ProcName],
*
FROM sys.dm_exec_procedure_stats ps
ORDER BY [DatabaseName], [ProcName];
About 5 times a year one of our most critical tables has a specific column where all the values are replaced with NULL. We have run log explorers against this and we cannot see any login/hostname populated with the update, we can just see that the records were changed. We have searched all of our sprocs, functions, etc. for any update statement that touches this table on all databases on our server. The table does have a foreign key constraint on this column. It is an integer value that is established during an update, but the update is identity key specific. There is also an index on this field. Any suggestions on what could be causing this outside of a t-sql update statement?
I would start by denying any client side dynamic SQL if at all possible. It is much easier to audit stored procedures to make sure they execute the correct sql including a proper where clause. Unless your sql server is terribly broken, they only way data is updated is because of the sql you are running against it.
All stored procs, scripts, etc. should be audited before being allowed to run.
If you don't have the mojo to enforce no dynamic client sql, add application logging that captures each client sql before it is executed. Personally, I would have the logging routine throw an exception (after logging it) when a where clause is missing, but at a minimum, you should be able to figure out where data gets blown out next time by reviewing the log. Make sure your log captures enough information that you can trace it back to the exact source. Assign a unique "name" to each possible dynamic sql statement executed, e.g., each assign a 3 char code to each program, and then number each possible call 1..nn in your program so you can tell which call blew up your data at "abc123" as well as the exact sql that was defective.
ADDED COMMENT
Thought of this later. You might be able to add / modify the update trigger on the sql table to look at the number of rows update prevent the update if the number of rows exceeds a threshhold that makes sense for your. So, did a little searching and found someone wrote an article on this already as in this snippet
CREATE TRIGGER [Purchasing].[uPreventWholeUpdate]
ON [Purchasing].[VendorContact]
FOR UPDATE AS
BEGIN
DECLARE #Count int
SET #Count = ##ROWCOUNT;
IF #Count >= (SELECT SUM(row_count)
FROM sys.dm_db_partition_stats
WHERE OBJECT_ID = OBJECT_ID('Purchasing.VendorContact' )
AND index_id = 1)
BEGIN
RAISERROR('Cannot update all rows',16,1)
ROLLBACK TRANSACTION
RETURN;
END
END
Though this is not really the right fix, if you log this appropriately, I bet you can figure out what tried to screw up your data and fix it.
Best of luck
Transaction log explorer should be able to see who executed command, when, and how specifically command looks like.
Which log explorer do you use? If you are using ApexSQL Log you need to enable connection monitor feature in order to capture additional login details.
This might be like using a sledgehammer to drive in a thumb tack, but have you considered using SQL Server Auditing (provided you are using SQL Server Enterprise 2008 or greater)?
With SMO objects using Server.JobServer.jobs to get a list of jobs, I can find the status of each job. For those that are currently executing I would like to find the SPID it is executing on. I can also get a list of the server's processes using Server.EnumProcesses(). This gives me a list of currently active SPIDs. I want to match the two.
The best I've been able to come up with is to convert the jobid to a string and substring the jobId out of the program string in the EnumProcesses table (which, at least on my system, embeds the jobId in this description). It's really ugly for a couple of reasons not the least of which is that the Guid in the program description and the guid for jobID have their bytes switched in the first 3 pieces of the string representation. Yuck.
Is there a better way to do that using SMO?
Using SMO, not that I know of.
I have done this using T/SQL. Take the job_id value and cast it to varbinary. That should then match the value in the application name (after the "SQL Agent" part).
Use SP_who2 , in the result "Program" field you will find SQLAgent-Jobid. you will get the jobID from sysjobs from MSDB table. now you can get the sp_id of the job