I have a large number of stored procedures (about 200) that need to be executed sequentially. Ideally I wanted to create a single "master" stored procedure that would execute each of the individual stored procedures one after another.
However, when I execute the master stored procedure it consistently freezes after running a long time. That being said, if I take all the SQL code from the 200 individual stored procedures and create one giant SQL script file, it runs without any issue.
The SQL code queries separate tables and inserts a subset of the data into a master "summary" table.
Any ideas why this would happen? Is there something about stored procedures that take more memory? I would prefer to keep everything in stored procedures so we could manage security and updates easier.
Any ideas why this would happen?
Compilation.
The master script likely is compiled batch by batch using the statistics valid at this point.
The SP will be compiled once at start, and if the statistics change during the run - as is typial for a sequence of loads - there you go. Especially if the statistical change is significant during processing. Basically the stats at teh beginning - when things are compiled - are totally off compared to the runtime stats for some tables.
There is a recompile option that you can se tin the individual statements to avoid this.
Related
At work, I have come across several SQL Server stored procedures that are only used by a single job. In that case, wouldn't it just make more sense to run the code in a job step? Is there some benefit from running statements in stored procedures?
These specific stored procedures do not require input variables, nor are they commonly used calculations; they are mostly just complex select statements. Looking for advice on best practice and performance impact.
There should be no material performance difference.
Code in a stored procedure is stored in the user database, present in backups, owned by the database owner, and can be invoked and debugged from anywhere.
Code in a job step is stored in the MSDB system database and owned by the job owner and can only be run through Agent.
I have few tables (base tables) which are getting inserted and updated twice a week. I have indexes created on these tables long back.
I'm applying logic on top of these tables in a stored procedure (without any parameter) and creating a final output table.
I'm scheduling this stored procedure twice a week using SQL Server agent job.
It is running slowly now (50 minutes) whereas if I run the stored procedure manually, it is running faster (15 - 18 minutes)
Do I have to drop the indexes whenever insert or update is happening in base tables and recreate it again after the insert or update?
If so do I have to do it every week?
What is its effect in performance of SQL Server agent jobs?
Indexes do require maintenance, but the rate at which they do depends entirely on how much data is changed, and how those changes are ordered. You can google around for any number of scripts to check your index fragmentation, and how to defragment them. Usually even for larger databases, weekly or nightly maintenances are more than enough.
Anyway, the fact that the execution time differs depending on how you run it, points to two possible causes:
Parametrization, or the SET properties used by the connection.
If your procedure uses parameters but you run the script manually giving the parameters values as you do, then SQL Server knows exactly which values you're using, and can optimize the query execution to use the correct indexes etc on the spot. If your agent calls the procedure with the same parameters, then the process is different. SQL Server may not know which values are being used, so it has to use covering indexes or worse yet, even full on table scans (reading all the data in the whole table, rendering indexes useless) to make sure that it will find all the relevant data for the query. Google SQL Server parametrization, and you can find out more.
The set properties on the other hand control specific session properties that are applied automatically when you connect directly to the database via Management Studio. But when you use an agent job, that may not be the case. This can also result in a different plan which will take far more time.
Both these cases, depend on your database settings and the way your procedure works. So we have to guess here.
But typically, you need to set the following properties in the beginning of a script in an agent job to match the session properties used in your regular Management Studio session:
SET ANSI_NULLS ON;
GO
SET QUOTED_IDENTIFIER ON;
GO
All of the terms here can be googled. I suggest you do so. Those articles can explain these things far better than I've the time for, especially given that - no disrespect intended - you're relatively new to SQL Server. So explaining these things with a suitable terminology here, is difficult. :)
A few days ago I was asked this question(tel intvw) and I was drawn blank. I said execution plan is stored in the server so its called STORED Procedure. But I am not sure I was correct.
My research after that has shown that there is plan cache or procedure cache inside of SQL Server that's dedicated to storing execution plans. In that article there is also reference to what is called compiled plan stub. So it appears that Compiled Plan Stub is first created and then execution plan is created.
So what I wanted to know is briefly
What are the steps that happen when I create a stored procedure?
Why is a Stored Procedure called stored procedure(if the question makes sense)?
If you can refer me to an existing SO question or some other article, that should also be fine.
Why is a Stored Procedure called stored procedure
Because it is a procedure that is stored in a database.
In other languages/environments, procedures that can be executed aren't usually stored. They are compiled in either bytecode or assembler. I.e. the procedure does not exist in its original textual form anymore. The original procedure cannot be retrieved as it was when it was created in those environments (although reverse engineering can retrieve the essence of that procedure).
When you create a stored procedure in SQL Server, it is completely stored in its original full-text form, same indentation, same casing, same lines, including comments and all. You can retrieve the text with which you created the stored procedure in its entirety.
Simplified explanation about executing a stored procedure
When SQL Server wants to execute a stored procedure, it will first check the cache to see if it has been compiled already. If it finds an entry in the cache (in the form of an execution plan) it will use this entry to execute. If it doesn't find an entry it will compile the procedure into an execution plan, store it in the cache for later use, then execute it using the execution plan.
There are cases that force a stored procedure to be recompiled, e.g. when the execution plan cache is cleared (schema changes, statistics updates, ...) or when supplying commands to the compiler that force recompilation (stored procedure WITH RECOMPILE, query option OPTION(RECOMPILE), ...).
I said execution plan is stored in the server so its called STORED Procedure.
Wrong. Execution plans also are stored on the server for dynamic SQL. I have no real idea why it is called as stored procedure, but the procedure as a whole is stored on the server (code etc.). I can assume this is the reason - but the execution plan (cache) is irrelevant here, because all execution plans are possibly stored there.
For 1: what do you care? Obviously a SQL Statement is executed. There is some parsing, to make sure it is valid. The rest is an implementation detail - and may vary even between versions. I would assume a SP is stored in some level of bytecode - but again, who cares? I do database level development for 25 years and that never even came into my consideration.
in my recent subject ,I have to do some queries through dynamic SQL,But I'm curious about
the efficiency in different ways:
1)combine the sql sentences in my server and then send them to the database ,do the query
2)send my variables to database and combine them in a certain procedure and finally do the query
Hope someone can help
BTW(I use .Net and Sqlserver)
Firstly, one of the main things you should do is to parameterise your SQL - whether that be by wrapping it up as a stored procedure in the DB, or by creating the SQL statement in your application code and then firing the whole thing in to the DB. This will mean:
prevention against SQL injection attacks by not directly concatenating user-entered values into a SQL statement
execution plan reuse (subsequent executions of that query, regardless of parameter values, will be able to reuse the original execution plan) (NB. this could be done if not parameterised yourself, via Forced Parameterisation)
Stored procedures do offer some extra advantages:
security ,only need to grant EXECUTE permissions to the stored procedures, you don't need to grant the user direct access to underlying db tables
maintainability, a change to a query does not involve an application code change, you can just change the sproc in the DB
network traffic, not necessarily a major point but you're sending less over the wire especially if the query is pretty large/complex
Personally, I use stored procedures most of the time. Though the times I need to build up SQL dynamically in application code, it is always parameterised.
Best is to use stored procedure and pass parameters from your application, as Stored procedures are precompiled queries and have execution plan ready which saves lot of time.
You can refer this url which has details http://mukund.wordpress.com/2005/10/14/advantages-and-disadvantages-of-stored-procedure/
Happy coding!!
What is the best way to get stored procedure useage data on a specific database out of SQL Server 2000?
The data I need is:
Total of all stored procedure calls over X time
Total of each specific stored procedure call over X time.
Total time spent processing all stored procedures over X time.
Total time spent processing specific stored procedures over X time.
My first hunch was to setup SQL Profiler wiht a bunch of filters to gather this data. What I don't like about this solution is that the data will have to be written to a file or table somewhere and I will have to do the number crunching to figure out the results I need. I would also like get these results ober the course of many days as I apply changes to see how the changes are impacting the database.
I do not have direct access to the server to run SQL Profiler so I would need to create the trace template file and submit it to my DBA and have them run it over X time and get back to me with the results.
Are there any better solutions to get the data I need? I would like to get even more data if possible but the above data is sufficient for my current needs and I don't have a lot of time to spend on this.
Edit: Maybe there are some recommended tools out there that can work on the trace file that profile creates to give me the stats I want?
Two options I see:
Re-script and recompile your sprocs to call a logging sproc. That sproc would be called by all your sprocs that want to have perf tracking. Write it to a table with the sproc name, current datetime, and anything else you'd like.
Pro: easily reversible, as you'd have a copy of your sprocs in a script that you could easily back out. Easily queryable!
Con: performance hit on each run of the sprocs that you are trying to gauge.
Recompile your data access layer with code that will write to a log text file at the start and end of each sproc call. Are you inheriting your DAL from a single class where you can insert this logging code in one place? Pro: No DB messiness, and you can switch in and out over an assembly when you want to stop this perf measurement. Could even be tweaked with on/off in app.config. Con: disk I/O.
Perhaps creating a SQL Server Trace outside of SQL Profiler might help.
http://support.microsoft.com/kb/283790
This solution involves creating a text file with all your tracing options. The output is put into a text file. Perhaps it could be modified to dump into a log table.
Monitoring the traces: http://support.microsoft.com/kb/283786/EN-US/