I have a very simple query
SELECT count(*) FROM MyTable
There are 508,000 rows in it. When I run the above query for the very first time it takes approximately 53 seconds to return the result. If I rerun the query again, it takes milliseconds to return the result.
I believe SQL Server is caching the query/results? Is it possible that I could tell SQL Server not to cache query/results, or somehow clear the cache/result?
In my application I am trying to do some performance tuning, but problems like above don't help me out.
Yes, you can flush the data buffer like this (but seriously, don't!):
DBCC DROPCLEANBUFFERS
Use DBCC DROPCLEANBUFFERS to test queries with a cold buffer cache
without shutting down and restarting the server.
To drop clean buffers from the buffer pool, first use CHECKPOINT to
produce a cold buffer cache. This forces all dirty pages for the
current database to be written to disk and cleans the buffers. After
you do this, you can issue DBCC DROPCLEANBUFFERS command to remove all
buffers from the buffer pool.
Ref
You have already ruled out the database as your bottleneck (which it is often), so rather than freeing the data buffer cache, I suggest you profile your code.
Related
Every time I am getting same execution time while executing the SQL query.
is there any chance to get same execution time for all the time if SQL query runs multiple times?
When you run your query for the first time and the data is not in cache, the server read the data from disk. It is time-comsuming. The second time you execute the same query data is already in cache so it requires less time.
is there any chance to get same execution time for all the time if SQL
query runs multiple times ?
If you want to test your queries with the cold cache (with no data cached, every time) you can use DBCC DROPCLEANBUFFERS before your query execution:
Use DBCC DROPCLEANBUFFERS to test queries with a cold buffer cache
without shutting down and restarting the server. To drop clean buffers
from the buffer pool and columnstore objects from the columnstore
object pool, first use CHECKPOINT to produce a cold buffer cache. This
forces all dirty pages for the current database to be written to disk
and cleans the buffers. After you do this, you can issue DBCC
DROPCLEANBUFFERS command to remove all buffers from the buffer pool.
Of course this is not to be used in production environment.
If
conversely you want to have your data always in cache, you should increase RAM for your server and don't restart it as long as possible.
I have a complex query that runs under 2 seconds (which is ok), however if i run
DBCC DROPCLEANBUFFERS
DBCC FREEPROCCACHE
first the query performans goes oder 40 seconds. Should I be worried?
No, Sql server is doing EXACTLY what you just told it to. You asked it to empty out ALL it's hard earned buffers.
It ran first previously because it had stored the information about the tables and the query in the caches and deleted the execution plans. Having purged them it has to rebuild them from scratch.
Don't do that unless you've a good reason to.
Use DBCC FREEPROCCACHE to clear the plan cache carefully. Freeing the plan cache causes, for example, a stored procedure to be recompiled instead of reused from the cache. This can cause a sudden, temporary decrease in query performance. For each cleared cachestore in the plan cache, the SQL Server error log will contain the following informational message: "SQL Server has encountered %d occurrence(s) of cachestore flush for the ‘%s’ cachestore (part of plan cache) due to ‘DBCC FREEPROCCACHE’ or ‘DBCC FREESYSTEMCACHE’ operations." This message is logged every five minutes as long as the cache is flushed within that time interval.
Use DBCC DROPCLEANBUFFERS to test queries with a cold buffer cache without shutting down and restarting the server.
To drop clean buffers from the buffer pool, first use CHECKPOINT to produce a cold buffer cache. This forces all dirty pages for the current database to be written to disk and cleans the buffers. After you do this, you can issue DBCC DROPCLEANBUFFERS command to remove all buffers from the buffer pool.
There is a "best practice" that you have to run
DBCC FREESESSIONCACHE
DBCC FREEPROCCACHE
DBCC DROPCLEANBUFFERS
Before doing performance analysis on a SQL query.
Yet, for example, the later one DROPCLEANBUFFERS:
Use DBCC DROPCLEANBUFFERS to test queries with a cold buffer cache
without shutting down and restarting the server.
To drop clean buffers from the buffer pool, first use CHECKPOINT to
produce a cold buffer cache. This forces all dirty pages for the
current database to be written to disk and cleans the buffers. After
you do this, you can issue DBCC DROPCLEANBUFFERS command to remove all
buffers from the buffer pool.
I guess, this means that you will test your query as if it was the first query that has run in the server, thus the actual "real-life" impact of the query will be lower.
Is it really advisable to run the three commands to know the query cost or does it get you to a rather empirical results that have no close relation to actual query time in live environment?
I disagree it is best practice and very rarely use it.
A query that I tune should be a popular, often run one. This gives me most bang for my buck. It should rarely be run "cold" for either plan or data.
I'm testing the query execution: not the disk read system or the Query Optimiser compilation
This was asked on DBA.SE a while ago. See these please
https://dba.stackexchange.com/a/10820/630
https://dba.stackexchange.com/a/7870/630
Is it really advisable to run the three commands to know the query cost or does it get you to a rather empirical results that have no close relation to actual query time in live environment?
It depends.
If you don't run DBCC DROPCLEANBUFFERS then there is a chance that you will end up with some odd results unless you are very careful about the way that you do your performance analysis. For example, generally speaking the second time you run a query it's going to be quicker because the required pages are probably cached in memory - running DBCC DROPCLEANBUFFERS helps here because it ensures that you have a consistent starting point in your testing and it ensures that your query is not artificially running quickly just because it is skipping the expensive disk access portions of your query.
Like you say however, in live environments it could be that this data is always cached and so your test is not representative of production conditions - it depends on whether or not you are analysing the performance based on the assumption that the data is frequently accessed and so will generally be cached, or infrequently accessed and so disk asscess is likely to be involved.
The short answer is that running those 3 statements can help ensure that you get consistent results while performance testing, however you shouldn't necessarily always run these before testing, instead you should try to understand what each one does and what impact it will have on your query when compared to a production environment.
As an aside, Never run any of those 3 statements on a production server unless you know exactly what you are doing!
I agree with what #gbn states in his answer, and I don't think I've ever used the three commands for anything other than demonstrating a difference between possible approaches.
In addition it would be ill-advised in most cases to run these three DBCCs on a production environment just for testing. And performance tuning queries in a test environment, with test data and test load, will often lead you to draw the wrong conclusions regarding your query, anyway.
Usually, when I tune a query, I use the profiler to get actual execution stats from live, I use SSMS to get execution plans from live and I do a few test runs (on test data) to see what differs. For the more tricky problems, I also use the Windows Performance Monitor - and always in a situation that is as close to the real one as possible. Running DBCC would just remove the tuning effort from the real deal.
I would like to check options to improve my queries.
sometimes, i want to do the tests on a production server so i can't use DBCC FREEPROCCACHE
and DBCC DROPCLEANBUFFERS to clear the entire server cache.
could you please share with me the way to do a kind of "cache clean" only for my connection/scope?
Thanks.
DBCC FREEPROCCACHE (plan_handle | sql_handle | pool_name)
By passing the plan_handle or the other options we will be able to cleare the cache of a particular sp or a query.
Buffers are not held in a user specific table, or stored per user - there is no way for SQL Server to selectively clear it, since it does not know what items are being held for which queries; doing so would produce unneeded overhead in almost every case (except what you are trying to do now. Sorry.) Despite this, there are options.
There are suggestions, however, to ameliorate the issue, even if the problem can't be avoided:
You can use with(RECOMPILE) to force the query to find a new plan, but that will not clear the cache.
You can run each query twice, to see how slowly/quickly it runs once the data is buffered.
You can repeatedly alternate the two methods, to see if they get faster, and what the speed difference converges towards.
You can run them a day or two apart, or after a server reset. (This is if the production server gets reset occasionally anyways.)
This article has additional ideas for testing in such a situation.
If downtime is OK you can take the database offline and take it online immediately afterwards.
Having looked around the net using Uncle Google, I cannot find an answer to this question:
What is the best way to monitor the performance and responsiveness of production servers running IIS and MS SQL Server 2005?
I'm currently using Pingdom and would like it to point to a URL which basically mimics a 'real world query' but for obvious reasons do not want the query to run from cache. The URL will be called every 5 minutes.
I cannot clear out the cache, buffers, etc since this would impact negatively on the production server. I have tried using a random generated number within the SELECT statement in order to generate unique queries, but the cached query is still used.
Is there any way to simulate the NO_CACHE in MySQL?
Regards
To clear the SQL buffer and plan cache:
DBCC DROPCLEANBUFFERS
GO
DBCC FREEPROCCACHE
GO
A little info about these commands from MSDN:
Use DROPCLEANBUFFERS to test queries with a cold buffer cache without shutting down and restarting the server. (source)
Use DBCC FREEPROCCACHE to clear the plan cache carefully. Freeing the plan cache causes, for example, a stored procedure to be recompiled instead of reused from the cache. (source)
SQL Server does not have a results cache like MySQL or Oracle, so I am a bit confused about your question. If you want the server to recompile the plan cache for a stored procedure, you can execute it WITH RECOMPILE. You can drop your buffer cache, but that would affect all queries as you know.
At my company, we test availability and performance separately. I would suggest you use this query just to make sure you your system is working together from front-end to database, then write other tests that check the individual components to judge performance. SQL Server comes with an amazing amount of ways to check if you are experiencing bottlenecks and where they are. I use PerfMon and DMVs extensively. Using PerfMon, I check CPU and page life expectancy, as well as seeing how long my disk queue is. Using DMVs, I can find out if my queries are taking too long (sys.dm_exec_query_stats) or if wait times are long (sys.dm_os_wait_stats).
The two biggest bottlenecks with IIS tend to be CPU and memory, and IIS comes with its own suite of PerfMon objects to query, but I am not as familiar with those.