I use a SqlTransaction in my C# project, and I use a Delete statement with an EcexuteNonQuery call.
This works very well and I have always the same amount of rows to delete, but 95% of the time, this needs 1 ms and approx 5% of the time, it is between 300 - 500 ms.
My code:
using (SqlTransaction DbTrans = conn.BeginTransaction(IsolationLevel.ReadCommitted))
{
SqlCommand dbQuery = conn.CreateCommand();
dbQuery.Transaction = DbTrans;
dbQuery.CommandType = CommandType.Text;
dbQuery.CommandText = "delete from xy where id = #ID";
dbQuery.Parameters.Add("ID", SqlDbType.Int).Value = x.ID;
dbQuery.ExecuteNonQuery();
}
Is something wrong with my code?
Read Understanding how SQL Server executes a query and How to analyse SQL Server performance to get you started on troubleshooting such issues.
Of course I assume you have an index on xy.id. Your DELETE is likely blocking from time to time. This an be caused by many causes:
data locks from other queries
IO block from your hardware
log growth events
etc
The gist of it is that using the techniques in the articles linked above (specially the second one) you can identify the cause and address it appropriately.
Changes to your C# code will have little impact, if any at all. Using a stored procedure is
not going to help. You need to root cause the problem.
Related
I've encountered a strange phenomena then investigatating a slow view of a typical ASP.NET MVC application. One of the queries is running ridiculously slow for no obvious reason. The LINQ query in question look like this (Db is DbContext):
var testResults = Db.CustomTestResults
.Include(tr => tr.TestMachine.Platform)
.Include(tr => tr.TestCase)
.Include(tr => tr.CustomTestResultAnalysis.Select(tra => tra.AnalysisOutcomeData))
.Where(tr => tr.CustomTestBuildId == testBuild.Id)
.ToList()
.AsReadOnly();
nothing special actually. Depending on filter query result set can vary in size, from 10 to 10000 records at max.
The SQL generated query (captured by LINQ debug log), executed from SSMS, runs fast, about 2 seconds for the largest sets and less than a second for smaller ones. However then run by IIS strange things happen. The queries began to run like ~1/100x slower speed. The smaller ones take ~10 seconds to execute, the larger are failing due to query execution timeout. I'm not sure if any other queries are affected, but this one is only that is dealing with large data sets, so it's most obvious to notice the problem.
As this was not confusing enough this same code was running perfectly as expected not so long ago. So the bug seems to be caused by some external factors. The database is SQL Server 2014 SP2, EF is at v6.2, IIS 7.5.
Would appreciate any ideas in what areas and how I could investigate this further.
As it turned out, the issue was in SQL Server optimizations, which start to work some time after multiple runs of the similar queries. This problem can be detected by any nonrelevant change to the original query, which fixes performance for some time.
This behaviour can be properly mitigated by controlling query command options. One of the solutions for EF is demonstrated here.
As a temporary "quick-and-dirty" solution I used this approach to randomize query each time, thus preventing optimizations by SQL Server engine:
private static IQueryable<CustomTestResult> RandomizeQuery(IQueryable<CustomTestResult> query)
{
const int minConditions = 1;
const int maxConditions = 5;
const int minId = -100;
const int maxId = -1;
var random = new Random();
var conditionsCount = random.Next(minConditions, maxConditions);
for (int i = 0; i < conditionsCount; i++)
{
var randomId = random.Next(minId, maxId);
query = query.Where(test => test.Id != randomId);
}
return query;
}
Since the SQL has not changed but it is having issues depending on what platform you run on I would start with your settings. A GREAT reference for the whys and hows is written by Erland Sommarskog this: http://www.sommarskog.se/query-plan-mysteries.html
It is long but I imagine you will find your answer in there.
I have a query that returns between 10k and 20k rows. I'm dumping this data into a IEnumerable<T> in what as far as I know is the fastest possible way:
using (var rdr = cmd.ExecuteReader()) {
var trackIdCol = rdr.GetOrdinal("TrackId");
var dateTimeCol = rdr.GetOrdinal("ActDateTime");
var clicksCol = rdr.GetOrdinal("Clicks");
var ipCol = rdr.GetOrdinal("IPAddress");
while (rdr.Read()) {
yield return new SiteClick() {
TrackId = (int)rdr[trackIdCol],
DateTime = (DateTime)rdr[dateTimeCol],
Clicks = (int)rdr[clicksCol],
IPAddress = rdr[ipCol] as string
};
}
}
The query takes about 11s to return all the results in SSMS and SSDT, but the code above takes over 2 minutes. There has to be something I'm doing wrong here. SqlDataAdapter.Fill() also takes about 2 minutes to run, if that helps.
It is worth noting that our database is horribly unoptimized. Just the fact that it takes 11 seconds to get results from that query in SSMS is ridiculous, but I gotta work with what I got. If the query executes quickly in SSMS, but an empty while(rdr.Read()){} still takes 2 minutes, it is possible that the DB is still the issue?
You can't dump into IEnumerable, because it is interface that must have implementation behind the scene. In your sample yield used as such implementation. But yield is rather expensive. For every iteration its state is saved, then restored and this is slow for database objects.
Database reader is the fastest iterator itself. So if you can use it directly - this will
be the fastest way. But there is a drawback, because such database cursors might lock
db tables for access from other threads. So use it in single-user environments or
when you can read uncommitted data.
To reduce locking time you can dump data into in-memory IEnumerable implementations.
For example, List. Just set larger capacity to avoid frequent allocations, so it
will be as fast as underlying array. The drawback of such solution is memory usage. But
in you case this will be less than 1M.
List<SiteClick> list = new List<SiteClick>(20000);
Also you can improve performance and memory usage if define SiteClick as struct, not class.
In this case list will contain objects, not references.
A little bit additional performance you can get if use typified reader methods.
TrackId = rdr.GetInt32(trackIdCol),
DateTime = rdr.GetDateTime(dateTimeCol),
Clicks = rdr.GetInt32(clicksCol),
IPAddress = rdr.GetString(ipCol)
UPDATE: For SQL Server it's a frequent case, when queries are tested in SSMS. But there are some issues, because SMSS has different options compared to database defaults. For example, ARITHABORT. So you can set it manually for connection to test.
// Use the same connection as for data reading
var cmdOptions = connection.CreateCommand();
cmdOptions.CommandType = CommandType.Text;
cmdOptions.CommandText = "SET ARITHABORT ON";
cmdOptions.ExecuteNonQuery();
According to MS recommendations - http://msdn.microsoft.com/en-us/library/aa933126%28SQL.80%29.aspx, its better to ensure that you have these options set:
SET ANSI_NULLS ON
SET ANSI_PADDING ON
SET ANSI_WARNINGS ON
SET CONCAT_NULL_YIELDS_NULL ON
SET NUMERIC_ROUNDABORT ON
SET QUOTED_IDENTIFIER ON
SET NUMERIC_ROUNDABORT OFF
My application makes about 5 queries per second to a SQL Server database. Each query results in 1500 rows on average. The application is written on C++/QT, database operations are implemented using QODBC driver. I determined that query processing takes about 25 ms, but fetching the result - 800 ms. Here is how code querying the data base looks like
QSqlQuery query(db)
query.prepare(queryStr);
query.setForwardOnly(true);
if(query.exec())
{
while( query.next() )
{
int v = query.value(0).toInt();
.....
}
}
How to optimize result fetching?
This does not directly answer your question as I haven't used QT in years. In the actual ODBC API you can often speed up the retrieval of rows by setting SQL_ATTR_ROW_ARRAY_SIZE to N then each call to SQLFetch returns N rows at once. I took a look at SqlQuery in qt and could not see a way to do this but it may be something you could look in to with QT or simply write to the ODBC API directly. You can find an example at Preparing to Return Multiple Rows
My profiler trace shows that exec sp_reset_connection is being called between every sql batch or procedure call. There are reasons for it, but can I prevent it from being called, if I'm confident that it's unnecessary, to improve performance?
UPDATE:
The reason I imagine this could improve performance is twofold:
SQL Server doesn't need to reset the connection state. I think this would be a relatively negligible improvement.
Reduced network latency because the client doesn't need to send down an exec sp_reset_connection, wait for response, then send whatever sql it really wants to execute.
The second benefit is the one I'm interested in, because in my architecture the clients are sometimes some distance from the database. If every sql batch or rpc requires a double round-trip this doubles the impact of any network latency. Eliminating such double calls could potentially improve performance.
Yes there are lots of other things I could do to improve performance like re-architect the app, and I'm a big fan of solving the root cause of problems, but in this case I just want to know if it's possible to prevent sp_reset_connection to be called. Then I can test if there is any performance improvement and properly assess the risks of not calling this.
This prompts another question: does the network communication with sp_reset_connection really occur like I outlined above? i.e. Does the client send exec sp_reset_connection, wait for a response, then send the real sql? Or does it all go in one chunk?
If you're using .NET to connect to SQL Server, disabling of the extra reset call was disabled as of .NET 3.5 -- see here. (The property remains, but it does nothing.)
I guess Microsoft realized (as someone did experimentally here) that opening the door to avoid the reset was far more dangerous than it was to get a (likely) small performance gain. Can't say I blame them.
Does the client send exec sp_reset_connection, wait for a response, then send the real sql?
EDIT: I was wrong -- see here -- the answer is no.
Summary: there is a special bit set in a TDS message that specifies that the connection should be reset, and SQL Server executes sp_reset_connection automatically. It appears as a separate batch in Profiler and would always be executed before the actual query you wanted to execute, so my test was invalid.
Yes, it's sent in a separate batch.
I put together a little C# test program to demonstrate this because I was curious:
using System.Data.SqlClient;
(...)
private void Form1_Load(object sender, EventArgs e)
{
SqlConnectionStringBuilder csb = new SqlConnectionStringBuilder();
csb.DataSource = #"MyInstanceName";
csb.IntegratedSecurity = true;
csb.InitialCatalog = "master";
csb.ApplicationName = "blarg";
for (int i = 0; i < 2; i++)
_RunQuery(csb);
}
private void _RunQuery(SqlConnectionStringBuilder csb)
{
using (SqlConnection conn = new SqlConnection(csb.ToString()))
{
conn.Open();
SqlCommand cmd = new SqlCommand("WAITFOR DELAY '00:00:05'", conn);
cmd.ExecuteNonQuery();
}
}
Start Profiler and attach it to your instance of choice, filtering on the dummy application name I provided. Then, put a breakpoint on the cmd.ExecuteNonQuery(); line and run the program.
The first time you step over, just the query runs, and all you get is the SQL:BatchCompleted event after the 5 second wait. When the breakpoint hits the second time, all you see in profiler is still just the one event. When you step over again, you immediately see the exec sp_reset_connection event, and then the SQL:BatchCompleted event shows up after the delay.
The only way to get rid of the exec sp_reset_connection call (which may or may not be a legitimate performance problem for you) would be to turn off .NET's connection pooling. And if you're planning to do that, you'd likely want to build your own connection pooling mechanism, because just turning it off and doing nothing else will probably hurt more overall than taking the hit of the extra roundtrip, and you will have to deal with the correctness issues manually.
This Q/A could be helpful:
What does "exec sp_reset_connection" mean in Sql Server Profiler?
However, I did a quick test using Entity Framework and MS-SQL 2008 R2. It shows that "exec sp_reset_connection" isn't time consuming after the first call:
for (int i = 0; i < n; i++)
{
using (ObjectContext context = new myEF())
{
DateTime timeStartOpenConnection = DateTime.Now;
context.Connection.Open();
Console.WriteLine();
Console.WriteLine("Opening connection time waste: {0} ticks.", (DateTime.Now - timeStartOpenConnection).Ticks);
ObjectSet<myEntity> query = context.CreateObjectSet<myEntity>();
DateTime timeStart = DateTime.Now;
myEntity e = query.OrderByDescending(x => x.EventDate).Skip(i).Take(1).SingleOrDefault<myEntity>();
Console.Write("{0}. Created By {1} on {2}... ", e.ID, e.CreatedBy, e.EventDate);
Console.WriteLine("({0} ticks).", (DateTime.Now - timeStart).Ticks);
DateTime timeStartCloseConnection = DateTime.Now;
context.Connection.Close();
context.Connection.Dispose();
Console.WriteLine("Closing connection time waste: {0} ticks.", (DateTime.Now - timeStartCloseConnection).Ticks);
Console.WriteLine();
}
}
And output was this:
Opening connection time waste: 5390101 ticks.
585. Created By sa on 12/20/2011 2:18:23 PM... (2560183 ticks).
Closing connection time waste: 0 ticks.
Opening connection time waste: 0 ticks.
584. Created By sa on 12/20/2011 2:18:20 PM... (1730173 ticks).
Closing connection time waste: 0 ticks.
Opening connection time waste: 0 ticks.
583. Created By sa on 12/20/2011 2:18:17 PM... (710071 ticks).
Closing connection time waste: 0 ticks.
Opening connection time waste: 0 ticks.
582. Created By sa on 12/20/2011 2:18:14 PM... (720072 ticks).
Closing connection time waste: 0 ticks.
Opening connection time waste: 0 ticks.
581. Created By sa on 12/20/2011 2:18:09 PM... (740074 ticks).
Closing connection time waste: 0 ticks.
So, the final conclusion is: Don't worry about "exec sp_reset_connection"! It wastes nothing.
Personally, I'd leave it.
Given what it does, I want to make sure I have no temp tables in scope or transactions left open.
To be fair, you will gain a bigger performance boost by not running profiler against your production database. And do you have any numbers or articles or recommendations about what you can gain from this please?
Just keep the connection open instead of returning it to the pool, and execute all commands on that one connection.
I am using SQL Server 2005 CE framework 3.5 and attempting to use merge replication between my hand held and my SQL Server. When I run the code to synchronise it just seems to sit forever, and when I put a breakpoint in my code it never gets past the call to Synchronize().
If I look at the replication monitor in sql server, it gets to the point where it says the subscription is no longer synchronising and doesn't show any errors. Therefore I am assuming this to mean the synchronisation is complete.
http://server/virtualdirectory/sqlcesa35.dll?diag does not report any issues.
This is my first attempt at any handheld development, so I may have done something daft. However, SQL Server seems to be reporting a successful synchronisation.
Any help would be greatly appreciated as I have spent ages on this !
Here is my code.
const string DatabasePath = #"SD Card\mydb.sdf";
var repl = new SqlCeReplication
{
ConnectionManager = true,
InternetUrl = #"http://server/virtualdirectory/sqlcesa35.dll",
Publisher = #"servername",
PublisherDatabase = #"databasename",
PublisherSecurityMode = SecurityType.DBAuthentication,
PublisherLogin = #"username",
PublisherPassword = #"password",
Publication = #"publicationname",
Subscriber = #"PPC",
SubscriberConnectionString = "Data Source=" + DatabasePath
};
try
{
Cursor.Current = Cursors.WaitCursor;
if (!File.Exists(DatabasePath))
{
repl.AddSubscription(AddOption.CreateDatabase);
}
repl.Synchronize();
MessageBox.Show("Successfully synchronised");
}
catch (SqlCeException e)
{
DisplaySqlCeErrors(e.Errors, e);
}
finally
{
repl.Dispose();
Cursor.Current = Cursors.Default;
}
Another thing you can do to speed up the Synchronize operation is to specify a db file path that is in your PDA's main program memory (instead of on the SD Card as in your example). You should see a speed improvement of up to 4X (meaning the Sync may take only 25% as long as it's taking now).
If you're running out of main program memory on your PDA, you can use System.IO.File.Move() to move the file to the SD Card after the Synchronize call. This seems a bit strange, I know, but it's much faster to sync to program memory and copy to the SD card then it is to sync directly to the SD card.
I have since discovered that it was just taking a long time to copy the data to the physical disk. Although the sql server replication had completed, it was still copying the data to the sd card.
I identified this by reducing the amount of tables I am replicating and I got a more immediate response (well another error but unrelated to this issue).
Thanks anyway :)