I have used the ADO-function from another website to make a query from Excel VBA 2016 on my SQL Server Express database on the same workstation. The SQL database is continuously updated with new records from a feed provider.
The query runs very smoothly and takes around 70 milliseconds to execute. I call the query every 3 seconds from a routine in Excel VBA. Although once or twice a minute the query takes 6000-12000 milliseconds. It seems the query is "hanging" at these moments for some reason.
When I turn off the external feed, the issue is not there!
I also have used Express Profiler to get insight in the cause. I have added the output below this post. Lines 33 and 173 are lines with a long interval. As you can see this duration is much longer than the other (identical queries).
I have set the READ_COMMIT_SNAPSHOT ON in options.
I have also set ALLOW_SNAPSHOT_ISOLATION ON in options.
I did not close down any connections, although SQL did give a message that it was closing all connections.
Although after testing the problem is still there.
After reading this:
https://learn.microsoft.com/en-us/sql/t-sql/statements/set-transaction-isolation-level-transact-sql I have tried using the command SET TRANSACTION ISOLATION LEVEL READ UNCOMMITTED but, after testing, the problem is still there.
Does anyone know a solution to this problem?
Related
I have a performance issue with a method that calls org.hibernate.Query#list. The duration of the method call vary over time: it usually lasts about one second but some days, for maybe half a day, it takes about 20 seconds.
How can this issue be resolved? How can the cause for this issue be determined?
More elements in the analysis of this issue:
Performance issues have been observed in production environment, but the described issue is in a test environment.
The issue has been observed for at least several weeks but the date of its origin is unknown.
The underlying query is a view (select) in MS SQL Server (2008 R2):
Database reads/writes in this test environment are from a few users at a time only: the database server should not be overly sollicited and the data only changes slowly over time.
Executing the exact query directly from a MS SQL Server client always takes less than a second.
Duplicating the database (using the MS SQL Server client to backup the database and restore this backup as a new database) does not allow to reproduce the problem: the method call results in being fast on the duplicate.
The application uses Hibernate (4.2.X) and Java 6.
Upgrading from Hibernate 3.5 to 4.2 has not changed anything about the problem.
The method call is always with the same arguments: there is a test method that does the operation.
Profiling the method call (using hprof) shows that when it is long, most of the time is spent on "Object.wait" and "ref.ReferenceQueue.remove".
Using log4jdbc to log the underlying query duration during the method call shows the following results :
query < 1s => method ~ 1s
query ~ 3s => method ~ 20s
The query generates POJO as described in the most up-voted answer from this issue.
I have not tried using a Constructor with all attributes as described in the most up-voted answer from this other similar issue because I do not understand what effect that would have.
A possible cause of apparently random slowness with an Hibernate query is the flushing of the session. If some statements (inserts, updates, deletes) in the same transaction are unflushed, the list method of Query might do an autoflush (depending on the current flush mode). If that's the case, the performance issue might not even be caused by the query on which list() is called.
It seems the issue is with MS SQL Server and the updating of procedure's plan: following DBCC FREEPROCCACHE, DBCC DROPCLEANBUFFERS the query and method times are consistent.
A solution to the issue may be to upgrade MS SQL Server: upgrading to MS SQL Server 2008 R2 SP2 resulted in the issue not appearing anymore.
It seems the difference between the duration of the query and that of the method is an exponential factor related to the objects being returned: most of the time is spent on a socket read of the result set.
I am writing some code that is importing a large amount of data into three tables currently around 6 million rows across the three tables. I am wanting to do this in a transaction so if there are any issues or the user cancels the import nothing is imported. This works fine on my own development machine however on a slower amazon ec2 instance and micro sql instance I am getting the following exception:
System.Data.SqlClient.SqlException: Timeout expired. The timeout period elapsed prior to completion of the operation or the server is not responding
Now I know that the commit is finishing eventually because the data is present in the tables when I look, so my question is; can this be easily avoided without adding the connection timeout property to my connection string (I only want this one operation to not timeout) or is this a really hard/dangerous thing to be doing?
I am not sure if maybe I should import into holding tables and then call stored procedures to move the data when I am ready because I would assume this will result in a shorter transaction)
I am using Ms Sql server 2012.
Do comment if I need to add more data.
Many thanks for your help
Check what is the SP getting timedout .. if you have any third party tool like Redgate or Avicode you can figure it out ..or use Profiler to figure it out.. then see the execution plan for the SP or query .. If you find any Key lookups or RID lookups then resolve it first and try again..
I have a SQL script that will refresh the dependent views of a table once the table has been modified, like adding new fields. The script will be run thru ExecuteNonQuery, see example below.
Using refreshCommand As New SqlClient.SqlCommand("EXEC RefreshDependentViews 'Customer','admin',0", SqlClient.SqlConnection, SqlClient.SqlTransaction)
refreshCommand.ExecuteNonQuery()
End Using
The above code when executed will take 4-5 seconds, but when I copy the script only and run it through MS SQL directly, it only takes 2-3 seconds.
My question is, why they have different intervals?
Please note that the MS SQL server is on my PC itself and also the code.
Thanks
SqlClient and SSMS have different connection-level options (SET options) by default, which can sometimes be a factor. I also wonder what the isolation level is for the two things, which could be compounded if you are using TransactionScope etc in your code.There could also simply be different system load at the time. Basically, hard to say just from that: but there are indeed some things that can impact this.
I have a reporting solution with several reports. Up to now, I have been able to add a dataset based on a SPROC with no problems. However, when I try to add the lastest dataset, and use a SPROC for its query type, when I click on Refresh Fields I get the following error:
Timeout expired. The timeout period elapsed prior to completion of the operation or the server is not responding.
I have tested the database connection in Data Source properties>Edit>Test Connection, and it's working fine.
I have increased the timeout to 100 in the following areas:
The connection string for the datasource, which is - Connect
Timeout=100
Tools>Options>Database Tools>Query and View
Designers. Cancel long running query is set to 100.
Tools>Options>Database Tools>Table and Database Designers>Checked
'Override Connection String time-out value for table designer
updates. Transaction time-out after is set to 100
The SPROC runs fine in the SQL database. It takes about 55 seconds.
Any other ideas?
Thanks.
UPDATE: I now can't add any dataset with a SPROC. Even thought the SPROCs are all working fine in SQL!!!!!!
If you are using Report Builder, you can increase timeout also in your DataSet.
I have also face same problem for adding the newly added column in stored procedure.
From the following way overcome this issue.
Alter the stored procedure as comment all the query except that final select command.
Now that new column has been added, then un-comment that quires in sp.
The thing to remember with your report is that when it is ran, it will attempt to run ALL the datasets just to make sure the are runnable, and the data they are requesting can be returned. So by running the each proc seperately you are in fact not duplicating with SSRS is trying to do...and to be honest don't bother.
What you could try is running sp_who while the report is running, or even just manually go through the procedures to see what table they have in common. Since your proc takes 52 seconds to return its dataset I'm going to assume its doing some heavy lifting. Without the queries nobody will be able to tell what the exact problem is.
I suggest using NO LOCK to see if that resolves your issue. If it does then your procs are fighting for data and blocking each other...possibly in a endless loop. Using NO LOCK is NOT a fix. Read what it does and judge for yourself however.
My solution was to go to the Dataset Properties for the given problem dataset, paste the query in the Query field, click Refresh Fields, and click Ok.
I've got a scenario when sometimes a user selects the right parameters and makes a query which takes several minutes or more to execute. I cannot prevent him to select such a combination of parameters (it's quite legal), so I'd like to set a timeout on the query.
Note that I really want to stop the query execution itself and rollback any transactions, because otherwise it hogs up most of server resources. Add an impatient user who restarts the application and tries the combination again, and you've got a recipe for a disaster (read: SQL Server DoS).
Can this be done and how?
As far as I know, apart from setting the command or connection timeouts in the client, there is no way to change timeouts on a query by query basis in the server.
You can indeed change the default 600 seconds using sp_configure, but these are server scoped.
Humm!
did you try LOCK_TIMEOUT
Note down what it was orginally before running the query
set it for your query
after running your query set it back to original value
SET LOCK_TIMEOUT 1800;
SELECT ##LOCK_TIMEOUT AS [Lock Timeout];
I might suggest 2 things.
1)
If your query takes a lot of time because it´s using several tables that might involve locks, a quite fast solution is to run your queries with the "NoLock" hint.
Simply add Select * from YourTable WITH (NOLOCK) in all your table references an that will prevent your query to block for concurrent transactions.
2) if you want to be sure that all of your queries runs in (let´s say) less than 5 seconds, then you could add what #talha proposed, that worked sweet for me
Just add at the top of your execution
SET LOCK_TIMEOUT 5000; --5 seconds.
And that will cause that your query takes less than 5 or fail. Then you should catch the exception and rollback if needed.
Hope it helps.
In management studio you can set the timeout in seconds.
menu Tools => Options set the field and then Ok
It sounds like more of an architectual issue, and any timeout/disconnect you can do would be more or less a band-aid. This has to be solved on SQL server side, by the way of read-only replica, transaction log shipping (to give you a read-only server to connect to), replication and such. Basically you give the DMZ sql server that heavy read can go to without killing stuff. This is very common. A well-designed SQL system won't be taken down by DDoS - that'd be like a car that dies if you step on the gas.
That said, if you are at the liberty to change the code, you could guesstimate if the query is too heavy and you could either reject or return only X rows in your stored procedure. If you are mated to some reporting tool and such and can't control the SELECT it generates, you could point it to a view and then do the safety valve in the view.
Also, if up-to-the-minute freshness isn't critical and you could compromise on that, like monthly sales data, then compiling a physical table of complex joins by job to avoid complex joins might do the trick - that way everything would be sub-second per query.
It entirely depends on what you are doing, but there is always a solution. Sometimes it takes extra coding to optimize it, sometimes it takes extra money to get you the secondary read-only DB, sometimes it needs time and attention in index tuning.
So it entirely depends, but I'd start with "what can I compromise? what can I change?" and go from there.
You can set Execution time-out in seconds.
If you have just one query I don't know how to set timeout on T-SQL level.
However if you have a few queries (i.e. collecting data into temporary tables) inside stored procedure you can just control time of execution with GETDATE(), DATEDIFF() and a few INT variables storing time of execution of each part.
You can specify the connection timeout within the SQL connection string, when you connect to the database, like so:
"Data Source=localhost;Initial Catalog=database;Connect Timeout=15"
On the server level, use MSSQLMS to view the server properties, and on the Connections page you can specify the default query timeout.
I'm not quite sure that queries keep on running after the client connection has closed. Queries should not take that long either, MSSQL can handle large databases, I've worked with GB's of data on it before. Run a performance profile on the queries, prehaps some well-placed indexes could speed it up, or rewriting the query could too.
Update:
According to this list, SQL timeouts happen when waiting for attention acknowledgement from server:
Suppose you execute a command, then the command times out. When this happens the SqlClient driver sends a special 8 byte packet to the server called an attention packet. This tells the server to stop executing the current command. When we send the attention packet, we have to wait for the attention acknowledgement from the server and this can in theory take a long time and time out. You can also send this packet by calling SqlCommand.Cancel on an asynchronous SqlCommand object. This one is a special case where we use a 5 second timeout. In most cases you will never hit this one, the server is usually very responsive to attention packets because these are handled very low in the network layer.
So it seems that after the client connection times out, a signal is sent to the server to cancel the running query too.