I have a stored procedure which when run from SQL Server Management Studio consistently takes 5 seconds to run when called like this.
exec dbo.MyStoredProc '2009-04-30 00:00:00', '2009-04-30 20:00:00'
When called from an excel spreadsheet via VBA it takes 6 minutes plus (not including the time taken to copy the recordset to a sheet. The VBA is nothing fancy simply using an ADO connection to return a recordset. Unfortunately the Excel approach is a client requirement that I can't get rid of yet.
Public Function GenerateSQL(strQueryName As String) As Recordset
Dim rs As Recordset, cm As Command, dbsConn As Connection
Set dbsConn = New ADODB.Connection
dbsConn.Open Configuration.Range("ConnectionString")
Set cm = New ADODB.Command
With cm
.CommandText = strQueryName
.CommandType = adCmdStoredProc
.CommandTimeout = 300
.ActiveConnection = dbsConn
Set rs = .Execute()
End With
Set GenerateSQL = rs
End Function
Does anyone have any idea why this would happen or how I could begin to trace what is happening?
Thanks,
Steve
Everything you need to know about this topic: Slow in the Application, Fast in SSMS? Understanding Performance Mysteries
I believe I have the same problem as Steve Homer.
In addition to this SO question I also found this thread on eggheadcafe.com Very slow SP execution when using .net - very fast in Management Studio - totico
The answers say it's about parameter sniffing and how that affects which execution plan is used. The answers there specifically mentions the arithabort set option and how that affects the selection of plan.
Now I just need to find out how to change the set options from VBA...
Finally thanks to this forum entry on social.msdn.com i managed to get it right. First, set multiple connections to false:
connectionObject.Properties("Multiple Connections") = False
and then use the following function on your connection to set arithabort on ...
Private Sub OptionSet(ByRef cnn As adodb.Connection)
Dim cmd As adodb.Command
Set cmd = New adodb.Command
With cmd
Set .ActiveConnection = cnn
.CommandType = adodb.CommandTypeEnum.adCmdText
.CommandText = "set arithabort on"
Call .Execute
End With
Set cmd = Nothing
End Sub
Use SQL Server Profiler
Set up a trace on your database.
Limit the trace only to the stored procedure object in question
Limit to the username used by the VBA code
An introduction to SQL Server Profiler
In particular, check the SET options used by the connection and compare these with the defaults used when running the stored procedure in SSMS.
I have come across scenarios before where the SET options were different between calling code and within SSMS and the difference in performance was HUGE.
Thanks I'll take a look at the trace tools.
In reply to the comments on the original question
Are you using the exact same parameter values?
Yes exactly the same.
How much data is being returned (roughly) - number of rows and columns (and are any of them particularly big)?
Under 200 rows, perhaps 15 fields mostly ints with a couple of 20 character varchars.
Can you run SQL profiler and confirm if the sql is the issue or the remains of the macro in excel ?
The SQL is pretty ugly, as is the underlying database schema, and unfortunately is under NDA so I can't post it. If the query were the issue then wouldn't it be slow in management studio too though?
Related
My ultimate goal is to run sql queries against sql-server and capture the returned data in a spreadsheet. The following code roughly reflects my current set-up and it works. The design allows me to read sql codes from text files and submit it to a sql-server. "Sub ExecuteCRUD" submits a first sql script to prepare data and dumps the result into a temp table. "Function loadRecordset" submits a relatively simple select query and captures the returned data in a recordset, which I then use to populate a spreadsheet.
There are a couple "variables" in my setup that could potentially be relevant for discussion.
My 4 set of Sql Codes
The ConnectionString (Part of my vba code)
The rest of my vba codes
a. Dbo.ConnectionString = "Provider=MSDASQL;DRIVER=SQL
Server;SERVER=myserver;UID=id;PWD=password;DATABASE=database;"
b. Dbo.ConnectionString = "Provider=SQLOLEDB;Data
Source=myserver;Initial Catalog=database;User
ID=id;Password=password;"
Initially, all I changed was the connection string. The immediate result was connection string version a works perfectly. Using b version, my setup would fail without any errors from sql-server.
Using Connection string version b, "Sub ExecuteCRUD" (data preparation step) would still work smoothly. I can verify that the temp table is created in tempdb as a result of my first sql script.
"Function loadRecordset" would run through the lines without any errors up to and including "rs.open". (I checked for errors, none whatsoever from the ado connection).
Only subsequent codes, when using the recordset to copy out the data would get an error: "Operation is not allowed when the object is closed."
Through some testing, I narrowed down the issue to the sql codes, sort of.
I have to reiterate here. My initial set of sql codes worked completely fine when using the ODBC provider. Things only went sideways using the OLEDB provider.
Using the OLEDB provider, the "offending" sql code was Use databaseABC. Furthermore, using ado, my setup submits 4 sets of sql codes to the sql server. The first set of sql codes prepares data (creating tables, inserting data, creating index, using while loops to populate data, using recursive ctes, etc). In this first set of sql codes, Use databaseABC was also included, and it would execute successfully. The other 3 set of sql codes submitted were only select queries aimed at obtaining data. When Use databaseABC was included in the select query sql codes, the operation failed without any errors. After I took out the Use databaseABC, everything would run correctly in the OLEDB provider world.
Use databaseABC is not a necessary part of the select queries. Using it saves me the trouble of specifying database names all the time in the join clauses.
At this point, my curiosity is two fold
why Use databaseABC causes failures, specifically only when using OLEDB provider, more specifically only when running select queries.
When the failure occurred, should sql-server or the driver generate any errors? What would be the proper way of checking and verifying that?
Private Dbo As New ADODB.Connection
Private rs As ADODB.Recordset
Public Sub ConnectServer()
If Dbo.State = adStateClosed Then
Dbo.ConnectionString = "Provider=SQLOLEDB;Data Source=*server*;" _
& "Initial Catalog=*database*;User ID=*id*;Password=*pwd*;"
Dbo.Open
End If
End Sub
Public Sub ExecuteCRUD(ByVal Sql As String)
On Error GoTo PANIC
Dbo.Execute Sql
Exit Sub
PANIC:
Debug.Print Err.Descript
Stop
End Sub
Public Function loadRecordset(ByVal Sql As String) As Long
On Error GoTo PANIC
Set rs = New ADODB.Recordset
With rs
.CursorLocation = adUseClient
.CursorType = adOpenStatic
.LockType = adLockReadOnly
.Source = Sql
Set .ActiveConnection = Dbo
End With
rs.Open
loadRecordset = rs.RecordCount
Exit Function
PANIC:
Debug.Print Err.Description
loadRecordset = 0
Stop
End Function
I don't have much experience with SQL Server, I use it currently to run some simple queries, and I link to SQL tables from Access where I have all my heavy queries. My goal is to run all of my queries in Access daily and then at the end write the finished tables up to SQL where my Access front end will read them (versus keeping them in my Access backend).
I've tried messing around with the code below to try and figure out how to do this, but I'm stuck at the driver and I can't find any references on how to do this with just a single table. Let's call it "PO_STATUS_TBL"
Public Sub ADOtest()
Dim ADOConn As New ADODB.Connection
Dim ADOCom As New ADODB.Command
On Error Resume Next
ADOConn.ConnectionString = "Driver =(SQL Server);DRIVER=SQL Server;SERVER=BUSINESS_BWP;Trusted_Connection=Yes"
ADOConn.Open
Debug.Print ADOConn.State
Dim db As Database
Set db = CurrentDb
'db.Execute "INSERT INTO [ODBC;DRIVER=SQL Server;ENCSQL28\BUSINESS_BWP;DATABASE=CurrentDb].SFTransfersDB ( ID, TO ) SELECT ID,TO FROM SFTransfersDB"
End Sub
I went through this process lately.
For migration you can use this tool by Microsoft. With this tool you can either migrate a table or a query to MS-SQL - or even both. Even with relations.
Simple export with this wizard and add your one table through ODBC Connector in Access. Important is to have the correct ODBC Driver.
If you are interested I resolved this issue with Parfait's advise. By using a simple INSERT statement.
INSERT INTO Dbo_PO_STATUS_ALL_TBL Select * FROM PO_STATUS_ALL_TBL
I get error 3035: 'System Resource Exceeded' when executing the following sub:
Private Sub delete_result_staging()
Dim db As DAO.Database
Dim qdf As DAO.QueryDef
Set db = CurrentDb
Set qdf = db.QueryDefs("qryWQPassthrough")
qdf.Connect = Me.con_str.Value
qdf.ReturnsRecords = False
qdf.SQL = "TRUNCATE TABLE tblWQResultStaging;"
qdf.Execute
Set qdf = Nothing
Set db = Nothing
End Sub
I am using MS Access 2013 and SQL Server 2012.
This code has worked previously; it stopped working when I refactored another sub per How to increase performance for bulk INSERTs to ODBC linked tables in Access?. I have changed the MaxLocksPerFile to 1,000,000 in the registry per other suggestions on the internet. I restarted Access and my computer, and I still get the error every time the sub fires. In the SQL Server Profiler I see no activity on the server when the code is executed. Forms and reports that point to linked tables on the same server but are connected via the GUI, rather than through VBA, still work as expected.
I figured out the issue was that I was reusing a single passthrough query multiple times in VBA. It didn't like the SQL statement that was in the query from the last time I used it in code, so it threw the error. Looks like I need to either be more careful with my earlier VBA code that uses that query, or use separate queries for every VBA sub, or both.
The weirdest thing. I have a simple procedure that I developed in Microsoft Access 2010 with a SQL Server 2012 backend. I am now trying to deploy this into production which is Access 2016 and a SQL Server 2014 backend.
I've compiled, compact and repaired in the new environment... but I can not get Access to execute this simple stored procedure. Even worse it still executes several other stored procedures fine... but a couple of them it times out and refuses to execute?
Here is my VBA and stored procedure:
Private Sub GenerateUnitKey(UnitColumns As String)
Dim Msg, Style, Title, Response As Variant
Dim lngProcessID As Long
Dim Conn As ADODB.Connection
Dim Cmd As ADODB.Command
Dim CurrentConnection As String
CurrentConnection = LinkMasterConnection()
Msg = "Are you sure you want to update the UnitKey with the selected columns?"
Style = vbYesNo + vbCritical + vbDefaultButton2
Title = "Save Campaign?"
Response = MsgBox(Msg, Style, Title)
If Response = vbYes Then
Call OpenSixHatLoader("Generating Unit Key Across Campaign Records", 1, "")
Set Conn = New ADODB.Connection
Conn.Open CurrentConnection
Set Cmd = New ADODB.Command
With Cmd
.ActiveConnection = CurrentConnection
.CommandText = "usp_GenerateUnitKey"
.CommandType = adCmdStoredProc
.CommandTimeout = 30
.Parameters.Append .CreateParameter("#UnitColumns", adVarChar, adParamInput, 4000, UnitColumns)
.Execute
End With
End If
End Sub
And stored procedure:
SET ANSI_NULLS ON
GO
SET QUOTED_IDENTIFIER ON
GO
ALTER PROCEDURE [dbo].[usp_GenerateUnitKey]
#UnitColumns AS VARCHAR(4000)
AS
SET NOCOUNT ON
DECLARE #SQL AS VARCHAR(MAX)
UPDATE tblStagingTable SET UnitKey =''
SET #SQL = 'UPDATE tblStagingTable SET UnitKey = ' + #UnitColumns + ' FROM tblStagingTable st'
EXEC(#SQL)
-- UPDATE Interests to match Staging Table
UPDATE tblInterests SET UnitKey = st.[UnitKey] FROM tblInterests i
INNER JOIN tblStagingTable st ON i.StagingTableID = st.StagingTableID
I am fairly confident there is nothing wrong with the code... as I said it worked fine in my development environment... even more I am manually able to execute the stored procedure within SQL Server. My SQL Server Native Client 11.0 connection works in executing other stored procedures... but for a couple of them it does not work. I am thinking I need to configure something within SQL Server itself or maybe within the Native Client 11.0 driver?
Unfortunately it gives no exception. I've set the CommandTimeout property to 0 and let it chug for a few hours hoping it would throw and exception to give me a clue but nothing... it just was frozen trying to execute. Any suggestions or ideas would be greatly appreciated because this one has me really stumped because it should be fine!
I would first launch SSMS, and from the SQL studio type in
Exec xxxxx ''
And ensure it runs (and use the SAME logon and connection to SSMS that you currently have for Access.
I would also consider creating a pass-though query, and saving that query in access. (set returns records = false if the sp does not return records). Then in code to run any proc, you can go:
With CurrentDb.QueryDefs("qryPass")
.SQL = "exec usp_GenerateUnitKey '" & UnitColumns & "'"
.Execute
End With
You note how simple the above code is - so if sp works from SSMS, then try the above code.
This was a difficult one that took me about 3 solid days of troubleshooting to get a solution to. Although I am not satisfied with the end solution as it should have just worked... but in the end my theory of the server being an Virtual Machine proved correct. When I deployed this exact same setup to Microsoft Access 2016 32 bit and SQL Server 2014 32 bit on a dedicated server it worked exactly as it was supposed to compared to the Azure VM and 1&1 Cloud Servers I had attempted to deploy to.
SQL Server integration with VM's is getting better from what all I have read, but apparently there is a ways to go. Maybe SQL Server needs to release a special VM version. Thank you to all those who took the time to look into this.
I have the following UDF in excel which uses ADO to connect to my MSSQL server. There it should execute the scalar udf "D100601RVDATABearingAllow".
For some reason the parameters that I try to append are not send to the sql server. At the server only:
SELECT dbo.D100601RVDATABearingAllow
arrives.
MY EXCEL UDF:
Function RVDATA(Fastener) As Long
Dim cnt As ADODB.Connection
Dim rst As ADODB.Recordset
Dim Cmd1 As ADODB.Command
Dim stSQL As String
Const stADO As String = "Provider=SQLOLEDB.1;Data ................"
'----------------------------------------------------------
Set cnt = New ADODB.Connection
With cnt
.ConnectionTimeout = 3
.CursorLocation = adUseClient
.Open stADO
.CommandTimeout = 3
End With
'----------------------------------------------------------
Set Cmd1 = New ADODB.Command
Cmd1.ActiveConnection = cnt
Cmd1.CommandText = "dbo.D100601RVDATABearingAllow"
Cmd1.CommandType = adCmdStoredProc
'----------------------------------------------------------
Set Param1 = Cmd1.CreateParameter("Fastener", adInteger, adParamInput, 5)
Param1.Value = Fastener
Cmd1.Parameters.Append Param1
Set Param1 = Nothing
'----------------------------------------------------------
Set rst = Cmd1.Execute()
RVDATA = rst.Fields(0).Value
'----------------------------------------------------------
rst.Close
cnt.Close
Set rst = Nothing
Set cnt = Nothing
'----------------------------------------------------------
End Function
When I use adCmdStoredProc the whole thing fails and in the vba debugger the properties of the recordset has a lot of "Operation is not allowed when object is closed" (may sound a bit different, the message is translated)
When I don't use adCmdStoredProc I get the message that the variable Fastener was not provided.
I think that maybe something is wrong in the way I open the recordset.
In other treads I read about using the "SET NOCOUNT ON" option, but that did not work either.
Does anyone have a idea?
Regards Lumpi
Ran into this error as well (in my case I am using a Stored Procedure to retrieve some information). I had made some changes which caused the execution to malfunction.
The error disappeared when I put SET NOCOUNT ON as the first statement of the Stored Procedure.
You do not need to SELECT the server side function, just provide its name ("[tra-CAE400-1].dbo.D100601RVDATABearingAllow") in the .CommandText property.
Also you should set the .CommandType property to "stored-procedure" (property reference on w3schools.com).
Then adodb will know that you are talking about calling a function, and not trying to send a plain sql-command.
Chances are that it will then allow you to define the parameters on the command object.
But the parameters you define on the command object should correspond exactly (in name and type) to the ones that are defined as the arguments of the function in the sql server.
An example from microsoft.com on using the command-object with a stored procedure
ADO Reference on microsoft.com
Another possible cause of this is debug statements. I just spent far too long trying to work out why this wouldn't work for me, the Proc on the database worked fine, the data it was supposed to insert was inserted, the VBA code worked fine, but there was nothing in the recordset.
Final solution was to go through the procs that had been built and remove the PRINT statements.
To test if this is the problem, run your proc on SQL Server manually, then look at the messages tab of the results, if there's anything there other than "Command(s) completed successfully." you need to eliminate those messages. "SET NOCOUNT ON" will get rid of the row count messages, but there may be others.
I'm assuming that after 5 years the OP has solved this particular problem, so this is just for anyone like me that finds this while searching for the same problem.
I also ran into this with a stored procedure. Did you SET NOCOUNT = OFF; at the bottom of your code? That is what worked for me after lots of googling. Also, if you have any other code that runs, you have to wrap it in Nocount = on/off, INCLUDING insert and update statements. You would think that an insert statement wouldn't matter but wrapping the code that way is what kept me from committing suicide today.
In our shop we often use lines like this in our stored procedures to assist with debugging:
RAISERROR('Debug message here',0,1) WITH NOWAIT;
This also breaks opening a recordset in Excel vba. I believe the complete answer for this question is, in the stored procedure:
use SET ROWCOUNT OFF
remove all PRINT statements
remove all RAISEERROR statements used for debugging (ie severity of 0)