I'm using ADO in Excel 2007 VBA to return data from SQL Server. I'm aggregating lots of Select statements using Union All.
It's working fine for up to 110 aggregated Select statements, but at 115 of them I get the ADO error "Unspecified Error". At 110 Select statements it's returning about 7,000 rows and 255 columns, so its not an unusually large ADO recordset.
I thought ADO was probably passing on an error from SQL Server just as Excel VBA passed it on from ADO (I know that the error is from ADO, not VBA, because the error is shown in the ADO Connection object's Errors collection), so I sent the problem SQL statement with the 115 Select statements to my colleague who owns the SQL Server database being queried and asked her if she could tell me what SQL Server didn't like. However, it ran fine for her in the SQL Server query tool, and returned the expected data.
So it seems it's just ADO that doesn't like it when I go to the 115 Select statements; the SQL Server database itself is fine with it.
BTW I haven't tested with 111 - 114 Select statements; I've just narrowed it down to working fine with 110 and breaking on 115.
Any idea what the problem might be and what I can do about it?
For now I'm going to chunk it into sets of 50 Select statements as a workaround, but if it's possible to send it all at once I'd like to.
Here's the relevant code:
Public Const SQL_UNION_ALL_OPERATOR As String = " UNION ALL "
(the loop that aggregates the Select statements starts here)
stSQL = _
"select * from LikeItemExtract" & _
" where Dept = " & varLikeDept & "" & _
" and sup_name = '" & varLikeSupp & "'" & _
" and VPN = '" & varLikeVPN & "'" & _
" and Loc_Key = " & varCADLoc & "" & _
" and SUPP_COLOR = '" & varLikeColor & "'"
stSQLAll = stSQLAll & stSQL & SQL_UNION_ALL_OPERATOR
(end loop here)
'remove the final " UNION ALL ":
If Right(stSQLAll, Len(SQL_UNION_ALL_OPERATOR)) = SQL_UNION_ALL_OPERATOR Then
stSQLAll = Left(stSQLAll, Len(stSQLAll) - Len(SQL_UNION_ALL_OPERATOR))
End If
If CBool(rs1.State And adStateOpen) = True Then rs1.Close
rs1.Open stSQLAll, con, adOpenStatic, adLockReadOnly, adCmdText
iRecordsetRowCount = shAggData.Cells(1).CopyFromRecordset(rs1)
Thanks,
Greg
Related
I see there are a lot of questions on this issue, but I thought I'd add to it again. I'm trying to run Pass Through queries to put the load on the server instead of wimpy Access. Almost every table I have is stored in a SQL server, but I have a large table I have to loop through and it's much faster if I copy it to a local table and then loop through it. Otherwise, everything is faster or fast enough when going through the pass through functions.
The table in question currently holds about 25k lines and if I just write a query to have access copy the local table to SQL server it takes about 1 hour. However, if I use a pass through query with insert, I can copy it over in about 47 seconds. My problem seems to come when I try to pass too long of a string of text.
Here is my function that run the pass through query
Public Sub RunPassThruQdf(sqlCode As String, Optional isTestDB As Boolean = False)
Dim qdf As QueryDef
Set qdf = CurrentDb.QueryDefs("vbaSQL")
If isTestDB Then
qdf.Connect = "ODBC;DSN=DataWarehouse_Test;Description=DataWarehouse_Test;UID=**username**;PWD=**password**;APP=Microsoft Office;DATABASE=DataWarehouse_test"
'The above line is where the error happens when I debug
Else
qdf.Connect = "ODBC;DSN=DataWarehouse;Description=DataWarehouse;UID=**username**;PWD=**password**;APP=Microsoft Office;DATABASE=DataWarehouse"
End If
qdf.ReturnsRecords = False
qdf.sql = sqlCode
Do Until InStr(qdf.sql, " ") = 0
qdf.sql = Replace(qdf.sql, " ", " ")
Loop
qdf.Execute
qdf.Close
End Sub
You'll notice that it connects with a special login (redacted) because I can't make changes to the server myself, but that user account can. And that last loop removes double spaces until everything is separated by a single space to reduce the string size as much as possible.
And this is the function that copies the local table to the server table
Public Sub LoadUnidentifiedFromLocal()
Dim rst As New RecordsetClass: rst.OpenR "tblUnidentifiedParts_Local"
Dim dtm As Date: dtm = Now
Dim baseSQL As String: baseSQL = "INSERT INTO [DataWarehouse_test].[dbo].[mT_SalesAttributeDB_UnidentifiedParts] (ID, PartNumber, FamilyNumber, AutoNote, ManualNote, Created, Updated) VALUES "
Dim sql As String
RunPassThruQdf "SET IDENTITY_INSERT [DataWarehouse_test].[dbo].[mT_SalesAttributeDB_UnidentifiedParts] ON", True
Dim i As Integer: i = 1
Do Until rst.EOF
If sql = vbNullString Then sql = baseSQL
If i Mod 500 = 0 Then
RunPassThruQdf sql, True
'Debug.Print i & ": " & Format(Now - dtm, "hh:mm:ss") & " (" & Format(Len(sql), "#,##0") & ")"
'DoEvents
sql = baseSQL
End If
Dim addSQL As String: addSQL = "(" & rst.Fields("ID") & ", '" & rst.Fields("PartNumber") & "', '" & rst.Fields("FamilyNumber") & "', '" & rst.Fields("AutoNote") & "', '" & rst.Fields("ManualNote") & "', '" & rst.Fields("Created") & "', '" & rst.Fields("Updated") & "')"
If sql = baseSQL Then
sql = sql & addSQL
Else
sql = sql & ", " & addSQL
End If
rst.MoveNext
i = i + 1
Loop
If sql <> vbNullString Then
RunPassThruQdf sql, True
Debug.Print i & " " & Format(Now - dtm, "hh:mm:ss")
End If
RunPassThruQdf "SET IDENTITY_INSERT [DataWarehouse_test].[dbo].[mT_SalesAttributeDB_UnidentifiedParts] OFF", True
rst.CloseR
End Sub
Where I have that mod 500 is me trying to figure out how many lines I can copy over until I get that System resources exceeded error. You'll probably also notice I have a special rst class I made. But just know that it mimics the normal one, I just have some special functions in there so it was my version of inheritance since VBA doesn't support that.
The first time I got it I googled and found that someone was able to fix their error by change the max lock to 1 million. And that worked for me too, until I exceeded it. DAO.DBEngine.SetOption dbMaxLocksPerFile, 1000000
I know it's not a problem with my computer, it's 6 core Xeon W-10855M with 64GB of ram. But I will see that error even after restart for a while and then it will just stop and work again until I try to send it too much and then I'm stuck seeing it for a bit. What is weird is even restarting my computer will not fix the issue. And I've tried compress and repair and that won't fix it either. So I'm not clear what changes so that it stops reporting that.
But my first question is if increasing the max locks per file helped, is there a way to clear what locks are currently there? My second question would be how big of a string can I send with a pass through, I think I saw somewhere in the neighborhood of 65k before I got the message (That was when I had it set at i mod 500.
I have created a macro/some VBA to UPDATE a SQL Server table which works fine.
In short the code pulls a defined amount of records from the the table to excel and then the end user updates some specific information and then clicks update. A connection is created to the table and an SQL update statement runs which updates the relevant records.
The problem is where the user has not had to update a NULL field (NULL is in the SQL Server table but shows as 'empty' in Excel), when the use clicks update the SQL statement is forcing the NULL to an 'empty' entry.
To get round this I would like my code in the For Each statement to check if the cell/record is NULL or Empty and to skip to the NEXT row so the SQL Execute command is not carried out.
Here is the VBA in question:
cnn.Open cnnstr
Dim row As Range
For Each row In [tbl_data].Rows
uSQL = "UPDATE BREACH_DATA SET [VAL_BREACH_REASON] = '" & (row.Columns(row.ListObject.ListColumns("VAL_BREACH_REASON").Index).Value) _
& "' ,[VAL_BREACH_DETAIL] = '" & (row.Columns(row.ListObject.ListColumns("VAL_BREACH_DETAIL").Index).Value) _
& "' ,[VAL_VALID] = '" & (row.Columns(row.ListObject.ListColumns("VAL_VALID").Index).Value) _
& "' ,[VAL_NOTES] = '" & (row.Columns(row.ListObject.ListColumns("VAL_NOTES").Index).Value) _
& "' WHERE [ATD_NUMBER] = '" & (row.Columns(row.ListObject.ListColumns("ATD_NUMBER").Index).Value) & "'"
'Debug.Print uSQL
cnn.Execute uSQL
Next
cnn.Close
Set cnn = Nothing
Any suggestions
Kind Regards
Dino
You are updating SQL Server data directly with strings from a cell. This is a classic example of opening a door for injection attacks - users can do all kinds of bad, bad things to your database. But given that you fix that here is a way to check that each cell is not empty or null (I assume if one of the fields are not empty or null you want to update...):
if not
(
(isempty(row.Columns(row.ListObject.ListColumns("VAL_BREACH_REASON").Index).Value)
and isnull(row.Columns(row.ListObject.ListColumns("VAL_BREACH_REASON").Index).Value)
and do same for the other cell values....
)
then update....
I get this error when i try to retrieve the data from database using the following piece of code.
Can someone help?
set rs = Server.CreateObject("ADODB.recordset")
sql = " SELECT * from COMPANY WHERE COMPANY_ID = " & Request.Form("CompanyId")
rs.Open sql, cnn
First of all, this is bad practice to do ad-hoc queries without using parameters. SQL Injection attack info: http://en.wikipedia.org/wiki/SQL_injection
To answer the question, though, you need to have single quotes around your varchar or char value that you are searching for.
set rs = Server.CreateObject("ADODB.recordset")
sql = " SELECT * from COMPANY WHERE COMPANY_ID = '" & Request.Form("CompanyId") & "'"
rs.Open sql, cnn
I need to insert new record into a SQL Server database, but get
Incorrect syntax error
The strange thing is when I try to query the same statement in SQL Server itself, it works properly.
The code in vb.net is as follows:
insertSql = "INSERT INTO Seg_LINE VALUES (" & OBJECTID & ", 'test" + "', '" + "test" + "','" + DrainName + "'," & UID & ")"
logger.Info("insert sql = " + insertSql)
Dim cmdInsert As New SqlClient.SqlCommand(insertSql, Sqlconnection)
cmdInsert.ExecuteNonQuery()
The OBJECTID and UID are number parameters.
I cannot figure out what's wrong with my code, I am using vb.net(vs2102).
Most likely you have a DrainName value with a single quote in it. You're lucky the query is just failing, and not executing unwanted commands on your DB server. Don't use string concatenation like that to build queries! You need to use query parameters, like this:
insertSql = "INSERT INTO Seg_LINE VALUES (#ObjectID, 'test', 'test', #DrainName, #UID)"
logger.Info("insert sql = " + insertSql)
Dim cmdInsert As New SqlClient.SqlCommand(insertSql, Sqlconnection)
'I'm guessing at these parameter types. Use the actual db types of the columns
cmdInsert.Parameters.Add("#ObjectID", SqlDbType.Int).Value = OBJECTID
cmdInsert.Parameters.Add("#DrainName", SqlDbType.NChar, 50).Value = DrainName
cmdInsert.Parameters.Add("#UID", SqlDbType.Int).Value = UID
cmdInsert.ExecuteNonQuery()
Changing the code this way will also likely fix your syntax error.
Started getting this error it seems since we upgraded to SQL Server 2008.
When inserting into the db and then returning the identity i get a 'Item cannot be found in the collection corresponding to the requested name or ordinal' error.
Here is the code:
SQL = "INSERT INTO PageFeatures(nPageFeatureFlagId,nPageFeatureFeatureId,nPageFeaturePageId) VALUES(" & nTemplateFlagId & "," & nFeatureId & "," & nPageId & "); SELECT SCOPE_IDENTITY() As nPageFeatureId;"
objrs.open SQL,objConn,1,1
nPageFeatureId = objrs("nPageFeatureId")
objrs.close
The insert is working as the record is in the db. It's not returning the id for some reason. It works fine and returns the id when running in SSMS. But ASP can't see the returned id so some reason.
Any help would be greatly appreciated!
You may have to try moving the recordset on? e.g.
SQL = "INSERT INTO PageFeatures(nPageFeatureFlagId,nPageFeatureFeatureId,nPageFeaturePageId) VALUES(" & nTemplateFlagId & "," & nFeatureId & "," & nPageId & "); SELECT SCOPE_IDENTITY() As nPageFeatureId;"
objrs.open SQL,objConn,1,1
objrs.NextRecordset
nPageFeatureId = objrs("nPageFeatureId")
objrs.close
Raj's comment about SQL injection is still relevant. :)
EDIT: Elaboration is that there are two recordsets in play here. The first is an empty one created by the insert statement, the second one is caused by everything after the semi-colon ;. This is the field you want. So initially, your objrs is attached to the first of its collection of recordsets (not rows in a recordset), you move to the NextRecordset - and you can then deal with that how you please.
Sorted:
SQL = "INSERT INTO PageFeatures(nPageFeatureFlagId,nPageFeatureFeatureId,nPageFeaturePageId) VALUES(" & nTemplateFlagId & "," & nFeatureId & "," & nPageId & ");"
objConn.execute(SQL)
Set oReturnValueRS = objConn.Execute("SELECT SCOPE_IDENTITY()")
nPageFeatureId = oReturnValueRS(0).Value
oReturnValueRS.close : set oReturnValueRS = nothing