Importing more than 1000 rows from Excel to SQL-server - sql-server

The process is run in the environment of Excel VBA 2010 and MS SQL Server 2008.
Assume that there is a simple one column data with 1500 rows in an Excel-sheet and we want to export it to the database with SQL-queries in VBA code (SQL procedure in VBA exports maximum 1000 rows at once in default mode).
There is one limitation in this problem: the export procedure must be with dbclass-connection instead of ADODB connection. (The code-owner is not me. The code-owner is using dbclass for a quite big VBA code, so probably he wouldn't accept to change the whole code).
I found an option like lngRecsAff for ADODB.Connection which is used like:
Sub test()
Dim cn As ADODB.Connection
Dim strSQL As String
Dim lngRecsAff As Long
Set cn = New ADODB.Connection
cn.Open "Provider=Microsoft.Jet.OLEDB.4.0;Data Source=C:\test.xls; Extended Properties=Excel 8.0"
strSQL = "Insert INTO [odbc;Driver={SQL Server};Server=SQL09;Database=Tom;UID=userID;PWD=password].tbl_test1 Select * FROM [Sheet1$]"
cn.Execute strSQL, lngRecsAff
cn.Close
Set cn = Nothing
End Sub
I tried to implement that lngRecsAff in my dbclass execution like:
Sub test()
Dim connOk As Boolean
Dim rs As New ADODB.Recordset
Set dbclass = New clsDB
Dim Value1() As Variant
Dim lngRecsAff As Long
Dim strSQL as String
Dim mstrErr as Boolean
dbclass.Database = "database_name"
dbclass.ConnectionType = SqlServer
dbclass.DataSource = "server_name"
dbclass.UserID = Application.UserName
connOk = dbclass.OpenConnection(False, True)
If connOk = False Then
MsgBox "Unsuccessful connection!"
Else
MsgBox "Successful connection"
End If
strSQL = "INSERT INTO [dbo].[table1](Column1) Values('" & Value1 & "')"
mstrErr = dbclass.ExecuteSQL(strSQL, lngRecsAff) ' The result mstrErr is a Boolean
' Some closing options here
End Sub
I got en error like lngRecsAff is not suitable for my ExecuteSQL procedure. Normally my execution mstrErr = dbclass.ExecuteSQL(strSQL) works without any problem.
Maybe I can do the SQL-procedure with a for-loop, then I can send the data in small pieces. But I want to find a more efficient, "nicer" solution which sends the whole array at once.
So is there any special option for dbclass which can send more than 1000 rows from Excel to the database?

You can query the Excel-Sheet directly using liked server.
In your Management Studio click to "Server Objects" and then right click onto "linked server". There you'll be able to add a new linked server.
If you need further help you can find tuorials easily. One is here:
https://www.mssqltips.com/sqlservertip/2018/using-a-sql-server-linked-server-to-query-excel-files/

Related

How to execute stored procedure which insert/update records using VBA?

I created many spreadsheets which use stored procedures. I have a custom VBA code which works fine when I try to get data from SQL. However, today I wanted to execute parametrized stored procedure which inserts and updates data on a database table. When I run macro no errors show up, however there's no insert/update action on database. I have no idea why. I established SQL connection in my workbook (myConn) as I do everytime I need to connect with SQL so it's correct for sure. This is my standard VBA code:
Sub SaveData()
Dim myValue As Double
myValue = Sheets("XYZ").Range("valueToSave").Value
With ActiveWorkbook.Connections("myConn").OLEDBConnection
.CommandText = Array( _
"EXEC DB.[dbo].[myProc] '" & myValue & "'")
End With
ActiveWorkbook.Connections("myConn").Refresh
End Sub
I need to insert data into column of decimal(6,4) type (in SQL table). myProc does it perfectly when I run it manually via SSMS but not here using VBA code. valueToSave is an Excel range which stores one decimal value (for example: 23,56, 11,21 etc.). When I run macro nothing happens. When I run macro and go to 'Connection Properties' > 'Definitions' > 'Command Text' then I can see there's a procedure with parameter (EXEC DB.[dbo].[myProc] '11,23'). So my acode above seems working but not executing stored procedure.
Has it something to do with data type? Honestly, I tried with other VBA types: String, Variant, Integer but it's not working. I also changed data type of that column in SQL table (to varchar, int etc.) but it also doesn't work. The most interesting thing is that the code above works fine when I withdraw data from db, it doesn't work when need to insert/update data.
PS. I guess I added all required refrences:
Using ADODB
Option Explicit
Sub SaveData()
Const PROC_NAME = "DB.dbo.myproc"
Dim wb As Workbook
Dim ado As ADODB.Connection, cmd As ADODB.Command
Dim sCon As String, v As Single
Set wb = ThisWorkbook
v = wb.Sheets("XYZ").Range("valueToSave").Value
' get connection string
sCon = wb.Connections("myConn").OLEDBConnection.Connection
sCon = Replace(sCon, "OLEDB;", "")
' open connection
Set ado = New ADODB.Connection
ado.Open sCon
Set cmd = New ADODB.Command
With cmd
.ActiveConnection = ado
.CommandType = adCmdStoredProc
.CommandText = PROC_NAME
.Parameters.Append .CreateParameter("P1", adDecimal, adParamInput)
With .Parameters(0)
.NumericScale = 2
.Precision = 18
.Value = v
End With
.Execute
End With
ado.Close
MsgBox PROC_NAME & " " & v, vbInformation, "Done"
End Sub

Speed up code for referencing / importing data from SQL Server into Excel

I have a problem here that I will probably need to solve in several iterations, this being the first. maybe you have an idea:
I need to speed up regular updates from 4 views in an SQL database on Azure to an Excel Worksheet. There are probably a lot of moving parts here, among them: The code, my connection to the internet, the service that provides me with a static IP address, the fact that I reference views and not tables, and the (rather basic) service level I booked in Azure.
What my code does is simple: it opens a connection, updates 4 worksheets from 4 views (3 of them with 5 - 10 rowns of data, 1 with about 2.000), and closes the connection again.
This takes up to 30 seconds, which seems an awfully long time.
I would like to make sure it is not my code that slows this down. My first attemt was to use Powerquery for the connections, call the connection, then set autofilters with VBA to have only the rows visible that I needed. That took ages.
My second try is: not use Powerquery, get rid of the autofilters, and use VBA instead to pass a SELECT to the SQL Server, so the work of selecting the data is done there, and send over only what I need. But it seems the difference that this makes is not really significant.
The code I use is:
Dim WS1 As Worksheet
Set WS1 = Worksheets("Eingabe")
Dim SelectedCustomer As Range
Set SelectedCustomer = WS1.Range("C39")
Dim cn As ADODB.Connection
Dim SQLStr1 As String
Dim rs1 As ADODB.Recordset
Set rs1 = New ADODB.Recordset
Dim SQLStr2 As String
Dim rs2 As ADODB.Recordset
Set rs2 = New ADODB.Recordset
Dim SQLStr3 As String
Dim rs3 As ADODB.Recordset
Set rs3 = New ADODB.Recordset
Dim SQLStr4 As String
Dim rs4 As ADODB.Recordset
Set rs4 = New ADODB.Recordset
'Open a connection to SQL Server - using the connection string provided by Azure
Set cn = New ADODB.Connection
cn.Open "Driver={ODBC Driver 17 for SQL Server};Server=tcp:server-
displacethis.database.windows.net,1433;Database=my-database;Uid=my-userID;Pwd=my-
password;Encrypt=yes;TrustServerCertificate=no;Connection Timeout=30;"
'First table
SQLStr1 = "SELECT CustomerID,CustomerUniqueName,CustomerFirstName,CustomerLastName,TransactionDate,EmissionenNeukauf,EmissionenSecondhand,600 As DurchschnittDE, 300 As Paris2030 FROM vAccumulatedEmissionsByCustomer WHERE CustomerUniqueName = '" & SelectedCustomer & " ' "
rs1.Open SQLStr1, cn, adOpenStatic
'Dump to spreadsheet
With Worksheets("vAccumulatedEmissions").Range("A2:I100")
.ClearContents
.CopyFromRecordset rs1
'Do some column formatting
Worksheets("vAccumulatedEmissions").Range("F:F").NumberFormat = "#,##0.00"
'Tidy up
rs1.Close
Set rs1 = Nothing
End With
'Then the next table follows. The connection os closed after the last table:
cn.Close
Set cn = Nothing
My questions is, very simply, is this highly ineffective code, and could that be the reason for the long time it runs, or should I look somewhere else to speed up the whole process?
Thank you in advance,
Ulrich

Update VBA to SQL Server

I'm trying to retrieve and update rows in Excel/SQL and found this script which works but I tried to make it update as soon as I enter the "existencia" value instead of hitting run on the update macro, found the Worksheet change function but I'm not sure how to add it to this macro.
Private Sub Worksheet_Change(ByVal Target As Range)
Dim con As New ADODB.Connection
Dim cmd As New ADODB.Command
Dim rst As New ADODB.Recordset
Dim i As Long
Dim vDB As Variant
Dim Ws As Worksheet
con.ConnectionString = "Provider=SQLOLEDB; data source=LAPTOP\SQLEXPRESS;initial catalog=Inventario;Integrated Security=SSPI;"
con.Open
Set cmd.ActiveConnection = con
Set Ws = ActiveSheet
'The assumption is that the data on the Excel sheet is listed from a1 Cell, including fields.
vDB = Ws.Range("a1").CurrentRegion
For i = 2 To UBound(vDB, 1)
cmd.CommandText = "UPDATE Productos SET Existencia='" & vDB(i, 4) & "' WHERE id_cod=" & vDB(i, 1) & " "
cmd.Execute
Next i
con.Close
Set con = Nothing
End Sub
I have used the Worksheet_Change event to do a lot of things, but I have never ever used Worksheet_Change to update data in a SQL Server table. This doesn't really make sense, if you think about it. You can make all kinds of changes in an Excel file, all day long. People do this all the time. When you are totally done with your updates, changes, or whatever, you can push the data to a table in a database. You should not do this intermittently. Otherwise, you will have to do all kinds of cleanup in your database table(s). You don't want to create this kind of overhead for yourself.

getting a column value from table using vb6

I'm trying to read a particular column value from a SQL result table. I know we use RowCount in c#. But I don’t know how it is done in vb6.0
For example a c# program code:
adapter.Fill(ds);
adapter.Dispose();
con.Close();
rowCount = ds.Tables[0].Rows.Count;// ds is dataset and I read that record set is used instead of dataset
if (rowCount > 1)
{
ab = ds.Tables[0].Rows[0][3].ToString();
ad = ds.Tables[0].Rows[0][8].ToString();
}
In VB6 you have a choice of ADO, DAO or RDO. ADO is newer of the three technologies, and the one MS recommends.
ADO Example
Sub Example()
Dim cn As ADODB.Connection
Dim rs As ADODB.Recordset
' Ready objects for use.
Set cn = New ADODB.Connection
Set rs = New ADODB.Recordset
' Connect.
cn.Open "Driver={SQL Server};Server=My_Server_Name;Database=Master;Trusted_Connection=yes;"
' Fetch a recordset.
rs.Open "SELECT TOP 10 Name FROM sys.Objects", cn, adOpenStatic, adLockReadOnly
' Display value, and total recordcount.
MsgBox rs.Fields(0).Value
MsgBox rs.RecordCount
' Close and release objects.
rs.Close
cn.Close
Set rs = Nothing
Set cn = Nothing
End Sub
The ADO Recordset object has a RecordCount property. Watch out! Certain cursor types do not populate this property. See the link for more on this.
ConnectionStrings.com is a great resource for finding the right connection string for you.
For this example to work you will need to add a reference to the Microsoft ActiveX Data Objects library.

How do I send data from Excel to SQL temp tables for processing using VBA?

My group uses an Excel macro-based tool to do some serious number crunching. I would like to move the number-crunching bit to SQL because using VBA has become intolerable due to the runtime. Users need to be able to use Excel as the interface and also need to be able to run their macros simultaneously as the Excel workbook is self-contained. I've been testing my plan to call an SQL stored procedure from VBA to pull the data from Excel into SQL temp tables, crunch it, and send it back to Excel. I'm able to pull the data from Excel if I run my SP in SQL Management Studio. Here is the SP:
ALTER PROCEDURE sp_test_import
-- Add the parameters for the stored procedure here
#path varchar(1000)
AS
BEGIN
SET NOCOUNT ON;
declare #SQL varchar(2500);
set #SQL = '
select *
from openrowset(''Microsoft.ACE.OLEDB.12.0'',
''Excel 12.0 Macro;
Database='+#path+''',
[Sheet1$]); '
create table [#test](
col1 varchar(15),
col2 varchar(15),
col3 varchar(15),
col4 varchar(15)
)
insert into #test
exec(#SQL)
select * from #mb_test
END
So that works fine. I then try to call this SP from the Excel file containing the data.
Option Explicit
Sub ado_test()
Dim adoConnection As ADODB.Connection
Dim adoRecordset As ADODB.Recordset
Dim connectString As String
Dim strSQL As String
Dim sPath As String
Worksheets("Sheet1").Select
sPath = ThisWorkbook.FullName
'-Create a new ADO connection --
Set adoConnection = New ADODB.Connection
'-Create a new ADO recordset --
Set adoRecordset = New ADODB.Recordset
'-Build our connection string to use when we open the connection --
connectString = "DRIVER=SQL Server;SERVER=MyServer;Trusted_Connection=yes;DATABASE=testDB"
adoConnection.ConnectionTimeout = 20
adoConnection.CommandTimeout = 20
adoConnection.Open connectString
strSQL = "EXEC testDB.dbo.sp_test_import " & vbCr _
& "#path = " & "'" & sPath & "'"
adoRecordset.Open strSQL, adoConnection
End Sub
The code hangs on the 'adoRecordset.Open' call. If I instead pass a path to a separate Excel file in the variable #path, then everything works swimmingly. Is there a simple way that I can make this SP call from the same workbook? I'm not worried about security since the SQL db will be a dedicated structure for pulling in and processing temporary data. I just need users to be able to run their Excel tools whenever they want to, so I don't want to use permanent tables in the DB in case their respective inputs get mixed up together.
Everything I've found online deals with ASP or ISS and I know nothing about ASP and ISS doesn't seem like the right solution to my particular problem. I could have VBA pass the data to external text files and then pass the paths to those text files to the SQL SP, but if there is a cleaner solution then I would like to know about it. Thanks in advance!
I think it is because you are passing in strSQL, a String data type, as the first parameter of the .Open method, but the Open method requires a Command object (according to MSDN).
What you'll want to do is declare an ADODB.Command object and pass that through. I've modified your code to do this:
Option Explicit
Sub ado_test()
Dim adoConnection As ADODB.Connection
Dim adoRecordset As ADODB.Recordset
Dim adoCommand As ADODB.Command
Dim connectString As String
Dim strSQL As String
Dim sPath As String
Worksheets("Sheet1").Select
sPath = ThisWorkbook.FullName
'-Create a new ADO connection --'
Set adoConnection = New ADODB.Connection
'-Create a new ADO recordset --'
Set adoRecordset = New ADODB.Recordset
'-Create a new ADO command --'
Set adoCommand = New ADODB.Command
'-Build our connection string to use when we open the connection --'
connectString = "DRIVER=SQL Server;SERVER=MyServer;Trusted_Connection=yes;DATABASE=testDB"
adoConnection.ConnectionTimeout = 20
adoConnection.CommandTimeout = 20
adoConnection.Open connectString
strSQL = "sp_test_import"
With adoCommand
.ActiveConnection = adoConnection
.CommandText = strSQL
.CommandType = adCmdStoredProc
.Parameters.Refresh
.Parameters(1).Value = sPath
End With
Set adoRecordset = adoCommand.Execute
If adoRecordset.EOF = False Then ThisWorkbook.Worksheets("YourSheetName").Cells(1, 1).CopyFromRecordset adoRecordset
'--adoRecordset.Open adoCommand, adoConnection'
'--Close the database connection'
adoConnection.Close
End Sub
More information on the Command object.
I also added how I get values from SQL Server into the Excel workbook using the CopyFromRecordset method.

Resources