I am trying to push data from Excel to SQL Server using bcp command for multiple tables. Currently I have setup my code such as it creates the text files of the data(in a loop) which needs to be pushed and then run a bcp command mentioned below
bcp dbname.schema_name.table_name out C:\table_name.txt -n -S localhost -U username -P password -b 10000
However I would like to do away with the loop and want to run a single bcp command with names of multiple tables as first argument and different text files as second arguments. I tried finding the solution for this online but couldn't come up with anything.
Any help will be much appreciated.
Just a bit of background: the reason I want to do away with the loop and run a single command is, I am observing strange behavior of my code which is when I step through my code it is able to push all the data to SQL Server, However when I run the entire process at once by clicking on the button to which macro is assigned, the code still runs but don't push any data to SQL Server.
To resolve this I tried putting doevents just after the bcp command but all went in vain, then I setup a wait for 5 seconds just after bcp command and it seems to work, but I feel setting up wait is kinda risky business I will never be sure what should be the wait time.
A single BCP command can only push data to a single table. There is no way to pass multiple table names to a single BCP command.
There's nothing wrong with a good old fashioned loop to handle the task.
Sub Button1_Click()
Dim conn As New ADODB.Connection
Dim iRowNo As Integer
Dim sCustomerId, sFirstName, sLastName As String
With Sheets("Sheet1")
'Open a connection to SQL Server
conn.Open "Provider=SQLOLEDB;Data Source=ASUSBOOK\SQL2012;Initial Catalog=ExcelDemo;Integrated Security=SSPI;"
'Skip the header row
iRowNo = 2
'Loop until empty cell in CustomerId
Do Until .Cells(iRowNo, 1) = ""
sCustomerId = .Cells(iRowNo, 1)
sFirstName = .Cells(iRowNo, 2)
sLastName = .Cells(iRowNo, 3)
'Generate and execute sql statement to import the excel rows to SQL Server table
conn.Execute "insert into dbo.Customers (CustomerId, FirstName, LastName) values ('" & sCustomerId & "', '" & sFirstName & "', '" & sLastName & "')"
iRowNo = iRowNo + 1
Loop
MsgBox "Customers imported."
conn.Close
Set conn = Nothing
End With
End Sub
Also, check out this link.
https://www.excel-sql-server.com/excel-sql-server-import-export-using-vba.htm
If the loop scenario is just too slow for you, consider saving your data sets as CSV files and do a bulk insert.
BULK
INSERT CSVTest
FROM 'c:\csvtest.txt'
WITH
(
FIELDTERMINATOR = ',',
ROWTERMINATOR = '\n'
)
GO
--Check the content of the table.
SELECT *
FROM CSVTest
GO
--Drop the table to clean up database.
DROP TABLE CSVTest
GO
https://blog.sqlauthority.com/2008/02/06/sql-server-import-csv-file-into-sql-server-using-bulk-insert-load-comma-delimited-file-into-sql-server/
Related
I'm trying to create in MS Access a pass trough query which will be connected to SQL server and use combo box from the form as a filter parameter in the WHERE statement part.
I know that connection and everything works because if I enter
SELECT * FROM mrch.Promo_Request_Base
I receive all the results.
Now when I try to enter something like
SELECT * FROM mrch.Promo_Request_Base WHERE mrch.Promo_Request_Base.Requestor_Name = 'UserABC';
then it also works.
It does not work for me if I enter SQL like this:
SELECT *
FROM mrch.Promo_Request_Base
WHERE (((mrch.Promo_Request_Base.Requestor_Name) = [Forms]![f_PromoRequest_VIEW_Header_001a]![Combo133]));
I also tried this:
SELECT *
FROM mrch.Promo_Request_Base
WHERE (((mrch.Promo_Request_Base.Requestor_Name) = [Forms]![f_PromoRequest_VIEW_Header_001a]![Combo133].Columns(0)));
[Combo133] has value 'UserABC' in it.
I would be very thankful if you could help me.
You don't.
Pass-through queries are passed to the data source as-is, and aren't parsed by Access at all. This means there's no way to use form-based parameters in a pass-through query.
Instead, use VBA to set parameters.
For how to do that, see How do I use parameters in VBA in the different contexts in Microsoft Access?. Specifically, the section on DAO applies to pass-through queries.
This also means you can't open the query for displaying. Use a form in datasheet view using VBA to set its own recordset instead. Do note that if a recordset requires parameters to be requeried, this can lead to trouble on sort/filter.
A pass-through query means it is executed server side. That server can no more go look into access and pull data then go and try and steal all your emails or financial data on your computer.
the simple solution then is to "process" or "evaluate" the expression BEFORE you send it to sql server. You can use the following:
Dim strSQL As String
strSQL = "SELECT * From mrch.Promo_Request_Base " & _
"WHERE mrch.Promo_Request_Base.Requestor_Name = '" & _
[Forms]![f_PromoRequest_VIEW_Header_001a]![Combo133] & "'"
With CurrentDb.QueryDefs("qryPass")
.SQL = strSQL
End With
' now code here to open form, or say launch report
DoCmd.OpenReport "rptPromoList", acViewPreview
Note that you have to ensure that the sql is formatted correctly for the server side.
So, in above, I am assume Requestor_Name is a text type of field. So,, that means you have to place quotes (I used single) around the expression. If that column you get from the combo box is a number, then the quotes are not required, and you would use this:
strSQL = "SELECT * From mrch.Promo_Request_Base " & _
"WHERE mrch.Promo_Request_Base.Requestor_Name = " & _
[Forms]![f_PromoRequest_VIEW_Header_001a]![Combo133]
So the other code would be the same - the only change in above was how I left out the adding of quotes (') around the expression.
Many thanks for your help.
At the end I did it differently. Im changing the PassTrough query via VBA :
Private Sub Command159_Click()
Dim db As dao.Database
Set db = CurrentDb
Dim qdf As dao.QueryDef
Set qdf = db.QueryDefs("q_PassThrough_VIEW_001a")
qdf.SQL = "SELECT * From mrch.Promo_Request_Last_Version_rpt_v " & _
"WHERE mrch.Promo_Request_Last_Version_rpt_v.F_Qtr_CD LIKE '" & _
[Forms]![f_PromoRequest_VIEW_Header_001a]![Combo145] & "%'"
qdf.Close
db.Close
Set qdf = Nothing
Set db = Nothing
End Sub
I have created a macro/some VBA to UPDATE a SQL Server table which works fine.
In short the code pulls a defined amount of records from the the table to excel and then the end user updates some specific information and then clicks update. A connection is created to the table and an SQL update statement runs which updates the relevant records.
The problem is where the user has not had to update a NULL field (NULL is in the SQL Server table but shows as 'empty' in Excel), when the use clicks update the SQL statement is forcing the NULL to an 'empty' entry.
To get round this I would like my code in the For Each statement to check if the cell/record is NULL or Empty and to skip to the NEXT row so the SQL Execute command is not carried out.
Here is the VBA in question:
cnn.Open cnnstr
Dim row As Range
For Each row In [tbl_data].Rows
uSQL = "UPDATE BREACH_DATA SET [VAL_BREACH_REASON] = '" & (row.Columns(row.ListObject.ListColumns("VAL_BREACH_REASON").Index).Value) _
& "' ,[VAL_BREACH_DETAIL] = '" & (row.Columns(row.ListObject.ListColumns("VAL_BREACH_DETAIL").Index).Value) _
& "' ,[VAL_VALID] = '" & (row.Columns(row.ListObject.ListColumns("VAL_VALID").Index).Value) _
& "' ,[VAL_NOTES] = '" & (row.Columns(row.ListObject.ListColumns("VAL_NOTES").Index).Value) _
& "' WHERE [ATD_NUMBER] = '" & (row.Columns(row.ListObject.ListColumns("ATD_NUMBER").Index).Value) & "'"
'Debug.Print uSQL
cnn.Execute uSQL
Next
cnn.Close
Set cnn = Nothing
Any suggestions
Kind Regards
Dino
You are updating SQL Server data directly with strings from a cell. This is a classic example of opening a door for injection attacks - users can do all kinds of bad, bad things to your database. But given that you fix that here is a way to check that each cell is not empty or null (I assume if one of the fields are not empty or null you want to update...):
if not
(
(isempty(row.Columns(row.ListObject.ListColumns("VAL_BREACH_REASON").Index).Value)
and isnull(row.Columns(row.ListObject.ListColumns("VAL_BREACH_REASON").Index).Value)
and do same for the other cell values....
)
then update....
I have a query against a linked table in MS Access that uses the getdate() function of SQL Server. However, I get this error when I attempt to run the query:
Undefined function GetDate in function
How do I create a linked table that allows the use of SQL Server T-SQL syntax? I see that this is called a pass through query but I don't know how to set it up to use the connection on the linked table as a pass through query.
Currently using Access 2010. The query is:
select getdate()
If it helps, I used the following vba code that generates the table link to SQL Server:
Function LinkTable(LinkedTableAlias As String, Server As String, Database As String, SourceTableName As String, OverwriteIfExists As Boolean, Username As String, Password As String)
'This method will also update the link if the underlying table definition has been modified.
If (InStr(1, LinkedTableAlias, "MSys") > 0) Then
Log "Skipping " & LinkedTableAlias
Exit Function
End If
'The overwrite parameter will cause it to re-map/refresh the link for LinktedTable Alias, but only if it was already a linked table.
' it will not overwrite an existing query or local table with the name specified in LinkedTableAlias.
'Links to a SQL Server table without the need to set up a DSN in the ODBC Console.
Dim tdfLinked As DAO.TableDef
' Open a database to which a linked table can be appended.
Dim dbsCurrent As Database
Set dbsCurrent = CurrentDb()
'Check for and deal with the scenario ofthe table alias already existing
If TableNameInUse(LinkedTableAlias) Then
'If InStr(dbsCurrent.TableDefs(LinkedTableAlias).Connect, "AccessBackup") Then
' Exit Function
'End If
If (Not OverwriteIfExists) Then
Log "Can't use name '" + LinkedTableAlias + "' because it would overwrite existing table."
Exit Function
End If
'delete existing table, but only if it is a linked table
'If IsLinkedTable(LinkedTableAlias) Then
dbsCurrent.TableDefs.Delete LinkedTableAlias
dbsCurrent.TableDefs.Refresh
'Else
' Log "Can't use name '" + LinkedTableAlias + "' because it would overwrite an existing query or local table."
' Exit Function
'End If
End If
'Create a linked table
Set tdfLinked = dbsCurrent.CreateTableDef(LinkedTableAlias)
tdfLinked.SourceTableName = SourceTableName
tdfLinked.Connect = "ODBC;DRIVER={SQL Server};SERVER=" & Server & ";DATABASE=" & Database & ";UID=" & Username & ";PWD=" & Password & ";"
On Error Resume Next
dbsCurrent.TableDefs.Append tdfLinked
If (err.Number = 3626) Then 'too many indexes on source table for Access
err.Clear
On Error GoTo 0
If LinkTable(LinkedTableAlias, Server, Database, "vw" & SourceTableName, OverwriteIfExists, Username, Password) Then
Log "Can't link directly to table '" + SourceTableName + "' because it contains too many indexes for Access to handle. Linked to view '" & "vw" & SourceTableName & "' instead."
LinkTable = True
Else
Log "Can't link table '" + SourceTableName + "' because it contains too many indexes for Access to handle. Create a view named '" & "vw" & SourceTableName & "' that selects all rows/columns from '" & SourceTableName & "' and try again to circumvent this."
LinkTable = False
End If
Exit Function
End If
On Error GoTo 0
'** Turn on error handling
On Error GoTo ErrorHandler:
tdfLinked.RefreshLink
LinkTable = True
Exit Function
ErrorHandler:
Log "refreshlink failed for " & tdfLinked.Name
LinkTable = True
I don't quite understand this statement:
How to I create a linked table that allows the use of SQL Server T-SQL
syntax?
But this is how you convert an existing MS Access querydef to a pass through query:
Go to design mode in the query, press the Query menu command, then SQL Specific then Pass Through
See this for screenshots.
http://www.mssqltips.com/sqlservertip/1482/microsoft-access-pass-through-queries-to-sql-server/
The reason why you are getting the error is that GETDATE() is not a function inside MSAccess. You probably need Now() to get the date and time or you may use Date() which provides the date
Here's a quick and dirty VBA way to create a pass-through query:
Set qdf = CurrentDb.CreateQueryDef("testqry")
' this is just your connection string
qdf.Connect = "ODBC;Driver={SQL Server};Server=MSSQL1; Database=MyDB;Trusted_Connection=Yes"
'anything here gets passed directly to and executed on the SQL Server
qdf.SQL = "select getdate()"
Set qdf = Nothing
Now you can use "testqry" as if it's any other Access query (as far as SELECTing from it goes, anyway)
Simple save your t-sql query as a pass-though
Select GetDate()
Then in VBA code, you can go:
TheSqlDate = currentdb.QueryDefs("qPass").OpenRecordset()(0)
Using ADO, and hardcoding connection strings, and the HUGE whacks of other code posted here is just a way to rack up billable hours and create world poveity. My posted solution IS ONLY ONE LINE OF CODE!
I am working on an application which backs up database tables and stores their data as CSV. When I want to restore a table back to the database I use a 'model' table which creates an existing table based on the model.
However there are multiple structures of these tables and I would like to generate the tables' 'Create' script and save it somewhere for restoring purposes.
Is there anyway for this? I am only interested to do this via coding and not sql script because we are using both SQL Server and Oracle and this process will be automated for more than 3000 tables.
In the code below I create the table based on a table model:
If ProductData.Tables(0).Rows(0).Item(0) = 0 Then 'Does not exist
'Create table based on a model
Dim sqlCopyFromModel As New SqlCommand("SELECT TOP 0 * INTO " & sFileName & " FROM s_20130702;", con)
con.Open()
Dim i As Integer = sqlCopyFromModel.ExecuteNonQuery()
con.Close()
'Enter data from CSV file
Dim sqlInsertDataFromCSV As New SqlCommand("BULK INSERT " & sFileName & " FROM '" & txtFilePath.Text.Trim & "' WITH (FIELDTERMINATOR = ',', ROWTERMINATOR = '\n', FIRSTROW=2 )", con)
con.Open()
sqlInsertDataFromCSV.ExecuteNonQuery()
con.Close()
MsgBox("success")
In Oracle you can use builtin package DBMS_METADATA where the function GET_DDL can give you the DDL ("create script") needed to create the table.
http://docs.oracle.com/database/121/ARPLS/d_metada.htm#ARPLS66885
SELECT DBMS_METADATA.GET_DDL('TABLE','EMP','SCOTT')
FROM DUAL;
(If you are trying to do something that is database independent and works both on SQL Server and Oracle, then you'd better not use "TOP 0" ;-)
This code currently is using a DROP TABLE to transfer data from Access to SQL Server, that is when it was intended for one person to use. Now it is going to be used by multiple people so I need to alter the code to APPEND. I am trying to figure out how to get it to check the destination table to see if the record already exists, if it does it will skip over it, if not it will write that employees data to the table. The SQL table is prebuilt and it has a ROW ID column and TIME STAMP column that the access table does not have in order to keep track of the records being entered.
The code currently looks like:
Public Function Update()
Dim cdb As DAO.Database, qdf As DAO.QueryDef
Dim err As DAO.Error
Const DestinationTableName = "AC_CDData"
Const ConnectionString = _
"ODBC;" & _
"Driver={SQL Server Native Client 10.0};" & _
"Server=SERVER;" & _
"Database=DB;" & _
"UID=ID;" & _
"PWD=PW;"
Set cdb = CurrentDb
Set qdf = cdb.CreateQueryDef("")
qdf.Connect = ConnectionString
qdf.SQL = _
"IF EXISTS " & _
"(" & _
"SELECT * FROM INFORMATION_SCHEMA.TABLES " & _
"WHERE TABLE_NAME='" & DestinationTableName & " '" & _
") " & _
"DROP TABLE [" & DestinationTableName & "]"
qdf.ReturnsRecords = False
On Error GoTo Update_qdfError
qdf.Execute dbFailOnError
On Error GoTo 0
Set qdf = Nothing
Set cdb = Nothing
DoCmd.TransferDatabase _
acExport, _
"ODBC Database", _
ConnectionString, _
acTable, _
"CDData", _
DestinationTableName, _
False
Exit Function
Update_qdfError:
For Each err In DAO.Errors
MsgBox err.Description, vbCritical, "Error " & err.Number
Next
End Function
The DoCmd.TransferDatabase does a complete transfer of the data from the Access table, to the SQL Server database. This function can not do partial inserts based on existing records.
What you can do, however, is export the data into a temporary new table (without dropping the existing table), followed by executing an SQL MERGE statement, to incorporate the new records of the temporary table, with the existing table.
You'll need to know how to perform SQL queries from VBA against your SQL database, and you'll need to know how to use the SQL MERGE statement. A google search will quickly give you the answer to both.
Just setup a simple standared linked table to SQL server in Access.
Since the column of the existing record is a primary key (or simply an index set = unique), then you only need ONE LINE of code to append your data.
This will work:
CurrentDb.Execute "INSERT INTO AC_CDData SELECT * FROM CDData;"
Any row that already exists will be ignored due to key (or index) violation. Thus only new non existing records will be appended with the above one line of code.
Edit:
As for the target table being a SQL linked table? Your application on startup does not care if you have some linked tables. Your application does not use nor care about such linked tables unless you use them.
The assuming here is you have a local data table. The linked table ONLY comes into play when you going to do the export. As noted you cannot export unless you are connected to SQL server. The “code” or testing if you are connected is not dealt with in this question but even if it was part of the question in both cases you still have to test for a connection and once again when you do finally determine you have a connect and you decide to export then again the linked table as per above will work.
So there is really no reason I can think of as to why having a linked table that is ONLY used during the append to the SQL table will be any kind of issue or problem.
As stated, then the one line of VBA code should suffice here.
In case of concurrent usage the process need to be changed:
DestinationTableName must be unique per session (thread) and need to be changed (from constant to variable)
Tabel with DestinationTableName neame need and need to be dropped before the export data (as it was in current solution).
DoCmd.TransferDatabase will not change - the data will be exported into the unique temp table.
For simplifying the Merge process the stored procedure can be written on the SQL Server side (with all needed to APSERT logic) with the source table name as parameter. This SP need to be called after the DoCmd.TransferDatabase completion
In this case the flow will be the following:
DestinationTableName unique name generation
Dropping this table on the SQL Server side (qdf.Execute ...)
Export data to the temp table (DoCmd.TransferDatabase ...)
Merge data in the SQL Server side (call to created SP with needed logic of update / insert)