SqlBulkCopy doesn't copy all the rows from Excel sheet - sql-server

I developed a Windows app to import Excel file and write the data into a SQL Server database table. The application works well but it starts writing from the line 27 or 30 and sometimes from line 29 of the Excel sheet. I need all the rows to be written to the database table from line 1 to line 4500.
My code:
Private Sub Button1_Click(sender As Object, e As EventArgs) Handles Button1.Click
Dim fdlg As OpenFileDialog = New OpenFileDialog
fdlg.Title = "Open File Dialog"
fdlg.InitialDirectory = "C:\"
fdlg.Filter = "All files (*.*)|*.*|All files (*.*)|*.*"
fdlg.FilterIndex = 2
fdlg.RestoreDirectory = True
If fdlg.ShowDialog() = Windows.Forms.DialogResult.OK Then
ExcelFileName = fdlg.FileName
End If
'Excel 2007
Dim ExcelConnection As New System.Data.OleDb.OleDbConnection("Provider=Microsoft.ACE.OLEDB.12.0;Data Source=" + ExcelFileName + ";Extended Properties=""Excel 12.0 Xml;HDR=No;""")
Try
ExcelConnection.Open()
Catch ex As Exception
End Try
Dim expr As String = "SELECT * FROM [Sheet1$]"
Dim objCmdSelect As OleDbCommand = New OleDbCommand(expr, ExcelConnection)
Dim objDR As OleDbDataReader
Dim SQLconn As New SqlConnection()
Dim ConnString As String = "Data Source=My_PC_Name;Initial Catalog=myDatabase;Integrated Security=True"
SQLconn.ConnectionString = ConnString
SQLconn.Open()
Using bulkCopy As SqlBulkCopy = New SqlBulkCopy(SQLconn)
bulkCopy.DestinationTableName = "ItemDetails"
Try
objDR = objCmdSelect.ExecuteReader
If objDR.HasRows = True Then
bulkCopy.WriteToServer(objDR)
MessageBox.Show("You Successfuly import the excel file")
objDR.Close()
SQLconn.Close()
End If
Catch ex As Exception
MsgBox(ex.ToString)
End Try
End Using
End Sub

SQL is per definition not ordered. So to guarantee the order you will have to add a helper column (say column A) with your row_id. YOu can then use that later to select your data ordered.

Related

MultipleActiveResultSets for SQL Server and VB.NET application

I am trying to get multiple data sets from SQL Server using a VB.NET application. The problem that every time I try to execute the query,
I get this message:
Cannot change property 'ConnectionString'. The current state of the connection is open
Then I tried to fix it by enabling MARS
<connectionStrings>
<add name="ConString"
providerName="System.Data.SqlClient"
connectionString="Data Source=my-PC;Initial Catalog=Project;Persist Security Info=True; MultipleActiveResultSets=true;User ID=user;Password=*****" />
</connectionStrings>
This is my code
Private Sub Button1_Click(sender As Object, e As EventArgs) Handles Button1.Click
Dim obj, body
obj = TextBox1.Text
body = TextBox2.Text
For Each mail In getemail()
Send_mail(mail, obj, body, getattachment(mail))
Next
MsgBox("Traitement effectué")
End Sub
Function getemail() As List(Of String)
Dim strMailTo As New List(Of String)
Dim SQL As String = "Select EMail FROM [USER] WHERE EMail Is Not NULL And MatriculeSalarie Is Not NULL And [EMail] <> '' and EtatPaie = 3 and BulletinDematerialise = 1 "
Dim cmd As New SqlCommand
Dim sqLdr As SqlDataReader
Dim dr As DataRow
Try
ConnServer()
cmd.Connection = con
cmd.CommandText = SQL
Using sda As New SqlDataAdapter(cmd)
Using ds As New DataTable()
sda.Fill(ds)
sqLdr = cmd.ExecuteReader()
For i = 0 To ds.Rows.Count - 1
dr = ds.Rows(i)
strMailTo.Add(dr("EMail"))
Next
End Using
End Using
Return strMailTo
sqLdr.Close()
Catch ex As Exception
MsgBox(ex.Message.ToString)
End Try
closeCon()
Return strMailTo
End Function
Function getattachment(email) As String()
Dim SQL As String = "Select MatriculeSalarie FROM [USER] WHERE [EMail]='" & email & "'"
Dim cmd As New SqlCommand
Dim sqLdr As SqlDataReader
ConnServer()
cmd.Connection = con
cmd.CommandText = SQL
Dim mat As String
mat = ""
Dim Dir As String = ConfigurationManager.AppSettings("path1").ToString
Dim file()
sqLdr = cmd.ExecuteReader()
While sqLdr.Read
mat = sqLdr.GetValue(sqLdr.GetOrdinal("MatriculeSalarie"))
End While
file = IO.Directory.GetFiles(Dir, mat.Substring(1) & "*.pdf")
sqLdr.Close()
Return file
End Function
If all you are going to do is show a message box in a Catch, don't do it in the database code. Let the error bubble up to the user interface code and put the Try around where the method is called.
Do not declare variables without a DataType. The button code with Option Infer on sets the type of obj and body.
Private ConStr As String = "Your connection string"
Private Sub Button1_Click(sender As Object, e As EventArgs) Handles Button1.Click
Dim obj = TextBox1.Text
Dim body = TextBox2.Text
Dim emails As New List(Of String)
Try
emails = getemail()
Catch ex As Exception
MessageBox.Show(ex.Message.ToString, "Error retrieving email list")
Exit Sub
End Try
For Each email In emails
Try
Send_mail(email, obj, body, getattachment(email))
Catch ex As Exception
MessageBox.Show(ex.Message, "Error getting attachments")
End Try
Next
MessageBox.Show("Traitement effectué")
End Sub
Parameters used by Sub and Function must have a DataType.
I don't know what you are doing here.
While sqLdr.Read
mat = sqLdr.GetValue(sqLdr.GetOrdinal("MatriculeSalarie"))
End While
Each iteration will overwrite the previous value of mat. I can only assume that you expect only a single value, in which case you can use ExecuteScalar to get the first column of the first row of the result set. Don't do anything with the data until after the connection is closed. Just get the raw data and close (End Using) the connection. Manipulate the data later.
Always use Parameters. Parameters are not treated as executable code by the database server. They are simply values. An example of executable code that could be inserted is "Drop table [USER];" where the value of a parameter belongs. Oops!
Function getemail() As List(Of String)
Dim SQL As String = "Select EMail FROM [USER]
WHERE EMail Is Not NULL
And MatriculeSalarie Is Not NULL
And [EMail] <> ''
And EtatPaie = 3
And BulletinDematerialise = 1;"
Dim dt As New DataTable
Using con As New SqlConnection("Your connection string"),
cmd As New SqlCommand(SQL, con)
con.Open()
Using reader = cmd.ExecuteReader
dt.Load(reader)
End Using
End Using
Dim strMailTo As New List(Of String)
strMailTo = (From row As DataRow In dt.AsEnumerable
Select row.Field(Of String)(0)).ToList
Return strMailTo
End Function
Function getattachment(email As String) As String()
Dim SQL As String = "Select MatriculeSalarie FROM [USER] WHERE [EMail]='" & email & "'"
Dim mat As String
Using con As New SqlConnection(ConStr),
cmd As New SqlCommand(SQL, con)
cmd.Parameters.Add("#email", SqlDbType.VarChar).Value = email
con.Open()
mat = cmd.ExecuteScalar().ToString()
End Using
Dim Dir As String = ConfigurationManager.AppSettings("path1").ToString
'Your original code was fine, no need for searchPattern.
'I added this so you could see if your search pattern was what you expected.
Dim searchPattern = mat.Substring(1) & "*.pdf"
Debug.Print(searchPattern) 'Appears in the Immediate window
Dim file = IO.Directory.GetFiles(Dir, searchPattern)
Return file
End Function

Auto complete textbox from mdf (master SQL Server database file)

I'm trying to use an autocomplete textbox, fed from a SQL Serve .mdf database file.
This is my code:
Dim cmd As New SqlCommand("Select col_name FROM college ", cn)
If cn.State = ConnectionState.Closed Then cn.Open()
Dim ds As New DataSet
Dim sqda As New SqlDataAdapter(cmd)
sqda.Fill(ds, "college")
Dim col As New AutoCompleteStringCollection
Dim i As Integer
For i = 0 To ds.Tables(0).Rows.Count - 1
col.Add(ds.Tables(0).Rows(i)("col_name").ToString())
Next
TextBox1.AutoCompleteSource = AutoCompleteSource.CustomSource
TextBox1.AutoCompleteCustomSource = col
TextBox1.AutoCompleteMode = AutoCompleteMode.Suggest
Since I am using SQL Server, the data not showing in the list.
I think its needs to use character N like
Insert data into SQL Server as:
cmd = New SqlCommand("insert into college (col_name) values (N'" & TextBox1.Text.Trim & "')", cn)
Keep your database objects local so you can control the closing and disposing. Using...End Using blocks handle this even if there is an error.
Always use parameters to avoid Sql injection.
In .net Char supports Unicode so strings (which are arrays of Char) also support Unicode. As long as the field in Sql Server is of type NVarChar. I don't see a problem
Private Sub SetUpAutoComplete()
Dim dt As New DataTable
Using cn As New SqlConnection("Your connection string")
Using cmd As New SqlCommand("Select col_name FROM college ", cn)
cn.Open()
dt.Load(cmd.ExecuteReader)
End Using
End Using
Dim col As New AutoCompleteStringCollection
For Each row As DataRow In dt.Rows
col.Add(row("col_name").ToString)
Next
'Just to check if we have some data
Debug.Print(col.Count.ToString)
TextBox3.AutoCompleteSource = AutoCompleteSource.CustomSource
TextBox3.AutoCompleteCustomSource = col
TextBox3.AutoCompleteMode = AutoCompleteMode.Suggest
End Sub
Private Sub Form1_Load(sender As Object, e As EventArgs) Handles MyBase.Load
SetUpAutoComplete()
End Sub
Private Sub InsertCollege()
Using cn As New SqlConnection("Your Connection string")
Using cmd As New SqlCommand("insert into college (col_name) values (#Name)", cn)
cmd.Parameters.Add("#Name", SqlDbType.NVarChar).Value = TextBox1.Text.Trim
cn.Open()
cmd.ExecuteNonQuery()
End Using
End Using
End Sub

Import Excel data to SQL Server table using VB.Net application

I found several articles on this topic but the issues are different. I'm trying to import data from an Excel 2016 file into SQL Server 2017 using a VB.net application, so that the end-users need not to have SQL Server installed on their machines. I ran the application in debug mode to identify the issue.
Again I read articles on various instances related to the following error:
The 'Microsoft.ACE.OLEDB.15.0' provider is not registered on the local machine
My code:
Dim TheData As DataTable = GetExcelData(OpenFileDialog1)
Private Function GetExcelData(File As OpenFileDialog) As DataTable
Dim oleDbConnection As New OleDbConnection
Try
OpenOLEDBConnections(oleDbConnection, "Provider=Microsoft.ACE.OLEDB.15.0;Data Source=" & File.InitialDirectory & _
File.FileName & "; ;Extended Properties=""Excel 12.0 XML;HDR=YES;""")
Dim oleDbDataAdapter As OleDbDataAdapter = New OleDbDataAdapter("Select '' as ID, 'AL' as Supplier, LTRIM(RTRIM(GRADE)) AS GRADE, [ship date] as shipdate, LTRIM(RTRIM(coil)) AS coil, L15*1000 AS L15, H15*1000 AS H15 FROM [sheet1$] where COIL is not null group by GRADE, [SHIP DATE], COIL, L15, H15", oleDbConnection)
Dim dataTable As DataTable = New DataTable()
oleDbDataAdapter.Fill(dataTable)
oleDbConnection.Close()
Return dataTable
Catch ex As Exception
oleDbConnection.Close()
Return Nothing
End Try
End Function
Private Sub OpenOLEDBConnections(ByRef cnData As OleDbConnection, cnDataConstr As String)
If Not cnData.State = ConnectionState.Closed Then cnData.Close()
If cnData.State = ConnectionState.Closed Then
cnData.ConnectionString = cnDataConstr
cnData.Open()
End If
End Sub
Not that this is the silver bullet, but i dont like the look of your connection string. There appears to be a ";" out of place. Try this one
Dim XLSFile As String = "C:\DATA\test.xlsx"
Dim connstring As String = "Provider=Microsoft.ACE.OLEDB.12.0;" &
"Data Source=" & XLSFile & ";Extended Properties=""Excel 12.0 Xml;HDR=YES;"""
Try installing ACE provider from the following link
https://www.microsoft.com/en-us/download/confirmation.aspx?id=13255
How to import Excel into SQL Server, check out on
https://www.red-gate.com/simple-talk/dotnet/c-programming/office-development-in-visual-studio/
Or try using this snippet
Imports System
Imports System.Collections.Generic
Imports System.Data
Imports System.Data.OleDb
Imports System.Data.SqlClient
Imports System.Runtime.InteropServices
Imports Excel = Microsoft.Office.Interop.Excel
Public Module Program
Public Sub Main()
Dim officeType = Type.GetTypeFromProgID("Excel.Application")
If officeType Is Nothing Then
Console.WriteLine("Sorry, Excel must be installed!")
Console.WriteLine("Press any key to exit")
Console.ReadLine()
Return
End If
Const fileToRead As String = "C:\TMP\FirstTest.xlsx"
Dim xlApp As Excel.Application = Nothing
Dim workbooks As Excel.Workbooks = Nothing
Dim xlWorkBook As Excel.Workbook = Nothing
Dim sheets As Excel.Sheets = Nothing
Dim t1 As Excel.Worksheet = Nothing
Try
xlApp = New Excel.Application()
Console.WriteLine($"Trying to open file {fileToRead}")
workbooks = xlApp.Workbooks
xlWorkBook = workbooks.Open(fileToRead, 0, True, 5, "", "", True, Origin:=Excel.XlPlatform.xlWindows, Delimiter:=vbTab, Editable:=False, Notify:=False, Converter:=0, AddToMru:=True, Local:=1, CorruptLoad:=0)
sheets = xlApp.ActiveWorkbook.Sheets
Dim dic = New List(Of String)()
For Each mSheet In sheets
dic.Add($"[{t1.Name}$]")
Next
Using myConnection = New OleDbConnection($"Provider=Microsoft.Ace.OLEDB.12.0;Data Source={fileToRead};Extended Properties='Excel 12.0 Xml;HDR = YES;'")
Using dtSet = New DataSet()
For Each s In dic
Console.WriteLine($" Processing {s} table")
Dim myCommand = New OleDbDataAdapter($"select * from {s};", myConnection)
myCommand.TableMappings.Add("Table", s)
myCommand.Fill(dtSet)
Next
For Each t As DataTable In dtSet.Tables
Console.WriteLine($" Table {t.TableName} has {t.Rows.Count} records")
Next
End Using
End Using
dic = Nothing
Console.WriteLine("Successfully imported!")
Console.WriteLine("After closing Console Windows start Task Manager and be sure that Excel instance is not there!")
Console.WriteLine("Press any key to exit")
Console.ReadLine()
Catch e As Exception
Console.WriteLine($"Error importing from Excel : {e.Message}")
Console.ReadLine()
Finally
GC.Collect()
GC.WaitForPendingFinalizers()
If sheets IsNot Nothing Then
Marshal.FinalReleaseComObject(sheets)
sheets = Nothing
End If
xlWorkBook.Close()
If xlWorkBook IsNot Nothing Then
Marshal.FinalReleaseComObject(xlWorkBook)
xlWorkBook = Nothing
End If
xlApp.Quit()
If xlApp IsNot Nothing Then
Marshal.FinalReleaseComObject(xlApp)
xlApp = Nothing
End If
GC.Collect()
GC.WaitForPendingFinalizers()
End Try
End Sub
End Module
After I spent a day and half, I somehow managed to resolve the issue by going through the following link and tried the option mentioned by Manju K Gowda Monday, May 14, 2012 12:14 PM
Microsoft.ACE.OLEDB.15.0 and also modified my connection string provider from OLEDB 15.0 to 12.0.
To be precise, while building the solution either in Debug or release mode, keep the Configuration Manager to x86 instead of AnyCPU

OleDbDataReader and bulkcopy.WriteToServer do not write to table

My script does not insert any data in the database table and it does not throw any error either.
The server, database and table names are correct.
The workbook is selected using filedialog (hence correct filepath) and the worksheet name seems correct too.
I have data in column A. My table is made of 2 fields:
ID (Identity, autocrement)
CustomerName
The While Loop performs 2 iterations but I have 4 records on the worksheet.
Any idea why the data does not insert in the table?
'Open the dialog box to select the file to upload
Dim fd As OpenFileDialog = New OpenFileDialog()
Dim strFileName As String
fd.InitialDirectory = "C:\"
fd.Filter = "Excel Files|*.xlsx"
fd.FilterIndex = 2
fd.RestoreDirectory = True
'declare variables - edit these based on your particular situation
Dim ssqltable As String = "tbl1"
Dim myexceldataquery As String = "select CustomerName from [A$]"
'create our connection strings
Dim sexcelconnectionstring As String = "Provider=Microsoft.ACE.OLEDB.12.0;Data Source=" & strFileName & ";Extended Properties=Excel 12.0;"
Dim ssqlconnectionstring As String = "Data Source=AA1\SQL001_DEV001;Initial Catalog=DB_Test;Integrated Security=True"
'execute a query to erase any previous data from our destination table
Dim sclearsql As String = Convert.ToString("delete from ") & ssqltable
Dim sqlconn As New SqlClient.SqlConnection(Connections.MyMainSQLServer)
Dim sqlcmd As New SqlCommand(sclearsql, sqlconn)
sqlconn.Open()
sqlcmd.ExecuteNonQuery()
sqlconn.Close()
'series of commands to bulk copy data from the excel file into our sql table
Dim oledbconn As New OleDbConnection(sexcelconnectionstring)
Dim oledbcmd As New OleDbCommand(myexceldataquery, oledbconn)
oledbconn.Open()
Dim dr As OleDbDataReader = oledbcmd.ExecuteReader()
Dim bulkcopy As New SqlBulkCopy(ssqlconnectionstring)
bulkcopy.DestinationTableName = ssqltable
While dr.Read()
bulkcopy.WriteToServer(dr)
End While
dr.Close()
oledbconn.Close()
Dim ExcelConnection As New System.Data.OleDb.OleDbConnection ("Provider=Microsoft.ACE.OLEDB.12.0;Data Source=C:\MyExcelSpreadsheet.xlsx;Extended Properties=""Excel 12.0 Xml;HDR=Yes""")
ExcelConnection.Open()
Dim expr As String = "SELECT * FROM [Sheet1$]"
Dim objCmdSelect As OleDbCommand = New OleDbCommand(expr, ExcelConnection)
Dim objDR As OleDbDataReader
Dim SQLconn As New SqlConnection()
Dim ConnString As String = "Data Source=MMSQL1;Initial Catalog=DbName; User Id=UserName; Password=password;"
SQLconn.ConnectionString = ConnString
SQLconn.Open()
Using bulkCopy As SqlBulkCopy = New SqlBulkCopy(SQLConn)
bulkCopy.DestinationTableName = "TableToWriteToInSQLSERVER"
Try
objDR = objCmdSelect.ExecuteReader
bulCopy.WriteToServer(objDR)
objDR.Close()
SQLConn.Close()
Catch ex As Exception
MsgBox(ex.ToString)
End Try
End Using

Populate Combobox from SQL Server vb.net

I'm trying to populate a combobox with data from SQL Server. This is my code so far. There are asterisks around the errors. Also, ignore the comments.
Private Sub frmOriginal_Load(sender As Object, e As EventArgs) Handles MyBase.Load
Dim connetionString As String = Nothing
Dim sqlcon As SqlConnection
Dim command As SqlCommand
Dim adapter As New SqlDataAdapter()
Dim ds As New DataSet()
Dim i As Integer = 0
Dim sql As String = Nothing
connetionString = "Data Source = RENEE\SQLEXPRESS;Initial Catalog=Stocks;Integrated Security = True"
sql = "select * from TickerSymbol"
sqlcon = New SqlConnection(connetionString)
Try
sqlcon.Open()
command = New SqlCommand(sql, sqlcon)
adapter.SelectCommand = command
adapter.Fill(ds)
adapter.Dispose()
command.Dispose()
sqlcon.Close()
cboID.DataSource = ds.Tables(0)
cboID.ValueMember = "TickerSymbol"
cboID.DisplayMember = "TickerSymbol"
Catch ex As Exception
'MessageBox.Show("Can not open connection ! ")'
End Try
End Sub
Private Sub cboID_SelectedIndexChanged(sender As Object, e As EventArgs) Handles cboID.SelectedIndexChanged
Dim dr As SqlDataReader
Dim command As New SqlCommand *(queryString, connection)*
Dim dataReader As SqlDataReader = command.ExecuteReader()
Dim sqlcon As SqlConnection
Dim cmd As SqlCommand
sqlcon = New SqlConnection
sqlcon.ConnectionString = "Data Source = RENEE\SQLEXPRESS;Initial Catalog=Stocks;Integrated Security = True"
Try
sqlcon.Open()
cmd = New SqlCommand
cmd.CommandText = " select * from TickerSymbol where TickerSymbol = '" & cboID.Text & "'"
cmd = New SqlCommand(cmd.CommandText, sqlcon)
dr = cmd.ExecuteReader
While dr.Read()
'TxtID.Text = dr.GetInt32(0)'
'TxtSN.Text = dr.GetString(1)'
'TxtGender.Text = dr.GetString(2)'
'TxtPhone.Text = dr.GetInt32(3)'
'TxtAdrress.Text = dr.GetString(4)'
lblCompanyName.Text = dataReader.GetString(1)
lblPurchasePrice.Text = dataReader.GetSqlMoney(2)
lblQtyPurchased.Text = dataReader.GetInt32(3)
lblPurchaseDate.Text = dataReader.GetDateTime(4)
End While
sqlcon.Close()
Catch ex As SqlException
MessageBox.Show(ex.Message)
End Try
sqlcon.Dispose()
End Sub
Please use parameterized queries as this will format values properly e.g. apostrophes in text will escape properly with parameters while without you must handle them, dates will be formatted properly too. Code is much cleaner also.
Example, syntax for Framework 3.5 and higher. If a connection string is used more than once then consider placing it in a private variable or under My.Settings under project properties.
Using cn As New SqlConnection With {.ConnectionString = "Data Source = RENEE\SQLEXPRESS;Initial Catalog=Stocks;Integrated Security = True"}
Using cmd As New SqlCommand With {.Connection = cn, .CommandText = "select * from TickerSymbol where TickerSymbol = #TickerSymbol"}
cmd.Parameters.AddWithValue("#TickerSymbol", cboID.Text)
cn.Open()
Dim dr As SqlDataReader = cmd.ExecuteReader
If dr.HasRows Then
While dr.Read
'
'
'
End While
End If
End Using
End Using

Resources