reduce string to compare - database

I would like to know the most effective way to compare long strings, having a database sql server, such as:
long string -> compare to-> 5,000 records in database
I happened to convert the records and the string to compare, using crc32, but I'm not sure if it would be a more efficient method

If u have a database,then u can go with a DataTable/DataSet/DataReader,let's assume u are using a DataReader
Dim cmd as new SqlCommand("Select * from table",connectionstring)
Dim dr as SqlDataReader = cmd.ExecuteReader
While dr.Read
If dr(1).ToString = "MyString" Then
''''Something will happen
End if
End While

If there are no other partially identifying fields you could check then make a new field of, perhaps the first 10 characters of the string. Make an index on this field and then take the first 10 characters of string to insert as a parameter. This should reduce the number of long comparisons you need to do. Of course my selection of 10 characters is arbitrary. You would select a number that you can see from your data that would reduce the number of rows returned significantly but be as small as possible.
Private Function OKayToInsert(stringToInsert As String) As Boolean
Using connection As New SqlConnection("Your connection string")
Dim stringShort As String = stringToInsert.Substring(0, 10)
Using cmd = New SqlCommand("Select LongString From StringTable Where NewShortField = #ShortString", connection)
cmd.Parameters.Add("#ShorString", SqlDbType.NVarChar, 100).Value = stringShort
Using reader As SqlDataReader = cmd.ExecuteReader()
If reader.HasRows Then
While reader.Read
If stringToInsert = reader.GetString(0) Then
Return False
End If
End While
Return True
Else
Return True
End If
End Using
End Using
End Using
End Function

Related

All Listview data show in textbox using loop

Dim Mysqlconn = New SqlConnection
Mysqlconn.ConnectionString = "Data Source=DESKTOP-D32ONKB;Initial Catalog=Attendance;Integrated Security=True"
Dim dt As DataTable = New DataTable("studentdata")
Mysqlconn.Open()
Dim query As String
query = "select ID from studentdata where Class='" & ComboBox1.Text & "'"
Dim Command = New SqlCommand(query, Mysqlconn)
Dim dr = Command.ExecuteReader(CommandBehavior.CloseConnection)
ListView1.Items.Clear()
Dim x As ListViewItem
Do While dr.Read = True
x = New ListViewItem(dr("ID").ToString)
ListView1.Items.Add(x)
Loop
For i = 0 To ListView1.Items.Count - 1
TextBox1.Text = ListView1.Items(i).SubItems(0).Text
Next
In this code, Textbox1 is showing the last row from Listview1. My requirement is all the Listview1 data show in textbox1 one after one from Listview1. Is this possible to show in textbox1 read all data from Listview1 using loop. Thank you...
A textbox only holds one string at a time. If it's set to allow multiline strings (not clear in the question) you can separate each item with a linebreak. Otherwise you can separate the strings using a delimiter like a comma.
Dim query As String = "select ID from studentdata where Class= #class"
Using conn As New SqlConnection("Data Source=DESKTOP-D32ONKB;Initial Catalog=Attendance;Integrated Security=True"), _
cmd As New SqlCommand(query, conn)
cmd.Parameters.Add("#class", SqlDbType.NVarChar, 20).Value = ComboBox1.Text
conn.Open()
Using dr As SqlDataReader = cmd.ExecuteReader()
While dr.Read()
ListView1.Items.Add(New ListViewItems(dr("ID").ToString()))
End While
End Using
End Using
TextBox1.Text = String.Join(",", ListView1.Items.Select(Function(i) i.SubItems(0).Text))
Also note how I used a query parameter to include the combobox value in the SQL command. That's a big deal; anything else will give you trouble, usually sooner than later.
Using as loop, the proper way would be like so:
Dim lines As New List(Of String)
For i = 0 To ListView1.Items.Count - 1
lines.Add(ListView1.Items(i).Text)
Next
TextBox1.Lines = lines.ToArray()
You can't keep setting the Text property to a new value and expect the old value to hang around for no reason. You could append to the Text each time, but that is inefficient. The proper way is to create a list of the values, convert that to a String array and then assign that to the Lines property.
Note that there is no point getting the Text of the first subitem because that is the same as the Text of the item.

Do While Loop doesn't stop looping in VB.net

I'm trying to generate a random number that's not in the database. If the randomly generated number happens to already be in the database, a message box appears saying the number exists. When you click Ok, it generates another number and if it's still in the database, it will repeat the same process. With my code, it keeps showing the message box and generating a number even after it has already generated a number that's not in the database. This is my code:
Private Sub BtnOrder_Click(sender As Object, e As EventArgs) Handles BtnOrder.Click
Dim rand As New Random
Dim num As Integer
num = rand.Next(1, 30)
TxtOrder.Text = "#" + num.ToString("0000")
BtnOrder.Enabled = False
Dim connection As New OleDbConnection("Provider=Microsoft.ACE.OLEDB.12.0;Data Source=Daily Sales.accdb;")
Dim command As New OleDbCommand("SELECT [Order No] FROM [Table1] WHERE [Order No] = orderno", connection)
Dim orderParam As New OleDbParameter("orderno", Me.TxtOrder.Text)
command.Parameters.Add(orderParam)
command.Connection.Open()
Dim reader As OleDbDataReader = command.ExecuteReader()
Do While reader.HasRows = True
If reader.HasRows = False Then
Exit Do
End If
MessageBox.Show("Order number exists.", "Invalid Order Number")
num = rand.Next(1, 30)
TxtOrder.Text = "#" + num.ToString("0000")
Loop
command.Connection.Close()
End Sub
I think many things need to be changed. Some are listed in comments on your question, but here they are
You hit the database multiple times.
You don't follow the Disposable pattern which is implemented by both the connection and the command.
You had a loop whose condition looked for rows, then immediately checks for no rows to exit, which can never happen.
You don't actually check that the order number is in the result set.
You only create random numbers from 1 to 29, but if they all exist, the loop will continue forever.
You perform the database interaction on the UI thread.
You may find that using Using blocks to properly dispose of the Disposable objects helps with memory, and moving the query off the UI helps your UI remain responsive. Also, I don't see a need to alert the user that a random number has found a match in the database - just find a proper random number without alerting the user. Lastly, throw an Exception if you can't generate another order number.
Dim lowerInclusive As Integer = 1, upperExclusive As Integer = 30
Private Async Sub BtnOrder_Click(sender As Object, e As EventArgs) Handles BtnOrder.Click
BtnOrder.Enabled = False
Try
Dim nextOrderNumber = Await GetNextOrderNumber()
TxtOrder.Text = $"#{nextOrderNumber:0000}"
Catch ex As Exception
TxtOrder.Text = "N/A"
MessageBox.Show(ex.Message)
Finally
BtnOrder.Enabled = True
End Try
End Sub
Private Function GetNextOrderNumber() As Task(Of Integer)
Return Task.Run(
Function()
Dim existingOrderNumbers As New List(Of Integer)()
Using connection As New OleDbConnection("Provider=Microsoft.ACE.OLEDB.12.0;Data Source=Daily Sales.accdb;")
connection.Open()
Using command As New OleDbCommand("SELECT [Order No] FROM [Table1]", connection)
Dim reader As OleDbDataReader = command.ExecuteReader()
While reader.Read()
existingOrderNumbers.Add(reader.GetInt32(0))
End While
End Using
End Using
Dim nextOrderNumbers = Enumerable.Range(lowerInclusive, upperExclusive - lowerInclusive).Except(existingOrderNumbers).ToList()
If Not nextOrderNumbers.Any() Then Throw New Exception("All possible order numbers are already used.")
Dim rand As New Random()
Return nextOrderNumbers(rand.Next(0, nextOrderNumbers.Count()))
End Function)
End Function
Thinking about it again, the new order number never goes into the database. So when does that happen? You may want that to be an atomic operation, if this code can be used by multiple people - multiple threads. You can use a transaction to lock down the table... or perhaps just use a PK ID as part of a hash to generate some random number.
So if you're looking for unique numbers, I suggest you change the DB column to become an IDENTITY column. That way, SQL server takes care of creating unique (increasing) numbers for you.
Query the current highest order number:
SELECT MAX([Order No]) AS OrderNoCurrent FROM Table1
Query the next order number:
SELECT (MAX([Order No])+1) AS OrderNoNext FROM Table1

Pull value from SQL Server in VB.NET

I am attempting to pull values from an SQL Server table from VB.NET.
On VB Form 1, the number from NoTable, Row 1, is pulled successfully, and Label1 is updated with the value.
Dim command As SqlCommand
Dim query As String = "SELECT Number FROM NoTable"
command = New SqlCommand(query, con)
con.Open()
Dim datareader As SqlDataReader = cmd.ExecuteReader()
If datareader.Read() Then
Label1.Text = datareader.GetValue(0)
End If
datareader.Close()
On VB Form 2 I am attempting to pull the value from the second row, using:
Dim query As String = "SELECT Number FROM NoTable"
command = New SqlCommand(query, con)
con.Open()
Dim datareader As SqlDataReader = cmd.ExecuteReader()
If datareader.Read() Then
Label1.Text = datareader.GetValue(1)
End If
datareader.Close()
However, this does not work, and the label is not updated with the value from the second row.
An unhandled exception of type 'System.IndexOutOfRangeException' occurred in System.Data.dll
Additional information: Index was outside the bounds of the array."
How would I go about fixing this, so that on Form 2, the value from Row 2 is pulled, and so forth?
Thank you.
Firstly, you only get one column back from the reader, but you are indexing the columns with that 0 or 1. So you should always pass 0 to GetValue.
To index the row instead, try this. Assign a form number to each form (first line in my example) and use that to determine which record to assign to the Label. There is probably a more efficient way to do this (not returning all the records before it) but this solution should fit in your environment.
' in form # 1
Dim formNumber = 1
Dim command As SqlCommand
Dim query As String = "SELECT Number FROM NoTable"
command = New SqlCommand(query, con)
con.Open()
Dim datareader As SqlDataReader = cmd.ExecuteReader()
Dim index = 0
While index < formNumber
If datareader.Read() AndAlso index = formNumber Then
Label1.Text = datareader.GetValue(0)
End If
index += 1
End While
datareader.Close()
See https://msdn.microsoft.com/en-us/library/system.data.sqlclient.sqldatareader.getvalue(v=vs.110).aspx
And another similar question in c# Access a specific row in DataReader
Another way is to just return the row you need in the first place, without iterating over the records on the client side. Assuming there is another column with an index which is in the same order as the row you want to return, called "ID"
' in form # 1
Dim formNumber = 1
Dim command As SqlCommand
Dim query As String =
"SELECT Number FROM " & _
" (SELECT Number, Row_Number() OVER (ORDER BY ID) AS RowNumber " & _
" FROM NoTable) AS Results " & _
" WHERE Results.RowNumber = " & formNumber.ToString()
command = New SqlCommand(query, con)
con.Open()
Dim datareader As SqlDataReader = cmd.ExecuteReader()
Label1.Text = datareader.GetValue(0)
datareader.Close()
See https://msdn.microsoft.com/en-us/library/ms186734.aspx
GetValue(1) does not exist, as this would refer to a second column in the select statement. You are only asking for [Number] which would be datareader.GetValue(0)

Best method for importing data from text file into MS SQL Server

In need of your opinions. Currently developing an application in VB.NET.
I have a text file which contains more than one thousand rows. Each rows contains the data needed to be inserted into the database. A sample of a row is as follows:
0001--------SCOMNET--------0.100---0.105
At first glance, one might figured that each column was separated with a tab but actually each column was separated by blank spaces (I used '-' to denote as blank spaces because somehow could not get SO text editor to show the blank spaces).
The first column is defined by
Substring(0, 12) which is the data [0001--------]
second column
Substring(12, 12) in which the data is [SCOMNET-----]
third column is
Substring(24, 8) in which the data is [---0.100]
and last column is
Substring(32, 8) in which the data is [---0.105]
My initial though is to extract the lines for the text file and stored in as a list of strings, then while looping, do the separation of the each string item in the list with the SubString() function and insert it one by one until the end of the list. But this will no doubt takes time.
In my scenario, how can I take advantage of the SqlBulkCopy? Or if there is any other ways to approach this for a faster insert? Say;
open file
start loop
read line
separate each column in the line with substring
save in a DataTable
end loop
BCP(DataTable)
This includes a method that may be a more efficient way of reading your text file.
Sub readFixWidthTextFileIntoSqlTable()
Dim sqlConn As New SqlConnection("Connection String Goes Here")
sqlConn.Open()
Dim sqlComm As New SqlCommand
sqlComm.Connection = sqlConn
sqlComm.CommandType = CommandType.Text
sqlComm.CommandText = "INSERT INTO YourTableNameHere VALUES(#Field1, #Field2, #Field3, #Field4)"
sqlComm.Parameters.Add("#Field1", SqlDbType.Text)
sqlComm.Parameters.Add("#Field2", SqlDbType.Text)
sqlComm.Parameters.Add("#Field3", SqlDbType.Text)
sqlComm.Parameters.Add("#Field4", SqlDbType.Text)
Using IFReader As New FileIO.TextFieldParser(FileNameWithPath)
IFReader.TextFieldType = FileIO.FieldType.FixedWidth
IFReader.SetFieldWidths(12, 12, 8, 8)
While Not IFReader.EndOfData
Dim fields As String() = IFReader.ReadFields
sqlComm.Parameters("#Field1").Value = fields(0)
sqlComm.Parameters("#Field2").Value = fields(1)
sqlComm.Parameters("#Field3").Value = fields(2)
sqlComm.Parameters("#Field4").Value = fields(3)
sqlComm.ExecuteNonQuery()
End While
End Using
sqlConn.Close()
End Sub
You've got it pretty much right. This approach is one that I take a lot. Here's a bit of sample code to get you started. It is ONLY an example, there's absolutely no validation or and no consideration for Primary Keys on the table. If you want to review your question with more details of the structure of the destination table then I can make this example much more specific.
Read_File:
Dim sFileContents As String = ""
Using sRead As New StreamReader("e:\ExampleFile.txt")
sFileContents = sRead.ReadToEnd
End Using
Dim sFileLines() As String = sFileContents.Split(vbCrLf)
Connect_To_DB:
Dim sqlConn As New SqlConnection
sqlConn.ConnectionString = "Data Source=YourServerName;Initial Catalog=YourDbName;Integrated Security=True"
sqlConn.Open()
Setup_DataTable:
Dim ExampleTable As New DataTable
ExampleTable.Load(New SqlCommand("Select Top 0 * From Example_Table", sqlConn).ExecuteReader)
'This is not absolutely necessary but avoids trouble with NOT NULL columns (like keys)'
For Each dcColumn As DataColumn In ExampleTable.Columns : dcColumn.AllowDBNull = True : Next dcColumn
Save_To_DataTable:
For Each sLine In sFileLines
Dim ExampleRow As DataRow = ExampleTable.NewRow
ExampleRow("First_Column_Name") = sLine.Substring(0, 12).TrimEnd
ExampleRow("Second_Column_Name") = sLine.Substring(12, 12).TrimEnd
ExampleRow("Third_Column_Name") = sLine.Substring(24, 8).TrimEnd
ExampleRow("Fourth_Column_Name") = sLine.Substring(32, 8).TrimEnd
ExampleTable.Rows.Add(ExampleRow)
Next
Update_Database:
If ExampleTable.Rows.Count <> 0 Then
Dim sqlBulk As SqlBulkCopy = New SqlBulkCopy(sqlMvConnection)
sqlBulk.DestinationTableName = "Example_Table"
sqlBulk.WriteToServer(ExampleTable)
End If
Disconnect_From_DB:
sqlConn.Close()
Also, as commented on above and if you have access to it SSIS will do this in a jiffy.

SQL Server parameterized query with a WHERE clause with Nulls

VB.NET 2012, ADO.NET, SQL Server 2014
I setup a parameterized query that works well. I essentially read records from a DataTable that comes from a different source than my SQL Server. It's small enough that I elected to read record by record and build a query and hit the SQL Server for a match.
However I am having trouble getting a match when one of my fields is supposed to be matched for a null. I know a record exist because I can look at it in SQL Server directly and see it. With my parameterized query somehow the null is being translated improperly. I tried manually replacing the parameter #EngSerialNo with DBNull.Value and still doesn't work. Almost seems like I need two different queries depending if my DataTable value is null.
sqQry.AppendLine("SELECT CityCode,CarNum,RegNum,Event,EngSerialNum)")
sqQry.AppendLine("FROM [MyDB].[dbo].[Events]")
sqQry.AppendLine("WHERE (CityCode=#City AND CarNum=#CarNo AND RegNum=#RegNo AND Event=#Event AND EngSerialNum=#EngSerialNo)") 'this looks for a value in EngSerialNo
'sqQry.AppendLine("WHERE (CityCode=#City AND CarNum=#CarNo AND RegNum=#RegNo AND Event=#Event AND EngSerialNum IS NULL)") 'this looks for a Null in EngSerialNo
Dim cmd As New SqlCommand
With cmd
.Connection = connMyDb
.CommandType = CommandType.Text
.CommandText = sqQry.ToString
'cycle through each DataRow in the DataTable and check for returns
Dim total As Integer = 0
For Each row As DataRow In dtMain.Rows
.Parameters.Clear()
.Parameters.AddWithValue("#City", row.Item("City"))
.Parameters.AddWithValue("#CarNo", row.Item("CarNo"))
.Parameters.AddWithValue("#RegNo", row.Item("RegNo"))
.Parameters.AddWithValue("#Event", row.Item("Event"))
.Parameters.AddWithValue("#EngSerialNo", row.Item("EngSerialNo")) 'how do I get this to look for a null value when the DataTable contains a null value?
Dim rowsAffected As Integer = .ExecuteNonQuery()
total += rowsAffected
Next row
End With
Update: I ended up creating a dynamic SQL for every DataRow. Basically for each DataRow I check key fields for NULL or an actual value and create the appropriate SQL command text. I have 4 fields that could contain a NULL but for sake of simplicity I only demonstrated one here. I think the developer can follow the example to create their own query.
Dim cmd As New SqlCommand
With cmd
.Connection = connMyDb
.CommandType = CommandType.Text
'cycle through each DataRow in the DataTable and check for returns
Dim total As Integer = 0
For Each row As DataRow In dtMain.Rows
.CommandText = BuildSql(row)
.Parameters.Clear()
.Parameters.AddWithValue("#City", row.Item("City"))
.Parameters.AddWithValue("#CarNo", row.Item("CarNo"))
.Parameters.AddWithValue("#RegNo", row.Item("RegNo"))
.Parameters.AddWithValue("#Event", row.Item("Event"))
.Parameters.AddWithValue("#EngSerialNo", row.Item("EngSerialNo"))
Dim rowsAffected As Integer = .ExecuteNonQuery()
total += rowsAffected
Next row
End With
Private Function BuildSql(ByVal dr As DataRow) As String
Dim sqQry As New StringBuilder
sqQry.AppendLine("SELECT CityCode,CarNum,RegNum,Event,EngSerialNum)")
sqQry.AppendLine("FROM [MyDB].[dbo].[Events]")
If dr.Item("EngSerialNo") Is DBNull.Value Then
sqQry.AppendLine("WHERE (CityCode=#City AND CarNum=#CarNo AND RegNum=#RegNo AND Event=#Event AND EngSerialNum IS NULL)") 'this looks for a Null in EngSerialNo
Else
sqQry.AppendLine("WHERE (CityCode=#City AND CarNum=#CarNo AND RegNum=#RegNo AND Event=#Event AND EngSerialNum=#EngSerialNo)") 'this looks for a value in EngSerialNo
End If
Return sqQry.ToString
End Function
In SQL you can't compare null values, i.e. EngSerialNum = null always evaluates to false, even if the value in the field is null.
Either you can create the query dynamically so that you use is null to match the null values, or you can use an expression like this:
((EngSerialNum is null and #EngSerialNo is null) or EngSerialNum = #EngSerialNo)

Resources