I am trying to figure out a way to load a very large .txt file and thought if I break it into sections (Listboxes) it would load faster and be easier to manipulate with less lag. Or is there a way to OFD with a background worker?
Here is how I am loading the .txt
TextBox1.Text = ""
OpenFileDialog1.Title = "Load File"
OpenFileDialog1.InitialDirectory = "C:temp"
OpenFileDialog1.ShowDialog()
Dim path As String = OpenFileDialog1.FileName
TextBox1.Text = path
Dim lines() As String = IO.File.ReadAllLines(TextBox1.Text)
I can go in and mark every 1/4 of the .txt with a delimiter if that would help?
I was thinking if I iterate through XX amount of lines then Next listbox etc. Maybe some form of items.count in a if not statement? My mind is going in circles please aim me in the best direction. My file is 25.MB and growing slowly. Notepad++ is the only thing handling it well ATM.
ListBox1.Items.Add(lines(1 - 10000))
throws an error ("Outside array index or similar")
ListBox1.Items.Add(lines(10000))
Loads the single line
Probably something along the lines of this. This is not 100% accurate code. But to give you an idea.
Dim dt As New DataTable()
Dim lines As New List(Of [String])()
lines = New List(Of [String])(File.ReadLines(ofDialog.FileName))
Task.Run(Function()
Dim options As New ParallelOptions()
options.MaxDegreeOfParallelism = CInt(1)//Number of threads to spawn
Parallel.ForEach(lines, options, Function(line)
dt.Rows.Add()
End Function)
Me.Invoke(DirectCast(Sub() listview.DataSource = dt, MethodInvoker))
End Function)
Related
I am trying to put data received from an API call that is comma-delimited into a datatable.
The data comes in something like this:
Name,ID,Date,Supervisior CRLF
Joe,123,1/1/2020,George CRLF
Mike,456,2/1/2020,George CRLF
Dan,789,4/1/2021,George
If there is only one row of data then my code works the data displays on screen just fine.
If there is more than one row I get an error "Input array is longer than the number of columns in this table."
I tried doing a split on comma and environment new line (also vbCRLF); none of those resolved the issue.
Any ideas on how I can resolve this?
Here is my code:
Dim vartable As DataTable = New DataTable()
vartable.Columns.Add("Name", GetType(String))
vartable.Columns.Add("ID", GetType(String))
vartable.Columns.Add("Date", GetType(String))
vartable.Columns.Add("Supervisior", GetType(String))
Dim inputstring As String
inputstring = (apiresponse) 'redacted API code as it works fine If I just display 'raw data to text field
Dim rowData As String() = inputstring.Split(New Char() {",",Environment.NewLine})
vartable.Rows.Add(rowData) 'this is where I get the input array error if 'more than one row of 'data
GridView1.DataSource = vartable
GridView1.DataBind()
It looks like you're expecting the Split() function to act on each of the delimiters separately, so you get an array of arrays, with each element in the outer array holding one line/row. This is not how it works.
You need to separate the lines first, and then in a loop for each line separate the contents by comma. The test way to do this is NOT by calling Split(). Instead, you can use a StringReader (which is different from StreamReader):
Using rdr As New StringReader(apiresponse)
Dim line As String = rdr.ReadLine()
While line IsNot Nothing
Dim rowData As String() = line.Split(","c)
vartable.Rows.Add(rowData)
line = rdr.ReadLine()
End While
End Using
But I would be surprised to learn the code to access the API doesn't also need to deal with streams at some point, even if you don't see it directly, meaning there exists a possible version of this that is even more efficient from using StreamReader connected to the api response directly.
For fun, since it's been a while since I've had to do this in VB, I made this extension module:
Public Module TextExt
<Extension()>
Public Iterator Function AsLines(data As TextReader) As IEnumerable(Of String)
Dim line As String = data.ReadLine()
While line IsNot Nothing
Yield line
line = data.ReadLine()
End While
End Function
<Extension()>
Public Iterator Function AsLines(data As String) As IEnumerable(Of String)
Using rdr As New StringReader(data)
For Each line As String In AsLines(rdr)
Yield line
Next
End Using
End Function
End Module
Which would let me write it like this:
For Each row As String() In apiresponse.AsLines().Select(Function(ln) ln.Split(","c))
vartable.Rows.Add(row)
Next
Finally, I need to add my customary warning about how it's a really bad idea to use .Split() as a CSV parser.
We have images stored on a DB and they are being used to replace an image within a Word document - that bit works perfectly, except where the replacement image is portrait and it's replacing a landscape one, so I'm trying to get the metadata to determine how the image is orientated using this function
Public Function GetImageTags(ImageFile() As Byte) As String()
Try
Dim vReturnArray() As String = Nothing
Using MS As New System.IO.MemoryStream(ImageFile)
Dim vDecoder As BitmapDecoder = BitmapDecoder.Create(MS, BitmapCreateOptions.None, BitmapCacheOption.Default)
Dim vFrame As BitmapFrame = vDecoder.Frames(0)
Dim vMetadata As BitmapMetadata = TryCast(vFrame.Metadata, BitmapMetadata)
If vMetadata IsNot Nothing And vMetadata.Keywords IsNot Nothing Then
vReturnArray = vMetadata.Keywords.ToArray()
End If
End Using
Return vReturnArray
Catch ex As Exception
EmailError(ex)
Return Nothing
End Try
End Function
...but it throws the toys out with...
This codec does not support the specified property.
at System.Windows.Media.Imaging.BitmapMetadata.GetQuery(String query)
at System.Windows.Media.Imaging.BitmapMetadata.get_Keywords()
...at BitMapMetadata.Keywords. Any idea how I can overcome this and get the keywords?
Thank you
================ UPDATE ================
It appears that the error, and I also tried...
vReturnArray = TryCast(vMetadata.GetQuery("System.Keywords"), String())
... is only returned for some images, but all that I tried returned Nothing for the String()
There is a really good EXIF class on Code Project that is easy to implement, either with a string link to the file
Dim vEXIF As New ImageEXIF(ImagePath)
Dim vOrientation As Integer = vEXIF.Orientation
or as BitMap
Dim vOrientation As Integer = 0
Using vBitmap As System.Drawing.Image = System.Drawing.Image.FromStream(New IO.MemoryStream(ImageFile))
Dim vEXIF As New ImageEXIF(vBitmap)
vOrientation = vEXIF.Orientation
End Using
It would not be difficult to add another Sub to the class for Byte(), but the above conversion is quite straightforward and the class should work with all image types.
You could use MetadataExtractor to access the image metadata.
Check for the presence of ExifDirectoryBase.TagOrientation on any of the contained Exif directories.
Something like this (sorry it's C# as I don't know VB.NET):
var orientation = ImageMetadataReader.ReadMetadata(imagePath)
.OfType<ExifSubIfdDirectory>()
.Select(d => d.GetObject(ExifDirectoryBase.TagOrientation))
.First(o => o != null);
I'm a beginner trying to learn VB.NET and I'm not quite sure how I'll explain this, but I'll give it a shot. Basically, I've written a txt file with about 10 lines of data in CSV form.
For example:
John, 10, 14
Michael, 14, 27
Billy, 13, 45
etc, etc....
I just want to be able to read and edit particular lines - not necessarily add new lines.
Just wondering if someone could just outline how I'd go about this - not asking anyone to write the program for me. I just don't know what to do and I couldn't understand other answers I've found on SO that attempted to solve the same problem. I don't know if I'm just a bit dense or something so it'd be great if someone could perhaps give a simple, 'dumbed-down' outline of what I need to do.
Thank you.
as stated in the comments you usually read all the file into memory, manipulate it and write it all back.
dotnet has a method that puts a file content in an array (line by line)
if you need to access individual cells in the CSV you probably want to run the split method on each line and join to merge them back together.
the split/join methods are either an method of the string/array object or its in the Strings namespace
a method for writing the file back is also in the System.IO Namespace
You don't asking to write the code for you, I can understand you but I think there is no way to show how can you do that. I used split/join methods as #weberik mentioned. It is a simple console application:
Public Class Sample
Public Shared Sub Main()
CreateExampleFileIfNotExists()
'lines variable is an "Array"
Dim lines() As String = IO.File.ReadAllLines("Example.txt")
'write items to the console
ShowItems(lines, "Values before edit:")
'edit the items
EditItems(lines)
'write edited items to the console
ShowItems(lines, "Values after edit:")
'save changes?
Save(lines)
'finish
Console.WriteLine("Press any key to exit.")
Console.ReadKey()
End Sub
Public Shared Sub ShowItems(lines() As String, header As String)
Console.WriteLine(header)
Dim headers() As String = {"Name:", "Value1:", "Value2:"}
WriteItems(headers)
For Each line In lines
WriteItems(line.Split(","))
Next
End Sub
Public Shared Sub EditItems(lines() As String)
For i As Integer = 0 To lines.Length - 1
Dim line As String = lines(i)
Dim values() As String = line.Split(",")
'edit the item
values(0) = "Edited " & values(0)
values(1) += i
values(2) *= i
lines(i) = String.Join(",", values)
Next
Console.WriteLine() 'empty line
End Sub
Public Shared Sub WriteItems(itemValues() As String)
Dim line As String = ""
For Each item In itemValues
line &= item & vbTab & vbTab
Next
Console.WriteLine(line)
End Sub
Public Shared Sub CreateExampleFileIfNotExists()
If Not IO.File.Exists("Example.txt") Then
IO.File.WriteAllLines("Example.txt", {"John,10,14", "Michael,14,27", "Billy,13,45"})
End If
End Sub
Public Shared Sub Save(lines() As String)
Console.WriteLine(vbCrLf & "Do you want to save the changes? Y/N")
Dim result = Console.ReadKey()
Console.WriteLine()
Select Case result.Key
Case ConsoleKey.Y
IO.File.WriteAllLines("Example.txt", lines)
Console.WriteLine("The changes has been saved.")
End Select
End Sub
End Class
I am newer to Visual Basic and I am trying to get into file reading. From what I've found online I haven't found anything that wasn't entirely confusing to me.
I am trying to make a simple game to test some dialogue things and I want the options of what you can say to be predetermined. Right now I have a few sentences in arrays, but what I really want to do is just type everything into a text file with a different item on each line and then convert each line to a separate item of an array.
What I have right now is:
Dim convoStarters() As String = {"Hello", "Who are you?", "Who the hell are you?"}
What I want to do is take information from a text file organized like so:
Hello
Who are you?
Who the hell are you?
and put each line into an array that looks exactly like the one I have above (except of course, I'd add more things to the text file).
Thank you for helping a new guy out, have a nice day.
First of all you have to find out, how many elements are necessary for your array.
By counting the numbers of '","' + 1 you have the numbers of elements existing in your string.
Please have a look in string methods like instr(), mid(), left(), rtrim(), ltrim() ..
With the number of elements you found you can REDIM array, or REDIM PRESERVER array
for your example REDIM strArr(nrElements) or if you need to add some elements without
loosing content use REDIM PRESERVE strArr(nrElements).
Then you can fill it up:
for x = LBound(strArr,1) to Ubound(strArr,1)
strArr(x) = Stringpart(x)
next x
Always open your "direct window" in VBA Editor under menu "view" and use debupg.print strArr(x)
Assuming you meant VB.NET and not VBA (both for the tag to Visual Studio and for the purpose of development a game), you can use a StreamReader to read the file line by line and adding the elements to the string then:
VB.NET SOLUTION
Note: if you didn't do yet, to interact with any file (input or output) you need to import the IO of the system, which means state this on top of your code:
Imports System.IO
...then...
GLOBAL VARIABLES (on top of the module, out of any sub/function)
Dim myStringElements As Integer
Dim convoStarters() As String
SUB CODE
Public Sub yourSub()
myStringElements = 0
Dim sr As StreamReader = New StreamReader(path) 'this object will read the text file
Do While sr.Peek() >= 0 'this will loop through the lines until the end
AddElementToStringArray(sr.ReadLine()) 'this will add the single line to the string array
Loop
sr.Close()
End Sub
PUBLIC FUNCTION BODY (another sub that makes the adding job)
Public Sub AddElementToStringArray(ByVal stringToAdd As String)
ReDim Preserve convoStarters(myStringElements)
convoStarters(myStringElements) = stringToAdd
myStringElements += 1
End Sub
VBA SOLUTION
But if as of your comment you rather meant VBA, then the logic is the same but the syntax is slightly different:
GLOBAL VARIABLES
Dim myStringElements As Integer
Dim convoStarters() As String
SUB CODE
myStringElements = 0
Open "C:\Users\Matteo\Desktop\Test.txt" For Input As #1
While Not EOF(1)
Line Input #1, DataLine ' read in data 1 line at a time
AddElementToStringArray (DataLine) 'this will add the single line to the string array
Wend
PUBLIC FUNCTION BODY
Public Sub AddElementToStringArray(ByVal stringToAdd As String)
ReDim Preserve convoStarters(myStringElements)
convoStarters(myStringElements) = stringToAdd
myStringElements += 1
End Sub
I've been reading and reading tons of answers, modifying the code, and still i cant figure out how to solve this problem.
I have a textbox that receives multiline comma-separated information from a .txt or .csv file. Example:
Pearl Harbour;Ticonderoga CG-45;300;1000 Everett;Ticonderoga
CG-46;310;1200 Pearl Harbour;Burke DDG-110;215;800
Now there will be a combobox to chose a port (in this example the options will be Pearl Harbour and Everett). After choosing "Pearl Harbour", another multiline textbox will show only the lines which have "Pearl Harbour" as the first element.
Now goes what I was able to write:
Public Sub Readfile()
TextBox1.Text = System.IO.File.ReadAllText("libro1.csv")<br>
Dim lines() As String<br>
lines = Split(TextBox1.Text, vbCrLf)<br>
Dim strline0 As String = lines(0)<br>
Dim strArray0() As String = strline0.Split(";")<br>
Dim strline1 As String = lines(1)<br>
Dim strArray1() As String = strline1.Split(";")<br>
...
End Sub
The first problem I find is that for every line the .csv has, I must write that two lines of code to have an array with all the information. But I cant do that because I cant know how many lines the .csv is going to have.
Im kind of lost here. Im not asking anyone to do magic and give me a code I can copy and paste, but I would be grateful if someone can guide me through this.
First off, you'd do better to use a List than an array. Particularly for a collection of strings, they're much easier to work with. With that, you're correct that you can't go individually naming your lines because you don't know how many there will be. That's why you need to create a list of lines and loop through them, like ...
Public Sub Readfile()
TextBox1.Text = System.IO.File.ReadAllText("libro1.csv")
Dim lines As List(of String)
Dim allResults As New List(of List(of String))
lines = Split(TextBox1.Text, vbCrLf)
For Each line In lines
Dim result As List(Of String) = line.Split(CChar(";"))
allResults.Add(result)
Next
End Sub
This will allow you to essentially say, "For each line in the file, take each semi-colon-separated part and put it into a list called 'result'. Then put 'result' into another list of results called 'allResults'."
Behold! The power of loops!