Text File to Array in Access VBA - arrays

I am looking to load a .txt file into a VBA array (in an access VBA), manipulate it there, and then paste the manipulated array into an access table. I will loop this macro then through round about 500 .txt files and populate the database with years of data.
I have done it before using Excel in this way:Populating the sheet for 1 .txt file, manipulating the data, loading into the database, clearing the sheet, and loading the next txt file and loop till all files have been processed. However, this takes years and it becomes obvious that Excel is not really needed, since everything is stored in Access anyway.
Since then, I have tried to figure out how to do it in access straight away, without using excel, but I have been struggling. I found some nice code on this board here:
Load csv file into a VBA array rather than Excel Sheet
and tried to amend it so it would work for me. Especially The_Barman's Answer at the bottom seems simple and very interesting.
I am new to arrays, VBA, SQL, basically everything, so maybe there is some big mistake I am making here that is easy to resolve. See my code below:
Sub LoadCSVtoArray()
Dim strfile As String
Dim strcon As String
Dim cn As ADODB.Connection
Dim rs As Recordset
Dim rsARR() As Variant
Set cn = New ADODB.Connection
strcon = "Provider=Microsoft.JET.OLEDB.4.0;Data Source=" & "\\filename.txt" &
";Extended Properties=""text;HDR=Yes;FMT=Delimited"";"
cn.Open strcon
strSQL = "SELECT * filename.txt;"
Set rs = cn.Execute(strSQL)
rsARR = WorksheetFunction.Transpose(rs.GetRows)
rs.Close
Set cn = Nothing
[a1].Resize(UBound(rsARR), UBound(Application.Transpose(rsARR))) = rsARR
End Sub
I dont even know if the bottom part of the code works, because an error message pops up that the file is not found. The interesting thing is, if I debug and copy the value of strcon into windows "run", it opens the correct file. So I guess the path is correct? Can I even open a .txt file through an ADODB connection? Right now I am a bit confused if this is possible and if it is the best solution.
Some more background regarding the text files I am trying to save into the array:
-They are output from another program, so it is always the same structure and very oreganized
it comes in this format:
Column a Column b ....
data 1 data 1
data 2 Data 2
...
and so on.
If possible, I would like to retain this structure, and even safe it as a table with the first row as column headers.

The Data Source is the folder path containing the file and the file is the table (SELECT * FROM ..). Replace "\\filename.txt" in strcon with the folder path.
http://www.connectionstrings.com/textfile/

Related

Pull data from multiple closed workbooks and sheets in excel

I want to pull data from multiple workbooks and worksheets that are closed.
I'm able to extract the data by using the below formula:
=INDEX('C:\Users\Shubham\Downloads\Ex_Files_Excel_Macros_and_VBA_for_Beginners\Ex_Files_Excel_Macros_and_VBA_for_Beginners\Exercise Files\[03_01 Undo.xlsx]Attendance'!C:C, 6)
but I want to provide the path in a cell using CONCATENATE function let's say in E5 so that value in E5 comes like below:
'C:\Users\Shubham\Downloads\Ex_Files_Excel_Macros_and_VBA_for_Beginners\Ex_Files_Excel_Macros_and_VBA_for_Beginners\Exercise Files\[03_01 Undo.xlsx]Attendance'!C:C
So I want to use the formula like:
= INDEX(E5, 6)
where E5 has the path and range defined but this doesn't take the value in E5 as table array.
I can't use the indirect formula since it doesn't work with closed workbooks. I have tried to understand the VBA code but doesn't work out.
I would really appreciate it if someone can help me out to resolve this issue since I have to extract data from multiple workbooks and multiple sheets and have to use dynamic reference to path and range.
I had the same problem, wanting to extract data from multiple workbooks with a volatile worksheet address. Indirect and Concatenate worked, but only when source files were open. There is no function which would work with closed files.
A macro was the solution, which I found here;
https://www.mrexcel.com/board/threads/vba-to-pull-data-from-dynamically-named-closed-workbooks.984156/
My solution uses a list of the source file name components with a volatile date component (date is the only variable in the source file names) and a volatile worksheet component (I need to take data for a person, where each person's name is the name of a worksheet in each file).
That list looks like this;List of File Names
I and K (Name) are volatile. These are modified in my list by having these cells reference separate input cells to simplify a query. Enter the name and the latest payroll date you want to query. It looks back a set number of cycles.
The formula to get the data I want is =COUNTIF(INDIRECT(CONCATENATE(H4,I4,J4,K4,L4)),"S")
How many times does S appear in range D63:U63 in tab Moe for each file.
The macro solution uses the Date information in column I to open all the source files as follows (I have added comments to hopefully make it clearer what each step does);
Public Sub Macro_Name()
' Opens Timesheets files to allow "Indirect" formula to obtain data
Dim r As Long
' I don't know what this does, but it is necessary
Application.ScreenUpdating = False
' Stops screen updating to speed up operation
Calculate
' Calculates to clear any previous data in the file. File is set to manual calculation.
With ThisWorkbook.ActiveSheet
For r = 4 To .Cells(Rows.Count, "I").End(xlUp).Row
' Starting at row 4, uses data in column I in the command below until the end of the data. You can't have any other data in column I.
Workbooks.Open Filename:="S:\Payroll\Weekly Timesheets " & .Cells(r, "I") & ".xlsm"
' Opens each file called Weekly Timesheets [date].xlsm
Next
Windows("[The name of your working file].xlsm").Activate
Calculate
Range("B2").Select
' This calculates your working file and goes to a convenient "home" cell
End With
Dim wb As Workbook
For Each wb In Application.Workbooks
If Not wb Is ThisWorkbook Then
wb.Close SaveChanges:=False
End If
Next
' This sub routine closes all open Excel files EXCEPT your working file. Best not to have other files open as it is set NOT to save.
Application.ScreenUpdating = True
End Sub
I found this page to set the file for manual calculation, which I preferred.
https://excel.tips.net/T001988_Forcing_Manual_Calculation_For_a_Workbook.html
I'm not sure if this solution is efficient, and it is not perhaps as elegant as I would like, but it has worked for me. I hope it helps.
Big shoutout to John_w at MrExcel.com who provided the core code above. I just fiddled a few things so it worked for me.

Real-Time Database Updating

I have ~300 Identical Excel Spreadsheets (.xlsx) that all have ten different cells I would like to project to an Access database. Some are single cells, a few ranges-- all of them are named in each individual workbook. The database should hold all values from the same worksheet on the same line, and all numbers should update in real time.
My goal is to use access to keep a running record on the contents of these cells. I have attempted to link the workbooks to the Master Access database using: External Data->Import and Link->Excel->Link to the data source by creating a linked table-> and here I would select one of the named ranges and click 'Finish', but I can only do this once per spreadsheet. This makes repeating this process for all spreadsheets unfeasible.
Is there a way to use Access VBA to create a linked Excel Table for each Workbook in the folder?
I am sort of teaching myself Access, and am still relatively new at it so any insight will help.
Cheers.
Luckily, I had just that lying around somewhere. This links all excel files in the same folder as the database.
Note that this just uses the file name as the tablename, and links everything using default settings. Change the DoCmd.TransferSpreadsheet line to customize it. Originally I have this linked to a form so I can choose what to import, and how it is handled.
Public Sub ImportExcel()
Dim objFSO As Object 'FileSystemObject
Dim databasefolderlocation As String
Dim objFolder As Object 'Folder
Dim objFile As Variant
Set objFSO = CreateObject("Scripting.FileSystemObject")
databasefolderlocation = Application.CurrentProject.Path 'This sets the folder, you can change it for another folder
Set objFolder = objFSO.GetFolder(databasefolderlocation)
For Each objFile In objFolder.Files
If objFile.Name Like "*.xls" Or objFile.Name Like "*.xlsx" Then
DoCmd.TransferSpreadsheet acLink, , objFile.Name, objFile.Path
End If
Next objFile
End Sub

[VB.NET][ACCESS] How do I check for if a database exists?

I cant find anything online to help.
I want to create a table if it doesnt already exist, or populate a listbox with what is stored in said table if it DOES exist. All I have so far is the populate and create table subroutines, but have no idea how to check the database so far.
Thank you
Checking if an MSAccess DATABASE exists or not is pretty simple because it is just a single file. So using File.Exists is enough
Suppose that your MDB file is
Dim accessFilePath = "D:\temp\myDatabase.mdb"
If File.Exists(accessFilePath) Then
... file exists
End if
Of course getting the content of the file (in terms of TABLES and QUERY) is a different thing and requires to open the connection and get the SCHEMA informations
Dim cnnString = "Provider=Microsoft.Jet.OLEDB.4.0;Data Source=" & accessFilePath
Using con = new OleDbConnection(cnnString)
con.Open()
Dim schema = con.GetSchema("Tables")
For Each row As DataRow in schema.Rows
Console.WriteLine(row.Field(Of String)("TABLE_NAME"))
Next
End Using
See how GetSchema works and what are its possible parameters and results

Bulk Insert from CSV file to MS SQL Database

I have this working script that I use to BULK INSERT A CSV FILE.
The code is:
' OPEN DATABASE
dim objConn,strQuery,objBULK,strConnection
set objConn = Server.CreateObject("ADODB.Connection")
objConn.ConnectionString = "Driver={SQL Server Native Client 11.0};Server=DemoSrvCld;Trusted_Connection=no;UID=dcdcdcdcd;PWD=blabla;database=demotestTST;"
objConn.Open strConnection
set objBULK = Server.CreateObject("ADODB.Recordset")
set objBULK.ActiveConnection = objConn
dim strAPPPATH
strAPPPATH="C:\DEMO001Test.CSV"
strQuery = "BULK INSERT EFS_OlderStyle FROM '" & strAPPPATH & "' WITH (firstrow=1, FIELDTERMINATOR=',', ROWTERMINATOR='\n')"
Set objBULK= objConn.Execute(strQuery)
objConn.Close
HERE IS A EXAMPLE OF THE .CSV FILE:
Date,Time,Card Number,Driver Id,Driver Name,Unit No,Sub-Fleet,Hub Miles,Odo Miles,Trip No,Invoice,T/S Code,In Dir,T/S Name,T/S City,ST,Total Inv,Fee,PPU,Fuel_UOM,Fuel_CUR,RFuel_UOM,RFuel_CUR,Oil_CUR,Add_CUR,Cash Adv,Tax,Amt Billed,Svc Bill,Chain,Ambest,MPU
10/08/13,03:20,70113531460800693,,,2100,,,,,0454591156,546200,Y,PILOT QUARTZSITE 328,QUARTZSITE,AZ,742.30,1.00,3.749,149.000,558.60,49.00,183.70,0.00,0.00,0.00,0.00,743.30,S, ,N,0.0
10/08/13,07:03,70110535170800735,,,6210,,,,,343723,512227,Y,PETRO WHEELER RIDGE,LEBEC,CA,678.78,1.00,4.169,139.140,580.08,23.68,98.70,0.00,0.00,0.00,0.00,679.78,S, ,N,0.0
But the .CSV FILE I HAVE NOW IS DIFFRENT then the one above.
HERE IS A EXAMPLE OF THE CURRENT .CSV FILE:
"BRANCH","CARD","BILL_TYPE","AUTH_CODE","INVOICE","UNIT","EMP_NUM","TRIP","TRAILER","HUB/SPEED","VEH_LICENSE","DRIVER","DATE","TIME","CHAIN","IN_NETWORK","TS#","TS_NAME","TS_CITY","TS_STATE","PPG","NET_PPG","FUEL_GALS","FUEL_AMT","RFR_GALS","RFR_AMT","CASH","MISC","INV_TOTAL","FEE","DISC","INV_BALANCE",1.00,1.00,"E","004ACS","02812","365","-","-","0",0.00,"-","JOHN S ",11/4/2013,"16:18:49E","IC","N",3257.00,"IRVING HOULTON","HOULTON","ME",3.95,3.95,121.57,480.08,0.00,0.00,0.00,0.00,480.08,1.50,0.00,481.58
"BRANCH","CARD","BILL_TYPE","AUTH_CODE","INVOICE","UNIT","EMP_NUM","TRIP","TRAILER","HUB/SPEED","VEH_LICENSE","DRIVER","DATE","TIME","CHAIN","IN_NETWORK","TS#","TS_NAME","TS_CITY","TS_STATE","PPG","NET_PPG","FUEL_GALS","FUEL_AMT","RFR_GALS","RFR_AMT","CASH","MISC","INV_TOTAL","FEE","DISC","INV_BALANCE",1.00,2.00,"E","014ACI","976234","430","-","-","0",0.00,"-","STACY ",11/4/2013,"00:21:16E","F","Y",8796.00,"PILOT 405","TIFTON","GA",3.77,3.77,172.65,650.73,0.00,0.00,0.00,0.00,650.73,1.50,0.00,652.23
I have edited the ms sql database fields to reflect the new .csv fields but the old and new .csv files do not store the info. in the same way. How do I fix this so that it works ?
I was thinking to first remove all of the " and then to remove all but one "BRANCH","CARD","BILL_TYPE","AUTH_CODE","INVOICE","UNIT","EMP_NUM","TRIP","TRAILER","HUB/SPEED","VEH_LICENSE","DRIVER","DATE","TIME","CHAIN","IN_NETWORK","TS#","TS_NAME","TS_CITY","TS_STATE","PPG","NET_PPG","FUEL_GALS","FUEL_AMT","RFR_GALS","RFR_AMT","CASH","MISC","INV_TOTAL","FEE","DISC","INV_BALANCE",
then save the .csv file and then reopen it.
But I think/hope there is another way ?
Please help...
Thank you.
Sure: there are a few ways you can work around this problem. The right solution for you will depend on the time and energy you have to dedicate to this problem, as well as whether this is a one time import or a process you want to streamline.
A few solutions:
1. Change the formating of your CSV file to resemble the old version.
this can be done realtively easily:
Download a text editor like notepad++
Open your CSV file in this editor
Do a find/Replace operation for:
"BRANCH","CARD","BILL_TYPE","AUTH_CODE","INVOICE","UNIT","EMP_NUM","TRIP","TRAILER","HUB/SPEED","VEH_LICENSE","DRIVER","DATE","TIME","CHAIN","IN_NETWORK","TS#","TS_NAME","TS_CITY","TS_STATE","PPG","NET_PPG","FUEL_GALS","FUEL_AMT","RFR_GALS","RFR_AMT","CASH","MISC","INV_TOTAL","FEE","DISC","INV_BALANCE"
replace with: ""
Finally add the line above as your header- quickly reformatting your new file in the same format as your old file.
Note: this may be the best option if you have a one time import.
2. Make the above changes in your code programatically
Since the beginning of each line contains the fields you wish to ignore, you can easily truncate each line based on the number of characters. The String.Replace function can be used to replace the initial (ignorable) part of the line with String.Empty before it is inserted into the DB.

How to convert data in excel spreasheet forms into format for upload into a database

I need a good way to convert input data from 900 spreadsheets into a format suitable for upload to a relational database (XML or flat file/s). The spreadsheets are multi-sheet, multi-line Excel 2007 each one consisting of 7 forms (so its definitely not a simple grid). There will be no formula data to get, just text, dates, integer data.
The 900 spreadsheets are all in the same format.
I will need some kind of scripted solution.
I'm expecting I should be able to do this with excel macros (and I expect a fancy scriptable editor could do it too) or possibly SSIS.
Can someone tell me how you would approach this if it was yours to do?
Can anyone give a link to some technical info on a good way to do this?
I'm new to excel macros but used to programming and scripting languages, sql, others.
Why? We're using spreadsheet forms as an interim solution and I then need to get the data into the database.
You probably want to write data out to a plain text file. Use the CreateTextFile method of FileSystemObject. Documentation here:
http://msdn.microsoft.com/en-us/library/aa265018(v=vs.60).aspx
There are many examples on the web of how to iterate over worksheets, capture the data and then use WriteLine method.
Sub ExampleTextFile()
Dim fso as Object
Dim oFile as Object
Dim fullExport as String
'Script that will capture data from worksheet belongs here _
' use the fullExport string variable to hold this data, for now we will _
' just create a dummy string for illustration purposes
fullExport = "Example string contents that will be inserted in to my text file!"
Set fso = CreateObject("Scripting.FileSystemObject")
'In the next line, replace "C:\filename.txt" with the specified file you want to create
set oFile = fso.CreateTextFile("C:\filename.txt", Overwrite:=True, unicode:=True)
oFile.WriteLine(fullExport) '<-- inserts the captured string to your new TXT file
oFile.Close
Set fso = Nothing
Set oFile = Nothing
End Sub
If you have character encoding issues (I recently ran in to a problem with UTF16LE vs. UTF8 encoding, you will need to use the ADODB.Stream object, but that will require a different method of writing the file.

Resources