Merge two files with different header value - file

I need to merge 2 files into a new file using .vbs. Both files contains header and data. For header, I need to sum both files row count & also the total amount. For the data, I need to merge both data from these two files.
Header format:
yyyymmdd|FILENAME|row count|total amount
Sample of files & format as below:
File 1:
20160204|FILENAME|3|17
20160204|SARAH|OWEN|SCIENCE BOOK|20090717|USD|2|SCIENCE|0|
20160204|KYLE|PARKER|ENGLISH BOOK|20090717|USD|10|ENGLISH|0|
20160204|LILY|MORGAN|POLITICAL BOOK|20090717|USD|5|POLITICS|0|
File 2:
20160204|FILENAME|1|4
20160204|ADAM|HANSEL|HISTORY BOOK|20090717|USD|4|HISTORY|0|
Output:
20160204|FILENAME|4|21
20160204|SARAH|OWEN|SCIENCE BOOK|20090717|USD|2|SCIENCE|0|
20160204|KYLE|PARKER|ENGLISH BOOK|20090717|USD|10|ENGLISH|0|
20160204|LILY|MORGAN|POLITICAL BOOK|20090717|USD|5|POLITICS|0|
20160204|ADAM|HANSEL|HISTORY BOOK|20090717|USD|4|HISTORY|0|
I only managed to merge the two files. But it is not the output I want.
So far I only have this:
Const ForReading = 1
Set objFSO = CreateObject("Scripting.FileSystemObject")
Set objOutputFile = objFSO.CreateTextFile("MYFILE.txt")
Set objTextFile = objFSO.OpenTextFile("D:\Testing\MYFILE1.txt", ForReading)
strText = objTextFile.ReadAll
objTextFile.Close
objOutputFile.WriteLine strText
Set objTextFile = objFSO.OpenTextFile("D:\Testing\MYFILE2.txt", ForReading)
strText = objTextFile.ReadAll
objTextFile.Close
objOutputFile.WriteLine strText
objOutputFile.Close

What you need to do is:
read the data rows of the two files
extract the 7th field of each data row
calculate the sum of all extracted values
write the new header with sum and total number of data rows
write all data rows to the file
For small-ish files one way would be something like this:
datafiles = Array("D:\Testing\MYFILE1.txt", "D:\Testing\MYFILE2.txt")
Set fso = CreateObject("Scripting.FileSystemObject")
Dim header
ReDim data(-1)
sum = 0
For Each filename In datafiles
Set f = fso.OpenTextFile(filename)
header = Split(f.ReadLine, "|")
Do Until f.AtEndOfStream
ReDim Preserve data(UBound(data)+1)
line = f.ReadLine
data(UBound(data)) = line
sum = sum + CInt(Split(line, "|")(6))
Loop
f.Close
Next
Set f = fso.CreateTextFile("MYFILE.txt")
f.WriteLine header(0) & "|" & header(1) & "|" & (UBound(data)+1) & "|" & sum
For Each row In data
f.WriteLine row
Next
f.Close
Beware that this won't perform too well for large files, though. If you need to process files like that you'll have to handle them differently (writing the data rows line by line to a temporary file instead of keeping them in memory, then writing header and merged data to the final file line by line in a second step).

Hope this helps:
<%
Class contFiles
Public Files
Private Sub Class_Initialize()
Set Files = Server.CreateObject("Scripting.Dictionary")
End Sub
Private Sub Class_Terminate()
If IsObject(Files) Then
Files.RemoveAll()
Set Files = Nothing
End If
End Sub
Public Sub addFile(theFileName)
Files.Add theFileName, theFileName
End Sub
Public Default Function makeFile(fileName)
On Error Resume Next
theKeys = Files.Keys
Set FSO = Server.CreateObject("Scripting.FileSystemObject")
Do Until IDx = Files.Count
Set TheTextStream = FSO.OpenTextFile(Server.MapPath(theKeys(IDx)), 1)
html = html & TheTextStream.ReadAll
IDx = IDx + 1
Loop
Set objTxt = FSo.CreateTextFile(Server.MapPath(fileName), True, False)
objTxt.Write(html)
objTxt.Close
If ( err.Number <> 0 ) Then
makeFile = False
Else
makeFile = True
End If
End Function
End Class
Set testFiles = New contFiles
testFiles.addFile("default.aspx")
testFiles.addFile("events.aspx")
testFiles.addFile("about.aspx")
response.write(testFiles.makeFile("test.txt"))
%>

Related

Store filehandles in array

I try to open all *.xlsx files in a specified folder and store the filehandles in an array.
My code looks like this
Dim Files() As Workbook
ReDim Files(Count)
File = Dir(Path & "\*.xlsx")
Count = 0
Do While File <> ""
Set Files(Count) = Workbooks.Open(Path & File, , True)
Count = Count + 1
File = Dir()
Loop
The code seems to work, however, when I run it a second time (hitting the run button again), I get an error number 13.
Debugging the code I tracked the problem to the line
Set Files(Count) = Workbooks.Open(Path & File, , True)
As I am unexperienced with vba I guess I didn't do this the right way...
What would be a preferable way to store filehandles to all files in a specific folder in an array?
you're missing a path separator
Set Files(Count) = Workbooks.Open(Path & "\" & File, , True)
The code should be:
Dim Files() As Workbook
Dim Count As Integer
ReDim Files(Count)
File = Dir(Path & "\*.xlsx")
Count = 0
Do While File <> ""
ReDim Preserve Files(Count)
Set Files(Count) = Workbooks.Open(Path & File, , True)
Count = Count + 1
File = Dir()
Loop
You need to redim your array. Preserve keeps the existing data.

moving oldest files

I was looking for a way to move the 5 oldest modified files in a folder to a different folder.
I came across some helpful pieces of code and I revised it to this:
Dim files
Dim startFolder
Dim destinationFolder
Dim oldestFile
Dim file
Dim FSO
startFolder = "C:\logs\current"
destinationFolder = "C:\logs\backup"
Set FSO = CreateObject("Scripting.FileSystemObject")
Set files = FSO.GetFolder(StartFolder).files
Set oldFiles = CreateObject("System.Collections.ArrayList")
If files.Count <= 5 Then
WScript.Quit
End If
For i = 0 To 4
Set files = FSO.GetFolder(StartFolder).files
Set oldFiles = Nothing
For Each file In files
If Not IsObject(oldestFile) Then
Set oldestFile = file
Else
If file.DateLastModified < oldestFile.DateLastModified Then
Set oldestFile = file
End If
End If
Next
WScript.Echo "OLDEST: " & oldestFile.Name
oldestFile.Move destinationFolder & "\" & oldestFile.Name
Next
Basically what it supposed to do is:
loop 5 times,
each time loop through the files and assign the oldest to oldestFile,
move the file to a different location.
However, it doesn't work, it's echoing the first file's name 5 times and move just this one.
I thought I should set the objects to Nothing to start fresh but to no avail.
You need to reset the variable oldestFile at the beginning (or end) of your loop, not the variable oldFiles.
For i = 0 To 4
Set files = FSO.GetFolder(StartFolder).files
Set oldestFile = Nothing
For Each file In files
...
Next
WScript.Echo "OLDEST: " & oldestFile.Name
oldestFile.Move destinationFolder & "\" & oldestFile.Name
Next
Otherwise the value of oldestFile will never change, because even after being moved the referenced file ramains the oldest file compared to the files in the source folder.

extract data from multiple text files in a folder into excel worksheet

I have multiple "datasheet" text files that are used with a program at work and need to harvest values from them and combine it all into a spreadsheet.
The text files are formatted as such:
[File]
DescText = "1756-IF16H 16 Channel Hart Analog Input Module";
CreateDate = 04-07-10;
CreateTime = 10:29;
Revision = 1.1;
HomeURL = "http://www.ab.com/networks/eds/XX/0001000A00A30100.eds";
[Device]
VendCode = 1;
VendName = "Allen-Bradley";
ProdType = 10;
ProdTypeStr = "Multi-Channel Analog I/O with HART";
ProdCode = 163;
MajRev = 1;
MinRev = 1;
ProdName = "1756-IF16H/A";
Catalog = "1756-IF16H/A";
Icon = "io_brown.ico";
The Tags are consistent through all the files and each lines ends with a semicolon [ ; ] so I'm assuming this should be pretty easy. I need to pull "DescText","VendCode","ProdType","MajRev","MinRev",and"ProdName" into separate columns.
There are about 100 individual data files, each with a nonsensical filename, so I'm looking to have the macro just go through and open each one in the folder.
Thanks for the help, here is the solution I came up with for this specific problem
Sub OpenFiles()
Dim MyFolder As String
Dim MyFile As String
MyFolder = "[directory of files]"
MyFile = Dir(MyFolder & "\*.txt")
Dim filename As String
Dim currentrow As Integer: currentrow = 2
Do While Myfile <> "" 'This will go through all files in the directory, "Dir() returns an empty string at the end of the list
'For i = 1 To 500 'this was my debug loop to only go through the first 500 files at first
filename = MyFolder & "\" & MyFile 'concatinates directory and filename
Open filename For Input As #1
Do Until EOF(1) 'reads the file Line by line
Line Input #1, textline
'Text = Text & textline
If textline = "" Then 'error handler, if line was empty, ignore
Else
Dim splitline() As String
splitline() = Split(textline, "=", -1, vbTextCompare)
'because of how my specific text was formatted, this splits the line into 2 strings. The Tag is in the first element, the data in the second
If IsError(splitline(0)) Then
splitline(0) = ""
End If
Select Case Trim(splitline(0)) 'removes whitespace
Case "DescText"
currentrow = currentrow + 1
'files that didn't have a description row, resulted in empty rows in the spreadsheet.
ActiveSheet.Range("A" & currentrow).Cells(1, 1).Value = splitline(1)
Case "Revision"
ActiveSheet.Range("B" & currentrow).Cells(1, 1).Value = splitline(1)
Case "ProdCode"
ActiveSheet.Range("C" & currentrow).Cells(1, 1).Value = splitline(1)
Case "ProdType"
ActiveSheet.Range("D" & currentrow).Cells(1, 1).Value = splitline(1)
'...etc. etc... so on for each "tag"
End Select
End If
Loop
Close #1
MyFile = Dir() 'reads filename of next file in directory
'currentrow = currentrow + 1
'Next i
Loop
End Sub
here how I would solve the complete task:
Private Sub importFiles(ByVal pFolder As String)
' create FSO
Dim oFSO As Object
Set oFSO = CreateObject("Scripting.FileSystemObject")
' create folder
Dim oFolder As Object
Set oFolder = oFSO.getFolder(pFolder)
' go thru the folder
Dim oFile As Object
For Each oFile In oFolder.Files
' check if is a text file
If UCase(Right(oFile.Name, 4)) = ".TXT" Then
Debug.Print "process file: " & oFolder.Path & "\" & oFile.Name
readFile oFolder.Path & "\" & oFile.Name
End If
Next
' clean up
Set oFolder = Nothing
Set oFSO = Nothing
End Sub
Private Sub readFile(ByVal pFile As String)
' get new file handle
Dim hnd As Integer
hnd = FreeFile
' open file
Open pFile For Input As hnd
Dim sContent As String
Dim sLine As String
' read file
Do Until EOF(hnd)
Line Input #hnd, sLine
sContent = sContent & sLine
Loop
' close file
Close hnd
' extract requiered data
Debug.Print getValue(sContent, "ProdName")
Debug.Print getValue(sContent, "DescText")
End Sub
Private Function getValue(ByVal pContent As String, ByVal pValueName As String) As String
Dim sRet As String
sRet = ""
If InStr(pContent, pValueName) Then
pContent = Mid(pContent, InStr(pContent, pValueName) + Len(pValueName) + 2)
sRet = Left(pContent, InStr(pContent, ";") - 1)
sRet = Trim(sRet)
End If
getValue = sRet
End Function
Overall the solution contains 3 different procedures:
importFiles reads the content of a given directory (which has to be handed over as parameter) and if it finds a .txt file it calls readFile() and passes the full path of the file to it
readFile() opens the text file and stores the content in a string variable. After this is done it calles getValue for each value you are interessted in.
getValue analyses the given content and extractes the given value.
Simply adjust the calls of getValue() so that you get all values you are interessted in and store them instead of showing via debug.print and call the first procedure with the right directory like importFiles "C:\Temp"

vba search through a folder and select files by name

I've got lots of files in a folder structered with names like:
"prof102122013#10.18.41.csv"
where the "02122013" bit is the date - 02/12/2013. Some of them have been made on the same day. I'd like to create a file that takes all of the workbooks that were made on the same day and put them all in one big file. So far I am struggling to get the selectivity to open a day-specific file. Does anyone have any idea what kind of code can help me with this?
Edit: Solved, thanks for the help, all! Here was the code that worked for me:
folder_location = Application.ActiveWorkbook.Path
i2 = 0
strFile = Dir(folder_location & "\")
'looping through to find the right file names and putting them all in an array
While strFile <> ""
If Right(strFile, 3) = "csv" Then
file_to_analyse = Split(strFile, "#")
If Right(file_to_analyse(0), 8) = date_we_want_to_analyse_on Then
found_files_to_analyse(i2) = strFile
i2 = i2 + 1
End If
End If
strFile = Dir
Wend
Do you want to consolidate files on the basis of file saved date or on the basis of the name of the file. A file named with yesterday's date can be saved today and will bear today's date. I guess you would have to parse the name of the file and apply logic around the date (in the file name) in a do while loop till all the files in the directory are searched for the condition. If the condition is met, you copy the opened file into a worksheet in your file. If the condition is not met, the file is skipped. Following might help
For each date which needs to be consolidated, do the following in a loop or an in a user prompted message box where the user inputs the date. You also need to chose whether you want the consolidation to happen in the workbook from where you are launching the macro or a separately opened workbook. The code below assumes you are consolidating in the same workbook.
Path = Path of the directory in which the files are stored
Flname = Dir(Path & "*.csv")
Do While Flname <> ""
If ' file check Condition' Then
Filecheckname = True ' Checks if file follows the naming conventions desired
Else
Filecheckname = False
End If
If Filecheckname Then
FlDate = getDate(Flname) ' extracts the date from the file name
Else
GoTo Errorhandler ' If there is an error, then the macro stops with a message to the user
End If
If FlDate<> Date Then
flsskpd = flsskpd + 1 ' If the date criteria is not met, the file is skipped with an increment to the fileskipped counter
Flname = Dir()
Else
Workbooks.Open Filename:=Path & Flname, ReadOnly:=True
'Code to Copy into consolidated workbook (ThisWorkbook)
filesmoved = filesmoved + 1
Flname = Dir()
End if
Loop
Message to user about how many files skipped and consolidated.
Prompt user whether to continue to the next date consolidation, if yes, continue or take the new date as an input and repeat the loop
If you want VBA to "search" for a file/folder in a directory, I think you need to use something like this:
Option Explicit
Option Compare Text
Public Enum xlSearchMode
xlFilesOnly = 0
xlFoldersOnly = 1
xlFilesAndFolders = 2
End Enum
Function SearchInDirectory(FName As String, Optional FoName As String, Optional SearchMode As xlSearchMode = xlFilesOnly, Optional ExactMatch As Boolean = True) As Variant
'By Abdallah Khaled Ali El-Yaddak
'Returns an array of strings with files/folders matching what you are searching for.
'If nothing is found, it returns an array of one empty string element.
'-------------'
'FName (String): The file/folder to look for
'[FoName] (String): The directory to search in, if omitted, CurDir will be used.
'[SreachMode] (xlSearchMode): xlFilesOnly (default) = Look for files only | xlFoldersOnly = Look for folders only | xlFilesAndFolders = Look for both
'[Exactmatch] (Boolean): True (default) = Look only for this string (case insenstive) | False = Sreach for any files/folders that includes this string in their name
Dim FSO As Object, File As Object, Folder As Object, Fnames() As String, i As Long, SubNames As Variant, SubFolder As Object
If FoName = "" Then FoName = CurDir
If Right(FoName, 1) <> "\" Then FoName = FoName & "\"
ReDim Fnames(1 To 1) As String
Set FSO = CreateObject("Scripting.FileSystemObject")
Set Folder = FSO.GetFolder(FoName)
If SearchMode = xlFilesOnly Or SearchMode = xlFilesAndFolders Then
For Each File In FSO.GetFolder(Folder).Files
If (ExactMatch And SubFolder.Name = FName) Or _
(Not ExactMatch And SubFolder.Name Like "*" & FName & "*") Then
Fnames(UBound(Fnames)) = File.Path
ReDim Preserve Fnames(1 To UBound(Fnames) + 1)
End If
Next
End If
If SearchMode = xlFoldersOnly Or SearchMode = xlFilesAndFolders Then
For Each SubFolder In FSO.GetFolder(Folder).subFolders
If (ExactMatch And SubFolder.Name = FName) Or _
(Not ExactMatch And SubFolder.Name Like "*" & FName & "*") Then
Fnames(UBound(Fnames)) = SubFolder.Path
ReDim Preserve Fnames(1 To UBound(Fnames) + 1)
End If
Next
End If
For Each SubFolder In FSO.GetFolder(Folder).subFolders
SubNames = SearchInDirectory(FName, SubFolder.Path, SearchMode, ExactMatch)
If SubNames(LBound(SubNames)) <> "" Then
For i = LBound(SubNames) To UBound(SubNames)
Fnames(UBound(Fnames)) = SubNames(i)
ReDim Preserve Fnames(1 To UBound(Fnames) + 1)
Next
End If
Next
If UBound(Fnames) > 1 Then ReDim Preserve Fnames(1 To UBound(Fnames) - 1)
SearchInDirectory = Fnames
End Function
To test, you need something like this:
Sub Test()
Dim a As Variant, i As Long
a = SearchInDirectory(date_we_want_to_analyse_on, folder_location, xlFilesOnly, Flase)
For i = LBound(a) To UBound(a)
Debug.Print a(i)
Next
End Sub
Notes:
This solution doesn't work on MAC (tested only on windows)
Searching will take longer for larger directories (The number of files/folders inside)

Excel - read 2 values from unknown number of CSV's with different worksheet names into array

I am another newb to scripting. I've done a lot of searching and haven't been able to find a solution that suits my problem.
I have a folder with a varying number of CSV's that each contain 1
worksheet and 2 values in the same cells (A2 & B2).
The worksheet names are not the same.
Logically I need to create a script that goes to each CSV in this folder, grabs the 2 values and puts them into a single worksheet in an XLS called MonthlyAvg in 2 columns
ie 1st Range goes to A2 & B2 in master document, second A2 & B2 goes into A3 & B3 in master document. My thoughts are that this should be handled in an array. Alas this is where I am stuck as I'm unable to put what I've found on creating arrays logically into what I've put together so far. Any help or guidance would be greatly appreciated.
strPath = "D:\MacWP\MacWork\Thermometers\TOT\MonthlyAvg\"
Set objExcel = CreateObject("Excel.Application")
objExcel.Visible = True
objExcel.DisplayAlerts = False
Set objFso = CreateObject("Scripting.FileSystemObject")
Set objFolder = objFso.GetFolder (strPath)
For Each objFile In objFolder.Files
If objFso.GetExtensionName (objFile.Path) = "csv" Then
Set objWorkbook = objExcel.Workbooks.Open(objFile.Path)
'Find the first worksheet name
FirstSheetName = objExcel.Activeworkbook.Worksheets(1).Name
'Set the first worksheet name
Set objSheet = objWorkbook.Sheets (FirstSheetname)
'Display the value in B2 of the first worksheet
MsgBox objSheet.Range("B2").Value
'Display the value in A2 of the first worksheet
MsgBox objSheet.Range ("A2").Value
objWorkbook.Close False 'Save changes
End If
Next
objExcel.Quit
If I understand, your file searching is already done, and your only question is how to storage the partial values in a master excel file, right ?
For that, you can use a For cycle with an "index".
Dim i As Integer, j As Integer, n As Integer
Dim filesOpen As Long
With Application.FileSearch
.LookIn = "C:\Examples"
.FileType = msoFileTypeExcelWorkbooks
'There are wb's
If .Execute > 0 Then
For n = 1 To .FoundFiles.Count
Workbooks.Open (.FoundFiles(i))
Next n
filesOpen = n
end With
For i = 2 To filesOpen
'i = 2 'fila
j = 1 'columna
'column A2
Application.Workbooks("MasterFile").Worksheets("nameOfYourMasterSheet").Cells(i, j).Value = Application.Workbooks("Book1.cvs").Worksheets("nameOfYourSheet").Cells(2, j).Value
'column B2
Application.Workbooks("MasterFile").Worksheets("nameOfYourMasterSheet").Cells(i, j+1).Value = Application.Workbooks("Book1.cvs").Worksheets("nameOfYourSheet").Cells(2, j+1).Value
next i
So, the last for cycle is for giving the values to your Master File. The "i" index is the only one which is ascending until you dont have more workbooks.
You don't have to use Arrays, just watch out the "i" value.
You could use a vbs as below to write all the values to a csv file directly. The code below creats a file output.csv under strPath
Dim objExcel
Dim objFSO
Dim objTF
Dim objWB
strpath = "c:\temp"
Set objExcel = CreateObject("Excel.Application")
Set objFSO = CreateObject("Scripting.FileSystemObject")
Set objFolder = objFSO.GetFolder(strpath)
Set objTF = objFSO.CreateTextFile(strpath & "\output.csv", True, False)
objExcel.DisplayAlerts = False
For Each objFile In objFolder.Files
If objFSO.GetExtensionName(objFile.Path) = "csv" Then
Set objWB = objExcel.Workbooks.Open(objFile.Path)
objTF.WriteLine objWB.Sheets(1).Range("B2") & "," & objWB.Sheets(1).Range("A2")
objWB.Close False 'Save changes
End If
Next
objTF.Close
With objExcel
.DisplayAlerts = True
.Quit
End With

Resources