Batch File to Extract values when variable names change in each file - loops

Question

OK, I have hundred of files that have a text header in them that includes one or more entries for a set of eight different variables (name, host, year, month, hour, minute, second). The first time the value is displayed it looks like:
#name 4 10
3 4 DHARLAN
Every time after this (and the variable sets can appear 1 to 100 times per file) it shows up only as:
3 4 DHARLAN
The problem comes in that what happens to be a 4 in this case can be any value between 1 & 99. So in the next file it might be:
3 15 DHARLAN
So really the of eavh variables entries are something like:
3 ## <value>
Where ## is determined earlier in the header by:
#name ## X
I don't understand enoyugh about how FOR /F TOKENS works to get anything that comes close.
What I need is to parse a directory and end up with a file something like:
<filenameA> <name1> <host1> <year1> <month1> <day1> <hour1> <minute1> <second1>
<filenameA> <name2> <host2> <year2> <month2> <day2> <hour2> <minute2> <second2>
<filenameB> <name1> <host1> <year1> <month1> <day1> <hour1> <minute1> <second1>
...
What I have so far:
FOR /f "tokens=*" %%i IN ('dir /a-d /s /b') DO call :findfile "%%i"
:findfile
REM Print Filename
FINDSTR /B /M "#UGC">>output.txt
REM set Name Variable, need 2nd word (7th & if exists (8th) character) from this line only
FINDSTR /B "#name">%%n
REM Find all lines with name variable
FINDSTR /B "3 %n%">>output.txt
Help is greatly appreciated, even suggestions to a program that can do this.

Answer 1

Thoughts
So...This is how I read your question. You have many files with many values and for each file you want output a single line of all the file's variables. Each variable is composed of two lines. 1. The variable declaration (name) and 2. The variable value (DHARLAN). These lines are associated by a number (1 to 99). Is this correct?
This should get you started...
#echo off
setlocal EnableExtensions EnableDelayedExpansion
<nul set /p "=">output.txt
for /f "delims=" %%F in ('dir /a:-d /b /s') do if exist "%%~fF" (
set "name="
set "namenum="
set "host="
set "hostnum="
set "year="
set "yearnum="
set "month="
set "day="
set "daynum="
set "monthnum="
set "hour="
set "hournum="
set "minute="
set "minutenum="
set "second="
set "secondnum="
set "count=0"
for /f "usebackq delims=" %%L in ("%%~fF") do (
for /f "tokens=1,2,3*" %%X in ("%%L") do (
if "%%Y"=="!namenum!" set "name=%%Z" & set /a "count+=1"
if "%%Y"=="!hostnum!" set "host=%%Z" & set /a "count+=2"
if "%%Y"=="!yearnum!" set "year=%%Z" & set /a "count+=4"
if "%%Y"=="!monthnum!" set "month=%%Z" & set /a "count+=8"
if "%%Y"=="!daynum!" set "day=%%Z" & set /a "count+=16"
if "%%Y"=="!hournum!" set "hour=%%Z" & set /a "count+=32"
if "%%Y"=="!minutenum!" set "minute=%%Z" & set /a "count+=64"
if "%%Y"=="!secondnum!" set "second=%%Z" & set /a "count+=128"
if /i "%%X"=="#name" set "namenum=%%Y"
if /i "%%X"=="#host" set "hostnum=%%Y"
if /i "%%X"=="#year" set "yearnum=%%Y"
if /i "%%X"=="#month" set "monthnum=%%Y"
if /i "%%X"=="#day" set "daynum=%%Y"
if /i "%%X"=="#hour" set "hournum=%%Y"
if /i "%%X"=="#minute" set "minutenum=%%Y"
if /i "%%X"=="#second" set "secondnum=%%Y"
if "!count!" equ "255" (
echo %%~nxF !name! !host! !year! !month! !day! !hour! !minute! !second!>>output.txt
set "name="
set "namenum="
set "host="
set "hostnum="
set "year="
set "yearnum="
set "month="
set "day="
set "daynum="
set "monthnum="
set "hour="
set "hournum="
set "minute="
set "minutenum="
set "second="
set "secondnum="
set "count=0"
)
)
)
)
goto :eof
:End
endlocal
Update
Here is an updated version of the script above with comments. It should also address your comment below with the setup section and the cd command.
#echo off
setlocal EnableExtensions EnableDelayedExpansion
:: Setup
set "ResultsFolder=results"
set "ExportFile=output.txt"
:: Set the Working Directory
cd data
:: Verify that the Results folder exists
if not exist "%ResultsFolder%\*" md "%ResultsFolder%"
set "OutputFile=%ResultsFolder%\%ExportFile%"
:: Empty the results file.
<nul set /p "=">%OutputFile%
:: Loop through the files /a:-d in the directory and its subdirectories /s.
for /f "delims=" %%F in ('dir /a:-d /b /s') do if exist "%%~fF" (
call :Reset
rem Loop through the file contents and parse each line.
for /f "usebackq delims=" %%L in ("%%~fF") do (
for /f "tokens=1,2,3*" %%X in ("%%L") do (
rem Keep track of the variables.
if "%%Y"=="!namenum!" set "name=%%Z" & set /a "count+=1"
if "%%Y"=="!hostnum!" set "host=%%Z" & set /a "count+=2"
if "%%Y"=="!yearnum!" set "year=%%Z" & set /a "count+=4"
if "%%Y"=="!monthnum!" set "month=%%Z" & set /a "count+=8"
if "%%Y"=="!daynum!" set "day=%%Z" & set /a "count+=16"
if "%%Y"=="!hournum!" set "hour=%%Z" & set /a "count+=32"
if "%%Y"=="!minutenum!" set "minute=%%Z" & set /a "count+=64"
if "%%Y"=="!secondnum!" set "second=%%Z" & set /a "count+=128"
if /i "%%X"=="#name" set "namenum=%%Y"
if /i "%%X"=="#host" set "hostnum=%%Y"
if /i "%%X"=="#year" set "yearnum=%%Y"
if /i "%%X"=="#month" set "monthnum=%%Y"
if /i "%%X"=="#day" set "daynum=%%Y"
if /i "%%X"=="#hour" set "hournum=%%Y"
if /i "%%X"=="#minute" set "minutenum=%%Y"
if /i "%%X"=="#second" set "secondnum=%%Y"
rem When a full set is reached print it out.
if "!count!" equ "255" (
echo %%~nxF !name! !host! !year! !month! !day! !hour! !minute! !second!>>%OutputFile%
call :Reset
)
if "!count!" gtr "255" (
echo %%~nxF: Incomplete Set Encountered. Stopping parsing of this file, continuing to the next.
rem You can also use other validations to identify incomplete sets.
)
)
)
)
goto :eof
:: Reset all of the variables for the next set.
:Reset
set "name="
set "namenum="
set "host="
set "hostnum="
set "year="
set "yearnum="
set "day="
set "daynum="
set "month="
set "monthnum="
set "hour="
set "hournum="
set "minute="
set "minutenum="
set "second="
set "secondnum="
set "count=0"
goto :eof
:End
endlocal
Summary
This is a summary of what the script is doing and how it works.
The Main for loop, will loop through all files located within the current working directory. dir /a:-d /b /s
Inside this loop, we first setup the variables needed to keep track of the file variables. :Reset
The next for loop will open the file and read each line of the file. %%L
The third for loop will then parse the line for the relevant variable information. Retrieve the first, second, and all remaining tokens from the line (1,2,*) into variables (%%X, %%Y, %%Z)
Now perform logic to keep track of the variables sets. If the variable number has been set and this line matches the variable number, then save the variable value. Else if the variable name matches, retrieve the variable number for comparison on the following lines.
Also increment the set count number and when a full set has been discovered, write it to the output file with the filename.
Reset the variables for use with the next set.
If the count does not equal 255, the file is incorrectly formatted or has an incomplete set of variables.
Notes
This script is just what seems to be the best solution for the questions presented and the information provided therein. I have had to make assumptions about the files being processed and therefore may requires some modifications. If you want to show us a full examples of the file to be parsed, it would help in determining what to do.