I have a file where I'm trying to read each line into a struct in C to further work with it.
The file looks like this:
Bread,212,2.7,36,6,9.8,0.01,0.01,10,500
Pasta,347,2.5,64,13,7,0.01,0.01,6,500
Honey,340,0.01,83,0.01,0.01,0.01,0.01,22.7,425
Olive-oil,824,92,0.01,0.01,0.01,0.01,13.8,35,500
White-beans,320,2.7,44,21,18,0.01,0.01,11,400
Flaxseed-oil,828,92,0.01,0.01,0.01,52,14,100,100
Cereal,363,6.5,58,13,9.9,0.01,0.01,11,1000
Hazelnuts,644,61.6,10.5,12,0.01,0.09,7.83,16.74,252
So I wrote a for-loop to iterate over the lines in the file, trying to store each value into fields of a struct. I try to print the fields of the struct, but its already going wrong with the first argument, the string.
It is printing:
scanresult: 1, name: ■B, kcal: 0.00, omega 3: 0.00, omega 6: 0.00, carb: 0.00, protein: 0.00, fib: 0.00, price: 0.00, weight: 0.00g
Scanres should be 10, not 1, and the values should match the ones of the first line of the file.
I have tried with or without whitespace in front of the argument in the formatted string. Also I tried compiler warnings -Wall and -pedantic. No issues found.
What else could cause this problem?
The code looks like this:
#include <stdio.h>
#define MAX_CHAR 100
#define SIZE_OF_SHELF 8
typedef struct {
char name[MAX_CHAR];
double kcal, fat, omega_3, omega_6, carb, protein, fib, price, weight;
} Food;
int main(void) {
int i = 0, scanresult;
Food Shelf[SIZE_OF_SHELF];
FILE *fp;
fp = fopen("foods.txt", "r");
if (! fp) {
printf("error loading file. bye.\n");
return 0;
}
for (i = 0; !feof(fp); i++) {
scanres = fscanf(fp, " %[^,],%lf,%lf,%lf,%lf,%lf,%lf,%lf,%lf,%lf ",
Shelf[i].name,
&Shelf[i].kcal, &Shelf[i].fat,
&Shelf[i].carb, &Shelf[i].protein,
&Shelf[i].fib, &Shelf[i].omega_3,
&Shelf[i].omega_6, &Shelf[i].price,
&Shelf[i].weight);
printf("scanres: %d, name: %s, kcal: %.2f, omega 3: %.2f, omega 6: %.2f, carb: %.2f, protein: %.2f, fib: %.2f, price: %.2f, weight: %.2fg\n",
scanres, Shelf[i].name, Shelf[i].kcal,
Shelf[i].omega_3, Shelf[i].omega_6, Shelf[i].carb,
Shelf[i].protein, Shelf[i].fib, Shelf[i].price,
Shelf[i].weight);
}
return 0;
}
If anybody can spot what I'm doing wrong, please let me know.
Check if the file has a Byte Order Mark (BOM) in the first three characters. You can use hexdump (or any binary editor) to inspect it.
File with BOM:
00000000 ef bb bf 42 72 65 61 64 2c 32 31 32 2c 32 2e 37 |...Bread,212,2.7|
00000010 2c 33 36 2c 36 2c 39 2e 38 2c 30 2e 30 31 2c 30 |,36,6,9.8,0.01,0|
00000020 2e 30 31 2c 31 30 2c 35 30 30 20 0a 50 61 73 74 |.01,10,500 .Past|
00000030 61 2c 33 34 37 2c 32 2e 35 2c 36 34 2c 31 33 2c |a,347,2.5,64,13,|
...
File without BOM :
00000000 42 72 65 61 64 2c 32 31 32 2c 32 2e 37 2c 33 36 |Bread,212,2.7,36|
00000010 2c 36 2c 39 2e 38 2c 30 2e 30 31 2c 30 2e 30 31 |,6,9.8,0.01,0.01|
00000020 2c 31 30 2c 35 30 30 20 0a 50 61 73 74 61 2c 33 |,10,500 .Pasta,3|
00000030 34 37 2c 32 2e 35 2c 36 34 2c 31 33 2c 37 2c 30 |47,2.5,64,13,7,0|
...
It's likely that, besides having a Byte Order Mark (BOM), the original copy of the foods.txt file was encoded using UTF-16, instead of ASCII or the more popular and compatible UTF-8. Taking a cue from wildplasser's answer, here is a hex dump of the first portion of the file in the little-endian variant of that encoding:
00000000 ff fe 42 00 72 00 65 00 61 00 64 00 2c 00 32 00 |..B.r.e.a.d.,.2.|
00000010 31 00 32 00 2c 00 32 00 2e 00 37 00 2c 00 33 00 |1.2.,.2...7.,.3.|
00000020 36 00 2c 00 36 00 2c 00 39 00 2e 00 38 00 2c 00 |6.,.6.,.9...8.,.|
00000030 30 00 2e 00 30 00 31 00 2c 00 30 00 2e 00 30 00 |0...0.1.,.0...0.|
00000040 31 00 2c 00 31 00 30 00 2c 00 35 00 30 00 30 00 |1.,.1.0.,.5.0.0.|
00000050 20 00 0a 00 50 00 61 00 73 00 74 00 61 00 2c 00 | ...P.a.s.t.a.,.|
00000060 33 00 34 00 37 00 2c 00 32 00 2e 00 35 00 2c 00 |3.4.7.,.2...5.,.|
The leading ff fe represents the byte order mark, and would account for the mysterious ■ that showed up in the output name: ■B. Thereafter, every other byte is 0, which is why "Bread" was truncated to "B". And then fscanf's first %lf sees "r\0e\0a\0d", and can't parse that as a double, which is why fscanf returns 1 instead of 10.
copying the contents of the .txt file into a new .txt file solved the problem. It was originated in an .xls file, my guess is, there the weird BOM stuff, mentioned by some of you, comes from.
I've been working on a little script that determines which disk is usable in our specific system. There has to be basically less than 1 TB of space on the disk in order to be considered as usable.
This is the complete code:
#echo off
chcp 65001>nul
setlocal enabledelayedexpansion
wmic logicaldisk get caption,freespace>c:\cmd\1\getDiskInfo.txt
for /f "tokens=1,2 eol=C" %%I in (C:\cmd\1\getDiskInfo.txt) do (
set diskCaption=%%I
set diskFreeSpace=%%J
set captionFreeSpace=!diskCaption! !diskFreeSpace!
for /f "tokens=1,2 delims= " %%X in ("!captionFreeSpace!") do (
if [%%Y] neq [] set usedDisks=%%X %%Y
for /f "tokens=1,2 delims= " %%A in ("!usedDisks!") do (
set freeSpaceFirstChar=%%B
set /a freeSpaceFirstChar=!freeSpaceFirstChar:~0,1!
if !freeSpaceFirstChar! gtr 1 set usableDisk=%%A
)
)
)
echo %usableDisk%
pause
But the output I get for %usableDisk% is always ECHO is off. which means that %usableDisk% does not even exist. I've done a little investigation and from my understanding it's because of a character coding related problem. I've copied the content of getDiskInfo.txt to another .txt file, and the batch file managed to give me the right output with that text file created by me.
The contents of getDiskInfo.txt and the other text file was both:
Caption FreeSpace
A:
B: 1098552672256
C: 40824201216
D:
E: 1042498560000
F: 40222941184
The output of the original script created file was ECHO is off.. The output with the text file created by me was F: which is the correct output because we can't use the system drive C:.
So I tried echo END OF FILE>>getDiskInfo.txt and it added the line 久⁄䙏䘠䱉 to the script created file, but the same command added END OF FILE to my text file. I'm completely lost on this one.
Do you have any suggestions or probable solutions?
There are multiple problems to solve for this task to get the drive letter of a (local) hard disk drive with less than one TiB (1 099 511 627 776 bytes) free space which is not the system drive.
1. Character encoding of WMIC output
WMIC outputs data always with character encoding UTF-16 Little Endian with byte order mark abbreviated as UTF-16LE+BOM.
So the data output
Caption FreeSpace
A:
B: 1098552672256
C: 40824201216
D:
E: 1042498560000
F: 40222941184
is as byte stream with byte offset left to : and ASCII representation right to ;:
0000h: FF FE 43 00 61 00 70 00 74 00 69 00 6F 00 6E 00 ; ÿþC.a.p.t.i.o.n.
0010h: 20 00 20 00 46 00 72 00 65 00 65 00 53 00 70 00 ; . .F.r.e.e.S.p.
0020h: 61 00 63 00 65 00 20 00 20 00 20 00 20 00 20 00 ; a.c.e. . . . . .
0030h: 20 00 0D 00 0A 00 41 00 3A 00 20 00 20 00 20 00 ; .....A.:. . . .
0040h: 20 00 20 00 20 00 20 00 20 00 20 00 20 00 20 00 ; . . . . . . . .
0050h: 20 00 20 00 20 00 20 00 20 00 20 00 20 00 20 00 ; . . . . . . . .
0060h: 20 00 20 00 20 00 0D 00 0A 00 42 00 3A 00 20 00 ; . . .....B.:. .
0070h: 20 00 20 00 20 00 20 00 20 00 20 00 31 00 30 00 ; . . . . . .1.0.
0080h: 39 00 38 00 35 00 35 00 32 00 36 00 37 00 32 00 ; 9.8.5.5.2.6.7.2.
0090h: 32 00 35 00 36 00 20 00 20 00 0D 00 0A 00 43 00 ; 2.5.6. . .....C.
00a0h: 3A 00 20 00 20 00 20 00 20 00 20 00 20 00 20 00 ; :. . . . . . . .
00b0h: 34 00 30 00 38 00 32 00 34 00 32 00 30 00 31 00 ; 4.0.8.2.4.2.0.1.
00c0h: 32 00 31 00 36 00 20 00 20 00 20 00 20 00 0D 00 ; 2.1.6. . . . ...
00d0h: 0A 00 44 00 3A 00 20 00 20 00 20 00 20 00 20 00 ; ..D.:. . . . . .
00e0h: 20 00 20 00 20 00 20 00 20 00 20 00 20 00 20 00 ; . . . . . . . .
00f0h: 20 00 20 00 20 00 20 00 20 00 20 00 20 00 20 00 ; . . . . . . . .
0100h: 20 00 0D 00 0A 00 45 00 3A 00 20 00 20 00 20 00 ; .....E.:. . . .
0110h: 20 00 20 00 20 00 20 00 31 00 30 00 34 00 32 00 ; . . . .1.0.4.2.
0120h: 34 00 39 00 38 00 35 00 36 00 30 00 30 00 30 00 ; 4.9.8.5.6.0.0.0.
0130h: 30 00 20 00 20 00 0D 00 0A 00 46 00 3A 00 20 00 ; 0. . .....F.:. .
0140h: 20 00 20 00 20 00 20 00 20 00 20 00 34 00 30 00 ; . . . . . .4.0.
0150h: 32 00 32 00 32 00 39 00 34 00 31 00 31 00 38 00 ; 2.2.2.9.4.1.1.8.
0160h: 34 00 20 00 20 00 20 00 20 00 0D 00 0A 00 ; 4. . . . .....
But Windows command processor expects a one byte per character encoding using the code page as output on running in an opened command prompt window the command chcp. The code page depends on which country is configured for the account used to run the command process processing the batch file.
The command line chcp 65001>nul to change to Unicode encoding UTF-8 is of no help here.
Processing UTF-16LE encoded output with FOR directly cause troubles as documented on Stack Overflow multiple times, see for example How to correct variable overwriting misbehavior when parsing output?
A solution would be to redirect the output of WMIC into a temporary file, output this temporary file to handle STDOUT (standard output) of a command process started in background with %ComSpec% /c using the command TYPE with capturing this output by the command process executing the batch file, process this ASCII output line by line, and finally delete the temporary file.
#echo off
setlocal EnableExtensions DisableDelayedExpansion
%SystemRoot%\System32\wbem\wmic.exe LOGICALDISK GET Caption,FreeSpace >"%TEMP%\%~n0.tmp"
if not exist "%TEMP%\%~n0.tmp" goto EndBatch
for /F "skip=1 tokens=1,2" %%I in ('type "%TEMP%\%~n0.tmp"') do echo %%I %%J
del "%TEMP%\%~n0.tmp"
:EndBatch
endlocal
In this case FOR processes the ASCII byte stream:
000h: 43 61 70 74 69 6F 6E 20 20 46 72 65 65 53 70 61 ; Caption FreeSpa
010h: 63 65 20 20 20 20 20 20 0D 0A 41 3A 20 20 20 20 ; ce ..A:
020h: 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 ;
030h: 20 20 0D 0A 42 3A 20 20 20 20 20 20 20 31 30 39 ; ..B: 109
040h: 38 35 35 32 36 37 32 32 35 36 20 20 0D 0A 43 3A ; 8552672256 ..C:
050h: 20 20 20 20 20 20 20 34 30 38 32 34 32 30 31 32 ; 408242012
060h: 31 36 20 20 20 20 0D 0A 44 3A 20 20 20 20 20 20 ; 16 ..D:
070h: 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 ;
080h: 0D 0A 45 3A 20 20 20 20 20 20 20 31 30 34 32 34 ; ..E: 10424
090h: 39 38 35 36 30 30 30 30 20 20 0D 0A 46 3A 20 20 ; 98560000 ..F:
0a0h: 20 20 20 20 20 34 30 32 32 32 39 34 31 31 38 34 ; 40222941184
0b0h: 20 20 20 20 0D 0A ; ..
But it is in general always better to avoid the usage of a temporary file as there is never the guarantee that the temporary file can be created at all during the execution of the batch file.
2. System drive is not always C:
Windows is installed by default on drive C: and so the system drive is C:. But Windows can be installed also to a different drive in which case the system drive is not C:. Any code depending on default data instead of using the appropriate data is a not good written code.
There is predefined the Windows environment variable SystemDrive with the drive letter and the colon of the drive on which active Windows is installed. The environment variable SystemRoot contains the path to the Windows directory which contains the directory System32 with all executables from the Windows Commands list which are not internal commands of cmd.exe.
All those system environment variables can be seen with their values on opening a command prompt window and running set system. Running just set outputs all environment variables with their values defined for the current user account.
3. Integer value range is limited to 32-bit signed integer
The Windows command processor cmd.exe uses always only 32-bit signed integer on evaluation of an arithmetic expression with set /A and for comparing integer values with command IF on using the operators EQU, NEQ, LSS, LEQ, GTR, GEQ.
Therefore the integer value range is −2147483648 to 2147483647. So maximum is one byte less than 2 GiB. The value is 1099511627776 requires 64-bit integer value range not supported by cmd.exe.
BTW: if [%%Y] neq [] is never a good comparison because of [ and ] have no special meaning for Windows command processor and neq results here first in the approach to convert the left string into a 32-bit signed integer value which fails because of [ is an invalid character for an integer value and therefore running next a string comparison with condition being true if the string comparison returns not 0, i.e. the compared strings are not equal. Better would be if not "%%Y" == "" which runs directly and more safely a string comparison on not equal strings. See Symbol equivalent to NEQ, LSS, GTR, etc. in Windows batch files for details on how command IF executes a string comparison.
Solution to get drives with less than one TiB free space
It is a good idea to read the documentation of a class of which properties are accessed using the Windows Management Instrumentation Command-line utility. This is here the Win32_LogicalDisk class.
There is in addition to FreeSpace of type uint64 and DeviceID of type string instead of Caption perhaps also useful the property DriveType of type uint32 to filter out drives of wrong type in addition to drives with too much free space and the system drive by using a where clause on wmic execution.
#echo off
setlocal EnableExtensions DisableDelayedExpansion
set "UsableDrive="
for /F "skip=1 tokens=1,2" %%I in ('""%SystemRoot%\System32\wbem\wmic.exe" LOGICALDISK where (DriveType=3 and FreeSpace^<1099511627776 and DeviceID!='C:') GET DeviceID,FreeSpace 2>nul"') do (
echo Drive %%I has %%J free bytes.
if not defined UsableDrive set "UsableDrive=%%I"
)
if defined UsableDrive echo Selected drive %UsableDrive%
endlocal
Important to know here is that FOR starts in background with %ComSpec% /c one more command process with the command line specified within ' appended as additional arguments. For that reason the command line with WMIC must fulfill the Windows command processor requirements described by help output on running cmd /? in a command prompt window on being in total three times parsed.
The first parsing is done by cmd.exe processing the batch file before executing command FOR.
The second parsing is done by cmd.exe instance on starting it in background by the cmd.exe instance processing the batch file with the following command line on Windows installed to C:\Windows.
C:\Windows\System32\cmd.exe /c ""C:\WINDOWS\System32\wbem\wmic.exe" LOGICALDISK where (DriveType=3 and FreeSpace^<1099511627776 and DeviceID!='C:') GET DeviceID,FreeSpace 2>nul"
The third parsing is done by the background command process before executing wmic.exe. The operator < in where clause must be interpreted as literal character and not as redirection operator which is the reason why < is escaped with ^ to run wmic.exe finally with:
"C:\Windows\System32\wbem\wmic.exe" LOGICALDISK where (DriveType=3 and FreeSpace<1099511627776 and DeviceID!='C:') GET DeviceID,FreeSpace
WMIC filters out with DriveType=3 all network drives, floppy disk drives, CD and DVD drives and other removable drives, RAM disks, etc. Hard disks connected to the computer using an external USB port or an eSATA port are not filtered out because of those drives have also value 3 for the drive type. Windows cannot determine if a hard disk is mounted inside the casing of the computer or outside. So a local hard disk is any hard disk connected to the computer, internal and external hard disks.
The system drive is filtered out with second condition DeviceID!='%SystemDrive%'.
The last condition FreeSpace<1099511627776 results in ignoring all drives with 1 TiB or more free space.
So the list is reduced already to those drives which fulfill all three conditions.
For understanding the used commands and how they work, open a command prompt window, execute there the following commands, and read entirely all help pages displayed for each command very carefully.
cmd /?
del /?
echo /?
endlocal /?
for /?
goto /?
if /?
set /?
setlocal /?
type /?
wmic /?
wmic logicaldisk /?
wmic logicaldisk get /?
See also the Microsoft article about Using command redirection operators for an explanation of > and 2>nul.
If I understand your requirements correctly, the following, much shorter, single line batch-file, should output only the usable drives, i.e. which, are not your system drive, and have less than 1 TB of free disk space.
#(For /F "Tokens=1-2" %%G In ('""%__AppDir__%wbem\WMIC.exe" LogicalDisk Where (DeviceID !='%SystemDrive%' And FreeSpace ^<'1000000000000') Get DeviceID,FreeSpace 2>NUL|"%__AppDir%find.exe" ":""')Do #Set "_= %%H"&SetLocal EnableDelayedExpansion&Echo %%G !_:~-14!&EndLocal)&Pause
As you did not explain in your question, what you were doing with the data retrieved, I have simply printed them to the console window in a similar format to your original getDiskInfo.txt, (I even right aligned the bytes column just for fun!).
TL;DR: I want to read raw h264 streams from AVI/MP4 files, even broken/incomplete.
Almost every document about h264 tells me that it consists of NAL packets. Okay. Almost everywhere told to me that the packet should start with a signature like 00 00 01 or 00 00 00 01. For example, https://stackoverflow.com/a/18638298/8167678, https://stackoverflow.com/a/17625537/8167678
The format of H.264 is that it’s made up of NAL Units, each starting
with a start prefix of three bytes with the values 0x00, 0x00, 0x01
and each unit has a different type depending on the value of the 4th
byte right after these 3 starting bytes. One NAL Unit IS NOT one frame
in the video, each frame is made up of a number of NAL Units.
Okay.
I downloaded random_youtube_video.mp4 and strip out one frame from it:
ffmpeg -ss 10 -i random_youtube_video.mp4 -frames 1 -c copy pic.avi
And got:
Red part - this is part of AVI container, other - actual data.
As you can see, here I have 00 00 24 A9 instead of 00 00 00 01
This AVI file plays perfectly
I do same for mp4 container:
As you can see, here exact same bytes.
This MP4 file plays perfectly
I try to strip out raw data:
ffmpeg -i pic.avi -c copy pic.h264
This file can't play in VLC or even ffmpeg, which produced this file, can't parse it:
I downloaded mp4 stream analyzer and got:
MP4Box tells me:
Cannot find H264 start code
Error importing pic.h264: BitStream Not Compliant
It very hard to learn internals of h264, when nothing works.
So, I have questions:
What actual data inside mp4?
What I must read to decode that data (I mean different annex-es)
How to read stream and get decoded image (even with ffmpeg) from this "broken" raw stream?
UPDATE:
It seems bug in ffmpeg:
When I do double conversion:
ffmpeg -ss 10 -i random_youtube_video.mp4 -frames 1 -c copy pic.mp4
ffmpeg pic.mp4 -c copy pic.h264
But when I convert file directly:
ffmpeg -ss 10 -i random_youtube_video.mp4 -frames 1 -c copy pic.h264
I have NALs signatures and one extra NAL unit. Other bytes are same (selected).
This is bug?
UPDATE
Not, this is not bug, U must use option -bsf h264_mp4toannexb to save stream as "Annex B" format (with prefixes)
"I want to read raw h264 streams from AVI files, even broken/incomplete."
"Almost everywhere told to me that the packet should start with a signature like : 00 00 01 or 00 00 00 01"
"...As you can see, here I have 00 00 24 A9 instead of 00 00 00 01"
Your H264 is in AVCC format which means it uses data sizes (instead of data start codes). It is only Annex-B that will have your mentioned signature as start code.
You seek frames, not by looking for start codes, but instead you just do skipping by frame sizes to reach the final correct offset of a (requested) frame...
AVI processing :
Read size (four) bytes (32-bit integer, Little Endian).
Extract the next following bytes up to size amount.
This is your H.264 frame (in AVCC format), decode the bytes to view image.
To convert into Annex-B, try replacing first 4 bytes of H.264 frame bytes with 00 00 00 01.
Consider your shown AVI bytes (see first picture) :
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
00 00 00 00 4C 49 53 54 BA 24 00 00 6D 6F 76 69 ....LISTº$..movi
30 30 64 63 AD 24 00 00 00 00 24 A9 65 88 84 27 00dc.$....$©eˆ„'
C7 11 FE B3 C7 83 08 00 08 2A 7B 6E 59 B5 71 E1 Ç.þ³Çƒ...*{nYµqá
E3 9C 0E 73 E7 10 50 00 18 E9 25 F7 AA 7D 9C 30 ãœ.sç.P..é%÷ª}œ0
E6 2F 0F 20 00 3A 64 AA CA 5E 4F CA FF AE 20 04 æ/. .:dªÊ^OÊÿ® .
07 81 40 00 48 00 0A 28 71 21 84 48 06 18 90 0C ..#.H..(q!„H....
31 14 57 9E 7A CD 63 A0 E0 9B 96 69 C5 18 AE F2 1.WžzÍc à›–iÅ.®ò
E6 07 02 29 01 20 10 70 A1 0F 8C BC 73 F0 78 FA æ..). .p¡.Œ¼sðxú
9E 1D E1 C2 BF 8C 62 CE CE AC 14 5A A4 E1 45 44 ž.á¿ŒbÎά.Z¤áED
38 38 85 DB 12 57 3E F6 E0 FB AE 03 04 21 62 8D 88…Û.W>öàû®..!b.
F6 F1 1E 37 1C A2 FF 75 1C F1 02 66 0C 92 07 06 öñ.7.¢ÿu.ñ.f.’..
15 7C 90 15 6F 7D FC BD 13 1E 2B 0C 14 3C 0C 00 .|..o}ü½..+..<..
B0 EA 6F 53 B4 98 D7 80 7A 68 3E 34 69 20 D2 FA °êoS´˜×€zh>4i Òú
F0 91 FC 75 C6 00 01 18 C0 00 3B 9A C5 E2 7D BF ð‘üuÆ...À.;šÅâ}¿
Some explanation :
Ignore leading multiple 00 bytes.
4C 49 53 54 D6 3C 00 00 6D 6F 76 69 including 30 30 64 63 = AVI "List" header.
AD 24 00 00 == decimal 9389 is AVI's own size of H264 item (must read in Little Endian).
Notice that the AVI bytes include...
- a note of item's total size (AD 24 00 00... or reverse for Little Endian : 00 00 24 AD)
- followed by item data (00 00 24 A9 65 88 84 27 ... etc ... C5 E2 7D BF).
This size includes both the 4 bytes of the AVI's"size" entry + expected bytes length of the item's own bytes. Can be written simply as:
AVI_Item_Size = ( 4 + item_H264_Frame.length );
H.264 video frame bytes in AVI :
Next follows the item data, which is the H.264 video frame. By sheer coincidence of formats/bytes layout, it too holds a 4-byte entry for data's size (since your H264 is in AVCC format, if it was Annex-B then you would be seeing start code bytes here instead of size bytes).
Unlike AVI bytes, these H264 size bytes are written in Big Endian format.
00 00 24 A9 = size of bytes for this video frame (instead of start code : 00 00 00 01).
65 88 84 27 C7 11 FE B3 C7 = H.264 keyframe (always begins X5, where the X value is based on other settings).
Remember after four size bytes (or even start codes) if followed by...
byte X5 = keyframe (IDR), example byte 65.
byte X1 = P or B frame, example byte 41.
byte X6 = SEI (Supplemental Enhancement Information).
byte X7 = SPS (Sequence Parameter Set).
byte X8 = PPS (Picture Parameter Set).
bytes 00 00 00 X9 = Access unit delimiter.
You can find the H.264 if you search for exact same bytes within AVI file. See third picture, these are your H.264 bytes (they are cut & pasted into the AVI container).
Sometimes a frame is sliced into different NAL units. So if you extract a key frame and it only shows 1/2 or 1/3 instead of full image, just grab next one or two NAL and re-try the decode.
I' m working on Mastercard Paypass transactions, I Have sent a READ RECORD command and got the result:
70 81 AB 57 11 54 13 33 00 89 60 10 83 D2 51 22
20 01 23 40 91 72 5A 08 54 13 33 00 89 60 10 83
5F 24 03 25 12 31 5F 25 03 04 01 01 5F 28 02 00
56 5F 34 01 01 8C 21 9F 02 06 9F 03 06 9F 1A 02
95 05 5F 2A 02 9A 03 9C 01 9F 37 04 9F 35 01 9F
45 02 9F 4C 08 9F 34 03 8D 0C 91 0A 8A 02 95 05
9F 37 04 9F 4C 08 8E 0E 00 00 00 00 00 00 00 00
42 03 5E 03 1F 03 9F 07 02 FF 00 9F 08 02 00 02
9F 0D 05 00 00 00 00 00 9F 0E 05 00 08 00 60 00
9F 0F 05 00 00 00 00 00 9F 42 02 09 78 9F 4A 01
82 9F 14 01 00 9F 23 01 00 9F 13 02 00 00
This response contains TLV data objects (without spaces). I have converted the response as described in the following:
// Read Record 1 with SFI2
//---------------------------------SEND READ RECORD-------------------
inCtlsSendVAPDU(0x2C,0x03,(unsigned char *)"\x00\xB2\x01\x14\x00",5);
clrscr();
inRet = inCTLSRecv2(Response, 269);
LOG_HEX_PRINTF("Essai EMV4 Read record 1 EMV Paypass:",Response,inRet);
if(Response[14]==0x70)
{
sprintf(Response_PPSE,"%02X%02X",Response[12],Response[13]);//To retrieve length of received data
t1=hexToInt(Response_PPSE);// Convert length to integer
t11=t1-2;
i=14;
k=0;
//--------------------------- Converting data to be used later---------
while(i<t11+14)// 14 to escape the header+ command+ status+ length
{
sprintf(READ1+(2*k),"%02X",Response[i]);
i++;
k++;
}
Now I should check if this Response contains the Mandatory Tags:
5A - Application Primary Account Number (PAN)
5F24 - Application Expiration Date
8C - Card Risk Management Data Object List 1 (CDOL1)
8D - Card Risk Management Data Object List 2 (CDOL2)
So I tried the following to check for the 5A tag (Application Primary Account Number (PAN)):
i=0;
t11=2*t11;
while(i<=t11)
{
strncpy(Response_PPSE,READ1+i,2);
if(strncmp(Response_PPSE,"\x05\x0A")==0)
{
write_at("true",4,1,1);// Just to test on the terminal display
goto end;
}
else
i=i+2;
}
goto end;
I don't know why nothing is displayed on the terminal, The if block is not executed!
I tried to print the 5A tag manually by:
strncpy(Response_PPSE,READ1+44,2);
write_at(Response_PPSE,strlen(Response_PPSE),1,1);
And it display the right value!!
Can someone helps to resolve this issue?
You don't find that tag because you are not searching for the string "5A" but for the string "\x05\x0A" (ENQ character + line feed character). Moreover, I wonder if the above code actually compiles as you did not specify the mandatory length argument to strncmp(). You could try something like
if(strncmp(Response_PPSE,"5A", 2)==0)
instead.
However, you should understand that you are scanning the whole response data for the value 5A. Therefore, finding this value could also mean that it was part of some other TLV tag's data field, length field or even part of a multi-byte tag field. It would therefore make sense to implement (or use an existing) TLV parser for the BER (Basic Encoding Rules) format.
It's not a good approach to search a specific byte in a raw byte-stream data using strings functions at first place.
The generic TLV parser is a very easy algorithm and you will do it in 30 minutes or so.
In general a pseudo-code for TLV parser that look for a specific Tag would be something like this:
index = 0
while (byte[i] != 0x5A or EOF)
{
length = DecodeLength(byte[i+1])
i += length + 2 // + 1 for L (length) byte itself, it might be encoded with
// 2 bytes so the function DecodeLength can return the number
// of bytes lenght has been encoded
// +1 for T (tag) byte
}
if(EOF) return tag_not_found
return byte[i + 2], length // pointer to data for Tag '5A'and length of data