Currently I'm trying to read the bytes from the IDAT chunk of a PNG image, in C. I am able to get all the other info, including the said array of bytes.
The problem arises whenever I try to decompress said array with zlib's uncompress() method.
[ ... ]
int decompress(Chunk * _chunk, Image * _image)
{
uLongf compressedSize = _chunk->length;
byte * uncompressedData = NULL;
uLongf uncompressedSize = 0;
int ret = uncompress(uncompressedData, &uncompressedSize, _chunk->data, compressedSize);
if(ret != Z_OK)
{
fprintf(stderr, "Error: failed to uncompress IDAT chunk data. ERR CODE: %d\n", ret);
return -1;
}
[ ... ]
}
The chunk struct is defined as such:
typedef struct chunk
{
uint32_t length;
byte chunkType[4];
byte *data;
} Chunk;
The byte type is just an unsigned char, and the image struct is defined as follows:
typedef struct image
{
uint32_t width;
uint32_t height;
byte bitDepth;
byte colorType;
byte compression;
byte filter;
byte interlace;
} Image;
The test image's HEX representation is:
89 50 4E 47 0D 0A 1A 0A 00 00 00 0D 49 48 44 52
00 00 00 11 00 00 00 12 04 03 00 00 00 4F D7 28
67 00 00 00 30 50 4C 54 45 00 00 00 80 00 00 00
80 00 80 80 00 00 00 80 80 00 80 00 80 80 80 80
80 C0 C0 C0 FF 00 00 00 FF 00 FF FF 00 00 00 FF
FF 00 FF 00 FF FF FF FF FF 7B 1F B1 C4 00 00 00
09 70 48 59 73 00 00 0E C4 00 00 0E C4 01 95 2B
0E 1B 00 00 00 28 49 44 41 54 08 D7 63 D8 0D 05
1B 18 36 30 00 01 FF FF FF 24 B1 FE FF FF C0 C0
40 0E 6B FF FF FF 20 73 48 60 C1 5D 0A 00 BB 1A
49 27 39 98 BC 6E 00 00 00 00 49 45 4E 44 AE 42
60 82
And the bytes of the IDAT chunk are:
08 D7 63 D8 0D 05 1B 18 36 30 00 01 FF FF FF 24 B1 FE FF FF C0 C0 40 0E 6B FF FF FF 20 73 48 60 C1 5D 0A 00 BB 1A 49 27
It must be noted that I'm not taking the CRC of the chunk as well; from my understanding it shouldn't be a problem.
Any idea as to why the uncompress() method is returning Z_DATA_ERROR?
You're not giving uncompress() anywhere to put the uncompressed data! uncompressedData cannot be NULL.
UPDATE:
CryptoAuthLib solve the problem. Many thanks #zaph
I'm working on ATSHA204 security chip, where a part of work is to calculate SHA-256 alike the chip does. I'm using
CryptoAuthentication ATSHA204A Development Library 2.0.1 [from hxxp://www.atmel.com/tools/CRYPTOAUTHENTICATIONATSHA204DEVELOPMENTLIBRARY.aspx] Implemented in C.
Well, while try any data >= 64 bytes, everything works fine. But anything < 64 bytes differ with the CHIP's calculation.
Have a look below,
message:
20 4D 56 EF 56 F6 27 71
AD CF D7 07 FC BA BE 21
A8 7D 7C AF F5 54 6F ED
27 2C F9 ED 75 B3 94 C7
1C 04 02 00 EE 01 23 00
00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00
0E EA 11 36 8E 89 D2 11
07 C1 1B 64 B6 D0 34 DA
A6 DB 66 0F 14 FB DB C6
26 7D 6A 3A 62 83 47 13
digest from Atmel's algorithm:
8C E2 28 98 6A 21 A3 40
A4 42 97 CE 4C 80 70 DF
87 97 61 B3 FC CD A2 BF
8D F0 89 EE 29 B8 68 BF
And while taking something < 64 length to same implementation,
message:
11 CC 32 8B 30 ED 04 81
73 96 A9 F8 3B F7 9E 72
23 DC 05 29 3C 65 36 70
3A 03 1C E2 C2 2D 99 4E
00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00
00 00 00 00 16 00 00
digest from Atmel's algorithm:
40 3D DC C0 93 A6 4B A5
01 36 7D 59 EC 8C C4 59
15 20 6D 52 A7 D5 ED 40
9B 95 31 D3 49 37 F3 F8
What should add with message which less than 64 bytes to get digest alike the CHIP calculation. [unfortunately yet failed any reply from ATMEL community]
I am trying to stream Olimex A13 encoded data to RTMP server with librtmp and view it on VLC. The problem is I cannot find how to correctly wrap Cedar data to flash container... I have found this example which looks exactly what I would need, but VLC has still problems understanding it -
No suitable decoder module:
VLC does not support the audio or video format "undf". Unfortunately there is no way for you to fix this.
I tried grabbing data through rtmpdump and it appears I cannot disable audio track which VLC is trying to get from the header
Format : Flash Video
File size : 195 KiB
Duration : 1mn 27s
Overall bit rate : 1 464 Kbps
_Server : NGINX RTMP (github.com/arut/nginx-rtmp-module)
_displayWidth : 640.000
_displayHeight : 480.000
_fps : 25.000
Video
Format : AVC
Format/Info : Advanced Video Codec
Codec ID : 7
Duration : 1mn 27s
Width : 640 pixels
Height : 480 pixels
Display aspect ratio : 4:3
Frame rate mode : Constant
Frame rate : 25.000 fps
Bit depth : 8 bits
Bits/(Pixel*Frame) : 0.191
Audio
The last line for 'Audio' is what I suspect is causing VLC to err. Moreover I am sure the video data I am trying to send is not correctly wrapped.
From what I understood looking into example is that FLV expects a general stream description which can be flagged for video and audio stream information, other examples I found (including ffmpeg ) use some magic number following onMetaData tag, but I dont think I do that by following an example from above:
STR2AVAL(av, "onMetaData");
enc = AMF_EncodeString(enc, pend, &av);
*enc++ = AMF_ECMA_ARRAY;
enc = AMF_EncodeInt32(enc, pend, 5+5+2); //5 - video 5 - audio
The dump of my rtmp header is:
sending 307 as header
00000000 02 00 0d 40 73 65 74 44 61 74 61 46 72 61 6d 65 ...#setDataFrame
00000010 02 00 0a 6f 6e 4d 65 74 61 44 61 74 61 03 00 06 ...onMetaData...
00000020 61 75 74 68 6f 72 02 00 00 00 09 63 6f 70 79 72 author.....copyr
00000030 69 67 68 74 02 00 00 00 0b 64 65 73 63 72 69 70 ight.....descrip
00000040 74 69 6f 6e 02 00 00 00 08 6b 65 79 77 6f 72 64 tion.....keyword
00000050 73 02 00 00 00 06 72 61 74 69 6e 67 02 00 00 00 s.....rating....
00000060 0a 70 72 65 73 65 74 6e 61 6d 65 02 00 06 43 75 .presetname...Cu
00000070 73 74 6f 6d 00 05 77 69 64 74 68 00 40 84 00 00 stom..width.#...
00000080 00 00 00 00 00 05 77 69 64 74 68 00 40 84 00 00 ......width.#...
00000090 00 00 00 00 00 06 68 65 69 67 68 74 00 40 7e 00 ......height.#~.
000000a0 00 00 00 00 00 00 09 66 72 61 6d 65 72 61 74 65 .......framerate
000000b0 00 40 39 00 00 00 00 00 00 00 0c 76 69 64 65 6f .#9........video
000000c0 63 6f 64 65 63 69 64 02 00 04 61 76 63 31 00 0d codecid...avc1..
000000d0 76 69 64 65 6f 64 61 74 61 72 61 74 65 00 40 96 videodatarate.#.
000000e0 e3 60 00 00 00 00 00 08 61 76 63 6c 65 76 65 6c .`......avclevel
000000f0 00 3f f0 00 00 00 00 00 00 00 0a 61 76 63 70 72 .?.........avcpr
00000100 6f 66 69 6c 65 00 40 50 80 00 00 00 00 00 00 17 ofile.#P........
00000110 76 69 64 65 6f 6b 65 79 66 72 61 6d 65 5f 66 72 videokeyframe_fr
00000120 65 71 75 65 6e 63 79 00 40 20 00 00 00 00 00 00 equency.# ......
00000130 00 00 09 ...
After that application send PSP and SPS data which I am not sure I encode correctly too:
This is what I have in my encoder application
psp struct start
00000000 34 32 30 31 66 00 00 00 00 00 5a 30 49 41 48 2b 4201f.....Z0IAH+
00000010 56 41 55 42 37 49 00 00 00 00 00 00 00 00 00 00 VAUB7I..........
00000020 00 00 00 00 00 00 00 00 61 4f 34 78 45 67 3d 3d ........aO4xEg==
00000030 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
00000040 00 00 00 00 00 00 ......
psp struct end
And this is what I send to RTMP server:
sending 38 as sps/pps info
00000000 17 00 00 00 00 01 42 c0 15 03 01 00 0d 67 5a 30 ......B......gZ0
00000010 49 41 48 2b 56 41 55 42 37 49 01 00 09 68 61 4f IAH+VAUB7I...haO
00000020 34 78 45 67 3d 3d 4xEg==
info frame sent
So the questions are:
How do you modify the header to explicitly state that its video only?
How does one wrap video data in NAL format and how often do you have to transmit SPS and PPS data?
Are there any online services to inspect the RTMP data to understand whats wrong with it? VLC doesnt help much...
I have a file in hex look like following.
Part 1
1F 00 1C 3A 1F 00 25 3A 1F 00 09 3A 1F 00 50 3A
1F 00 5A 3A 1F 00 5C 3A 1F 00 5B 3A 1F 00 59 3A
1F 00 5D 3A 03 00 FE 0F 1F 00 01 30 1F 00 06 3A
1F 00 11 3A 1F 00 44 3A 1F 00 4F 3A 1F 00 45 3A
1F 10 56 3A 1F 10 54 3A 1F 00 03 30 1F 00 02 30
03 00 55 3A 03 00 71 3A 1F 00 29 3A 1F 00 27 3A
1F 00 2A 3A 1F 00 28 3A 1F 00 26 3A 1F 00 51 3A
1F 00 08 3A 1F 00 24 3A 1F 00 21 3A 1F 00 16 3A
1F 00 17 3A 1F 00 18 3A 1F 00 19 3A 1F 00 0A 80
1F 00 48 3A 1F 10 58 3A 02 00 4D 3A 40 00 42 3A
40 00 41 3A 1F 00 04 30 1F 10 00 80 03 00 01 80
02 01 FF 0F
Part 2
40 00 08 30 03 00 71 3A 03 00 55 3A 1F 00 02 30
1F 00 03 30 1F 10 54 3A 1F 10 56 3A 1F 00 06 3A
1F 00 01 30 03 00 FE 0F 02 01 FF 0F
Part 3
40 00 08 30 03 00 71 3A 03 00 55 3A 1F 00 02 30
1F 00 03 30 1F 10 54 3A 1F 10 56 3A 1F 00 11 3A
1F 00 06 3A 1F 00 01 30 03 00 FE 0F 02 01 FF 0F
Part 4
1F 00 5D 3A 03 00 71 3A 03 00 55 3A 1F 00 02 30
1F 00 03 30 1F 10 54 3A 1F 10 56 3A 1F 00 45 3A
1F 00 4F 3A 1F 00 44 3A 1F 00 11 3A 1F 00 06 3A
1F 00 01 30 02 01 FF 0F
Part 5
40 00 08 30 1F 00 03 30 1F 00 02 30 1F 00 01 30
02 01 FF 0F
My file has so many data parts like above. Each data part ending with bytes 02 01 FF 0F. Here I am showing only five parts.
Each time I don't know how many bytes are available in each data part. All the parts are together in one file. I want to read all of the parts and store all parts in a separate memory, i.e. array or linked list, so that I am become able to access all bytes.
My code is as follows:
int n = 500; //where n is the number of parts in my file
for(int i = 0; i < n; i++)
{
rewind(pFile);
fread(&a, 1, 4, pFile);
if((a==0x0FFF0102) || (a==0x8004001F) || (a==0x800D001F))
{
continue;
}
fseek(pFile, -4, SEEK_CUR);
while(a!=0x0FFF0102)
{
fread(&a, 1, 4, pFile);
// now what can I do here. where I store all above hex data.
}
}
This answer stands if your file is much smaller than the memory available on your machine.
Allocate a large buffer that you think will fit the entire file using calloc().
Start reading the file by chunks.
Keep track of how much you have read.
Put the read contents into the allocated buffer.
if the buffer can't fit the read data, use realloc() to enlarge the buffer
Once you have read the entire file, create another array, this time of char* pointers - for the parts start list
Add pointer to the beginning of the read buffer to the parts start list
Start reading the read buffer and search for the 02 01 FF 0F part-ending bytes
Add the location after each found byte sequence to the parts start list. Again, realloc as needed.
Replace the end bytes with '\0' if the file is a text file; otherwise do nothing
Repeat until reached end of the buffer
treat each entry in parts start list as a string and perform your operations on it. Its end is marked by the start of next entry.
I am building my answer upon Dariusz' answer. Normally I would put that into a comment, but I write too much for it.
The problem is that, depending on your program's general usage of memory, malloc()ing smaller chunks of memory might be more successful.
malloc() enough memory so that the pointers will fit in.
Read chunk by chunk into some temporary buffer.
Once you have a full chunk, malloc() memory for it and put the data in. Remember to note the size somewhere.
Put that pointer into the pointer list, realloc()ing it as needed.
Dariusz' solution has advantages - e.g., you can get the size of the chunks by just subtracting pointers - but, as said, one large chunk for the while file might lead to problems, depending on the file size.
Another solution would be to use mmap(), which allows you to map a disk file in memory.
This gives you a pointer to virtual memory where you exactly find the bytes from your disk file.
In this case, stick closer to Dariusz' answer, starting at point 4.
Sledgehammer, meet Nut; Nut, Sledgehammer.
#include <assert.h>
#include <stdint.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include "imageprt.h"
#include "stderr.h"
enum { EOS_MARKER = 0x0FFF0102 };
typedef struct Section
{
size_t length;
int32_t *data;
} Section;
typedef struct Description
{
size_t n_sections;
Section *sections;
int32_t *data;
} Description;
static void free_description(Description *dp);
static Description *read_description(FILE *fp, char const *fn);
static void dump_description(char const *tag, Description const *desc);
int main(int argc, char **argv)
{
err_setarg0(argv[0]);
for (int i = 1; i < argc; i++)
{
FILE *fp = fopen(argv[i], "rb");
if (fp == 0)
err_sysrem("Failed to open file %s for reading\n", argv[i]);
else
{
Description *desc = read_description(fp, argv[i]);
dump_description("Description", desc);
fclose(fp);
free_description(desc);
}
}
return(0);
}
static void dump_description(char const *tag, Description const *desc)
{
assert(desc != 0);
printf("%s: %p\n", tag, (void *)desc);
printf("Number of sections: %zu\n", desc->n_sections);
if (desc->n_sections != 0)
{
assert(desc->sections != 0);
assert(desc->data != 0);
for (size_t i = 0; i < desc->n_sections; i++)
{
size_t offset = (desc->sections[i].data - desc->data) * sizeof(int32_t);
printf("Section %zu:\n", i);
image_print(stdout, offset, (char *)desc->sections[i].data,
desc->sections[i].length * sizeof(int32_t));
}
}
}
static void free_description(Description *dp)
{
assert(dp != 0);
free(dp->sections);
free(dp->data);
free(dp);
}
static Description *read_description(FILE *fp, char const *fn)
{
fseek(fp, 0L, SEEK_END);
size_t n_bytes = ftell(fp);
fseek(fp, 0L, SEEK_SET);
if (n_bytes % sizeof(int32_t) != 0)
{
err_remark("Length of file (%zu) is not a multiple of %zu bytes\n",
n_bytes, sizeof(int32_t));
return 0;
}
Description *desc = (Description *)calloc(1, sizeof(Description));
if (desc == 0)
err_syserr("Failed to allocate memory\n");
desc->data = (int32_t *)malloc(n_bytes);
if (desc->data == 0)
err_syserr("Failed to allocate memory\n");
size_t n_read = fread(desc->data, 1, n_bytes, fp);
if (n_read != n_bytes)
err_syserr("Short read on file %s\n", fn);
//image_print(stderr, 0, (char *)desc->data, n_bytes);
/* All data in memory — how many sections? */
size_t n_values = n_bytes / sizeof(int32_t);
size_t n_sects = 0;
for (size_t i = 0; i < n_values; i++)
{
if (desc->data[i] == EOS_MARKER)
n_sects++;
}
//err_remark("Found %zu sections\n", n_sects);
desc->sections = (Section *)malloc(n_sects * sizeof(Section));
size_t sec_num = 0;
int32_t p_value = EOS_MARKER;
for (size_t i = 0; i < n_values; i++)
{
if (p_value == EOS_MARKER)
{
//err_remark("Found EOS_MARKER: section %zu, index %zu\n", sec_num, i);
//image_print(stderr, 0, (char *)&desc->data[i], (n_values - i) * sizeof(int32_t));
desc->sections[sec_num].data = &desc->data[i];
//err_remark("Section %zu: data %p\n", sec_num, (void *)desc->sections[sec_num].data);
if (i > 0)
{
assert(sec_num > 0);
desc->sections[sec_num-1].length = &desc->data[i] - desc->sections[sec_num-1].data;
//err_remark("Section %zu: length %zu\n", sec_num-1, desc->sections[sec_num-1].length);
}
sec_num++;
}
p_value = desc->data[i];
}
assert(sec_num == n_sects);
desc->sections[sec_num-1].length = &desc->data[n_values] - desc->sections[sec_num-1].data;
if (p_value != EOS_MARKER)
err_syserr("The file %s did not finish with the section marker!\n", fn);
desc->n_sections = n_sects;
return desc;
}
The header imageprt.h declares 'image_print(), a function in my personal library that formats hex dumps. The headerstderr.hdefines error reporting functions such aserr_remark()anderr_syserr()` (which report a message and continue, and report a message, the system error, and stop, respectively).
The code slurps the entire file into a single chunk of memory, then divvies it up into sections. The Description structure contains the description. The code scans the data twice while reading (and again for printing). If the file is going to be multiple gigabytes, it might be better to build up the sections list in a single pass. You could also consider memory mapping the file.
Hex dump of input data
0x0000: 1F 00 1C 3A 1F 00 25 3A 1F 00 09 3A 1F 00 50 3A ...:..%:...:..P:
0x0010: 1F 00 5A 3A 1F 00 5C 3A 1F 00 5B 3A 1F 00 59 3A ..Z:..\:..[:..Y:
0x0020: 1F 00 5D 3A 03 00 FE 0F 1F 00 01 30 1F 00 06 3A ..]:.......0...:
0x0030: 1F 00 11 3A 1F 00 44 3A 1F 00 4F 3A 1F 00 45 3A ...:..D:..O:..E:
0x0040: 1F 10 56 3A 1F 10 54 3A 1F 00 03 30 1F 00 02 30 ..V:..T:...0...0
0x0050: 03 00 55 3A 03 00 71 3A 1F 00 29 3A 1F 00 27 3A ..U:..q:..):..':
0x0060: 1F 00 2A 3A 1F 00 28 3A 1F 00 26 3A 1F 00 51 3A ..*:..(:..&:..Q:
0x0070: 1F 00 08 3A 1F 00 24 3A 1F 00 21 3A 1F 00 16 3A ...:..$:..!:...:
0x0080: 1F 00 17 3A 1F 00 18 3A 1F 00 19 3A 1F 00 0A 80 ...:...:...:....
0x0090: 1F 00 48 3A 1F 10 58 3A 02 00 4D 3A 40 00 42 3A ..H:..X:..M:#.B:
0x00A0: 40 00 41 3A 1F 00 04 30 1F 10 00 80 03 00 01 80 #.A:...0........
0x00B0: 02 01 FF 0F 40 00 08 30 03 00 71 3A 03 00 55 3A ....#..0..q:..U:
0x00C0: 1F 00 02 30 1F 00 03 30 1F 10 54 3A 1F 10 56 3A ...0...0..T:..V:
0x00D0: 1F 00 06 3A 1F 00 01 30 03 00 FE 0F 02 01 FF 0F ...:...0........
0x00E0: 40 00 08 30 03 00 71 3A 03 00 55 3A 1F 00 02 30 #..0..q:..U:...0
0x00F0: 1F 00 03 30 1F 10 54 3A 1F 10 56 3A 1F 00 11 3A ...0..T:..V:...:
0x0100: 1F 00 06 3A 1F 00 01 30 03 00 FE 0F 02 01 FF 0F ...:...0........
0x0110: 1F 00 5D 3A 03 00 71 3A 03 00 55 3A 1F 00 02 30 ..]:..q:..U:...0
0x0120: 1F 00 03 30 1F 10 54 3A 1F 10 56 3A 1F 00 45 3A ...0..T:..V:..E:
0x0130: 1F 00 4F 3A 1F 00 44 3A 1F 00 11 3A 1F 00 06 3A ..O:..D:...:...:
0x0140: 1F 00 01 30 02 01 FF 0F 40 00 08 30 1F 00 03 30 ...0....#..0...0
0x0150: 1F 00 02 30 1F 00 01 30 02 01 FF 0F ...0...0....
0x015C:
Example output
Description: 0x7fc58bc03a20
Number of sections: 5
Section 0:
0x0000: 1F 00 1C 3A 1F 00 25 3A 1F 00 09 3A 1F 00 50 3A ...:..%:...:..P:
0x0010: 1F 00 5A 3A 1F 00 5C 3A 1F 00 5B 3A 1F 00 59 3A ..Z:..\:..[:..Y:
0x0020: 1F 00 5D 3A 03 00 FE 0F 1F 00 01 30 1F 00 06 3A ..]:.......0...:
0x0030: 1F 00 11 3A 1F 00 44 3A 1F 00 4F 3A 1F 00 45 3A ...:..D:..O:..E:
0x0040: 1F 10 56 3A 1F 10 54 3A 1F 00 03 30 1F 00 02 30 ..V:..T:...0...0
0x0050: 03 00 55 3A 03 00 71 3A 1F 00 29 3A 1F 00 27 3A ..U:..q:..):..':
0x0060: 1F 00 2A 3A 1F 00 28 3A 1F 00 26 3A 1F 00 51 3A ..*:..(:..&:..Q:
0x0070: 1F 00 08 3A 1F 00 24 3A 1F 00 21 3A 1F 00 16 3A ...:..$:..!:...:
0x0080: 1F 00 17 3A 1F 00 18 3A 1F 00 19 3A 1F 00 0A 80 ...:...:...:....
0x0090: 1F 00 48 3A 1F 10 58 3A 02 00 4D 3A 40 00 42 3A ..H:..X:..M:#.B:
0x00A0: 40 00 41 3A 1F 00 04 30 1F 10 00 80 03 00 01 80 #.A:...0........
0x00B0: 02 01 FF 0F ....
Section 1:
0x00B4: 40 00 08 30 03 00 71 3A 03 00 55 3A 1F 00 02 30 #..0..q:..U:...0
0x00C4: 1F 00 03 30 1F 10 54 3A 1F 10 56 3A 1F 00 06 3A ...0..T:..V:...:
0x00D4: 1F 00 01 30 03 00 FE 0F 02 01 FF 0F ...0........
Section 2:
0x00E0: 40 00 08 30 03 00 71 3A 03 00 55 3A 1F 00 02 30 #..0..q:..U:...0
0x00F0: 1F 00 03 30 1F 10 54 3A 1F 10 56 3A 1F 00 11 3A ...0..T:..V:...:
0x0100: 1F 00 06 3A 1F 00 01 30 03 00 FE 0F 02 01 FF 0F ...:...0........
Section 3:
0x0110: 1F 00 5D 3A 03 00 71 3A 03 00 55 3A 1F 00 02 30 ..]:..q:..U:...0
0x0120: 1F 00 03 30 1F 10 54 3A 1F 10 56 3A 1F 00 45 3A ...0..T:..V:..E:
0x0130: 1F 00 4F 3A 1F 00 44 3A 1F 00 11 3A 1F 00 06 3A ..O:..D:...:...:
0x0140: 1F 00 01 30 02 01 FF 0F ...0....
Section 4:
0x0148: 40 00 08 30 1F 00 03 30 1F 00 02 30 1F 00 01 30 #..0...0...0...0
0x0158: 02 01 FF 0F
Yes, the same image_print() function is used in my hex dump program as in this program.
We see these packets being injected in an FTP-DTP channel during a downlink file transfer on Telstra's NEXTG mobile network. We are not sure if these are network level packets, a problem with our 3G modem (HC25 based) or something like our firewall injecting in the stream.
Using a tool we noticed that the PPP framing fails with protocol length errors, so they are mostly likely mobile network packets.
I am hoping someone here can identify the signature of the packets so that I can chase this up with the appropriate vendor.
There is definitely a format to these packets: -
Packet1:
00 00 00 24 c4 b8 7b 1a 00 90 7f 43 0f a1 08 00 45 00 01 10 f4 4e 00 00 40 06 2f 13 cb 7a 9d e9 7b d0 71 52 7a ed 04 06 8c 61 5d a9 01 f7 0c eb 50 10 ff ff 58 b9 00 00
Packet2:
00 00 00 24 c4 b8 7b 1a 00 90 7f 43 0f a1 08 00 45 00 00 ff 6b 50 00 00 40 06 b8 22 cb 7a 9d e9 7b d0 71 52 7a ed 04 06 8c 61 7b 82 01 f7 0c eb 50 10 ff ff a3 79 00 00
Packet3:
00 00 00 24 c4 b8 7b 1a 00 90 7f 43 0f a1 08 00 45 00 02 20 5b 50 00 00 40 06 c7 01 cb 7a 9d e9 7b d0 71 52 7a ed 04 06 8c 61 7c 59 01 f7 0c eb 50 10 ff ff e2 5d 00 00
Packet4:
00 00 00 24 c4 b8 7b 1a 00 90 7f 43 0f a1 08 00 45 00 01 38 d8 52 00 00 40 06 4a e7 cb 7a 9d e9 7b d0 71 52 7a ed 04 06 8c 62 42 f9 01 f7 0c eb 50 10 ff ff 20 91 00 00
Packet5:
00 00 00 24 c4 b8 7b 1a 00 90 7f 43 0f a1 08 00 45 00 00 d0 4d 58 00 00 40 06 d6 49 cb 7a 9d e9 7b d0 71 52 7a ee 04 08 4b fb 0b 8f 03 5d 51 1a 50 10 ff ff e9 88 00 00
I converted your packet trace snippet into a format understood by text2pcap so I could convert them into the pcap format for viewing in Wireshark (a very handy packet capture and analysis tool):
Looks like some sort of IPv4 multicast traffic at a very rough guess. Here's what I got from the first packet (rest came up as malformed):
No. Time Source Destination Protocol Info
1 0.000000 7b:1a:00:90:7f:43 00:00:00_24:c4:b8 0x0fa1 Ethernet II
Frame 1 (31 bytes on wire, 31 bytes captured)
Arrival Time: Dec 1, 2009 00:33:05.000000000
[Time delta from previous captured frame: 0.000000000 seconds]
[Time delta from previous displayed frame: 0.000000000 seconds]
[Time since reference or first frame: 0.000000000 seconds]
Frame Number: 1
Frame Length: 31 bytes
Capture Length: 31 bytes
[Frame is marked: False]
[Protocols in frame: eth:data]
Ethernet II, Src: 7b:1a:00:90:7f:43 (7b:1a:00:90:7f:43), Dst: 00:00:00_24:c4:b8 (00:00:00:24:c4:b8)
Destination: 00:00:00_24:c4:b8 (00:00:00:24:c4:b8)
Address: 00:00:00_24:c4:b8 (00:00:00:24:c4:b8)
.... ...0 .... .... .... .... = IG bit: Individual address (unicast)
.... ..0. .... .... .... .... = LG bit: Globally unique address (factory default)
Source: 7b:1a:00:90:7f:43 (7b:1a:00:90:7f:43)
Address: 7b:1a:00:90:7f:43 (7b:1a:00:90:7f:43)
.... ...1 .... .... .... .... = IG bit: Group address (multicast/broadcast)
.... ..1. .... .... .... .... = LG bit: Locally administered address (this is NOT the factory default)
Type: Unknown (0x0fa1)
Data (17 bytes)
0000 08 00 45 00 01 10 f4 4e 00 00 40 06 2f 13 cb 7a ..E....N..#./..z
0010 9d .
Data: 080045000110F44E000040062F13CB7A9D
These look like ordinary TCP packets but with two extra 00 bytes tagged on at the front. Not sure why that would happen, but they appear to be from 00-90-7f-43-0f-a1 (Watchguard) to 00-24-c4-b8-7b-1a (Cisco).
IP header is 45 00 01 10 f4 4e 00 00 40 06 2f 13 cb 7a 9d e9 7b d0 71 52
TCP header is 7a ed 04 06 8c 61 5d a9 01 f7 0c eb 50 10 ff ff 58 b9 00 00
So you can get the rest of the details from there.
00:24:c4 is a NIC from Cisco and 00:90:7F is a NIC from WatchGuard.
From the IEEE OUI Registry.
How much help that might be ... don't know. Might therefore be an attempted VPN connection.
As already decoded by others:
first 6+6+2 bytes identifying NIC and Ethernet II.
bytes 0x0800 EtherType telling that it is IP. http://en.wikipedia.org/wiki/EtherType
next octet starting with nibble "4" is IPv4
etc.