i coded a small program to show you the casting problem
#include <stdlib.h>
struct flags {
u_char flag1;
u_char flag2;
u_short flag3;
u_char flag4;
u_short flag5;
u_char flag7[5];
};
int main(){
char buffer[] = "\x01\x02\x04\x03\x05\x07\x06\xff\xff\xff\xff\xff";
struct flags *flag;
flag = (struct flags *) buffer;
return 0;
}
my problem is when i cast the flag 5 wrongly takes the "\x06\xff" bytes ignoring the "\x07" and the flag 7 wrongly takes the next 4 "\xff" bytes plus a nul which is the next byte.I also run gdb
(gdb) p/x flag->flag5
$1 = 0xff06
(gdb) p/x flag->flag7
$2 = {0xff, 0xff, 0xff, 0xff, 0x0}
(gdb) x/15xb flag
0xbffff53f: 0x01 0x02 0x04 0x03 0x05 0x07 0x06 0xff
0xbffff547: 0xff 0xff 0xff 0xff 0x00 0x00 0x8a
why this is happening and how i can handle it correctly?
thanks
It seems like structure member alignment issues. Unless you know how your compiler packs structure members, you should not make assumptions about the positions of those members in memory.
The reason that the 0x07 is apparently lost, is because the compiler is probably aligning the flag5 member on a 16-bit boundary, skipping the odd memory location that holds the 0x07 value. That value is lost in the padding. Also, what you are doing is overflowing the buffer, a big no-no. In other words:
struct flags {
u_char flag1; // 0x01
u_char flag2; // 0x02
u_short flag3; // 0x04 0x03
u_char flag4; // 0x05
// 0x07 is in the padding
u_short flag5; // 0x06 0xff
u_char flag7[5]; // 0xff 0xff 0xff 0xff ... oops, buffer overrun, because your
// buffer was less than the sizeof(flags)
};
You can often control the packing of structure members with most compilers, but the mechanism is compiler specific.
The compiler is free to put some unused padding between members of the structure to (for instance) arrange the alignment to it's conveneince. Your compiler may provide a #pragma packed or a command line argument to insure tight structure packing.
How structures are stored is implementation defined, and thus, you can't rely on a specific memory layout for serialization like that.
To serialize your structure to a byte array, write a function which serializes each field in a set order.
You might need to pack the struct:
struct flags __attribute__ ((__packed__)) {
u_char flag1;
u_char flag2;
u_short flag3;
u_char flag4;
u_short flag5;
u_char flag7[5];
};
Note: This is GCC -- I don't know how portable it is.
This has to do with padding. The compiler is adding garbage memory into your struct in order to get it to align with your memory correctly for efficiency.
See the following examples:
http://msdn.microsoft.com/en-us/library/71kf49f1(v=vs.80).aspx
http://en.wikipedia.org/wiki/Data_structure_alignment#Typical_alignment_of_C_structs_on_x86
Related
I'm trying to force a buffer overflow to change the value of a variable. The idea is to overflow var_a to change var_b in the following code, which runs Contiki-NG operating system in an ARM Cortex-M4:
#include "contiki.h"
#include "board.h"
#include <dev/leds.h>
#include <stdio.h>
#include <string.h>
PROCESS(main_process, "main_process");
AUTOSTART_PROCESSES(&main_process);
PROCESS_THREAD(main_process, ev, data) {
uint8_t data_buffer[16] = {
0x00, 0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07,
0x08, 0x09, 0x0a, 0x0b, 0x0c, 0x0d, 0x0e, 0x0f
};
uint32_t var_b;
uint8_t var_a[4];
PROCESS_BEGIN();
var_b = 0;
printf("var_b = %08lx\n", var_b);
memcpy(var_a, data_buffer, 8);
printf("var_b = %08lx\n", var_b);
if (var_b) {
leds_arch_set(LEDS_RED);
}
PROCESS_END();
}
The problem is that overflow is not affecting var_b, but data_buffer. I used a debugger to check the addresses of the local variables in the process, and got the following:
Looking at this explains why the overflow is affecting to data_buffer, since it is located right after var_a. But what I didn't expect was that the address of var_b is <outofscope>, which suggests that this variable might be allocated in other memory region different from the current stack frame.
What's the reason for this happening when allocating var_b? Is there any way I can make it local?
A couple suggestions:
You could try disable compiler optimization with -O0 cflags.
Add volatile
But what I didn't expect was that the address of var_b is <outofscope>, which suggests that this variable might be allocated in other memory region different from the current stack frame.
What's the reason for this happening when allocating var_b? Is there any way I can make it local?
When it comes to debugging, there's really no other way to read variables except to compile with -ggdb -O0. ie: turn ON debug flags with -g or -ggdb or similar, and turn OFF optimization with -O0. You must do both. See my answer here: What's the difference between a compiler's -O0 option and -Og option?
Memory pools and pointer arithmetic
Whatever it is you're trying to do is very confusing. Some questions for you to think about:
Why are you trying to "overflow a buffer" to change variable var_b? Why don't you just change var_b directly?
memcpy(var_a, data_buffer, 8); copies the first 8 bytes from data_buffer into var_a. Since var_a is only 4 bytes, the first 4 bytes go into it successfully, and then the latter 4 bytes have undefined behavior by writing out of the bounds of the variable. Why don't you just write the first 4 bytes of data_buffer into var_a and the next 4 bytes into var_b?
If you think writing past the bounds of var_a should write into var_b, why did you put var_b first instead of var_a first? You did:
uint32_t var_b;
uint8_t var_a[4];
The compiler can do whatever it wants when determining where to place variables in memory, so there's no guarantee here, but it seems more logical that you would have at least put var_a first, like this:
uint8_t var_a[4];
uint32_t var_b;
Why didn't you?
Alright, that about covers it. Let's go over some things.
1. If you want to have guaranteed relative locations of variables, force it via a memory pool!
No matter what order you write your variables, the compiler is not compelled to abide by that order nor location, unless you use memory pools or otherwise manually specify the address for a given variable, such as a hardware register.
// How to force a memory layout of 4 bytes of `uint8_t var_a[4]` followed by 4
// bytes of `uint32_t var_b` (8 bytes total)
#define MEM_POOL_SIZE 8 // 8 bytes
// Step 1. Create an 8 byte memory pool. Choose **one** of the following
// options:
// Option 1: 8 byte memory pool **on the stack** (statically allocated)
uint8_t mem_pool[MEM_POOL_SIZE];
// Option 2: 8 byte memory pool **on the heap** (dynamically allocated)
uint8_t* mem_pool = malloc(MEM_POOL_SIZE);
if (mem_pool == NULL)
{
// do error handling here: out of memory
}
// Option 3: 8 byte memory pool **neither on the stack nor the heap**
// (`static` makes it take global RAM not allocated for either)
static uint8_t mem_pool[MEM_POOL_SIZE];
// Step 2: point `var_a` and `var_b` into the memory pool. Voila! They now
// magically take this pool of memory, with `var_b` **guaranteed** to be
// right after `var_a`.
uint8_t* var_a = mem_pool; // 4 byte array of uint8_t
// Note: the `+ 4` is pointer math: since `mem_pool` is a ptr to `uint8_t`,
// this moves forward `4*sizeof(uint8_t)` bytes.
uint32_t* var_b = mem_pool + 4;
// Step 3: use the variables! Here are some examples:
// write some bytes into the `var_a` array
var_a[0] = 0x01;
var_a[1] = 0x02;
var_a[2] = 0x03;
var_a[3] = 0x04;
// write a value into `var_b`
*var_b = 12345;
// write the first 4 bytes of data_buffer into `var_a` and the next 4 into
// `var_b`
memcpy(var_a, data_buffer, MEM_POOL_SIZE);
2. Just write the first 4 bytes of data_buffer into var_a and the next 4 bytes into var_b directly
See my questions at the top of my answer. I don't understand what you are doing, or why. If you want to write the first 4 bytes of data_buffer into var_a and the next 4 bytes into var_b, just do that! No need for any undefined-behavior "overflow" tricks, nor memory pools!
uint8_t var_a[4];
uint32_t var_b;
uint8_t data_buffer[16] = {
0x00, 0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07,
0x08, 0x09, 0x0a, 0x0b, 0x0c, 0x0d, 0x0e, 0x0f,
};
memcpy(var_a, data_buffer, 4);
// remember, the `+ 4` is pointer arithmetic, and moves forward 4 x the size
// of the thing being pointed to, which is `uint8_t` (1 byte) in this case
memcpy(&var_b, data_buffer + 4, 4);
// Or (better), same thing as just above, but written more-clearly:
memcpy(var_a, data_buffer, sizeof(var_a);
memcpy(&var_b, data_buffer + sizeof(var_a), sizeof(var_b));
First, I'm a student still. So I am not very experienced.
I'm working with a piece of bluetooth hardware and I am using its protocol to send it commands. The protocol requires packets to be sent with LSB first for each packet field.
I was getting error packets back to me indicating my CRC values were wrong so I did some investigating. I found the problem, but I became confused in the process.
Here is Some GDB output and other information elucidating my confusion.
I'm sending a packet that should look like this:
|Start Flag| Packet Num | Command | Payload | CRC | End Flag|
0xfc 0x1 0x0 0x8 0x0 0x5 0x59 0x42 0xfd
Here is some GDB output:
print /x reqId_ep
$1 = {start_flag = 0xfc, data = {packet_num = 0x1, command = {0x0, 0x8}, payload = {
0x0, 0x5}}, crc = 0x5942, end_flag = 0xfd}
reqId_ep is the variable name of the packet I'm sending. It looks all good there, but I am receiving the CRC error codes from it so something must be wrong.
Here I examine 9 bytes in hex starting from the address of my packet to send:
x/9bx 0x7fffffffdee0
0xfc 0x01 0x00 0x08 0x00 0x05 0x42 0x59 0xfd
And here the problem becomes apparent. The CRC is not LSB first. (0x42 0x59)
To fix my problem I removed the htons() that I set my CRC value equal with.
And here is the same output above without htons():
p/x reqId_ep
$1 = {start_flag = 0xfc, data = {packet_num = 0x1, command = {0x0, 0x8}, payload = {
0x0, 0x5}}, crc = 0x4259, end_flag = 0xfd}
Here the CRC value is not LSB.
But then:
x/9bx 0x7fffffffdee0
0xfc 0x01 0x00 0x08 0x00 0x05 0x59 0x42 0xfd
Here the CRC value is LSB first.
So apparently the storing of C is LSB first? Can someone please cast a light of knowledge upon me for this situation? Thank you kindly.
This has to do with Endianness in computing:
http://en.wikipedia.org/wiki/Endianness#Endianness_and_operating_systems_on_architectures
For example, the value 4660 (base-ten) is 0x1234 in hex. On a Big Endian system, it would be stored in memory as 1234 while on a Little Endian system it would be stored as 3412
If you want to avoid this sort of issue in the future, it might just be easiest to create a large array or struct of unsigned char, and store individual values in it.
eg:
|Start Flag| Packet Num | Command | Payload | CRC | End Flag|
0xfc 0x1 0x0 0x8 0x0 0x5 0x59 0x42 0xfd
typedef struct packet {
unsigned char startFlag;
unsigned char packetNum;
unsigned char commandMSB;
unsigned char commandLSB;
unsigned char payloadMSB;
unsigned char payloadLSB;
unsigned char crcMSB;
unsigned char crcLSB;
unsigned char endFlag;
} packet_t;
You could then create a function that you compile differently based on the type of system you are building for using preprocessor macros.
eg:
/* Uncomment the line below if you are using a little endian system;
/* otherwise, leave it commented
*/
//#define LITTLE_ENDIAN_SYSTEM
// Function protocol
void writeCommand(int cmd);
//Function definition
void writeCommand(int cmd, packet_t* pkt)
{
if(!pkt)
{
printf("Error, invalid pointer!");
return;
}
#if LITTLE_ENDIAN_SYSTEM
pkt->commandMSB = (cmd && 0xFF00) >> 8;
pkt->commandLSB = (cmd && 0x00FF);
# else // Big Endian system
pkt->commandMSB = (cmd && 0x00FF);
pkt->commandLSB = (cmd && 0xFF00) >> 8;
#endif
// Done
}
int main void()
{
packet_t myPacket = {0}; //Initialize so it is zeroed out
writeCommand(0x1234,&myPacket);
return 0;
}
One final note: avoid sending structs as a stream of data, send it's individual elements one-at-a-time instead! ie: don't assume that the struct is stored internally in this case like a giant array of unsigned characters. There are things that the compiler and system put in place like packing and allignment, and the struct could actually be larger than 9 x sizeof(unsigned char).
Good luck!
This is architecture dependent based on which processor you're targeting. There are what is known as "Big Endian" systems, which store the most significant byte of a word first, and "Little Endian" systems that store the least significant byte first. It looks like you're looking at a Little Endian system there.
i trying to parse a packet. till the ip header everything is fine(i'm able to retrieve all the values correctly). but for the udp header( checked if the protocol is 17) , the values are coming out to be wrong( all the 4 fields).
I'm trying to do this:
struct udp_header{
uint16_t sport;
uint16_t dport;
uint16_t len;
uint16_t chksum;
};
struct udp_header* udp= (struct udp_header*)(packet + 14 + ip_hdr->ip_hl*4);
Packet is the pointer pointing to the beginning of the packet. 14 is for ethernet header.The header length ip when checked is giving out the correct value. But after performing this operation i'm getting all the fields wrongly. when tried with uint8_t as data type( i know its wrong! ) the destintion port somehow is coming out correct.
You have run into endianness. IP packets have all fields in network byte order (aka "big-endian"), and your host system probably runs little-endian. Look into ntohs() and friends for one approach.
The proper approach is to not copy the structure as-is from the network data, but instead extract each field manually and byte-swap it if necessary. This also works around any issues with padding and alignment, there's no guarantee that your struct is mapped into your computer's memory in exactly the same way as the packet is serialized.
So you would do e.g.:
udp_header.sport = ntohs(*(unsigned short*) (packet + 14 + 4 * ip_hdr->ip_hl));
This is also a bit iffy, since it assumes the resulting address can validly be cast into a pointer to unsigned short. On x86 that will work, but it's not epic.
Even better, in my opinion, is to drop the use of pointers and instead write a function called e.g. unsigned short read_u16(void *packet, size_t offset) that extracts the value byte-by-byte and returns it. Then you'd just do:
udp_header.sport = read_u16(packet, 14 + 4 * ip_hdr->ip_hl);
I always use this struct for IP header:
struct sniff_ip {
u_char ip_vhl; /* version << 4 | header length >> 2 */
u_char ip_tos; /* type of service */
u_short ip_len; /* total length */
u_short ip_id; /* identification */
u_short ip_off; /* fragment offset field */
#define IP_RF 0x8000 /* reserved fragment flag */
#define IP_DF 0x4000 /* dont fragment flag */
#define IP_MF 0x2000 /* more fragments flag */
#define IP_OFFMASK 0x1fff /* mask for fragmenting bits */
u_char ip_ttl; /* time to live */
u_char ip_p; /* protocol */
u_short ip_sum; /* checksum */
struct in_addr ip_src,ip_dst; /* source and dest address */
};
#define IP_HL(ip) (((ip)->ip_vhl) & 0x0f)
#define IP_V(ip) (((ip)->ip_vhl) >> 4)
And to get the UDP struct pointer:
udp = (struct sniff_udp*)(packet + SIZE_ETHERNET + (IP_HL(ip)*4));
As another answer remarked, you have to deal with endianness of your data.
The other thing you need to deal with is byte alignment. For speed, when you define a structure in C like this:
struct udp_header{
uint16_t sport;
uint16_t dport;
uint16_t len;
uint16_t chksum;
};
The C compiler may leave padding bytes between these fields so that member accesses can be done with faster single-instruction memory access assembly instructions. You can check if your c compiler is doing this by printf("struct size is: %u\n", sizeof(struct udp_header));
Assuming you are using GCC, you must disable padding bytes by adding #pragma pack(1) before the structure definition. To re-enable padding for speed you should then use #pragma pack().
I am populating an array with data to be sent on the I2C, my code below shows what I am doing. To me this looks horrible. Is there a better/neater way that anyone can recommend?
Thanks
#define DS1337_SECONDS_REG 0x00
// Default values used for initialisation
#define DS1337_DEFAULT_SECONDS 0x00
#define DS1337_DEFAULT_MINUTES 0x00
#define DS1337_DEFAULT_HOURS_HR_MODE 0x40
#define DS1337_DEFAULT_DAY 0x00 /* Sun */
#define DS1337_DEFAULT_DATE 0x01
#define DS1337_DEFAULT_MONTH 0x01
#define DS1337_DEFAULT_YEAR 0x0C /* 2012 */
#define DS1337_DEFAULT_ALM1_SECS 0x00
#define DS1337_DEFAULT_ALM1_MINS 0x00
#define DS1337_DEFAULT_ALM1_HRS 0x00
#define DS1337_DEFAULT_ALM1_DAY_DATE 0x00
#define DS1337_DEFAULT_ALM2_MINS 0x00
#define DS1337_DEFAULT_ALM2_HRS 0x00
#define DS1337_DEFAULT_ALM2_DAY_DATE 0x00
extern i2c_err_t i2c_send(const i2c_ch_t channel, const uint8_t data[], const uint32_t length, const i2c_stop_t stop);
rtc_err_t ds1337_init(void)
{
uint8_t data_to_send[17] = { DS1337_SECONDS_REG, /* Address of first register */
DS1337_DEFAULT_SECONDS, DS1337_DEFAULT_MINUTES, DS1337_DEFAULT_HOURS_HR_MODE,
DS1337_DEFAULT_DAY, DS1337_DEFAULT_DATE, DS1337_DEFAULT_MONTH, DS1337_DEFAULT_YEAR,
DS1337_DEFAULT_ALM1_SECS, DS1337_DEFAULT_ALM1_MINS, DS1337_DEFAULT_ALM1_HRS, DS1337_DEFAULT_ALM1_DAY_DATE,
DS1337_DEFAULT_ALM2_MINS, DS1337_DEFAULT_ALM2_HRS, DS1337_DEFAULT_ALM2_DAY_DATE,
DS1337_CTRL_REG_INIT_VAL, /* Turn off the squarewave output pin */
DS1337_CLEAR_STATUS_VAL /* Clear the status registers */
};
if (i2c_set_address(DS1337_CHANNEL, DS1337_SPI_ADDRESS) != I2C_NO_ERROR)
return RTC_I2C_ADDRESS_ERROR;
if (i2c_send(DS1337_CHANNEL, data_to_send, sizeof(data_to_send), STOP_CONDITION) != I2C_NO_ERROR)
return RTC_I2C_SEND_ERROR;
//enable_interrupts(GLOBAL);
return RTC_NO_ERROR;
}
Use enums rather than #defines, and I would say "12" rather than "0x0C" for 2012.
Consider just hard-coding "0" in place of the apparently-unused *ALM* values.
#include a header file to get the declaration of i2c_send(), rather than explicitly declaring it.
Omit (i.e. make implicit) the [17] array size--just say [] and let the actual data you pack determine the size.
This is a pretty standard approach. Enums (instead of defines) won't make your source code much shorter. Hardcoding the constants in the code will make it unreadable. Having multiple calls to i2c_send() and no array will be about the same, though probably slightly bigger and less readable. Defining a number of functions to configure every little thing is a sure way to make the source code bigger. I'd probably just keep the code the way it is.
See the definition of TCP header in /netinet/tcp.h:
struct tcphdr
{
u_int16_t th_sport; /* source port */
u_int16_t th_dport; /* destination port */
tcp_seq th_seq; /* sequence number */
tcp_seq th_ack; /* acknowledgement number */
# if __BYTE_ORDER == __LITTLE_ENDIAN
u_int8_t th_x2:4; /* (unused) */
u_int8_t th_off:4; /* data offset */
# endif
# if __BYTE_ORDER == __BIG_ENDIAN
u_int8_t th_off:4; /* data offset */
u_int8_t th_x2:4; /* (unused) */
# endif
u_int8_t th_flags;
# define TH_FIN 0x01
# define TH_SYN 0x02
# define TH_RST 0x04
# define TH_PUSH 0x08
# define TH_ACK 0x10
# define TH_URG 0x20
u_int16_t th_win; /* window */
u_int16_t th_sum; /* checksum */
u_int16_t th_urp; /* urgent pointer */
};
Why does the 8-bit field have a different order in endianness? I thought only 16-bit and 32-bit fields mattered with byte order, and you could convert between endians with ntohs and ntohl, respectively. What would the function be for handling 8-bit things? If there is none, it seems that a TCP using this header on a little endian machine would not work with a TCP on a big endian machine.
There are two kind of order. One is byte order, one is bitfield order.
There is no standard order about the bitfield order in C language. It depends on the compiler. Typically, the order of bitfields are reversed between big and little endian.
This is compiler-dependent and non-portable. How bit fields are ordered is implementation dependent, it would be far better here to use an 8-bit field and shift/mask to obtain the subfields.
Its possible that in this machine the endianess also refers to the bit order as well as the byte order. This wikipedia article mentions that this is sometimes the case.
My understanding is that bit ordering and endianness are generally two different things. Structures with bit-fields are generally not portable across compilers/architectures. Sometimes ifdefs can be used to support different bit orderings. In this case the endianness is really irrelevant and it should be an ifdef about the bit ordering. The assumption that some endianesses have a certain bit-order may be true in some cases.
My reading of the comment is that the two one-byte fields together are construed as a two-byte value (or were -- seems like one byte is unused anyway). Rather than declare one two-byte value, they declare two one-byte values but reverse the order of declaration depending on endian-ness.
It may help to know that this is code is only run if "# ifdef __FAVOR_BSD". It is from /usr/include/netinet/tcp.h
# ifdef __FAVOR_BSD
typedef u_int32_t tcp_seq;
/*
* TCP header.
* Per RFC 793, September, 1981.
*/
struct tcphdr