Related
I am working on a project in C to implement CBC mode on top of a skeleton code for DES with OpenSSL. We are not allowed to use a function that does the CBC mode automatically, in the sense that we must implement it ourselves. I am getting output but I have result files and my output is not matching up completely with the intended results. I also am stuck on figuring out how to pad the file to ensure all the blocks are of equal size, which is probably one of the reasons why I'm not receiving the correct output. Any help would be appreciated. Here's my modification of the skeleton code so far:
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <openssl/des.h>
#include <sys/time.h>
#include <unistd.h>
#define ENC 1
#define DEC 0
DES_key_schedule key;
int append(char*s, size_t size, char c) {
if(strlen(s) + 1 >= size) {
return 1;
}
int len = strlen(s);
s[len] = c;
s[len+1] = '\0';
return 0;
}
int getSize (char * s) {
char * t;
for (t = s; *t != '\0'; t++)
;
return t - s;
}
void strToHex(const_DES_cblock input, unsigned char *output) {
int arSize = 8;
unsigned int byte;
for(int i=0; i<arSize; i++) {
if(sscanf(input, "%2x", &byte) != 1) {
break;
}
output[i] = byte;
input += 2;
}
}
void doBitwiseXor(DES_LONG *xorValue, DES_LONG* data, const_DES_cblock roundOutput) {
DES_LONG temp[2];
memcpy(temp, roundOutput, 8*sizeof(unsigned char));
for(int i=0; i<2; i++) {
xorValue[i] = temp[i] ^ data[i];
}
}
void doCBCenc(DES_LONG *data, const_DES_cblock roundOutput, FILE *outFile) {
DES_LONG in[2];
doBitwiseXor(in, data, roundOutput);
DES_encrypt1(in,&key,ENC);
printf("ENCRYPTED\n");
printvalueOfDES_LONG(in);
printf("%s","\n");
fwrite(in, 8, 1, outFile);
memcpy(roundOutput, in, 2*sizeof(DES_LONG));
}
int main(int argc, char** argv)
{
const_DES_cblock cbc_key = {0x01,0x23,0x45,0x67,0x89,0xab,0xcd,0xef};
const_DES_cblock IV = {0x01,0x23,0x45,0x67,0x89,0xab,0xcd,0xef};
// Initialize the timing function
struct timeval start, end;
gettimeofday(&start, NULL);
int l;
if ((l = DES_set_key_checked(&cbc_key,&key)) != 0)
printf("\nkey error\n");
FILE *inpFile;
FILE *outFile;
inpFile = fopen("test.txt", "r");
outFile = fopen("test_results.txt", "wb");
if(inpFile && outFile) {
unsigned char ch;
// A char array that will hold all 8 ch values.
// each ch value is appended to this.
unsigned char eight_bits[8];
// counter for the loop that ensures that only 8 chars are done at a time.
int count = 0;
while(!feof(inpFile)) {
// read in a character
ch = fgetc(inpFile);
// print the character
printf("%c",ch);
// append the character to eight_bits
append(eight_bits,1,ch);
// increment the count so that we only go to 8.
count++;
const_DES_cblock roundOutput;
// When count gets to 8
if(count == 8) {
// for formatting
printf("%s","\n");
// Encrypt the eight characters and store them back in the char array.
//DES_encrypt1(eight_bits,&key,ENC);
doCBCenc(eight_bits, roundOutput, outFile);
// prints out the encrypted string
int k;
for(k = 0; k < getSize(eight_bits); k++){
printf("%c", eight_bits[k]);
}
// Sets count back to 0 so that we can do another 8 characters.
count = 0;
// so we just do the first 8. When everything works REMOVE THE BREAK.
//break;
}
}
} else {
printf("Error in opening file\n");
}
fclose(inpFile);
fclose(outFile);
// End the timing
gettimeofday(&end, NULL);
// Initialize seconds and micros to hold values for the time output
long seconds = (end.tv_sec - start.tv_sec);
long micros = ((seconds * 1000000) + end.tv_usec) - (start.tv_usec);
// Output the time
printf("The elapsed time is %d seconds and %d microseconds\n", seconds, micros);
}
Your crypto is at least half correct, but you have a lot of actual or potential other errors.
As you identified, raw CBC mode can only encrypt data which is a multiple of the block size, for DES 64 bits or 8 bytes (on most modern computers and all where you could use OpenSSL). In some applications this is okay; for example if the data is (always) an MD5 or SHA-256 or SHA-512 hash, or a GUID, or an IPv6 (binary) address, then it is a block multiple. But most applications want to handle at least any length in bytes, so they need to use some scheme to pad on encrypt and unpad on decrypt the last block (all blocks before the last already have the correct size). Many different schemes have been developed for this, so you need to know which to use. I assume this is a school assignment (since no real customer would set such a stupid and wasteful combination of requirements) and this should either have been specified or clearly left as a choice. One padding scheme very common today (although not for single-DES, because that is broken, unsafe, obsolete, and not common) is the one defined by PKCS5 and generalized by PKCS7 and variously called PKCS5, PKCS7, or PKCS5/7 padding, so I used that as an example.
Other than that:
you try to test feof(inpFile) before doing fgetc(inpFile). This doesn't work in C. It results in your code treating the low 8 bits of EOF (255 aka 0xFF on practically all implementations) as a valid data character added to the characters that were actually in the file. The common idiom is to store the return of getchar/getc/fgetc in a signed int and compare to EOF, but that would have required more changes so I used an alternate.
you don't initialize eight_bits which is a local-scope automatic duration variable, so its contents are undefined and depending on the implementation are often garbage, which means trying to 'append' to it by using strlen() to look for the end won't work right and might even crash. Although on some implementations at least some times it might happen to contain zero bytes, and 'work'. In addition it is possible in C for a byte read from a file (and stored here) to be \0 which will also make this work wrong, although if this file contains text, as its name suggests, it probably doesn't contain any \0 bytes.
once you fill eight_bits you write 'off-the-end' into element [8] which doesn't exist. Technically this is Undefined Behavior and anything at all can happen, traditionally expressed on Usenet as nasal demons. Plus after main finishes the first block it doesn't change anything in eight_bits so all further calls to append find it full and discard the new character.
while you could fix the above points separately, a much simple solution is available: you are already using count to count the number of bytes in the current block, so just use it as the subscript.
roundOutput is also an uninitialized local/auto variable within the loop, which is then used as the previous block for the CBC step, possibly with garbage or wrong value(s). And you don't use the IV at all, as is needed. You should allocate this before the loop (so it retains its value through all iterations) and initialize it to the IV, and then for each block in the loop your doCBCenc can properly XOR it to the new block and then leave the encrypted new block to be used next time.
your code labelled 'prints out the encrypted string' prints plaintext not ciphertext -- which is binary and shouldn't be printed directly anyway -- and is not needed because your file-read loop already echoes each character read. But if you do want to print a (validly null-terminated) string it's easier to just use fputs(s) or [f]printf([f,]"%s",s) or even fwrite(s,1,strlen(s),f).
your doCBCenc has a reference to printvalueofDES_LONG which isn't defined anywhere, and which along with two surrounding printf is clearly not needed.
you should use a cast to convert the first argument to doCBCenc -- this isn't strictly required but is good style and a good compiler (like mine) complains if you don't
finally, when an error occurs you usually print a message but then continue running, which will never work right and may produce symptoms that disguise the problem and make it hard to fix.
The below code fixes the above except that last (which would have been more work for less benefit) plus I removed routines that are now superfluous, and the timing code which is just silly: Unix already has builtin tools to measure and display process time more easily and reliably than writing code. Code I 'removed' is under #if 0 for reference, and code I added under #else or #if 1 except for the cast. The logic for PKCS5/7 padding is under #if MAYBE so it can be either selected or not. Some consider it better style to use sizeof(DES_block) or define a macro instead of the magic 8's, but I didn't bother -- especially since it would have required changes that aren't really necessary.
// SO70209636
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <openssl/des.h>
#include <sys/time.h>
#include <unistd.h>
#define ENC 1
#define DEC 0
DES_key_schedule key;
#if 0
int append(char*s, size_t size, char c) {
if(strlen(s) + 1 >= size) {
return 1;
}
int len = strlen(s);
s[len] = c;
s[len+1] = '\0';
return 0;
}
int getSize (char * s) {
char * t;
for (t = s; *t != '\0'; t++)
;
return t - s;
}
void strToHex(const_DES_cblock input, unsigned char *output) {
int arSize = 8;
unsigned int byte;
for(int i=0; i<arSize; i++) {
if(sscanf(input, "%2x", &byte) != 1) {
break;
}
output[i] = byte;
input += 2;
}
}
#endif
void doBitwiseXor(DES_LONG *xorValue, DES_LONG* data, const_DES_cblock roundOutput) {
DES_LONG temp[2];
memcpy(temp, roundOutput, 8*sizeof(unsigned char));
for(int i=0; i<2; i++) {
xorValue[i] = temp[i] ^ data[i];
}
}
void doCBCenc(DES_LONG *data, const_DES_cblock roundOutput, FILE *outFile) {
DES_LONG in[2];
doBitwiseXor(in, data, roundOutput);
DES_encrypt1(in,&key,ENC);
#if 0
printf("ENCRYPTED\n");
printvalueOfDES_LONG(in);
printf("%s","\n");
#endif
fwrite(in, 8, 1, outFile);
memcpy(roundOutput, in, 2*sizeof(DES_LONG));
}
int main(int argc, char** argv)
{
const_DES_cblock cbc_key = {0x01,0x23,0x45,0x67,0x89,0xab,0xcd,0xef};
const_DES_cblock IV = {0x01,0x23,0x45,0x67,0x89,0xab,0xcd,0xef};
#if 0
// Initialize the timing function
struct timeval start, end;
gettimeofday(&start, NULL);
#endif
int l;
if ((l = DES_set_key_checked(&cbc_key,&key)) != 0)
printf("\nkey error\n");
#if 1
DES_cblock roundOutput; // must be outside the loop
memcpy (roundOutput, IV, 8); // and initialized
#endif
FILE *inpFile;
FILE *outFile;
inpFile = fopen("test.txt", "r");
outFile = fopen("test.encrypt", "wb");
if(inpFile && outFile) {
unsigned char ch;
// A char array that will hold all 8 ch values.
// each ch value is appended to this.
unsigned char eight_bits[8];
// counter for the loop that ensures that only 8 chars are done at a time.
int count = 0;
#if 0
while(!feof(inpFile)) {
// read in a character
ch = fgetc(inpFile);
#else
while( ch = fgetc(inpFile), !feof(inpFile) ){
#endif
// print the character
printf("%c",ch);
#if 0
// append the character to eight_bits
append(eight_bits,1,ch);
// increment the count so that we only go to 8.
count++;
#else
eight_bits[count++] = ch;
#endif
#if 0
const_DES_cblock roundOutput;
#endif
// When count gets to 8
if(count == 8) {
// for formatting
printf("%s","\n");
// Encrypt the eight characters and store them back in the char array.
//DES_encrypt1(eight_bits,&key,ENC);
doCBCenc((DES_LONG*)eight_bits, roundOutput, outFile);
#if 0
// prints out the encrypted string
int k;
for(k = 0; k < getSize(eight_bits); k++){
printf("%c", eight_bits[k]);
}
#endif
// Sets count back to 0 so that we can do another 8 characters.
count = 0;
// so we just do the first 8. When everything works REMOVE THE BREAK.
//break;
}
}
#if MAYBE
memset (eight_bits+count, 8-count, 8-count); // PKCS5/7 padding
doCBCenc((DES_LONG*)eight_bits, roundOutput, outFile);
#endif
} else {
printf("Error in opening file\n");
}
fclose(inpFile);
fclose(outFile);
#if 0
// End the timing
gettimeofday(&end, NULL);
// Initialize seconds and micros to hold values for the time output
long seconds = (end.tv_sec - start.tv_sec);
long micros = ((seconds * 1000000) + end.tv_usec) - (start.tv_usec);
// Output the time
printf("The elapsed time is %d seconds and %d microseconds\n", seconds, micros);
#endif
}
PS: personally I wouldn't put the fwrite in doCBCenc; I would only do the encryption and let the caller do whatever I/O is appropriate which might in some cases not be fwrite. But what you have is not wrong for the requirements you apparently have.
So I have a binary file that I want to read that is structured such that there are 256 segments of the following structure:
First Byte: integer representing the length of the bits of the field following it that you need to read. It doesn't necessarily end at a byte boundary
Variable number of bits: The field you want to read. It doesn't necessarily end at a byte boundary
The file ends with 0's padded out so that it ends at a byte boundary.
I'm struggling to figure out an ideal method that involves reading as few individual bits as possible. I'm thinking of maybe reading the length first, dividing this value by 8 and reading that number of bytes next, and then using remainder of the previous operation (if any) to read the rest of the field bit by bit. I'm not sure if this is an ideal method however. Any suggestions?
Edit: Attached is a link to the files. The readable file is the format I would like to print out the binary file as.
To take an example from the desired output:
length for 9c: 4
code for 9c: 1101
4 would be the first byte read from the binary file, and 1101 would be the variable number of bits
https://ln2.sync.com/dl/e85dc8b40/3f5wbhaq-kxz3ijv8-wuts3t32-442gbsh2
The naive method works excellently (for small files)
The input is actually completely unaligned, just a series of bits, without any padding.
[I'll delete this answer in 1 minute, because I dont want do do someone's homework]
#include <stdio.h>
#include <stdlib.h>
#define the_path "/home/Download/binary_file.dict"
struct bitfile {
FILE *fp;
unsigned char byte;
unsigned char left;
};
struct bitfile * bfopen(char *path)
{
struct bitfile *bp;
bp = malloc(sizeof *bp);
bp->fp = fopen(path, "rb" );
bp->byte = 0;
bp->left = 0;
return bp;
}
int bfclose(struct bitfile * bp)
{
int rc;
rc = fclose(bp->fp);
free(bp);
return rc;
}
int bfgetb(struct bitfile * bp)
{
int ch;
if (!bp->left) {
ch = fgetc(bp->fp);
if (ch < 0) return EOF;
bp->byte = ch;
bp->left = 8;
}
bp->left -= 1;
ch = bp->byte & (1u << bp->left) ? 1 : 0;
// bp->byte >>= 1;
return ch;
}
void bfflush(struct bitfile * bp)
{
bp->left =0;
}
unsigned bp_get_n( struct bitfile *bp, unsigned bitcount)
{
unsigned val=0;
while(bitcount--) {
int ch;
ch = bfgetb(bp);
if (ch < 0) return EOF;
val <<=1;
val |= ch;
}
return val;
}
int main(void)
{
struct bitfile *bp;
int ch;
unsigned iseg, ibit, nbit;
bp = bfopen( the_path);
for (iseg =0; iseg <16*16; iseg++) {
// bfflush(bp);
nbit = bp_get_n(bp, 8);
fprintf(stdout, "Seg%u: %u bits\n", iseg, nbit);
fprintf(stdout, "payload:");
for (ibit=0; ibit < nbit; ibit++) {
ch = bfgetb(bp);
if (ch < 0) break;
fputc( '0'+ ch, stdout);
}
fprintf(stdout, ".\n");
}
bfclose(bp);
return 0;
}
Closed. This question needs details or clarity. It is not currently accepting answers.
Want to improve this question? Add details and clarify the problem by editing this post.
Closed 5 years ago.
Improve this question
I understand the general idea of C and how making a log file would go. Reading/writing to a file and such.
My concern is the following format that is desired:
[![enter image description here][1]][1]
I've gotten a good chunk done now but am concerned with how to append to my log file after the first record. I increment the file's record count (in the top 2 bytes) and write the first record after it. How would I then setup to add the 2nd/3rd/etc records to showup after each other?
//confirm a file exists in the directory
bool fileExists(const char* file)
{
struct stat buf;
return (stat(file, &buf) == 0);
}
int rightBitShift(int val, int space)
{
return ((val >> space) & 0xFF);
}
int leftBitShift(int val, int space)
{
return (val << space);
}
int determineRecordCount(char * logName)
{
unsigned char record[2];
FILE *fp = fopen(logName, "rb");
fread(record, sizeof(record), 1, fp);
//display the record number
int recordNum = (record[0] << 8) | record[1];
recordNum = recordNum +1;
return (recordNum);
}
void createRecord(int argc, char **argv)
{
int recordNum;
int aux = 0;
int dst;
char* logName;
char message[30];
memset(message,' ',30);
//check argument count and validation
if (argc == 7 && strcmp("-a", argv[2]) ==0 && strcmp("-f", argv[3]) ==0 && strcmp("-t", argv[5]) ==0)
{
//aux flag on
aux = 1;
logName = argv[4];
strncpy(message, argv[6],strlen(argv[6]));
}
else if (argc == 6 && strcmp("-f", argv[2]) ==0 && strcmp("-t", argv[4]) ==0)
{
logName = argv[3];
strncpy(message, argv[5],strlen(argv[5]));
}
else
{
printf("Invalid Arguments\n");
exit(0);
}
//check if log exists to get latest recordNum
if (fileExists(logName))
{
recordNum = determineRecordCount(logName);
printf("%i\n",recordNum);
}
else
{
printf("Logfile %s not found\n", logName);
recordNum = 1;
}
//Begin creating record
unsigned char record[40]; /* One record takes up 40 bytes of space */
memset(record, 0, sizeof(record));
//recordCount---------------------------------------------------------------------
record[0] = rightBitShift (recordNum, 8); /* Upper byte of sequence number */
record[1] = rightBitShift (recordNum, 0); /* Lower byte of sequence number */
//get aux/dst flags---------------------------------------------------------------
//get date and time
time_t timeStamp = time(NULL);
struct tm *date = localtime( &timeStamp );
if (date->tm_isdst)
dst = 1;
record[2] |= aux << 7; //set 7th bit
record[2] |= dst << 6; //set 6th
//timeStamp-----------------------------------------------------------------------
record[3] |= rightBitShift(timeStamp, 24);//high byte
record[4] |= rightBitShift(timeStamp, 16);
record[5] |= rightBitShift(timeStamp, 8);
record[6] |= rightBitShift(timeStamp, 0); //low byte
//leave bytes 7-8, set to 0 -----------------------------------------
record[7] = 0;
record[8] = 0;
//store message--------------------------------------------
strncpy(&record[9], message, strlen(message));
//write record to log-----------------------------------------------------------------
FILE *fp = fopen(logName, "w+");
unsigned char recordCount[4];
recordCount[0] = rightBitShift (recordNum, 8); /* Upper byte of sequence number */
recordCount[1] = rightBitShift (recordNum, 0); /* Lower byte of sequence number */
recordCount[2] = 0;
recordCount[3] = 0;
fwrite(recordCount, sizeof(recordCount), 1, fp);
fwrite(record, sizeof(record), 1, fp);
fclose(fp);
printf("Record saved successfully\n");
}
NOTE: I've never had to do this before in C, take it with a grain of salt.
This is a very specific binary formatting where each bit is precisely accounted for. It's using the Least-Significant-Bit numbering scheme (LSB 0) where the bits are numbered from 7 to 0.
Specifying that the "upper byte" comes first means this format is big-endian. The most significant bits come first. This is like how we write our numbers, four thousand, three hundred, and twenty one is 4321. 1234 would be little-endian. For example, the Number Of Records and Sequence are both 16 bit big-endian numbers.
Finally, the checksum is a number calculated from the rest of the record to verify there were no mistakes in transmission. The spec defines how to make the checksum.
Your job is to precisely reproduce this format, probably using the fixed-sized types found in stdint.h or unsigned char. For example, the sequence would be a uint16_t or unsigned char[2].
The function to produce a record might have a signature like this:
unsigned char *make_record( const char *message, bool aux );
The user only has to supply you with the message and the aux flag. The rest you can be figured out by the function. You might decide to let them pass in the timestamp and sequence. Point is, the function needs to be passed just the data, it takes care of the formatting.
This byte-ordering means you can't just write out integers, they might be the wrong size or the wrong byte order. That means any multi-byte integers must be serialized before you can write them to the record. This answer covers ways to do that and I'll be using the ones from this answer because they proved a bit more convenient.
#include <stdio.h>
#include <stdint.h>
#include <time.h>
#include <stdbool.h>
#include <stdlib.h>
#include <string.h>
unsigned char *make_record( const char *message, bool aux ) {
// Allocate and zero memory for the buffer.
// Zeroing means no risk of accidentally sending garbage.
unsigned char *buffer = calloc( 40, sizeof(unsigned char) );
// As we add to the buffer, pos will track the next byte to be written.
unsigned char *pos = buffer;
// I decided not make the user responsible for
// the sequence number. YMMV.
static uint16_t sequence = 1;
pos = serialize_uint16( pos, sequence );
// Get the timestamp and DST.
time_t timestamp = time(NULL);
struct tm *date = localtime( ×tamp );
// 2nd row is all flags and a bunch of 0s. Start with them all off.
uint8_t flags = 0;
if( aux ) {
// Flip the 7th bit on.
flags |= 0x80;
}
if( date->tm_isdst ) {
// Flip the 6th bit on.
flags |= 0x40;
}
// That an 8 bit integer has no endianness, this is to ensure
// pos is consistently incremented.
pos = serialize_uint8(pos, flags);
// I don't know what their timestamp format is.
// This is just a guess. It's probably wrong.
pos = serialize_uint32(pos, (uint32_t)timestamp);
// "Spare" is all zeros.
// The spec says this is 3 bytes, but only gives it bytes
// 7 and 8. I'm going with 2 bytes.
pos = serialize_uint16(pos, 0);
// Copy the message in, 30 bytes.
// strncpy() does not guarantee the message will be null
// terminated. This is probably fine as the field is fixed width.
// More info about the format would be necessary to know for sure.
strncpy( pos, message, 30 );
pos += 30;
// Checksum the first 39 bytes.
// Sorry, I don't know how to do 1's compliment sums.
pos = serialize_uint8( pos, record_checksum( buffer, 39 ) );
// pos has moved around, but buffer remains at the start
return buffer;
}
int main() {
unsigned char *record = make_record("Basset hounds got long ears", true);
fwrite(record, sizeof(unsigned char), 40, stdout);
}
At this point my expertise is exhausted, I've never had to do this before. I'd appreciate folks fixing up the little mistakes in edits and suggesting better ways to do it in the comments, like what to do with the timestamp. And maybe someone else can cover how to do 1's compliment checksums in another answer.
As a byte is composed by 8 bits (from 0 to 7) you can use bitwise operations to modify them as asked in your specifications. Take a look for general information (https://en.wikipedia.org/wiki/Bitwise_operations_in_C). As a preview, you can use >> or << operators to determine which bit to modify, and use logical operators | and & to set it's values.
I need to put into a char* some uint32_t and uint16_t numbers. Then I need to get them back from the buffer.
I have read some questions and I've tried to use sprintf to put them into the char* and sscanf get the original numbers again. However, I'm not able to get them correctly.
Here's an example of my code with only 2 numbers. But I need more than 2, that's why I use realloc. Also, I don't know how to use sprintf and sscanf properly with uint16_t
uint32_t gid = 1100;
uint32_t uid = 1000;
char* buffer = NULL;
uint32_t offset = 0;
buffer = realloc(buffer, sizeof(uint32_t));
sprintf(buffer, "%d", gid);
offset += sizeof(uint32_t);
buffer = realloc(buffer, sizeof(uint32_t) + sizeof(buffer));
sprintf(buffer+sizeof(uint32_t), "%d", uid);
uint32_t valorGID;
uint32_t valorUID;
sscanf(buffer, "%d", &valorGID);
buffer += sizeof(uint32_t);
sscanf(buffer, "%d", &valorUID);
printf("ValorGID %d ValorUID %d \n", valorGID, valorUID);
And what I get is
ValorGID 11001000 ValorUID 1000
What I need to get is
ValorGID 1100 ValorUID 1000
I am new in C, so any help would be appreciated.
buffer = realloc(buffer, sizeof(uint32_t));
sprintf(buffer, "%d", gid);
offset += sizeof(uint32_t);
buffer = realloc(buffer, sizeof(uint32_t) + sizeof(buffer));
sprintf(buffer+sizeof(uint32_t), "%d", uid);
This doesn't really make sense, and will not work as intended except in lucky circumstances.
Let us assume that the usual CHAR_BIT == 8 holds, so sizeof(uint32_t) == 4. Further, let us assume that int is a signed 32-bit integer in two's complement representation without padding bits.
sprintf(buffer, "%d", gid) prints the decimal string representation of the bit-pattern of gid interpreted as an int to buffer. Under the above assumptions, gid is interpreted as a number between -2147483648 and 2147483647 inclusive. Thus the decimal string representation may contain a '-', contains 1 to 10 digits and the 0-terminator, altogether it uses two to twelve bytes. But you have allocated only four bytes, so whenever 999 < gid < 2^32-99 (the signed two's complement interpretation is > 999 or < -99), sprintf writes past the allocated buffer size.
That is undefined behaviour.
It's likely to not crash immediately because allocating four bytes usually gives you a larger chunk of memory effectively (if e.g. malloc always returns 16-byte aligned blocks, the twelve bytes directly behind the allocated four cannot be used by other parts of the programme, but belong to the programme's address space, and writing to them will probably go undetected). But it can easily crash later when the end of the allocated chunk lies on a page boundary.
Also, since you advance the write offset by four bytes for subsequent sprintfs, part of the previous number gets overwritten if the string representation (excluding the 0-termnator) used more than four bytes (while the programme didn't yet crash due to writing to non-allocated memory).
The line
buffer = realloc(buffer, sizeof(uint32_t) + sizeof(buffer));
contains further errors.
buffer = realloc(buffer, new_size); loses the reference to the allocated memory and causes a leak if realloc fails. Use a temporary and check for success
char *temp = realloc(buffer, new_size);
if (temp == NULL) {
/* reallocation failed, recover or cleanup */
free(buffer);
exit(EXIT_FAILURE);
}
/* it worked */
buffer = temp;
/* temp = NULL; or let temp go out of scope */
The new size sizeof(uint32_t) + sizeof(buffer) of the new allocation is always the same, sizeof(uint32_t) + sizeof(char*). That's typically eight or twelve bytes, so it doesn't take many numbers to write outside the allocated area and cause a crash or memory corruption (which may cause a crash much later).
You must keep track of the number of bytes allocated to buffer and use that to calculate the new size. There is no (portable¹) way to determine the size of the allocated memory block from the pointer to its start.
Now the question is whether you want to store the string representations or the bit patterns in the buffer.
Storing the string representations has the problem that the length of the string representation varies with the value. So you need to include separators between the representations of the numbers, or ensure that all representations have the same length by padding (with spaces or leading zeros) if necessary. That would for example work like
#include <stdint.h>
#include <inttypes.h>
#define MAKESTR(x) # x
#define STR(x) MAKESTR(x)
/* A uint32_t can use 10 decimal digits, so let each field be 10 chars wide */
#define FIELD_WIDTH 10
uint32_t gid = 1100;
uint32_t uid = 1000;
size_t buf_size = 0, offset = 0;
char *buffer = NULL, *temp = NULL;
buffer = realloc(buffer, FIELD_WIDTH + 1); /* one for the '\0' */
if (buffer == NULL) {
exit(EXIT_FAILURE);
}
buf_size = FIELD_WIDTH + 1;
sprintf(buffer, "%0" STR(FIELD_WIDTH) PRIu32, gid);
offset += FIELD_WIDTH;
temp = realloc(buffer, buf_size + FIELD_WIDTH);
if (temp == NULL) {
free(buffer);
exit(EXIT_FAILURE);
}
buffer = temp;
temp = NULL;
buf_size += FIELD_WIDTH;
sprintf(buffer + offset, "%0" STR(FIELD_WIDTH) PRIu32, uid);
offset += FIELD_WIDTH;
/* more */
uint32_t valorGID;
uint32_t valorUID;
/* rewind for scanning */
offset = 0;
sscanf(buffer + offset, "%" STR(FIELD_WIDTH) SCNu32, &valorGID);
offset += FIELD_WIDTH;
sscanf(buffer + offset, "%" STR(FIELD_WIDTH) SCNu32, &valorUID);
printf("ValorGID %u ValorUID %u \n", valorGID, valorUID);
with zero-padded fixed-width fields. If you'd rather use separators than a fixed width, the calculation of the required length and the offsets becomes more complicated, but unless the numbers are large, it would use less space.
If you'd rather store the bit-patterns, which would be the most compact way of storing, you'd use something like
size_t buf_size = 0, offset = 0;
unsigned char *buffer = NULL, temp = NULL;
buffer = realloc(buffer, sizeof(uint32_t));
if (buffer == NULL) {
exit(EXIT_FAILURE);
}
buf_size = sizeof(uint32_t);
for(size_t b = 0; b < sizeof(uint32_t); ++b) {
buffer[offset + b] = (gid >> b*8) & 0xFF;
}
offset += sizeof(uint32_t);
temp = realloc(buffer, buf_size + sizeof(uint32_t));
if (temp == NULL) {
free(buffer);
exit(EXIT_FAILURE);
}
buffer = temp;
temp = NULL;
buf_size += sizeof(uint32_t);
for(size_t b = 0; b < sizeof(uint32_t); ++b) {
buffer[offset + b] = (uid >> b*8) & 0xFF;
}
offset += sizeof(uint32_t);
/* And for reading the values */
uint32_t valorGID, valorUID;
/* rewind */
offset = 0;
valorGID = 0;
for(size_t b = 0; b < sizeof(uint32_t); ++b) {
valorGID |= buffer[offset + b] << b*8;
}
offset += sizeof(uint32_t);
valorUID = 0;
for(size_t b = 0; b < sizeof(uint32_t); ++b) {
valorUID |= buffer[offset + b] << b*8;
}
offset += sizeof(uint32_t);
¹ If you know how malloc etc. work in your implementation, it may be possible to find the size from malloc's bookkeeping data.
The format specifier '%d' is for int and thus is wrong for uint32_t. First uint32_t is an unsigned type, so you should at least use '%u', but then it might also have a different width than int or unsigned. There are macros foreseen in the standard: PRIu32 for printf and SCNu32 for scanf. As an example:
sprintf(buffer, "%" PRIu32, gid);
The representation returned by sprintf is a char*. If you are trying to store an array of integers as their string representatins then your fundamental data type is a char**. This is a ragged matrix of char if we are storing only the string data itself, but since the longest string a uint32_t can yield is 10 chars, plus one for the terminating null, it makes sense to preallocate this many bytes to hold each string.
So to store n uint32_t's from array a in array s as strings:
const size_t kMaxIntLen=11;
uint32_t *a,b;
// fill a somehow
...
size_t n,i;
char **s.*d;
if((d=(char*)malloc(n*kMaxIntLen))==NULL)
// error!
if((s=(char**)malloc(n*sizeof(char*)))==NULL)
// error!
for(i=0;i<n;i++)
{
s[i]=d+i; // this is incremented by sizeof(char*) each iteration
snprintf(s[i],kMaxIntLen,"%u",a[i]); // snprintf to be safe
}
Now the ith number is at s[i] so to print it is just printf("%s",s[i]);, and to retrieve it as an integer into b is sscanf(s[i],"%u",&b);.
Subsequent memory management is a bit trickier. Rather than constantly using using realloc() to grow the buffer, it is better to preallocate a chunk of memory and only alter it when exhausted. If realloc() fails it returns NULL, so store a pointer to your main buffer before calling it and that way you won't lose a reference to your data. Reallocate the d buffer first - again allocate enough room for several more strings - then if it succeeds see if d has changed. If so, destroy (free()) the s buffer, malloc() it again and rebuild the indices (you have to do this since if d has changed all your indices are stale). If not, realloc() s and fix up the new indices. I would suggest wrapping this whole thing in a structure and having a set of routines to operate on it, e.g.:
typedef struct StringArray
{
char **strArray;
char *data;
size_t nStrings;
} StringArray;
This is a lot of work. Do you have to use C? This is vastly easier as a C++ STL vector<string> or list<string> with the istringstream classes and the push_back() container method.
uint32_t gid = 1100;
uint32_t uid = 1000;
char* buffer = NULL;
uint32_t offset = 0;
buffer = realloc(buffer, sizeof(uint32_t));
sprintf(buffer, "%d", gid);
offset += sizeof(uint32_t);
buffer = realloc(buffer, sizeof(uint32_t) + sizeof(buffer));
sprintf(buffer+sizeof(uint32_t), "%d", uid);
uint32_t valorGID;
uint32_t valorUID;
sscanf(buffer, "%4d", &valorGID);
buffer += sizeof(uint32_t);
sscanf(buffer, "%d", &valorUID);
printf("ValorGID %d ValorUID %d \n", valorGID, valorUID);
`
I think this may resolve the issue !
I have a char * buffer that is filled by an API function. I need to take the data that is contained with that pointer, cast it to unsigned shorts and translate it into network (htons()) format to send it over UDP. Here is my code (not all though for a few reasons)
The code below will work but that data on the other side is bad (not shorts or network translated)
char * pcZap;
while(1)
{
unsigned short *ps;
unsigned short short_buffer[4096];
write_reg(to start xfer);
return_val = get_packet(fd, &pcZap, &uLen, &uLob);
check_size_of_uLen_and_uLob(); //make sure we got a packet
// here I need to chage pcZap to (unsigned short *) and translate to network
sendto(sockFd,pcZap,size,0,(struct sockaddr *)Server_addr,
sizeof(struct sockaddr));
return_val = free_packet(fd, pcZap);
thread_check_for_exit();
}
Any help would be appreciated. Thank you.
Assuming you have 4080 bytes in your buffer that are composed of 16-bit samples, that would mean you have 2040 total 16-bit samples in the 4080 bytes of your buffer (16-bytes are reserved for the header). Therefore you can do the following:
#define MAXBUFSIZE 4096
#define MAXSHORTSIZE 2040
unsigned char pcZap[MAXBUFSIZE];
unsigned ushort[MAXSHORTSIZE];
//get the value of the returned packed length in uLen, and the header in uLob
unsigned short* ptr = (unsigned short*)(pcZap + uLob);
for (int i=0; i < ((uLen - uLob) / 2); i++)
{
ushort[i] = htons(*ptr++);
}
Now your ushort array will be composed of network-byte-order unsigned short values converted from the original values in the pcZap array. Then, when you call sendto(), make sure to use the values from ushort, not the values from pcZap.
If your array of chars is null terminated then you can simply do:
for (int i=0; i<strlen(CHAR_ARRAY); i++)
short_buffer[i] = (unsigned short) CHAR_ARRAY[i];
If the array isn't null terminated then you'll need to figure out how long it is exactly and then replace strlen(CHAR_ARRAY) with that value.
If all you need to do is convert a chunk of bytes, representing short ints in host endian to network endian, you do this:
size_t i;
size_t len = uLen - 16 - uLob;
size_t offset = uLob + 16;
if(len % 2 != 0) {
..error not a multiple of 16 bit shorts...
}
//now, if you're on a little endian host (assuming the shorts in
//pcZap is laid out as the host endian...), just swap around the bytes
//to convert the shorts to network endian.
for(i = 0; i < len; i+=2) {
//swap(&pcZap[offset + i],&pcZap[offset + i + 1]);
char tmp = pcZap[offset + i];
pcZap[offset + i] = pcZap[offset + i + 1];
pcZap[offset + i + 1] = tmp;
}
//if you're on a big endian host, forget the above loop, the data
//is already in big/network endian layout.
//and just send the data.
if(sendto(sockFd,pcZap + offset,len,0,(struct sockaddr *)&Server_addr,
sizeof Server_addr) == -1) {
perror("sendto");
}
Note that your code had sizeof(struct sockaddr) in the sendto() call, which is wrong, you want it to be the actual size of Server_addr.