Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 6 years ago.
Improve this question
Currently, I'm writing a program to calculate the Integral of an equation by using the Trapezoidal rule and a combination of these to archive higher precision. Right now as you can see below, I have hard-coded the function. Is it possible to read in a mathematical equation and evaluate it? I know I could read in the input character list and then evaluate the function (like if char[i] == '+' do ....) but is there an easier way?
Thank you in advance!
void Integral_TN (double* TN_ptr,double a,double b,int n,int my_rank,int p){
int i;
double a_loc;
double b_loc;
double hN;
*TN_ptr = 0;
hN = (b-a)/n;
a_loc = a + my_rank*n/p*hN;
b_loc = a + (my_rank+1)*n/p*hN;
*TN_ptr += (function(a_loc)+function(b_loc))/2; /*Evaluate f at the borders*/
for(i = 1; i < n/p; i++){
*TN_ptr += function(a_loc + i*hN); /*Evaluate f at the inner nodes*/
}
*TN_ptr = *TN_ptr*hN;
}
double function(double x){
double y;
y = 1/(1+x*x);
return y;
}
skrtbhtngr already answered the stated question, but I'd like to address the underlying problem.
Apply the Unix philosophy. Use one tool to generate the data, and another to consume it -- here, to compute the integral using the trapezoid rule.
The easiest format you can use is the same supported by Gnuplot:
Empty lines are ignored
Lines beginning with # are ignored, and can be used for comments
Each line defines one sample
Essentially, you could describe a sine curve, very roughly, using
#x sin(x)
0.000 0.000000000
0.100 0.099833417
0.200 0.198669331
0.300 0.295520207
0.400 0.389418342
0.500 0.479425539
0.600 0.564642473
0.700 0.644217687
0.800 0.717356091
0.900 0.783326910
1.000 0.841470985
1.100 0.891207360
1.200 0.932039086
1.300 0.963558185
1.400 0.985449730
1.500 0.997494987
1.600 0.999573603
1.700 0.991664810
1.800 0.973847631
1.900 0.946300088
2.000 0.909297427
2.100 0.863209367
2.200 0.808496404
2.300 0.745705212
2.400 0.675463181
2.500 0.598472144
2.600 0.515501372
2.700 0.427379880
2.800 0.334988150
2.900 0.239249329
3.000 0.141120008
3.100 0.041580662
3.200 -0.058374143
3.300 -0.157745694
3.400 -0.255541102
3.500 -0.350783228
3.600 -0.442520443
3.700 -0.529836141
3.800 -0.611857891
3.900 -0.687766159
4.000 -0.756802495
4.100 -0.818277111
4.200 -0.871575772
4.300 -0.916165937
4.400 -0.951602074
4.500 -0.977530118
4.600 -0.993691004
4.700 -0.999923258
4.800 -0.996164609
4.900 -0.982452613
5.000 -0.958924275
5.100 -0.925814682
5.200 -0.883454656
5.300 -0.832267442
5.400 -0.772764488
5.500 -0.705540326
5.600 -0.631266638
5.700 -0.550685543
5.800 -0.464602179
5.900 -0.373876665
6.000 -0.279415498
6.100 -0.182162504
6.200 -0.083089403
You could use eg. awk to generate that, like I did:
awk 'BEGIN { printf "#x sin(x)\n" ; for (x=0.0; x<6.3; x+=0.1) printf "%.3f %11.9f\n", x, sin(x) }'
If you save that to a file (appending > data.txt to the above command), you can plot it in Gnuplot using
plot "data.txt" using 1:2 notitle with lines
Such data is easy to read in a C program. Because I only use POSIX.1 systems (Linux, BSDs, macOS), and POSIX.1 provides the very useful getline() function -- it lets you read in lines of any length, dynamically allocating a large enough buffer --, this particular implementation also requires POSIX.1 support. In other words, it works basically everywhere except in Windows.
#define _POSIX_C_SOURCE 200809L
#include <stdlib.h>
#include <string.h>
#include <stdio.h>
#include <errno.h>
/* Read x y(x) values, one pair per line, from a stream.
The arrays are dynamically allocated, and pointers stored
to *xptr and *yptr. The size of the arrays is stored at *nptr.
They are initially cleared to NULL/zero.
The function returns 0 if success, an errno error code otherwise:
EINVAL: Invalid function parameters
EIO: Read error
ENOMEM: Out of memory
EBADMSG: Malformed line
*/
int read_xy(double **xptr, double **yptr, size_t *nptr, FILE *in)
{
/* Line input buffer variables. */
char *line = NULL;
size_t size = 0;
ssize_t len;
/* Data array variables. */
double *x = NULL;
double *y = NULL;
size_t n = 0; /* Entries in x[] and y[] */
size_t nmax = 0; /* Entries allocated */
/* Temporary variables. */
double xval, yval, *newx, *newy;
/* We clear the output parameters to NULL or zero,
in case the caller is careless and does not check
the return value. Clearing them ensures they do
not contain garbage in such a case. */
if (xptr)
*xptr = NULL;
if (yptr)
*yptr = NULL;
if (nptr)
*nptr = 0;
/* We need in and nptr, and at least one of xptr and yptr. */
if (!in || !nptr || (!xptr && !yptr))
return errno = EINVAL;
/* If an error has already occurred in 'in',
we do not even try to read from it. */
if (ferror(in))
return EIO;
while (1) {
/* Read next input line. */
len = getline(&line, &size, in);
/* End of input or error? */
if (len < 0)
break;
/* Skip empty and comment lines. */
if (len == 0 ||
line[0] == '\n' || (line[0] == '\r' && line[1] == '\n') ||
line[0] == '#')
continue;
/* Parse the line. */
if (sscanf(line, " %lf %lf", &xval, &yval) != 2)
break;
/* Need to grow the dynamically allocated arrays? */
if (n >= nmax) {
/* Allocation policy.
We allocate room for at least 16 doubles,
then double the size up to 1048576 (=2^20),
then adjust to the next full multiple of 1048576.
This is not 'the best', but it is robust,
and not too wasteful.
*/
if (n < 16)
nmax = 16;
else
if (n < 1048576)
nmax = n * 2;
else
nmax = (n | 1048575) + 1048576;
/* Note: realloc(NULL, size) is equivalent to malloc(size).
If the realloc() call fails, it returns NULL,
but the original array is still valid.
Also note that free(NULL) is safe, and does nothing.
*/
newx = realloc(x, nmax * sizeof x[0]);
newy = realloc(y, nmax * sizeof y[0]);
if (newx)
x = newx;
if (newy)
y = newy;
if (!newx || !newy) {
/* One or both of the allocations failed. */
free(line);
free(x);
free(y);
return ENOMEM;
}
}
/* Save the parsed values to the arrays. */
x[n] = xval;
y[n] = yval;
n++;
}
/* We no longer need the line buffer. */
free(line);
/* Did a read error occur? */
if (ferror(in)) {
free(x);
free(y);
return EIO;
}
/* Was there no data to read? */
if (n < 1) {
free(x);
free(y);
return 0;
}
/* Reallocate the arrays to their exact sizes
(actually, allow for one extra double at the end,
because it is often useful to copy the initial
ones there if the data is considered cyclic).
*/
nmax = n + 1; /* One extra just because it is so often useful. */
newx = realloc(x, nmax * sizeof x[0]);
newy = realloc(y, nmax * sizeof y[0]);
if (newx)
x = newx;
if (newy)
y = newy;
if (!newx || !newy) {
free(x);
free(y);
return ENOMEM;
}
/* Save the array pointers. */
if (xptr)
*xptr = x;
else
free(x);
if (yptr)
*yptr = y;
else
free(y);
/* Save the number of samples read. */
*nptr = n;
/* If feof(in) is true, then we read everything
up to end of input. Otherwise, we stopped at
a line we could not parse.
*/
if (!feof(in))
return EBADMSG;
return 0;
}
That function, or something like it, should be in the course materials for every numerical computation course. They are just so darn useful. This particular one has no inherent limits to the size of data it can read, other than possible memory allocation limits set by the system administrator for each process. I know for a fact that it happily reads billions of lines of data, successfully, if you just have enough RAM.
Using the function is very simple. Here is an example main() that just reads such data from standard input -- remember that you can make it read from a file by appending < file to the command when you run it --, and prints the data out.
int main(void)
{
double *x, *y;
size_t i, n;
int result;
result = read_xy(&x, &y, &n, stdin);
switch (result) {
case 0: /* No errors */
break;
case EBADMSG:
if (n > 1)
fprintf(stderr, "Invalid line after %zu data samples.\n", n);
else
fprintf(stderr, "Cannot parse first input line.\n");
return EXIT_FAILURE;
case ENOMEM:
fprintf(stderr, "Out of memory.\n");
return EXIT_FAILURE;
case EIO:
fprintf(stderr, "Read error.\n");
return EXIT_FAILURE;
case EINVAL:
fprintf(stderr, "Invalid parameters to the read_xy() function!\n");
return EXIT_FAILURE;
default:
fprintf(stderr, "%s.\n", strerror(result));
return EXIT_FAILURE;
}
printf("Read %zu samples:\n", n);
for (i = 0; i < n; i++)
printf("%.9f %.9f\n", x[i], y[i]);
return EXIT_SUCCESS;
}
Note that most of it is the switch (result) { .. } error-reporting code. This example is very careful to tell you, if an error occurs; the reason is, you, as the user, need to know when the program knows they might spew garbage, and will in real life prefer for the programs to abort than spew that garbage silently -- perhaps making you believe it is valid.
While it is possible to modify the above code to work even on Windows (you replace the getline() with fgets(), for example, and hope that the buffer size you use for it will suffice; and also may have to change some of the errno error codes). However, there is a reason there are no Windows machines in the Top 500 supercomputer list: POSIXy systems (Unix and Linux) are just better suited for scientific computing. So, if you do intend to work on some scientific computing, you might as well just set up a Linux or BSD virtual machine, and do your development there.
There is no easier way for what you wish to achieve. If you want to apply a specific formula to some values, you have to define a function for it and input the values.
If you wish to type in the whole expression (with values and operators) as input and get the desired result as output, you would have to go beyond basic programming. You would need to create a parser.
For instance, if you provide 3+2*4 as input, you might expect 11 as output without reading in separate values 3, 2 and 4. This can be made possible by implementing a custom parser in a parser-generator like YACC. Basically, you would create and define new rules on how the input should be interpreted.
Related
I am working on a project in C to implement CBC mode on top of a skeleton code for DES with OpenSSL. We are not allowed to use a function that does the CBC mode automatically, in the sense that we must implement it ourselves. I am getting output but I have result files and my output is not matching up completely with the intended results. I also am stuck on figuring out how to pad the file to ensure all the blocks are of equal size, which is probably one of the reasons why I'm not receiving the correct output. Any help would be appreciated. Here's my modification of the skeleton code so far:
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <openssl/des.h>
#include <sys/time.h>
#include <unistd.h>
#define ENC 1
#define DEC 0
DES_key_schedule key;
int append(char*s, size_t size, char c) {
if(strlen(s) + 1 >= size) {
return 1;
}
int len = strlen(s);
s[len] = c;
s[len+1] = '\0';
return 0;
}
int getSize (char * s) {
char * t;
for (t = s; *t != '\0'; t++)
;
return t - s;
}
void strToHex(const_DES_cblock input, unsigned char *output) {
int arSize = 8;
unsigned int byte;
for(int i=0; i<arSize; i++) {
if(sscanf(input, "%2x", &byte) != 1) {
break;
}
output[i] = byte;
input += 2;
}
}
void doBitwiseXor(DES_LONG *xorValue, DES_LONG* data, const_DES_cblock roundOutput) {
DES_LONG temp[2];
memcpy(temp, roundOutput, 8*sizeof(unsigned char));
for(int i=0; i<2; i++) {
xorValue[i] = temp[i] ^ data[i];
}
}
void doCBCenc(DES_LONG *data, const_DES_cblock roundOutput, FILE *outFile) {
DES_LONG in[2];
doBitwiseXor(in, data, roundOutput);
DES_encrypt1(in,&key,ENC);
printf("ENCRYPTED\n");
printvalueOfDES_LONG(in);
printf("%s","\n");
fwrite(in, 8, 1, outFile);
memcpy(roundOutput, in, 2*sizeof(DES_LONG));
}
int main(int argc, char** argv)
{
const_DES_cblock cbc_key = {0x01,0x23,0x45,0x67,0x89,0xab,0xcd,0xef};
const_DES_cblock IV = {0x01,0x23,0x45,0x67,0x89,0xab,0xcd,0xef};
// Initialize the timing function
struct timeval start, end;
gettimeofday(&start, NULL);
int l;
if ((l = DES_set_key_checked(&cbc_key,&key)) != 0)
printf("\nkey error\n");
FILE *inpFile;
FILE *outFile;
inpFile = fopen("test.txt", "r");
outFile = fopen("test_results.txt", "wb");
if(inpFile && outFile) {
unsigned char ch;
// A char array that will hold all 8 ch values.
// each ch value is appended to this.
unsigned char eight_bits[8];
// counter for the loop that ensures that only 8 chars are done at a time.
int count = 0;
while(!feof(inpFile)) {
// read in a character
ch = fgetc(inpFile);
// print the character
printf("%c",ch);
// append the character to eight_bits
append(eight_bits,1,ch);
// increment the count so that we only go to 8.
count++;
const_DES_cblock roundOutput;
// When count gets to 8
if(count == 8) {
// for formatting
printf("%s","\n");
// Encrypt the eight characters and store them back in the char array.
//DES_encrypt1(eight_bits,&key,ENC);
doCBCenc(eight_bits, roundOutput, outFile);
// prints out the encrypted string
int k;
for(k = 0; k < getSize(eight_bits); k++){
printf("%c", eight_bits[k]);
}
// Sets count back to 0 so that we can do another 8 characters.
count = 0;
// so we just do the first 8. When everything works REMOVE THE BREAK.
//break;
}
}
} else {
printf("Error in opening file\n");
}
fclose(inpFile);
fclose(outFile);
// End the timing
gettimeofday(&end, NULL);
// Initialize seconds and micros to hold values for the time output
long seconds = (end.tv_sec - start.tv_sec);
long micros = ((seconds * 1000000) + end.tv_usec) - (start.tv_usec);
// Output the time
printf("The elapsed time is %d seconds and %d microseconds\n", seconds, micros);
}
Your crypto is at least half correct, but you have a lot of actual or potential other errors.
As you identified, raw CBC mode can only encrypt data which is a multiple of the block size, for DES 64 bits or 8 bytes (on most modern computers and all where you could use OpenSSL). In some applications this is okay; for example if the data is (always) an MD5 or SHA-256 or SHA-512 hash, or a GUID, or an IPv6 (binary) address, then it is a block multiple. But most applications want to handle at least any length in bytes, so they need to use some scheme to pad on encrypt and unpad on decrypt the last block (all blocks before the last already have the correct size). Many different schemes have been developed for this, so you need to know which to use. I assume this is a school assignment (since no real customer would set such a stupid and wasteful combination of requirements) and this should either have been specified or clearly left as a choice. One padding scheme very common today (although not for single-DES, because that is broken, unsafe, obsolete, and not common) is the one defined by PKCS5 and generalized by PKCS7 and variously called PKCS5, PKCS7, or PKCS5/7 padding, so I used that as an example.
Other than that:
you try to test feof(inpFile) before doing fgetc(inpFile). This doesn't work in C. It results in your code treating the low 8 bits of EOF (255 aka 0xFF on practically all implementations) as a valid data character added to the characters that were actually in the file. The common idiom is to store the return of getchar/getc/fgetc in a signed int and compare to EOF, but that would have required more changes so I used an alternate.
you don't initialize eight_bits which is a local-scope automatic duration variable, so its contents are undefined and depending on the implementation are often garbage, which means trying to 'append' to it by using strlen() to look for the end won't work right and might even crash. Although on some implementations at least some times it might happen to contain zero bytes, and 'work'. In addition it is possible in C for a byte read from a file (and stored here) to be \0 which will also make this work wrong, although if this file contains text, as its name suggests, it probably doesn't contain any \0 bytes.
once you fill eight_bits you write 'off-the-end' into element [8] which doesn't exist. Technically this is Undefined Behavior and anything at all can happen, traditionally expressed on Usenet as nasal demons. Plus after main finishes the first block it doesn't change anything in eight_bits so all further calls to append find it full and discard the new character.
while you could fix the above points separately, a much simple solution is available: you are already using count to count the number of bytes in the current block, so just use it as the subscript.
roundOutput is also an uninitialized local/auto variable within the loop, which is then used as the previous block for the CBC step, possibly with garbage or wrong value(s). And you don't use the IV at all, as is needed. You should allocate this before the loop (so it retains its value through all iterations) and initialize it to the IV, and then for each block in the loop your doCBCenc can properly XOR it to the new block and then leave the encrypted new block to be used next time.
your code labelled 'prints out the encrypted string' prints plaintext not ciphertext -- which is binary and shouldn't be printed directly anyway -- and is not needed because your file-read loop already echoes each character read. But if you do want to print a (validly null-terminated) string it's easier to just use fputs(s) or [f]printf([f,]"%s",s) or even fwrite(s,1,strlen(s),f).
your doCBCenc has a reference to printvalueofDES_LONG which isn't defined anywhere, and which along with two surrounding printf is clearly not needed.
you should use a cast to convert the first argument to doCBCenc -- this isn't strictly required but is good style and a good compiler (like mine) complains if you don't
finally, when an error occurs you usually print a message but then continue running, which will never work right and may produce symptoms that disguise the problem and make it hard to fix.
The below code fixes the above except that last (which would have been more work for less benefit) plus I removed routines that are now superfluous, and the timing code which is just silly: Unix already has builtin tools to measure and display process time more easily and reliably than writing code. Code I 'removed' is under #if 0 for reference, and code I added under #else or #if 1 except for the cast. The logic for PKCS5/7 padding is under #if MAYBE so it can be either selected or not. Some consider it better style to use sizeof(DES_block) or define a macro instead of the magic 8's, but I didn't bother -- especially since it would have required changes that aren't really necessary.
// SO70209636
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <openssl/des.h>
#include <sys/time.h>
#include <unistd.h>
#define ENC 1
#define DEC 0
DES_key_schedule key;
#if 0
int append(char*s, size_t size, char c) {
if(strlen(s) + 1 >= size) {
return 1;
}
int len = strlen(s);
s[len] = c;
s[len+1] = '\0';
return 0;
}
int getSize (char * s) {
char * t;
for (t = s; *t != '\0'; t++)
;
return t - s;
}
void strToHex(const_DES_cblock input, unsigned char *output) {
int arSize = 8;
unsigned int byte;
for(int i=0; i<arSize; i++) {
if(sscanf(input, "%2x", &byte) != 1) {
break;
}
output[i] = byte;
input += 2;
}
}
#endif
void doBitwiseXor(DES_LONG *xorValue, DES_LONG* data, const_DES_cblock roundOutput) {
DES_LONG temp[2];
memcpy(temp, roundOutput, 8*sizeof(unsigned char));
for(int i=0; i<2; i++) {
xorValue[i] = temp[i] ^ data[i];
}
}
void doCBCenc(DES_LONG *data, const_DES_cblock roundOutput, FILE *outFile) {
DES_LONG in[2];
doBitwiseXor(in, data, roundOutput);
DES_encrypt1(in,&key,ENC);
#if 0
printf("ENCRYPTED\n");
printvalueOfDES_LONG(in);
printf("%s","\n");
#endif
fwrite(in, 8, 1, outFile);
memcpy(roundOutput, in, 2*sizeof(DES_LONG));
}
int main(int argc, char** argv)
{
const_DES_cblock cbc_key = {0x01,0x23,0x45,0x67,0x89,0xab,0xcd,0xef};
const_DES_cblock IV = {0x01,0x23,0x45,0x67,0x89,0xab,0xcd,0xef};
#if 0
// Initialize the timing function
struct timeval start, end;
gettimeofday(&start, NULL);
#endif
int l;
if ((l = DES_set_key_checked(&cbc_key,&key)) != 0)
printf("\nkey error\n");
#if 1
DES_cblock roundOutput; // must be outside the loop
memcpy (roundOutput, IV, 8); // and initialized
#endif
FILE *inpFile;
FILE *outFile;
inpFile = fopen("test.txt", "r");
outFile = fopen("test.encrypt", "wb");
if(inpFile && outFile) {
unsigned char ch;
// A char array that will hold all 8 ch values.
// each ch value is appended to this.
unsigned char eight_bits[8];
// counter for the loop that ensures that only 8 chars are done at a time.
int count = 0;
#if 0
while(!feof(inpFile)) {
// read in a character
ch = fgetc(inpFile);
#else
while( ch = fgetc(inpFile), !feof(inpFile) ){
#endif
// print the character
printf("%c",ch);
#if 0
// append the character to eight_bits
append(eight_bits,1,ch);
// increment the count so that we only go to 8.
count++;
#else
eight_bits[count++] = ch;
#endif
#if 0
const_DES_cblock roundOutput;
#endif
// When count gets to 8
if(count == 8) {
// for formatting
printf("%s","\n");
// Encrypt the eight characters and store them back in the char array.
//DES_encrypt1(eight_bits,&key,ENC);
doCBCenc((DES_LONG*)eight_bits, roundOutput, outFile);
#if 0
// prints out the encrypted string
int k;
for(k = 0; k < getSize(eight_bits); k++){
printf("%c", eight_bits[k]);
}
#endif
// Sets count back to 0 so that we can do another 8 characters.
count = 0;
// so we just do the first 8. When everything works REMOVE THE BREAK.
//break;
}
}
#if MAYBE
memset (eight_bits+count, 8-count, 8-count); // PKCS5/7 padding
doCBCenc((DES_LONG*)eight_bits, roundOutput, outFile);
#endif
} else {
printf("Error in opening file\n");
}
fclose(inpFile);
fclose(outFile);
#if 0
// End the timing
gettimeofday(&end, NULL);
// Initialize seconds and micros to hold values for the time output
long seconds = (end.tv_sec - start.tv_sec);
long micros = ((seconds * 1000000) + end.tv_usec) - (start.tv_usec);
// Output the time
printf("The elapsed time is %d seconds and %d microseconds\n", seconds, micros);
#endif
}
PS: personally I wouldn't put the fwrite in doCBCenc; I would only do the encryption and let the caller do whatever I/O is appropriate which might in some cases not be fwrite. But what you have is not wrong for the requirements you apparently have.
I have a string creating function in C which accepts an array of structs as it's argument and outputs a string based on a predefined format (like a list of list in python).
Here's the function
typedef struct
{
PacketInfo_t PacketInfo;
char Gnss60[1900];
//and other stuff...
} Track_json_t;
typedef struct
{
double latitude;
double longitude;
} GPSPoint_t;
typedef struct
{
UInt16 GPS_StatusCode;
UInt32 fixtime;
GPSPoint_t point;
double altitude;
unsigned char GPS_Satilite_Num;
} GPS_periodic_t;
unsigned short SendTrack()
{
Track_json_t i_sTrack_S;
memset(&i_sTrack_S, 0x00, sizeof(Track_json_t));
getEvent_Track(&i_sTrack_S);
//Many other stuff added to the i_sTrack_S struct...
//Make a JSON format out of it
BuildTrackPacket_json(&i_sTrack_S, XPORT_MODE_GPRS);
}
Track_json_t *getEvent_Track(Track_json_t *trk)
{
GPS_periodic_t l_gps_60Sec[60];
memset(&l_gps_60Sec, 0x00,
sizeof(GPS_periodic_t) * GPS_PERIODIC_ARRAY_SIZE);
getLastMinGPSdata(l_gps_60Sec, o_gps_base);
get_gps60secString(l_gps_60Sec, trk->Gnss60);
return trk;
}
void get_gps60secString(GPS_periodic_t input[60], char *output)
{
int i = 0;
memcpy(output, "[", 1); ///< Copy the first char as [
char temp[31];
for (i = 0; i < 59; i++) { //Run for n-1 elements
memset(temp, 0, sizeof(temp));
snprintf(temp, sizeof(temp), "[%0.8f,%0.8f],",
input[i].point.latitude, input[i].point.longitude);
strncat(output, temp, sizeof(temp));
}
memset(temp, 0, sizeof(temp)); //assign last element
snprintf(temp, sizeof(temp), "[%0.8f,%0.8f]]",
input[i].point.latitude, input[i].point.longitude);
strncat(output, temp, sizeof(temp));
}
So the output of the function must be a string of format
[[12.12345678,12.12345678],[12.12345678,12.12345678],...]
But at times I get a string which looks like
[[12.12345678,12.12345678],[55.01[12.12345678,12.12345678],...]
[[21.28211567,84.13454083],[21.28211533,21.22[21.28211517,84.13454000],..]
Previously, I had a buffer overflow at the function get_gps60secString, I fixed that by using snprintf and strncat.
Note: This is an embedded application and this error occur once or twice a day (out of 1440 packets)
Question
1. Could this be caused by an interrupt during the snprintf/strncat process?
2. Could this be caused by a memory leak, overwriting the stack or some other segmentation issue caused else where?
Basically I would like to understand what might be causing a corrupt string.
Having a hard time finding the cause and fixing this bug.
EDIT:
I used chux's function. Below is the Minimal, Complete, and Verifiable Example
/*
* Test code for SO question https://stackoverflow.com/questions/5216413
* A Minimal, Complete, and Verifiable Example
*/
#include <stdio.h>
#include <stdlib.h>
#include <memory.h>
#include <stdbool.h>
#include <signal.h>
#include <unistd.h>
typedef unsigned short UInt16;
typedef unsigned long UInt32;
#define GPS_PERIODIC_ARRAY_SIZE 60
#define GPS_STRING_SIZE 1900
/* ---------------------- Data Structs --------------------------*/
typedef struct
{
char Gnss60[GPS_STRING_SIZE];
} Track_json_t;
typedef struct
{
double latitude;
double longitude;
} GPSPoint_t;
typedef struct
{
UInt16 GPS_StatusCode;
UInt32 fixtime;
GPSPoint_t point;
double altitude;
unsigned char GPS_Satilite_Num;
} GPS_periodic_t;
/* ----------------------- Global --------------------------------*/
FILE *fptr; //Global file pointer
int res = 0;
int g_last = 0;
GPS_periodic_t l_gps_60Sec[GPS_PERIODIC_ARRAY_SIZE];
/* ----------------------- Function defs --------------------------*/
/* At signal interrupt this function is called.
* Flush and close the file. And safly exit the program */
void userSignalInterrupt()
{
fflush(fptr);
fclose(fptr);
res = 1;
exit(0);
}
/* #brief From the array of GPS structs we create a string of the format
* [[lat,long],[lat,long],..]
* #param input The input array of GPS structs
* #param output The output string which will contain lat, long
* #param sz Size left in the output buffer
* #return 0 Successfully completed operation
* 1 Failed / Error
*/
int get_gps60secString(GPS_periodic_t input[GPS_PERIODIC_ARRAY_SIZE],
char *output, size_t sz)
{
int cnt = snprintf(output, sz, "[");
if (cnt < 0 || cnt >= sz)
return 1;
output += cnt;
sz -= cnt;
int i = 0;
for (i = 0; i < GPS_PERIODIC_ARRAY_SIZE; i++) {
cnt = snprintf(output, sz, "[%0.8f,%0.8f]%s",
input[i].point.latitude, input[i].point.longitude,
i + 1 == GPS_PERIODIC_ARRAY_SIZE ? "" : ",");
if (cnt < 0 || cnt >= sz)
return 1;
output += cnt;
sz -= cnt;
}
cnt = snprintf(output, sz, "]");
if (cnt < 0 || cnt >= sz)
return 1;
return 0; // no error
}
/* #brief Create a GPS struct with data for testing. It will populate the
* point field of GPS_periodic_t. Lat starts from 0.0 and increases by 1*10^(-8)
* and Long will dstart at 99.99999999 and dec by 1*10^(-8)
*
* #param o_gps_60sec Output array of GPS structs
*/
void getLastMinGPSdata(GPS_periodic_t *o_gps_60sec)
{
//Fill in GPS related data here
int i = 0;
double latitude = o_gps_60sec[0].point.latitude;
double longitude = o_gps_60sec[0].point.longitude;
for (i = 0; i < 60; i++)
{
o_gps_60sec[i].point.latitude = latitude + (0.00000001 * (float)g_last +
0.00000001 * (float)i);
o_gps_60sec[i].point.longitude = longitude - (0.00000001 * (float)g_last +
0.00000001 * (float)i);
}
g_last = 60;
}
/* #brief Get the GPS data and convert it into a string
* #param trk Track structure with GPS string
*/
int getEvent_Track(Track_json_t *trk)
{
getLastMinGPSdata(l_gps_60Sec);
get_gps60secString(l_gps_60Sec, trk->Gnss60, GPS_STRING_SIZE);
return 0;
}
int main()
{
fptr = fopen("gpsAno.txt", "a");
if (fptr == NULL) {
printf("Error!!\n");
exit(1);
}
//Quit at signal interrupt
signal(SIGINT, userSignalInterrupt);
Track_json_t trk;
memset(&l_gps_60Sec, 0x00, sizeof(GPS_periodic_t) * GPS_PERIODIC_ARRAY_SIZE);
//Init Points to be zero and 99.99999999
int i = 0;
for (i = 0; i < 60; i++) {
l_gps_60Sec[i].point.latitude = 00.00000000;
l_gps_60Sec[i].point.longitude = 99.99999999;
}
do {
memset(&trk, 0, sizeof(Track_json_t));
getEvent_Track(&trk);
//Write to file
fprintf(fptr, "%s", trk.Gnss60);
fflush(fptr);
sleep(1);
} while (res == 0);
//close and exit
fclose(fptr);
return 0;
}
Note: Error was not recreated in the above code.
Because this doesn't have the strcat pitfalls.
I tested this function in the embedded application.
Through this I was able to find that the snprintf returns an error and the string created ended up to be:
[17.42401750,78.46098717],[17.42402083,53.62
It ended there (because of the return 1).
Does this mean that the data which was passed to snprints corrupted? It's a float value. How can it get corrupted?
Solution
The error have not been seen since I changed the sprintf function with one that doesn't directly deal with 64 bits of data.
Here's the function modp_dtoa2
/** \brief convert a floating point number to char buffer with a
* variable-precision format, and no trailing zeros
*
* This is similar to "%.[0-9]f" in the printf style, except it will
* NOT include trailing zeros after the decimal point. This type
* of format oddly does not exists with printf.
*
* If the input value is greater than 1<<31, then the output format
* will be switched exponential format.
*
* \param[in] value
* \param[out] buf The allocated output buffer. Should be 32 chars or more.
* \param[in] precision Number of digits to the right of the decimal point.
* Can only be 0-9.
*/
void modp_dtoa2(double value, char* str, int prec)
{
/* if input is larger than thres_max, revert to exponential */
const double thres_max = (double)(0x7FFFFFFF);
int count;
double diff = 0.0;
char* wstr = str;
int neg= 0;
int whole;
double tmp;
uint32_t frac;
/* Hacky test for NaN
* under -fast-math this won't work, but then you also won't
* have correct nan values anyways. The alternative is
* to link with libmath (bad) or hack IEEE double bits (bad)
*/
if (! (value == value)) {
str[0] = 'n'; str[1] = 'a'; str[2] = 'n'; str[3] = '\0';
return;
}
if (prec < 0) {
prec = 0;
} else if (prec > 9) {
/* precision of >= 10 can lead to overflow errors */
prec = 9;
}
/* we'll work in positive values and deal with the
negative sign issue later */
if (value < 0) {
neg = 1;
value = -value;
}
whole = (int) value;
tmp = (value - whole) * pow10[prec];
frac = (uint32_t)(tmp);
diff = tmp - frac;
if (diff > 0.5) {
++frac;
/* handle rollover, e.g. case 0.99 with prec 1 is 1.0 */
if (frac >= pow10[prec]) {
frac = 0;
++whole;
}
} else if (diff == 0.5 && ((frac == 0) || (frac & 1))) {
/* if halfway, round up if odd, OR
if last digit is 0. That last part is strange */
++frac;
}
/* for very large numbers switch back to native sprintf for exponentials.
anyone want to write code to replace this? */
/*
normal printf behavior is to print EVERY whole number digit
which can be 100s of characters overflowing your buffers == bad
*/
if (value > thres_max) {
sprintf(str, "%e", neg ? -value : value);
return;
}
if (prec == 0) {
diff = value - whole;
if (diff > 0.5) {
/* greater than 0.5, round up, e.g. 1.6 -> 2 */
++whole;
} else if (diff == 0.5 && (whole & 1)) {
/* exactly 0.5 and ODD, then round up */
/* 1.5 -> 2, but 2.5 -> 2 */
++whole;
}
//vvvvvvvvvvvvvvvvvvv Diff from modp_dto2
} else if (frac) {
count = prec;
// now do fractional part, as an unsigned number
// we know it is not 0 but we can have leading zeros, these
// should be removed
while (!(frac % 10)) {
--count;
frac /= 10;
}
//^^^^^^^^^^^^^^^^^^^ Diff from modp_dto2
// now do fractional part, as an unsigned number
do {
--count;
*wstr++ = (char)(48 + (frac % 10));
} while (frac /= 10);
// add extra 0s
while (count-- > 0) *wstr++ = '0';
// add decimal
*wstr++ = '.';
}
// do whole part
// Take care of sign
// Conversion. Number is reversed.
do *wstr++ = (char)(48 + (whole % 10)); while (whole /= 10);
if (neg) {
*wstr++ = '-';
}
*wstr='\0';
strreverse(str, wstr-1);
}
Here's (part of) my unabashedly opinionated guide on safe string handling in C. Normally, I would promote dynamic memory allocation instead of fixed-length strings, but in this case I'm assuming that in the embedded environment that might be problematic. (Although assumptions like that should always be checked.)
So, first things first:
Any function which creates a string in a buffer must be told explicitly how long the buffer is. This is non-negotiable.
As should be obvious, it's impossible for a function filling a buffer to check for buffer overflow unless it knows where the buffer ends. "Hope that the buffer is long enough" is not a viable strategy. "Document the needed buffer length" would be fine if everyone carefully read the documentation (they don't) and if the required length never changes (it will). The only thing that's left is an extra argument, which should be of type size_t (because that's the type of buffer lengths in the C library functions which require lengths).
Forget that strncpy and strncat exist. Also forget about strcat. They are not your friends.
strncpy is designed for a specific use case: ensuring that an entire fixed-length buffer is initialised. It is not designed for normal strings, and since it doesn't guarantee that the output is NUL-terminated, it doesn't produce a string.
If you're going to NUL-terminate yourself anyway, you might as well use memmove, or memcpy if you know that the source and destination don't overlap, which should almost always be the case. Since you'll want the memmove to stop at the end of the string for short strings (which strncpy does not do), measure the string length first with strnlen: strnlen takes a maximum length, which is precisely what you want in the case that you are going move a maximum number of characters.
Sample code:
/* Safely copy src to dst where dst has capacity dstlen. */
if (dstlen) {
/* Adjust to_move will have maximum value dstlen - 1 */
size_t to_move = strnlen(src, dstlen - 1);
/* copy the characters */
memmove(dst, src, to_move);
/* NUL-terminate the string */
dst[to_move] = 0;
}
strncat has a slightly more sensible semantic, but it's practically never useful because in order to use it, you already have to know how many bytes you could copy. In order to know that, in practice, you need to know how much space is left in your output buffer, and to know that you need to know where in the output buffer the copy will start. [Note 1]. But if you already know where the copy will start, what's the point of searching through the buffer from the beginning to find the copy point? And if you do let strncat do the search, how sure are you that your previously computed start point is correct?
In the above code snippet, we already computed the length of the copy. We can extend that to do an append without rescanning:
/* Safely copy src1 and then src2 to dst where dst has capacity dstlen. */
/* Assumes that src1 and src2 are not contained in dst. */
if (dstlen) {
/* Adjust to_move will have maximum value dstlen - 1 */
size_t to_move = strnlen(src1, dstlen - 1);
/* Copy the characters from src1 */
memcpy(dst, src1, to_move);
/* Adjust the output pointer and length */
dst += to_move;
dstlen -= to_move;
/* Now safely copy src2 to just after src1. */
to_move = strnlen(src2, dstlen - 1);
memcpy(dst, src2, to_move);
/* NUL-terminate the string */
dst[to_move] = 0;
}
It might be that we want the original values of dst and dstlen after creating the string, and it might also be that we want to know how many bytes we inserted into dst in all. In that case, we would probably want to make copies of those variables before doing the copies, and save the cumulative sum of moves.
The above assumes that we're starting with an empty output buffer, but perhaps that isn't the case. Since we still need to know where the copy will start in order to know how many characters we can put at the end, we can still use memcpy; we just need to scan the output buffer first to find the copy point. (Only do this if there is no alternative. Doing it in a loop instead of recording the next copy point is Shlemiel the Painter's algorithm.)
/* Safely append src to dst where dst has capacity dstlen and starts
* with a string of unknown length.
*/
if (dstlen) {
/* The following code will "work" even if the existing string
* is not correctly NUL-terminated; the code will not copy anything
* from src, but it will put a NUL terminator at the end of the
* output buffer.
*/
/* Figure out where the existing string ends. */
size_t prefixlen = strnlen(dst, dstlen - 1);
/* Update dst and dstlen */
dst += prefixlen;
dstlen -= prefixlen;
/* Proceed with the append, as above. */
size_t to_move = strnlen(src, dstlen - 1);
memmove(dst, src, to_move);
dst[to_move] = 0;
}
Embrace snprintf. It really is your friend. But always check its return value.
Using memmove, as above, is slightly awkward. It requires you to manually check that the buffer's length is not zero (otherwise subtracting one would be disastrous since the length is unsigned), and it requires you to manually NUL-terminate the output buffer, which is easy to forget and the source of many bugs. It is very efficient, but sometimes it's worth sacrificing a little efficiency so that your code is easier to write and easier to read and verify.
And that leads us directly to snprintf. For example, you can replace:
if (dstlen) {
size_t to_move = strnlen(src, dstlen - 1);
memcpy(dst, src, to_move);
dst[to_move] = 0;
}
with the much simpler
int copylen = snprintf(dst, dstlen, "%s", src);
That does everything: checks that dstlen is not 0; only copies the characters from src which can fit in dst, and correctly NUL-terminates dst (unless dstlen was 0). And the cost is minimal; it takes very little time to parse the format string "%s" and most implementations are pretty well optimised for this case. [Note 2]
But snprintf is not a panacea. There are still a couple of really important warnings.
First, the documentation for snprintf makes clear that it is not permitted for any input argument to overlap the output range. (So it replaces memcpy but not memmove.) Remember that overlap includes NUL-terminators, so the following code which attempts to double the string in str instead leads to Undefined Behaviour:
char str[BUFLEN];
/* Put something into str */
get_some_data(str, BUFLEN);
/* DO NOT DO THIS: input overlaps output */
int result = snprintf(str, BUFLEN, "%s%s", str, str);
/* DO NOT DO THIS EITHER; IT IS STILL UB */
size_t len = strnlen(str, cap - 1);
int result = snprintf(str + len, cap - len, "%s", str);
The problem with the second invocation of snprintf is that the NUL which terminates str is precisely at str + len, the first byte of the output buffer. That's an overlap, so it's illegal.
The second important note about snprintf is that it returns a value, which must not be ignored. The value returned is not the length of the string created by snprintf. It's the length the string would have been had it not been truncated to fit in the output buffer.
If no truncation occurred, then the result is the length of the result, which must be strictly less than the size of the output buffer (because there must be room for a NUL terminator, which is not considered part of the length of the result.) You can use this fact to check whether truncation occurred:
if (result >= dstlen) /* Output was truncated */
This can be used, for example, to redo the snprintf with a larger, dynamically-allocated buffer (of size result + 1; never forget the need to NUL-terminate).
But remember that the result is an int -- that is, a signed value. That means that snprintf cannot cope with very long strings. That's not likely to be an issue in embedded code, but on systems where it's conceivable that strings exceed 2GB, you may not be able to safely use %s formats in snprintf. It also means that snprintf is allowed to return a negative value to indicate an error. Very old implementations of snprintf returned -1 to indicate truncation, or in response to being called with buffer length 0. That's not standard behaviour according to C99 (nor recent versions of Posix), but you should be prepared for it.
Standard-compliant implementations of snprintf will return a negative value if the buffer length argument is too big to fit in a (signed) int; it's not clear to me what the expected return value is if the buffer length is OK but the untruncated length is too big for an int. A negative value will also be returned if you used a conversion which resulted in an encoding error; for example, a %lc conversion whose corresponding argument contains an integer which cannot be converted to a multibyte (typically UTF-8) sequence.
In short, you should always check the return value of snprintf (recent gcc/glibc versions will produce a warning if you do not), and you should be prepared for it to be negative.
So, with all that behind us, let's write a function which produces a string of co-ordinate pairs:
/* Arguments:
* buf the output buffer.
* buflen the capacity of buf (including room for trailing NUL).
* points a vector of struct Point pairs.
* npoints the number of objects in points.
* Description:
* buf is overwritten with a comma-separated list of points enclosed in
* square brackets. Each point is output as a comma-separated pair of
* decimal floating point numbers enclosed in square brackets. No more
* than buflen - 1 characters are written. Unless buflen is 0, a NUL is
* written following the (possibly-truncated) output.
* Return value:
* If the output buffer contains the full output, the number of characters
* written to the output buffer, not including the NUL terminator.
* If the output was truncated, (size_t)(-1) is returned.
*/
size_t sprint_points(char* buf, size_t buflen,
struct Point const* points, size_t npoints)
{
if (buflen == 0) return (size_t)(-1);
size_t avail = buflen;
char delim = '['
while (npoints) {
int res = snprintf(buf, avail, "%c[%f,%f]",
delim, points->lat, points->lon);
if (res < 0 || res >= avail) return (size_t)(-1);
buf += res; avail -= res;
++points; --npoints;
delim = ',';
}
if (avail <= 1) return (size_t)(-1);
strcpy(buf, "]");
return buflen - (avail - 1);
}
Notes
You will often see code like this:
strncat(dst, src, sizeof(src)); /* NEVER EVER DO THIS! */
Telling strncat not to append more characters from src than can fit in src is obviously pointless (unless src is not correctly NUL-terminated, in which case you have a bigger problem). More importantly, it does absolutely nothing to protect you from writing beyond the end of the output buffer, since you have not done anything to check that dst has room for all those characters. So about all it does is get rid of compiler warnings about the unsafety of strcat. Since this code is exactly as unsafe as strcat was, you probably would be better off with the warning.
You might even find a compiler which understands snprintf will enough to parse the format string at compile time, so the convenience comes at no cost at all. (And if your current compiler doesn't do this, no doubt a future version will.) As with any use of the *printf family, you should never try to economize keystrokes by
leaving out the format string (snprintf(dst, dstlen, src) instead of snprintf(dst, dstlen, "%s", src).) That's unsafe (it has undefined behaviour if src contains an unduplicated %). And it's much slower because the library function has to parse the entire string to be copied looking for percent signs, instead of just copying it to the output.
Code is using functions that expect pointers to string, yet not always passing pointers to strings as arguments.
Stray characters seen at output of snprintf
A string must have a terminating null character.
strncat(char *, .... expects the first parameter to be a pointer to a string. memcpy(output, "[",1); does not insure that. #Jeremy
memcpy(output, "[",1);
...
strncat(output, temp,sizeof(temp));
This is a candidate source of stray characters.
strncat(...., ..., size_t size). itself is a problem as the size is the amount of space available for concatenating (minus the null character). The size available to char * output is not passed in. #Jonathan Leffler. Might as well do strcat() here.
Instead, pass in the size available to output to prevent buffer overflow.
#define N 60
int get_gps60secString(GPS_periodic_t input[N], char *output, size_t sz) {
int cnt = snprintf(output, sz, "[");
if (cnt < 0 || cnt >= sz)
return 1;
output += cnt;
sz -= cnt;
int i = 0;
for (i = 0; i < N; i++) {
cnt = snprintf(output, size, "[%0.8f,%0.8f]%s", input[i].point.latitude,
input[i].point.longitude, i + 1 == N ? "" : ",");
if (cnt < 0 || cnt >= sz)
return 1;
output += cnt;
sz -= cnt;
}
cnt = snprintf(output, sz, "]");
if (cnt < 0 || cnt >= sz)
return 1;
return 0; // no error
}
OP has posted more code - will review.
Apparently the buffer char *output is pre-filled with 0 before the get_gps60secString() so the missing null character from memcpy(output, "[",1); should not cause the issue - hmmmmmm
unsigned short SendTrack() does not return a value. 1) Using its result value is UB. 2) Enable all compiler warnings.
I have a simple C program that is supposed to expand the string FX according to the following rules:
X -> X+YF+
Y-> -FX-Y
The program is as follows:
#include <stdio.h>
void expandOnce( )
{
char* s = (char*)malloc(sizeof(char)*100);
int i=0;
char str[]="FX";
while(str[i]!='\0')
{
if(str[i]=='X')
{
s[i]="X+YF+";
}
if(str[i]=='Y')
{
s[i]="-FX-Y";
}
i++;
}
printf("%s",s);
}
void expandNtimes (int n)
{
for (int i =0; i < n; i++){
expandOnce();
}
}
int main(){
expandNtimes(2);
return 0;
}
If the program executes correctly, it should print out:
FX+YF++-FX-YF+
This is after 2 expansions of FX using the above rules.
Questions:
1. How can you make this work without printing out gibberish?
2. How can I return the expanded array from expandNtimes() and expandOnce() so that I can, for instance, use it in the main function?
I try this but I keep getting memory errors.
After your suggestions, I made the following changes:
#include <stdio.h>
#include <string.h>
#define NTIMES 2
#define MAXC 100
void expandOnce( )
{
char s[MAXC];
int i=0;
s[0]='F';
s[1]='X';
while(s[i]!='\0')
{
if(s[i]=='X'){
//shift all letters to the right by 5 steps
for (i; i < MAXC; i++){ //warning: statement with no effect (Wunused value)
s[i + 5]= s[i];
if(s[i]=='\0'){
break;}
}
//Fill in the expansion
s[i]='X';
s[i+1] ='+';
s[i+2] ='Y';
s[i+3]='F';
s[i+4] ='+';
}
if(s[i]=='Y'){
//shift all letters to the right by 5 steps
for (i; i < MAXC; i++){ //warning: statement with no effect (Wunused value)
s[i + 5]=s[i];
if(s[i]=='\0'){
break;}
}
//Fill in the expansion
s[i]='-';
s[i+1] ='F';
s[i+2] ='X';
s[i+3]='-';
s[i+4] ='Y';
}
i++;
}
printf("%s\n",s);
printf("Me\n");
}
void expandNtimes (int n)
{
for (int i =0; i < n; i++){
expandOnce();
}
}
int main(){
expandOnce();
expandNtimes(NTIMES);
return 0;
}
Both of my for loops have a warning warning: statement with no effect (Wunused value) which I do not understand. Finally, I get this error:
Segmentation fault (core dumped)
Is it allowed to do a for-loop inside a while-loop?
Oh Boy... Do you have a whole list of "you can't do that..." in your code.
To begin with there is no need to cast the return of malloc, it is unnecessary. See: Do I cast the result of malloc?.
char *s = malloc (sizeof *s * 100);
is all that is needed (note: if you always use sizeof dereference_pointer you will always have your type size correct.
This brings up another point, if you need constants #define one (or more) or use a global enum to accomplish the same thing, e.g.
#define NTIMES 2
#define MAXC 100
or
enum { NTIMES = 2, MAXC = 100 };
Then you can use them like,
char *s = malloc (sizeof *s * MAXC);
and
expandNtimes(NTIMES);
By defining constants, you then have one convenient place at the top of your code to make changes if needed in the future and not have to pick through each allocation or function call changing all your hardcoded magic numbers.
Your Memory Leak
You allocate for s in expandOnce, but never free s and don't return a pointer so that it could be later freed in main() (or any other calling function).
You should include free (s); after printf("%s",s); (which you should also include a newline in to prevent multiple outputs from running together, and so that your program is POSIX compliant -- outputting a final '\n'), e.g. printf("%s\n",s);
Moreover, "Why are you dynamically allocating anyway?" Just use a fixed buffer, you are not trying to return it. Just use:
char s[MAXC] = "";
s is Uninitialized and You Leave s[0] Uninitialized
s[i]="X+YF+"; and s[i]="-FX-Y"; write 4-characters to s from index 1. s[0] is never initialized which causes any subsequent attempt to read from s invoke Undefined Behavior. You cannot simply assign the pointer to string literal "X+YF+"; to a character s[i] -- your compiler should be screaming warnings and errors at you.
You can copy the literal to s with:
strcpy (s, "X+YF+");
or you can simply loop repeatedly copying as you go, e.g.
int i;
for (i = 0; str[i]; i++) /* loop over each char in str */
s[i] = str[i]; /* assign to s */
s[i] = 0; /* don't forget to nul-terminate s */
Compile with Warnings Enabled
Always compile with warnings enabled, and do not accept code until it compiles cleanly without warning. To enable warnings add -Wall -Wextra to your gcc or clang compile string. (add -pedantic for several additional warnings). For clang, instead you can use -Weverything. For VS (cl.exe on windoze), add /W3 (or use /Wall to enable all warnings).
Read and understand each warning. They will identify any problems, and the exact line on which they occur. You can learn as much about coding by simply listening to what your compiler is telling you as you can from most tutorials.
Put these suggestions to use and then edit your question (or ask a new one) if you have further problems.
Additional Comments After Edit
Gakuo, it is obvious you are still struggling to sort out string/character array handling in order to make the expansions work. As I mentioned in the comments, whenever you have a problem like this, do not pick up the keyboard and start pecking away hoping that you can tinker long enough that divine intervention will somehow guide you to a solution (it won't...).
Instead first pick up a 8 1/2 x 11 sheet of paper (graph paper works really well) and write out your original string, and then plan each step required to get you from your original string to the final string you need (don't forget to represent the '\0' in your layout as well)
For example, you could start with your original string:
+---+---+---+
| F | X |\0 |
+---+---+---+
The next step would be to represent the shift to make room for your expansion (you leave the original character in place since you will simply overwrite it):
+---+---+---+---+---+---+---+
| F | X | | | | |\0 |
+---+---+---+---+---+---+---+
Now it is a matter of copying "X+YF+" to the position of 'X' resulting in:
+---+---+---+---+---+---+---+
| F | X | + | Y | F | + |\0 |
+---+---+---+---+---+---+---+
Then simply repeat this process until you have fully exanded N times. As you do, you will be able to determine the pattern for expanding N times -- and that is when it is time to pick up the keyboard.
For your case, rather than manually looping everything, make use of memmove and memcpy to help shift and copy within the string. (you are free to manually shift and copy by looping -- as long as you can find more letters for you loop variable than just 'i'). Letting the memory functions help, you could do something like:
(note: edited to accommodate changing length of s in expandonce())
#include <stdio.h>
#include <string.h>
#define NTIMES 2
#define MAXC 100
#define SDEF "FX" /* string (default) */
#define XEXP "X+YF+" /* X expansion */
#define YEXP "-FX-Y" /* Y expansion */
/* returns pointer to 1-past last expansion on success, NULL otherwise */
char *expandonce (char *s)
{
char *p = s,
*expanded = NULL;
size_t lxexp = strlen (XEXP),
lyexp = strlen (YEXP);
while (*p) {
if (*p == 'X') { /* handle 'X' found */
size_t lens = strlen (s); /* get new length of s */
if (MAXC > lens + lxexp) { /* room to expand? */
size_t lenp = strlen (p); /* total to shift */
/* shift chars to right of 'X' by lxexp-1 including '\0' */
memmove (p + lxexp, p + 1, lenp + lxexp - 1);
memcpy (p, XEXP, lxexp); /* copy expansion */
p += lxexp; /* add offset */
expanded = p; /* set return */
}
}
else if (*p == 'Y') { /* handle 'Y' found */
size_t lens = strlen (s); /* same for 'Y' */
if (MAXC > lens + lyexp) {
size_t lenp = strlen (p);
memmove (p + lyexp, p + 1, lenp + lyexp - 1);
memcpy (p, YEXP, lyexp);
p += lyexp;
expanded = p;
}
}
else /* handle all other chars */
p++;
}
return expanded;
}
/* returns number of successful expansion loops */
int expandntimes (char *s, int n)
{
int i = 0;
char *p = s;
for (i = 0; i < n; i++)
if ((p = expandonce (s)) == NULL)
break;
return i;
}
int main (int argc, char **argv) {
char str[MAXC] = "";
int n = 0;
if (argc > 1 && strlen (argv[1]) < MAXC - 1)
strcpy (str, argv[1]);
else
strcpy (str, SDEF);
printf ("string: %s\nX_exp : %s\nY_exp : %s\n\n", str, XEXP, YEXP);
n = expandntimes (str, NTIMES);
if (n == 0) {
fputs ("error: no expansions possible.\n", stderr);
return 1;
}
printf ("%d expansions => %s\n", n, str);
return 0;
}
(note: this is a minimum example -- you should verify that all corner cases are covered as you step through your design)
Example Use/Output
Your example:
$ ./bin/expand2 "FX"
string: FX
X_exp : X+YF+
Y_exp : -FX-Y
2 expansions => FX+YF++-FX-YF+
A slightly different string:
$ ./bin/expand2 "XY"
string: XY
X_exp : X+YF+
Y_exp : -FX-Y
2 expansions => X+YF++-FX-YF+-FX+YF+--FX-Y
You getting memory errors because you try to write in memory where are you not supposed to do anything.if you want to append string you must use strcat() function! Line s[i]="X+YF+"; will never compile because you try to write multiple characters in place where is allowed only one character. i think it is better to make s global and append charracters one by one to s with strcat until you make string. You can access to global variables in main.Then print string s.
I'm about to implement a dynamic matrix structure (that stores double values) and I got some problems with reading from a file.
The idea is, that the program doesn't know the number of rows and columns in advance. It has to scan the first line in order to find the number of columns.
The problem with simply using fscanf() to scan doubles is, that (as far as I know) it can't differentiate between the newline and space characters, so it would read the whole file as one line.
To fix this I first fscanf() the line character-by-character with a function. It stores the values in a string, that represents exactly one line.
Then I use sscanf() to scan the string for double values and store them in a double array. After the conversion I free the string. This is done in the chararray_to_doublearray function.
Now after a bit of testing I suspect that the chararray_to_doublearray function is not working as intended.
/* Converts a character array to a double array and returns a pointer to it. Frees the space of the character array, as it's no longer needed. */
double *chararray_to_doublearray(char **chararray)
{
int i;
int elements = 0;
double *numbers=NULL;
double newnumber;
while (sscanf(*chararray, "%lf ", &newnumber) == 1) {
double* newarray = (double*) malloc(sizeof(double) * (elements+1));
for (i = 0; i < elements; ++i)
newarray[i] = numbers[i];
free(numbers);
numbers = newarray;
numbers[elements] = newnumber;
++elements;
}
free(*chararray);
return numbers;
}
And the main() function calling only the chararray_to_doublearray function:
main ()
{
int i;
double *numbers;
char string[50]="12.3 1.2 3.4 4 0.3";
numbers=chararray_to_doublearray(&string);
free(numbers)
return 0;
}
So to summarize: I couldn't find any good implementation of reading double values from the user (or from a file) until the end of line. This is my implementation.
Do you have any ideas, what might be wrong with this?
Regards,
naroslife
This is an XY problem. Do you really need to "fscanf() the line character-by-character"? Has this caused you to ask too much of your question in the wrong direction?
Consider this: %lf denotes a conversion of characters to the double that you choose... It stops immediately when there are no more suitable characters to convert... and a newline is not a suitable character to convert... Is there a light bulb shining in your head, yet?
In your case, the space that follows %lf in the format string causes the useful information (whether the white-space is a newline or not) to be discarded. STOP! You've gone too far, and the consequence is that you now need an intermediate character array conversion function, which is unnecessary bloat.
With this new-found realisation that removing the white-space from the format string will cause a post-fixed newline to be left onto the stream, consider using fgetc to handle the distinction between regular white-space and newlines.
e.g.
double f;
int x = scanf("%lf", &f);
int c;
do {
c = getchar();
} while (isspace(c) && c != '\n');
if (c != '\n') {
ungetc(c, stdin);
}
See above, how I was able to distinguish between newline and non-newline white-space?
There is nothing difficult about reading an unknown number of double values from a file or stdin and storing them in a simulated 2D array. (pointer-to-pointer-to-type) Since you have to assume the number of columns may also differ per-row, you need a similar way to allocate column storage, keep track of the number of values allocated/read, and a way to reallocate the column storage if/when the maximum number of columns are reached. This allows handling a jagged array as easily as an array with a fixed size of columns.
There is one subtle trick that greatly helps in managing jagged arrays. Since you do not know before hand how many column values may be present -- once read, you need a way to store the number of column elements present (for each row in the array). A simple and robust method is simply to store the number of column elements per-row as the first column value. Then after the data is collected, you have the information as part of the array that provides a key to iterating over all rows and columns in the array.
Included as part of this approach, I have created specialty functions xstrtod, xcalloc, xrealloc_sp (realloc of single-pointer array) and realloc_dp (realloc for double-pointer). These are nothing more than the standard functions with appropriate error-checking moved to the function so the myriad of validation checks don't cloud the main body of the code.
A quick implementation that reads values from stdin could be coded as follows:
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <limits.h>
#include <errno.h>
#include <math.h> /* for HUGE_VALF, HUGE_VALL */
#define ROWS 32
#define COLS 32
#define MAXC 256
double xstrtod (char *str, char **ep);
void *xcalloc (size_t n, size_t s);
void *xrealloc_sp (void *p, size_t sz, size_t *n);
void *xrealloc_dp (void **p, size_t *n);
int main (void) {
char line[MAXC] = {0}; /* line buffer for fgets */
char *p, *ep; /* pointers for strtod */
double **array = NULL; /* array of values */
size_t row = 0, col = 0, nrows = 0; /* indexes, number of rows */
size_t rmax = ROWS, cmax = COLS; /* row/col allocation size */
/* allocate ROWS number of pointers to array of double */
array = xcalloc (ROWS, sizeof *array);
/* read each line in file */
while (fgets(line, MAXC, stdin))
{
p = ep = line; /* initize pointer/end pointer */
col = 1; /* start col at 1, store ncols in 0 */
cmax = COLS; /* reset cmax for each row */
/* allocate COLS number of double for each row */
array[row] = xcalloc (COLS, sizeof **array);
/* convert each string of digits to number */
while (errno == 0)
{
array[row][col++] = xstrtod (p, &ep);
if (col == cmax) /* if cmax reached, realloc array[row] */
array[row] = xrealloc_sp (array[row], sizeof *array[row], &cmax);
/* skip delimiters/move pointer to next digit */
while (*ep && *ep != '-' && (*ep < '0' || *ep > '9')) ep++;
if (*ep)
p = ep;
else /* break if end of string */
break;
}
array[row++][0] = col; /* store ncols in array[row][0] */
/* realloc rows if needed */
if (row == rmax) array = xrealloc_dp ((void **)array, &rmax);
}
nrows = row; /* set nrows to final number of rows */
printf ("\n the simulated 2D array elements are:\n\n");
for (row = 0; row < nrows; row++) {
for (col = 1; col < (size_t)array[row][0]; col++)
printf (" %8.2lf", array[row][col]);
putchar ('\n');
}
putchar ('\n');
/* free all allocated memory */
for (row = 0; row < nrows; row++)
free (array[row]);
free (array);
return 0;
}
/** string to double with error checking.
* #include <math.h> for HUGE_VALF, HUGE_VALL
*/
double xstrtod (char *str, char **ep)
{
errno = 0;
double val = strtod (str, ep);
/* Check for various possible errors */
if ((errno == ERANGE && (val == HUGE_VAL || val == HUGE_VALL)) ||
(errno != 0 && val == 0)) {
perror ("strtod");
exit (EXIT_FAILURE);
}
if (*ep == str) {
fprintf (stderr, "No digits were found\n");
exit (EXIT_FAILURE);
}
return val;
}
/** xcalloc allocates memory using calloc and validates the return.
* xcalloc allocates memory and reports an error if the value is
* null, returning a memory address only if the value is nonzero
* freeing the caller of validating within the body of code.
*/
void *xcalloc (size_t n, size_t s)
{
register void *memptr = calloc (n, s);
if (memptr == 0)
{
fprintf (stderr, "%s() error: virtual memory exhausted.\n", __func__);
exit (EXIT_FAILURE);
}
return memptr;
}
/** reallocate array of type size 'sz', to 2 * 'n'.
* accepts any pointer p, with current allocation 'n',
* with the type size 'sz' and reallocates memory to
* 2 * 'n', updating the value of 'n' and returning a
* pointer to the newly allocated block of memory on
* success, exits otherwise. all new memory is
* initialized to '0' with memset.
*/
void *xrealloc_sp (void *p, size_t sz, size_t *n)
{
void *tmp = realloc (p, 2 * *n * sz);
#ifdef DEBUG
printf ("\n reallocating '%zu' to '%zu', size '%zu'\n", *n, *n * 2, sz);
#endif
if (!tmp) {
fprintf (stderr, "%s() error: virtual memory exhausted.\n", __func__);
exit (EXIT_FAILURE);
}
p = tmp;
memset (p + *n * sz, 0, *n * sz); /* zero new memory */
*n *= 2;
return p;
}
/** reallocate memory for array of pointers to 2 * 'n'.
* accepts any pointer 'p', with current allocation of,
* 'n' pointers and reallocates to 2 * 'n' pointers
* intializing the new pointers to NULL and returning
* a pointer to the newly allocated block of memory on
* success, exits otherwise.
*/
void *xrealloc_dp (void **p, size_t *n)
{
void *tmp = realloc (p, 2 * *n * sizeof tmp);
#ifdef DEBUG
printf ("\n reallocating %zu to %zu\n", *n, *n * 2);
#endif
if (!tmp) {
fprintf (stderr, "%s() error: virtual memory exhausted.\n", __func__);
exit (EXIT_FAILURE);
}
p = tmp;
memset (p + *n, 0, *n * sizeof tmp); /* set new pointers NULL */
*n *= 2;
return p;
}
Compile
gcc -Wall -Wextra -Ofast -o bin/fgets_strtod_dyn fgets_strtod_dyn.c
Input
$ cat dat/float_4col.txt
2078.62 5.69982 -0.17815 -0.04732
5234.95 8.40361 0.04028 0.10852
2143.66 5.35245 0.10747 -0.11584
7216.99 2.93732 -0.18327 -0.20545
1687.24 3.37211 0.14195 -0.14865
2065.23 34.0188 0.1828 0.21199
2664.57 2.91035 0.19513 0.35112
7815.15 9.48227 -0.11522 0.19523
5166.16 5.12382 -0.29997 -0.40592
6777.11 5.53529 -0.37287 -0.43299
4596.48 1.51918 -0.33986 0.09597
6720.56 15.4161 -0.00158 -0.0433
2652.65 5.51849 0.41896 -0.61039
Output
$ ./bin/fgets_strtod_dyn <dat/float_4col.txt
the simulated 2D array elements are:
2078.62 5.70 -0.18 -0.05
5234.95 8.40 0.04 0.11
2143.66 5.35 0.11 -0.12
7216.99 2.94 -0.18 -0.21
1687.24 3.37 0.14 -0.15
2065.23 34.02 0.18 0.21
2664.57 2.91 0.20 0.35
7815.15 9.48 -0.12 0.20
5166.16 5.12 -0.30 -0.41
6777.11 5.54 -0.37 -0.43
4596.48 1.52 -0.34 0.10
6720.56 15.42 -0.00 -0.04
2652.65 5.52 0.42 -0.61
Memory Check
In any code your write that dynamically allocates memory, it is imperative that you use a memory error checking program to insure you haven't written beyond/outside your allocated block of memory and to confirm that you have freed all the memory you have allocated. For Linux valgrind is the normal choice. There are so many subtle ways to misuse a block of memory that can cause real problems, there is no excuse not to do it. There are similar memory checkers for every platform. They are all simple to use. Just run your program through it.
$ valgrind ./bin/fgets_strtod_dyn <dat/float_4col.txt
==28022== Memcheck, a memory error detector
==28022== Copyright (C) 2002-2012, and GNU GPL'd, by Julian Seward et al.
==28022== Using Valgrind-3.8.1 and LibVEX; rerun with -h for copyright info
==28022== Command: ./bin/fgets_strtod_dyn
==28022==
the simulated 2D array elements are:
2078.62 5.70 -0.18 -0.05
5234.95 8.40 0.04 0.11
2143.66 5.35 0.11 -0.12
7216.99 2.94 -0.18 -0.21
1687.24 3.37 0.14 -0.15
2065.23 34.02 0.18 0.21
2664.57 2.91 0.20 0.35
7815.15 9.48 -0.12 0.20
5166.16 5.12 -0.30 -0.41
6777.11 5.54 -0.37 -0.43
4596.48 1.52 -0.34 0.10
6720.56 15.42 -0.00 -0.04
2652.65 5.52 0.42 -0.61
==28022==
==28022== HEAP SUMMARY:
==28022== in use at exit: 0 bytes in 0 blocks
==28022== total heap usage: 14 allocs, 14 frees, 3,584 bytes allocated
==28022==
==28022== All heap blocks were freed -- no leaks are possible
==28022==
==28022== For counts of detected and suppressed errors, rerun with: -v
==28022== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 2 from 2)
There is nothing difficult about reading an unknown number of rows and unknown number of columns from a file in C, but you must pay particular attention to how you do it. While you can limit the array to a square (NxN) array, there is no reason every row can't have a different number of columns (a jagged-array).
Your basic approach is to allocate memory for an array or pointers to type double for some reasonable anticipated number of rows. ( #define ROWS 32 ) You will then read each line. For every line you read you then allocate a block of memory for an array of 'double' for some reasonably anticipated number of doubles. ( #define COLS 32 )
You then convert each string of digits encountered to an double value and store the number at array[row][col]. (we actually start storing values at col = 1 and save col = 0 to hold the final number of cols for that row) You keep track of the number you have added to the array and if your number of columns reaches the number you allocated, you then realloc the array to hold additional doubles.
You continue reading lines until you have read all the lines. If you reach your original limit on the number of rows, you simply realloc the array much like you did with cols.
You now have all your data stored and can do with it what you will. When you are done, do not forget to free all memory you have allocated. Let me know if you have questions.
Quick Brown Fox Separated File
There is one further bit of additional robustness that you can build into the code that will basically allow you to read any row of data no matter how much junk may be included in the file. It doesn't matter if the row-values are comma separated, semi-colon separated, space separated, or separated by the quick brown fox. With a little parsing help, you can prevent read failures by manually advancing to the beginning of the next number. A quick addition in context would be:
while (errno == 0)
{
/* skip any non-digit characters */
while (*p && ((*p != '-' && (*p < '0' || *p > '9')) ||
(*p == '-' && (*(p+1) < '0' || *(p+1) > '9')))) p++;
if (!*p) break;
array[row][col++] = xstrtod (p, &ep);
...
Skipping the non-digits will allow you to read almost any sane file with any type of delimiter without issue. Take for example, the same numbers used originally, but now formatted as follows in the data file:
$ cat dat/float_4colmess.txt
The, 2078.62 quick 5.69982 brown -0.17815 fox; -0.04732 jumps
5234.95 over 8.40361 the 0.04028 lazy 0.10852 dog
and the 2143.66 dish ran 5.35245 away 0.10747 with -0.11584
the spoon, 7216.99 2.93732 -0.18327 -0.20545
1687.24 3.37211 0.14195 -0.14865
2065.23 34.0188 0.1828 0.21199
2664.57 2.91035 0.19513 0.35112
7815.15 9.48227 -0.11522 0.19523
5166.16 5.12382 -0.29997 -0.40592
6777.11 5.53529 -0.37287 -0.43299
4596.48 1.51918 -0.33986 0.09597
6720.56 15.4161 -0.00158 -0.0433
2652.65 5.51849 0.41896 -0.61039
Even with this insane format, the code has no problems properly reading all numeric values into the array properly.
I have a function which writes binary data to a file or stdout. However the call to fwrite() sometimes fails, UNLESS I fflush(stderr) or print a report to stderr before attempting to fwrite.
Is this normal behaviour, or indicative of some underlying memory problem? It's difficult to debug because as soon as I fprint to stderr the data I'm trying to fwrite to stdout, the "problem" disappears.
Here's a very simplified version of the function.
int writebinary(FILE *fpout, void *data, long ntowrite) {
long i;
/* lets say we know the input data is double float */
double *p = data;
/* including this test line makes the function work! */
for(i=0; i<ntowrite;i++) fprintf(stderr,"data[%d]=%g\n",i,p[i]);
/* attempt to write data[] as a single block of bytes */
m= fwrite(data,(size_t)(ntowrite*sizeof(double)),1,fpout);
if(m!=1) {
fprintf(stderr,"Write error: %d\n",ferror(fpout));
return(-1);
}
else return(m);
}
Any wisdom appreciated :)
This is not normal behavior.
Recommend the following. There are a number of things in the posting which are of concern (see far below) but do not point to the problem - but something is wrong.
int writebinary2(FILE *fpout, void *data, long ntowrite) {
size_t n = (size_t) ntowrite;
double *dp = (double *) data;
// pedantic checks
if ((!fpout) || (!data) || (dp != data) || (ntowrite <= 0) || (n != ntowrite)) {
fprintf(stderr,"Bad parameter\n");
return(0);
}
clearerr(stderr); // only needed for debug
clearerr(fpout);
/* attempt to write blocks of doubles */
size_t m = fwrite(dp, sizeof(double), n, fpout);
if (m != n) {
fprintf(stderr,"Write error: %d\n",ferror(fpout));
clearerr(fpout);
return(-1);
}
if (fflush(fpout)) {
fprintf(stderr,"Flush error: %d\n",ferror(fpout));
clearerr(fpout);
return(-1);
}
return(1);
}
ntowrite s/b size_t, an unsigned integer. ntowrite = 0 may cause confusion on m. ntowrite < 0 will cause issues.
m is not declared, assumed size_t.
format "data[%d]=%g\n" s/b "data[%ld]=%g\n"
i s/b size_t.
(size_t)(ntowrite*sizeof(double)) s/b ((size_t)ntowrite)*sizeof(double). See #7.
fprintf(stderr... should not be needed. That is the question.
fwrite(..., size, n ...) works even if size * n overflow. So safer than OP.