I have a server side C based CGI code as:
cgiFormFileSize("UPDATEFILE", &size); //UPDATEFILE = file being uploaded
cgiFormFileName("UPDATEFILE", file_name, 1024);
cgiFormFileContentType("UPDATEFILE", mime_type, 1024);
buffer = malloc(sizeof(char) * size);
if (cgiFormFileOpen("UPDATEFILE", &file) != cgiFormSuccess) {
exit(1);
}
output = fopen("/tmp/cgi.tar.gz", "w+");
inc = size/(1024*100);
fptr = fopen("progress_bar.txt", "w+");
while (cgiFormFileRead(file, b, sizeof(b), &got_count) == cgiFormSuccess)
{
fwrite(b,sizeof(char),got_count,output);
i++;
if(i == inc && j<=100)
{
fprintf(fptr,"%d", j);
fflush(fptr);
i = 0;
j++; // j is the progress bar increment value
}
}
fclose(fptr);
cgiFormFileClose(file);
retval = system("mkdir /tmp/update-tmp;\
cd /tmp/update-tmp;\
tar -xzf ../cgi.tar.gz;\
bash -c /tmp/update-tmp/update.sh");
However, this doesn't work the way as is seen above. Instead of printing 1,2,...100 to progress_bar.txt (referred by fptr)one by one it prints at ONE GO, seems it buffers and then writes to the file.
fflush() also didn't work.
Any clue/suggestion would be really appreciated.
First, open the file before the loop and close after it ends. Too much IO.
The problem is here w+ - this truncates your file. use a+. (fopen help)
It is writing it one-by-one, it's just that it does it so fast that you're vanishingly unlikely to ever see the file with a value other than 99 in it.
This is easily demonstrated if you put a sleep(1) within the loop, so that it's slow enough for you to catch it.
Related
I want to change lines which contain the # symbol in a text file with heet using C.
I have tried it this way, but it did not work thoroughly, it just replaces the characters & overwrites not the whole string, like I want.
Is there any other trick to remove or delete a whole line from the file? So, we can easily replace it.
myfile.txt: (before execution)
Joy
#Smith
Lee
Sara#
Priyanka
#Addy
Code:
#include <stdio.h>
#include <string.h>
int main() {
FILE *pFile;
fpos_t pos1, pos2;
int line = 0;
char buf[68]
char *p;
char temp[10] = "heet";
pFile = fopen("myfile.txt", "r+");
printf("changes are made in this lines:\t");
while (!feof(pFile)) {
++line;
fgetpos(pFile, &pos1);
if (fgets(buf, 68, pFile) == NULL)
break;
fgetpos(pFile, &pos2);
p = strchr(buf, '#');
if (p != NULL) {
printf("%d, " , line);
fsetpos(pFile, &pos1);
fputs(temp, pFile);
}
fsetpos(pFile, &pos2);
}
fclose(pFile);
return 0;
}
myfile.txt: (after execution)
Joy
heetth
Lee
heet#
Priyanka
heety
Output:
changes are made in this lines: 2, 4, 6,
myfile.txt: (I want to get)
Joy
heet
Lee
heet
Priyanka
heet
The best way of doing what you want is to use a utility like sed. It is faster and uses less memory than anything you (or I) would write.
That aside, let's assume you want to go ahead and write it yourself anyway.
A file is just like a long array of bytes. If you want to increase or decrease the length of one line, it affects the position of every byte in the rest of the file. The result can be shorter (or longer) than the original. As the result can be shorter, modifying the file in place is a bad idea.
The following pseudo-code illustrates a simple approach:
open original file
open output file
allocate a line buffer that is large enough
read a line from the original file
do
return an error if the buffer is too small
manipulate the line
write the manipulated line to the output file
read a line from the original file
loop until read returns nothing
sed does it much smarter. I once saw an explanation on how sed works, but my google karma can't seem to find it.
Edit:
How to do it using sed:
sed -e 's/.*\#.*/heet/g' myfile.txt
The s, or substitute, command of sed can replace one string, or regular expression, with another string.
The above command is interpreted as:
replace any line that has a # somewhere in it with heet. The final g tells sed to do this globally, i.e. in the entire file.
Edit2:
By default, sed writes to standard output.
To rewrite the file you should redirect the output to a file and then rename it.
In linux, do the following (you can run command line stuff from C with system):
sed -e 's/.*\#.*/heet/g' myfile.txt > temp_file123.txt
rm myfile.txt
mv temp_file123.txt myfile.txt
From C:
system("sed -e 's/.*\#.*/heet/g' myfile.txt > temp_file123.txt");
system("rm myfile.txt");
system("mv temp_file123.txt myfile.txt");
If you want to do it with just one call to system, put all the command line stuff in a shell script.
You should probably treat input/output like a UNIX utility and replace the line by reading in the whole input and writing the whole output like sed would or something. It's going to be a pain to edit the line in place as you need to shift the following text 'down' in order to make it work.
You cannot achieve your goal by overwriting the file in place like you do in the code because heet is 3 bytes longer than # and there is no standard function to insert bytes in the middle of a file.
Note also these important issues:
you do not test if fopen() succeeds at opening the file. You have undefined behavior if the file does not exist or cannot be open for read+update mode.
while (!feof(pFile)) does not stop exactly at the end of file because the end of file indicator returned by feof() is only set when a read operation fails, not before. You should instead write:
while (fgets(buf, 68, pFile) != NULL) {
if the file has lines longer than 66 characters, the line numbers will be computed incorrectly.
There are 2 ways to replace the text in the file:
you can create a temporary file and write the modified contents to it. Once the contents have all been converted, delete the original file with remove() and rename the temporary file to the original name with rename(). This method uses extra space on the storage device, and requires that you can create a new file and determine a file name that does not conflict with existing file names.
alternately, you can read the complete contents of the original file and overwrite it with the modified contents from the start. This works because the modified contents is longer than the original contents. This method may fail if the file is very large and does not fit in memory, which is rather rare today for regular text files.
Here is a modified version using the second approach:
#include <stdio.h>
#include <string.h>
#include <stdlib.h>
int main() {
FILE *pFile;
int c, line, changes;
unsigned char *buf;
size_t pos, length, size;
char replacement[] = "heet";
/* open the file */
pFile = fopen("myfile.txt", "r+");
if (pFile == NULL) {
printf("cannot open myfile.txt\n");
return 1;
}
/* read the file */
buf = NULL;
length = size = 0;
while ((c = getc(pFile)) != EOF) {
if (length == size) {
size = size + size / 2 + 128;
buf = realloc(buf, size);
if (buf == NULL) {
printf("not enough memory to read myfile.txt\n");
fclose(pFile);
return 1;
}
}
buf[length++] = c;
}
/* write the modified contents */
rewind(pFile);
line = 1;
changes = 0;
for (pos = 0; pos < length; pos++) {
c = buf[pos];
if (c == '\n')
line++;
if (c == '#') {
if (changes++ == 0)
printf("changes are made in this lines:\t");
else
printf(", ");
printf("%d", line);
fputs(replacement, pFile);
} else {
putc(c, pFile);
}
}
free(buf);
fclose(pFile);
if (changes == 0)
printf("no changes were made\n");
else
printf("\n");
return 0;
}
To rewrite a word in file using fwrite or any file writing function, use fgetpos and fsetpos. Otherwise seeking file pointer alone will not work. Still this work, if the file pointer is end of the file, it means append is possible.
I need to create a program that does this:
execute a command with popen
do things with the output of popen(use the FILE in a lot of things)
stay checking for popen output changes, if have one, re execute everything.
The source code is here: https://gitorious.org/clyv/clyv
So I only want to execute all the rest of the program AGAIN if there is a change in the output of popen (must be compared with the first output)
The program should do everything first time, and after, only do everything and print again if there is a change on popen output. The popen should verified once a second.
Update
I didn't get any answer that solve my problem here, but reading a C tutorial i saw something about threads, and it sounds like the solution to me, i will see what i can do.
You are free to call popen() as many times as needed. But, to properly release resources used by a call to popen(), you need to call pclose().
In your case, you probably want to just poll the output occasionally, and emit something whenever it is necessary to do so.
first_time = 1;
need_to_print = 1;
for (;;) {
FILE *fp = popen(...);
/* read input ... */
pclose(fp);
/* parse input ... */
if (first_time) {
/* save contents for future comparison... */
first_time = 0;
} else {
need_to_print = /* result of comparing saved contents with new contents */;
}
if (need_to_print) {
/* print something ... */
}
sleep(INTERVAL);
}
You can use
//an array of 2 string buffers
char output[2][1024];
//the string buffer index in use
int flip = 0;
FILE *fp;
void executeTestCommand() {
int resultSize;
fp = popen("free -m | awk 'NR==3 {print $3}'" "r");
if (fp == NULL) {
perror("Failed to run command");
}
else {
result_size = fread(output, 1, 1024 - 1, fp);
//place string terminator
output[flip][result_size] = '\0';
}
//flip the buffers
flip = flip ^ 1;
pclose(fp);
}
int main() {
//init with empty string
output[0][0] = '\0';
output[1][0] = '\0';
//for ever
for(;;) {
executeTestCommand();
//if the last command output differs from the current
//command output
if (strcmp(output[0], output[1])) {
//execute the rest of the command here
}
}
return 0;
}
Cheers!
You want to close the file pointer (you don't want any memory leaks) since you are going to use the same FILE * fp variable again with popen.
Later edit: hope it helps:). You might want to insert a sleep statement inside the for ever block because this will be cpu intensive.
#AlinUngureanu #JonathanLeffler #jxh
I published the source on git! Now you can see what i want
https://gitorious.org/clyv/clyv
I'm working on opening a file from the mainframe. currently, I can't access the input file I wanted. I don't know if there is something wrong with the C code or my JCL. Anyone help me out?
Here is my code:
int main()
{
FILE *ifp, *ofp;
printf("CTRACE1\n");
ifp = fopen("dd:INPUTF", "rb, recfm=vb, lrecl=50");
printf("CTRACE2\n");
ofp = fopen("dd:OUTPUTF","w");
printf("CTRACE3\n");
fread( buffer, sizeof( char ), LINESZ, ifp );
printf( "Number of characters read = %i\n", num );
printf( "buffer = %s\n", buffer );
dstr = (DATASTR*) buffer;
printf("VAR_A = %.*s\n", sizeof(dstr->VAR_A), dstr->VAR_A);
printf("VAR_B = %.*s\n", sizeof(dstr->VAR_B), dstr->VAR_B);
printf("VAR_C = %.*s\n", sizeof(dstr->VAR_C), dstr->VAR_C);
printf("CTRACE4\n");
x[sizeof(x)+1]='\0';
y[sizeof(y)+1]='\0';
z[sizeof(z)+1]='\0';
printf("CTRACE5\n");
memcpy(x, dstr->VAR_A,sizeof(dstr->VAR_A));
memcpy(y, dstr->VAR_B,sizeof(dstr->VAR_B));
memcpy(z, dstr->VAR_C,sizeof(dstr->VAR_C));
printf("CTRACE6\n");
printf("%s,%s,%s",x,y,z);
printf("CTRACE7\n");
fwrite(buffer, sizeof(char), LINESZ, ofp);
fprintf(ofp,"%s,%s,%s\n",x,y,z);
fclose(ifp);
fclose(ofp);
return(0);
}
Here's my JCL:
...
//* TYPRUN=SCAN
//JOBLIB DD DSN=X543863.LOADLIB1,DISP=SHR
//STEP1 EXEC PGM=CCCGRATE
//INPUTF DD DSN=X543863.SAMPLE.INPUT01,DISP=SHR
//OUTPUTF DD DSN=X543863.SAMPLE.OUTPUT01,
// DISP=(NEW,CATLG,DELETE),
// SPACE=(CYL,(1,1,45)),
// DCB=(RECFM=FB,LRECL=50)
//SYSOUT DD SYSOUT=*
//
add an
#include <errno.h>
rewrite the open() calls to trap errors
if (!ifp = fopen("dd:INPUTF", "rb, recfm=vb, lrecl=50"))
{
perror("ifp");
exit(1);
}
printf("CTRACE2\n");
if (!ofp = fopen("dd:OUTPUTF","w"))
{
perror("ofp");
exit(1);
}
And you should get a clue on why the input file does not work
Without knowing exactly what output you get, at best it is difficult to say.
Are you trying to copy variable-length records to fixed-length records?
Two things. On the fopen for an input file you do not need to specify DCB information - so recfm=vb and lrecl=50 are not needed (the information if not specified, will be taken from the JCL (if present) or from the catalog (which will be correct)).
Since they are not needed, you have probably got them wrong :-)
Take them off, and try your program.
Looking at what you have there and have specified for the output, either the vb is wrong (you are writing an fb) or the 50 is wrong (if you have 50 bytes of data, which you logically have from your fb definition, the the lrecl for a vb should be 54, because four bytes extra to the data are needed to include the RDW (Record Descriptor Word)).
The is nothing wrong with your JCL, but no way to tell whether the files and definitions of files are correct.
Down to you now. If you still can't fix it, provide all the likely information.
I have a programme which is writing results to a file and I would like to read in real-time from that file. It is a normal text file and external programme always write a whole line. I need to run it just on a Linux system.
int last_read = 0;
int res;
FILE *file;
char *buf;
char *end = buf + MAXLINE - 1;
int c;
int fileSize;
char *dst;
while (external_programme_is_running()) {
file = fopen(logfile, "r"); //without opening and closing it's not working
fseek(file, 0, SEEK_END);
fileSize = ftell(file);
if (fileSize > last_read) {
fseek(file, last_read, SEEK_SET);
while (!feof(file)) {
dst = buf;
while ((c = fgetc(file)) != EOF && c != '\n' && dst < end)
*dst++ = c;
*dst = '\0';
res = ((c == EOF && dst == buf) ? EOF : dst - buf);
if (res != -1) {
last_read = ftell(file);
parse_result(buf)
}
}
}
fclose(file);
}
Is this a correct approach? Or would it be better to check the modification time and then open the file? Is is possible that reading would crash in case that the file would be modified at the very same time?
To avoid the need to close, re-open, and re-seek for every loop iteration, call clearerr on the stream after reading EOF.
You shouldn't have any problems if you read at the same time the other program writes. The worst that would happen is that you wouldn't get to see what was written until the next time you open the file.
Also, your approach of comparing the last seek position to the end of the file is a fine way to look for additions to the file, since the external program is simply writing additional lines. I would recommend adding a sleep(1) at the end of your loop, though, so you don't use a ton of CPU.
There's no problem in reading a file while another process is writing to it. The standard tail -f utility is used often for that very purpose.
The only caveat is that the writing process must not exclusively lock the file.
Regarding your read code (ie. using fgetc()), since you said that the writing process will be writing a line at a time, you might want to look at fgets() instead.
I want to change lines which contain the # symbol in a text file with heet using C.
I have tried it this way, but it did not work thoroughly, it just replaces the characters & overwrites not the whole string, like I want.
Is there any other trick to remove or delete a whole line from the file? So, we can easily replace it.
myfile.txt: (before execution)
Joy
#Smith
Lee
Sara#
Priyanka
#Addy
Code:
#include <stdio.h>
#include <string.h>
int main() {
FILE *pFile;
fpos_t pos1, pos2;
int line = 0;
char buf[68]
char *p;
char temp[10] = "heet";
pFile = fopen("myfile.txt", "r+");
printf("changes are made in this lines:\t");
while (!feof(pFile)) {
++line;
fgetpos(pFile, &pos1);
if (fgets(buf, 68, pFile) == NULL)
break;
fgetpos(pFile, &pos2);
p = strchr(buf, '#');
if (p != NULL) {
printf("%d, " , line);
fsetpos(pFile, &pos1);
fputs(temp, pFile);
}
fsetpos(pFile, &pos2);
}
fclose(pFile);
return 0;
}
myfile.txt: (after execution)
Joy
heetth
Lee
heet#
Priyanka
heety
Output:
changes are made in this lines: 2, 4, 6,
myfile.txt: (I want to get)
Joy
heet
Lee
heet
Priyanka
heet
The best way of doing what you want is to use a utility like sed. It is faster and uses less memory than anything you (or I) would write.
That aside, let's assume you want to go ahead and write it yourself anyway.
A file is just like a long array of bytes. If you want to increase or decrease the length of one line, it affects the position of every byte in the rest of the file. The result can be shorter (or longer) than the original. As the result can be shorter, modifying the file in place is a bad idea.
The following pseudo-code illustrates a simple approach:
open original file
open output file
allocate a line buffer that is large enough
read a line from the original file
do
return an error if the buffer is too small
manipulate the line
write the manipulated line to the output file
read a line from the original file
loop until read returns nothing
sed does it much smarter. I once saw an explanation on how sed works, but my google karma can't seem to find it.
Edit:
How to do it using sed:
sed -e 's/.*\#.*/heet/g' myfile.txt
The s, or substitute, command of sed can replace one string, or regular expression, with another string.
The above command is interpreted as:
replace any line that has a # somewhere in it with heet. The final g tells sed to do this globally, i.e. in the entire file.
Edit2:
By default, sed writes to standard output.
To rewrite the file you should redirect the output to a file and then rename it.
In linux, do the following (you can run command line stuff from C with system):
sed -e 's/.*\#.*/heet/g' myfile.txt > temp_file123.txt
rm myfile.txt
mv temp_file123.txt myfile.txt
From C:
system("sed -e 's/.*\#.*/heet/g' myfile.txt > temp_file123.txt");
system("rm myfile.txt");
system("mv temp_file123.txt myfile.txt");
If you want to do it with just one call to system, put all the command line stuff in a shell script.
You should probably treat input/output like a UNIX utility and replace the line by reading in the whole input and writing the whole output like sed would or something. It's going to be a pain to edit the line in place as you need to shift the following text 'down' in order to make it work.
You cannot achieve your goal by overwriting the file in place like you do in the code because heet is 3 bytes longer than # and there is no standard function to insert bytes in the middle of a file.
Note also these important issues:
you do not test if fopen() succeeds at opening the file. You have undefined behavior if the file does not exist or cannot be open for read+update mode.
while (!feof(pFile)) does not stop exactly at the end of file because the end of file indicator returned by feof() is only set when a read operation fails, not before. You should instead write:
while (fgets(buf, 68, pFile) != NULL) {
if the file has lines longer than 66 characters, the line numbers will be computed incorrectly.
There are 2 ways to replace the text in the file:
you can create a temporary file and write the modified contents to it. Once the contents have all been converted, delete the original file with remove() and rename the temporary file to the original name with rename(). This method uses extra space on the storage device, and requires that you can create a new file and determine a file name that does not conflict with existing file names.
alternately, you can read the complete contents of the original file and overwrite it with the modified contents from the start. This works because the modified contents is longer than the original contents. This method may fail if the file is very large and does not fit in memory, which is rather rare today for regular text files.
Here is a modified version using the second approach:
#include <stdio.h>
#include <string.h>
#include <stdlib.h>
int main() {
FILE *pFile;
int c, line, changes;
unsigned char *buf;
size_t pos, length, size;
char replacement[] = "heet";
/* open the file */
pFile = fopen("myfile.txt", "r+");
if (pFile == NULL) {
printf("cannot open myfile.txt\n");
return 1;
}
/* read the file */
buf = NULL;
length = size = 0;
while ((c = getc(pFile)) != EOF) {
if (length == size) {
size = size + size / 2 + 128;
buf = realloc(buf, size);
if (buf == NULL) {
printf("not enough memory to read myfile.txt\n");
fclose(pFile);
return 1;
}
}
buf[length++] = c;
}
/* write the modified contents */
rewind(pFile);
line = 1;
changes = 0;
for (pos = 0; pos < length; pos++) {
c = buf[pos];
if (c == '\n')
line++;
if (c == '#') {
if (changes++ == 0)
printf("changes are made in this lines:\t");
else
printf(", ");
printf("%d", line);
fputs(replacement, pFile);
} else {
putc(c, pFile);
}
}
free(buf);
fclose(pFile);
if (changes == 0)
printf("no changes were made\n");
else
printf("\n");
return 0;
}
To rewrite a word in file using fwrite or any file writing function, use fgetpos and fsetpos. Otherwise seeking file pointer alone will not work. Still this work, if the file pointer is end of the file, it means append is possible.