I have a pipe that contains a series of numbers.
I will read it and at each iteration I will write what has been read in a txt file.
How can I convert binary numbers to decimal numbers and record them, one per line, in my txt file?
PS
The pipe is created in another file and the numbers are written using this command:
write (fp, &mynum, sizeof (mynum));
Main file
#include <fcntl.h>
#include <stdio.h>
#include <stdlib.h>
#include <sys/stat.h>
#include <unistd.h>
#define MAX_BUF 1024
int main()
{
int fd, fp ;
int bytesread;
char * myfifo = "myfifo";
char buf[MAX_BUF];
int count = 0;
char *filename = "memorizza.txt";
fp = open(filename,O_WRONLY | O_APPEND);
if ((fd = open(myfifo, O_RDONLY))==-1 ){ // opening pipe
perror(myfifo);
return 1;
}
while(1)
{
if((bytesread = read( fd, buf, MAX_BUF - 1)) > 0) //read pipe
{
buf[bytesread] = '\0';
count++;
write(fp , buf , MAX_BUF );
}
else {
printf("Ho ricevuto %d numeri primi \n" , count);
break;
}
}
close(fd);
close(fp);
return 0;
}
To read a binary encoded int from a file descriptor (assuming host byte order), you have to pass a a pointer to int to read, and read exactly sizeof(int) bytes. A naive implementation would look like this:
int i;
ssize_t e = read(fd, &i, sizeof(i));
The problem with this implementation is that read is not guaranteed to give you all the bytes you asked for in one go. So we have to keep reading until we have all the bytes:
int i;
char *buffer = (char *)&i;
size_t left_to_read = sizeof(i);
while (left_to_read)
{
ssize_t e = read(fd, buffer, left_to_read);
if (e < 0 && errno != EINTR)
{
perror("Failure reading from file descriptor");
exit(EXIT_FAILURE);
}
else if (e == 0)
{
/*
Handle EOF
*/
}
else if (e > 0)
{
left_to_read -= e;
buffer += e;
}
}
EDIT:
Naturally after reading the int can be converted to ascii the normal way:
fprintf(outfile, "%d\n", i);
To a first approximation, if you write to the pipe with
write (fp, & mynum, sizeof (mynum));
(quoted from comments), where mynum has type int, then the corresponding read would be
int fp = /* open read end of the pipe */;
int my_read_num;
read(fp, &my_read_num, sizeof(my_read_num));
When that works as intended, it gets you the wanted integer as an int, and you can then emit it without further manipulation via fprintf(), or otherwise do anything with it that you might ordinarily do with an int.
But there are complications, including:
Neither write() nor read() is guaranteed to transfer the full number of bytes requested. Their return values tell you how many bytes they actually did transfer, and robust code needs to account for short writes and reads, generally by transferring any remaining bytes via additional write or read calls, as appropriate.
Calls to write() and read() are not guaranteed to pair up automatically. That is, data written by multiple write()s might be read by a single read(), and vice versa. That's not so much a problem for your particular case, however, (once you handle (1)) because you are transferring units of known size. It normally bites people who try to transfer varying length data, such as lines of text, without establishing a means to also convey message lengths or boundaries.
In the fullest generality, you cannot necessarily rely on the two endpoints to use the same integer representation. For instance, one end might use 32-bit big-endian for int, whereas the other end uses 16-bit little-endian for int. There are ways to account for that, but you do not need to worry about it in your specific case, where both ends are running on the same machine and use the same C data type for the object they transfer.
Related
I need to make a program that takes string data from one file and copy every third char from it to another file.
I am not sure if I am doing it right. The idea I got is to first create one array where I will store original data from file1 and then using 'for' loop I will modify the data and store in in the second array:
(eg for(i=0; i < arraysize; i+=3);
The thing is I dont have an idea how to transfer input to my array and how to do it backwards to have my modified data go to file2.
#include <stdio.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
#define BUFFER_SIZE 50
int main(int argc, char *argv[]) {
char buffer[BUFFER_SIZE];
char modified[BUFFER_SIZE];
int input_fd, output_fd;
ssize_t ret_in, ret_out;
if(argc !=3 || strcmp(argv[1], "--help") == 0)
{
printf("Usage: %s file_origin file_destination\n", argv[0]);
return 2;
}
input_fd = open(argv[1], O_RDONLY);
if(input_fd == -1)
{
perror("There is no such file");
return 2;
}
out_fd = open(argv[2], O_RDWR | O_CREAT | O_TRUNC, 0644);
if(output_fd == -1)
{
perror("create");
return 3;
}
Could someone please tell me how to use function read/write correctly to stream my data to array and how to do it the other way.
Welcome to Stackoverflow!
Given the exact description of your assignment, I would not use a buffer; you could simply read from the input file one byte at a time, and write every third byte to the output file. This avoids any buffer-management overhead.
But, if you do read from the input file into a buffer, you do not need to modify that buffer in any way, nor do you need a second buffer. After reading all the data, simply iterate through the input buffer, outputting every third byte to the output file.
But, if you want/need to reuse the output in some way, you can simply populate a second buffer from the input buffer in the same manner (loop over the input buffer, skipping two bytes each iteration), and then write that second buffer to the output file. (This way, you still have the same output in that second buffer, and you can reuse it in some manner.)
The approach you take will dictate the best functions to use. I see you already know about open(). Read up on read(), write() and close(), but also read up on fopen(), fgetc(), fgets(), fread(), fwrite() and fclose(). There is a lot for you to learn from reading about these various functions, how they are similar to each other, how they differ from each other, and the pros and cons of each. Reading about them will lead you to learn about other related file operations (like seeking, rewinding, etc.), which will serve you well as you learn more about C and programming in general.
Please note that for the approaches using buffers, you need to be very careful about the size of your buffers vs. the size of the input file. There are many pitfalls here. If this is an assignment for a class of some sort, then those considerations might show up in later lessons / assignments, and maybe it's too much to take on just now. But it's never too early to start thinking about what you do and don't know about the input your program will need to handle.
If you do not need cin or cout, I would suggest the following (I assumed strings are ended with newline and those should be preserved in the output and that counting the 3rd character starts anew in every line read):
FILE *f1=fopen("_infile.txt","rt");
FILE *f2=fopen("_outfuile.txt","wt");
char buffer[MAXBUFLEN];
while (!feof(f1)) {
if (fgets(buffer,MAXBUFLEN,f1)>0) {
for(int i=2;i<strlen(buffer);i+=3) {
fprintf(f2,"%c",buffer[i]);
}
fprintf(f2,"\n");
} else break;
}
fclose(f1);
fclose(f2);
This will read input file and reprint every third character to output. You can adapt it to you situation.
#include <stdio.h>
#include <stdlib.h>
int main()
{
size_t i; // index
int c; // char read
FILE *FIN, *FOUT; // file streams
if ((FIN = fopen("in.txt", "rb")) == NULL) {
printf("Error opening input file.\n Exiting.\n");
exit(1);
}
if ((FOUT = fopen("out.txt", "wb")) == NULL) {
printf("Error opening output file.\n Exiting.\n");
exit(1);
}
// read input and reprint every third character
for(i=0;;i++)
{
c = fgetc(FIN); // read byte
if(c == EOF)
{
break; // reached end of file (input), leave loop
}
if((i%3)==2) // get every third character by modulo(i)
{
fputc(c, FOUT); // write output
}
}
fclose(FIN);
fclose(FOUT);
return 0;
}
I'm working on linux, I have a file that contains a line like this:
328abc
I would like, in C, to read the integer part (328) and the characters 'a','b','c', using only the function:
ssize_t read (int filedes, void *buffer, size_t size))
This is the only thing the file contains.
I know there are better ways to do that with other functions, but I haven't coded in C for a long time, and trying to help a friend, only this function is alowed.
How do I play with the buffer to do that?
Thanks
edit:
I understand that I need to parse the buffer manually. and my question is how?
If that's the only thing in the file. This will do:
#include <stdio.h>
#include <stdlib.h>
#include <fcntl.h>
#include <unistd.h>
#include <string.h>
int main() {
char buffer[6];
char intBuffer[4];
ssize_t bytesRead;
int number;
int fd;
if ((fd = open("file.txt", O_RDONLY)) == -1) {
perror("Error opening file");
exit(EXIT_FAILURE);
}
if ((bytesRead = read(fd, buffer, 6)) == -1) {
perror("Error reading file");
exit(EXIT_FAILURE);
}
memcpy(intBuffer, buffer, 3);
intBuffer[3] = '\0';
number = atoi(intBuffer);
printf("The number is %d\n", number);
exit(EXIT_SUCCESS);
}
The following code will print "The number is 328".
Is this some kind of homework?
I am asking because there are better ways to do that than using the read function.
Anyway to answer your question, read reads size bytes from the file whose file descriptor is filedes and places them to the buffer.
It does not know anything about line breaks etc. So you need to manually find where a line ends, etc. If you want to only use read, then you need to manually parse the buffer after each call to read (supposing your files contains many lines, that you want to parse).
Beware that a line may be split between two read calls, so you need to handle that case with caution.
I want to take all characters past location 900 from a file called WWW, and put all of these in an array:
//Keep track of all characters past position 900 in WWW.
int Seek900InWWW = lseek(WWW, 900, 0); //goes to position 900 in WWW
printf("%d \n", Seek900InWWW);
if(Seek900InWWW < 0)
printf("Error seeking to position 900 in WWW.txt");
char EverythingPast900[appropriatesize];
int NextRead;
char NextChar[1];
int i = 0;
while((NextRead = read(WWW, NextChar, sizeof(NextChar))) > 0) {
EverythingPast900[i] = NextChar[0];
printf("%c \n", NextChar[0]);
i++;
}
I try to create a char array of length 1, since the read system call requires a pointer, I cannot use a regular char. The above code does not work. In fact, it does not print any characters to the terminal as expected by the loop. I think my logic is correct, but perhaps a misunderstanding of whats going on behind the scenes is what is making this hard for me. Or maybe i missed something simple (hope not).
If you already know how many bytes to read (e.g. in appropriatesize) then just read in that many bytes at once, rather than reading in bytes one at a time.
char everythingPast900[appropriatesize];
ssize_t bytesRead = read(WWW, everythingPast900, sizeof everythingPast900);
if (bytesRead > 0 && bytesRead != appropriatesize)
{
// only everythingPast900[0] to everythingPast900[bytesRead - 1] is valid
}
I made a test version of your code and added bits you left out. Why did you leave them out?
I also made a file named www.txt that has a hundred lines of "This is a test line." in it.
And I found a potential problem, depending on how big your appropriatesize value is and how big the file is. If you write past the end of EverythingPast900 it is possible for you to kill your program and crash it before you ever produce any output to display. That might happen on Windows where stdout may not be line buffered depending on which libraries you used.
See the MSDN setvbuf page, in particular "For some systems, this provides line buffering. However, for Win32, the behavior is the same as _IOFBF - Full Buffering."
This seems to work:
#include <fcntl.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <unistd.h>
#include <stdio.h>
int main()
{
int WWW = open("www.txt", O_RDONLY);
if(WWW < 0)
printf("Error opening www.txt\n");
//Keep track of all characters past position 900 in WWW.
int Seek900InWWW = lseek(WWW, 900, 0); //goes to position 900 in WWW
printf("%d \n", Seek900InWWW);
if(Seek900InWWW < 0)
printf("Error seeking to position 900 in WWW.txt");
int appropriatesize = 1000;
char EverythingPast900[appropriatesize];
int NextRead;
char NextChar[1];
int i = 0;
while(i < appropriatesize && (NextRead = read(WWW, NextChar, sizeof(NextChar))) > 0) {
EverythingPast900[i] = NextChar[0];
printf("%c \n", NextChar[0]);
i++;
}
return 0;
}
As stated in another answer, read more than one byte. The theory behind "buffers" is to reduce the amount of read/write operations due to how slow disk I/O (or network I/O) is compared to memory speed and CPU speed. Look at it as if it is code and consider which is faster: adding 1 to the file size N times and writing N bytes individually, or adding N to the file size once and writing N bytes at once?
Another thing worth mentioning is the fact that read may read fewer than the number of bytes you requested, even if there is more to read. The answer written by #dreamlax illustrates this fact. If you want, you can use a loop to read as many bytes as possible, filling the buffer. Note that I used a function, but you can do the same thing in your main code:
#include <sys/types.h>
/* Read from a file descriptor, filling the buffer with the requested
* number of bytes. If the end-of-file is encountered, the number of
* bytes returned may be less than the requested number of bytes.
* On error, -1 is returned. See read(2) or read(3) for possible
* values of errno.
* Otherwise, the number of bytes read is returned.
*/
ssize_t
read_fill (int fd, char *readbuf, ssize_t nrequested)
{
ssize_t nread, nsum = 0;
while (nrequested > 0
&& (nread = read (fd, readbuf, nrequested)) > 0)
{
nsum += nread;
nrequested -= nread;
readbuf += nread;
}
return nsum;
}
Note that the buffer is not null-terminated as not all data is necessarily text. You can pass buffer_size - 1 as the requested number of bytes and use the return value to add a null terminator where necessary. This is useful primarily when interacting with functions that will expect a null-terminated string:
char readbuf[4096];
ssize_t n;
int fd;
fd = open ("WWW", O_RDONLY);
if (fd == -1)
{
perror ("unable to open WWW");
exit (1);
}
n = lseek (fd, 900, SEEK_SET);
if (n == -1)
{
fprintf (stderr,
"warning: seek operation failed: %s\n"
" reading 900 bytes instead\n",
strerror (errno));
n = read_fill (fd, readbuf, 900);
if (n < 900)
{
fprintf (stderr, "error: fewer than 900 bytes in file\n");
close (fd);
exit (1);
}
}
/* Read a file, printing its contents to the screen.
*
* Caveat:
* Not safe for UTF-8 or other variable-width/multibyte
* encodings since required bytes may get cut off.
*/
while ((n = read_fill (fd, readbuf, (ssize_t) sizeof readbuf - 1)) > 0)
{
readbuf[n] = 0;
printf ("Read\n****\n%s\n****\n", readbuf);
}
if (n == -1)
{
close (fd);
perror ("error reading from WWW");
exit (1);
}
close (fd);
I could also have avoided the null termination operation and filled all 4096 bytes of the buffer, electing to use the precision part of the format specifiers of printf in this case, changing the format specification from %s to %.4096s. However, this may not be feasible with unusually large buffers (perhaps allocated by malloc to avoid stack overflow) because the buffer size may not be representable with the int type.
Also, you can use a regular char just fine:
char c;
nread = read (fd, &c, 1);
Apparently you didn't know that the unary & operator gets the address of whatever variable is its operand, creating a value of type pointer-to-{typeof var}? Either way, it takes up the same amount of memory, but reading 1 byte at a time is something that normally isn't done as I've explained.
Mixing declarations and code is a no no. Also, no, that is not a valid declaration. C should complain about it along the lines of it being variably defined.
What you want is dynamically allocating the memory for your char buffer[]. You'll have to use pointers.
http://www.ontko.com/pub/rayo/cs35/pointers.html
Then read this one.
http://www.cprogramming.com/tutorial/c/lesson6.html
Then research a function called memcpy().
Enjoy.
Read through that guide, then you should be able to solve your problem in an entirely different way.
Psuedo code.
declare a buffer of char(pointer related)
allocate memory for said buffer(dynamic memory related)
Find location of where you want to start at
point to it(pointer related)
Figure out how much you want to store(technically a part of allocating memory^^^)
Use memcpy() to store what you want in the buffer
I see that the code below uses memcpy which i can use to exploit this program and cause a buffer overflow, but i cant seem to make it crash. No matter what character argument i pass to it i just get "error opening packet file." Any ideas how?
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <sys/stat.h>
#include <fcntl.h>
#define MAX_ADDR_LEN 128
#define ADDR_LENGTH_OFFSET 4
#define ADDR_OFFSET 8
typedef unsigned char shsize_t;
typedef struct{
char addr[MAX_ADDR_LEN];
shsize_t len;
} arp_addr;
void
print_address(char *packet)
{
arp_addr hwaddr;
int i;
hwaddr.len = (shsize_t) *(packet + ADDR_LENGTH_OFFSET);
memcpy(hwaddr.addr, packet + ADDR_OFFSET, hwaddr.len);
printf("Sender hardware address: ");
for (i = 0; i < hwaddr.len - 1; i ++)
printf("%02hhx::", hwaddr.addr[i]);
printf("%02hhx\n", hwaddr.addr[hwaddr.len - 1]);
return;
}
int main(int argc, char *argv[])
{
struct stat sbuf;
char *packet;
int fd;
if (argc != 2){
printf("Usage: %s <packet file>\n", argv[0]);
return EXIT_FAILURE;
}
if ((stat(argv[1], &sbuf)) < 0){
printf("Error opening packet file\n");
return EXIT_FAILURE;
}
if ((fd = open(argv[1], O_RDONLY)) < 0){
printf("Error opening packet file\n");
return EXIT_FAILURE;
}
if ((packet = (char *)malloc(sbuf.st_size * sizeof(char))) == NULL){
printf("Error allocating memory\n");
return EXIT_FAILURE;
}
if (read(fd, packet, sbuf.st_size) < 0){
printf("Error reading packet from file\n");
return EXIT_FAILURE;
}
close(fd);
print_address(packet);
free(packet);
return EXIT_SUCCESS;
}
When you do something like write past the end of a buffer there is no guarantee that the program will crash. This is called undefined behavior, which literally means that you can make no reasonable assumptions as to what will happen.
The program itself appears relatively well behaved. As long as len is calculated properly I don't see any way for you to cause an overrun via input. Just because a program uses memcpy doesn't mean that it is vulnerable to attack. The only attack vector I see is if you pass it a carefully crafted file such that the length is calculated incorrectly:
hwaddr.len = (shsize_t) *(packet + ADDR_LENGTH_OFFSET)
In this line the program reads ADDR_LENGTH_OFFSET bytes from the address of packet to get the data length. Obviously that is problematic if you craft a file with an erroneous value for the data length in the header (i.e., a data length > MAX_ADDR_LEN).
BTW, the argument is a file, not a character. You won't be able to do anything passing it nonsense input because read will fail.
No matter what character argument i pass to it i just get "error
opening packet file."
You need to pass a valid file name as an argument, not random characters.
As others have indicated, the memcpy() isn't the security problem. The problem is that the length parameter passed to memcpy() comes from user input (the file you specified). If you specify a file that has a length field of, say, a billion, you will probably see a crash (and, yes, 'crash' is accepted vernacular).
Since there is rather limited checking on the size of the packet, you can pass it the name of an empty or very short file and the print_address() code will mess around out of bounds.
Also, since the code reads a length from the data read from the file, you can place an arbitrary number at relevant position and make the code go running around most places in memory.
I'm trying to build an instruction pipeline simulator and I'm having a lot of trouble getting started. What I need to do is read binary from stdin, and then store it in memory somehow while I manipulate the data. I need to read in chunks of exactly 32 bits one after the other.
How do I read in chunks of exactly 32 bits at a time? Secondly, how do I store it for manipulation later?
Here's what I've got so far, but examining the binary chunks I read further, it just doesn't look right, I don't think I'm reading exactly 32 bits like I need.
char buffer[4] = { 0 }; // initialize to 0
unsigned long c = 0;
int bytesize = 4; // read in 32 bits
while (fgets(buffer, bytesize, stdin)) {
memcpy(&c, buffer, bytesize); // copy the data to a more usable structure for bit manipulation later
// more stuff
buffer[0] = 0; buffer[1] = 0; buffer[2] = 0; buffer[3] = 0; // set to zero before next loop
}
fclose(stdin);
How do I read in 32 bits at a time (they are all 1/0, no newlines etc), and what do I store it in, is char[] okay?
EDIT: I'm able to read the binary in but none of the answers produce the bits in the correct order — they are all mangled up, I suspect endianness and problems reading and moving 8 bits around ( 1 char) at a time — this needs to work on Windows and C ... ?
What you need is freopen(). From the manpage:
If filename is a null pointer, the freopen() function shall attempt to change the mode of the stream to that specified by mode, as if the name of the file currently associated with the stream had been used. In this case, the file descriptor associated with the stream need not be closed if the call to freopen() succeeds. It is implementation-defined which changes of mode are permitted (if any), and under what circumstances.
Basically, the best you can really do is this:
freopen(NULL, "rb", stdin);
This will reopen stdin to be the same input stream, but in binary mode. In the normal mode, reading from stdin on Windows will convert \r\n (Windows newline) to the single character ASCII 10. Using the "rb" mode disables this conversion so that you can properly read in binary data.
freopen() returns a filehandle, but it's the previous value (before we put it in binary mode), so don't use it for anything. After that, use fread() as has been mentioned.
As to your concerns, however, you may not be reading in "32 bits" but if you use fread() you will be reading in 4 chars (which is the best you can do in C - char is guaranteed to be at least 8 bits but some historical and embedded platforms have 16 bit chars (some even have 18 or worse)). If you use fgets() you will never read in 4 bytes. You will read in at least 3 (depending on whether any of them are newlines), and the 4th byte will be '\0' because C strings are nul-terminated and fgets() nul-terminates what it reads (like a good function). Obviously, this is not what you want, so you should use fread().
Consider using SET_BINARY_MODE macro and setmode:
#ifdef _WIN32
# include <io.h>
# include <fcntl.h>
# define SET_BINARY_MODE(handle) setmode(handle, O_BINARY)
#else
# define SET_BINARY_MODE(handle) ((void)0)
#endif
More details about SET_BINARY_MODE macro here: "Handling binary files via standard I/O"
More details about setmode here: "_setmode"
I had to piece the answer together from the various comments from the kind people above, so here is a fully-working sample that works - only for Windows, but you can probably translate the windows-specific stuff to your platform.
#include "stdafx.h"
#include "stdio.h"
#include "stdlib.h"
#include "windows.h"
#include <io.h>
#include <fcntl.h>
int main()
{
char rbuf[4096];
char *deffile = "c:\\temp\\outvideo.bin";
size_t r;
char *outfilename = deffile;
FILE *newin;
freopen(NULL, "rb", stdin);
_setmode(_fileno(stdin), _O_BINARY);
FILE *f = fopen(outfilename, "w+b");
if (f == NULL)
{
printf("unable to open %s\n", outfilename);
exit(1);
}
for (;; )
{
r = fread(rbuf, 1, sizeof(rbuf), stdin);
if (r > 0)
{
size_t w;
for (size_t nleft = r; nleft > 0; )
{
w = fwrite(rbuf, 1, nleft, f);
if (w == 0)
{
printf("error: unable to write %d bytes to %s\n", nleft, outfilename);
exit(1);
}
nleft -= w;
fflush(f);
}
}
else
{
Sleep(10); // wait for more input, but not in a tight loop
}
}
return 0;
}
For Windows, this Microsoft _setmode example specifically shows how to change stdin to binary mode:
// crt_setmode.c
// This program uses _setmode to change
// stdin from text mode to binary mode.
#include <stdio.h>
#include <fcntl.h>
#include <io.h>
int main( void )
{
int result;
// Set "stdin" to have binary mode:
result = _setmode( _fileno( stdin ), _O_BINARY );
if( result == -1 )
perror( "Cannot set mode" );
else
printf( "'stdin' successfully changed to binary mode\n" );
}
fgets() is all wrong here. It's aimed at human-readable ASCII text terminated by end-of-line characters, not binary data, and won't get you what you need.
I recently did exactly what you want using the read() call. Unless your program has explicitly closed stdin, for the first argument (the file descriptor), you can use a constant value of 0 for stdin. Or, if you're on a POSIX system (Linux, Mac OS X, or some other modern variant of Unix), you can use STDIN_FILENO.
fread() suits best for reading binary data.
Yes, char array is OK, if you are planning to process them bytewise.
I don't know what OS you are running, but you typically cannot "open stdin in binary". You can try things like
int fd = fdreopen (fileno (stdin), outfname, O_RDONLY | OPEN_O_BINARY);
to try to force it. Then use
uint32_t opcode;
read(fd, &opcode, sizeof (opcode));
But I have no actually tried it myself. :)
I had it right the first time, except, I needed ntohl ... C Endian Conversion : bit by bit