I've got a little problem while experimenting with some C code. I've tried to use read()-Command to read a text out of a file and store the results in a charArray. But when I print the results they're always different from the file.
Here is the code:
#include <stdio.h>
#include <fcntl.h>
#include <unistd.h>
void main() {
int fd = open("file", 2);
char buf[2];
printf("Read elements: %ld\n", read(fd, buf, 2));
printf("%s\n", buf);
close(fd);
}
The file "file" was created in the same directory using the following UNIX commands:
cat > file
Hi
So it contains just the word "Hi". When I run it, I expect it to read 2 bytes from the file (which are 'H' and 'i') and store them at buf[0] and buf[1]. But when I want to print the result, it appears, that there was an issue, because besides the word "Hi" there are several wierd characters printed (indicating a memory reading/writing problem i guess, due to bad buffer size). I've tried to increase the size of the buf-Array and it appears that when i change the size, the wierd characters printed change. The problem is removed when size reaches 32 bytes.
Can someone explain to me in detail why this is happening?
I've understood so far that read() does not read \'0' when it reads something, and that the third parameter of read() indicates the maximum number of bytes to read.
Antoher thing I've noticed while experimenting with the above code is the following: Let's assume one changes the third parameter (maximum bytes to read) of read() to 3, and the size of buf-Array to 512 (overkill i know, but I really wanted to see what will happen). Now read will acutally read a third character (in my case 'e') and store it into the buffer, even tho this third character does not exist.
I've searched for a while now #stackoverflow and I found many similiar cases, but none of them made me understand my problem. If there is any other thread i missed, it would be a pleasure if u could link me to it.
At last: sry for my bad english, it's not my native language.
Clearly you need to make buf 3 bytes long and use the last byte as the null byte (0 or '\0'). That way, when you print the string, your computer doesn't carry on until he finds another 0 !
The way strings (char arrays really) are handled in C is quite straightforward. Indeed, when dealing with strings (most) if not all functions take under the assumption that string parameters are null terminated (puts) or return null terminated strings (strdup).
The point is that, by default the computer can't tell where a string ends unless it is given the strings size each time he processes it. The easiest implementation around this approach was to append after each string a 0 (namely the null byte). That way, the computer just need to iterate over the string's characters and stop when he finds the termination character (other name for null byte).
Related
The Scoop:
I am creating a method that runs through a lengthy file in chunks: using pthreads. I am calling fread() to read the file in this sort of fashion:
fread( thread_data[i].buffer, 1, 50, f )
/*
thread_data is a data structure for each thread (hence i)
buffer is in thread_data as an array of length 50
*/
I am then directly calling a print statement to see what each thread is doing, as a weird pattern was showing up in some of the parts that I was printing. Namely, my print statement would look something like this:
this is suppose to be 50 characters, but it is only a fewgD4
That D4 directly above is what I have my question on. Every thread that I make, at the end of the string, we are printing D4, and in this case, followed by a g. Other times, it is followed by a d, and most commonly a �. Now, I did read the wikipedia page on this character, which states:
replacement character used to replace an unknown or unrepresentable character
My question:
What kind of an error am I running into? Why is the end of each read statement containing unknown characters, especially the weird gD4 guy?
Aside:
I am trying to make a function in c that utilizes pthreads to find the frequency of each word in a file, in case anyone was wondering. These weird characters were showing up in my list, which is something that I find slightly unpleasent. Finally, don't bother linking me to the Obligaroty Unicode article, I am already aware of it, and the characters are not outside of what I am working with.
The strings you are printing out are not null-terminated — fread() does not null-terminate its output, it simply reads in as many raw bytes as you asked for (or fewer). So when you print out your buffer, your print function is walking past the end of the data and printing out whatever garbage memory comes after the buffer, which in your case just happens to be gD4.
You need to either explicitly null-terminate your buffer; or, if your print function supports it, tell it exactly how many characters to print. Either way, you need to save the return value from fread to know how many characters you read. For example:
int n = fread(thread_data[i].buffer, 1, 50, f);
if (n < 0) /* Handle error */ ;
// Explicitly add a null terminator -- make sure the buffer has room for it!
thread_data[i].buffer[n] = 0;
I encrypted a text file using an offset cipher in C. For this, I simply added 128 to each character and got the file size decreased by 3 bytes. I tried the same on some other files too just to get the same result, i.e. decrease in file size by 3 bytes. I got the original size after decryption.
Could you please tell me why does it so happen?
Code for the main logic is given below:
while((ch=fgetc(fs))!=EOF){
fputc(ch+128, ft);
Could you please tell me why does it so happen?
Your ch probably has the wrong declaration. The fputc() function returns an int, not a char, and if you cast to char you will lose the distinction between (char) 0xff and EOF.
// WRONG WRONG WRONG
// char ch = fgetc(fs);
The right declaration:
int ch = fgetc(fs);
Otherwise, it shouldn't happen. Is your process exiting cleanly? If you abort(), then there might be data still in FILE * buffers. Show more code. Run with Valgrind. Check the exit status of your process.
I think the file size should have doubled as two bytes were taken for one character after encryption as something greater than 127 can not be stored in 1 byte.
No, fputc() does not work that way. The fputc() man page (run man fputc in a terminal, unless on Windows):
fputc() writes the character c, cast to an unsigned char, to stream.
Conversion to unsigned char is done by taking the value modulo 256*. So fputc() always writes exactly one byte of data (unlesss it fails).
* This is true all but exceedingly rare systems.
If you talk about Windows, I could imagine that you have opened the file in text mode, not in binary mode.
That leads to the following:
Writing \n leads to a \r\n written to the file.
Reading \r\n from the file gives only \n to the user.
Reading stops at the first \x1A, being a EOF character.
If you add 128 to each byte, the data-to-be-written rolls over at 256. While it may be undefined behaviour to call fputc() with a value > 256 (you should write (ch+128)%256 or (ch+128) & 0xFF), on your systems it obviously writes the value wrapped by 256 and thus you may get \n or \x1A by accident.
Consider the following piece of code for reading the contents of the file into a buffer
#include <stdio.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <unistd.h>
#define BLOCK_SIZE 4096
int main()
{
int fd=-1;
ssize_t bytes_read=-1;
int i=0;
char buff[50];
//Arbitary size for the buffer?? How to optimise.
//Dynamic allocation is a choice but what is the
//right way to relate the file size to bufffer size.
fd=open("./file-to-buff.txt",O_RDONLY);
if(-1 == fd)
{
perror("Open Failed");
return 1;
}
while((bytes_read=read(fd,buff,BLOCK_SIZE))>0)
{
printf("bytes_read=%d\n",bytes_read);
}
//Test to characters read from the file to buffer.The file contains "Hello"
while(buff[i]!='\0')
{
printf("buff[%d]=%d\n",i,buff[i]);
i++;
//buff[5]=\n-How?
}
//buff[6]=`\0`-How?
close(fd);
return 0;
}
Code Description:
The input file contains a string "Hello"
This content needs to be copied into the buffer.
The objective is acheived by open and read POSIX API's.
The read API uses a pointer to a buffer of an*arbitary size* to copy the data in.
Questions:
Dynamic allocation is the method that must be used to optimize the size of the buffer.What is the right procedure to relate/derive the buffer size from the input file size?
I see at the end of the read operation the read has copied a new line character and a NULL character in addition to the characters "Hello". Please elaborate more on this behavior of read.
Sample Output
bytes_read=6
buff[0]=H
buff[1]=e
buff[2]=l
buff[3]=l
buff[4]=o
buff[5]=
PS: Input file is user created file not created by a program (using write API). Just to mention here, in case if it makes any difference.
Since you want to read the whole file, the best way is to make the buffer as big as the file size. There's no point in resizing the buffer as you go. That just hurts performance without good reason.
You can get the file size in several ways. The quick-and-dirty way is to lseek() to the end of the file:
// Get size.
off_t size = lseek(fd, 0, SEEK_END); // You should check for an error return in real code
// Seek back to the beginning.
lseek(fd, 0, SEEK_SET);
// Allocate enough to hold the whole contents plus a '\0' char.
char *buff = malloc(size + 1);
The other way is to get the information using fstat():
struct stat fileStat;
fstat(fd, &fileStat); // Don't forget to check for an error return in real code
// Allocate enough to hold the whole contents plus a '\0' char.
char *buff = malloc(fileStat.st_size + 1);
To get all the needed types and function prototypes, make sure you include the needed header:
#include <sys/stat.h> // For fstat()
#include <unistd.h> // For lseek()
Note that read() does not automatically terminate the data with \0. You need to do that manually, which is why we allocate an extra character (size+1) for the buffer. The reason why there's already a \0 character there in your case is pure random chance.
Of course, since buf is now a dynamically allocated array, don't forget to free it again when you don't need it anymore:
free(buff);
Be aware though, that allocating a buffer that's as large as the file you want to read into it can be dangerous. Imagine if (by mistake or on purpose, doesn't matter) the file is several GB big. For cases like this, it's good to have a maximum allowable size in place. If you don't want any such limitations, however, then you should switch to another method of reading from files: mmap(). With mmap(), you can map parts of a file to memory. That way, it doesn't matter how big the file is, since you can work only on parts of it at a time, keeping memory usage under control.
1, you can get the file size with stat(filename, &stat), but define the buffer to page size is just fine
2, first, there is no NULL character after "Hello", it must be accident that the stack area you allocated was 0 before your code executed, please refer to APUE chapter 7.6. In fact you must initialize the local variable before using it.
I tried to generate the text file with vim, emacs and echo -n Hello > file-to-buff.txt, only vim adds a line break automatically
You could consider allocating the buffer dynamically by first creating a buffer of a fixed size using malloc and doubling (with realloc) the size when you fill it up. This would have a good time complexity and space trade off.
At the moment you repeatedly read into the same buffer. You should increase the point in the buffer after each read otherwise you will overwrite the buffer contents with the next section of the file.
The code you supply allocates 50 bytes for the buffer yet you pass 4096 as the size to the read. This could result in a buffer overflow for any files over the size of 50 bytes.
As for the `\n' and '\0'. The newline is probably in the file and the '\0' was just already in the buffer. The buffer is allocated on the stack in your code and if that section of the stack had not been used yet it would probably contain zeros, placed there by the operating system when your program was loaded.
The operating system makes no attempt to terminate the data read from the file, it might be binary data or in a character set that it doesn't understand. Terminating the string, if needed, is up to you.
A few other points that are more a matter of style:
You could consider using a for (i = 0; buff[i]; ++i) loop instead of a while for the printing out at the end. This way if anyone messes with the index variable i you will be unaffected.
You could close the file earlier, after you finish reading from it, to avoid having the file open for an extended period of time (and maybe forgetting to close it if some kind of error happens).
For your second question, read don't add automatically a character '\0'.
If you consider that your file is a textual file, your must add a '\0' after calling read, for indicate the end of string.
In C, the end of string is represented by this caracter. If read set 4 characters, printf will read these 4 characters, and will test the 5th: if it's not '\0', it will continue to print until next '\0'.
It's also a source of buffer overflow
For the '\n', it is probably in the input file.
I have the code below:
#include <stdio.h>
#include <unistd.h>
int main () {
int fd = open("filename.dat", O_CREAT|O_WRONLY|O_TRUNC, 0600);
int result = write(fd, "abcdefghijklmnopqrstuvxz", 100);
printf("\n\nfd = %d, result = %d, errno = %d", fd, result, errno);
close(fd);
return 0;
}
I am trying to understand what happens when I try to write more bytes to a file than I have available. So I am calling write and asking the program to write 100 bytes while I have much less than that. The result: a bunch of stuff from stdout ends up on filename.dat. If instead of 100 I use strlen("abcdefghijklmnopqrstuvxz"), I get the desired result. My question then is: why is the program trying to write beyond the '\0' character on my string? Is there some undefined behavior going on here?
My question then is: why is the program trying to write beyond the
'\0' character on my string?
The function write(2) doesn't care about 0-terminators. It actually doesn't care about buffer contents at all: it will try to write as many bytes as you tell it.
Is there some undefined behavior going on here
Of course, trying to write more than you have might incur the wrath of the OS who could decide to terminate your process if it touches inaccessible memory.
The write() function you are using does not care about the content. It just writes the no. of bytes you tell it to write in the file.
So when you say it to write 100 bytes and provide less than 100 bytes. The remaining bytes are taken as garbage value.
But when you are using strlen("abcdefghijklmnopqrstuvxz"), you are asking the write() to write bytes equal to the length of the string. So it works fine there
Because there are two techniques to represent a string. There is the null-terminated version, and there is another when you define its size and the pointer to the first byte. Write uses the second one. It needs a pointer where your data begins and a length to know how much data should copy to the file, but it doesn't see the null values. Sometimes these methods wraps a simple memcpy.
So when you defined the 100 length, in the memory after your abcdefghijklmnopqrstuvxz the program stored your "bunch of stdout stuff". That's why you see garbage. You were lucky because you can get SEGFAULT easily in these cases!
My question then is: why is the program trying to write beyond a \0 Because you want it to write 100 chars.
Is there some undefined behavior going on here? If you increase that 100 to a large number and if that area is on a non-privilage area, it is undefined behaviour.
I think that the basic issue here is that you're thinking of C strings as values, you think you're passing this value to the write function, and the write function is writing out your value plus extra junk.
C is lower level than that. In C, we don't really pass strings around, instead we pass pointers to strings, which are 'char *' values but with the added promise that they point to a valid block of memory that should be treated as a null-terminated string.
The write() function doesn't care about the null-terminated string convention. The parameters in the write call provide a file descriptor, a char *, and a buffer length.
Also, the compiler converts string constants into const char arrays. The equivalent of this happens at the top level:
const char *stringconst00001[27] = { 'a', 'b', 'c', ... 'y', 'z', '\0' }
And it does this in main():
int result = write(fd, stringconst00001, 100);
This code comes from K&R. I have read it several times, but it still seems to escape my grasp.
#define BUFSIZE 100
char buf[BUFSIZE];
int bufp = 0;
int getch(void)
{
return(bufp>0)?buf[--bufp]:getchar();
}
int ungetch(int c)
{
if(bufp>=BUFSIZE)
printf("too many characters");
else buf[bufp++]=c;
}
The purpose of these two functions, so K&R says, is to prevent a program from reading too much input. i.e. without this code a function might not be able to determine it has read enough data without first reading too much. But I don't understand how it works.
For example, consider getch().
As far as I can see this is the steps it takes:
check if bufp is greater than 0.
if so then return the char value of buf[--bufp].
else return getchar().
I would like to ask a more specific question, but I literally dont know how this code achieves what it is intended to achieve, so my question is: What is (a) the purpose and (b) the reasoning of this code?
Thanks in advance.
NOTE: For any K&R fans, this code can be found on page 79 (depending on your edition, I suppose)
(a) The purpose of this code is to be able to read a character and then "un-read" it if it turns out you accidentally read a character too many (with a max. of 100 characters to be "un-read"). This is useful in parsers with lookahead.
(b) getch reads from buf if it has contents, indicated by bufp>0. If buf is empty, it calls getchar. Note that it uses buf as a stack: it reads it from right-to-left.
ungetch pushes a character onto the stack buf after doing a check to see if the stack isn't full.
The code is not really for "reading too much input", instead is it so you can put back characters already read.
For example, you read one character with getch, see if it is a letter, put it back with ungetch and read all letters in a loop. This is a way of predicting what the next character will be.
This block of code is intended for use by programs that make decisions based on what they read from the stream. Sometimes such programs need to look at a few character from the stream without actually consuming the input. For example, if your input looks like abcde12xy789 and you must split it into abcde, 12, xy, 789 (i.e. separate groups of consecutive letters from groups of consecutive digits) you do not know that you have reached the end of a group of letters until you see a digit. However, you do not want to consume that digit at the time you see it: all you need is to know that the group of letters is ending; you need a way to "put back" that digit. An ungetch comes in handy in this situation: once you see a digit after a group of letters, you put the digit back by calling ungetch. Your next iteration will pick that digit back up through the same getch mechanism, sparing you the need to preserve the character that you read but did not consume.
1. The other idea also shown here can be also called as a very primitive I/O stack mangement system and gives the implementation of the function getch() and ungetch().
2. To go a step further , suppose you want to design an Operating System , how can you handle the memory which stores all the keystrokes?
This is solved by the above code snippet.An extension of this concept is used in file handling , especially in editing files .In that case instead of using getchar() which is used to take input from Standard input , a file is used as a source of input.
I have a problem with code given in question. Using buffer (in form of stack) in this code is not correct as when getting more than one extra inputs and pushing into stack will have undesired effect in latter processing (getting input from buffer).
This is because when latter processing (getting input) going on ,this buffer (stack) will give extra input in reverse order (means last extra input given first).
Because of LIFO (Last in first out ) property of stack , the buffer in this code must be quene as it will work better in case of more than one extra input.
This mistake in code confused me and finally this buffer must be quene as shown below.
#define BUFSIZE 100
char buf[BUFSIZE];
int bufr = 0;
int buff = 0;
int getch(void)
{
if (bufr ==BUFSIZE)
bufr=0;
return(bufr>=0)?buf[bufr++]:getchar();
}
int ungetch(int c)
{
if(buff>=BUFSIZE && bufr == 0)
printf("too many characters");
else if(buff ==BUFSIZE)
buff=0;
if(buff<=BUFSIZE)
buf[buff++]=c;
}