TL;DR:
I am asking you to tell me what would be the most efficient approach to double my strings and print them out?
Full story:
I had trouble with the title, and the actual problem may be a bit different than you expect.
Imagine I have a main buffer.
At some index determined by the program, I want to insert a string.
But every char in that string needs to be doubled.
So "abc", inserted at index 10 of buffer[999], needs to be "aabbcc".
Now, the second part of the problem - this needs to be as efficient as possible. I could make this easily, but I need the fastest option.
I thought I had devised several approaches, but it boils down to:
fill buffer(1000) with single chars and double the chars when printing (pushing to stdout)
fill buffer(2000) with double chars and print like normal
The variations to the second approach would be When to double the chars (when copying or generating "aabbcc" from the start and copy the full thing).
The first approach would be the most intuitive, but I fear I would need to devise a low level char-doubling function because putc and printf and any large amount of function calls will have much overhead. (There are allegedly very efficient functions in libc with bitshifting and pointer magic, but I couldn't find them. I can only find the very dissappointing versions where fgets() is just a wrapper for getc() - which can't be efficient.)
The second approach obviously wastes a lot of memory and requires a lot of copying, but it could probably put everything into stdout more efficiently as a chunk without the overhead of copying single chars.
I am unsure if under everything there is just a system write call, and I also lack the knowledge how it works. I am just going with my research that says that fgets is about 12 times faster than fgetc for equal data. And so I assume it is with all the single-char vs line functions.
So in conclusion, I am asking you to tell me what would be the most efficient approach to double my strings and print them out?
Related
I have a text file and I should allocate an array with as many entries as the number of lines in the file. What's more efficient: to read the file twice (first to find out the number of lines) and allocate the array once, or to read the file once, and use "realloc" after each line read? thank you in advance.
Reading the file twice is a bad idea, regardless of efficiency. (It's also almost certainly less efficient.)
If your application insists on reading its input teice, that means its input must be rewindable, which excludes terminal input and pipes. That's a limitation so annoying that apps which really need to read their input more than once (like sort) generally have logic to make a temporary copy if the input is unseekable.
In this case, you are only trying to avoid the trivial overhead of a few extra malloc calls. That's not justification to limit the application's input options.
If that's not convincing enough, imagine what will happen if someone appends to the file between the first time you read it and the second time. If your implementation trusts the count it got on the first read, it will overrun the vector of line pointers on the second read, leading to Undefined Behaviour and a potential security vulnerability.
I presume you want to store the read lines also and not just allocate an array of that many entries.
Also that you don't want to change the lines and then write them back as in that case you might be better off using mmap.
Reading a file twice is always bad, even if it is cached the 2nd time, too many system calls are needed. Also allocing every line separately if a waste of time if you don't need to dealloc them in a random order.
Instead read the entire file at once, into an allocated area.
Find the number of lines by finding line feeds.
Alloc an array
Put the start pointers into the array by finding the same line feeds again.
If you need it as strings, then replace the line feed with \0
This might also be improved upon on modern cpu-architectures, instead of reading the array twice it might be faster simply allocating a "large enough" array for the pointer and scan the array once. This will cause a realloc at the end to have the right size and potentially a couple of times to make the array larger if it wasn't large enough at start.
Why is this faster? because you have a lot of if's that can take a lot of time for each line. So its better to only have to do this once, the cost is the reallocation, but copying large arrays with memcpy can be a bit cheaper.
But you have to measure it, your system settings, buffer sizes etc. will influence things too.
The answer to "What's more efficient/faster/better? ..." is always:
Try each one on the system you're going to use it on, measure your results accurately, and find out.
The term is "benchmarking".
Anything else is a guess.
I want to ask you if is it possible to read the same input (stdin) multiple times? I am about to get really big number, containing thousands of digits (so I am unable to store it in variable, (and also I can not use folders!). My idea is to put the digits into int array, but I don't know how big the array should be, because amount of digits in input may vary. I have to write general solution.
So my question is, how to solve it, and how to find out amount of digits (so I can initialize array), before I copy digits into array. I tried using scanf(), multiple times, or scanf() and getchar, but it is not working. See my code:
int main(){
int c;
int amountOfDigits=5;
while(scanf("%1d",&c)!=' '){//finding out number of digits with scanf
if(isdigit(c)==0){
break;
}
amountOfDigits++;
}
int digits[amountOfDigits];//now i know lenght of array, and initialize it
for(int i=0;i<amountOfDigits;i++){//putting digits into array
digits[i]=getchar();
}
for(int i=0;i<amountOfDigits;i++){//printing array
printf("%d",digits[i]);
}
printf("\n");
return 0;
}
is it possible to read the same input (stdin) multiple times?
(I am guessing you are a student beginning to learn programming, and you are using Linux; adapt my answer if not)
For your homework, you don't need to read the same input several times. In some cases, it is possible (when the standard input is a genuine file -seekable-, that is when you use some redirection in your command). In other cases (e.g. when the standard input is a pipe, e.g. with a command pipeline; or with here documents in your shell command...) it is not possible to read several times stdin (but you don't need to). In general, don't expect stdin to be seekable with fseek or rewind (it usually is not).
(I am not going to do your homework, but here are useful hints)
so I am unable to store it in variable, (and also I can not use folders!)
You could do several things:
(since you mentioned folders....) you might use some more sophisticated ways of storing data on the disk (but in your particular case, I don't recommend that ...). These ways could be some direct-accessed file (ugly), or some indexed file à la gdbm, or some database à la sqlite or even some RDBMS server like PostGreSQL.
In your case, you don't need any of these; I'm mentioning it since you mentioned "folders" and you meant "directories"!
you really should use some heap allocated memory, so read about C dynamic memory allocation and read carefully the documentation of each standard memory management functions like malloc, realloc, free. Your program should probably use all these three functions (don't forget that malloc & realloc could fail).
Read this and that answers. Both are surprisingly relevant.
You probably should keep somehow:
a pointer to heap allocated int-s (actually, you could use char-s)
the allocated size of that pointer
the used length of that thing, that is the actual number of useful digits.
You certainly don't want to grow your array by repeated realloc at each loop (that is inefficient). In practice, you would adapt some growing scheme like newsize = 3*oldsize/2 + 10 to avoid reallocating memory at each step (of your input loop).
you should thank your teacher for a so useful exercise, but you should not expect StackOverflow to do your homework!
Be also aware of arbitrary-precision arithmetic (called bignums or bigints). It is actually hard to code efficiently, so in real-life you would use some library like GMPlib.
Here is my approach:
int linesize=1
int ReadStatus;
char buff[200];
ReadStatus=read(file,buff,linesize)
while(buff[linesize-1]!='\n' && ReadStatus!=0)
{
linesize++;
ReadStatus=read(file,buf,linesize)
}
Is this idea right?
I think my code is a bit inefficient because the run time is O(FileWidth); however I think it can be O(log(FileWidth)) if we exponentially increase linesize to find the linefeed character.
What do you think?
....
I just saw a new problem. How do we read the second line?. Is there anyway to delimit the bytes?
Is this idea right?
No. At the heart of a comment written by Siguza, lies the summary of an issue:
1) read doesn't read lines, it just reads bytes. There's no reason buff should end with \n.
Additionally, there's no reason buff shouldn't contain multiple newline characters, and as there's no [posix] tag here there's no reason to suggest what read does, let alone whether it's a syscall. Assuming you're referring to the POSIX function, there's no error handling. Where's your logic to handle the return value/s reserved for errors?
I think my code is a bit inefficient because the run time is O(FileWidth); however I think it can be O(log(FileWidth)) if we exponentially increase linesize to find the linefeed character.
Providing you fix the issues mentioned above (more on that later), if you were to test this theory, you'd likely find, also at the heart of the comment by Siguza,
Disks usually work on a 512-byte basis and file system caches and even CPU/memory caches are a lot larger than that.
To an extent, you can expect your idea to approach O(log n), but your bottleneck will be one of those cache lines (likely the one closest to your keyboard/the filesystem/whatever is feeding the stream with information). At that point, you should stop guzzling memory which other programs might need because your optimisation becomes less and less effective.
What do you think?
I think you should just STOP! You're guessing!
Once you've written your program, decide whether or not it's too slow. If it's not too slow, it doesn't need optimisation, and you probably won't shave enough nanoseconds to make optimisation worthwhile.
If it is to slow, then you should:
Use a profiler to determine what the most significant bottleneck is,
apply optimisations based on what your profiler tells you, then
use your profiler again, with the same inputs as before, to measure the effect your optimisation had.
If you don't use a profiler, your guess-work could result in slower code, or you might miss opportunities for more significant optimisations...
How do we read the second line?
Naturally, it makes sense to read character by character, rather than two hundred characters at a time, because there's no other way to stop reading the moment you reach a line terminating character.
Is there anyway to delimit the bytes?
Yes. The most sensible tools to use are provided by the C standard, and syscalls are managed automatically to be most efficient based on configurations decided by the standard library devs (who are much likely better at this than you are). Those tools are:
fgets to attempt to read a line (by reading one character at a time), up to a threshold (the size of your buffer). You get to decide how large a line should be, because it's more often the case that you won't expect a user/program to input huge lines.
strchr or strcspn to detect newlines from within your buffer, in order to determine whether you read a complete line.
scanf("%*[^\n]"); to discard the remainder of an incomplete line, when you detect those.
realloc to reallocate your buffer, if you decide you want to resize it and call fgets a second time to retrieve more data rather than discarding the remainder. Note: this will have an effect on the runtime complexity of your code, not that I think you should care about that...
Other options are available for the first three. You could use fgetc (or even read one character at a time) like I did at the end of this answer, for example...
In fact, that answer is highly relevant to your question, as it does make an attempt to exponentially increase the size. I wrote another example of this here.
It should be pointed out that the reason to address these problems is not so much optimisation, but the need to read a large, yet variadic in size chunk of memory. Remember, if you haven't yet written the code, it's likely you won't know whether it's worthwhile optimising it!
Suffice to say, it isn't the read function you should try to reduce your dependence upon, but the malloc/realloc/calloc function... That's the real kicker! If you don't absolutely need to store the entire line, then don't!
MAC Addresses are 48 bits. That is equivalent to three shorts. MAC
addresses are sometimes written like this: 01:23:45:67:89:ab where
each pair of digits represents a hexadecimal number.
Write a function that will take in a character pointer pointing to a
null terminated string of characters like in the example and will
break it apart and then store it in an array of three 16-bit shorts.
The address of the array will also be passed into the function.
I figured the function header should look something like void convertMacToShort(char *macAddr, short *shorts);. What I'm having difficulty with is parsing the char*. I feel like it's possible if I loop over it, but that doesn't feel efficient enough. I don't even need to make this a universal function of sorts--the MAC address will always be a char* in the format of 01:23:45:67:89:ab.
What's a good way to go about parsing this?
Well efficiency is one thing ... robustness another.
If you have very defined circumstances like a list of millions of MAC addresses which are all in the same format (only lower case letters, always leading zeroes, ...) then I would suggest using a quick function accessing the characters directly.
If you're parsing user input and need to detect input errors as well, execution speed is not the thing to worry about. In this scenario you have to make sure that you detect all possible mistakes a user is able to do (and this is quite a feat). This leads to sscanf(..) and in that case I would even suggest to write your own function which parses the string (for my experience sscanf(..) sometimes causes trouble depending on the input string and therefore I avoid using it when processing user input).
Another thing: If you're worrying about efficiency in the means of execution time, write a little benchmark which runs the parsing function a few million times and compare execution time. This is easily done and sometimes brings up surprises...
I need to write a program where during run time, a set of integers of arbitrary size will taken as input. They will be seperated by white space. At the end, a new line is given, showing the end of input. How do I save them into an array of integers so that i can display them later. I think it is a little difficult because the number of values that will be entered is not known during compilation
Sounds like homework.
Correct me if I am wrong and I will give you more than hints.
You can either declare an array of a really large size that would not possibly be filled by the user input, then use scanf or something like that to grab the integers until you hit '\n', or you can grab each integer at a time, allocating memory as you go, using a combination of malloc and memcpy calls. The first option should never be done in a real world problem, and I am certainly not advocating such practices even though your textbook probably tells you to do it this way.
There is an example just like this in K&R.
This is a typical problem you will have in C. The solution is usually one of two options.
Use a really large array that is large enough to hold the input. Sometimes this is a poor option when the data could be really large. An example of when it would be a bad idea is when you are saving a video frame or a large text file to the array. This also opens you up to a buffer overrun attack in older versions of Windows. However, this is sometimes a good quick hack solution for smaller (homework) programs where you can count on the user (i.e. your professor who is not trying to break your program) to not input 1000's of characters. Usually this is considered bad practice, please consider my 2nd option for the security reason I mentioned before.
Use dynamic arrays (i.e. malloc). This is probably what your professor wants you to do as this sounds like a typical problem to use when a student is first learning pointers and arrays. This is a great approach, just remember to call free on your memory when you are finished. The tricky part here is that you still have to know the size of the array you want ahead of time (not at compile time though of course).