Fread - large amount of integers - c

i need to read large amount of integers from stdin. Speed is very important here, for example getchar_unlocked is too slow for me(hundredths of a second really matters)
My code works with getchar_unlocked, but now I'm trying to read wholle line of unknown number of integers to buffer with fread. What is wrong down there?
code below:
inline int fastRead_int(int *sum) {
char buffer[sizeof(int)*sizeof(int)];
register int i = 0;
fread(buffer,sizeof(buffer),1,stdin);
register int c = buffer[0];
int x = 0;
while(c != NULL)
{
c = buffer[++i];
for(; ((c<48 || c>57) && c != ' '); c = buffer[++i]);
for(; c>47 && c<58 && c != ' ' ; c = buffer[++i]) {
x = (x<<1) + (x<<3) + c - 48;
}
*sum = *sum+x;
}

// what exactly does the input look like?
// I doubt that the input is a series of single digit numbers.
// and for multidigit numbers, the following will not work
// the posted code fails to compile for several reasons,
// including that the 'int' return value is never set
// and the code is much to big for inlining
inline int fastRead_int(int *sum)
{
char buffer[sizeof(int)*sizeof(int)]; //<-- = 64bytes = 16 binary ints
// but as little as 4 character ints
register int i = 0;
fread(buffer,sizeof(buffer),1,stdin);
// <-- should check/save the returned value
// as that would be the number of characters actually read
// and fread() does not NUL terminate the input
// should also check for and handle read failures
register int c = buffer[0]; // <-- this gets one byte/char only,
// however, unless the input is a binary file
// this will only get the first byte
// of a numeric char string.
int x = 0;
// <-- suggest the following code block be:
// for( i=0; i<(returned value from fread(); i++)
// {
// if( isdigit(c) )
// {
// x += (c-48);
// }
// c = buffer[++i];
// } // end for
// *sum += x;
while(c != NULL) // <-- 'c' is a integer in the range 0...255
// but NULL is a 4 byte pointer to 0
// perhaps you meant: 'while(c)'
{
c = buffer[++i]; // <-- see my comment about reading buffer, above
// step by non-numeric characters
// what if the input does not end in a numeric character?
// then would be reading past the end of the buffer[]
// resulting in undefined behaviour leading to a seg fault event
for(; ((c<48 || c>57) && c != ' '); c = buffer[++i]);
//<-- much better/clearer to #include ctype.h and then
// for( ; !isdigit(c); c = buffer[++i] ;)
// <-- see above comments
for(; c>47 && c<58 && c != ' ' ; c = buffer[++i])
{
x = (x<<1) + (x<<3) + c - 48;
}
*sum = *sum+x;
} // end while

Related

How to Convert the last digit of a number to a char and display it on LCD?

I recently got a STM8 MCU and it has the built in function LCD_GLASS_DisplayString("STRING")
The problem with that function is, as you can see below, that I cannot directly display an integer on it:
void LCD_GLASS_DisplayString(uint8_t* ptr)
{
uint8_t i = 0x01;
LCD_GLASS_Clear();
/* Send the string character by character on lCD */
while ((*ptr != 0) & (i < 8))
{
/* Display one character on LCD */
LCD_GLASS_WriteChar(ptr, FALSE, FALSE, i);
/* Point on the next character */
ptr++;
/* Increment the character counter */
i++;
}
}
How could I modify it so I could send integers directly? Also, I'm not sure I can use any libraries, so just pure C would help.
I was thinking of something like this, but it didn't work:
void LCD_GLASS_DisplayINT(uint16_t integer)
{
uint8_t i = 0x01;
LCD_GLASS_Clear();
/* Send the string character by character on lCD */
while ((integer != 0) & (i < 8))
{
/* Display one number on LCD */
LCD_GLASS_WriteChar("0" + integer%10, FALSE, FALSE, i);
/* Point on the next number*/
integer=integer/10;
/* Increment the character counter */
i++;
}
}
Any idea on how to make it work? I need to either make a function to display the integers or a way to convert them to strings before I send them over to the LCD. The code is pure C, as what I'm programming are pure drivers right now.
You're not far off with "0" + integer%10 - but you need to treat it as a character - '0' + integer%10 - and you need to pass LCD_GLASS_WriteChar a pointer to this character.
One way to do this is:
char* digits = "0123456789";
LCD_GLASS_WriteChar(&digits[integer % 10], FALSE, FALSE, i);
Also, your loop condition - while ((integer != 0) & (i < 8)) should not use bitwise and (&), but rather logical and (&&).
while ((integer != 0) && (i < 8))
You'll want to store the digits in a buffer to get the left-to-right order. On a microcontroller you'll probably want to allocate that buffer in .bss so that it doesn't take up stack. Use static to make that happen. For a 16 bit number you'll have at most 5 digits + null term, so:
static char buf[5+1] = {0};
The conversion is essentially the code you've written, except you mixed up '0' and "0":
for(uint8_t i=0; i<5; i++)
{
buf[5-i-1] = val%10 + '0';
val/=10;
}
Note that this code always leaves item buf[5] untouched and leaves a zero null terminator there.
Complete code with tests in standard C. Obviously drop puts for your custom LCD routine.
#include <stdio.h>
#include <stdint.h>
void display_int (uint16_t val)
{
static char buf[5+1] = {0};
char* p;
for(uint8_t i=0; i<5; i++)
{
buf[5-i-1] = val%10 + '0';
val/=10;
}
// trim leading zeroes
for(p=buf; *p!='\0'; p++)
{
if(*p!='0' || // stop looking at first non zero or
p[1]=='\0') // in case there is only one zero character
break;
}
puts(p); // use whatever string printing routine you got
}
int main (void)
{
display_int(0);
display_int(5);
display_int(666);
display_int(12345);
}
Division on STM8 is very expensive. We can optimize the code a bit by not filling up the whole buffer if we don't have to. This saves us several DIV calls. Optimized version, faster but harder to read:
void display_int (uint16_t val)
{
static char buf[5+1] = {0};
char* p;
char* start;
for(uint8_t i=0;;i++)
{
buf[5-i-1] = val%10 + '0';
val/=10;
if(val==0)
{
start = &buf[5-i-1];
break;
}
}
// trim leading zeroes
for(p=start; *p!='\0'; p++)
{
if(*p!='0' || // stop looking at first non zero or
p[1]=='\0') // in case there is only one zero character
break;
}
puts(p); // use whatever string printing routine you got
}

c string: put ' ' if a word found in the sentence

I made a code and my target is to put spacewhere the input word was found in a sentence.
i neet to replece the small word with space
like:
Three witches watched three watches
tch
output:
Three wi es wa ed three wa es
I made this code:
#include<stdio.h>
#define S 8
#define B 50
void main() {
char small[S] = {"ol"};
char big[B] = {"my older gradmom see my older sister"};
int i = 0, j = 0;
for (i = 0; i < B; i++)
{
for(j=0;j<S;j++)
{
if(small[j]!=big[i])
{
j=0;
break;
}
if(small[j]=='\0')
{
while (i-(j-1)!=i)
{
i = i - j;
big[i] = '\n';
i++;
}
}
}
}
puts(big);
}
First of all, in your exemple you work with newline '\n' and not with space.
Consider this simple example:
#include<stdio.h>
#define S 8
#define B 50
void main() {
char small[S] = {"ol"};
char big[B] = {"my older gradmom see my older sister"};
int i = 0, j = 0;
int cpt = 0;
int smallSize = 0;
// loop to retrieve smallSize
for (i = 0; i < S; i++)
{
if (small[i] != '\0')
smallSize++;
}
// main loop
for (i = 0; i < B; i++)
{
// stop if we hit the end of the string
if (big[i] == '\0')
break;
// increment the cpt and small index while the content of big and small are equal
if (big[i] == small[j])
{
cpt++;
j++;
}
// we didn't found the full small word
else
{
j = 0;
cpt = 0;
}
// test if we found the full word, if yes replace char in big by space
if (cpt == smallSize)
{
for (int k = 0; k < smallSize; k++)
{
big[i-k] = ' ';
}
j = 0;
cpt = 0;
}
}
puts(big);
}
You need first to retrieve the real size of the small array.
Once done, next step is to look inside "big" if there is the word small inside. If we find it, then replace all those char by spaces.
If you want to replace the whole small word with a single space, then you'll need to adapt this example !
I hope this help !
A possible way is to use to pointers to the string, one for reading and one for writing. This will allow to replace an arbitrary number of chars (the ones from small) with a single space. And you do not really want to nest loops but une only one to process every char from big.
Last but not least, void main() should never be used except in stand alone environment (kernel or embedded development). Code could become:
#include <stdio.h>
#define S 8
#define B 50
int main() { // void main is deprecated...
char small[S] = {"ol"};
char big[B] = {"my older gradmom see my older sister"};
int i = 0, j = 0;
int k = 0; // pointer to written back big
for (i = 0; i < B; i++)
{
if (big[i] == 0) break; // do not process beyond end of string
if(small[j]!=big[i])
{
for(int l=0; l<j; l++) big[k++] = small[l]; // copy an eventual partial small
big[k++] = big[i]; // copy the incoming character
j=0; // reset pointer to small
continue;
}
else if(small[++j] == 0) // reached end of small
{
big[k++] = ' '; // replace chars from small with a single space
j = 0; // reset pointer to small
}
}
big[k] = '\0';
puts(big);
return 0;
}
or even better (no need for fixed sizes of strings):
#include <stdio.h>
int main() { // void main is deprecated...
char small[] = {"ol"};
char big[] = {"my older gradmom see my older sister"};
int i = 0, j = 0;
int k = 0; // pointer to written back big
for (i = 0; i < sizeof(big); i++)
{
if(small[j]!=big[i])
...
In C strings are terminated with a null character '\0'. Your code defines a somehow random number at the beginning (B and S) and iterates over that much characters instead of the exact number of characters, the strings actually contain. You can use the fact that the string is terminated by testing the content of the string in a while loop.
i = 0;
while (str[i]) {
...
i = i + 1;
}
If you prefer for loops you can write it also as a for loop.
for (i = 0; str[i]; i++) {
...
}
Your code does not move the contents of the remaining string to the left. If you replace two characters ol with one character , you have to move the remaining characters to the left by one character. Otherwise you would have a hole in the string.
#include <stdio.h>
int main() {
char small[] = "ol";
char big[] = "my older gradmom see my older sister";
int s; // index, which loops through the small string
int b; // index, which loops through the big string
int m; // index, which loops through the characters to be modified
// The following loops through the big string up to the terminating
// null character in the big string.
b = 0;
while (big[b]) {
// The following loops through the small string up to the
// terminating null character, if the character in the small
// string matches the corresponding character in the big string.
s = 0;
while (small[s] && big[b+s] == small[s]) {
// In case of a match, continue with the next character in the
// small string.
s = s + 1;
}
// If we are at the end of the small string, we found in the
// big string.
if (small[s] == '\0') {
// Now we have to modify the big string. The modification
// starts at the current position in the big string.
m = b;
// First we have to put the space at the current position in the
// big string.
big[m] = ' ';
// And next the rest of the big string has to be moved left. The
// rest of the big string starts, where the match has ended.
while (big[b+s]) {
m = m + 1;
big[m] = big[b+s];
s = s + 1;
}
// Finally the big string has to be terminated by a null
// character.
big[m+1] = '\0';
}
// Continue at next character in big string.
b = b + 1;
}
puts(big);
return 0;
}

Reading ints from serial port Arduino

I've done a C program that writes an integer array into the Arduino:
// ...
FILE* file;
file = fopen("/dev/ttyuSB0","w");
for (int i = 0; i < 3; i++) {
fprintf(file, "%d ", rgb[i]);
}
fclose(file);
// ...
How can I do from my arduino code (.ino) to catch the three integers from the file?
while (Serial.available() > 0) {
// What can I do here ???
}
You need to read the data and put it into a buffer. After you encounter a ' ' character you terminate the string inside the buffer and convert it into an int.
When you do this three times you have read all three ints.
const uint8_t buff_len = 7; // buffer size
char buff[buff_len]; // buffer
uint8_t buff_i = 0; // buffer index
int arr[3] = {0,0,0}; // number array
uint8_t arr_i = 0; // number array index
void loop() {
while (Serial.available() > 0) {
char c = Serial.read();
if (buff_i < buff_len-1) { // check if buffer is full
if (c == ' ') { // check if got number terminator
buff[buff_i++] = 0; // terminate the string
buff_i = 0; // reset the buffer index
arr[arr_i++] = atoi(buff); // convert the string to int
if (arr_i == 3) { // if got all three numbers
arr_i = 0; // reset the number array index
// do something with the three integers
}
}
else if (c == '-' || ('0' <= c && c <= '9')) // if negative sign or valid digit
buff[buff_i++] = c; // put the char into the buffer
}
}
// maybe do some other stuff
}
Or if you don't mind blocking code[1] you can use builtin ParseInt.
void loop() {
while (Serial.available() > 0) {
arr[0] = Serial.parseInt();
arr[1] = Serial.parseInt();
arr[2] = Serial.parseInt();
Serial.read(); // read off the last space
// do something with the three integers
}
// maybe do some other stuff, but might be blocked by serial read
}
[1] If your computer has a hiccup and doesn't send all the data at once, your Arduino code will just wait for data and won't be doing anything else. Read more here.

Why does my conversion method in C continue to fail?

I am trying to convert from a popen pass, to a float as the final result. I have tried converting to a char, and then into a float in every possible way I can find, however the output I have seen using printf seems to be wrong every time. I have tried using a tostring function, as well as using a %s like in the printf function that returns the correct function, however it all seems to give me the wrong output as soon as I try to convert the output. Should I be trying a different conversion method?
Here is the code.
FILE * uname;
char os[80];
int lastchar;
char n;
uname = popen("sudo python ./return63.py", "r");
lastchar = fread(os, 1, 80, uname);
os[lastchar] = "\0";
n = toString(("%s", os));
printf("THE DIRECT OUTPUT FROM PY IS %s", os);
printf("THE DIRECT OUTPUT For n IS %c", n);
float ia = n - 0;
long p = ia - 0;
float dd = p - 0;
printf("Your OS is %f", dd);
Output from the PY is 'THE DIRECT OUTPUT FROM PY IS 63.0' , which is the correct value,
output from the n is 'THE DIRECT OUTPUT For n IS �'
output from the dd is 'Your OS is Your OS is 236.000000'
The function tostring was pulled from an answered question about how to get the output from another answered question. I have tried with and without this code.
int toString(char a[]) {
int c, sign, offset, n;
if (a[0] == '-') { // Handle negative integers
sign = -1;
}
if (sign == -1) { // Set starting position to convert
offset = 1;
}
else {
offset = 0;
}
n = 0;
for (c = offset; a[c] != '\0'; c++) {
n = n * 10 + a[c] - '0';
}
if (sign == -1) {
n = -n;
}
return n;
}
toString returns an int, so store an int and output an int.
int n = toString(os); // Also removed the obfuscating '("%s", ..)'
printf("THE DIRECT OUTPUT For n IS %d", n);
Also your toString function has undefined behavior because sign might be read without being initialized.
if (a[0] == '-') { // Handle negative integers
sign = -1;
offset = 1;
}
else {
sign = 1;
offset = 0;
}
You have a potential os buffer overflow and you are not doing the null termination of os correctly:
lastchar = fread(os, 1, sizeof(os) - 1, uname); // Only read one byte less
os[lastchar] = '\0'; // changed from string "\0" to char '\0'
And finally you are not checking the input string for digits, you are accepting every input (also the '.' in "63.0"). You might want to stop at the first non-digit character:
for (c = offset; !isdigit((unsigned char)a[c]); c++) {

any one know how to convert a huge char array to float, very huge array, performance better than the atof/strtod/sscanf

I got a char array, a huge array char p[n] read from a txt like.
//1.txt
194.919 -241.808 234.896
195.569 -246.179 234.482
194.919 -241.808 234.896
...
foo(char *p, float x, float y, float z)
{
}
I tried to use atof, strtod, but they are real time consuming when the array is too huge, because they will call the strlen(). and the sscanf is also very slow....
I debug into the code and find that both atof() and strtod call the strlen() in the visual studio, we can check the crt code.
strtod() call:
answer = _fltin2( &answerstruct, ptr, (int)strlen(ptr), 0, 0, _loc_update.GetLocaleT());
atof() call:
return( *(double *)&(_fltin2( &fltstruct, nptr, (int)strlen(nptr), 0, 0, _loc_update.GetLocaleT())->dval) );
I also try to use strtok, but we should not change any data in the 1.txt.
so any one have the best way to convert all these to float x, y, z.
Visual studio 2008 + WIN7
If you can make additional assumptions about the format of the floating point values, parsing them yourself might increase performance.
Example code for parsing ' ' or '\n'-separated values without exponents and no input validation:
float parsef(const char **str)
{
const char *cc = *str;
_Bool neg = (*cc == '-');
if(neg) ++cc;
float value = 0, e = 1;
for(; *cc != '.'; ++cc)
{
if(*cc == ' ' || *cc == '\n' || !*cc)
{
*str = cc;
return neg ? -value : value;
}
value *= 10;
value += *cc - '0';
}
for(++cc;; ++cc)
{
if(*cc == ' ' || *cc == '\n' || !*cc)
{
*str = cc;
return neg ? -value : value;
}
e /= 10;
value += (*cc - '0') * e;
}
}
Example code:
const char *str = "42 -15.4\n23.001";
do printf("%f\n", parsef(&str));
while(*str++);
Okay, how about doing the tokenization yourself and then calling strtod.
What I'm thinking is something like this:
char *current = ...; // initialited to the head of your character array
while (*current != '\0')
{
char buffer[64];
unsigned int idx = 0;
// copy over current number
while (*current != '\0' && !isspace(*current))
{
buffer[idx++] = *current++;
}
buffer[idx] = '\0';
// move forward to next number
while (*current != '\0' && isspace(*current))
{
current++;
}
// use strtod to convert buffer
}
Some issues with this is the tokenization is very simple. It will work for the format you posted, but if the format varies (another line uses : to separate the numbers), it won't work.
Another issue is that the code assumes all numbers have < 64 characters. If they are longer, you'll get a buffer overflow.
Also, the copying to a temporary buffer will add some overhead (but hopefully less then the overhead of constantly doing a strlen on the entire buffer). I know you said you can't change the original buffer, but can you do a temporary change (i.e. the buffer can change as as long as you return it to it's original state before you return):
char *current = ...; // initialited to the head of your character array
while (*current != '\0')
{
char *next_sep = current;
while (*next_sep != '\0' && !isspace(*next_sep))
{
next_sep++;
}
// save the separator before overwriting it
char tmp = *next_sep;
*next_sep = '\0';
// use strtod on current
// Restore the separator.
*next_sep = tmp;
current = next_sep;
// move forward to next number
while (*current != '\0' && isspace(*current))
{
current++;
}
}
This technique means no copying and no worries about buffer overflow. You do need to temporarily modify the buffer; hopefully that is
Check out this code.
It can be further optimized if there's no need to support scientific representation, '+' sign, or leading tabs.
It doesn't use strlen, or any other standard library string routine.
// convert floating-point value in string represention to it's numerical value
// return false if NaN
// F is float/double
// T is char or wchar_t
// '1234.567' -> 1234.567
template <class F, class T> inline bool StrToDouble(const T* pczSrc, F& f)
{
f= 0;
if (!pczSrc)
return false;
while ((32 == *pczSrc) || (9 == *pczSrc))
pczSrc++;
bool bNegative= (_T('-') == *pczSrc);
if ( (_T('-') == *pczSrc) || (_T('+') == *pczSrc) )
pczSrc++;
if ( (*pczSrc < _T('0')) || (*pczSrc > _T('9')) )
return false;
// todo: return false if number of digits is too large
while ( (*pczSrc >= _T('0')) && (*pczSrc<=_T('9')) )
{
f= f*10. + (*pczSrc-_T('0'));
pczSrc++;
}
if (_T('.') == *pczSrc)
{
pczSrc++;
double e= 0.;
double g= 1.;
while ( (*pczSrc >= _T('0')) && (*pczSrc<=_T('9')) )
{
e= e*10. + (*pczSrc-_T('0'));
g= g*10. ;
pczSrc++;
}
f+= e/g;
}
if ( (_T('e') == *pczSrc) || (_T('E') == *pczSrc) ) // exponent, such in 7.32e-2
{
pczSrc++;
bool bNegativeExp= (_T('-') == *pczSrc);
if ( (_T('-') == *pczSrc) || (_T('+') == *pczSrc) )
pczSrc++;
int nExp= 0;
while ( (*pczSrc >= _T('0')) && (*pczSrc <= _T('9')) )
{
nExp= nExp*10 + (*pczSrc-_T('0'));
pczSrc++;
}
if (bNegativeExp)
nExp= -nExp;
// todo: return false if exponent / number of digits of exponent is too large
f*= pow(10., nExp);
}
if (bNegative)
f= -f;
return true;
}
As long as you are not using a particularly bad standard library (impossible these times, they are all good) it's not possible to do it faster than atof.
I don't see any reason why strod() should call strlen(). Of course it might, but nothing in its specification requires it and I'd be suprised if it did. And I'd say that strtod() about as fast as you'll get, short of writing some FPU processor-specific stuff yourself.
Why do you think atof, strtod use strlen? I've never implemented them, but I can't imagine why they'd need to know the length of the input string. It would be of no value to them. I'd use strtod as per Jason's answer. That's what it's for.
And yes, if you have a very large amount of text, it's going to take some time to convert. That's just the way it is.
Use strtod. It almost certainly does not call strlen. Why would it need to know the length of the input? It merely runs past leading whitespace, then consumes as many characters as possible that make sense for a floating point literal, and then returns a pointer just past that. You can see an example implementation Perhaps you're using it non-optimally? Here's a sample of how to use strtod:
#include <stdio.h>
#include <stdlib.h>
int main() {
char *p = "1.txt 194.919 -241.808 234.896 195.569 -246.179 234.482 194.919 -241.808 234.896";
char *end = p;
char *q;
double d;
while(*end++ != ' '); // move past "1.txt"
do {
q = end;
d = strtod(q, &end);
printf("%g\n", d);
} while(*end != '\0');
}
This outputs:
194.919
-241.808
234.896
195.569
-246.179
234.482
194.919
-241.808
234.896
on my machine.
As others have said, I don't think you're going to do much better than the standard library calls. They have been around for a long time and are quite highly optimized (well, they should be, at least in good implementations).
That said, there are some things that aren't clear to me. Are you reading the whole file into memory and then converting the array to another array? If so, you might want to check that the system you are running on has enough memory to do that with swapping. If you are doing this, would it be possible to just convert one line at a time as you read them off disk instead of storing them?
You could consider multithreading your program. One thread to read and buffer lines off disk, and n threads to process the lines. Dr. Dobb's Journal published a great single-reader/single-writer lockless queue implementation you could use. I've used this in a similar app. My worker threads each have an input queue, and then reader thread reads data off disk and places them into these queues in round robin style.
How about something like:
#include <stdio.h>
#include <string.h>
#include <stdlib.h>
static float frac[] =
{
0.000,
0.001,
0.002,
... // fill in
0.997,
0.998,
0.999,
};
static float exp[] =
{
1e-38,
1e-37,
1e-36,
... // fill in
1e+36,
1e+37,
1e+38,
};
float cvt(char* p)
{
char* d = strchr(p, '.'); // Find the decimal point.
char* e = strchr(p, 'e'); // Find the exponent.
if (e == NULL)
e = strchr(p, 'E');
float num = atoi(p);
if (num > 0) {
num += frac[atoi(d + 1)];
} else {
num -= frac[atoi(d + 1)];
}
if (e)
num *= exp[atoi(e)];
return num;
}
int main()
{
char line[100];
while(gets(line)) {
printf("in %s, out %g\n", line, cvt(line));
}
}
Should be good to three significant digits.
Edit: watch out for big mantissas.
Edit again: and negative exponents. :-(
I doubt if strlen is costing you much.
If you can take advantage of your numbers falling in a relatively restricted range, then what I suggest is to parse it yourself, doing as little computation as possible, such as:
#define DIGIT(c) ((c)>='0' && (c)<='9')
BOOL parseNum(char* *p0, float *f){
char* p = *p0;
int n = 0, frac = 1;
BOOL bNeg = FALSE;
while(*p == ' ') p++;
if (*p == '-'){p++; bNeg = TRUE;}
if (!(DIGIT(*p) || *p=='.')) return FALSE;
while(DIGIT(*p)){
n = n * 10 + (*p++ - '0');
}
if (*p == '.'){
p++;
while(DIGIT(*p)){
n = n * 10 + (*p++ - '0');
frac *= 10;
}
}
*f = (float)n/(float)frac;
if (bNeg) *f = -*f;
*p0 = p;
return TRUE;
}

Resources