Reading bytes from progmem - arrays

I'm trying to write a simple program (as a pre-cursor to a more complicated one) that stores an array of bytes to progmem, and then reads and prints the array. I've looked through a million blog/forums posts online and think I'm doing everything fine, but I'm still getting utter gibberish as output.
Here is my code, any help would be much appreciated!
void setup() {
byte hello[10] PROGMEM = {1,2,3,4,5,6,7,8,9,10};
byte buffer[10];
Serial.begin(9600);
memcpy_P(buffer, (char*)pgm_read_byte(&hello), 10);
for(int i=0;i<10;i++){
//buffer[i] = pgm_read_byte(&(hello[i])); //output is wrong even if i use this
Serial.println(buffer[i]);
}
}
void loop() {
}
If I use memcpy, I get the output:
148
93
0
12
148
93
0
12
148
93
And if I use the buffer = .... statement in the for loop (instead of memcpy):
49
5
9
240
108
192
138
173
155
173

You're thinking about two magnitudes too complicated.
memcpy_P wants a source pointer, a destination pointer and a byte count. And the PROGMEM pointer is simply the array. So, your memcpy_P line should like like
memcpy_P (buffer, hello, 10);
that's it.
memcpy (without the "P") will not be able to reach program memory and copy stuff from data RAM instead. That is not what you want.

Related

strstr() function returns the address 0x0000

I'm trying to check whether (and where) a substring ("DATA") is located in a big string (located in a buffer - linearBuffer) by strstr() function, but it doesn't seem to work and I don't know why eventhough my source string (located in the linearBuffer) in null terminated.
What really happended is that a ringbuffer (buf) fills with characters for every USART interrupt. Then, in some point of the code its content copied into a linear buffer (through ringBuff_to_linearBuff()) and I apply the strstr() function on it in order to find a wanted substring. The value that I get when the function strstr() returns is the value 244 and not the location of the substring eventhough I know its there from setting a breakpoint
** Note that my code is spread on many files so I tried to gather all question related code together.
#include <string.h>
#define BUFFER_SIZE 400
#define LINEAR_BUFFER_SIZE (BUFFER_SIZE+1)
#define WIFI_CMD_DATA "DATA"
typedef RingBuff_Data_t uint8_t;
typedef struct
{
RingBuff_Data_t Buffer[BUFFER_SIZE]; /**< Internal ring buffer data, referenced by the buffer pointers. */
RingBuff_Data_t* In; /**< Current storage location in the circular buffer */
RingBuff_Data_t* Out; /**< Current retrieval location in the circular buffer */
} RingBuff_t;
volatile RingBuff_t buf;
uint8_t linearBuffer[LINEAR_BUFFER_SIZE]="";
static inline void RingBuffer_Insert(RingBuff_t* const Buffer, const RingBuff_Data_t Data)
{
*Buffer->In = Data;
if (++Buffer->In == &Buffer->Buffer[BUFFER_SIZE])
Buffer->In = Buffer->Buffer;
ATOMIC_BLOCK(ATOMIC_RESTORESTATE)
{
Buffer->Count++;
}
}
ISR(USART1_RX_vect)
{
//code to be executed when the rx pin of the USART receives a char
uint8_t c = UDR_N;
if (c != '\n')
RingBuffer_Insert(&buf,c);
else
RingBuffer_Insert(&buf,'\0');
}
void ringBuff_to_linearBuff(uint8_t linearBuffer[])
{
memset(linearBuffer,0,LINEAR_BUFFER_SIZE);
RingBuff_Data_t* tempIn = buf.In;
if (buf.Out < tempIn){
memcpy(linearBuffer, buf.Out, tempIn - buf.Out);
}
else if (buf.Out > tempIn){
size_t s1 = buf.Buffer + BUFFER_SIZE - buf.Out;
size_t s2 = buf.In - buf.Buffer;
memcpy(linearBuffer, buf.Out, s1);
memcpy(linearBuffer + s1, buf.Buffer, s2);
}
}
void main ()
{
uint8_t* linearBufferp;
while (1)
{
if (buf.Out != buf.In)
{
ringBuff_to_linearBuff(linearBuffer);
linearBufferp = strstr(linearBuffer, WIFI_CMD_DATA); // Checking if a new DATA msg from a client had arrived
if (linearBufferp != NULL)
{
//do something
}
}
}
}
debugging
When strstr returns NULL it means that it didn't find the substring (and when that happens, the value it points to have no meaning at all, so forget about the 224).
So your question should be:
Why doesn't strstr find my substring?
When looking at the debug picture you posted, your linearBuffercontains:
13
0
13
0
43
....
....
68 <---- This is what you want to find
65
....
However, there are multiple strings in your buffer:
13 <----- Start of first string
0 <----- End of first string
13 <----- Start of second string
0 <----- End of second string
43 <----- Start of thrid string
....
....
68 <---- This is what you want to find
65
....
strstr will only search the first string. When strstr sees the first 0 (index [1]), it returns NULL because it didn't find what it was looking for.
In other words - strstr never looks at the part of the buffer where the match is. It returns long before that.
So what's wrong with your code?
It is hard to say since you haven't posted a complete code base. So this is a guess. I think you receive a number of "newlines" in the form:
13 10 13 10
before the message. So you receive:
13 10 13 10 43 ...... 68 65 .....
Your ISR turns the 10 into 0 so the buffer becomes
13 0 13 0 43 ...... 68 65 .....
which is 3 strings instead of 1 string.
What to do?
Well, there could be several different solutions. The correct depends on your system requirements. A simple solution would be to skip the extra 13 0 before calling strstr. Something like:
ringBuff_to_linearBuff(linearBuffer);
// Skip "13 0"
while (*linearBuffer == 13 && *(linearBuffer+1) == 0)
{
linearBuffer += 2;
}
linearBufferp = strstr(linearBuffer, WIFI_CMD_DATA);
Note: You should add some range check also so that linearBuffer isn't incremented so much that you read out of bounds

Having issues iterating through machine code

I'm attempting to recreate the wc command in c and having issues getting the proper number of words in any file containing machine code (core files or compiled c). The number of logged words always comes up around 90% short of the amount returned by wc.
For reference here is the project info
Compile statement
gcc -ggdb wordCount.c -o wordCount -std=c99
wordCount.c
/*
* Author(s) - Colin McGrath
* Description - Lab 3 - WC LINUX
* Date - January 28, 2015
*/
#include<stdio.h>
#include<string.h>
#include<dirent.h>
#include<sys/stat.h>
#include<ctype.h>
struct counterStruct {
int newlines;
int words;
int bt;
};
typedef struct counterStruct ct;
ct totals = {0};
struct stat st;
void wc(ct counter, char *arg)
{
printf("%6lu %6lu %6lu %s\n", counter.newlines, counter.words, counter.bt, arg);
}
void process(char *arg)
{
lstat(arg, &st);
if (S_ISDIR(st.st_mode))
{
char message[4056] = "wc: ";
strcat(message, arg);
strcat(message, ": Is a directory\n");
printf(message);
ct counter = {0};
wc(counter, arg);
}
else if (S_ISREG(st.st_mode))
{
FILE *file;
file = fopen(arg, "r");
ct currentCount = {0};
if (file != NULL)
{
char holder[65536];
while (fgets(holder, 65536, file) != NULL)
{
totals.newlines++;
currentCount.newlines++;
int c = 0;
for (int i=0; i<strlen(holder); i++)
{
if (isspace(holder[i]))
{
if (c != 0)
{
totals.words++;
currentCount.words++;
c = 0;
}
}
else
c = 1;
}
}
}
currentCount.bt = st.st_size;
totals.bt = totals.bt + st.st_size;
wc(currentCount, arg);
}
}
int main(int argc, char *argv[])
{
if (argc > 1)
{
for (int i=1; i<argc; i++)
{
//printf("%s\n", argv[i]);
process(argv[i]);
}
}
wc(totals, "total");
return 0;
}
Sample wc output:
135 742 360448 /home/cpmcgrat/53/labs/lab-2/core.22321
231 1189 192512 /home/cpmcgrat/53/labs/lab-2/core.26554
5372 40960 365441 /home/cpmcgrat/53/labs/lab-2/file
24 224 12494 /home/cpmcgrat/53/labs/lab-2/frequency
45 116 869 /home/cpmcgrat/53/labs/lab-2/frequency.c
5372 40960 365441 /home/cpmcgrat/53/labs/lab-2/lineIn
12 50 1013 /home/cpmcgrat/53/labs/lab-2/lineIn2
0 0 0 /home/cpmcgrat/53/labs/lab-2/lineOut
39 247 11225 /home/cpmcgrat/53/labs/lab-2/parseURL
138 318 2151 /home/cpmcgrat/53/labs/lab-2/parseURL.c
41 230 10942 /home/cpmcgrat/53/labs/lab-2/roman
66 162 1164 /home/cpmcgrat/53/labs/lab-2/roman.c
13 13 83 /home/cpmcgrat/53/labs/lab-2/romanIn
13 39 169 /home/cpmcgrat/53/labs/lab-2/romanOut
7 6 287 /home/cpmcgrat/53/labs/lab-2/URLs
11508 85256 1324239 total
Sample rebuild output (./wordCount):
139 76 360448 /home/cpmcgrat/53/labs/lab-2/core.22321
233 493 192512 /home/cpmcgrat/53/labs/lab-2/core.26554
5372 40960 365441 /home/cpmcgrat/53/labs/lab-2/file
25 3 12494 /home/cpmcgrat/53/labs/lab-2/frequency
45 116 869 /home/cpmcgrat/53/labs/lab-2/frequency.c
5372 40960 365441 /home/cpmcgrat/53/labs/lab-2/lineIn
12 50 1013 /home/cpmcgrat/53/labs/lab-2/lineIn2
0 0 0 /home/cpmcgrat/53/labs/lab-2/lineOut
40 6 11225 /home/cpmcgrat/53/labs/lab-2/parseURL
138 318 2151 /home/cpmcgrat/53/labs/lab-2/parseURL.c
42 3 10942 /home/cpmcgrat/53/labs/lab-2/roman
66 162 1164 /home/cpmcgrat/53/labs/lab-2/roman.c
13 13 83 /home/cpmcgrat/53/labs/lab-2/romanIn
13 39 169 /home/cpmcgrat/53/labs/lab-2/romanOut
7 6 287 /home/cpmcgrat/53/labs/lab-2/URLs
11517 83205 1324239 total
Notice the difference in the word count (second int) from the first two files (core files) as well as the roman file and parseURL files (machine code, no extension).
C strings do not store their length. They are terminated by a single NUL (0) byte.
Consequently, strlen needs to scan the entire string, character by character, until it reaches the NUL. That makes this:
for (int i=0; i<strlen(holder); i++)
desperately inefficient: for every character in holder, it needs to count all the characters in holder in order to test whether i is still in range. That transforms a simple linear Θ(N) algorithm into an Θ(N2) cycle-burner.
But in this case, it also produces the wrong result, since binary files typically include lots of NUL characters. Since strlen will actually tell you where the first NUL is, rather than how long the "line" is, you'll end up skipping a lot of bytes in the file. (On the bright side, that makes the scan quadratically faster, but computing the wrong result more rapidly is not really a win.)
You cannot use fgets to read binary files because the fgets interface doesn't tell you how much it read. You can use the Posix 2008 getline interface instead, or you can do binary input with fread, which is more efficient but will force you to count newlines yourself. (Not the worst thing in the world; you seem to be getting that count wrong, too.)
Or, of course, you could read the file one character at a time with fgetc. For a school exercise, that's not a bad solution; the resulting code is easy to write and understand, and typical implementations of fgetc are more efficient than the FUD would indicate.

FAT BPB and little endian reversal

My CPU is little endian, which documentation has told me conforms to the byte-order of the FAT specification. Why then, am I getting a valid address for the BS_jmpBoot, bytes 0-3 of first sector, but not getting a valid number for BPB_BytesPerSec, bytes 11-12 of the first sector.
116 int fd = open (diskpath, O_RDONLY, S_IROTH);
117
118 read (fd, BS_jmpBoot, 3);
119 printf("BS_jmpBoot = 0x%02x%02x%02x\n", BS_jmpBoot[0], S_jmpBoot[1], S_jmpBoot[2]);
120
121 read (fd, OEMName, 8);
122 OEMName[8] = '\0';
123 printf("OEMName = %s\n", OEMName);
124
125 read (fd, BPB_BytesPerSec, 2);
126 printf("BPB_BytesPerSec = 0x%02x%02x\n",BPB_BytesPerSec[0], BPB_BytesPerSec[1]);
Yields
BS_jmpBoot = 0xeb5890 //valid address, while 0x9058eb would not be
OEMName = MSDOS5.0
BPB_BytesPerSec = 0x0002 //Should be 0x0200
I would like figure out why BS_jmpBoot and OEMName print valid but BPB_BytesPerSec does not. If anyone could enlighten me I would be greatly appreciative.
Thanks
EDIT: Thanks for the help everyone, it was my types that were making everything go awry. I got it to work by writing the bytes to an unsigned short, as uesp suggested(kinda), but I would still like to know why this didn't work:
unsigned char BPB_BytesPerSec[2];
...
125 read (fd, BPB_BytesPerSec, 2);
126 printf("BPB_BytesPerSec = 0x%04x\n", *BPB_BytesPerSec);
yielded
BPB_BytesPerSec = 0x0000
I would like to use char arrays to allocate the space because I want to be sure of the space I'm writing to on any machine; or should I not?
Thanks again!
You are reading BPB_BytesPerSec incorrectly. The structure of the Bpb is (from here):
BYTE BS_jmpBoot[3];
BYTE BS_OEMName[8];
WORD BPB_BytesPerSec;
...
The first two fields are bytes so their endianness is irrelevant (I think). BPB_BytesPerSec is a WORD (assuming 2 bytes) so you should define/read it like:
WORD BPB_BytesPerSec; //Assuming WORD is defined on your system
read (fd, &BPB_BytesPerSec, 2);
printf("BPB_BytesPerSec = 0x%04x\n", BPB_BytesPerSec);
Since when you read the bytes directly you get 00 02, which is 0x0200 in little endian, you should correctly read BPB_BytesPerSec like this.
First of all, this line:
printf("BPB_BytesPerSec = 0x%02x%02x\n",BPB_BytesPerSec[0], BPB_BytesPerSec[1]);
is printing the value out in big endian format. If it prints 0x0002 here, the actual value would be 0x0200 in little endian.
As for the BS_jmpBoot value, according to this site:
The first three bytes EB 3C and 90 disassemble to JMP SHORT 3C NOP. (The 3C value may be different.) The reason for this is to jump over the disk format information (the BPB and EBPB). Since the first sector of the disk is loaded into ram at location 0x0000:0x7c00 and executed, without this jump, the processor would attempt to execute data that isn't code.
In other words, the first 3 bytes are opcodes which are three separate bytes, not one little endian value.

Reading an indefinite number of integers from a file

I am in the early stages of coding a homework assignment. The larger goal is a little bit bigger and beyond the scope of this question. The immediate goal is to take one or more two digit numbers from the command line which correspond to years (e.g. 52). Then open the file that goes with that year. The files are formatted thusly:
1952 Topps baseball
-------------------
8 10 15 17 20 47 48 49 59 71 136
153 155 159 162 168 170 175 176 186 188 202
215 233 248 252 253 254 257 259 264 270 271 272 274
282 283 284 285 287 293 294 295 297 299 300 308 310 311
312
Each file has a random (between 1-50) number of 1-3 digit integers. I store the year in an int. Then I store each of the later digits into an array. Then I will use that array to do other cool stuff. My problem is, how to I scan for a random number of integer inputs from the file. THis is what I have done so far:
#include <stdio.h>
#include <string.h>
main(int argc, char** argv) {
char filename[30];
int cards[100];
FILE *fp;
int year,n,i;
for (i=1; i<argc; i++) {
n=atoi(argv[i]);
sprintf (filename,"topps.%d",n);
if (!(fp=fopen(filename,"r"))){
printf("cannot open %s for reading\n",filename);
exit(3);
}
fscanf (fp, "%d%*s%*s%*s%d%d%d%d%d%d%d%d%d%d%d%d",
&year,
&cards[i],
&cards[i+1],
&cards[i+2], //this is what needs to be improved upon
&cards[i+3],
&cards[i+4],
&cards[i+5],
&cards[i+6],
&cards[i+7],
&cards[i+8],
&cards[i+9],
&cards[i+10],
&cards[i+11],
&cards[i+12]);
printf ("%d\n",year);
printf ("%d\n",cards[i+11]);
}
}
The current fscanf is just a sort of stopgap to make sure I can read and print the info. It stores up to the 12th integer and prints it. For obvious reasons I didn't want to go to the 50th, because it's pointless. Some files only have 2 or 3 numbers in them. Can anyone help guide me to a more ideal solution for reading files like this? Thanks for having a look.
Something like this does the trick:
Declare 3 new variables at the top:
char sData[10000];
char * pch;
int j = 0;
Then replace your number reading code with the snippet below:
fscanf (fp, "%d%*s%*s%*s", &year);
/* ignore the line with all the dashes (crude, but works)*/
fgets(sData, 10000, fp);
/* read all the number data in */
fgets(sData, 10000, fp);
pch = strtok (sData," ");
j = 0;
while (pch != NULL)
{
cards[j++] = atoi(pch);
pch = strtok (NULL, " ");
}
At the end of this code, cards[] should have all your numbers, and j should contain the count.
I greatly appreciate the help I got from everyone. It definitely led me down the right path. However, this is the answer to the problem that eventually worked for me:
fscanf(fp,"%*[^\n]%*c"); //Skip first two
fscanf(fp,"%*[^\n]%*c"); //lines of file
while (!feof(fp)) { //Read ints into array
fscanf(fp,"%d ",&cards[i++]);
}

How do I sscan for time?

I'm having a hard time using sscanf to scan hour and minutes from a list. Below is a small snip of the list.
1704 86 2:30p 5:50p Daily
1711 17 10:40a 2:15p 5
1712 86 3:10p 6:30p 1
1731 48 6:25a 9:30a 156
1732 100 10:15a 1:30p Daily
1733 6 2:15p 3:39p Daily
I've tried this, but it keeps getting me segmentation Fault.(I'm putting this information into structures).
for(i=0;i<check_enter;i++){
sscanf(all_flights[i],
"%d %d %d:%d%c %d:%d%c %s",
&all_flights_divid[1].flight_number,
&all_flights_divid[i].route_id,
&all_flights_divid[i].departure_time_hour,
&all_flights_divid[i].departure_time_minute,
&all_flights_divid[i].departure_time_format,
&all_flights_divid[i].arrival_time_minute,
&all_flights_divid[i].arrival_time_minute,
&all_flights_divid[i].arrival_time_format,
&all_flights_divid[i].frequency);
printf("%d ",all_flights_divid[i].flight_number);
printf("%d ",all_flights_divid[i].route_id);
printf("%d ",all_flights_divid[i].departure_time_hour);
printf("%d ",all_flights_divid[i].departure_time_minute);
printf("%c ",all_flights_divid[i].departure_time_format);
printf("%d ",all_flights_divid[i].arrival_time_hour);
printf("%d ",all_flights_divid[i].arrival_time_minute);
printf("%c ",all_flights_divid[i].arrival_time_format);
printf("%s\n",all_flights_divid[i].frequency);
}
This is how I declared it.
struct all_flights{
int flight_number;
int route_id;
int departure_time_hour;
int departure_time_minute;
char departure_time_format;
int arrival_time_hour;
int arrival_time_minute;
char arrival_time_format;
char frequency[10];
};
struct all_flights all_flights_divid[3000];
These are the results I get
0 86 2 30 p 0 50 p Daily
0 17 10 40 a 0 15 p 5
0 86 3 10 p 0 30 p 1
0 48 6 25 a 0 30 a 156
0 100 10 15 a 0 30 p Daily
0 6 2 15 p 0 39 p Daily
Look carefully at your list of output targets in sscanf. Do you see the difference between &all_flights_divid[i].departure_time_minute and all_flights_divid[i].departure_time_format? Similary for .arrival_time_format and .frequency.
What do you think the & ampersand is for? Hint: what is one way of returning multiple values from a single function call, and what does this have to do with the & ampersand?
A segmentation fault arises when your program tries to write data into memory the operating system has never instructed the CPU to make available to the program. Segmentation faults do not always occur when data is misplaced, because sometimes data is misplaced within available memory. By way of analogy, if you inadvertently put a book in the wrong place on the bookshelf, you'll not easily find the book later, but the book is still on a bookshelf and does not seem to anyone to be out of place. On the other hand, if you inadvertently put the same book in the refrigerator, well, when mother goes to get the milk she's going to issue you a segmentation fault! That's the analogy, anyway.
In general, it is hard to guess whether misplacing data will cause a segmentation fault (as misplaced into the refrigerator) or not (as misplaced on the bookshelf) until you run the program. The segmentation fault (refrigerator) is preferable because it makes the mistake obvious, so the operating system tries to give you as many segmentation faults as it can by affording the program as little memory as possible.
I am avoiding giving a 100 percent direct answer because of your "homework" tag. See if you cannot figure out the & ampersand matter, then come back here if it still does not make sense.
Your pointers are all messed up. Perhaps a series of local variables specifically for reading this stuff in would help you organize this all in your head.
int flightNum, routeID, depHour, depMin, arrHour, arrMin;
char depFormat, arrFormat;
char * freq;
for(i=0;i<check_enter;i++){
sscanf(
all_flights[i],"%d %d %d:%d%c %d:%d%c %s"
&flightNum, &routeID,
&depHour, &depMin, &depFormat,
&arrHour, $arrMin, $arrFormat,
freq
);
all_flights_divid[i].flight_number = flightNum;
all_flights_divid[i].route_id = routeID;
all_flights_divid[i].departure_time_hour = depHour;
all_flights_divid[i].departure_time_minute = depMin;
all_flights_divid[i].departure_time_format = depFormat;
all_flights_divid[i].arrival_time_hour = arrHour;
all_flights_divid[i].arrival_time_minute = arrMin;
all_flights_divid[i].arrival_time_format = arrFormat;
strcpy(all_flights_divid[i].frequency, freq);
}

Resources