I'm currently trying to read some star data from the BSC. I've managed to read in the header and that shows up more or less correct, but I'm having trouble reading in the star data itself. The specification states that values are stored as 4/8-byte "Real" numbers, which I assumed meant floats/doubles, but the Ascension and Declination I get are all wrong, a good bit above the trillions for one and zero for the other. The magnitude is also wrong, despite it just being an integer, which I could read fine in the header. Here's and image of the output thus far. Any know what I'm doing wrong?
Alright, after some more testing, I managed to solve my problem. The crucial step was to abandon the binary file altogether and use the ASCII file instead. I had some problems reading from it before due to how it was formatted, but I came up with a method that worked:
/* Struct to store all the attributes I'm interested in */
struct StarData_t{
char Name[11];
char SpType[21];
float GLON, GLAT, Vmag;
};
int main()
{
/* Allocate a list of the structs
(the BSC has 9110 entries) */
struct StarData_t stars[9110];
/* Open the catalog */
FILE *fptr = fopen("catalog", "r");
if(fptr != NULL){
/* Create a buffer for storing the star entries.
The ASCII file has one entry per line.
Each line has a max length of 197,
which becomes 199 with the newline and null terminator,
so I round up to 200. */
size_t star_size = 200;
char *star_buffer;
star_buffer = (char *)malloc(star_size * sizeof(char));
/* Create a buffer for reading in the numbers.
The catalog has no numbers longer than 6 characters,
So I allocate 7 to account for the newline. */
char data_buffer[7];
/* For each entry in the BSC... */
for(int i = 0; i < 9110; i++){
/* Read the line to the buffer */
getline(&star_buffer, &star_size, fptr);
/* And put the data in the matching index,
Using the data buffer to create the floats */
// GLON
strncpy(data_buffer, &(star_buffer[90]), 6);
data_buffer[6] = '\0';
stars[i].GLON = fmod(atof(data_buffer)+180, 360)-180;
// GLAT
strncpy(data_buffer, &(star_buffer[96]), 6);
data_buffer[6] = '\0';
stars[i].GLAT = atof(data_buffer);
// Vmag
strncpy(data_buffer, &(star_buffer[102]), 5);
data_buffer[5] = '\0';
stars[i].Vmag = atof(data_buffer);
// Name
strncpy(stars[i].Name, &(star_buffer[4]), 10);
stars[i].Name[10] = '\0';
// Spectral Type
strncpy(stars[i].SpType, &(star_buffer[127]), 20);
stars[i].SpType[20] = '\0';
printf("Name: %s, Long: %7.2f, Lat: %6.2f, Vmag: %4.2f, SpType: %s\n", stars[i].Name, stars[i].GLON, stars[i].GLAT, stars[i].Vmag, stars[i].SpType);
}
free(star_buffer);
}
}
Hope this is useful!
Related
I am trying to output the statistics from my program into a file. I first create some empty strings, then amend them using sprintf, this is because I need to turn floats into chars. I then write them to a file.
I sort of works, but the output .txt file only returns 4 digits of precision regardless what I specify in sprintf.
CODE:
METRIC.RESP_TIME =(( (long int )(tval_after.tv_sec*1000000 + tval_after.tv_usec) - (long int )(tval_before.tv_sec*1000000 + tval_before.tv_usec)));
METRIC.RESP_TIME = (float) METRIC.RESP_TIME/1000000;
float ave_resp_time = METRIC.RESP_TIME/R;
float ave_through = METRIC.BYTES_RECEIVED/METRIC.RESP_TIME;
FILE *fp;
char size_str [30]; //malloc(((int)strlen(DOWNLOAD_FILE)+ (int)strlen(Q[LOCAL_QUEUE_COUNT].CHORE_NAME)))];
char rate_str [30];
char through_put_str [30];
sprintf(size_str,"TOTAL BYTES RECIEVED [B]: %5.0d ", METRIC.BYTES_RECEIVED);
sprintf(rate_str,"TOTAL TIME REQUIRED [s]: %2.8f ", ave_resp_time);
sprintf(through_put_str,"AVERAGE THROUGHPUT [B/s]: %2.8f ", ave_through);
fprintf(stdout,"%d\n",METRIC.BYTES_RECEIVED);
fp = fopen( METRICS_FILE, "w");
if(NULL == fp){
printf("Could not make metrics file: error %d ",errno);
return 0;
}
fwrite(size_str,(size_t)sizeof(size_str),1,fp);
fwrite(rate_str,sizeof(rate_str),1,fp);
fwrite(through_put_str,sizeof(through_put_str),1,fp);
fclose(fp);
return 0;
OUTPUT:
TOTAL BYTES RECIEVED [B]: 5526TOTAL TIME REQUIRED [s]: 0.001AVERAGE THROUGHPUT [B/s]: 2992
Hoping to make it look like:
TOTAL BYTES RECIEVED [B]: 55264892
TOTAL TIME REQUIRED [s]: 0.0019634
AVERAGE THROUGHPUT [B/s]: 29929054
You need an explicit new line character; fwrite() does not add one, and also you can directly use fprintf() instead of what you have.
To fix your code, do this
sprintf(size_str, "TOTAL BYTES RECIEVED [B]: %5.0d\n", METRIC.BYTES_RECEIVED);
/* ^ this will break the line */
the strcat(size_str, "\0"); is not needed.
You don't have to do all this, because you can just
fprintf(fp, "TOTAL BYTES RECIEVED [B]: %5.0d\n", METRIC.BYTES_RECEIVED);
/* ^ this will break the line */
As mentioned by #Jonathan Leffler, use well sized buffers rather than hoping that 30 is ample.
One method that has worked well is to size the buffer per the sprintf()
// char size_str [30];
// sprintf(size_str,"TOTAL BYTES RECIEVED [B]: %5.0d ", METRIC.BYTES_RECEIVED);
#define INT_MAX_PRT (sizeof(int) * CHAR_BIT/3 + 3)
const char rcv_fmt[] = "TOTAL BYTES RECEIVED [B]: %5.0d\n";
char size_str [sizeof rcv_fmt + INT_MAX_PRT];
sprintf(size_str, rcv_fmt, METRIC.BYTES_RECEIVED);
This approach is a bit more challenging with floating point as the "%f" width could be so large.
// char rate_str [30];
// sprintf(rate_str,"TOTAL TIME REQUIRED [s]: %2.8f ", ave_resp_time);
#define FLT_MAX_PRT (1 /* sign */ + FLT_MAX_10_EXP + 1)
const char *time_fmt[] = "TOTAL TIME REQUIRED [s]: %2.8f\n";
char rate_str[sizeof time_fmt + FLT_MAX_PRT + 8];
sprintf(rate_str, time_fmt, ave_resp_time);
Still, since it is possible to mistake the needed buffer size, code could also use snprintf() to minimize the harm. But in the end, a proper size buffer is needed.
Note: added '\n' to the formats.
First things first, you should get out of the habit of using sizeof on any array. This will get you into trouble faster than you realize.
Second, you seem to be doing double-duty here. There is no need to use sprintf + fwrite at all. You should just fprintf and pass you fp as the first arg.
I am quite rusty with C and system calls and pointers in general, so this is a good refresher exercise to get back on track. All I need to do is, given a file such as this:
YYY.txt: "somerandomcharacters"
Change it to be like this:
YYY.txt: "somerandomabcdefghijklmnopqrstuvwxyzcharacters"
So all that is done is some characters added to the middle of the file. Obviously, this is quite simple, but in C you must keep track and manage the size of the file in advance before adding the additional characters.
Here is my naive try:
//(Assume a file called YYY.txt exists and an int YYY is the file descriptor.)
char ToBeInserted[26] = "abcdefghijklmnopqrstuvwxyz";
//Determine the current length of YYY
int LengthOfYYY = lseek(YYY, 0, 2);
if(LengthOfYYY < 0)
printf("Error upon using lseek to get length of YYY");
//Assume we want to insert at position 900 in YYY.txt, and length of YYY is over 1000.
//1.] Keep track of all characters past position 900 in YYY and store in a char array.
lseek(YYY, 900, 0); //Seeks to position 900 in YYY, so reading begins there.
char NextChar;
char EverythingPast900[LengthOfYYY-900];
int i = 0;
while(i < (LengthOfYYY - 900)) {
int NextRead = read(YYY, NextChar, 1); //Puts next character from YYY in NextChar
EverythingPast900[i] = NextChar;
i++;
}
//2.] Overwrite what used to be at position 900 in YYY:
lseek(YYY, 900, 0); //Moves to position 900.
int WriteToYYY = write(YYY, ToBeInserted, sizeof(ToBeInserted));
if(WriteToYYY < 0)
printf("Error upon writing to YYY");
//3.] Move to position 900 + length of ToBeInserted, and write the characters that were saved.
lseek(YYY, 926, 0);
int WriteMoreToYYY = write(YYY, EverythingPast900, sizeof(EverythingPast900));
if (WriteMoreToYYY < 0) {
printf("Error writing the saved characters back into YYY.");
}
I think the logic is sound, mostly, although there are much better ways to do it in C. I need help on my C pointers, basically, as well as the UNIX system calls. Does anyone mind walking me through how to properly implement this in C?
That's the basic idea. If you had to really conserve RAM and the file was a lot bigger you'd want to copy block by block in reverse order. But the simpler way is to read the entire thing into memory and rewrite the entire file.
also, I prefer the stream functions: fopen, fseek, fread. But the file descriptor method works.
I am writing a c program that opens a txt file and want to read the last line of the txt file.
I am not that proficient in C so bear in mind that I may not know all of the concepts in C. I am stuck at the part where I use fscanf to read all the lines of my txt file but I want to take the last line of the txt file and get the values as described below.
Here is my incomplete code:
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
FILE *sync;
void check()
{
int success; //to hold the results if the timestamps match
sync = fopen("database.txt","r");
char file[] = "database.txt";
while (fscanf(sync, "%d.%06d", &file) != EOF)
{
}
fclose(sync);
}
sample txt file:
/////// / //// ///// ///// //////////////// Time: 1385144574.787665 //////// /
/////// / //// ///// ///// //////////////// Time: 1385144574.787727 //////// /
/////// / //// ///// ///// //////////////// Time: 1385144574.787738 //////// /
/////// / //// ///// ///// //////////////// Time: 1385144574.787746 //////// /
/////// / //// ///// ///// //////////////// Time: 1385144574.787753 //////// /
The / are some words, symbols and numbers I do not want, just the numbers in sample txt as shown above
I appreciate any examples and pointing out errors I made so I can understand this much better.
Since I made some people confused about the text file, here is what it really is. This is the format it will be so I should know the length of each line. However, I will not be able to know how many lines there will be as it may be updated.
Socket: 0 PGN: 65308 Data: 381f008300000000 Time: 1385144574.787925 Address: 28
Socket: 0 PGN: 65398 Data: 0000000100000000 Time: 1385144574.787932 Address: 118
Socket: 0 PGN: 61444 Data: f07d83351f00ffff Time: 1385144574.787940 Address: 4
Socket: 0 PGN: 65266 Data: 260000000000ffff Time: 1385144574.787947 Address: 242
Socket: 0 PGN: 65309 Data: 2600494678fff33c Time: 1385144574.787956 Address: 29
Socket: 0 PGN: 65398 Data: 0000000100000000 Time: 1385144574.787963 Address: 118
Socket: 0 PGN: 61444 Data: f07d833d1f00ffff Time: 1385144574.787971 Address: 4
Socket: 0 PGN: 65398 Data: 0000000100000000 Time: 1385144574.787978 Address: 118
Socket: 0 PGN: 61443 Data: d1000600ffffffff Time: 1385144574.787985 Address: 3
Socket: 0 PGN: 65308 Data: 451f008300000000 Time: 1385144574.787993 Address: 28
Socket: 0 PGN: 65317 Data: e703000000000000 Time: 1385144574.788001 Address: 37
Again I am after the Time values (eg. 1385144574.787925) at the last line of the txt file.
Hope this helps.
Since you're after the last line of the file, and you didn't mention how large the file might be, it could be worth while to start reading the file from the end, and work your way backwards from there:
FILE *fp = fopen("database.txt", "r");
fseek(fp, 0, SEEK_END);//sets fp to the very end of your file
From there, you can use fseek(fp, -x, SEEK_CUR); where x is the number of bytes you want to go back, until you get to where you want... other than that, Jekyll's answer should work just fine.
However, to get the last line, I tend to do something like this:
FILE *fp = fopen("database.txt", "r");
char line[1024] = "";
char c;
int len = 0;
if (fp == NULL) exit (EXIT_FAILURE);
fseek(fp, -1, SEEK_END);//next to last char, last is EOF
c = fgetc(fp);
while(c == '\n')//define macro EOL
{
fseek(fp, -2, SEEK_CUR);
c = fgetc(fp);
}
while(c != '\n')
{
fseek(fp, -2, SEEK_CUR);
++len;
c = fgetc(fp);
}
fseek(fp, 1, SEEK_CUR);
if (fgets(line, len, fp) != NULL) puts(line);
else printf("Error\n");
fclose(fp);
The reasoning behind my len var is so that I can allocate enough memory to accomodate the entire line. Using an array of 1024 chars should suffice, but if you want to play it safe:
char *line = NULL;
//read line
line = calloc(len+1, sizeof(char));
if (line == NULL)
{
fclose(fp);
exit( EXIT_FAILURE);
}
//add:
free(line);//this line!
fclose(fp);
Once you've gotten that line, you can use Jekyll's sscanf examples to determine the best way to extract whatever you want from that line.
The way you are using fscanf is wrong as the actual vector of arguments needs to match what you are collecting (as you can see in the manpage). Instead of using fscanf you may consider using fgets and then filtering for what you are looking for in the latest raw with a regex through sscanf.
Note:: I collected the value in double format, you may choose the format that suits you the most for your problem (string?int.int?float?), in order to do this you should check for regex using scanf. Please come back if you cannot accomplish this task.
update:: due to some requests I wrote some few examples of different pattern matching. These should be a good starting point to fix your problems.
update::
1. I have seen that you added the pattern of your db file so we can now state that both #3 and #4 match and put the 3 here (faster).
2. I removed the feof check as for your request, but note that the check is fine if you know what you are doing. Basically you have to keep in mind that stream's internal position indicator may point to the end-of-file for the next operation, but still, the end-of-file indicator may not be set until an operation attempts to read at that point.
3. You asked to remove the char line[1024]={0,}; This instruction is used to initialize the line[1024] array which will contain the lines that you read from the file. This is needed! To know what that instruction is please see here
Code:
void check()
{
char line[1024]={0,}; // Initialize memory! You have to do this (as for your question)
int n2=0;
int n3=0;
sync = fopen("database.txt", "r");
if( sync ) {
while( fgets(line, 1024, sync) !=NULL ) {
// Just search for the latest line, do nothing in the loop
}
printf("Last line %s\n", line); //<this is just a log... you can remove it
fclose(sync);
// This will look for Time and it will discard it collecting the number you are looking for in n2 and n3
if (sscanf(line, "%*[^T]Time: %d.%d", &n2, &n3) ) {
printf( "%d.%d\n", n2, n3);
}
}
}
Example 2
if for instance you need to collect the value using two integers you will need to replace the sscanf of the example above with the following code:
unsigned int n2, n3;
if (sscanf(line, "%*[^0-9]%d.%d", &n2, &n3) ) {
printf( "%d.%d\n", n2, n3);
}
said this you should figure out how to collect other formats.
Example 3
A better regex. In case there are others number in the file before the giving pattern you may want to match on Time, so let's say that there isn't any T before. A regex for this can be:
if (sscanf(line, "%*[^T]Time: %d.%d", &n2, &n3) ) {
printf( "%d.%d\n", n2, n3);
}
The regex using sscanf can be not suitable for your pattern, in that case you need to consider the usage of gnu regex library or you can mix strstr and sscanf like I did in the following example.
Example 4
This can be useful if you don't find a common pattern. In that case you may want to trigger on the string "Time" using strstr before calling the sscanf
char *ptr = strstr( line, "Time:" );
if( ptr != NULL ) {
if (sscanf(ptr, "%*[^0-9]%d.%d", &n2, &n3) ) {
printf( "%d.%d\n", n2, n3);
}
}
* Note *
You may need to find your way to parse the file and those above can be only suggestions because you may have more specific or different patterns in your file but the instruction I posted here should be enough to give you the instruments to do the job in that case
The Overview
I am using the low-level calls in the libbzip2 library: BZ2_bzCompressInit(), BZ2_bzCompress() and BZ2_bzCompressEnd() to compress chunks of data to standard output.
I am migrating working code from higher-level calls, because I have a stream of bytes coming in and I want to compress those bytes in sets of discrete chunks (a discrete chunk is a set of bytes that contains a group of tokens of interest — my input is logically divided into groups of these chunks).
A complete group of chunks might contain, say, 500 chunks, which I want to compress to one bzip2 stream and write to standard output.
Within a set, using the pseudocode I outline below, if my example buffer is able to hold 101 chunks at a time, I would open a new stream, compress 500 chunks in runs of 101, 101, 101, 101, and one final run of 96 chunks that closes the stream.
The Problem
The issue is that my bz_stream structure instance, which keeps tracks of the number of compressed bytes in a single pass of the BZ2_bzCompress() routine, seems to claim to be writing more compressed bytes than the total bytes in the final, compressed file.
For example, the compressed output could be a file with a true size of 1234 bytes, while the number of reported compressed bytes (which I track while debugging) is somewhat higher than 1234 bytes (say 2345 bytes).
My rough pseudocode is in two parts.
The first part is a rough sketch of what I do to compress a subset of chunks (and I know that I have another subset coming after this one):
bz_stream bzStream;
unsigned char bzBuffer[BZIP2_BUFFER_MAX_LENGTH] = {0};
unsigned long bzBytesWritten = 0UL;
unsigned long long cumulativeBytesWritten = 0ULL;
unsigned char myBuffer[UNCOMPRESSED_MAX_LENGTH] = {0};
size_t myBufferLength = 0;
/* initialize bzStream */
bzStream.next_in = NULL;
bzStream.avail_in = 0U;
bzStream.avail_out = 0U;
bzStream.bzalloc = NULL;
bzStream.bzfree = NULL;
bzStream.opaque = NULL;
int bzError = BZ2_bzCompressInit(&bzStream, 9, 0, 0);
/* bzError checking... */
do
{
/* read some bytes into myBuffer... */
/* compress bytes in myBuffer */
bzStream.next_in = myBuffer;
bzStream.avail_in = myBufferLength;
bzStream.next_out = bzBuffer;
bzStream.avail_out = BZIP2_BUFFER_MAX_LENGTH;
do
{
bzStream.next_out = bzBuffer;
bzStream.avail_out = BZIP2_BUFFER_MAX_LENGTH;
bzError = BZ2_bzCompress(&bzStream, BZ_RUN);
/* error checking... */
bzBytesWritten = ((unsigned long) bzStream.total_out_hi32 << 32) + bzStream.total_out_lo32;
cumulativeBytesWritten += bzBytesWritten;
/* write compressed data in bzBuffer to standard output */
fwrite(bzBuffer, 1, bzBytesWritten, stdout);
fflush(stdout);
}
while (bzError == BZ_OK);
}
while (/* while there is a non-final myBuffer full of discrete chunks left to compress... */);
Now we wrap up the output:
/* read in the final batch of bytes into myBuffer (with a total byte size of `myBufferLength`... */
/* compress remaining myBufferLength bytes in myBuffer */
bzStream.next_in = myBuffer;
bzStream.avail_in = myBufferLength;
bzStream.next_out = bzBuffer;
bzStream.avail_out = BZIP2_BUFFER_MAX_LENGTH;
do
{
bzStream.next_out = bzBuffer;
bzStream.avail_out = BZIP2_BUFFER_MAX_LENGTH;
bzError = BZ2_bzCompress(&bzStream, (bzStream.avail_in) ? BZ_RUN : BZ_FINISH);
/* bzError error checking... */
/* increment cumulativeBytesWritten by `bz_stream` struct `total_out_*` members */
bzBytesWritten = ((unsigned long) bzStream.total_out_hi32 << 32) + bzStream.total_out_lo32;
cumulativeBytesWritten += bzBytesWritten;
/* write compressed data in bzBuffer to standard output */
fwrite(bzBuffer, 1, bzBytesWritten, stdout);
fflush(stdout);
}
while (bzError != BZ_STREAM_END);
/* close stream */
bzError = BZ2_bzCompressEnd(&bzStream);
/* bzError checking... */
The Questions
Am I calculating cumulativeBytesWritten (or, specifically, bzBytesWritten) incorrectly, and how would I fix that?
I have been tracking these values in a debug build, and I do not seem to be "double counting" the bzBytesWritten value. This value is counted and used once to increment cumulativeBytesWritten after each successful BZ2_bzCompress() pass.
Alternatively, am I not understanding the correct use of the bz_stream state flags?
For example, does the following compress and keep the bzip2 stream open, so long as I keep sending some bytes?
bzError = BZ2_bzCompress(&bzStream, BZ_RUN);
Likewise, can the following statement compress data, so long as there are at least some bytes are available to access from the bzStream.next_in pointer (BZ_RUN), and then the stream is wrapped up when there are no more bytes available (BZ_FINISH)?
bzError = BZ2_bzCompress(&bzStream, (bzStream.avail_in) ? BZ_RUN : BZ_FINISH);
Or, am I not using these low-level calls correctly at all? Should I go back to using the higher-level calls to continuously append a grouping of compressed chunks of data to one main file?
There's probably a simple solution to this, but I've been banging my head on the table for a couple days in the course of debugging what could be wrong, and I'm not making much progress. Thank you for any advice.
In answer to my own question, it appears I am miscalculating the number of bytes written. I should not use the total_out_* members. The following correction works properly:
bzBytesWritten = sizeof(bzBuffer) - bzStream.avail_out;
The rest of the calculations follow.
I have written some code, and the Vigenere encryption is used in it. This is a simple program for encrypting/decrypting any files.
#include<stdio.h>
/*
LANGUAGE: C.
STANDARD: C89.
ABOUT PROGRAM:
This is a simple program for encrypting/decrypting any files.
The size of source file coincide with size of result file.
For encryption of file are use any string key. For decrypting,
you must to use the same key, which was used for encryption.
NOTES:
The Vigenere encryption are used in it.
Info at the site: http://en.wikipedia.org/wiki/Vigen%C3%A8re_cipher.
This simple algorithm is often used at commercial products. The
Vigenere's algorithm are using a string key, and 'XOR' for
encrypting/decrypting information.
WARNING!
The use of this method, doesn't give 100% of a warranty
for protection of your information. Don't create the keys,
consisting of identical characters, for example" "aaaaa",
"zzz", "xxxx" e.t.c. - it is very feeble protection!
Don't forget your encrypting keys... :)
SYNTAX OF USING:
vigenere StringKey SourceFileName ResultFileName
where:
vigenere - program name;
StringKey - string key for encrypting/decrypting;
SourceFileName - source file name;
ResultFileName - result file name;
EXAMPLE OF USING:
vigenere "G5$hj4*df7f3+x" "c:\temp\source.txt" "c:\temp\result.txt"
*/
int main(int argc, char *args[]){
/****************************************************/
/* All variables must be defined on top in function, otherwise
some compilers can't compile this code (for example - MS
Visual Studio 2012. */
char ch; /* The next char for encrypting/decrypting. */
char *x; /* String key. */
FILE *srcFile; /* Source file. */
FILE *trgFile; /* Result file. */
/****************************************************/
/* The first argument always is a program file name. */
if (4 != argc)
return 1; /* Invalid arguments count. */
if (!*args[1] || !*args[2] || !*args[3])
return 2; /* Contains the empty argument. */
x = args[1];
if ((srcFile = fopen(args[2], "rb")) != NULL){
if ((trgFile = fopen(args[3], "wb")) != NULL){
while((ch = getc(srcFile)) != EOF){
if(!*x++)
x = args[1];
putc((ch ^= *x), trgFile);
}
fclose(trgFile);
}
else
return 4; /* Result file wasn't created. */
fclose(srcFile);
}
else
return 3; /* Source file wasn't opened. */
return 0; /* Successful operation. */
}
But this code does not always work well. I don't understand why it occurs. I do XOR for each byte. I have tested this code on such TXT files. Where is my mistake?
char ch;
/* ... */
while((ch = getc(srcFile)) != EOF)
ch must be an int. EOF is defined as a negative int.
In addition to ouah's answer, the pointer value increment looks off.
your if statement, if(!*x++), is bad for two reasons:
By doing an increment before the actual XOR operation, you're skipping the first character of your key in the initial loop.
There's no point in incrementing the pointer if you already reach the null-terminating character.
The better code would be:
while((ch = getc(srcFile)) != EOF){
putc((ch ^= *x), trgFile);
if(!*++x)
x = args[1];
}