Hi fellow stack overflowers,
I'm currently parsing a file which both contains text and binary data. Currently, I'm reading the file in following manner:
QTextStream in(&file);
int index = 0;
while(!in.atEnd()) {
if (index==0) {
QString line = in.readLine(); // parse file here
} else {
QByteArray raw_data(in.readAll().toAscii());
data = new QByteArray(raw_data);
}
index++;
}
where data refers to the binary data I'm looking for. I'm not sure if this is what I want, since the QString is encoded into ascii and I have no idea if some bytes are lost.
I checked the documentation, and it recommends using a QDataStream. How can I combine both approaches, i.e. read lines with an encoding and also read the binary dump, after one line break?
Help is greatly appreciated!
This will do what you want.
QTextStream t(&in);
QString line;
QByteArray raw_data;
if(!in.atEnd()) {line = t.readLine();}
in.reset();
int lineSize = line.toLocal8Bit().size() + 1;
in.seek(lineSize);
if(!in.atEnd())
{
int len = in.size() - lineSize;
QDataStream d(&in);
char *raw = new char[len]();
d.readRawData(raw, len);
raw_data = QByteArray(raw, len);
delete raw;
}
PS: if file format is yours, it will be better to create file with QDataStream and write data with <<, read with >>. This way you can store QByteArray and QString in file without such problems.
Related
Input: a stream of ogg/vorbis coming from an encoder chip of an embedded system.
Problem: create output chunks of one second without transcoding.
Issue: the stream is being read "in the middle", so the first page with BOS (Beginning of Stream) is not available. Since the encoder chip has always the same parameters, I'd like to recreate the BOS page using the BOS page of a stream that was read from the start (reference stream).
I am trying to use vcut. I modified it so that it creates infinite chunks of one second. It was easy, and it works with files and streams with BOS.
I also hacked it so that I wrote to a file the first pages of the reference stream and then read them before reading the production stream with no BOS. In this way, vs->headers are populated. When I detect a page serial number change, I change it so that vcut and libogg do not freak:
int process_page(vcut_state *s, ogg_page *page) {
...
else if(vs->serial != ogg_page_serialno(page))
{
// fprintf(stderr, _("Multiplexed bitstreams are not supported.\n"));
vs->stream_in.serialno = ogg_page_serialno(page);
vs->serial = ogg_page_serialno(page);
vs->granulepos = -1;
vs->initial_granpos = 0;
// ogg_stream_init(&vs->stream_in, vs->serial);
// vorbis_info_init(&vs->vi);
// vorbis_comment_init(&vs->vc);
s->vorbis_init = 1;
}
However, this gigantic hack does not work. How to solve this issue?
It actually works: see VS1053 split ogg.
What I needed to do was to consider that starting reading in the middle of the stream, granulepos was naturally high. So it was mine logical mistake.
In process_audio_packet, I added:
int process_audio_packet(vcut_state *s,
vcut_vorbis_stream *vs, ogg_packet *packet)
{
...
if(packet->granulepos >= 0)
{
if (!firstNonZeroGranule) { // my addition
firstNonZeroGranule = 1;
vs->initial_granpos = packet->granulepos - bs;
if(vs->initial_granpos < 0)
vs->initial_granpos = 0;
} else if(vs->granulepos == 0 && packet->granulepos != bs) {
...
We use testing equipment (1995 year manufacturing) powered by MS DOS. Analog-digital converter records information in the file.
In [picture1] is shown the structure of that file.
In [picture2] is shown the oscillogram that constructed according to the data from the file (program for opening the file on MS DOS).
Below I placed link to this file (google drive).
This file contains the data that need for me - the massive of point of oscillogram. I want have opportunities to keep, analyze and print this chart on Windows or Linux (not MS DOS). So I need to extract data from the file.
But I can't make it. And no program (known to me) can't open this file. I analyzed a few first byte and they point to program 'TRAS v4.99'. This program is on MS DOS.
But I really hope, that it is really to get data without this program.
P.S. If anyone will say it is impossible - it is will well too because I haven't found point of view yet:)
Thank you for your time! Best regards!
LINK TO FILE ON GOOGLE DISK - 00014380.K00
STRUCTURE OF FILE
OPENING FILE VIA PROGRAM IN MS DOS
Here is an idea on how you can tackle this problem. Since the format is relatively well specified in the handbook you can use the Java programming language for example with something like java.io.RandomAccessFile to read arrays of bytes. These arrays of bytes can then be converted to Java primitive types OR to string according to the data type. After this conversion you can the print out the data in a human readable format.
Below you can find some sample code to give you an idea of what you could do with this approach (I have not tested the code, it is not complete, it is just to give you an idea of what you can do):
public static void readBinaryfile() throws IOException {
java.io.RandomAccessFile randomAccessFile = new RandomAccessFile("test.bin", "r");
byte[] addKenStrBytes = new byte[12];
randomAccessFile.read(addKenStrBytes);
String addKenStr = new String(addKenStrBytes, "UTF-8");
// TODO: Do something with addKenStr.
System.out.println(addKenStr);
byte[] kopfSizeBytes = new byte[2];
randomAccessFile.read(kopfSizeBytes);
// TODO: Do something with kopfSizeBytes
System.out.println(convertToInt(kopfSizeBytes));
byte[] addRufNrCounterBytes = new byte[6];
randomAccessFile.read(addRufNrCounterBytes);
long addRufNrCounter = convertToLong(addRufNrCounterBytes);
// TODO: Do something with addRufNrCounter
System.out.println(addRufNrCounter);
byte[] endAdrBytes = new byte[4];
randomAccessFile.read(endAdrBytes);
// TODO: Do something with endAdrBytes
System.out.println(convertToLong(endAdrBytes));
// Continue here and after you reached the end of the record repeat until you reached the end off the file
}
private static int convertToInt(byte[] bytes) {
if(bytes.length > 4) {
throw new IllegalArgumentException();
}
int buffer = 0;
for(byte b : bytes) {
buffer |= b;
buffer = buffer << 8;
}
return buffer;
}
private static long convertToLong(byte[] bytes) {
if(bytes.length > 8) {
throw new IllegalArgumentException();
}
long buffer = 0L;
for(byte b : bytes) {
buffer |= b;
buffer = buffer << 8;
}
return buffer;
}
Note that fields with more than 8 bytes need to be most probably converted to strings. This is not complete code, just an example to give you an idea on how you can tackle this problem.
I've been trying to work this out for a while.
Reading a text file.
Using this code the formatting gets replaced with the getline...but the time to load the file is simply too long...
std::string line = "";
std::string file = "";
std::ifstream filepath(path);
if (filepath.is_open())
{
while (std::getline (filepath,line) )
{
file = file + line + "\r\n";
}
filepath.close();
}
Using this code the time to load the file is around 10X faster, but the formatting is lost:
std::ifstream in(path);
std::stringstream stream;
stream << in.rdbuf();
std::string file(stream.str());
Is it possible to get the speed of the second method with the formatting of the first...? Or better yet faster speeds and no change in formatting?
I had considered trying not to keep loading the same string in the first example at each getline in the loop but early attempts didn't seem to help.
Okay after checking again, not sure what this is called but this string optimization took care of the delay and the formatting... whew!
If anyone knows how to make it even faster I'd love to learn how...
std::string line = "";
std::string file = "";
std::string tmp_str = "";
std::ifstream filepath(path);
unsigned int c = 0;
if (filepath.is_open())
{
while (std::getline (filepath,line) )
{
tmp_str = tmp_str + line + "\r\n";
c++;
if (c > 100)
{
file = file + tmp_str;
tmp_str = "";
c = 0;
}
}
if (c != 0)
{
file = file + tmp_str;
}
filepath.close();
I'm trying to write a wchar array to a file in C, however there is some sort of corruption and unrelevant data like variables and paths like this
c.:.\.p.r.o.g.r.a.m. .f.i.l.e.s.\.m.i.c.r.o.s.o.f.t. .v.i.s.u.a.l. .s.t.u.d.i.o. 1.0...0.\.v.c.\.i.n.c.l.u.d.e.\.x.s.t.r.i.n.g..l.i.s.t...i.n.s.e.r.t
are written on to the file along with the correct data (example) I have confirmed that the buffer is null-terminated and contains proper data.
Heres my code:
myfile = fopen("logs.txt","ab+");
fseek(myfile,0,SEEK_END);
long int size = ftell(myfile);
fseek(myfile,0,SEEK_SET);
if (size == 0)
{
wchar_t bom_mark = 0xFFFE;
size_t written = fwrite(&bom_mark,sizeof(wchar_t),1,myfile);
}
// in another func
while (true)
{
[..]
unsigned char Temp[512];
iBytesRcvd = recv(sclient_socket,(char*)&Temp,iSize,NULL);
if(iBytesRcvd > 0 )
{
WCHAR* unicode_recv = (WCHAR*)&Temp;
fwrite(unicode_recv,sizeof(WCHAR),wcslen(unicode_recv),myfile);
fflush(myfile);
}
[..]
}
What could be causing this?
recv() will not null-terminate &Temp, so wcslen() runs over the bytes actually written by recv(). You will get correct results if you just use iBytesReceived as byte count for fwrite() instead of using wcslen() and hoping the data received is correctly null-terminated (wide-NULL-terminated, that is):
fwrite(unicode_recv, 1, iBytesReceived, myfile);
I want to delete a string from a particular position in the file. Is thre a function to do that?
Can I delete last line of a file through a function?
You have two option
To read whole file, remove what you need and write it back
If the file is big, read file sequentially, remove given part, and move content after that forward
No there is no such function that will let you do this directly on a file.
You should load up the file content in memory and modify there and write back to file.
I don't feel like looking up all the io functions, so here's pseudo-c on how to implement option 2 of ArsenMkrt's answer
char buffer[N]; // N >= 1
int str_start_pos = starting position of the string to remove
int str_end_pos = ending position of the string to remove
int file_size = the size of the file in bytes
int copy_to = str_start_pos
int copy_from = str_end_pos + 1
while(copy_from < file_size){
set_file_pos(file, copy_from)
int bytes_read = read(buffer, N, file)
copy_from += bytes_read
set_file_pos(file, copy_to)
write(buffer, file, bytes_read)
copy_to += bytes_read
}
truncate_file(file,file_size - (str_end_pos - str_start_pos + 1))
something to that effect