What is the file format/encoding of this binary data stream - file-format

I have some binary blobs (blob from MySQL). These are suppose to be an audio stream recorded using JS web-app.
I took one of these blobs and save it as a.wtf file. When I ran strings a.wtf, I get some useful information.
webmB
QTmuxingAppLibWebM-0.0.1WA
QTwritingAppLibWebM-0.0.1
A_OPUSc
OpusHead
OPUS
...
I also tried the following in the terminal (tips on google).
[dilawars#chutki data (master)]$ mkvextract a.wtf tracks 0:audio.opus
Error: (mkvextract) The file 'a.wtf' could not be opened for reading: Not a valid Matroska file (no segment/level 0 element found).
Download a.wtf.
Any help is very much appreciated? Ideally, I'd like to convert them to WAV format.
Update
I used this tool.
[dilawars#chutki data (master)]$ hachoir-metadata a.wtf
[err!] Unable to parse file: a.wtf

Thanks to the tip by #bryc, I managed to find a solution. The data in MySQL is in base64 encoding (uploaded file a.wtf is already in binary format). I decode them back to a binary stream and saved it as a.webm file. After that, I ran the following command.
$ ffmpeg -i a.webm -ac 1 -f wav -vn -ar 20500 a.wav

Related

How to save a .zabw file (compressed abiword)?

I want to save a file in .zabw format, I can only find answers on "how to open .zabw files".
I found out that .zabw is a native Abiword format. It is gzip compressed .abw (https://www.abisource.com/help/en-US/info/infoformats.html).
When saving a file in Abiword it shows an option in "Save file as type: Abiword(.abw, .zabw, abw.gz)" but the file is saved as file-name.abw and not file-name.zabw.
So when I use gzip to compress an .abw file does it become filename.abw.gz?
gzip file-name.abw
Are .zabw and abw.gz same?
Also, how can I obtain a file in .zabw format?
Abiword does not save files to .zabw, but it can read them. It can also read .abw.gz files, which is the same as an .zabw file (both are an XML file compressed in gzip format).
When you open an .zabw file, Abiword will read it, but if you modify and save that file, it will convert it into an uncompressed XML file and it won't change the filename, which is unfortunate (and that should be considered a bug). The help page recommends to only use .zabw to send the document over the internet, in other words, they tell you to compress the file to transfer it, but edit and work with an uncompressed file.

Check contents of RC file

I am trying to investigate the working of RC file and hence stored the file in hadoop cluster using row group size as 3 bytes to ensure my data is stored in 2-3 rowgroups.
After loading, inorder to check how the contents are organized in my file, I downloaded the file to be in RC file format and used xxd /Path/To/Downloaded/File to open it. The content which was in hexadecimal format is displayed but I hope there was some other format too in the same file due to which we are not able to check content.
The file in text and binary format opened using xxd is as follows;
Could someone help me understanding the contents of file in RC format.
Thanks,
Sree
There is hive utility rcfilecat to read RC file. Something like:
ggk#hadoop4:~/Downloads$ hive --rcfilecat 000000_0
References:
Documentation
Java doc
I wanted to see the file content as is. rcfilecat deserializes the data and rearranges in record format. I used the file to see contents.
sudo xxd /path/to/downloaded/file
Thanks,
Sree

FFmpeg decoding .mp4 video file

I'm working on a project that needs to open .mp4 file format, read it's frames 1 by 1, decode them and encode them with better type of lossless compression and save them into a file.
Please correct me if i'm wrong with order of doing things, because i'm not 100% sure how this particular thing should be done. From my understanding it should go like this:
1. Open input .mp4 file
2. Find stream info -> find video stream index
3. Copy codec pointer of found video stream index into AVCodecContext type pointer
4. Find decoder -> allocate codec context -> open codec
5. Read frame by frame -> decode the frame -> encode the frame -> save it into a file
So far i encountered couple of problems. For example, if i want to save a frame using av_interleaved_write_frame() function, i can't open input .mp4 file using avformat_open_input() since it's gonna populate filename part of the AVFormatContext structure with input file name and therefore i can't "write" into that file. I've tried different solution using av_guess_format() but when i dump format using dump_format() i get nothing so i can't find stream information about which codec is it using.
So if anyone have any suggestions, i would really appreciate them. Thank you in advance.
See the "detailed description" in the muxing docs. You:
set ctx->oformat using av_guess_format
set ctx->pb using avio_open2
call avformat_new_stream for each stream in the output file. If you're re-encoding, this is by adding each stream of the input file into the output file.
call avformat_write_header
call av_interleaved_write_frame in a loop
call av_write_trailer
close the file (avio_close) and clear up all allocated memory
You can convert a video to a sequence of losses images with:
ffmpeg -i video.mp4 image-%05d.png
and then from a series of images back to a video with:
ffmpeg -i image-%05d.png video.mp4
The functionality is also available via wrappers.
You can see a similar question at: Extracting frames from MP4/FLV?

combine a binary file and a .txt file to a single file in python

I have a binary file (.bin) and a (.txt) file.
Using Python3, is there any way to combine these two files into one file (WITHOUT using any compressor tool if possible)?
And if I have to use a compressor, I want to do this with python.
As an example, I have 'file.txt' and 'file.bin', I want a library that gets these two and gives me one file, and also be able to un-merge the file.
Thank you
Just create a tar archive, a module that let's you accomplish this task is already bundled with Cpython, and it's called tarfile.
more examples here.
there are a lot of solutions for compressing!
gzip or zlib would allows compression and decompression and could be a solution for your problem.
Example of how to GZIP compress an existing file from [http://docs.python.org]:
import gzip
f_in = open('file.txt', 'rb')
f_out = gzip.open('file.txt.gz', 'wb')
f_out.writelines(f_in)
f_out.close()
f_in.close()
but also tarfile is a good solution!
Tar's the best solution to get binary file.
If you want the output to be a text, you can use base64 to transform binary file into a text data, then concatenate them into one file (using some unique string (or other technique) to mark the point they were merged).

How to dump raw RTSP stream to file?

Is it possible to dump a raw RTSP stream to file and then later decode the file to something playable?
Currently I'm using FFmpeg to receive and decode the stream, saving it to an mp4 file. This works perfectly, but is CPU intensive, and will severely limit the number of RTSP streams I can receive simultaneously on my server.
I would like to save the stream to file without decoding it, and delay the decoding part to when the file needs to be opened.
Is this possible?
I have tried VLC, which is even more CPU intensive than FFmpeg. I've also looked at this question where the answer says dumping RTSP to file is not useful, and this question, where the comment below the question says "Raw RTSP content is not well suited for save and replay...", which seems to indicate that there is way.
EDIT
Here is the command I'm using for FFmpeg:
ffmpeg -i rtsp://#192.168.241.1:62159 -r 15 C:/DB_Videos/2013-04-30 17_18_34.703.mp4
If you are reencoding in your ffmpeg command line, that may be the reason why it is CPU intensive. You need to simply copy the streams to the single container. Since I do not have your command line I cannot suggest a specific improvement here. Your acodec and vcodec should be set to copy is all I can say.
EDIT: On seeing your command line and given you have already tried it, this is for the benefit of others who come across the same question. The command:
ffmpeg -i rtsp://#192.168.241.1:62156 -acodec copy -vcodec copy c:/abc.mp4
will not do transcoding and dump the file for you in an mp4. Of course this is assuming the streamed contents are compatible with an mp4 (which in all probability they are).
With this command I had poor image quality
ffmpeg -i rtsp://192.168.XXX.XXX:554/live.sdp -vcodec copy -acodec copy -f mp4 -y MyVideoFFmpeg.mp4
With this, almost without delay, I got good image quality.
ffmpeg -i rtsp://192.168.XXX.XXX:554/live.sdp -b 900k -vcodec copy -r 60 -y MyVdeoFFmpeg.avi
You can use mplayer.
mencoder -nocache -rtsp-stream-over-tcp rtsp://192.168.XXX.XXX/test.sdp -oac copy -ovc copy -o test.avi
The "copy" codec is just a dumb copy of the stream. Mencoder adds a header and stuff you probably want.
In the mplayer source file "stream/stream_rtsp.c" is a prebuffer_size setting of 640k and no option to change the size other then recompile. The result is that writing the stream is always delayed, which can be annoying for things like cameras, but besides this, you get an output file, and can play it back most places without a problem.

Resources