I am trying to write a GStreamer (0.10.34) plugin. I need manipulate an incoming image. I have my Sink caps set as "video/x-raw-yuv" so I know I'll be getting video.
I am having trouble in understanding how to use the GstBuffer, more specifically:
How do I get the bits per pixel?
Given the bpp, how do I determine the dimensions of the buffer?
I am currently elbows deep in 0.10.34 core documentation reading about GstStructure and GstQuarks... I think I'm in the wrong area.
As always, thanks for any advice.
After some source code hunting (jpegenc), I found the BaseLib plugins, most importantly GstVideo. This gives you the function gst_video_format_parse_caps
GstVideoFormat seems to be what you use to parse incoming video information.
Related
I want to achieve deep learning-based video compression. But it's difficult to get the intermediate results. So I want to ask if there are some convenient methods.
Could you specify your question 'intermediate results'?
If you mean reconstructed frame of VTM, you can get a buffer from picture class.
EncGOP encodes every frame in GOP and executes in-loop filters, therefore you can get intermediate frame from EncGOP while encoding.
At decoder side, you can get same buffer at DecLib.
I hope this answer helps you.
Using Windows API, I want to implement something like following:
i.e. Getting current microphone input level.
I am not allowed to use external audio libraries, but I can use Windows libraries. So I tried using waveIn functions, but I do not know how to process audio input data in real time.
This is the method I am currently using:
Record for 100 milliseconds
Select highest value from the recorded data buffer
Repeat forever
But I think this is way too hacky, and not a recommended way. How can I do this properly?
Having built a tuning wizard for a very dated, but well known, A/V conferencing applicaiton, what you describe is nearly identical to what I did.
A few considerations:
Enqueue 5 to 10 of those 100ms buffers into the audio device via waveInAddBuffer. IIRC, when the waveIn queue goes empty, weird things happen. Then as the waveInProc callbacks occurs, search for the sample with the highest absolute value in the completed buffer as you describe. Then plot that onto your visualization. Requeue the completed buffers.
It might seem obvious to map the sample value as follows onto your visualization linearly.
For example, to plot a 16-bit sample
// convert sample magnitude from 0..32768 to 0..N
length = (sample * N) / 32768;
DrawLine(length);
But then when you speak into the microphone, that visualization won't seem as "active" or "vibrant".
But a better approach would be to give more strength to those lower energy samples. Easy way to do this is to replot along the μ-law curve (or use a table lookup).
length = (sample * N) / 32768;
length = log(1+length)/log(N);
length = max(length,N)
DrawLine(length);
You can tweak the above approach to whatever looks good.
Instead of computing the values yourself, you can rely on values from Windows. This is actually the values displayed in your screenshot from the Windows Settings.
See the following sample for the IAudioMeterInformation interface:
https://learn.microsoft.com/en-us/windows/win32/coreaudio/peak-meters.
It is made for the playback but you can use it for capture also.
Some remarks, if you open the IAudioMeterInformation for a microphone but no application opened a stream from this microphone, then the level will be 0.
It means that while you want to display your microphone peak meter, you will need to open a microphone stream, like you already did.
Also read the documentation about IAudioMeterInformation it may not be what you need as it is the peak value. It depends on what you want to do with it.
I want to print out some attributes of video frames: I've looked into AVFrame struct, but only found the following disappointments:
attribute_deprecated short * dct_coeff
attribute_deprecated uint32_t * mb_type
It seems to me everything I am interested in is already obsolete. Btw, I didn't find
int16_t(*[2] motion_val )[2]
attribute in the actual frame I captured. My question is: how can i get access to those attributes such as dct_coeff or motion_vector or mb_type of a frame at all?
See av_frame_get_side_data(frame,AV_FRAME_DATA_MOTION_VECTORS) for the motion vectors. The other two have no replacement. The documentation states that they're mpeg-specific and using internal implementation details, which is why no replacement was provided.
(Don't forget to set avctx->flags2 & AV_CODEC_FLAG2_EXPORT_MVS, otherwise it's not exported.)
For the two with no replacement, I understand you might want that type of information if you're e.g. writing a stream analyzer, but FFmpeg really doesn't provide a stream-analyzer-level API right now. They could - if there's a more generic API - obviously be added as a separate side-data type. If you want that, you should probably become a FFmpeg developer and work on a broader API that is not MPEG-specific (e.g. does not use internal macros for mb_type), possibly even implement it for other codecs. In any other case, I don't really see why you would want that information. Can you elaborate?
I'm trying to save color of vertices using vcglib but failed. Even if I read a file in and save it out without doing anything, the color of the original file is lost.
Here is the code I wrote:
vcg::tri::io::ImporterPLY<MyMesh>::Open(*srcMesh,"bunny.ply");
vcg::tri::io::ExporterPLY<MyMesh>::Save(*srcMesh,"out.ply");
After doing this, out.ply has no color while the source ply bunny.ply does.
Could anybody give me some sample code to make this thing done?
Thank you!
I had the exact same problem a couple of weeks ago.
After spending some time with the debugger and browsing through lots of source code, I discovered that the the open and save methods need to share an int mask. This allows the Open method to convey which attributes have been read from the original mesh (Also, make sure you've added the Colour4b attribute to your mesh definition.
int mask=0;
vcg::tri::io::ImporterPLY<MyMesh>::Open(*srcMesh,"bunny.ply",mask);
vcg::tri::io::ExporterPLY<MyMesh>::Save(*srcMesh,"out.ply",mask);
I hope that helps.
Need to use c for a project and i saw this screenshot in a pdf which gave me the idea
http://i983.photobucket.com/albums/ae313/edmoney777/Screenshotfrom2013-11-10015540_zps3f09b5aa.png
It say's you can treat each pixel of an image as a graph node(or vertex i guess) so i was wondering how
i would do this using OpenCV and the CvGraph set of functions. Im trying to do this to learn about and how
to use graphs in computer vision and i think this would be a good starting point.
I know i can add a vetex to a graph with
int cvGraphAddVtx(CvGraph* graph, const CvGraphVtx* vtx=NULL, CvGraphVtx** inserted_vtx=NULL )
and the documentation says for the above functions vtx parameter
"Optional input argument used to initialize the added vertex (only user-defined fields beyond sizeof(CvGraphVtx) are copied)"
is this how i would represent a pixel as a graph vertex or am i barking up the wrong tree...I would love to learn more about
graphs so if someone could help me by maybe posting code, links, or good ol' fashioned advice...Id be grateful=)
http://vision.csd.uwo.ca/code has an implementation on Mulit-label optimization. GCoptimization.cpp file has a GCoptimizationGridGraph class, which I guess is what you need. I am not a C++ expert, so can't still figure out how it works. I am also looking for some simpler solution.