The problem
I am writing a simple program in C language which makes use of libavcodec (https://github.com/Dr-Noob/framepos/blob/master/framepos.c)
I have been working with h264 videos. While the decoding works, I can see that it's very slow, since it only uses 1 CPU core (I checked it with top). On the other hand, I know that ffmpeg, which uses the same libavocdec installed in my system, makes use of the parallel h264 decoder. I can test it with:
ffmpeg -c:v h264 -i test.mkv -f null -
With top I can see that it's running in parallel, and the speed is noticeably faster. I would like a solution that always gives me the possibility of decoding the video using all the CPU cores, not only in the case of the h264 codec.
My research so far
Looking at the ffmpeg code, one can see that, to obtain the AVCodec, it uses the function find_codec_or_die. This will end up using avcodec_find_decoder_by_name. Actually, if in my program I use this function, asking for the h264 decoder, I still get the sequential version. Moreover, using gdb in ffmpeg I have seen that the AVCodec in ffmpeg is called ff_h264_decoder, while in my code, gdb does not know which specific type is the codec. The ff suffix makes me think that this one is the parallel decoder (because it looks like ff has something to do with parallel in the ffmpeg context (https://ffmpeg.org/doxygen/2.7/pthread__frame_8c.html)). However, it seems that I am unable to get this codec.
What can I do to decode video in parallel using libavcodec in C?
Posting gkv311 comment as an answer for future reference.
AVCodec does not have multi-threading functionality. It is stored inside AVCodecContex. So, a posible scheme to run the codec in parallel:
AVCodec *codec = avcodec_find_decoder
AVCodecContext *ctx = avcodec_alloc_context3
ctx->thread_count = n_threads;
ctx->thread_type = FF_THREAD_FRAME;
avcodec_open2(ctx, fmt_ctx->video_codec, NULL)
Related
I am currently working on the gstreamer tutorials, in particular the one about playback speed adjustments. I pasted the example code into a file which I compiled with the flags pkg-config --cflags --libs gstreamer-1.0 (my gstreamer version is 1.20.5)
I tried to change the playback rates using the keys S / s, got the corresponding prints (Current rate: 0.5 etc.) but the playback speed stayed constant at 1. I thought that the failure to change the playback speed was due to the source being a remote file, so I changed the code to use local files (as command line arguments) instead:
gchar buffer[4096];
g_snprintf(buffer, 4096, "playbin uri=file://%s", argv[1]);
/* Build the pipeline */
data.pipeline = gst_parse_launch(buffer, &error);
I also switched from the video sink to an audio sink (audio is sufficient for my cases).
I then noticed that whether or not the rate changed works is (apparently) up to the type of the file I am opening: When I open a local ogg file, the playback speed changes, when I open an mp3 instead, nothing happens.
Is this a bug in gstreamer, or do I need a more sophisticated pipeline in order to get the approach to work with different media types (local files would be sufficient for my needs)?
Edit: Complete code, sample mp3
We have written a short code in C code to read a video file, using common libraries as libavcodec, libavformat, etc.
The code is running smoothly but only using the CPU resources. We'd need to run the code on the GPU (Nvidia GeForce 940MX and 1080Ti). Is there a way to force the code to be run on the GPU?
While using the command line (e.g., ffmpeg -hwaccel cuvid -i vid.mp4 out.avi) things are fine, we are not able to have it working on the GPU from the source code.
We are working with Ubuntu 18.04, and ffmpeg correctly compiled with CUDA 9.2
There are pretty good examples for using libav (ffmpeg) for encoding and decoding video at https://github.com/FFmpeg/FFmpeg/tree/master/doc/examples.
For what you need is demuxing_decoding.c example and change the lines 166 which is:
/* find decoder for the stream */
dec = avcodec_find_decoder(st->codecpar->codec_id);
with
/* find decoder for the stream */
if (st->codecpar->codec_id == AV_CODEC_ID_H264)
{
dec = avcodec_find_decoder_by_name("h264_cuvid");
}
else if (st->codecpar->codec_id == AV_CODEC_ID_HEVC)
{
dec = avcodec_find_decoder_by_name("hevc_cuvid");
}
else
{
dec = avcodec_find_decoder(st->codecpar->codec_id);
}
add/change lines for other formats. And make sure your FFmpeg compiled with --enable-cuda --enable-cuvid
In my tests I got error comes from line 85: because nvdec (hevc_cuvid) uses p010 internal format for 10bit (input is yuv420p10). Which means decoded frame will be either NV12 pixel format or P010 depending on bit depth. I hope you are familiar with pixel formats.
Hope that helps.
using this code example (dranger - ffmpeg):
https://github.com/arashafiei/dranger-ffmpeg-tuto/blob/master/tutorial03.c
and dranger tutorial for ffmpeg:
http://dranger.com/ffmpeg/tutorial02.html
The video runs as fast as possible but it makes sense because there is no timer and we just extract the frames as soon as we have them ready. But for some reason, the sound also runs as fast as possible even though he says that it shouldn't.
I'm using mac os x (Maybe that has something to do with it).
Any suggestions?
Try adding:
aCodecCtx = pFormatCtx->streams[audioStream]->codec;
> aCodecCtx->request_sample_fmt = AV_SAMPLE_FMT_S16;
I am transcoding a video using FFMPEG API in c code.
I am trying to set the video bit rate using the ffmpeg API as shown below:
ovCodecCtx->bit_rate = 100 * 1000;
The Encoder I am using is libx264.
But this parameter is not taken into effect and the resulting video quality is very bad.
I have even tried setting related parameters like rc_min_rate, rc_max_rate, etc.. but the video quality is still very low as these related parameters are not taken into effect.
Could any expert tell how one can set the bit rate correctly using the FFMPEG API?
Thanks
I have found the solution to my problem. In fact somebody who was facing the same problem has posted the solution in ffmpeg(libav) user forum. This seems to work in my case too. I am posting the answer to my own question so that other users facing similar issue might benefit from this post.
Problem:
Setting the Video Bit Rate programmatically for the H264 Video Codec was not honoured by the libx264 Codec. Even though it was working for MPEG1, 2 and MPEG4 video codecs, this setting was not recognised for H264 Video Codec. And the resulting video quality was very bad.
Solution:
We need to set the pts for the decoded/resized frames before they are fed to encoder.
The person who found the solution has gone through ffmpeg.c source and was able to figure this out. We need to first rescale the AVFrame's pts from the stream's time_base to the codec time_base to get a simple frame number (e.g. 1, 2, 3).
pic->pts = av_rescale_q(pic->pts, ost->time_base, ovCodecCtx->time_base);
avcodec_encode_video2(ovCodecCtx, &newpkt, pic, &got_packet_ptr);
And when we receive back the encoded packet from the libx264 codec, we need to rescale the pts and dts of the encoded video packet to the stream time base
newpkt.pts = av_rescale_q(newpkt.pts, ovCodecCtx->time_base, ost->time_base);
newpkt.dts = av_rescale_q(newpkt.dts, ovCodecCtx->time_base, ost->time_base);
Thanks
I am implementing the operation of encoding video with TI DM365 mpeg4 encoder and containerizing it with ffmpeg mp4 container using a dummy FMP4 codec to produce headers and footers. While the container is proven to be working correctly using similar Intel based mpeg4 encoder, the dm365 gives a mosaic result if P frames are used at all. Using only I frames works, but I would like to minimize amount of data stored.
The example of the result can be viewed here. Settings are 1-Iframe, 9-Pframes
TI developers didn't answer my question regarding this in 2 days, so I am trying to get help here.
This may help, a TI data sheet on the various settings/parameters and their effect. Apologies if it is telling you stuff you already know...
TI Data Sheet spraba9.pdf