H264 decoding using ffmpeg - c

I'm trying to decode a video stream with ffmpeg library, that's how I do it basically:
void video_decode(const char *filename)
{
AVCodec *codec;
AVCodecContext *c= NULL;
int frame_count=0;
FILE *f;
AVFrame *frame;
uint8_t inbuf[INBUF_SIZE + AV_INPUT_BUFFER_PADDING_SIZE];
AVPacket avpkt;
av_init_packet(&avpkt);
memset(inbuf + INBUF_SIZE, 0, AV_INPUT_BUFFER_PADDING_SIZE);
printf("Decoding video file...\n");
/* find the h264 video decoder */
codec = avcodec_find_decoder(AV_CODEC_ID_H264);
c = avcodec_alloc_context3(codec);
c->bit_rate = 400000;
c->width = 1920;
c->height = 1080;
if (avcodec_open2(c, codec, NULL) < 0) {
fprintf(stderr, "Could not open codec\n");
exit(1);
}
frame = av_frame_alloc();
for (;;) {
avpkt.size = fread(inbuf, 1, INBUF_SIZE, f);
if (avpkt.size == 0)
break;
avpkt.data = inbuf;
while(avpkt.size > 0){
int len, got_frame;
len = avcodec_decode_video2(c, frame, &got_frame, &avpkt);
if (len < 0) {
fprintf(stderr, "Errr while decding frame %d\n", frame_count);
exit (1);
}
if (got_frame) {
//Print out frame information..
}
if (avpkt.data) {
avpkt.size -= len;
avpkt.data += len;
}
}
}
}
But I got the following outputs:
Decoding video file...
[h264 # 0x303c040] decode_slice_header error
[h264 # 0x303c040] decode_slice_header error
[h264 # 0x303c040] decode_slice_header error
[h264 # 0x303c040] no frame!
Errr while decding frame 0
Obviously the initiation of codec was incomplete. Do you have experience with h264 api? Any help would be appreciated.

You cant just put a random number (INBUF_SIZE ) of bytes into an AV packet. It should be a full AU or NALU, and if it is not annex B, you must first set the extra data field. For your case, I would recommend use libavformat to open the file and read the AVPackets.

The answer to this question is given under demuxing_decoding.c example of ffmpeg.
One can compile this file independently as below, if your ffmpeg installation directory is default "usr/local/".
gcc -I/usr/local/include -c demuxing_decoding.c -o demuxing_decoding.o
gcc demuxing_decoding.o -pthread -L/usr/local/lib -lavdevice -lavfilter -lpostproc -lavformat -lavcodec -lva-x11 -lva -lxcb-xfixes -lxcb-render -lxcb-shape -lxcb -lX11 -lx264 -lpthread -ldl -lz -lswresample -lswscale -lavutil -lm -o demuxing_decoding
In case, if the installation directory is changed to say something like "ffmpeg_ins_dir" replace "usr/local/" in above commands with "path upto ffmpef_ins_dir/".
Run the binary as below.
./demuxing_decoding test.h264 v_out.yuv a_out.yuv
Note : This example has many things you may not need at all, remove things that isn't needed and recompile it again.

may be you should try to use AVCODECPARSER between your fread() and avcodec_decode_video2();
av_parser_parse2(). make your rawdata change into h264 fromat data to feed your decoder.
C code like this :
#
while (1)
{
cur_size = fread(in_buffer, 1, in_buffer_size, fp_in);
cur_ptr = in_buffer;
while (cur_size > 0)
{
int len = av_parser_parse2(pCodecParserCtx, pCodecCtx, &packet.data, &packet.size, cur_ptr, cur_size, AV_NOPTS_VALUE, AV_NOPTS_VALUE, AV_NOPTS_VALUE);
cur_ptr += len;
cur_size -= len;
ret = avcodec_decode_video2(pCodecCtx, pFrame, &got_picture, &packet);
if (ret < 0)
{
printf("Decode Error\n");
return ret;
}
if (got_picture)
{
printf("Succeed to decode 1 frame!\n");
}
}
inspired by this :
https://blog.csdn.net/leixiaohua1020/article/details/42181571

Related

Libav: Protocol not found

I am trying to follow a libav tutorial on how to create a video player. My "Hello World" program consists of dumping the format of the video passed on the command line. The code is as follows:
#include <stdio.h>
#include <libavformat/avformat.h>
int main(int argc, char *argv[]) {
AVFormatContext *pFormatCtx = NULL;
int ret;
char err[128];
if(argc < 2) {
printf("Please provide a movie file\n");
return -1;
}
ret = avformat_open_input(&pFormatCtx, argv[1], NULL, NULL);
av_strerror(ret, err, 128);
// Open video file
if(ret != 0)
{
printf("Failed to open file: %s\n", err);
return -1; // Couldn't open file
}
// Retrieve stream information
if(avformat_find_stream_info(pFormatCtx, NULL) < 0)
{
printf("Failed to obtain stream information\n");
return -1; // Couldn't find stream information
}
// Dump information about file onto standard error
av_dump_format(pFormatCtx, 0, argv[1], 0);
// Close the video file
avformat_close_input(&pFormatCtx);
return 0;
}
When I pass this code a video (I have confirmed with several) it prints Failed to open file: Protocol not found. This was unexpected since this exact same code with the exact same file works on my Linux virtual machine. My compilation command is as follows: gcc file.c -lavformat -lavcodec -lavutil -lws2_32 -o prog. I invoke my program by writing prog sample.mpeg. In case it helps my libav version is 56.1.0 and my ffmpeg version is 5.1.2-essentials_build

Setting ffmpeg properly in ubuntu 16.04

I am following this website for ffmpeg tutorial: http://dranger.com
I tried to compile the programs after setting up ffmpeg in ubuntu by looking on some online videos but none of them worked. Some times GCC gives me undefined reference error and sometimes header not found error. I looked on some of the answers on SO that said that we need to do some change in the code as the new api is not backwards compatible but still GCC gives me undefined reference error.
Here is the code that I am trying to compile:
// tutorial01.c
// Code based on a tutorial by Martin Bohme (boehme#inb.uni-luebeckREMOVETHIS.de)
// Tested on Gentoo, CVS version 5/01/07 compiled with GCC 4.1.1
// With updates from https://github.com/chelyaev/ffmpeg-tutorial
// Updates tested on:
// LAVC 54.59.100, LAVF 54.29.104, LSWS 2.1.101
// on GCC 4.7.2 in Debian February 2015
// A small sample program that shows how to use libavformat and libavcodec to
// read video from a file.
//
// Use
//
// gcc -o tutorial01 tutorial01.c -lavformat -lavcodec -lswscale -lz
//
// to build (assuming libavformat and libavcodec are correctly installed
// your system).
//
// Run using
//
// tutorial01 myvideofile.mpg
//
// to write the first five frames from "myvideofile.mpg" to disk in PPM
// format.
#include <libavcodec/avcodec.h>
#include <libavformat/avformat.h>
#include <libswscale/swscale.h>
#include <stdio.h>
// compatibility with newer API
#if LIBAVCODEC_VERSION_INT < AV_VERSION_INT(55,28,1)
#define av_frame_alloc avcodec_alloc_frame
#define av_frame_free avcodec_free_frame
#endif
void SaveFrame(AVFrame *pFrame, int width, int height, int iFrame) {
FILE *pFile;
char szFilename[32];
int y;
// Open file
sprintf(szFilename, "frame%d.ppm", iFrame);
pFile=fopen(szFilename, "wb");
if(pFile==NULL)
return;
// Write header
fprintf(pFile, "P6\n%d %d\n255\n", width, height);
// Write pixel data
for(y=0; y<height; y++)
fwrite(pFrame->data[0]+y*pFrame->linesize[0], 1, width*3, pFile);
// Close file
fclose(pFile);
}
int main(int argc, char *argv[]) {
// Initalizing these to NULL prevents segfaults!
AVFormatContext *pFormatCtx = NULL;
int i, videoStream;
AVCodecContext *pCodecCtxOrig = NULL;
AVCodecContext *pCodecCtx = NULL;
AVCodec *pCodec = NULL;
AVFrame *pFrame = NULL;
AVFrame *pFrameRGB = NULL;
AVPacket packet;
int frameFinished;
int numBytes;
uint8_t *buffer = NULL;
struct SwsContext *sws_ctx = NULL;
if(argc < 2) {
printf("Please provide a movie file\n");
return -1;
}
// Register all formats and codecs
av_register_all();
// Open video file
if(avformat_open_input(&pFormatCtx, argv[1], NULL, NULL)!=0)
return -1; // Couldn't open file
// Retrieve stream information
if(avformat_find_stream_info(pFormatCtx, NULL)<0)
return -1; // Couldn't find stream information
// Dump information about file onto standard error
av_dump_format(pFormatCtx, 0, argv[1], 0);
// Find the first video stream
videoStream=-1;
for(i=0; i<pFormatCtx->nb_streams; i++)
if(pFormatCtx->streams[i]->codec->codec_type==AVMEDIA_TYPE_VIDEO) {
videoStream=i;
break;
}
if(videoStream==-1)
return -1; // Didn't find a video stream
// Get a pointer to the codec context for the video stream
pCodecCtxOrig=pFormatCtx->streams[videoStream]->codec;
// Find the decoder for the video stream
pCodec=avcodec_find_decoder(pCodecCtxOrig->codec_id);
if(pCodec==NULL) {
fprintf(stderr, "Unsupported codec!\n");
return -1; // Codec not found
}
// Copy context
pCodecCtx = avcodec_alloc_context3(pCodec);
if(avcodec_copy_context(pCodecCtx, pCodecCtxOrig) != 0) {
fprintf(stderr, "Couldn't copy codec context");
return -1; // Error copying codec context
}
// Open codec
if(avcodec_open2(pCodecCtx, pCodec, NULL)<0)
return -1; // Could not open codec
// Allocate video frame
pFrame=av_frame_alloc();
// Allocate an AVFrame structure
pFrameRGB=av_frame_alloc();
if(pFrameRGB==NULL)
return -1;
// Determine required buffer size and allocate buffer
numBytes=avpicture_get_size(PIX_FMT_RGB24, pCodecCtx->width,
pCodecCtx->height);
buffer=(uint8_t *)av_malloc(numBytes*sizeof(uint8_t));
// Assign appropriate parts of buffer to image planes in pFrameRGB
// Note that pFrameRGB is an AVFrame, but AVFrame is a superset
// of AVPicture
avpicture_fill((AVPicture *)pFrameRGB, buffer, PIX_FMT_RGB24,
pCodecCtx->width, pCodecCtx->height);
// initialize SWS context for software scaling
sws_ctx = sws_getContext(pCodecCtx->width,
pCodecCtx->height,
pCodecCtx->pix_fmt,
pCodecCtx->width,
pCodecCtx->height,
PIX_FMT_RGB24,
SWS_BILINEAR,
NULL,
NULL,
NULL
);
// Read frames and save first five frames to disk
i=0;
while(av_read_frame(pFormatCtx, &packet)>=0) {
// Is this a packet from the video stream?
if(packet.stream_index==videoStream) {
// Decode video frame
avcodec_decode_video2(pCodecCtx, pFrame, &frameFinished, &packet);
// Did we get a video frame?
if(frameFinished) {
// Convert the image from its native format to RGB
sws_scale(sws_ctx, (uint8_t const * const *)pFrame->data,
pFrame->linesize, 0, pCodecCtx->height,
pFrameRGB->data, pFrameRGB->linesize);
// Save the frame to disk
if(++i<=5)
SaveFrame(pFrameRGB, pCodecCtx->width, pCodecCtx->height,
i);
}
}
// Free the packet that was allocated by av_read_frame
av_free_packet(&packet);
}
// Free the RGB image
av_free(buffer);
av_frame_free(&pFrameRGB);
// Free the YUV frame
av_frame_free(&pFrame);
// Close the codecs
avcodec_close(pCodecCtx);
avcodec_close(pCodecCtxOrig);
// Close the video file
avformat_close_input(&pFormatCtx);
return 0;
}
This is the command I use to compile:
gcc -o tutorial01 tutorial01.c -lavformat -lavcodec -lswscale -lz

Remux RTSP stream into container? (Write read frame without decoding it)

I'm trying to use dranger tutorials for writing a RTSP h264 streamed video directly to a file without decoding and encoding it (ffmpeg 3.0/3.1 library). But I'm a bit lost on how do I need to populate the AVFormatContext pointer for the av_write_frame once I get the corresponding AVPacket.
Trying to clarify. What I want to do is this
1. Open webcam stream in h264
2. Read a frame
3. Save it to a file without decoding and encoding it.
EDIT: I've also tried to use the remuxing example in ffmpeg's documentation (doing a network init()) but I'm getting dts and pts sync errors when going from rtsp -> .mp4
Copy pasting the code from the tutorial:
#include <stdio.h>
#include <libavutil/pixfmt.h>
#include <libavcodec/avcodec.h>
#include <libavutil/avconfig.h>
#include <libswscale/swscale.h>
#include <libavformat/avformat.h>
int main(int argc, char *argv[]) {
av_register_all();
avcodec_register_all();
avformat_network_init();
AVFormatContext *pFormatCtx = avformat_alloc_context();
// Camera comes from argv[1]
avformat_open_input(&pFormatCtx, argv[1], NULL, NULL);
avformat_find_stream_info(pFormatCtx, NULL);
av_dump_format(pFormatCtx, 0, argv[1], 0);
int video_stream_idx = -1;
for (int i = 0; i < pFormatCtx->nb_streams; i++) {
if (pFormatCtx->streams[i]->codecpar->codec_type == AVMEDIA_TYPE_VIDEO) {
video_stream_idx = i;
break;
}
}
AVCodecContext *pCodecContext = NULL;
AVCodecContext *pCodecContextOrig = NULL;
pCodecContextOrig = pFormatCtx->streams[video_stream_idx]->codec;
AVCodec *pCodec;
pCodec = avcodec_find_decoder(pCodecContextOrig->codec_id);
pCodecContext = avcodec_alloc_context3(pCodec);
avcodec_copy_context(pCodecContext, pCodecContextOrig);
avcodec_open2(pCodecContext, pCodec, NULL);
AVFrame *pFrame = av_frame_alloc();
AVFrame *pFrameRGB = av_frame_alloc();
uint8_t *buffer = NULL;
int buffer_size = avpicture_get_size(AV_PIX_FMT_RGB24, pCodecContext->width,
pCodecContext->height);
buffer = (uint8_t *)av_malloc(buffer_size * sizeof(uint8_t));
// fill buffer
avpicture_fill((AVPicture *)pFrameRGB, buffer, AV_PIX_FMT_RGB24,
pCodecContext->width, pCodecContext->height);
struct SwsContext *sws_ctx = NULL;
int frame_finished = 0;
AVPacket packet;
// Size(src), fmt(src), Size(dst), fmt(dst) .. flags
sws_ctx = sws_getContext(pCodecContext->width, pCodecContext->height,
pCodecContext->pix_fmt, pCodecContext->width,
pCodecContext->height, AV_PIX_FMT_RGB24,
SWS_BILINEAR, NULL, NULL, NULL);
AVFormatContext *out_fmt_ctx;
avformat_write_header(out_fmt_ctx, NULL);
int i = 0;
while (i < 100 && (av_read_frame(pFormatCtx, &packet) >= 0)) {
// Want to write these frames to disk
i++;
}
av_write_trailer(out_fmt_ctx);
av_free(buffer);
av_free(pFrameRGB);
av_free(pFrame);
avcodec_close(pCodecContext);
avcodec_close(pCodecContextOrig);
avformat_close_input(&pFormatCtx);
return 0;
}
I think a lot of stuff of this code can be removed. I'm trying to learn :).
Linking against -lavutil -lavformat -lavcodec -lz -lavutil -lm -lswscale
I fixed by using the remuxing.c example from FFmpeg's documentation. The problem was that for some reason (I still dont know why) the first packet dts and pts was higher than the second (and following ones, which increased monotonically). Fixed it by skipping the first packet write :).

encode h264 video using ffmpeg library memory issues

I'm trying to do screen capture on OS X using ffmpeg's avfoundation library. I capture frames from the screen and encode it using H264 into an flv container.
Here's the command line output of the program:
Input #0, avfoundation, from 'Capture screen 0':
Duration: N/A, start: 9.253649, bitrate: N/A
Stream #0:0: Video: rawvideo (UYVY / 0x59565955), uyvy422, 1440x900, 14.58 tbr, 1000k tbn, 1000k tbc
raw video is inCodec
FLV (Flash Video)http://localhost:8090/test.flv
[libx264 # 0x102038e00] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX
[libx264 # 0x102038e00] profile High, level 4.0
[libx264 # 0x102038e00] 264 - core 142 r2495 6a301b6 - H.264/MPEG-4 AVC codec - Copyleft 2003-2014 - http://www.videolan.org/x264.html - options: cabac=1 ref=3 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=7 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=6 lookahead_threads=1 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=1 weightp=2 keyint=50 keyint_min=5 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=abr mbtree=1 bitrate=400 ratetol=1.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00
[tcp # 0x101a5fe70] Connection to tcp://localhost:8090 failed (Connection refused), trying next address
[tcp # 0x101a5fe70] Connection to tcp://localhost:8090 failed: Connection refused
url_fopen failed: Operation now in progress
[flv # 0x102038800] Using AVStream.codec.time_base as a timebase hint to the muxer is deprecated. Set AVStream.time_base instead.
encoded frame #0
encoded frame #1
......
encoded frame #49
encoded frame #50
testmee(8404,0x7fff7e05c300) malloc: *** error for object 0x102053e08: incorrect checksum for freed object - object was probably modified after being freed.
*** set a breakpoint in malloc_error_break to debug
(lldb) bt
* thread #10: tid = 0x43873, 0x00007fff95639286 libsystem_kernel.dylib`__pthread_kill + 10, stop reason = signal SIGABRT
* frame #0: 0x00007fff95639286 libsystem_kernel.dylib`__pthread_kill + 10
frame #1: 0x00007fff9623742f libsystem_pthread.dylib`pthread_kill + 90
frame #2: 0x00007fff977ceb53 libsystem_c.dylib`abort + 129
frame #3: 0x00007fff9ab59e06 libsystem_malloc.dylib`szone_error + 625
frame #4: 0x00007fff9ab4f799 libsystem_malloc.dylib`small_malloc_from_free_list + 1105
frame #5: 0x00007fff9ab4d3bc libsystem_malloc.dylib`szone_malloc_should_clear + 1449
frame #6: 0x00007fff9ab4c877 libsystem_malloc.dylib`malloc_zone_malloc + 71
frame #7: 0x00007fff9ab4b395 libsystem_malloc.dylib`malloc + 42
frame #8: 0x00007fff94aa63d2 IOSurface`IOSurfaceClientLookupFromMachPort + 40
frame #9: 0x00007fff94aa6b38 IOSurface`IOSurfaceLookupFromMachPort + 12
frame #10: 0x00007fff92bfa6b2 CoreGraphics`_CGYDisplayStreamFrameAvailable + 342
frame #11: 0x00007fff92f6759c CoreGraphics`CGYDisplayStreamNotification_server + 336
frame #12: 0x00007fff92bfada6 CoreGraphics`display_stream_runloop_callout + 46
frame #13: 0x00007fff956eba07 CoreFoundation`__CFMachPortPerform + 247
frame #14: 0x00007fff956eb8f9 CoreFoundation`__CFRUNLOOP_IS_CALLING_OUT_TO_A_SOURCE1_PERFORM_FUNCTION__ + 41
frame #15: 0x00007fff956eb86b CoreFoundation`__CFRunLoopDoSource1 + 475
frame #16: 0x00007fff956dd3e7 CoreFoundation`__CFRunLoopRun + 2375
frame #17: 0x00007fff956dc858 CoreFoundation`CFRunLoopRunSpecific + 296
frame #18: 0x00007fff95792ef1 CoreFoundation`CFRunLoopRun + 97
frame #19: 0x0000000105f79ff1 CMIOUnits`___lldb_unnamed_function2148$$CMIOUnits + 875
frame #20: 0x0000000105f6f2c2 CMIOUnits`___lldb_unnamed_function2127$$CMIOUnits + 14
frame #21: 0x00007fff97051765 CoreMedia`figThreadMain + 417
frame #22: 0x00007fff96235268 libsystem_pthread.dylib`_pthread_body + 131
frame #23: 0x00007fff962351e5 libsystem_pthread.dylib`_pthread_start + 176
frame #24: 0x00007fff9623341d libsystem_pthread.dylib`thread_start + 13
I've attached the code I used below.
#include <libavcodec/avcodec.h>
#include <libavformat/avformat.h>
#include <libswscale/swscale.h>
#include <libavdevice/avdevice.h>
#include <libavutil/opt.h>
#include <stdio.h>
#include <stdint.h>
#include <unistd.h>
/* compile using
gcc -g -o stream test.c -lavformat -lavutil -lavcodec -lavdevice -lswscale
*/
// void show_av_device() {
// inFmt->get_device_list(inFmtCtx, device_list);
// printf("Device Info=============\n");
// //avformat_open_input(&inFmtCtx,"video=Capture screen 0",inFmt,&inOptions);
// printf("===============================\n");
// }
void AVFAIL (int code, const char *what) {
char msg[500];
av_strerror(code, msg, sizeof(msg));
fprintf(stderr, "failed: %s\nerror: %s\n", what, msg);
exit(2);
}
#define AVCHECK(f) do { int e = (f); if (e < 0) AVFAIL(e, #f); } while (0)
#define AVCHECKPTR(p,f) do { p = (f); if (!p) AVFAIL(AVERROR_UNKNOWN, #f); } while (0)
void registerLibs() {
av_register_all();
avdevice_register_all();
avformat_network_init();
avcodec_register_all();
}
int main(int argc, char *argv[]) {
//conversion variables
struct SwsContext *swsCtx = NULL;
//input stream variables
AVFormatContext *inFmtCtx = NULL;
AVCodecContext *inCodecCtx = NULL;
AVCodec *inCodec = NULL;
AVInputFormat *inFmt = NULL;
AVFrame *inFrame = NULL;
AVDictionary *inOptions = NULL;
const char *streamURL = "http://localhost:8090/test.flv";
const char *name = "avfoundation";
// AVFrame *inFrameYUV = NULL;
AVPacket inPacket;
//output stream variables
AVCodecContext *outCodecCtx = NULL;
AVCodec *outCodec;
AVFormatContext *outFmtCtx = NULL;
AVOutputFormat *outFmt = NULL;
AVFrame *outFrameYUV = NULL;
AVStream *stream = NULL;
int i, videostream, ret;
int numBytes, frameFinished;
registerLibs();
inFmtCtx = avformat_alloc_context(); //alloc input context
av_dict_set(&inOptions, "pixel_format", "uyvy422", 0);
av_dict_set(&inOptions, "probesize", "7000000", 0);
inFmt = av_find_input_format(name);
ret = avformat_open_input(&inFmtCtx, "Capture screen 0:", inFmt, &inOptions);
if (ret < 0) {
printf("Could not load the context for the input device\n");
return -1;
}
if (avformat_find_stream_info(inFmtCtx, NULL) < 0) {
printf("Could not find stream info for screen\n");
return -1;
}
av_dump_format(inFmtCtx, 0, "Capture screen 0", 0);
// inFmtCtx->streams is an array of pointers of size inFmtCtx->nb_stream
videostream = av_find_best_stream(inFmtCtx, AVMEDIA_TYPE_VIDEO, -1, -1, &inCodec, 0);
if (videostream == -1) {
printf("no video stream found\n");
return -1;
} else {
printf("%s is inCodec\n", inCodec->long_name);
}
inCodecCtx = inFmtCtx->streams[videostream]->codec;
// open codec
if (avcodec_open2(inCodecCtx, inCodec, NULL) > 0) {
printf("Couldn't open codec");
return -1; // couldn't open codec
}
//setup output params
outFmt = av_guess_format(NULL, streamURL, NULL);
if(outFmt == NULL) {
printf("output format was not guessed properly");
return -1;
}
if((outFmtCtx = avformat_alloc_context()) < 0) {
printf("output context not allocated. ERROR");
return -1;
}
printf("%s", outFmt->long_name);
outFmtCtx->oformat = outFmt;
snprintf(outFmtCtx->filename, sizeof(outFmtCtx->filename), streamURL);
printf("%s\n", outFmtCtx->filename);
outCodec = avcodec_find_encoder(AV_CODEC_ID_H264);
if(!outCodec) {
printf("could not find encoder for H264 \n" );
return -1;
}
stream = avformat_new_stream(outFmtCtx, outCodec);
outCodecCtx = stream->codec;
avcodec_get_context_defaults3(outCodecCtx, outCodec);
outCodecCtx->codec_id = AV_CODEC_ID_H264;
outCodecCtx->codec_type = AVMEDIA_TYPE_VIDEO;
outCodecCtx->flags = CODEC_FLAG_GLOBAL_HEADER;
outCodecCtx->width = inCodecCtx->width;
outCodecCtx->height = inCodecCtx->height;
outCodecCtx->time_base.den = 25;
outCodecCtx->time_base.num = 1;
outCodecCtx->pix_fmt = AV_PIX_FMT_YUV420P;
outCodecCtx->gop_size = 50;
outCodecCtx->bit_rate = 400000;
//setup output encoders etc
if(stream) {
ret = avcodec_open2(outCodecCtx, outCodec, NULL);
if (ret < 0) {
printf("Could not open output encoder");
return -1;
}
}
if (avio_open(&outFmtCtx->pb, outFmtCtx->filename, AVIO_FLAG_WRITE ) < 0) {
perror("url_fopen failed");
}
avio_open_dyn_buf(&outFmtCtx->pb);
ret = avformat_write_header(outFmtCtx, NULL);
if (ret != 0) {
printf("was not able to write header to output format");
return -1;
}
unsigned char *pb_buffer;
int len = avio_close_dyn_buf(outFmtCtx->pb, (unsigned char **)(&pb_buffer));
avio_write(outFmtCtx->pb, (unsigned char *)pb_buffer, len);
numBytes = avpicture_get_size(PIX_FMT_UYVY422, inCodecCtx->width, inCodecCtx->height);
// Allocate video frame
inFrame = av_frame_alloc();
swsCtx = sws_getContext(inCodecCtx->width, inCodecCtx->height, inCodecCtx->pix_fmt, inCodecCtx->width,
inCodecCtx->height, PIX_FMT_YUV420P, SWS_BILINEAR, NULL, NULL, NULL);
int frame_count = 0;
while(av_read_frame(inFmtCtx, &inPacket) >= 0) {
if(inPacket.stream_index == videostream) {
avcodec_decode_video2(inCodecCtx, inFrame, &frameFinished, &inPacket);
// 1 Frame might need more than 1 packet to be filled
if(frameFinished) {
outFrameYUV = av_frame_alloc();
uint8_t *buffer = (uint8_t *)av_malloc(numBytes);
int ret = avpicture_fill((AVPicture *)outFrameYUV, buffer, PIX_FMT_YUV420P,
inCodecCtx->width, inCodecCtx->height);
if(ret < 0){
printf("%d is return val for fill\n", ret);
return -1;
}
//convert image to YUV
sws_scale(swsCtx, (uint8_t const * const* )inFrame->data,
inFrame->linesize, 0, inCodecCtx->height,
outFrameYUV->data, outFrameYUV->linesize);
//outFrameYUV now holds the YUV scaled frame/picture
outFrameYUV->format = outCodecCtx->pix_fmt;
outFrameYUV->width = outCodecCtx->width;
outFrameYUV->height = outCodecCtx->height;
AVPacket pkt;
int got_output;
av_init_packet(&pkt);
pkt.data = NULL;
pkt.size = 0;
outFrameYUV->pts = frame_count;
ret = avcodec_encode_video2(outCodecCtx, &pkt, outFrameYUV, &got_output);
if (ret < 0) {
fprintf(stderr, "Error encoding video frame: %s\n", av_err2str(ret));
return -1;
}
if(got_output) {
if(stream->codec->coded_frame->key_frame) {
pkt.flags |= AV_PKT_FLAG_KEY;
}
pkt.stream_index = stream->index;
if(pkt.pts != AV_NOPTS_VALUE)
pkt.pts = av_rescale_q(pkt.pts, stream->codec->time_base, stream->time_base);
if(pkt.dts != AV_NOPTS_VALUE)
pkt.dts = av_rescale_q(pkt.dts, stream->codec->time_base, stream->time_base);
if(avio_open_dyn_buf(&outFmtCtx->pb)!= 0) {
printf("ERROR: Unable to open dynamic buffer\n");
}
ret = av_interleaved_write_frame(outFmtCtx, &pkt);
unsigned char *pb_buffer;
int len = avio_close_dyn_buf(outFmtCtx->pb, (unsigned char **)&pb_buffer);
avio_write(outFmtCtx->pb, (unsigned char *)pb_buffer, len);
} else {
ret = 0;
}
if(ret != 0) {
fprintf(stderr, "Error while writing video frame: %s\n", av_err2str(ret));
exit(1);
}
fprintf(stderr, "encoded frame #%d\n", frame_count);
frame_count++;
av_free_packet(&pkt);
av_frame_free(&outFrameYUV);
av_free(buffer);
}
}
av_free_packet(&inPacket);
}
av_write_trailer(outFmtCtx);
//close video stream
if(stream) {
avcodec_close(outCodecCtx);
}
for (i = 0; i < outFmtCtx->nb_streams; i++) {
av_freep(&outFmtCtx->streams[i]->codec);
av_freep(&outFmtCtx->streams[i]);
}
if (!(outFmt->flags & AVFMT_NOFILE))
/* Close the output file. */
avio_close(outFmtCtx->pb);
/* free the output format context */
avformat_free_context(outFmtCtx);
// Free the YUV frame populated by the decoder
av_free(inFrame);
// Close the video codec (decoder)
avcodec_close(inCodecCtx);
// Close the input video file
avformat_close_input(&inFmtCtx);
return 1;
}
I'm not sure what I've done wrong here. But, what I've observed is that for each frame thats been encoded, my memory usage goes up by about 6MB. Backtracking afterward usually leads one of the following two culprits:
avf_read_frame function in avfoundation.m
av_dup_packet function in avpacket.h
Can I also get advice on the way I'm using avio_open_dyn_buff function to be able to stream over http? I've also attached my ffmpeg library versions below:
ffmpeg version N-70876-g294bb6c Copyright (c) 2000-2015 the FFmpeg developers
built with Apple LLVM version 6.0 (clang-600.0.56) (based on LLVM 3.5svn)
configuration: --prefix=/usr/local --enable-gpl --enable-postproc --enable-pthreads --enable-libmp3lame --enable-libtheora --enable-libx264 --enable-libvorbis --disable-mmx --disable-ssse3 --disable-armv5te --disable-armv6 --disable-neon --enable-shared --disable-static --disable-stripping
libavutil 54. 20.100 / 54. 20.100
libavcodec 56. 29.100 / 56. 29.100
libavformat 56. 26.101 / 56. 26.101
libavdevice 56. 4.100 / 56. 4.100
libavfilter 5. 13.101 / 5. 13.101
libswscale 3. 1.101 / 3. 1.101
libswresample 1. 1.100 / 1. 1.100
libpostproc 53. 3.100 / 53. 3.100
Hyper fast Audio and Video encoder
Valgrind analysis attached here because I exceeded stack overflow's character limit. http://pastebin.com/MPeRhjhN
Your code contains the following allocation:
uint8_t *buffer = (uint8_t *)av_malloc(numBytes*sizeof(uint8_t));
But has no matching av_free(). That's probably where you're losing 6MB/frame. av_free(outFrameYUV) won't free the memory inside the struct, just the struct itself. More correct would be to use av_frame_unref() instead of these individual av_free()s.
I also see you're calling this in your loop:
av_free(inFrame);
But the allocation for that frame is done outside your loop, so after the first run, it would presumably be dead. You want to allocate and free the frame both inside the loop. And for unreferencing, use av_frame_unref() instead of av_free().
I'd recommend running your program using asan or valgrind to detect more such issues, it'll keep track of what memory is accessed when and if incorrect, where it was free'ed.

libpng -- error: Not a PNG file -- png_process_data()

I am attempting to write a progressive reader using libpng. I have tried to follow the code in example.c provided with libpng source but it does not work. I have also looked at libpng doc. Which instructs you to use png_set_progressive_read_fn() in connection with png_process_data(). After setting up things as specified in the documentation. When I go to start the process of having the png file read I get the following error from libpng.
Not a PNG file
Stepping through the code reveals that when png_process_data() goes to check the signature of the PNG file, it identifies that it is not a png file. This is very peculiar behavior since before I call my read function I verify the file is a PNG file with the following bit of code.
int check_png_sig(const char * file_name)
{
FILE *fp = fopen(file_name, "rb");
if(!fp) {
return (0);
}
enum HeaderSize {Size = 8};
png_byte header[Size];
fread(header, 1, Size, fp);
int is_png = !png_sig_cmp(header, 0, Size);
if (!is_png) {
fclose(fp);
return (0);
}
fclose(fp);
return (1);
}
I close the FILE* and then reopen a new one in my read function so I don't have to tell libpng that I have already read 8 bytes with png_set_sig_bytes(png_ptr, 8);. My read function is as follows.
int read_png_file(const char * file_name)
{
fprintf(stdout, "Reading PNG File %s\n", file_name);
fflush(stdout);
if (!png_ptr || !info_ptr) {
return (0);
}
FILE *fp = fopen(file_name, "rb");
if(!fp) {
return (0);
}
png_init_io(png_ptr, fp);
png_voidp user_chunkp = png_get_user_chunk_ptr(png_ptr);
/*
* Tell libpng to call read_chunk_callback if an unknown chunk is
* encountered
*/
png_set_read_user_chunk_fn(png_ptr, user_chunkp, read_chunk_callback);
/*
* Tell libpng to call read_row_callback if an known chunk is
* encountered.
*/
png_set_read_status_fn(png_ptr, read_row_callback);
/*
* Tell libpng to call the specified functions on info, rows, or
* end.
*/
png_set_progressive_read_fn(png_ptr, user_chunkp, info_callback,
row_callback, end_callback);
enum Size {DataSize = 8};
unsigned char rawbuf[DataSize];
int process = 1;
if (setjmp(png_jmpbuf(png_ptr))) {
free_png_resources();
process = 0;
return process;
}
/*
* Check to see if libpng is confused
*/
//png_set_sig_bytes(png_ptr, 8);
while (process) {
memset(rawbuf, 0, DataSize);
png_process_data(png_ptr, info_ptr, rawbuf, DataSize);
}
fclose(fp);
return (process);
}
I also have a working version of a non-porgressive reader at (non-progressive reader). It works just fine on my file. So it is not an issue with my file.
COMPLETE CODE
#include <png.h>
#include <stdlib.h>
static png_structp png_ptr;
static png_infop info_ptr;
static png_bytep old_row = NULL;
/* Variable that loops untill data is read */
int run = 0;
//-------------------------------------------------------------------
// Callbacks
int read_chunk_callback(png_structp png_ptr, png_unknown_chunkp chunk)
{
struct chunk
{
png_byte name[5];
png_byte *data;
png_size_t size;
};
return (0); /* did not recognize */
}
/*
* This method will be called each time a row is read
* and differs from the row_callback in that ...
*/
void read_row_callback(png_structp ptr, png_uint_32 row, int pass)
{
fprintf(stderr, "read_row_callback\n");
}
int process_data(png_bytep buffer, png_uint_32 length)
{
fprintf(stderr, "process_data\n");
png_process_data(png_ptr, info_ptr, buffer, length);
return 0;
}
void info_callback(png_structp png_ptr, png_infop info)
{
fprintf(stderr, "info_callback\n");
png_uint_32 width;
png_uint_32 height;
int bit_depth;
int color_type;
int interlace_type;
int compression_type;
int filter_type;
png_byte channels;
png_uint_32 rowbytes;
png_bytep signature;
/*
* This will get the information stored in the header
* of the PNG file.
*/
png_get_IHDR(png_ptr, info_ptr, &width, &height, &bit_depth, &color_type,
&interlace_type, &compression_type, &filter_type);
/*
* Get the rest of the header information.
*/
channels = png_get_channels(png_ptr, info_ptr);
/* This is subject to change with transformations */
rowbytes = png_get_rowbytes(png_ptr, info_ptr);
signature = png_get_signature(png_ptr, info_ptr);
fprintf(stdout,
"width: %u"
"height: %u"
"bit_depth: %d"
"color_type: %d"
"interlace_type: %d"
"compression_type: %d"
"filter_type: %d"
"channles: %d"
"rowbytes: %u"
"signature: %s", (unsigned int)width, (unsigned int)height, bit_depth, color_type,
interlace_type, compression_type, filter_type, channels,
(unsigned int)rowbytes, signature);
}
void row_callback(png_structp png_ptr, png_bytep new_row, png_uint_32 row_num,
int pass)
{
fprintf(stderr, "row_callback\n");
png_progressive_combine_row(png_ptr, old_row, new_row);
}
void end_callback(png_structp png_ptr, png_infop info)
{
fprintf(stderr, "end_callback\n");
run = 1;
}
/*
* Error handler local to his translation unit
* Used by png_create_read_struct function.
*/
void progressive_reader_error_fn(png_structp png_ptr, png_const_charp msg)
{
fprintf(stderr, "error: %s\n", msg);
fflush(stderr);
longjmp(png_jmpbuf(png_ptr), 1);
exit(0);
}
//-------------------------------------------------------------------
/*
* Free PNG resources on close
*/
void free_png_resources()
{
png_destroy_read_struct(&png_ptr, &info_ptr, NULL);
png_ptr = NULL;
info_ptr = NULL;
}
/*
* All strings to this function must
* be null terminated -- return 0 if
* file is not in png format
*/
int check_png_sig(const char * file_name)
{
FILE *fp = fopen(file_name, "rb");
if(!fp) {
return (0);
}
enum HeaderSize {Size = 8};
png_byte header[Size];
fread(header, 1, Size, fp);
int is_png = !png_sig_cmp(header, 0, Size);
if (!is_png) {
fclose(fp);
return (0);
}
fclose(fp);
return (1);
}
/*
* Create the png_structp and png_infop
*/
int create_png_structs()
{
png_ptr = png_create_read_struct(PNG_LIBPNG_VER_STRING, NULL,
progressive_reader_error_fn, NULL);
if (!png_ptr) {
return 0;
}
info_ptr = png_create_info_struct(png_ptr);
if (!info_ptr) {
png_destroy_read_struct(&png_ptr, NULL, NULL);
return 0;
}
if (setjmp(png_jmpbuf(png_ptr))) {
png_destroy_read_struct(&png_ptr, &info_ptr, NULL);
return 0;
}
return 1;
}
/*
* This method does all the work
*/
int read_png_file(const char * file_name)
{
fprintf(stdout, "Reading PNG File %s\n", file_name);
fflush(stdout);
if (!png_ptr || !info_ptr) {
return (0);
}
FILE *fp = fopen(file_name, "rb");
if(!fp) {
return (0);
}
png_init_io(png_ptr, fp);
png_voidp user_chunkp = png_get_user_chunk_ptr(png_ptr);
/*
* Tell libpng to call read_chunk_callback if an unknown chunk is
* encountered
*/
png_set_read_user_chunk_fn(png_ptr, user_chunkp, read_chunk_callback);
/*
* Tell libpng to call read_row_callback if an known chunk is
* encountered.
*/
png_set_read_status_fn(png_ptr, read_row_callback);
/*
* Tell libpng to call the specified functions on info, rows, or
* end.
*/
png_set_progressive_read_fn(png_ptr, user_chunkp, info_callback,
row_callback, end_callback);
enum Size {DataSize = 8};
unsigned char rawbuf[DataSize];
int process = 1;
if (setjmp(png_jmpbuf(png_ptr))) {
free_png_resources();
process = 0;
return process;
}
/*
* Check to see if libpng is confused
*/
//png_set_sig_bytes(png_ptr, 8);
while (process) {
memset(rawbuf, 0, DataSize);
png_process_data(png_ptr, info_ptr, rawbuf, DataSize);
}
fclose(fp);
return (process);
}
int main(int argc, char *argv[])
{
if (check_png_sig("/home/matt6809/Downloads/png_image.png")) {
if (create_png_structs()) {
if (read_png_file("/home/matt6809/Downloads/png_image.png")) {
fprintf(stderr, "SUCCESS!!!\n");
}
}
}
return 0;
}
Makefile
CC = /usr/bin/gcc
LD = /usr/bin/gcc
CFLAGS = -c -Wall -g
LDFLAGS = -lpng
SRC = $(wildcard *.c)
OBJ = $(SRC:%.c=%.o)
TARGET = progressive
all : $(TARGET)
$(TARGET) : $(OBJ)
$(LD) $(LDFLAGS) $^ -o $(TARGET)
%.o : %.c
$(CC) $(CFLAGS) -o $# $<
clean :
rm -rf *.o $(TARGET)
You've forgotten to read the content from your file.
Replacing:
while (process) {
memset(rawbuf, 0, DataSize);
png_process_data(png_ptr, info_ptr, rawbuf, DataSize);
}
with
size_t read;
while ((read = fread(rawbuf, 1, DataSize, fp))) {
png_process_data(png_ptr, info_ptr, rawbuf, read);
}
is enough to allow decoding to proceed.
I did notice one other thing: the second parameters passed to png_set_read_user_chunk_fn() and png_set_progressive_read_fn() are supposed to be arbitrary values that the callback functions can get hold of later (via png_get_user_chunk_ptr() and png_get_progressive_ptr(), respectively).
These can be set to whatever values you like (or to NULL if you don't need them), but you shouldn't be calling the png_get_*_ptr() functions yourself to get a value to pass to png_set_*_fn(). (I think it's harmless, since they'll return NULL to start with, but it's confusing at best.)

Resources