No-op remuxing of .avi file - c

I want to demux and then mux .avi file without changing anything.
My program is this (redacted for brevity):
AVFormatContext *input_format_context = NULL;
avformat_open_input(
&input_format_context,
input_url,
NULL, // fmt
NULL // options
);
avformat_find_stream_info(input_format_context, NULL);
AVFormatContext *output_format_context = NULL;
avformat_alloc_output_context2(
&output_format_context,
NULL, // oformat
NULL, // format_name
output_url
);
avio_open2(
&output_format_context->pb,
output_url,
AVIO_FLAG_WRITE,
NULL, // int_cb,
NULL // options
);
for (int i = 0; i < input_format_context->nb_streams; i++) {
avformat_new_stream(output_format_context, NULL);
AVStream *input_stream = input_format_context->streams[i];
AVStream *output_stream = output_format_context->streams[i];
AVCodecParameters *params = avcodec_parameters_alloc();
avcodec_parameters_copy(params, input_stream->codecpar);
output_stream->codecpar = params;
}
avformat_write_header(output_format_context, NULL))
AVPacket *input_packet = NULL;
input_packet = av_packet_alloc();
while (!av_read_frame(
input_format_context,
input_packet
)) {
av_write_frame(output_format_context, input_packet);
av_packet_unref(input_packet);
}
av_write_trailer(output_format_context);
Problem:
Output file is created but instead of close to 10 minute video it is a 24-second slide show consisting of around 3 frames.
It seems that the problem is (perhaps not the only one) lack of PTS on the packet.
When I explicitly print it (input_packet->pts) for each packet it is -9223372036854775808. And also the following warning is printed:
[avi # 0x562868c6c000] Timestamps are unset in a packet for stream 0. This is deprecated and will stop working in the future. Fix your code to set the timestamps properly
How do I then "fix my code to set the timestamps properly"?

I just found a solution.
I added this:
output_stream->time_base = input_stream->time_base;
which then, I understand, allows the video player to calculate PTS on the fly.
This does not remove the warning itself, though. I understand that .avi simply does not have PTS, so it's not a bug as such. To get rid of the warning one can manually set PTS on the packets:
input_packet->pts = calculated_ts;
I would think I should be able to also just do:
output_format_context->oformat->flags |= AVFMT_NOTIMESTAMPS;
However, I cannot do that:
error: assignment of member ‘flags’ in read-only object
So, it looks like ffmpeg is requiring PTS even for .avi or there's a bug or I'm still doing something wrong.

Related

av_write_header - Error with sample format

I'm writing program with libav/ffmpeg to download internet radio stream and play it on soundcard with alsa.
I've managed to download stream and extract packet and frame.
I'm having problem with av_write_header() function which (according to this https://www.ffmpeg.org/doxygen/3.2/group__lavf__encoding.html#details) I must call. It crashes and gives me the following error:
[alsa # 0x55d7ba32e580] sample format 0x15001 is not supported
Number 0x15001 is 86017 in decimal, which is index in enum AVCodecID of MP3 format(AV_CODEC_ID_MP3) used by this stream. The sample format has index 3. I can't figure out why libav parses the header wrong.
Here is a part of my code that is responsible for configuring output:
avdevice_register_all();
AVOutputFormat *output = av_guess_format("alsa",NULL,NULL);
AVFormatContext *outputFormatContext = avformat_alloc_context();
outputFormatContext->oformat = output;
outputFormatContext->flags = AVFMT_NOFILE;
AVStream *stream = avformat_new_stream(outputFormatContext,NULL);
AVCodecParameters *oCodecParameters = avcodec_parameters_alloc();
ret = avcodec_parameters_copy(oCodecParameters,iCodecParameters);
if(ret < 0){
printf("avformat_parameters_copy\n");
exit(0);
}
stream->codecpar = oCodecParameters;
if(avformat_write_header(outputFormatContext,NULL)<0){
dumpParameters(stream->codecpar);
printf("avformat_write_header\n");
exit(0);
}
The full code is here: https://github.com/szymonbarszcz99/C-internet-radio
It seems that in libav we can't do simple copy. Instead I have to manually give it requested parameters. Changing avcodec_parameters_copy() to this
AVCodecParameters *oCodecParameters = avcodec_parameters_alloc();
oCodecParameters->format = 8;
oCodecParameters->codec_type = 1;
oCodecParameters->sample_rate = 44100;
oCodecParameters->channels = 2;
stream->codecpar = oCodecParameters;
fixes this problem

FFMPEG remux sample without writing to file

Let's consider this very nice and easy to use remux sample by horgh.
I'd like to achieve the same task: convert an RTSP H264 encoded stream to a fragmented MP4 stream.
This code does exactly this task.
However I don't want to write the mp4 onto disk at all, but I need to get a byte buffer or array in C with the contents that would normally written to disk.
How is that achievable?
This sample uses vs_open_output to define the output format and this function needs an output url.
If I would get rid of outputting the contents to disk, how shall I modify this code?
Or there might be better alternatives as well, those are also welcomed.
Update:
As szatmary recommended, I have checked his example link.
However as I stated in the question I need the output as buffer instead of a file.
This example demonstrates nicely how can I read my custom source and give it to ffmpeg.
What I need is how can open the input as standard (with avformat_open_input) then do my custom modification with the packets and then instead writing to file, write to a buffer.
What have I tried?
Based on szatmary's example I created some buffers and initialization:
uint8_t *buffer;
buffer = (uint8_t *)av_malloc(4096);
format_ctx = avformat_alloc_context();
format_ctx->pb = avio_alloc_context(
buffer, 4096, // internal buffer and its size
1, // write flag (1=true, 0=false)
opaque, // user data, will be passed to our callback functions
0, // no read
&IOWriteFunc,
&IOSeekFunc
);
format_ctx->flags |= AVFMT_FLAG_CUSTOM_IO;
AVOutputFormat * const output_format = av_guess_format("mp4", NULL, NULL);
format_ctx->oformat = output_format;
avformat_alloc_output_context2(&format_ctx, output_format,
NULL, NULL)
Then of course I have created 'IOWriteFunc' and 'IOSeekFunc':
static int IOWriteFunc(void *opaque, uint8_t *buf, int buf_size) {
printf("Bytes read: %d\n", buf_size);
int len = buf_size;
return (int)len;
}
static int64_t IOSeekFunc (void *opaque, int64_t offset, int whence) {
switch(whence){
case SEEK_SET:
return 1;
break;
case SEEK_CUR:
return 1;
break;
case SEEK_END:
return 1;
break;
case AVSEEK_SIZE:
return 4096;
break;
default:
return -1;
}
return 1;
}
Then I need to write the header to the output buffer, and the expected behaviour here is to print "Bytes read: x":
AVDictionary * opts = NULL;
av_dict_set(&opts, "movflags", "frag_keyframe+empty_moov", 0);
av_dict_set_int(&opts, "flush_packets", 1, 0);
avformat_write_header(output->format_ctx, &opts)
In the last line during execution, it always runs into segfault, here is the backtrace:
#0 0x00007ffff7a6ee30 in () at /usr/lib/x86_64-linux-gnu/libavformat.so.57
#1 0x00007ffff7a98189 in avformat_init_output () at /usr/lib/x86_64-linux-gnu/libavformat.so.57
#2 0x00007ffff7a98ca5 in avformat_write_header () at /usr/lib/x86_64-linux-gnu/libavformat.so.57
...
The hard thing for me with the example is that it uses avformat_open_input.
However there is no such thing for the output (no avformat_open_ouput).
Update2:
I have found another example for reading: doc/examples/avio_reading.c.
There are mentions of a similar example for writing (avio_writing.c), but ffmpeg does not have this available (at least in my google search).
Is this task really this hard to solve? standard rtsp input to custom avio?
Fortunately ffmpeg.org is down. Great.
It was a silly mistake:
In the initialization part I called this:
avformat_alloc_output_context2(&format_ctx, output_format,
NULL, NULL)
However before this I already put the avio buffers into format_ctx:
format_ctx->pb = ...
Also, this line is unnecessary:
format_ctx = avformat_alloc_context();
Correct order:
AVOutputFormat * const output_format = av_guess_format("mp4", NULL, NULL);
avformat_alloc_output_context2(&format_ctx, output_format,
NULL, NULL)
format_ctx->pb = avio_alloc_context(
buffer, 4096, // internal buffer and its size
1, // write flag (1=true, 0=false)
opaque, // user data, will be passed to our callback functions
0, // no read
&IOWriteFunc,
&IOSeekFunc
);
format_ctx->flags |= AVFMT_FLAG_CUSTOM_IO;
format_ctx->oformat = output_format; //might be unncessary too
Segfault is gone now.
You need to write a AVIOContext implementation.

How to set pts, dts and duration in ffmpeg library?

I want to pack some compressed video packets(h.264) to ".mp4" container.
One word, Muxing, no decoding and no encoding.
And I have no idea how to set pts, dts and duration.
I get the packets with "pcap" library.
I removed headers before compressed video data show up. e.g. Ethernet, VLAN.
I collected data until one frame and decoded it for getting information of data. e.g. width, height. (I am not sure that it is necessary)
I initialized output context, stream and codec context.
I started to receive packets with "pcap" library again. (now for muxing)
I made one frame and put that data in AVPacket structure.
I try to set PTS, DTS and duration. (I think here is wrong part, not sure though)
*7-1. At the first frame, I saved time(msec) with packet header structure.
*7-2. whenever I made one frame, I set parameters like this : PTS(current time - start time), DTS(same PTS value), duration(current PTS - before PTS)
I think it has some error because :
I don't know how far is suitable long for dts from pts.
At least, I think duration means how long time show this frame from now to next frame, so It should have value(next PTS - current PTS), but I can not know the value next PTS at that time.
It has I-frame only.
// make input context for decoding
AVFormatContext *&ic = gInputContext;
ic = avformat_alloc_context();
AVCodec *cd = avcodec_find_decoder(AV_CODEC_ID_H264);
AVStream *st = avformat_new_stream(ic, cd);
AVCodecContext *cc = st->codec;
avcodec_open2(cc, cd, NULL);
// make packet and decode it after collect packets is be one frame
gPacket.stream_index = 0;
gPacket.size = gPacketLength[0];
gPacket.data = gPacketData[0];
gPacket.pts = AV_NOPTS_VALUE;
gPacket.dts = AV_NOPTS_VALUE;
gPacket.flags = AV_PKT_FLAG_KEY;
avcodec_decode_video2(cc, gFrame, &got_picture, &gPacket);
// I checked automatically it initialized after "avcodec_decode_video2"
// put some info that I know that not initialized
cc->time_base.den = 90000;
cc->time_base.num = 1;
cc->bit_rate = 2500000;
cc->gop_size = 1;
// make output context with input context
AVFormatContext *&oc = gOutputContext;
avformat_alloc_output_context2(&oc, NULL, NULL, filename);
AVFormatContext *&ic = gInputContext;
AVStream *ist = ic->streams[0];
AVCodecContext *&icc = ist->codec;
AVStream *ost = avformat_new_stream(oc, icc->codec);
AVCodecContext *occ = ost->codec;
avcodec_copy_context(occ, icc);
occ->flags |= CODEC_FLAG_GLOBAL_HEADER;
avio_open(&(oc->pb), filename, AVIO_FLAG_WRITE);
// repeated part for muxing
AVRational Millisecond = { 1, 1000 };
gPacket.stream_index = 0;
gPacket.data = gPacketData[0];
gPacket.size = gPacketLength[0];
gPacket.pts = av_rescale_rnd(pkthdr->ts.tv_sec * 1000 /
+ pkthdr->ts.tv_usec / 1000 /
- gStartTime, Millisecond.den, ost->time_base.den, /
(AVRounding)(AV_ROUND_NEAR_INF | AV_ROUND_PASS_MINMAX));
gPacket.dts = gPacket.pts;
gPacket.duration = gPacket.pts - gPrev;
gPacket.flags = AV_PKT_FLAG_KEY;
gPrev = gPacket.pts;
av_interleaved_write_frame(gOutputContext, &gPacket);
Expected and actual results is a .mp4 video file that can play.

IMFTransform::ProcessOutput returns E_INVALIDARG

The problem
I am trying to get call ProcessOutput to get decoded data from my decoder and get the following error:
E_INVALIDARG One or more arguments are invalid.
What I have tried
As ProcessOutput has many arguments I have tried to pinpoint what the error might be. Documentation for ProcessOutput does not mention E_INVALIDARG. However, the documentation for MFT_OUTPUT_DATA_BUFFER, the datatype for one of the arguments, mentions in its Remarks section that:
Any other combinations are invalid and cause ProcessOutput to return E_INVALIDARG
What it talks about there is how the MFT_OUTPUT_DATA_BUFFER struct is setup. So an incorrectly setup MFT_OUTPUT_DATA_BUFFER might cause that error. I have however tried to set it up correctly.
By calling GetOutputStreamInfo I find that I need to allocate the sample sent to ProcessOutput which is what I do. I'm using pretty much the same method that worked for ProcessInput so I don't know what I am doing wrong here.
I have also tried to make sure that the other arguments, who logically should also be able to cause an E_INVALIDARG. They look good to me and I have not been able to find any other leads to which of my arguments to ProcessOutput might be invalid.
The code
I have tried to post only the relevant parts of the code below. I have removed or shortened many of the error checks for brevity. Note that I am using plain C.
"Prelude"
...
hr = pDecoder->lpVtbl->SetOutputType(pDecoder, dwOutputStreamID, pMediaOut, dwFlags);
...
// Send input to decoder
hr = pDecoder->lpVtbl->ProcessInput(pDecoder, dwInputStreamID, pSample, dwFlags);
if (FAILED(hr)) { /* did not fail */ }
So before the interesting code below I have successfully setup things (I hope) and sent them to ProcessInput which did not fail. I have 1 input stream and 1 output stream, AAC in, PCM out.
Code directly leading to the error
// Input has now been sent to the decoder
// To extract a sample from the decoder we need to create a strucure to hold the output
// First we ask the OutputStream for what type of output sample it will produce and who should allocate it
// Then we create both the sample in question (if we should allocate it that is) and the MFT_OUTPUT_DATA_BUFFER
// which holds the sample and some other information that the decoder will fill in.
#define SAMPLES_PER_BUFFER 1 // hardcoded here, should depend on GetStreamIDs results, which right now is 1
MFT_OUTPUT_DATA_BUFFER pOutputSamples[SAMPLES_PER_BUFFER];
DWORD *pdwStatus = NULL;
// There are different allocation models, find out which one is required here.
MFT_OUTPUT_STREAM_INFO streamInfo = { 0,0,0 };
MFT_OUTPUT_STREAM_INFO *pStreamInfo = &streamInfo;
hr = pDecoder->lpVtbl->GetOutputStreamInfo(pDecoder, dwOutputStreamID, pStreamInfo);
if (FAILED(hr)) { ... }
if (pStreamInfo->dwFlags == MFT_OUTPUT_STREAM_PROVIDES_SAMPLES) { ... }
else if (pStreamInfo->dwFlags == MFT_OUTPUT_STREAM_CAN_PROVIDE_SAMPLES) { ... }
else {
// default, the client must allocate the output samples for the stream
IMFSample *pOutSample = NULL;
DWORD minimumSizeOfBuffer = pStreamInfo->cbSize;
IMFMediaBuffer *pBuffer = NULL;
// CreateMediaSample is explained further down.
hr = CreateMediaSample(minimumSizeOfBuffer, sampleDuration, &pBuffer, &pOutSample);
if (FAILED(hr)) {
BGLOG_ERROR("error");
}
pOutputSamples[0].pSample = pOutSample;
}
// since GetStreamIDs return E_NOTIMPL then dwStreamID does not matter
// but its recomended that it is set to the array index, 0 in this case.
// dwOutputStreamID will be 0 when E_NOTIMPL is returned by GetStremIDs
pOutputSamples[0].dwStreamID = dwOutputStreamID; // = 0
pOutputSamples[0].dwStatus = 0;
pOutputSamples[0].pEvents = NULL; // have tried init this myself, but MFT_OUTPUT_DATA_BUFFER documentation says not to.
hr = pDecoder->lpVtbl->ProcessOutput(pDecoder, dwFlags, outputStreamCount, pOutputSamples, pdwStatus);
if (FAILED(hr)) {
// here E_INVALIDARG is found.
}
CreateMediaSample that is used in the code is derived from an example from the official documentation but modified to call SetSampleDuration and SetSampleTime. I get the same error by not setting those two though so it should be something else causing the problem.
Some of the actual data that was sent to ProcessOutput
In case I might have missed something which is easy to see from the actual data:
hr = pDecoder->lpVtbl->ProcessOutput(
pDecoder, // my decoder
dwFlags, // 0
outputStreamCount, // 1 (from GetStreamCount)
pOutputSamples, // se comment below
pdwStatus // NULL
);
// pOutputSamples[0] holds this struct:
// dwStreamID = 0,
// pSample = SampleDefinedBelow
// dwStatus = 0,
// pEvents = NULL
// SampleDefinedBelow:
// time = 0
// duration = 0.9523..
// buffer = with max length set correctly
// attributes[] = NULL
Question
So anyone have any ideas on what I am doing wrong or how I could debug this further?
ProcessOutput needs a valid pointer as the last argument, so this does not work:
DWORD *pdwStatus = NULL;
pDecoder->lpVtbl->ProcessOutput(..., pdwStatus);
This is okay:
DWORD dwStatus;
pDecoder->lpVtbl->ProcessOutput(..., &dwStatus);
Regarding further E_FAIL - your findings above, in general, looks good. It is not that I see something obvious, and also the error code does not suggest that the problem is with MFT data flow. Perhaps it could be bad data or data not matching media types set.

Using BASS_StreamCreateFile in WPF

BASS_StreamCreateFile(path,offset,length,BassFlags) always returns '0'. I am not understanding how to use this function. Need help on the usage of BassFlags.
PS : Using this with the help of WPF Sound Visualization Library.
Since 0 only informs you that there's an error, you should check what kind of error it is:
int BASS_ErrorGetCode();
This gives you the errorcode for the recent error.
Here's the list of possible error codes (= return values):
BASS_ERROR_INIT // BASS_Init has not been successfully called.
BASS_ERROR_NOTAVAIL // Only decoding channels (BASS_STREAM_DECODE) are allowed when using the "no sound" device. The BASS_STREAM_AUTOFREE // flag is also unavailable to decoding channels.
BASS_ERROR_ILLPARAM // The length must be specified when streaming from memory.
BASS_ERROR_FILEOPEN // The file could not be opened.
BASS_ERROR_FILEFORM // The file's format is not recognised/supported.
BASS_ERROR_CODEC // The file uses a codec that is not available/supported. This can apply to WAV and AIFF files, and also MP3 files when using the "MP3-free" BASS version.
BASS_ERROR_FORMAT // The sample format is not supported by the device/drivers. If the stream is more than stereo or the BASS_SAMPLE_FLOAT flag is used, it could be that they are not supported.
BASS_ERROR_SPEAKER // The specified SPEAKER flags are invalid. The device/drivers do not support them, they are attempting to assign a stereo stream to a mono speaker or 3D functionality is enabled.
BASS_ERROR_MEM // There is insufficient memory.
BASS_ERROR_NO3D // Could not initialize 3D support.
BASS_ERROR_UNKNOWN // Some other mystery problem!
(from bass.h)
Also make shure you have initialised BASS properly - BASS_Init() must get called before you create a stream:
BOOL BASS_Init(
int device, // The device to use... -1 = default device, 0 = no sound, 1 = first real output device
DWORD freq, // Output sample rate
DWORD flags, // A combination of flags
HWND win, // The application's main window... 0 = the current foreground window (use this for console applications)
GUID *clsid // Class identifier of the object to create, that will be used to initialize DirectSound... NULL = use default
);
Example:
int device = -1; // Default device
int freq = 44100; // Sample rate
BASS_Init(device, freq, 0, 0, NULL); // Init BASS

Resources