Modify data in a stream using GStreamer - c

Question

I've been looking for a way to modify a GStreamer stream in order to display the quality (by showing the raw video next to the result of the subtraction between the raw video and the enc/dec stream for example).
I've been learning GStreamer for a week and so far I was able to do what I was asked but right now I'm stuck.
I looked into the compositor element that seems to mix streams but I'm pretty sure it cannot do what I need.
Then I checked appsrc and appsink in some code. I tried to build a pipeline: filesrc - appsink - appsrc - filesink. But for obvious reasons it did not work. I browsed github projects but most of appsrc/appsink uses were just to programmaticaly do a task like reading a file.
Lastly I found someone with the same problem like me. He "solved" it by creating 2 pipelines (filesrc - appsink & appsrc - filesink) but he still got allocation errors. I was not even able to run the code he shared.
Does anyone have any idea on how to get it done in a clean way?

Answer 1

I found a way to modify a stream.
And it basically was to create my own plugin because by creating a element I can modify the stream between the source pad and the sink pad of my element.
If someone is interrested there is the documentation explaining how to create a plugin and here's my chain function where I modify the data between the pads:
static GstFlowReturn
gst_my_filter_chain (GstPad * pad, GstObject * parent, GstBuffer * buf)
{
GstMyFilter *filter;
filter = GST_MYFILTER (parent);
if (filter->silent == FALSE) {
GstMapInfo info;
if(gst_buffer_map(buf, &info, GST_MAP_READWRITE)) {
guint8* offset = info.data;
int i = 0;
while(i < info.maxsize) {
guint8* pos = offset + i;
guint8 c = 100; // Value that will be added to make the color lighter
int cc = (int) (*pos + c); // Adding the value to the color
guint8 color;
if(cc > 255) { // Making sure the color stays valid
color = 255;
}
else {
color = (guint8) cc;
}
memset(pos, color, 1); // Setting the value
i++;
}
}
gst_buffer_unmap(buf, &info);
}
return gst_pad_push (filter->srcpad, buf);
}
It's quite rudimentary and only lighten the video's colors but I can modify a stream so it's only a matter of time before I get the thing I want done.
Just hope it'll help someone

Answer 2

Compositor may be the simplest solution for what you're trying. You would have to use tee in order to fork into 2 sub-pipelines, one for direct video, and another one going thru encoding/decoding (here using jpeg), and compose both:
gst-launch-1.0 videotestsrc ! video/x-raw,width=640,height=480,framerate=30/1 ! tee name=t t. ! queue ! comp.sink_0 t. ! queue ! jpegenc ! jpegdec ! comp.sink_1 compositor name=comp sink_0::xpos=0 sink_1::xpos=640 ! autovideosink