Since upgrading my PC from Fedora 35 to Fedora 36, kdenlive has failed to render. I traced the problem to mlt-melt.
$ mlt-melt export.mlt
+-----+ +-----+ +-----+ +-----+ +-----+ +-----+ +-----+ +-----+ +-----+
|1=-10| |2= -5| |3= -2| |4= -1| |5= 0| |6= 1| |7= 2| |8= 5| |9= 10|
+-----+ +-----+ +-----+ +-----+ +-----+ +-----+ +-----+ +-----+ +-----+
+---------------------------------------------------------------------+
| H = back 1 minute, L = forward 1 minute |
| h = previous frame, l = next frame |
| g = start of clip, j = next clip, k = previous clip |
| 0 = restart, q = quit, space = play |
+---------------------------------------------------------------------+
[consumer avformat] error with audio encode: -22 (frame 1)
[consumer avformat] error with audio encode: -22 (frame 11)
[consumer avformat] error with audio encode: -22 (frame 21)
Current Position: 20
$ mlt-melt --version
mlt-melt 7.4.0
...
$ dnf list mlt\*
...
Installed Packages
mlt.x86_64 7.4.0-3.fc36 #fedora
mlt-freeworld.x86_64 7.4.0-1.fc36 #rpmfusion-free
...
The exact same mlt file (which I generated on kdenlive 22.04.0 on F36) works fine on Fedora 35, which has mlt-melt 6.26.1 installed. I didn't see any instructions that additional steps were needed to make mlt work on F36. I get the same "audo encode -22" errors if I pull a working .mlt file from F35 to F36.
Any thoughts or suggestions? I cannot roll back to F35. I need a go-forward solution. Thanks.
Error code -22 maps to the posix error EINVAL (Invalid argument). THis is the code FFMpeg is returning from the MLT call to avcodec_receive_packet().
https://github.com/mltframework/mlt/blob/0e0e526124f4392275bdeb2510e8fb086d23da8f/src/modules/avformat/consumer_avformat.c#L1376
If you trace the code in FFMpeg, there are any number of places that EINVAL is returned: https://ffmpeg.org/doxygen/trunk/encode_8c_source.html#l00391
I think the most likely is that your FFMpeg does not have the codec you are trying to use - but you say it works when you use FFMpeg directly. Unless MLT is not finding the same FFMpeg that you are testing with. A good way to see if MLT is finding the codecs you want is to run:
melt -query audio_codecs
But that is just a guess. Maybe EINVAL error code is caused by a different problem.
As for other ideas... Well, you could use the AppImage or Flatpack provided by the developers themselves:
https://kdenlive.org/en/download/
Related
I tried to concatenate a number of video files:
$ melt file01.mp4 file02.mp4 filen.mp4 170_ot_proof.mp4 170_thanks.mp4 -consumer "avformat:total.mp4" acodec=libmp3lame vcodec=libx264
+-----+ +-----+ +-----+ +-----+ +-----+ +-----+ +-----+ +-----+ +-----+
|1=-10| |2= -5| |3= -2| |4= -1| |5= 0| |6= 1| |7= 2| |8= 5| |9= 10|
+-----+ +-----+ +-----+ +-----+ +-----+ +-----+ +-----+ +-----+ +-----+
+---------------------------------------------------------------------+
| H = back 1 minute, L = forward 1 minute |
| h = previous frame, l = next frame |
| g = start of clip, j = next clip, k = previous clip |
| 0 = restart, q = quit, space = play |
+---------------------------------------------------------------------+
[mp4 # 0x7fc5b4001180] Using AVStream.codec to pass codec parameters to muxers is deprecated, use AVStream.codecpar instead.
[mp4 # 0x7fc5b4001180] Using AVStream.codec to pass codec parameters to muxers is deprecated, use AVStream.codecpar instead.
[mp4 # 0x7fc5b4001180] Timestamps are unset in a packet for stream 1. This is deprecated and will stop working in the future. Fix your code to set the timestamps properly
[mp4 # 0x7fc5b4001180] Encoder did not produce proper pts, making some up.
Current Position: 4599
The problem is that the produced file is 1 frame too long:
$ function get_fps() {melt "$1" -consumer xml | grep length | sed 's/[^0-9]//g';}
$ get_fps "total.mp4"
4601
$ tot=0; for file in file01.mp4 file02.mp4 filen.mp4 170_ot_proof.mp4 170_thanks.mp4; do tot=$(($tot + $(get_fps "$file"))); done; echo $tot
4600
After inspecting with KDEnlive, it seems like the added frame is in the last clip, not sure if it's just the last frame that is duplicated or another one.
This always occurs for the same set of files, but if I remove some files the issue is not always here… I made sure to set all files at 24fps.
I really don't understand what's going wrong here. Any idea?
Edit
On the other hand, ffprobe gives:
$ ffprobe -select_streams v -show_streams ../total.mp4 | grep nb_frame
nb_frames=4600
and this also gives the same number (but takes much more time, like if it were decoding the video instead of reading the headers):
$ ffprobe -v error -select_streams v:0 -count_frames -show_entries stream=nb_read_frames -print_format csv ../total.mp4
stream,4600
I also tried to use cv2.CAP_PROP_FRAME_COUNT that also gives me 4600, so I'm a bit lost… If KDEnlive/melt give a different number than ffprobe, who should I trust?
I haven't programmed C for quite some time and my pointer-fu had degraded. I made a very elementary mistake and it took me well over an hour this morning to find what I'd done. The bug is minimally reproduced here: https://godbolt.org/z/3MdzarP67 (I am aware the program is absurd memory-management wise, just showing what happens).
The first call to realloc() breaks because of course, the pointer it's given points to stack memory, valgrind made this quite obvious.
I have a rule with myself where any time I track down a bug, if there is a warning that could have caught it I enable it on my projects. Often times this is not the case, since many bugs come from logic errors the compiler can't be expected to check.
However here I am a bit surprised. We malloc() and then immediately reassign that pointer which leaves the allocated memory inaccessible. It's obvious the returned pointer does not live outside the scope of that if block, and is never free()'d. Maybe it's too much to expect the compiler to analyze the calls and realize we're attempting to realloc() stack memory but I am surprised that I can't find anything to yell at me about the leaking of the malloc() returned pointer. Even clang's static analyzer scan-build doesn't pick up on it, I've tried various relevant options.
The best I could find was -fsanitize=address which at least prints out some cluing information during the crash instead of:
mremap_chunk(): invalid pointer
on Godbolt, or
realloc(): invalid old size
Aborted (core dumped)
on my machine, both of which are somewhat cryptic (although yes they do show plainly that there is some memory issue occurring). Still, this compiles without issues.
Since Godbolt links don't live forever here is the critical section of the code:
void add_foo_to_bar(struct Bar** b, Foo* f) {
if ((*b)->foos == NULL) {
(*b)->foos = (Foo*)malloc(sizeof(Foo));
// uncomment for correction
//(*b)->foos[(*b)->n_foos] = *f;
// obvious bug here, we leak memory by losing the returned pointer from malloc
// and assign the pointer to a stack address (&f1)
// comment below line for correction
(*b)->foos = f; // completely wrong
(*b)->n_foos++;
} else {
(*b)->foos = (Foo*)realloc((*b)->foos, ((*b)->n_foos + 1) * sizeof(Foo));
(*b)->foos[(*b)->n_foos] = *f;
(*b)->n_foos++;
}
}
the error occurs because f is a pointer to stack memory (intentional) but we obviously can't assign something that was supposed to have been malloc()'d to that.
Try -fanalyzer if your compiler is recent enough. When running it I get:
../main.c:30:28: warning: ‘realloc’ of ‘&f1’ which points to memory not on the heap [CWE-590] [-Wanalyzer-free-of-non-heap]
30 | (*b)->foos = (Foo*)realloc((*b)->foos, ((*b)->n_foos + 1) * sizeof(Foo));
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
‘main’: events 1-2
|
| 37 | int main() {
| | ^~~~
| | |
| | (1) entry to ‘main’
|......
| 45 | add_foo_to_bar(&b, &f1);
| | ~~~~~~~~~~~~~~~~~~~~~~~
| | |
| | (2) calling ‘add_foo_to_bar’ from ‘main’
|
+--> ‘add_foo_to_bar’: events 3-5
|
| 19 | void add_foo_to_bar(struct Bar** b, Foo* f) {
| | ^~~~~~~~~~~~~~
| | |
| | (3) entry to ‘add_foo_to_bar’
| 20 | if ((*b)->foos == NULL) {
| | ~
| | |
| | (4) following ‘true’ branch...
| 21 | (*b)->foos = (Foo*)malloc(sizeof(Foo));
| | ~~~~
| | |
| | (5) ...to here
|
<------+
|
‘main’: events 6-7
|
| 45 | add_foo_to_bar(&b, &f1);
| | ^~~~~~~~~~~~~~~~~~~~~~~
| | |
| | (6) returning to ‘main’ from ‘add_foo_to_bar’
| 46 | add_foo_to_bar(&b, &f2);
| | ~~~~~~~~~~~~~~~~~~~~~~~
| | |
| | (7) calling ‘add_foo_to_bar’ from ‘main’
|
+--> ‘add_foo_to_bar’: events 8-11
|
| 19 | void add_foo_to_bar(struct Bar** b, Foo* f) {
| | ^~~~~~~~~~~~~~
| | |
| | (8) entry to ‘add_foo_to_bar’
| 20 | if ((*b)->foos == NULL) {
| | ~
| | |
| | (9) following ‘false’ branch...
|......
| 30 | (*b)->foos = (Foo*)realloc((*b)->foos, ((*b)->n_foos + 1) * sizeof(Foo));
| | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
| | | |
| | | (10) ...to here
| | (11) call to ‘realloc’ here
|
No, but, runtime testing can save you.
If you can spare the execution overhead, I have seen many applications add an extra layer to memory allocation to track the allocations made and find leaks/errors.
Usually they replace malloc() and free() with a macros that include FILE and LINE
One example can be seen here (check the Heap.c and Heap.h files)
https://github.com/eclipse/paho.mqtt.c/tree/master/src
Googling "memory heap debugger" will probably turn up other examples. Or you could roll your own.
i run this command
./darknet detector train data/obj.data cfg/yolov3_training.cfg back/last_4_4_7pm.weights /back -dont_show -gpus 0
but gou is not been used and 0 %
here is my makefile;:
%cd darknet
!sed -i 's/OPENCV=0/OPENCV=1/' Makefile
!sed -i 's/GPU=0/GPU=1/' Makefile
!sed -i 's/CUDNN=0/CUDNN=1/' Makefile
here is the out put
CUDA-version: 11020 (11000)
Warning: CUDA-version is higher than Driver-version!
, cuDNN: 8.1.0, GPU count: 1
OpenCV version: 3.4.11
0
yolov3_training
0 : compute_capability = 610, cudnn_half = 0, GPU: Quadro P5000
net.optimized_memory = 0
mini_batch = 4, batch = 64, time_steps = 1, train = 1
layer filters size/strd(dil) input output
0 Create CUDA-stream - 0
Create cudnn-handle 0
here is my nvidia smi:
root#n5qr6jidhm:/notebooks/Untitled Folder/darknet# nvidia-smi
Fri Jun 25 17:53:45 2021
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 450.36.06 Driver Version: 450.36.06 CUDA Version: 11.0 |
|-------------------------------+----------------------+----------------------+
| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|===============================+======================+======================|
| 0 Quadro P5000 On | 00000000:00:05.0 Off | Off |
| 39% 62C P0 126W / 180W | 5725MiB / 16278MiB | 92% Default |
| | | N/A |
+-------------------------------+----------------------+----------------------+
First, start with upgrading(downgrading) your drives by gpu model. That can be easily found on google. Then check driver versions with darknet min. req. before installing them. Don't use -gpus flag as nothing changes.
I was wondering if there is a preferred way to print a nice visual representation of a Graph, like the one seen here:
+------+
In1 ~>| |~> Out1
| bidi |
Out2 <~| |<~ In2
+------+
I am using version 3.0.3, and running my queries in the shell.
I have ~58 million record nodes with 4 properties each, specifically an ID string, a epoch time integer, and lat/lon floats.
When I run a query like profile MATCH (r:record) RETURN count(r); I get a very quick response:
+----------+
| count(r) |
+----------+
| 58430739 |
+----------+
1 row
29 ms
Compiler CYPHER 3.0
Planner COST
Runtime INTERPRETED
+--------------------------+----------------+------+---------+-----------+--------------------------------+
| Operator | Estimated Rows | Rows | DB Hits | Variables | Other |
+--------------------------+----------------+------+---------+-----------+--------------------------------+
| +ProduceResults | 7644 | 1 | 0 | count(r) | count(r) |
| | +----------------+------+---------+-----------+--------------------------------+
| +NodeCountFromCountStore | 7644 | 1 | 0 | count(r) | count( (:record) ) AS count(r) |
+--------------------------+----------------+------+---------+-----------+--------------------------------+
Total database accesses: 0
The Total database accesses: 0 and NodeCountFromCountStore tells me that neo4j uses a counting mechanism here that avoids iterating over all the nodes.
However, when I run profile MATCH (r:record) WHERE r.time < 10000000000 RETURN count(r);, I get a very slow response:
+----------+
| count(r) |
+----------+
| 58430739 |
+----------+
1 row
151278 ms
Compiler CYPHER 3.0
Planner COST
Runtime INTERPRETED
+-----------------------+----------------+----------+----------+-----------+------------------------------+
| Operator | Estimated Rows | Rows | DB Hits | Variables | Other |
+-----------------------+----------------+----------+----------+-----------+------------------------------+
| +ProduceResults | 1324 | 1 | 0 | count(r) | count(r) |
| | +----------------+----------+----------+-----------+------------------------------+
| +EagerAggregation | 1324 | 1 | 0 | count(r) | |
| | +----------------+----------+----------+-----------+------------------------------+
| +NodeIndexSeekByRange | 1752922 | 58430739 | 58430740 | r | :record(time) < { AUTOINT0} |
+-----------------------+----------------+----------+----------+-----------+------------------------------+
Total database accesses: 58430740
The count is correct, as I chose a time value larger than all of my records. What surprises me here is that Neo4j is accessing EVERY single record. The profiler states that Neo4j is using the NodeIndexSeekByRange as an alternative method here.
My question is, why does Neo4j access EVERY record when all it is returning is a count? Are there no intelligent mechanisms inside the system to count a range of values after seeking the boundary/threshold value within the index?
I use Apache Solr for the same data, and returning a count after searching an index is extremely fast (about 5 seconds). If I recall correctly, both platforms are built on top of Apache Lucene. While I don't know much about that software internally, I would assume that the index support is fairly similar for both Neo4j and Solr.
I am working on a proxy service that will deliver results in a paginated form (using the SKIP n LIMIT m technique) by first getting a count, and then iterating over results in chunks. This works really well for Solr, but I am afraid that Neo4j may not perform well in this scenario.
Any thoughts?
The later query does a NodeIndexSeekByRange operation. This is going through all your matched nodes with the record label to look up the value of the node property time and does a comparison if its value is less than 10000000000.
This query actually has to get every node and read some info for comparison, and that's the reason why it is much slower.