OpenCV 3.4.3 - GStream not called when VideoWriting - file

I'd like to succeed writing a file with OpenCV encoding with GStreamer.
Thus I'm using the code :
VideoWriter.open("appsrc ! autovideoconvert ! omxh264enc ! matroskamux ! filesink location=test2.mkv sync=false",
//cv::CAP_GSTREAMER, //set or not, I get the same problem
0, (double)25, //fourcc, fps
cv::Size(1024, 1024),
true);
And my problem :
A video file called appsrc ! autovideoconvert ! omxh264enc ! matroskamux ! filesink location=test2.mkv sync=false is created (rather than test2.mkv), and it is empty. The same problem is corrected and seems to work in the following thread, without any effect for me : Opening a GStreamer pipeline from OpenCV with VideoWriter
I tried, without success :
Add quotes to the file name test2 :
"appsrc ! autovideoconvert ! omxh264enc ! matroskamux ! filesink location=\"test2.mkv\" sync=false" neither
Encode with different format : .flv rather than .mkv
Encode with different codec : omxh265enc rather than omxh264en
I checked :
That GStreamer is well compiled with OpenCV ; the CMake of the OpenCV project outputs :
Video I/O:Video for Windows: YES
DC1394: NO
FFMPEG: YES (prebuilt binaries)
avcodec: YES (ver 57.107.100)
avformat: YES (ver 57.83.100)
avutil: YES (ver 55.78.100)
swscale: YES (ver 4.8.100)
avresample: YES (ver 3.7.0)
GStreamer:
base: YES (ver 1.0)
video: YES (ver 1.0)
app: YES (ver 1.0)
riff: YES (ver 1.0)
pbutils: YES (ver 1.0)
DirectShow: YES
Media Foundation: YES
That FFMpeg is installed on my computer ; for this I used VLC :
File -> Stream -> Add a file then stream
New destination : RTP/MPEG Transport Stream
Profile : Video - H.264 + MP3 (MP4)
Then no error happens ; Note : an error occured with Video - H.265 + MP3
(MP4), but I'm using omxh264enc in my code, not 265
I'm struggled :/ Do you have any idea ?
Thanks a lot :)
Have a good day,
Thibaut
PS - Complete OpenCV CMake output :
Selecting Windows SDK version 10.0.17134.0 to target Windows 10.0.17763.
AVX_512F is not supported by C++ compiler
AVX512_SKX is not supported by C++ compiler
Dispatch optimization AVX512_SKX is not available, skipped
libjpeg-turbo: VERSION = 1.5.3, BUILD = opencv-3.4.3-libjpeg-turbo
Looking for Mfapi.h
Looking for Mfapi.h - found
found Intel IPP (ICV version): 2017.0.3 [2017.0.3]
at: E:/OpenCV/3.4.3/build-cuda92-gstreamer/3rdparty/ippicv/ippicv_win
found Intel IPP IW sources: 2017.0.3
at: E:/OpenCV/3.4.3/build-cuda92-gstreamer/3rdparty/ippicv/ippiw_win
CUDA detected: 9.2
CUDA NVCC target flags: -gencode;arch=compute_30,code=sm_30;-gencode;arch=compute_35,code=sm_35;-gencode;arch=compute_37,code=sm_37;-gencode;arch=compute_50,code=sm_50;-gencode;arch=compute_52,code=sm_52;-gencode;arch=compute_60,code=sm_60;-gencode;arch=compute_61,code=sm_61;-gencode;arch=compute_70,code=sm_70;-D_FORCE_INLINES
Found apache ant: C:/NVPACK/apache-ant-1.8.2/bin/ant.bat (1.8.2)
Could NOT find Matlab (missing: MATLAB_MEX_SCRIPT MATLAB_INCLUDE_DIRS MATLAB_ROOT_DIR MATLAB_LIBRARIES MATLAB_LIBRARY_DIRS MATLAB_MEXEXT MATLAB_ARCH MATLAB_BIN)
VTK is not found. Please set -DVTK_DIR in CMake to VTK build directory, or to VTK install subdirectory with VTKConfig.cmake file
Caffe: NO
Protobuf: NO
Glog: YES
freetype2: NO
harfbuzz: NO
Module opencv_ovis disabled because OGRE3D was not found
No preference for use of exported gflags CMake configuration set, and no hints for include/library directories provided. Defaulting to preferring an installed/exported gflags CMake configuration if available.
Found installed version of gflags: E:/Git/gflags/BUILD-VS2015
Detected gflags version: 2.2.1
Checking SFM deps... FALSE
Module opencv_sfm disabled because the following dependencies are not found: Eigen Glog/Gflags
Excluding from source files list: <BUILD>/modules/dnn/layers/layers_common.avx512_skx.cpp
Tesseract: NO
General configuration for OpenCV 3.4.3 =====================================
Version control: unknown
Extra modules:
Location (extra): E:/OpenCV/3.4.3/opencv_contrib-3.4.3/modules
Version control (extra): unknown
Platform:
Timestamp: 2018-11-07T16:57:18Z
Host: Windows 10.0.17763 AMD64
CMake: 3.12.0-rc2
CMake generator: Visual Studio 14 2015 Win64
CMake build tool: C:/Program Files (x86)/MSBuild/14.0/bin/MSBuild.exe
MSVC: 1900
CPU/HW features:
Baseline: SSE SSE2 SSE3
requested: SSE3
Dispatched code generation: SSE4_1 SSE4_2 FP16 AVX AVX2
requested: SSE4_1 SSE4_2 AVX FP16 AVX2 AVX512_SKX
SSE4_1 (5 files): + SSSE3 SSE4_1
SSE4_2 (2 files): + SSSE3 SSE4_1 POPCNT SSE4_2
FP16 (2 files): + SSSE3 SSE4_1 POPCNT SSE4_2 FP16 AVX
AVX (6 files): + SSSE3 SSE4_1 POPCNT SSE4_2 AVX
AVX2 (11 files): + SSSE3 SSE4_1 POPCNT SSE4_2 FP16 FMA3 AVX AVX2
C/C++:
Built as dynamic libs?: YES
C++11: YES
C++ Compiler: C:/Program Files (x86)/Microsoft Visual Studio 14.0/VC/bin/x86_amd64/cl.exe (ver 19.0.24215.1)
C++ flags (Release): /DWIN32 /D_WINDOWS /W4 /GR /D _CRT_SECURE_NO_DEPRECATE /D _CRT_NONSTDC_NO_DEPRECATE /D _SCL_SECURE_NO_WARNINGS /Gy /bigobj /Oi /EHa /wd4127 /wd4251 /wd4324 /wd4275 /wd4512 /wd4589 /MP16 /MD /O2 /Ob2 /DNDEBUG
C++ flags (Debug): /DWIN32 /D_WINDOWS /W4 /GR /D _CRT_SECURE_NO_DEPRECATE /D _CRT_NONSTDC_NO_DEPRECATE /D _SCL_SECURE_NO_WARNINGS /Gy /bigobj /Oi /EHa /wd4127 /wd4251 /wd4324 /wd4275 /wd4512 /wd4589 /MP16 /MDd /Zi /Ob0 /Od /RTC1
C Compiler: C:/Program Files (x86)/Microsoft Visual Studio 14.0/VC/bin/x86_amd64/cl.exe
C flags (Release): /DWIN32 /D_WINDOWS /W3 /D _CRT_SECURE_NO_DEPRECATE /D _CRT_NONSTDC_NO_DEPRECATE /D _SCL_SECURE_NO_WARNINGS /Gy /bigobj /Oi /MP16 /MD /O2 /Ob2 /DNDEBUG
C flags (Debug): /DWIN32 /D_WINDOWS /W3 /D _CRT_SECURE_NO_DEPRECATE /D _CRT_NONSTDC_NO_DEPRECATE /D _SCL_SECURE_NO_WARNINGS /Gy /bigobj /Oi /MP16 /MDd /Zi /Ob0 /Od /RTC1
Linker flags (Release): /machine:x64 /INCREMENTAL:NO
Linker flags (Debug): /machine:x64 /debug /INCREMENTAL
ccache: NO
Precompiled headers: YES
Extra dependencies: cudart.lib nppc.lib nppial.lib nppicc.lib nppicom.lib nppidei.lib nppif.lib nppig.lib nppim.lib nppist.lib nppisu.lib nppitc.lib npps.lib cublas.lib cufft.lib -LIBPATH:C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v9.2/lib/x64
3rdparty dependencies:
OpenCV modules:
To be built: aruco bgsegm bioinspired calib3d ccalib core cudaarithm cudabgsegm cudacodec cudafeatures2d cudafilters cudaimgproc cudalegacy cudaobjdetect cudaoptflow cudastereo cudawarping cudev datasets dnn dnn_objdetect dpm face features2d flann fuzzy hfs highgui img_hash imgcodecs imgproc java java_bindings_generator line_descriptor ml objdetect optflow phase_unwrapping photo plot python_bindings_generator reg rgbd saliency shape stereo stitching structured_light superres surface_matching text tracking ts video videoio videostab xfeatures2d ximgproc xobjdetect xphoto
Disabled: js world
Disabled by dependency: -
Unavailable: cnn_3dobj cvv freetype hdf matlab ovis python2 python3 sfm viz
Applications: tests perf_tests apps
Documentation: NO
Non-free algorithms: NO
Windows RT support: NO
GUI:
Win32 UI: YES
VTK support: NO
Media I/O:
ZLib: build (ver 1.2.11)
JPEG: build-libjpeg-turbo (ver 1.5.3-62)
WEBP: build (ver encoder: 0x020e)
PNG: build (ver 1.6.34)
TIFF: build (ver 42 - 4.0.9)
JPEG 2000: build (ver 1.900.1)
OpenEXR: build (ver 1.7.1)
HDR: YES
SUNRASTER: YES
PXM: YES
Video I/O:
Video for Windows: YES
DC1394: NO
FFMPEG: YES (prebuilt binaries)
avcodec: YES (ver 57.107.100)
avformat: YES (ver 57.83.100)
avutil: YES (ver 55.78.100)
swscale: YES (ver 4.8.100)
avresample: YES (ver 3.7.0)
GStreamer:
base: YES (ver 1.0)
video: YES (ver 1.0)
app: YES (ver 1.0)
riff: YES (ver 1.0)
pbutils: YES (ver 1.0)
DirectShow: YES
Media Foundation: YES
Parallel framework: Concurrency
Trace: YES (with Intel ITT)
Other third-party libraries:
Intel IPP: 2017.0.3 [2017.0.3]
at: E:/OpenCV/3.4.3/build-cuda92-gstreamer/3rdparty/ippicv/ippicv_win
Intel IPP IW: sources (2017.0.3)
at: E:/OpenCV/3.4.3/build-cuda92-gstreamer/3rdparty/ippicv/ippiw_win
Custom HAL: NO
Protobuf: build (3.5.1)
NVIDIA CUDA: YES (ver 9.2, CUFFT CUBLAS NVCUVID)
NVIDIA GPU arch: 30 35 37 50 52 60 61 70
NVIDIA PTX archs:
OpenCL: YES (no extra features)
Include path: E:/OpenCV/3.4.3/opencv-3.4.3/3rdparty/include/opencl/1.2
Link libraries: Dynamic load
Python (for build): C:/ProgramData/Miniconda3/python.exe
Java:
ant: C:/NVPACK/apache-ant-1.8.2/bin/ant.bat (ver 1.8.2)
JNI: C:/NVPACK/jdk1.8.0_77/include C:/NVPACK/jdk1.8.0_77/include/win32 C:/NVPACK/jdk1.8.0_77/include
Java wrappers: YES
Java tests: YES
Matlab: NO
Install to: E:/OpenCV/3.4.3/build/install
-----------------------------------------------------------------
Configuring done

Ok, I've eventually found the problem's origin.
By stepping into VideoWriter::open(...) I finally noticed that std::vector backends did not contain any element with "id=CAP_GSTREAMER" -> it did not correctly compile with OpenCV. By investigating more, I discovered it was because I compiled with the x86 version of Gstreamer by mistake (whereas I was building OpenCV for x64).
Have a good day,
Thibaut

Related

Discrepancy in behavior of Linux loaders (ld-linux-x86-64) between Glibc 2.12 and Glibc 2.17

I'm trying to compile the same lib on two x86 separate machines.
Both use the same toolchain (exactly same set of files) but have different Glibc versions.
When I run command LD_DEBUG=libs /lib64/ld-linux-x86-64.so.2 --list ./libl2ps.so I notice the following discrepancy between the 2 Linux loaders:
Machine 1 (with Glibc 2.12):
19943: find library=libm.so.6 [0]; searching
19943: search path=/ebs/frperies/repo/gnb/uplane/build/prefix-root/asik-x86_64-ps_lfs-dynamic-linker-on/toolchain/sysroots/core2-64-pc-linux-gnu/usr/lib64:...:/ebs/frperies/repo/gnb/uplane/build_bbp/l2_ps/build/. (RPATH from file ./libl2ps.so)
19943: trying file=/ebs/frperies/repo/gnb/uplane/build/prefix-root/asik-x86_64-ps_lfs-dynamic-linker-on/toolchain/sysroots/core2-64-pc-linux-gnu/usr/lib64/libm.so.6
19943:
19943: find library=libgcc_s.so.1 [0]; searching
...
In this case the Linux loader selects lib libm.so.6 from the toolchain path based on RPATH of lib libl2ps.so.
Machine 2 (with Glibc 2.17):
10699: find library=libm.so.6 [0]; searching
10699: search path=/home/frperies/repo/gnb/uplane/build/prefix-root/asik-x86_64-ps_lfs-dynamic-linker-on/toolchain/sysroots/core2-64-pc-linux-gnu/usr/lib64:/home/frperies/repo/gnb/uplane/build/prefix-root/asik-x86_64-ps_lfs-dynamic-linker-on/toolchain/sysroots/core2-64-pc-linux-gnu/lib64:/home/frperies/repo/gnb/uplane/build/prefix-root/asik-x86_64-ps_lfs-dynamic-linker-on/toolchain/sysroots/core2-64-pc-linux-gnu/usr/lib:/home/frperies/repo/gnb/uplane/build_bbp/l2_ps/build/. (RPATH from file ./libl2ps.so)
10699: trying file=/home/frperies/repo/gnb/uplane/build/prefix-root/asik-x86_64-ps_lfs-dynamic-linker-on/toolchain/sysroots/core2-64-pc-linux-gnu/usr/lib64/libm.so.6
10699: trying file=/home/frperies/repo/gnb/uplane/build/prefix-root/asik-x86_64-ps_lfs-dynamic-linker-on/toolchain/sysroots/core2-64-pc-linux-gnu/lib64/libm.so.6
10699: trying file=/home/frperies/repo/gnb/uplane/build/prefix-root/asik-x86_64-ps_lfs-dynamic-linker-on/toolchain/sysroots/core2-64-pc-linux-gnu/usr/lib/libm.so.6
10699: trying file=/home/frperies/repo/gnb/uplane/build_bbp/l2_ps/build/./libm.so.6
10699: search cache=/etc/ld.so.cache
10699: trying file=/lib64/libm.so.6
As for Machine 1, the loader attempts from RPATH of libl2ps.so to select lib libm.so.6 from toolchain path but skip it for some reason and try further other paths. Finally it selects libm.so.6from the system path /lib64/.
The RPATH of the 2 libs lib2ps.so are exactly the same. The two files libm.so.6 are also exactly the same on both machines (checked with md5sum).
I don't understand this differences of behavior between the 2 Linux loaders.
Do you see any reason what would explain this discrepancy ?
Thank you very much for your answers.
Update:
Thank you yugr for your answer.
Output of readelf -h gives only differences on fields "Entry point address" and "Start of section headers" and there is no other differences so I think it will not help.
Regarding using dlopen()/dlerror(), I've done a little executable with the following statement:
dlopen("/home/frperies/repo/gnb/uplane/build/prefix-root/asik-x86_64-ps_lfs-dynamic-linker-on/toolchain/sysroots/core2-64-pc-linux-gnu/usr/lib64/libm-2.28.so", RTLD_LAZY);
On machine 1 it works as expected:
C++ dlopen demo
Opening libm-2.28.so...
Closing library...
On machine 2 it fails and dlerror() gives the following output:
Cannot open library: /home/frperies/repo/gnb/uplane/build/prefix-root/asik-x86_64-ps_lfs-dynamic-linker-on/toolchain/sysroots/core2-64-pc-linux-gnu/usr/lib64/libm-2.28.so: cannot open shared object file: No such file or directory
but the file libm-2-28.so really exists on my file system:
$ ls -l /home/frperies/repo/gnb/uplane/build/prefix-root/asik-x86_64-ps_lfs-dynamic-linker-on/toolchain/sysroots/core2-64-pc-linux-gnu/usr/lib64/libm-2.28.so
-rwxr-xr-x 1 frperies linseeusers_lte_espoo 1682944 Oct 5 13:50 /home/frperies/repo/gnb/uplane/build/prefix-root/asik-x86_64-ps_lfs-dynamic- linker-on/toolchain/sysroots/core2-64-pc-linux-gnu/usr/lib64/libm-2.28.so
This is very weird, what could lead to this situation ???
Thanks
Update 2:
That is true that I haven't pointed out that machine 1 is a RHEL6.8 distro while machine 2 is RHEL7.4 distro. I (naively?) didn't think this really matters...
On machine 1:
$ cat /proc/sys/kernel/osrelease
4.4.115-1.NSN.el6.x86_64
$ uname -a
Linux sq24-3 4.4.115-1.NSN.el6.x86_64 #1 SMP Mon Feb 12 12:35:46 CET 2018 x86_64 x86_64 x86_64 GNU/Linux
$ readelf -n libl2ps.so
Notes at offset 0x00000270 with length 0x00000024:
Owner Data size Description
GNU 0x00000014 NT_GNU_BUILD_ID (unique build ID bitstring)
Build ID: b598468830fdf2f61eda25553b9a367c4d28cdc9
On machine 2:
$ cat /proc/sys/kernel/osrelease
3.10.0-693.el7.x86_64
$ uname -a
Linux localhost.localdomain 3.10.0-693.el7.x86_64 #1 SMP Thu Jul 6 19:56:57 EDT 2017 x86_64 x86_64 x86_64 GNU/Linux
$ readelf -n libl2ps.so
Displaying notes found at file offset 0x00000270 with length 0x00000024:
Owner Data size Description
GNU 0x00000014 NT_GNU_BUILD_ID (unique build ID bitstring)
Build ID: 5829181bc0502233748149369108915ea7b10e8f
Does it help ?
Thanks
Update 3:
$ readelf -n libm.so.6
Notes at offset 0x00000238 with length 0x00000024:
Owner Data size Description
GNU 0x00000014 NT_GNU_BUILD_ID (unique build ID bitstring)
Build ID: 0d84c7247dd76008c096719043e5592735a1c4bd
Notes at offset 0x0000025c with length 0x00000020:
Owner Data size Description
GNU 0x00000010 NT_GNU_ABI_TAG (ABI version tag)
OS: Linux, ABI: 4.4.0
So, how to interpret this ABI version number set to 4.4.0 ??
Thanks
Thank you yugr and Employed Russian for your answers!!
I will give it a try by upgrading my Kernel version on Machine 2.
Thanks
Regards
The error message that you see is the infamously confusing ENOENT errno. I see two instances of it in dl-load.c:
checking OS compatibility
loading non-setuid to setuid process
I suspect the first one fails in your case which would mean that OS kernel is incompatible between two machines. ld.so manpage indeed says that
Each shared object can inform the dynamic linker of the
minimum kernel ABI version that it requires. (This
requirement is encoded in an ELF note section that is viewable
via readelf -n as a section labeled NT_GNU_ABI_TAG.) At run
time, the dynamic linker determines the ABI version of the
running kernel and will reject loading shared objects that
specify minimum ABI versions that exceed that ABI version.
NT_GNU_ABI_TAG is 4.4.0 which means that you run a program expecting a minimum 4.4 kernel on a 3.10 kernel. Theoretically newer Glibc should run on older kernels as well but in your case Glibc was probly built with explicit --enable-kernel flag which prevents it's usage on kernels before 4.4 (see e.g. this explanation of --enable-kernel).
As a workaround, you may try to fool Glibc by overriding kernel version on machine 2 via
export LD_ASSUME_KERNEL=4.4.0
but it may not work if libm makes 4.4-specific syscalls that are not really present on 3.10.

How to builld OS/ABI for SystemV using crosstool-ng

I am trying to build a kernel module for my DNS-320L NAS.
I have built crosstool-ng in a Debian lenny chroot environment but after I have compiled my kernel module and try and install it, I get:
insmod: error inserting 'kernel/net/ip4/ipip.ko': -1 Invalid module format
When I examine a working kernel module with readelf, I get:
~/ct-ng-build$ readelf -h ~/ct-ng-build/kernel/orig/ipip.ko
ELF Header:
Magic: 7f 45 4c 46 01 01 01 00 00 00 00 00 00 00 00 00
Class: ELF32
Data: 2's complement, little endian
Version: 1 (current)
OS/ABI: UNIX - System V
ABI Version: 0
Type: REL (Relocatable file)
Machine: ARM
Version: 0x1
Entry point address: 0x0
Start of program headers: 0 (bytes into file)
Start of section headers: 6696 (bytes into file)
Flags: 0x5000000, Version5 EABI
Size of this header: 52 (bytes)
Size of program headers: 0 (bytes)
Number of program headers: 0
Size of section headers: 40 (bytes)
Number of section headers: 51
But my module has a few differences:
~/ct-ng-build$ readelf -h ~/ct-ng-build/kernel/modules/lib/modules/2.6.31.8/kernel/net/ipv4/ipip.ko
ELF Header:
Magic: 7f 45 4c 46 01 01 01 61 00 00 00 00 00 00 00 00
Class: ELF32
Data: 2's complement, little endian
Version: 1 (current)
OS/ABI: ARM
ABI Version: 0
Type: REL (Relocatable file)
Machine: ARM
Version: 0x1
Entry point address: 0x0
Start of program headers: 0 (bytes into file)
Start of section headers: 6280 (bytes into file)
Flags: 0x600, GNU EABI, software FP, VFP
Size of this header: 52 (bytes)
Size of program headers: 0 (bytes)
Number of program headers: 0
Size of section headers: 40 (bytes)
Number of section headers: 23
Section header string table index: 20
My cross_compile environment is:
$ echo $CROSS_COMPILE
arm-none-eabi-
make CROSS_COMPILE=${CROSS_COMPILE} INSTALL_PATH=~/ct-ng-build/kernel/install INSTALL_MOD_PATH=~/ct-ng-build/kernel/modules INSTALL_FW_PATH=~/ct-ng-build/kernel/firmware
My gcc is compiled:
$ ${CROSS_COMPILE}gcc -v
Using built-in specs.
Target: arm-none-eabi
Configured with: ~/ct-ng-build/targets/src/gcc-4.3.2/configure --build=x86_64-build_unknown-linux-gnu --host=x86_64-build_unknown-linux-gnu --target=arm-none-eabi --prefix=~/x-tools/arm-none-eabi --with-local-prefix=~/x-tools/arm-none-eabi/arm-none-eabi//sys-root --disable-multilib --disable-libmudflap --with-sysroot=~/x-tools/arm-none-eabi/arm-none-eabi//sys-root --with-newlib --enable-threads=no --disable-shared --with-pkgversion=crosstool-NG-1.9.0 --with-arch=armv5te --with-tune=arm926ej-s --disable-__cxa_atexit --with-gmp=~/ct-ng-build/targets/arm-none-eabi/build/static --with-mpfr=~/ct-ng-build/targets/arm-none-eabi/build/static --enable-target-optspace --disable-nls --enable-symvers=gnu --enable-languages=c,c++
Thread model: single
gcc version 4.3.2 (crosstool-NG-1.9.0)
Any ideas how to make the OS/ABI to "UNIX - System V" and the Flags to "0x5000000, Version5 EABI" instead of "Arm" and "0x600, GNU EABI, software FP, VFP" ?
Or is there another problem?
Thanks!
--- EDIT ----
I am specifically trying to target my existing NAS OS:
Kernel 2.6.31.8 #1 armv5tel
C library gcc-4.3-mt-1.44.0
# /lib/libc-2.8.so
GNU C Library stable release version 2.8, by Roland McGrath et al.
Copyright (C) 2008 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.
There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A
PARTICULAR PURPOSE.
Compiled by GNU CC version 4.3.2.
I have only been able to build a crosstool-ng environment so far using crosstool-ng-1.9.0, but it produced the binaries above.
I tried to set -mabi=apcs-gnu as per 3.17.2 ARM Options but the C library would not compile with strange "missing headers" errors (I may try again - possibly something with my chroot environment).
Build a new toolchain using crosstool-NG, but configure it for OABI rather than EABI.
See Configuring crosstool-NG.
All debian releases (for ARM) upto and including Lenny were OABI.
Ref: https://wiki.embeddedarm.com/wiki/EABI_vs_OABI
Even though your toolchain prefix is arm-none-eabi-, the binary it has produced looks like OABI.
I have OABI binaries that produce identical readelf output, and EABI binaries that have identical readelf output ("UNIX - System V") to what you want.
This question describes a similar but reversed situation to yours, i.e. his toolchain is generating EABI binaries but he needs OABI.
Apparently your toolchain was built to produce OABI but a misleading prefix was used.
You need to build a new toolchain using crosstool-NG, but configure it for EABI rather than OABI.
Newer versions of crosstool-NG actually make configuration for OABI difficult by forcing the selection EABI unless Use obsolete features is enabled.
ADDENDUM
You say "but configure it for OABI rather than EABI.", but how?
Using a crosstool-NG v1.18 installation, I can specify & build
* Linux kernel version 2.6.31.14,
* gcc version 4.3.2,
* binutils version 2.18a 2.19.1a,
* glibc version 2.8.
In order to unselect Target options ---> Use EABI (unselecting means use OABI),
I first have to select Paths and misc options ---> Use obsolete features.
In this version of crosstool-NG, Use EABI is automatically selected.
The description for the Use EABI menu item has:
Set up the toolchain so that it generates EABI-compliant binaries.
If you say 'n' here, then the toolchain will generate OABI binaries.
OABI has long been deprecated, and is now considered legacy.
Since OABI is considered deprecated in this version of crosstool-NG, the config symbol ARCH_ARM_EABI_FORCE is active unless Use obsolete features is specified.
I'm extracting some final comments here:
As #sawdust recommends, I am using EABI in the following CT-NG environment:
crosstool-NG version: 1.9.0
gcc 4.3.2
binutils 2.19.1
libc 2.8
gmp 4.3.2
mpfr 2.4.2
In order to get the GCC compiler to compile, I had to do a couple of hacks!
Namely:
remove res_hconf.c from targets/src/glibc-2.8/nscd (once, after the sources have been extracted)
remove res_hconf from the list of objects in the nscd/Makefile
create a soft link to stubs-32.h from stubs-.h in ~/x-tools/arm-unknown-linux-gnueabi/arm-unknown-linux-gnueabi//sys-root/usr/include/gnu (after the build had started).
I just had to download the correct Linux kernel and install it. The missing command was:
make ARCH=arm kirkwood_defconfig
Thereafter I just had to use make ARCH=arm menuconfig to customise things.
The gcc -v says:
Using built-in specs.
Target: arm-unknown-linux-gnueabi
Configured with: ~/ct-ng-build/targets/src/gcc-4.3.2/configure --build=x86_64-build_unknown-linux-gnu --host=x86_64-build_unknown-linux-gnu --target=arm-unknown-linux-gnueabi --prefix=~/x-tools/arm-unknown-linux-gnueabi --with-sysroot=~/x-tools/arm-unknown-linux-gnueabi/arm-unknown-linux-gnueabi//sys-root --enable-languages=c,c++ --disable-multilib --with-arch=armv5te --with-tune=arm926ej-s --with-float=soft --disable-shared --with-pkgversion=crosstool-NG-1.9.0 --disable-sjlj-exceptions --enable-__cxa_atexit --disable-libmudflap --disable-libgomp --disable-libssp --with-gmp=~/ct-ng-build/targets/arm-unknown-linux-gnueabi/build/static --with-mpfr=~/ct-ng-build/targets/arm-unknown-linux-gnueabi/build/static --enable-threads=posix --enable-target-optspace --with-local-prefix=~/x-tools/arm-unknown-linux-gnueabi/arm-unknown-linux-gnueabi//sys-root --disable-nls --enable-symvers=gnu --enable-c99 --enable-long-long
Thread model: posix
gcc version 4.3.2 (crosstool-NG-1.9.0)
Finally, to generate SYSV modules, I had to hack the kernel arch/arm/Makefile to remove the -mabi=aapcs-linux setting from the EABI section.
Now the error is:
"modprobe: can't load module ntfs (kernel/fs/ntfs/ntfs.ko): unknown symbol in module, or unknown parameter
Which is for another question :D
I still can't build a working kernel module (maybe because of the hacks!) but it might also be because I don't have a compatible kernel .config for the NAS.
The final changes were:
Not use unwind
Disable some kernel debugging and tracing
use SLAB instead of SLUB
The changes were selected by tracking the dmsg messages.
The resulting compiled kernel modules can now be loaded, if they are compatible with the compiled kernel.
(Note of warning: I got into a little trouble when I started adding iptables modules when I got locked out of my network connection! Be careful! Fortunately the modules are not reloaded at reboot. )

Error when running DPDK app on valgrind

When I run my DPDK based app on valgrind, it cannot execute it and throws error
ERROR: This system does not support "RDRAND". Please check that
RTE_MACHINE is set correctly.
My CPU supports RDRAND, still it is throwing the same error. For valgrind to support hugepages which are being used by my app, I'm using the following patched version of valgrind.
https://github.com/bisdn/valgrind-hugepages.git
I had this same problem on a Haswell architecture CPU, and was able to fix it by modifying one of the makefiles to remove a handful of options. Specifically, AVX/AVX2, RDRND, FSGSBASE, and F16C. You might need to remove other options that valgrind is balking at and recompile DPDK, it was an iterative process for me. There's probably a more elegant way to do this using the .config file but I didn't find it. See this patch:
diff -u dpdk-2.2.0-orig/mk/rte.cpuflags.mk dpdk-2.2.0/mk/rte.cpuflags.mk
--- dpdk-2.2.0-orig/mk/rte.cpuflags.mk^I2015-12-15 12:06:58.000000000 -0500
+++ dpdk-2.2.0/mk/rte.cpuflags.mk^I2016-08-24 08:53:22.911258783 -0400
## -69,26 +69,6 ##
CPUFLAGS += PCLMULQDQ
endif
-ifneq ($(filter $(AUTO_CPUFLAGS),__AVX__),)
-CPUFLAGS += AVX
-endif
-
-ifneq ($(filter $(AUTO_CPUFLAGS),__RDRND__),)
-CPUFLAGS += RDRAND
-endif
-
-ifneq ($(filter $(AUTO_CPUFLAGS),__FSGSBASE__),)
-CPUFLAGS += FSGSBASE
-endif
-
-ifneq ($(filter $(AUTO_CPUFLAGS),__F16C__),)
-CPUFLAGS += F16C
-endif
-
-ifneq ($(filter $(AUTO_CPUFLAGS),__AVX2__),)
-CPUFLAGS += AVX2
-endif
-
# IBM Power CPU flags
ifneq ($(filter $(AUTO_CPUFLAGS),__PPC64__),)
CPUFLAGS += PPC64
RDRAND was introduced on IvyBridge, you can build dpdk with a specific subset of instructions using "CONFIG_RTE_MACHINE". For this case you can use SandyBridge as the machine.
Modify $RTE_SDK/$RTE_TARGET/.config, set CONFIG_RTE_MACHINE="snb", and rebuild the DPDK library (make -C $RTE_SDK/$RTE_TARGET).
I found another solution to this problem. DPDK supports EXTRA_CFLAGS variable which you can use to specify your own flags for GCC. Initial makefile runs gcc with options -dN -E to check what is supported by the platform. If you want to disable some instruction sets, e.g. RDRAND, you can specify option
export EXTRA_CFLAGS=-mno-rdrnd
and this will disable RDRAND in built DPDK library binaries.

Verification of poly1305-donna-16.h code

The RFC from May 2015 Y. Nir et al, ChaCha20 and Poly1305 for IETF Protocols
(https://www.rfc-editor.org/rfc/rfc7539) contains a reference to the MIT/Public domain C library https://github.com/floodyberry/poly1305-donna.
I am just porting the C code to Pascal. The 8 bit code works OK (self-test, example, and RFC test vectors).
The port from poly1305-donna-16.h using 16->32 bit multiples and 32 bit additions failed. After some testing I compiled the original source with DJGPP GCC 4.7.3, MS VC 6.0 and BC 3.1 and all three failed too (poly1305 self-test).
Questions: Does this C version (build with -DPOLY1305_16BIT) fail for other compilers too? Is there a known fix available? (The blog of the author Andrew Moon at https://floodyberry.wordpress.com/ is inactive since 6 years)
I can confirm a build failure on a pretty vanilla Fedora 22 system:
% gcc poly1305-donna.c -c -DPOLY1305_16BIT
% gcc example-poly1305.c -o ex poly1305-donna.o -DPOLY1305_16BIT
% ./ex
poly1305 self test: failed
Notice that the test succeeds when I omit -DPOLY1305_16BIT.
Also notice:
% uname -rmp
4.0.8-300.fc22.x86_64 x86_64 x86_64
% gcc --version
gcc (GCC) 5.1.1 20150618 (Red Hat 5.1.1-4)
I suggest you submit a bug report. Andrew has been responsive in the past.
EDIT:
Compiling with clang version 3.5.0 yields the same results as the above gcc test.

ffmpeg - operation not permitted error while conversion

I am developing an android app. My requirement is that to implement an rtsp streaming server on android. It has to live stream video and audio captured using MediaRecorder. Another requirement is that I have to use live555 as the streaming server. What I get from MediaRecorder is in MP4 or 3GP format. live555 cannot able to stream both. But it can stream audio if I recorded it only in 'RAW_AMR' format. Since live555 support 'mpg' format for streaming, I decided to put someone in middle who can convert 'mp4' or '3gp' to 'mpg', and I chose ffmpeg.
I have ported live555 and ffmpeg to android. ffmpeg is able to convert the file recorded by MediaRecorder once it is finished. But the problem is that ffmpeg cannot be able to do it concurrently. That is, ffmpeg is not able to convert the file while recording. It shows an Operation not permitted error. I tried the same on my linux machine, using VLC to record instead of MediaRecorder on android. The result is same. ffmpeg is able to convert once the recording is finished, and not able to do the same while recording.
Here is the ffmpeg command I issued on my linux box:
ffmpeg -v 9 -loglevel 99 -i test.mp4 test.mpg
Where test.mp4 is the file to which VLC is recording in mp4 format. and test.mpg is my destination file. The following is the output by ffmpeg on terminal.
ffmpeg version 0.8.9, Copyright (c) 2000-2011 the FFmpeg developers
built on Feb 1 2012 18:29:27 with gcc 4.6.2 20111027 (Red Hat 4.6.2-1)
configuration: --prefix=/usr --bindir=/usr/bin --datadir=/usr/share/ffmpeg --incdir=/usr/include/ffmpeg --libdir=/usr/lib64 --mandir=/usr/share/man --arch=x86_64 --extra-cflags='-O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic' --enable-bzlib --enable-libcelt --enable-libdc1394 --enable-libdirac --enable-libfreetype --enable-libgsm --enable-libmp3lame --enable-libopenjpeg --enable-librtmp --enable-libschroedinger --enable-libspeex --enable-libtheora --enable-libvorbis --enable-libvpx --enable-libx264 --enable-libxvid --enable-x11grab --enable-avfilter --enable-postproc --enable-pthreads --disable-static --enable-shared --enable-gpl --disable-debug --disable-stripping --shlibdir=/usr/lib64 --enable-runtime-cpudetect
libavutil 51. 9. 1 / 51. 9. 1
libavcodec 53. 8. 0 / 53. 8. 0
libavformat 53. 5. 0 / 53. 5. 0
libavdevice 53. 1. 1 / 53. 1. 1
libavfilter 2. 23. 0 / 2. 23. 0
libswscale 2. 0. 0 / 2. 0. 0
libpostproc 51. 2. 0 / 51. 2. 0
[mov,mp4,m4a,3gp,3g2,mj2 # 0x1672600] Format mov,mp4,m4a,3gp,3g2,mj2 probed with size=2048 and score=100
[mov,mp4,m4a,3gp,3g2,mj2 # 0x1672600] ISO: File Type Major Brand: isom
[mov,mp4,m4a,3gp,3g2,mj2 # 0x1672600] moov atom not found
test.mp4: Operation not permitted
Would anyone please tell me what is causing the problem? Or is the scenario above is possible by ffmpeg. That is, is ffmpeg is able to do the conversion at the same time as that of recording? If it is not possible by ffmpeg, would you please suggest any alternative solutions?
NOTE: I am putting a C tag because if it possible by some tweaking in C on ffmpeg, I am ready to do that(I want the solution that badly). But please provide some pointers to the right direction.
Both 3gp and mp4 formats include moov atom (chunk of data) that is written when the file is finalized. Until then the file is incomplete.
You can use FLV as the "middle" format. Other formats that support live streaming can be used too. Option -re may be helpful to tell encoder to run at the stream rate, if needed.
See also Is it possible to play an output video file from an encoder as it's being encoded?.

Resources