I am trying to fuzz pngquant using AFL and noticed that I am seeing the odd, check syntax! warning. A previous answer says that AFL is probably not reading the input file I specified. I am able to invoke pngquant by providing the png file located in the same input sample directory I used while fuzzing with AFL.
The AFL command I executed is:
afl-fuzz -i ./in-filtered -o ./out -x ./dictionary/png.dict -- pngquant ##
The AFL documentation mentions that the odd, check syntax! warning may pop up when AFL is not able to find new paths.
Additionally, I notice that the warning pops up as soon as AFL begins using the havoc fuzzing strategy, the images below show that the cycle counts start incrementing as soon as the fuzzer begins using havoc.
This is because of incorrect usage of pngquant.
When using pngquant, it produces an output file with your results, when you do it twice you will encounter an error:
➜ pngquant git:(master) ✗ ./pngquant ./test/img/metadata.png
➜ pngquant git:(master) ✗ ./pngquant ./test/img/metadata.png
error: './test/img/metadata-fs8.png' exists; not overwriting
AFL has no chance to explore the target, since it gets blocked every time.
After a quick look, the easiest fix is like this:
afl-fuzz -i ./in -o ./out -- ./pngquant -f -- ##
This forces pngquant to overwrite the resultfile and therefore enables afl-fuzz. However, be aware that this produces a lot of IO. So try to circumvent this using /dev/null or similar tricks.
Happy fuzzing!
I've been compiling OpenSSL (and thus Perl >5.10, as it is a dependency) on multiple platforms. I've managed to get 1.1.0b compiled on every single platform except AIX, which I can't even compile Perl. I've tried several versions and looked at the documentation Perl provides online. From what I can tell, it suggests version 5.12.2.
When I attempt to compile version 5.12.2,
I take the following from the documentation, and fill in a few local system variables, such as using XLC rv7.
export OBJECT_MODE=64
./Configure \
-d \
-Dcc=/usr/vac/bin/xlc_r7 \
-Duseshrplib \
-Duse64bitall \
-Dprefix=`pwd`/../PERL
Then I attempt to make as prompted, and get the following error:
/usr/vac/bin/xlc_r7 -q64 -o miniperl -brtl -bdynamic -L/usr/local/lib -b64 gv.o toke.o perly.o pad.o regcomp.o dump.o util.o mg.o reentr.o mro.o hv.o av.o run.o pp_hot.o sv.o pp.o scope.o pp_ctl.o pp_sys.o doop.o doio.o regexec.o utf8.o taint.o deb.o universal.o globals.o perlio.o perlapi.o numeric.o mathoms.o locale.o pp_pack.o pp_sort.o miniperlmain.o opmini.o perlmini.o -lbind -lnsl -ldl -lld -lm -lcrypt -lc
LIBPATH=.../perl-5.12.2 ./miniperl -w -Ilib -MExporter -e '<?>' || make minitest
LIBPATH=.../perl-5.12.2 ./miniperl -Ilib autodoc.pl
/usr/bin/ln -s perl5122delta.pod pod/perldelta.pod
LIBPATH=.../perl-5.12.2 ./miniperl -Ilib -Icpan/Cwd -Icpan/Cwd/lib pod/perlmodlib.PL -q
readdir(./../../../../..): Bad file number at lib/FindBin.pm line 116
stat(/pod/): No such file or directory at lib/FindBin.pm line 197
stat(/pod/): No such file or directory at lib/FindBin.pm line 200
Use of chdir('') or chdir(undef) as chdir() is deprecated at pod/perlmodlib.PL line 9.
No such file or directory at pod/perlmodlib.PL line 19.
make: The error code from the last command is 2.
Taking a look at pod/perlmodlib.PL, we see the following:
# MANIFEST itself is Unix style filenames, so we have to assume that Unix style
# filenames will work.
open (MANIFEST, "../MANIFEST") or die $!;
In my desperation I tried to hack it up and avoid writing to the manifest, but then I get this issue:
Creating Makefile.PL in cpan/Archive-Extract for Archive::Extract
Running Makefile.PL in cpan/Archive-Extract
../../miniperl Makefile.PL INSTALLDIRS=perl INSTALLMAN1DIR=none INSTALLMAN3DIR=none PERL_CORE=1 LIBPERL_A=libperl.a
readdir(./../../../../../../..): No such file or directory at ../../lib/File/Find.pm line 610
Use of chdir('') or chdir(undef) as chdir() is deprecated at ../../lib/File/Find.pm line 773.
readdir(./../../..): No such file or directory at ../../cpan/Cwd/lib/File/Spec/Unix.pm line 483
Could not open 'lib/Archive/Extract.pm': No such file or directory at ../../cpan/ExtUtils-MakeMaker/lib/ExtUtils/MM_Unix.pm line 2588.
512 from cpan/Archive-Extract's Makefile.PL at make_ext.pl line 390.
Warning: No Makefile!
make: Cannot find a rule to create target config from dependencies.
Stop.
make config PERL_CORE=1 LIBPERL_A=libperl.a failed, continuing anyway...
Making all in cpan/Archive-Extract
make all PERL_CORE=1 LIBPERL_A=libperl.a
make: Cannot find a rule to create target all from dependencies.
Stop.
Unsuccessful make(cpan/Archive-Extract): code=512 at make_ext.pl line 449.
make: The error code from the last command is 2.
The frequency of errors makes me feel like perhaps I don't have something configured right...
My next thought was to try the most updated version, which at the time of writing is 5.24. Using the same configuration and attempting to make I get the following issue:
Can't locate strict.pm in #INC (you may need to install the strict module) (#INC contains: /cpan/AutoLoader/lib /dist/Carp/lib /dist/PathTools /dist/PathTools/lib /cpan/ExtUtils-Install/lib /cpan/ExtUtils-MakeMaker/lib /cpan/ExtUtils-Manifest/lib /cpan/File-Path/lib /ext/re /dist/Term-ReadLine/lib /dist/Exporter/lib /ext/File-Find/lib /cpan/Text-Tabs/lib /dist/constant/lib /cpan/version/lib /lib .) at autodoc.pl line 25.
BEGIN failed--compilation aborted at autodoc.pl line 25.
make: The error code from the last command is 2.
Which I know from other compilations means I need to edit the PERL5LIB variable.
If I keep adding all the modules to the path:
export PERL5LIB=`pwd`/dist/Exporter/lib:$PERL5LIB
export PERL5LIB=`pwd`/cpan/Text-Tabs/lib:$PERL5LIB
export PERL5LIB=`pwd`/ext/re:$PERL5LIB
export PERL5LIB=`pwd`/dist/constant/lib:$PERL5LIB
export PERL5LIB=`pwd`/cpan/ExtUtils-MakeMaker/lib:$PERL5LIB
export PERL5LIB=`pwd`/dist/Carp/lib:$PERL5LIB
export PERL5LIB=`pwd`/cpan/File-Path/lib:$PERL5LIB
export PERL5LIB=`pwd`/dist/PathTools:$PERL5LIB
export PERL5LIB=`pwd`/dist/PathTools/lib:$PERL5LIB
export PERL5LIB=`pwd`/ext/File-Find/lib:$PERL5LIB
Even so, I'll still get an error for a module that isn't even present in the 5.24.0 source!
Can't locate ExtUtils/MakeMaker/version/vpp.pm in #INC (you may need to install the ExtUtils::MakeMaker::version::vpp module) (#INC contains: <removed from post>) at (eval 2) line 2.
BEGIN failed--compilation aborted at (eval 2) line 2.
Compilation failed in require at .../perl-5.24.0/cpan/ExtUtils-MakeMaker/lib/ExtUtils/MakeMaker.pm line 10.
BEGIN failed--compilation aborted at .../perl-5.24.0/cpan/ExtUtils-MakeMaker/lib/ExtUtils/MakeMaker.pm line 10.
Compilation failed in require at Makefile.PL line 7.
BEGIN failed--compilation aborted at Makefile.PL line 7.
Unsuccessful Makefile.PL(cpan/Archive-Tar): code=512 at make_ext.pl line 517.
make: The error code from the last command is 2.
I have MakeMaker, but the 5.24 version does not contain vpp.pm! Again, in desperation I attempted to put it in there from a different source, and I get this error:
LIBPATH=.../perl-5.24.0 ./miniperl -Ilib make_ext.pl cpan/Archive-Tar/pm_to_blib MAKE="make" LIBPERL_A=libperl.a
readdir(./../../../../../../..): No such file or directory at .../perl-5.24.0/ext/File-Find/lib/File/Find.pm line 142.
Can't cd to : No such file or directory
Unsuccessful Makefile.PL(cpan/Archive-Tar): code=512 at make_ext.pl line 517.
make: The error code from the last command is 2.
All of this makes me feel like perhaps I'm not configuring something right... Can anyone with some experience help me out with some cut and dry installation instructions for Perl on AIX? I'd be super grateful. Thanks!
Below is some information about my system:
> prtconf
System Model: IBM,8231-E1D
Machine Serial Number: Not Available
Processor Type: Not Available
Processor Implementation Mode: POWER 7
Processor Version: PV_7_Compat
Number Of Processors: 0
Processor Clock Speed: Not Available
CPU Type: 64-bit
Kernel Type: 64-bit
Memory Size: 10240 MB
Good Memory Size: Not Available
Platform Firmware level: Not Available
Firmware Version: IBM,AL770_092
Console Login: disable
Auto Restart: true
Full Core: false
I pull down and compile openssh and openssl frequently. I use gcc and no real magic that I can recall.
Here is my script to build it. I don't know which perl is native to AIX but it seems to work for me.
https://github.com/pedz/aix-build-scripts/blob/master/build-scripts/build-openssl
what have i done wrong (or didn't do) that gdb is not working properly for me?
root#6be3d60ab7c6:/# cat minimal.c
int main()
{
int i = 1337;
return 0;
}
root#6be3d60ab7c6:/# gcc -g minimal.c -o minimal
root#6be3d60ab7c6:/# gdb minimal
GNU gdb (Ubuntu 7.7.1-0ubuntu5~14.04.2) 7.7.1
.
.
.
Reading symbols from minimal...done.
(gdb) break main
Breakpoint 1 at 0x4004f1: file minimal.c, line 3.
(gdb) run
Starting program: /minimal
warning: Error disabling address space randomization: Operation not permitted
During startup program exited normally.
(gdb)
(gdb) print i
No symbol "i" in current context.
If you're using Docker, you probably need the --security-opt seccomp=unconfined option (as well as enabling ptrace):
docker run --cap-add=SYS_PTRACE --security-opt seccomp=unconfined
For whatever reason, your user account doesn't have permission to disable the kernel's address space layout randomisation for this process. By default, gdb turns this off because it makes some sorts of debugging easier (in particular, it means the address of stack objects will be the same each time you run your program). Read more here.
You can work around this problem by disabling this feature of gdb with set disable-randomization off.
As for getting your user the permission needed to disable ASLR, it probably boils down to having write permission to /proc/sys/kernel/randomize_va_space. Read more here.
Building on wisbucky's answer (thank you!), here are the same settings for Docker compose:
security_opt:
- seccomp:unconfined
cap_add:
- SYS_PTRACE
The security option seccomp:unconfined fixed the address space randomization warnings.
The capability SYS_PTRACE didn't seem to have a noticeable effect even though the Docker documentation states that SYS_PTRACE is a capability that is "not granted by default". Perhaps I don't know what to look for.
I'm currently trying to track down some phantom I/O in a PostgreSQL build I'm testing. It's a multi-process server and it isn't simple to associate disk I/O back to a particular back-end and query.
I thought Linux's perf tool would be ideal for this, but I'm struggling to capture block I/O performance counter metrics and associate them with user-space activity.
It's easy to record block I/O requests and completions with, eg:
sudo perf record -g -T -u postgres -e 'block:block_rq_*'
and the user-space pid is recorded, but there's no kernel or user-space stack captured, or ability to snapshot bits of the user-space process's heap (say, query text) etc. So while you have the pid, you don't know what the process was doing at that point. Just perf script output like:
postgres 7462 [002] 301125.113632: block:block_rq_issue: 8,0 W 0 () 208078848 + 1024 [postgres]
If I add the -g flag to perf record it'll take snapshots of the kernel stack, but doesn't capture user-space state for perf events captured in the kernel. The user-space stack only goes up to the entry-point from userspace, like LWLockRelease, LWLockAcquire, memcpy (mmap'd IO), __GI___libc_write, etc.
So. Any tips? Being able to capture a snapshot of the user-space stack in response to kernel events would be ideal.
I'm on Fedora 19, 3.11.3-201.fc19.x86_64, Schrödinger’s Cat, with perf version 3.10.9-200.fc19.x86_64.
OK, looks like there are several parts to this:
I'm on x86_64, where most distros build with -fomit-frame-pointer by default, and perf can't follow the stack without frame pointers;
.... unless it's a newer version built with libunwind support, in which case it supports perf record -g dwarf.
See:
the patch adding libunwind support to Perf
Debian bug 725075.
linux perf: how to interpret and find hotspots
I'm on Fedora 18, but the same issue applies. So if you're profiling code you're working on (as is likely on Stack Overflow), rebuild with -fno-omit-frame-pointer and -ggdb.
I landed up rebuilding perf because I wanted to be able to compare to the stock RPMs:
sudo yum build-dep perf
sudo yum install yum-utils rpmdevtools libunwind-devel
yumdownloader --source perf or download the appropriate kernel-.....src.rpm srpm
rpmdev-setuptree
rpm -Uvh kernel-*.src.rpm
cd $HOME/rpmbuild/SPECS
rpmbuild -bp --target=$(uname -m) kernel.spec
At this point you can just build a new perf if you want:
cd $HOME/rpmbuild/BUILD/kernel-*/linux-*/tools/perf
make
... which I did and tested that the updated perf does in fact capture a useful stack if built with libunwind available.
You can also build a new rpm:
edit kernel.spec, uncomment the line %define buildid ..., change buildid to something like .perfunwind. Note it's %define not % define.
In the same spec file, find:
%global perf_make \
make %{?_smp_mflags} -C tools/perf -s V=1 WERROR=0 NO_LIBUNWIND=1 HAVE_CPLUS_DEMANGLE=1 NO_GTK2=1 NO_LIBNUMA=1 NO_STRLCPY=1 prefix=%{_prefix}
and delete NO_LIBUNWIND=1
rpmbuild -bb --without up --without mp --without pae --without debug --without doc --without headers --without debuginfo --without bootwrapper --without with_vdso_install --with perf kernel.spec to produce new perf RPMs without building the whole kernel. Or if you want, omit the --without for the kernel flavour you want, in which case you'll also want to build headers, debuginfo, etc.
sudo rpm -Uvh $HOME/rpmbuild/RPMS/x86_64/perf-*.fc19.x86_64.rpm
See the fedora project guide on building a custom kernel.
I've reported the issue to Fedora; they shouldn't be using NO_LIBUNWIND=1. See bug 1025603.
Once you have a rebuilt perf you can use perf record -g dwarf to get full stacks.
I'm trying to attach a program with gdb but it returns:
Attaching to process 29139
Could not attach to process. If your uid matches the uid of the target
process, check the setting of /proc/sys/kernel/yama/ptrace_scope, or try
again as the root user. For more details, see /etc/sysctl.d/10-ptrace.conf
ptrace: Operation not permitted.
gdb-debugger returns "Failed to attach to process, please check privileges and try again."
strace returns "attach: ptrace(PTRACE_ATTACH, ...): Operation not permitted"
I changed "kernel.yama.ptrace_scope" 1 to 0 and /proc/sys/kernel/yama/ptrace_scope 1 to 0 and tried set environment LD_PRELOAD=./ptrace.so with this:
#include <stdio.h>
int ptrace(int i, int j, int k, int l) {
printf(" ptrace(%i, %i, %i, %i), returning -1\n", i, j, k, l);
return 0;
}
But it still returns the same error. How can I attach it to debuggers?
If you are using Docker, you will probably need these options:
docker run --cap-add=SYS_PTRACE --security-opt seccomp=unconfined
If you are using Podman, you will probably need its --cap-add option too:
podman run --cap-add=SYS_PTRACE
This is due to kernel hardening in Linux; you can disable this behavior by echo 0 > /proc/sys/kernel/yama/ptrace_scope or by modifying it in /etc/sysctl.d/10-ptrace.conf
See also this article about it in Fedora 22 (with links to the documentation) and this comment thread about Ubuntu and .
I would like to add that I needed --security-opt apparmor=unconfined along with the options that #wisbucky mentioned. This was on Ubuntu 18.04 (both Docker client and host). Therefore, the full invocation for enabling gdb debugging within a container is:
docker run --cap-add=SYS_PTRACE --security-opt seccomp=unconfined --security-opt apparmor=unconfined
Just want to emphasize a related answer. Let's say that you're root and you've done:
strace -p 700
and get:
strace: attach: ptrace(PTRACE_SEIZE, 700): Operation not permitted
Check:
grep TracerPid /proc/700/status
If you see something like TracerPid: 12, i.e. not 0, that's the PID of the program that is already using the ptrace system call. Both gdb and strace use it, and there can only be one active at a time.
Not really addressing the above use-case but I had this problem:
Problem: It happened that I started my program with sudo, so when launching gdb it was giving me ptrace: Operation not permitted.
Solution: sudo gdb ...
As most of us land here for Docker issues I'll add the Kubernetes answer as it might come in handy for someone...
You must add the SYS_PTRACE capability in your pod's security context
at spec.containers.securityContext:
securityContext:
capabilities:
add: [ "SYS_PTRACE" ]
There are 2 securityContext keys at 2 different places. If it tells you that the key is not recognized than you missplaced it. Try the other one.
You probably need to have a root user too as default. So in the other security context (spec.securityContext) add :
securityContext:
runAsUser: 0
runAsGroup: 0
fsGroup: 101
FYI : 0 is root. But the fsGroup value is unknown to me. For what I'm doing I don't care but you might.
Now you can do :
strace -s 100000 -e write=1 -e trace=write -p 16
You won't get the permission denied anymore !
BEWARE : This is the Pandora box. Having this in production it NOT recommended.
I was running my code with higher privileges to deal with Ethernet Raw Sockets by setting set capability command in Debian Distribution. I tried the above solution: echo 0 > /proc/sys/kernel/yama/ptrace_scope
or by modifying it in /etc/sysctl.d/10-ptrace.conf but that did not work for me.
Additionally, I also tried with set capabilities command for gdb in installed directory (usr/bin/gdb) and it works: /sbin/setcap CAP_SYS_PTRACE=+eip /usr/bin/gdb.
Be sure to run this command with root privileges.
Jesup's answer is correct; it is due to Linux kernel hardening. In my case, I am using Docker Community for Mac, and in order to do change the flag I must enter the LinuxKit shell using justin cormack's nsenter (ref: https://www.bretfisher.com/docker-for-mac-commands-for-getting-into-local-docker-vm/ ).
docker run -it --rm --privileged --pid=host justincormack/nsenter1
/ # cat /etc/issue
Welcome to LinuxKit
## .
## ## ## ==
## ## ## ## ## ===
/"""""""""""""""""\___/ ===
{ / ===-
\______ O __/
\ \ __/
\____\_______/
/ # cat /proc/sys/kernel/yama/ptrace_scope
1
/ # echo 0 > /proc/sys/kernel/yama/ptrace_scope
/ # exit
Maybe someone has attached this process with gdb.
ps -ef | grep gdb
can't gdb attach the same process twice.
I was going to answer this old question as it is unaccepted and any other answers are not got the point. The real answer may be already written in /etc/sysctl.d/10-ptrace.conf as it is my case under Ubuntu. This file says:
For applications launching crash handlers that need PTRACE, exceptions can
be registered by the debugee by declaring in the segfault handler
specifically which process will be using PTRACE on the debugee:
prctl(PR_SET_PTRACER, debugger_pid, 0, 0, 0);
So just do the same thing as above: keep /proc/sys/kernel/yama/ptrace_scope as 1 and add prctl(PR_SET_PTRACER, debugger_pid, 0, 0, 0); in the debugee. Then the debugee will allow debugger to debug it. This works without sudo and without reboot.
Usually, debugee also need to call waitpid to avoid exit after crash so debugger can find the pid of debugee.
If permissions are a problem, you probably will want to use gdbserver. (I almost always use gdbserver when I gdb, docker or no, for numerous reasons.) You will need gdbserver (Deb) or gdb-gdbserver (RH) installed in the docker image. Run the program in docker with
$ sudo gdbserver :34567 myprogram arguments
(pick a port number, 1025-65535). Then, in gdb on the host, say
(gdb) target remote 172.17.0.4:34567
where 172.17.0.4 is the IP address of the docker image as reported by /sbin/ip addr list run in the docker image. This will attach at a point before main runs. You can tb main and c to stop at main, or wherever you like. Run gdb under cgdb, emacs, vim, or even in some IDE, or plain. You can run gdb in your source or build tree, so it knows where everything is. (If it can't find your sources, use the dir command.) This is usually much better than running it in the docker image.
gdbserver relies on ptrace, so you will also need to do the other things suggested above. --privileged --pid=host sufficed for me.
If you deploy to other OSes or embedded targets, you can run gdbserver or a gdb stub there, and run gdb the same way, connecting across a real network or even via a serial port (/dev/ttyS0).
I don't know what you are doing with LD_PRELOAD or your ptrace function.
Why don't you try attaching gdb to a very simple program? Make a program that simply repeatedly prints Hello or something and use gdb --pid [hello program PID] to attach to it.
If that does not work then you really do have a problem.
Another issue is the user ID. Is the program that you are tracing setting itself to another UID? If it is then you cannot ptrace it unless you are using the same user ID or are root.
I have faced the same problem and try a lot of solution but finally, I have found the solution, but really I don't know what the problem was. First I modified the ptrace_conf value and login into Ubuntu as a root but the problem still appears. But the most strange thing that happened is the gdb showed me a message that says:
Could not attach to process. If your uid matches the uid of the target process, check the setting of /proc/sys/kernel/yama/ptrace_scope, or try again as the root user.
For more details, see /etc/sysctl.d/10-ptrace.conf
warning: process 3767 is already traced by process 3755 ptrace: Operation not permitted.
With ps command terminal, the process 3755 was not listed.
I found the process 3755 in /proc/$pid but I don't understand what was it!!
Finally, I deleted the target file (foo.c) that I try to attach it vid gdb and tracer c program using PTRACE_ATTACH syscall, and in the other folder, I created another c program and compiled it.
the problem is solved and I was enabled to attach to another process either by gdb or ptrace_attach syscall.
(gdb) attach 4416
Attaching to process 4416
and I send a lot of signals to process 4416. I tested it with both gdb and ptrace, both of them run correctly.
really I don't know the problem what was, but I think it is not a bug in Ubuntu as a lot of sites have referred to it, such https://askubuntu.com/questions/143561/why-wont-strace-gdb-attach-to-a-process-even-though-im-root
Extra information
If you wanna make changes in the interfaces such as add the ovs bridge, you must use --privileged instead of --cap-add NET_ADMIN.
sudo docker run -itd --name=testliz --privileged --cap-add=SYS_PTRACE --security-opt seccomp=unconfined ubuntu
If you are using FreeBSD, edit /etc/sysctl.conf, change the line
security.bsd.unprivileged_proc_debug=0
to
security.bsd.unprivileged_proc_debug=1
Then reboot.