I've been trying to get my Laravel Forge Ubuntu server to connect to a remote MSSQL server. I've finally got the server set up to where it can reach the remote database and make queries through terminal. However, when I try and use the connection within Laravel I'm getting a "502 Bad Gateway". I've done quite a bit of searching at this point and I'm still none the wiser on how to debug this.
Any help would be greatly appreciated! I've included my nginx and php-fpm logs edited for security.
var/log/nginx/XXX.co-error.log
2017/06/12 13:51:40 [error] 5682#5682: *84 recv() failed (104:
Connection reset by peer) while reading response header from upstream,
client: 000.000.000.000, server: XXX.co, request: "GET
/tools/labels/refreshsalesorderitems HTTP/1.1", upstream:
"fastcgi://unix:/var/run/php/php7.1-fpm.sock:", host: "XXX.co",
referrer: "http://XXX.co/tools/labels"
var/log/php7.1-fpm.log
[12-Jun-2017 13:51:40] WARNING: [pool www] child 8538 exited on signal 11 (SIGSEGV) after 3135.874644
seconds from start
[12-Jun-2017 13:51:40] NOTICE: [pool www] child 9087 started
core dump backtrace
/etc/php/7.1/fpm: Success.
[New LWP 22567]
Core was generated by `php-fpm: pool www '.
Program terminated with signal SIGSEGV, Segmentation fault.
#0 0x00007fdb7d6bfbe9 in ?? ()
(gdb) backtrace
#0 0x00007fdb7d6bfbe9 in ?? ()
#1 0x372e350a0000005b in ?? ()
#2 0x3921ba89f34d7f00 in ?? ()
#3 0x0004000400000006 in ?? ()
#4 0x000055d8b19043a0 in ?? ()
#5 0x000055d8b1904810 in ?? ()
#6 0x0000000000000000 in ?? ()
(gdb)
Related
I'm having some issues with enabling the res_srtp module in Asterisk. Every attempt results in this not-so-helpful error message:
WARNING[47044] res_srtp.c: Failed to initialize libsrtp
ERROR[47044] loader.c: *** Failed to load module res_srtp.so
ERROR[47044] asterisk.c: Module initialization failed. ASTERISK EXITING!
I'd like to fix this and get it running, so now we get into the programming side of things. I'm not a C programmer by trade, but I was able to use my limited gdb skills to trace the error back through libsrtp's crypto_kernel_init() function. The error occurs when trying to enable the AES-GCM-128 cipher.
I was only able to get as far as PK11_Encrypt() in the NSS library, where I was unable to step into this line of code. I think this is because (again, very limited C knowledge) it's a macro and not a true function?
crv = PK11_GETTAB(slot)->C_EncryptInit(session, &mech, symKey->objectID);
// returns 113
So my question is either, how can I proceed with debugging this to a point where I can file a bug report with someone, or (preferably) has anyone got libsrtp working in this environment? There were very few other reports of similar problems, likely because EL9 is not in wide use yet.
My distro (AlmaLinux) is running NSS 3.71, and I've tried updating to NSS 3.79 with no change.
Here's the backtrace from where I was able to get to, if it's of any help.
#0 PK11_Encrypt (symKey=0x555555ea1a00, mechanism=mechanism#entry=4231, param=param#entry=0x7fffffffc9a0,
out=out#entry=0x7fffffffcb50 "\331\061\062%\370\204\006\345\245Y\tů\365&\232\206\247\251S\025\064\367\332.L0=\212\061\212r\034<\f\225\225h\tS/\317\016$I\246\265%\261j\355\365\252\r\346W\272c{9", outLen=outLen#entry=0x7fffffffca3c, maxLen=<optimized out>,
data=0x7fffffffcb50 "\331\061\062%\370\204\006\345\245Y\tů\365&\232\206\247\251S\025\064\367\332.L0=\212\061\212r\034<\f\225\225h\tS/\317\016$I\246\265%\261j\355\365\252\r\346W\272c{9", dataLen=60) at ../pk11wrap/pk11obj.c:972
#1 0x00007ffff456a455 in srtp_aes_gcm_nss_do_crypto (enc_len=0x7fffffffca3c,
buf=0x7fffffffcb50 "\331\061\062%\370\204\006\345\245Y\tů\365&\232\206\247\251S\025\064\367\332.L0=\212\061\212r\034<\f\225\225h\tS/\317\016$I\246\265%\261j\355\365\252\r\346W\272c{9", encrypt=1, cv=0x5555562a19c0) at crypto/cipher/aes_gcm_nss.c:297
#2 srtp_aes_gcm_nss_encrypt (cv=0x5555562a19c0, buf=<optimized out>, enc_len=0x7fffffffca3c) at crypto/cipher/aes_gcm_nss.c:345
#3 0x00007ffff456cb24 in srtp_cipher_type_test (ct=0x7ffff457a6c0 <srtp_aes_gcm_128>, test_data=0x7ffff457a420 <srtp_aes_gcm_test_case_0>)
at crypto/cipher/cipher.c:297
#4 0x00007ffff456d545 in srtp_cipher_type_test (ct=<optimized out>, test_data=<optimized out>) at crypto/cipher/cipher.c:605
#5 0x00007ffff456d58d in srtp_cipher_type_self_test (ct=<optimized out>) at crypto/cipher/cipher.c:613
#6 0x00007ffff457005d in srtp_crypto_kernel_do_load_cipher_type (replace=0, id=6, new_ct=0x7ffff457a6c0 <srtp_aes_gcm_128>)
at crypto/kernel/crypto_kernel.c:293
#7 srtp_crypto_kernel_load_cipher_type (new_ct=new_ct#entry=0x7ffff457a6c0 <srtp_aes_gcm_128>, id=id#entry=6) at crypto/kernel/crypto_kernel.c:343
#8 0x00007ffff457036e in srtp_crypto_kernel_init () at crypto/kernel/crypto_kernel.c:139
#9 srtp_crypto_kernel_init () at crypto/kernel/crypto_kernel.c:72
#10 0x00007ffff457040d in srtp_init () at srtp/srtp.c:2729
#11 0x00007ffff458718d in res_srtp_init () at /usr/src/debug/asterisk-16.28.0-0.el9.x86_64/res/res_srtp.c:1237
#12 load_module () at /usr/src/debug/asterisk-16.28.0-0.el9.x86_64/res/res_srtp.c:1272
#13 0x000055555566c4dc in start_resource.part.0.lto_priv.0 (mod=0x555555a469d0) at /usr/src/debug/asterisk-16.28.0-0.el9.x86_64/main/loader.c:1718
#14 0x0000555555665517 in start_resource (mod=0x555555a469d0) at /usr/src/debug/asterisk-16.28.0-0.el9.x86_64/main/loader.c:1692
#15 start_resource_attempt (mod=mod#entry=0x555555a469d0, count=count#entry=0x7fffffffce94)
at /usr/src/debug/asterisk-16.28.0-0.el9.x86_64/main/loader.c:1894
#16 0x000055555566862f in start_resource_list (mod_count=0x7fffffffce94, resources=0x7fffffffceb0)
at /usr/src/debug/asterisk-16.28.0-0.el9.x86_64/main/loader.c:1991
#17 load_resource_list (mod_count=<synthetic pointer>, load_order=0x7fffffffcea0) at /usr/src/debug/asterisk-16.28.0-0.el9.x86_64/main/loader.c:2173
#18 load_modules () at /usr/src/debug/asterisk-16.28.0-0.el9.x86_64/main/loader.c:2396
#19 0x000055555559e074 in asterisk_daemon (isroot=<optimized out>, rungroup=<optimized out>, runuser=<optimized out>)
at /usr/src/debug/asterisk-16.28.0-0.el9.x86_64/main/asterisk.c:4258
#20 main (argc=<optimized out>, argv=<optimized out>) at /usr/src/debug/asterisk-16.28.0-0.el9.x86_64/main/asterisk.c:4025
After going through the debug steps a few more times, I tried searching the initial return value from srtp_kernel_init() which was srtp_err_status_cipher_fail; I ended up finding a resolved issue with the same symptoms for Fedora Linux:
The issue turns out to be an incompatibility with nss-3.63 shipped with F34. The attached patch adds NSS_PKCS11_2_0_COMPAT defines to 2 header files to enable the backward compatibility. The same fix is already in the upstream github repo and is targeted for the next libsrtp release.
I was able to apply their patch to the libsrtp 2.3.0 codebase, and the library is successfully loading now:
index 4d6031f..b1da343 100644
--- a/crypto/include/aes_gcm.h
+++ b/crypto/include/aes_gcm.h
## -66,6 +66,8 ## typedef struct {
#ifdef NSS
+#define NSS_PKCS11_2_0_COMPAT 1
+
#include <nss.h>
#include <pk11pub.h>
index ad306dd..a57564f 100644
--- a/crypto/include/aes_icm_ext.h
+++ b/crypto/include/aes_icm_ext.h
## -65,6 +65,8 ## typedef struct {
#ifdef NSS
+#define NSS_PKCS11_2_0_COMPAT 1
+
#include <nss.h>
#include <pk11pub.h>
I'm not sure the second hunk is needed, as ICM ciphers were loading fine, but I applied it anyway.
When I try to debug core file with gdb I dont see any valid stack trace to proceed further (similar to the reported issue in the link below). Can you please help me how I should debug further to debug the issue. Any pointers or gdb commands which will help in triaging the problem.
GDB debugging trace with no relevant info (#0 0x2e6e6f69 in ?? ())
(gdb) where
#0 0x76c0da28 in ?? ()
#1 0x76c0d9e0 in ?? ()
#2 0x76c0d9e0 in ?? ()
Backtrace stopped: previous frame identical to this frame (corrupt stack?)
(gdb)
EDIT: To rule out the possibility of the -g flag and host environment issues,I have intentionally added a code to crash and was able to get the correct stack trace from the corefile.
When I try to debug core file with gdb I dont see any valid stack trace
Was the core produced on the same host where it is being analyzed?
If not, this answer explains what you need to do.
I have a segmentation fault in my program, and I'm using gdb to identify where it's happening. However, I am not able to see a clear line number where the error is occurring.
Below is a screenshot of my output.
Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 20065168 (LWP 4645)]
0x007e537f in _int_free () from /lib/libc.so.6
(gdb) backtrace
#0 0x007e537f in _int_free () from /lib/libc.so.6
#1 0x007e90f0 in free () from /lib/libc.so.6
#2 0x080d9e67 in CRYPTO_free ()
#3 0xbfd15f7c in ?? ()
#4 0xbfd16108 in ?? ()
#5 0x08070b3e in function_random.19532 ()
#6 0x00000001 in ?? ()
#7 0x00000000 in ?? ()
(gdb)
frame 5 is the piece of code that I have written, but I don't quite understand what it means.
Can someone please explain?
Most likely, in your case, debug symbols are not present in the binary. That is why, gdb is not able to read the debugging info and display them.
Re-compile your code, with the debugging enabled.
Example: for gcc, use the -g options.
I have a small C program that loads a custom DLL and uses a couple of functions. I can run the program from the console and it works as intended. (I'm compiling with MinGW on Windows XP)
But if I run it from gdb, when it gets to loading the DLL, I get:
56 ldll = LoadLibrary("gsp810.dll");
(gdb) n
Program received signal SIGSEGV, Segmentation fault.
0x7c929af2 in ntdll!RtlpWaitForCriticalSection () from C:\WINDOWS\system32\ntdll.dll
The weird thing is, if I make a backtrace at this point, I get a strange stack of Windows functions, which doesn't even contain my own program's stack (see below). However, if I keep running, it'll eventually return to my main() function and everything seems to be back to normal. The program works as expected and the functions from the DLL can be called.
(gdb) backtrace
#0 0x7c929af2 in ntdll!RtlpWaitForCriticalSection () from C:\WINDOWS\system32\ntdll.dll
#1 0x7c911046 in ntdll!RtlEnterCriticalSection () from C:\WINDOWS\system32\ntdll.dll
#2 0x00e161a0 in ?? ()
#3 0x77da6cf8 in RegCloseKey () from C:\WINDOWS\system32\advapi32.dll
#4 0x77da78e4 in RegOpenKeyExA () from C:\WINDOWS\system32\advapi32.dll
#5 0x77f44fcd in SHLWAPI!PathMakeSystemFolderW () from C:\WINDOWS\system32\shlwapi.dll
#6 0x77f452e8 in SHLWAPI!PathMakeSystemFolderW () from C:\WINDOWS\system32\shlwapi.dll
#7 0x77f45252 in SHLWAPI!PathMakeSystemFolderW () from C:\WINDOWS\system32\shlwapi.dll
#8 0x7c91118a in ntdll!LdrInitializeThunk () from C:\WINDOWS\system32\ntdll.dll
#9 0x77f40000 in ?? ()
#10 0x7c92b5d2 in ntdll!LdrFindResourceDirectory_U () from C:\WINDOWS\system32\ntdll.dll
#11 0x7c9262db in ntdll!RtlValidateUnicodeString () from C:\WINDOWS\system32\ntdll.dll
#12 0x7c92643d in ntdll!LdrLoadDll () from C:\WINDOWS\system32\ntdll.dll
#13 0x00000000 in ?? ()
Is this SEGFAULT normal, or it is indicating an underlying problem with the DLL?
EDIT: Ok, looks like the problem is in the DLL itself. What I don't understand is the backtrace gdb is showing, as it does not contain the functions in my application. Then, at a certain point, it somehow "switches" to my stack, and the program keeps running as if nothing had happened.
Is it possible that Windows is somehow "handling" the segmentation fault, and the it returns control to the application?
I'm using gdb to attach a running process, however, it always stops at __kernel_vsyscall. It looks like it stopped at my system call msgrcv(). I have to constantly "cont" it and don't know when it could jump out of kernel and go back to application. How can I make it continue? The following is my procedure.
How did I get this situation?
How to make it continue?
Thanks!
gdb
(gdb) attach PID
...
Reading symbols from /lib/ld-linux.so.2...(no debugging symbols found)...done.
Loaded symbols for /lib/ld-linux.so.2
0xffffe410 in __kernel_vsyscall ()
(gdb)bt
#0 0xffffe410 in __kernel_vsyscall ()
#1 0x009ed573 in msgrcv () from /lib/libc.so.6
#2 0xf7f3a487 in _UX_wgetmsg (mode=0, msgp=0xffbb4178, pmaxtime=0xffbb4164,
pdata=0xf7f7a860, ux_type=0) at ../../../ux/com_ux/libux/com/UXipc.c:2550
#3 0xf7f3ad05 in UX_wgetmsg_v2 (mode=0, msgp=0xffbb4178, maxtime=10000,
ux_type=0) at ../../../ux/com_ux/libux/com/UXipc.c:2237
#4 0x0804bb9b in main (argc=1, argv=0xffbb5394)
at /path/to/my_application:243
How did I get this situation?
That situation is completely normal for when you attach to a process which is blocked in a system call (waiting for message, or for read to complete).
How to make it continue?
You type continue (at which point the application would again block, waiting for a message). If you want to debug some part of the application, set breakpoints before continuing.