How to run SSL with mongoose webserver API - c

I use the Mongoose webserver API in C to implement a small HTTP API. It's working fine so far but if I want to use SSL, the server rejects the requests.
To generate the ssl key and certificate, I used the following approach:
# openssl version
OpenSSL 1.0.0-beta2 21 Apr 2009
# openssl req -x509 -newkey rsa:2048 -keyout key.pem -out cert.pem -days 1000 -nodes
# cat key.pem > ssl.pem; cat cert.pem >> ssl.pem
According to this manual, mongoose wants the key and cert in one file.
I also set the NS_ENABLE_SSL compile flag to enable the ssl part of mongoose.
To initialize mongoose I do the following in my C source:
struct mg_server *server;
server = mg_create_server(NULL, ev_handler);
printf("%s",mg_set_option(server, "document_root", "/var/www"));
printf("%s",mg_set_option(server, "listening_port", "80"));
printf("%s",mg_set_option(server, "ssl_certificate", "ssl.pem"));
My event handler does nothing but logging the requests on the console at the moment:
static int ev_handler(struct mg_connection *conn, enum mg_event ev) {
switch(ev){
case MG_AUTH: printf("MG_AUTH event handler called with uri = '%s'\n", conn->uri);
break;
case MG_REQUEST:
printf("MG_REQUEST event handler called with uri = '%s'\n", conn->uri);
break;
case MG_POLL:
printf("MG_POLL event handler called with uri = '%s'\n", conn->uri);
break;
case MG_HTTP_ERROR:
printf("MG_HTTP_ERROR event handler called with uri = '%s'\n", conn->uri);
break;
default:
DBG("MG_CLOSE event handler called with uri = '%s'\n", conn->uri);
break;
}
return MG_FALSE;
}
If I try to open a HTML file from the document root in the browser, the server closes the connection without delivering content.
Console output:
(null)
(null)
(null)
MG_POLL event handler called with uri = '(null)'
MG_CLOSE event handler called with uri = '(null)'
The first three (null) are the return values from the mg_set_option which are NULL if no problem occurs while setting an option.
What's going wrong here?

Make sure you're using "https://my.ip:80" when connecting to the server. Otherwise, browser is going to use HTTP protocol, and mongoose will reject the connection because it expects SSL handshake, and gets plain HTTP data. I think that is what's going on.
You're returning MG_FALSE on MG_AUTH event, which tells mongoose "connection is not authenticated, close it immediately."
Solution:
return MG_TRUE on MG_AUTH event. For other events, returning MG_FALSE is fine.
use listening_port 443, and https:// when connecting to the server

Related

How to obtain the key agreement algorithm used in TLS 1.3 handshake using OpenSSL API?

I was modifying the example client code given on the OpenSSL s_client page to understand a few functions. I added the following lines after the chain verification part :
const SSL_CIPHER* curr_cipher = SSL_get_current_cipher(ssl);
printf("SSL_CIPHER_standard_name: %s\n", SSL_CIPHER_standard_name(curr_cipher));
printf("SSL_get_cipher_version: %s\n", SSL_get_cipher_version(ssl));
int kx_nid = SSL_CIPHER_get_kx_nid(curr_cipher);
printf("SSL_CIPHER_get_kx_nid: %d\n", kx_nid);
printf("OBJ_nid2ln(%d): %s\n", kx_nid, OBJ_nid2ln(kx_nid));
printf("SSL_CIPHER_description: %s\n", SSL_CIPHER_description(curr_cipher, NULL, 0));
Instead of getting the key exchange algorithm used, I got the following output after connecting to random.org using TLS 1.3 :
SSL_CIPHER_standard_name: TLS_AES_256_GCM_SHA384
SSL_get_cipher_version: TLSv1.3
SSL_CIPHER_get_kx_nid: 1063
OBJ_nid2ln(1063): kx-any
SSL_CIPHER_description: TLS_AES_256_GCM_SHA384 TLSv1.3 Kx=any Au=any Enc=AESGCM(256) Mac=AEAD
The man page for SSL_CIPHER_get_kx_nid says it returns NID_kx_any for TLS 1.3. How can I programatically get the actual key agreement (NID_kx_ecdhe) and authentication protocol (NID_auth_rsa) used in the handshake from the SSL object?

com.ctc.wstx.exc.WstxParsingException: Unexpected close tag </span>; expected </br>

I created the stub classes using CXF wsdl2java tool.
I am using Apache CXF library, with JCIFS. I validated the WSDL file itself through couple tools, it is good. Here is the code. It looks like some setting I must do.
//JCIFS Authentication related code
jcifs.Config.setProperty("jcifs.smb.client.domain", "NTS");
jcifs.Config.setProperty("jcifs.netbios.wins", "ecmchat.mark.gov");
jcifs.Config.setProperty("jcifs.smb.client.soTimeout", "300000"); // 5 minutes
jcifs.Config.setProperty("jcifs.netbios.cachePolicy", "1200"); // 20 minutes
jcifs.Config.setProperty("jcifs.smb.client.username", "user");
jcifs.Config.setProperty("jcifs.smb.client.password", "password");
//Register the jcifs URL handler to enable NTLM
jcifs.Config.registerSmbURLHandler();
//WSDL and Client settings
URL wsdlURL = BF.WSDL_LOCATION;
if (args.length > 0 && args[0] != null && !"".equals(args[0])) {
File wsdlFile = new File(args[0]);
try {
if (wsdlFile.exists()) {
wsdlURL = wsdlFile.toURI().toURL();
} else {
wsdlURL = new URL(args[0]);
}
} catch (MalformedURLException e) {
e.printStackTrace();
}
}
BF ss = new BF(wsdlURL, SERVICE_NAME);
BFSoap port = ss.getBFSoap12();
Client client = ClientProxy.getClient(port);
HTTPConduit http = (HTTPConduit) client.getConduit();
HTTPClientPolicy httpClientPolicy = new HTTPClientPolicy();
httpClientPolicy.setConnectionTimeout(36000);
httpClientPolicy.setAllowChunking(false);
httpClientPolicy.setReceiveTimeout(32000);
http.setClient(httpClientPolicy);
// Calling the method
System.out.println("Invoking testMethod...");
String _testMethod__return = port.testMethod();
System.out.println("testMethod.result=" + _testMethod__return);
I am getting the following exception
Caused by: com.ctc.wstx.exc.WstxParsingException: Unexpected close tag </span>; expected </br>.
at [row,col,system-id]: [59,22,"https://ecmchat.mark.gov/BF/BF.asmx"]
at com.ctc.wstx.sr.StreamScanner.constructWfcException(StreamScanner.java:621)
at com.ctc.wstx.sr.StreamScanner.throwParseError(StreamScanner.java:491)
at com.ctc.wstx.sr.StreamScanner.throwParseError(StreamScanner.java:475)
at com.ctc.wstx.sr.BasicStreamReader.reportWrongEndElem(BasicStreamReader.java:3365)
at com.ctc.wstx.sr.BasicStreamReader.readEndElem(BasicStreamReader.java:3292)
at com.ctc.wstx.sr.BasicStreamReader.nextFromTree(BasicStreamReader.java:2911)
at com.ctc.wstx.sr.BasicStreamReader.next(BasicStreamReader.java:1123)
at org.apache.cxf.staxutils.StaxUtils.readDocElements(StaxUtils.java:1361)
at org.apache.cxf.staxutils.StaxUtils.readDocElements(StaxUtils.java:1255)
at org.apache.cxf.staxutils.StaxUtils.read(StaxUtils.java:1183)
at org.apache.cxf.wsdl11.WSDLManagerImpl.loadDefinition(WSDLManagerImpl.java:235)
... 9 more
If I comment out the JCIFS NTLM authentication code, I get a HTTP 401 error. Therefore, I believe, at least it is passing some kind of authorization step.
And, if I use local WSDL in place of remote URL WSDL, then I get a different error like "method not implemented" on the call to the method. May be this is due to me not using the local WSDL correctly. I do not even know if we can use the local WSDL reference for remote service.
Then, I created a SoapUI dummy service with this WSDL, and the same code (but without the JCIFS authentication code) works good, and successfully calls the methods.
It appears to me that I must add some more appropriate settings in the configuration related code.
Am I right, and are you aware of any, for NTLM authentication and Apache CXF?
But parsing error is confusing???
I do not know if this is related.
My original WSDL URL that I gave was this.
https://ecmchat.mark.gov/BF/BF.asmx
I added a ?wsdl like below
https://ecmchat.mark.gov/BF/BF.asmx?wsdl
Then I am getting a different error.
I wonder why it is working if I access my local SoapUI version of the same WSDL service, but not for the remote one.
Invoking testMethod...
Jan 07, 2020 10:47:25 AM org.apache.cxf.phase.PhaseInterceptorChain doDefaultLogging
WARNING: Interceptor for {https://ecmchat.mark.gov}BF#{https://ecmchat.mark.gov}testMethod has thrown exception, unwinding now
java.lang.UnsupportedOperationException: Method not implemented.
at java.net.URLStreamHandler.openConnection(URLStreamHandler.java:96)
at java.net.URL.openConnection(URL.java:1028)
at org.apache.cxf.transport.https.HttpsURLConnectionFactory.createConnection(HttpsURLConnectionFactory.java:92)
at org.apache.cxf.transport.http.URLConnectionHTTPConduit.createConnection(URLConnectionHTTPConduit.java:121)
at org.apache.cxf.transport.http.URLConnectionHTTPConduit.setupConnection(URLConnectionHTTPConduit.java:125)
at org.apache.cxf.transport.http.HTTPConduit.prepare(HTTPConduit.java:505)
at org.apache.cxf.interceptor.MessageSenderInterceptor.handleMessage(MessageSenderInterceptor.java:47)
at org.apache.cxf.phase.PhaseInterceptorChain.doIntercept(PhaseInterceptorChain.java:308)
at org.apache.cxf.endpoint.ClientImpl.doInvoke(ClientImpl.java:530)
at org.apache.cxf.endpoint.ClientImpl.invoke(ClientImpl.java:441)
at org.apache.cxf.endpoint.ClientImpl.invoke(ClientImpl.java:356)
at org.apache.cxf.endpoint.ClientImpl.invoke(ClientImpl.java:314)
at org.apache.cxf.frontend.ClientProxy.invokeSync(ClientProxy.java:96)
at org.apache.cxf.jaxws.JaxWsClientProxy.invoke(JaxWsClientProxy.java:140)
at com.sun.proxy.$Proxy33.testMethod(Unknown Source)
at edison.learn.BFSoap_BFSoap12_Client.main(BFSoap_BFSoap12_Client.java:90)
Exception in thread "main" javax.xml.ws.soap.SOAPFaultException: Method not implemented.
at org.apache.cxf.jaxws.JaxWsClientProxy.mapException(JaxWsClientProxy.java:195)
at org.apache.cxf.jaxws.JaxWsClientProxy.invoke(JaxWsClientProxy.java:145)
at com.sun.proxy.$Proxy33.testMethod(Unknown Source)
at edison.learn.BFSoap_BFSoap12_Client.main(BFSoap_BFSoap12_Client.java:90)
Caused by: java.lang.UnsupportedOperationException: Method not implemented.
at java.net.URLStreamHandler.openConnection(URLStreamHandler.java:96)
at java.net.URL.openConnection(URL.java:1028)
at org.apache.cxf.transport.https.HttpsURLConnectionFactory.createConnection(HttpsURLConnectionFactory.java:92)
at org.apache.cxf.transport.http.URLConnectionHTTPConduit.createConnection(URLConnectionHTTPConduit.java:121)
at org.apache.cxf.transport.http.URLConnectionHTTPConduit.setupConnection(URLConnectionHTTPConduit.java:125)
at org.apache.cxf.transport.http.HTTPConduit.prepare(HTTPConduit.java:505)
at org.apache.cxf.interceptor.MessageSenderInterceptor.handleMessage(MessageSenderInterceptor.java:47)
at org.apache.cxf.phase.PhaseInterceptorChain.doIntercept(PhaseInterceptorChain.java:308)
at org.apache.cxf.endpoint.ClientImpl.doInvoke(ClientImpl.java:530)
at org.apache.cxf.endpoint.ClientImpl.invoke(ClientImpl.java:441)
at org.apache.cxf.endpoint.ClientImpl.invoke(ClientImpl.java:356)
at org.apache.cxf.endpoint.ClientImpl.invoke(ClientImpl.java:314)
at org.apache.cxf.frontend.ClientProxy.invokeSync(ClientProxy.java:96)
at org.apache.cxf.jaxws.JaxWsClientProxy.invoke(JaxWsClientProxy.java:140)
... 2 more

SSL Cipher help in C

I am trying to Use SSL on top of tcp/ip to send an HTTPS request to a site using C. I have no access to curl or other standard libraries. Pretend like i can't load any libraries at all.
I need to set an SSL Profile Cipher. When I successfully use curl on my linux box to talk with the server I see: SSL Connection using ECDHE-RSA-AES128-SHA
If my options for setting the cipher are:
SSL_kRSA (RSA Key Exchange)
SSL_kEDH (tmp DH key no DH cert)
SSL_aRSA (Authenticate with RSA)
SSL-aDSS (Authenticate with DSS)
SSL_DES (DES)
SSL_3DES (3DES)
SSL_RC4 (RC4)
SSL_RC2 (RC2)
SSL_AES (AES)
SSL_MD5 (MD5)
SSL_SHA1 (SHA1)
SSL_SHA256 (SHA256)
SSL_SHA384 (SHA384)
SSL_RSA ([SSL_kRSA|SSL_aRSA] RSA)
SSL_DSS ([SSL_aDSS] Authenticate with DSS)
I can set multiple things by something like:
SSL_RSA | SSL_AES
Protocol is TLSv1.2
What should my cipher look like?
"Pretend like i can't load any libraries at all." If that is true, you will need to implement the cipher itself plus the SSL handling layer ^_^.
Assuming you are using OpenSSL and have TCP established with socket_fd, you need to create a SSL_CTX with SSL_CTX_new (SSLv23_client_method()). Normally, to set the cipher list, you use SSL_CTX_set_cipher_list(ctx, "HIGH:!aNULL:!eNULL:#STRENGTH"), see http://openssl.org/docs/apps/ciphers.html for all available options, you may specific a particular cipher.
Then create a SSL session with SSL_new(ctx) and SSL_set_fd (ssl, socket_fd), after that use SSL_connect(...), SSL_read(...)/SSL_write(...) to communicate with server.
After all have been done, SSL_shutdown(...) and SSL_Free(...), SSL_CTX_Free(...).

How to add PFS to socket server written in c and openssl

I try to add PFS (perfect forward secrecy) to my client-server application.
When I run a server with the following command:
openssl s_server -key ./key.pem -cert ./cert.pem -accept 443 -cipher ECDHE-RSA-AES128-SHA -tls1_2
I am able to connect with my client given the following ctx:
SSL_CTX* initCTX() {
SSL_METHOD *method;
SSL_CTX *ctx;
SSL_library_init();
OpenSSL_add_all_algorithms();
SSL_load_error_strings();
method = TLSv1_2_client_method();
ctx = SSL_CTX_new(method);
if(ctx == NULL) {
ERR_print_errors_fp(stderr);
return NULL;
}
SSL_CTX_set_cipher_list(ctx, "ECDHE-RSA-AES128-SHA");
return ctx;
}
When I run my server application with the following ctx:
SSL_CTX* init_ssl_ctx() {
SSL_METHOD const *method;
SSL_CTX *ctx;
SSL_library_init();
OpenSSL_add_all_algorithms();
SSL_load_error_strings();
method = TLSv1_2_server_method();
ctx = SSL_CTX_new(method);
if(ctx == NULL) {
ERR_print_errors_fp(stderr);
abort();
}
SSL_CTX_set_session_cache_mode(ctx, SSL_SESS_CACHE_OFF);
SSL_CTX_set_cipher_list(ctx, "ECDHE-RSA-AES128-SHA");
// ADDITIONAL CTX MODIFICATIONS TO ENABLE ECDHE
SSL_CTX_use_certificate_file(ctx, "./cert.pem", SSL_FILETYPE_PEM);
SSL_CTX_use_PrivateKey_file(ctx, "./key.pem", SSL_FILETYPE_PEM);
return ctx;
}
and try to connect with the client, then I get an no shared cipher error.
The private key has been created with openssl genrsa.
Well my question is: How do I have to modify the ctx to add ECDHE support. I guess that I have to select a curve and I probably need to create and exchange keys for every connection.
Do I still need the private key file? And when yes — what is it used for?
Well what I actually missed was the configuration of the Diffie-Hellman parameters and the Elliptic curve Diffie-Hellman. If you do not configure them ...
the PFS cipher suites will be silently ignored.
More information and examples on how to configure and include Diffie-Hellman parameters and Elliptic curve Diffie-Hellman in your C socket server can be found here: http://wiki.openssl.org/index.php/Diffie-Hellman_parameters
... then I get an no shared cipher error.
How do I have to modify the ctx to add ECDHE support?
The cipher suite is a product of both client and server capabilities. I find that I need to add 12 to 16 to ensure most clients can be accommodated.
Here's the cipher list I use. It includes the upcoming ChaCha and Poly cipher suites, and include the downlevel client suites. If you only want ECDHE, the rdeuce the list even further.
// *_CHACHA20_POLY1305 are 3x to 4x faster than existing cipher suites.
// http://googleonlinesecurity.blogspot.com/2014/04/speeding-up-and-strengthening-https.html
// Use them if available. Normative names can be found at (TLS spec depends on IPSec spec):
// http://tools.ietf.org/html/draft-nir-ipsecme-chacha20-poly1305-01
// http://tools.ietf.org/html/draft-mavrogiannopoulos-chacha-tls-02
"TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305:"
"TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305:"
"TLS_ECDHE_ECDSA_WITH_CHACHA20_SHA:"
"TLS_ECDHE_RSA_WITH_CHACHA20_SHA:"
"TLS_DHE_RSA_WITH_CHACHA20_POLY1305:"
"TLS_RSA_WITH_CHACHA20_POLY1305:"
"TLS_DHE_RSA_WITH_CHACHA20_SHA:"
"TLS_RSA_WITH_CHACHA20_SHA:"
// Done with bleeding edge, back to TLS v1.2 and below
"TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384:"
"TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384:"
"TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256:"
"TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256:"
"TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384:"
"TLS_DHE_DSS_WITH_AES_256_GCM_SHA384:"
"TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256:"
"TLS_DHE_DSS_WITH_AES_128_GCM_SHA256:"
// TLS v1.0 (with some SSLv3 interop)
"TLS_DHE_RSA_WITH_AES_256_CBC_SHA384:"
"TLS_DHE_DSS_WITH_AES_256_CBC_SHA256:"
"TLS_DHE_RSA_WITH_AES_128_CBC_SHA:"
"TLS_DHE_DSS_WITH_AES_128_CBC_SHA:"
"TLS_DHE_RSA_WITH_3DES_EDE_CBC_SHA:"
"TLS_DHE_DSS_WITH_3DES_EDE_CBC_SHA:"
"SSL_DH_RSA_WITH_3DES_EDE_CBC_SHA:"
"SSL_DH_DSS_WITH_3DES_EDE_CBC_SHA:"
// RSA key transport sucks, but they are needed as a fallback.
// For example, microsoft.com fails under all versions of TLS
// if they are not included. If only TLS 1.0 is available at
// the client, then google.com will fail too. TLS v1.3 is
// trying to deprecate them, so it will be interesteng to see
// what happens.
"TLS_RSA_WITH_AES_256_CBC_SHA256:"
"TLS_RSA_WITH_AES_256_CBC_SHA:"
"TLS_RSA_WITH_AES_128_CBC_SHA256:"
"TLS_RSA_WITH_AES_128_CBC_SHA:"
The ChaCha/Poly cipher suites are available in OpenSSL 1.0.2. So you can test Google's implementation if interested.
The RFCs do not specify who picks the cipher. By convention, the server usually honors the client's preference. To ensure your server picks the cipher suite, you should add SSL_OP_CIPHER_SERVER_PREFERENCE to the server's context options. See SSL_CTX_set_options(3).

C HTTP Proxy, the browser shows "waiting for" indefinetely

I have a hard time implementing a proxy server in C. It works for a few first webpages but then I get blocked while waiting for a new request.
Design:
Firefox -> Proxy -> Webserver --.
Firefox <- Proxy <- Webserver <-'
So each request is a round-trip from the browser to the proxy and the server and then back. Until the response from the request comes back, nothing should happen. I use no pipelining, threads or anything like that but rather only recv() and send() in a linear serial manner (for simplicity and intuition). I also don't close any sockets as I want to have a persistent connection.
I expect to be able and fetch one whole webpage including subrequested resources like css, img, js, etc..
In my implementation I manage to fetch a few first requests for a webpage. Then it hangs at step 1.
Implementation:
puts("Waiting for user to connect..");
int sock_user = accept(sock, (struct sockaddr*)NULL, NULL);
int sock_host = -1;
printf("User connected.\n");
// Accept requests
while(1){
http_request req;
http_response resp;
// 1. Client ==> Proxy Server
http_parse_request(sock_user, &req); // uses recv()
// 2. Client Proxy ==> Server
if (sock_host < 0)
sock_host = proxy_connect_host(req.header->host);
write(sock_host, req.header->raw_data, req.header->raw_size);
// 3. Client Proxy <== Server
http_parse_response(sock_host, &resp); // uses recv()
// 4. Client <== Proxy Server
write(sock_user, resp.header->raw_data, resp.header->raw_size);
write(sock_user, resp.body ->first_block->data, resp.body ->first_block->size);
}
Log:
---- ......................................... ----
---- after succesfully responded to 4 requests ----
Client ==> Proxy Server
Received 389
Client Proxy ==> Server
Sending.. 389
Sent 389
Client Proxy <== Server
Got header 312
Got body 1437
Response total 1749
Client <== Proxy Server
Sending header.. 312
Sent 312
Sending body.. 1437
Sent 1437
Client ==> Proxy Server
---- Hangs/blocks here ----
Firebug:
Wireshark:
I have no intuition as to what the reason for that block is and I have spent a whole week trying to resolve this problem without a breakthrough.
Among things tried to resolve problem:
Sending some extra CRLF for each response body
Checked the return value from each recv() and send(). (in the log above,
the values getting printed are the return values from recv and
send)
I hope someone could give at least some direction as to how to troubleshoot this or my brain will soon explode :)
You have to be very careful not to read too much data. E.g. make sure that:
header is read only up to the double CRLF; store extra data and sent them with the body
sending of body starts not before the server has sent the complete header (does not apply to this GET case but is important for POST or CONNECT)
there are received and sent only Content-Length bytes from the body
This applies to both the client -> proxy request and the server -> proxy response.
Your sample code stays in an endless loop (while (1) ...). How do you abort this? Do you honor the "Proxy-Connection" header?

Resources