I am using haproxy as a load balancer.
One of type incoming request use multipart/form-data( content type : application/pdf ).
This type of requests are not forwarded.
There is information in log:
qwerty qwerty/<NOSRV> 13/-1/-1/-1/13 500 208 - - PR-- 1/1/0/0/0 0/0
{|Apache-HttpClient/4.5.5 (Java/12.0.1)
Other requests ( content type: application/json ) are forwarded correctly
Haproxy 2.4.19
I try to add:
acl multipart hdr_beg(Content-Type) -i multipart/form-data
use_backend xyz if multipart
Without result.
Hproxy cfg:
global
#log 127.0.0.1 local0 debug - rsyslog logging
log stdout len 65535 format raw local0 debug
maxconn 4000
stats socket :9000 mode 660 level admin
lua-load /tmp/lua/haproxy_okapi_connector.lua
h1-case-adjust authorization Authorization
#---------------------------------------------------------------------
# common defaults that all the 'listen' and 'backend' sections will
# use if not designated in their block
#---------------------------------------------------------------------
defaults
mode http
log global
option dontlognull
option httplog
option http-buffer-request
option forwardfor
retries 3
timeout http-request 90s
timeout queue 2m
timeout connect 90s
timeout client 2m
timeout server 2m
timeout http-keep-alive 90s
timeout check 90s
maxconn 30000
#default log
log-format "%[src] %si:%sp %ci:%cp [%tr] %ft %b/%s %TR/%Tw/%Tc/%Tr/%Ta %ST %B %CC %CS %tsc %ac/%fc/%bc/%sc/%rc %sq/%bq %hr %hs %{+Q}r %[capture.req.hdr(1)] %[capture.req.hdr(2)] %[capture.req.hdr(3)] %[capture.req.hdr(4)] %[capture.req.hdr(5)]"
#---------------------------------------------------------------------
# Frontend with Headers
#---------------------------------------------------------------------
frontend iwa-proxy
bind *:8080
http-request set-header X-Forwarded-Proto http
http-request set-header forwarded "host=xxxx;proto=http;proto-version="
http-request set-header x-forwarded-for xyz
http-request set-header host xyz
option h1-case-adjust-bogus-client
http-request set-var(req.token) str(""),lua.create_access_token
http-request add-header "authorization" 'Bearer %[var(req.token)]'
declare capture request len 800000
declare capture response len 800000
http-request capture req.body id 0
http-response capture res.body id 0
capture request header user-agent len 500
capture request header Host len 500
capture request header X-Forwarded-For len 500
capture request header X-Forwarded-Proto len 500
capture request header X-Forwarded-Host len 500
default_backend xyz
#---------------------------------------------------------------------
# Backend
#---------------------------------------------------------------------
backend iwa-ogate
mode http
balance roundrobin
option forwardfor
option h1-case-adjust-bogus-server
server xyz xxx:443 ssl verify none
Related
my react app on production mode (https://www.cli-domain.com) -> main domain, it's using laravel API server (https://admin.cli-domain.com) -> subdomain created by apache virtual host, both domain are running on same server,
whenever i'm trying to send request from react app to API, it showing CORS error.
i'm using axios for api request, i set headers fields Access-Control-Allow-Origin * on my client side,
on my laravel API i used fruitcake/cors package to handle middleware via allow cross origin requests, as well i tried with htaccess Header allow cross origin snippets, and i use laravel Cors.php file to allow cross origin method,
everything ended up with failure result,
still i can't able to send a successful request to my laravel API,
please assist me achieve this is possible
Here i attach .htaccess method for reference
Header always set Access-Control-Allow-Origin "*"
Header always set Access-Control-Allow-Credentials "true"
Header always set Access-Control-Allow-Methods "POST, GET, OPTIONS, DELETE, PUT"
Header always set Access-Control-Max-Age "1000"
Header always set Access-Control-Allow-Headers "x-requested-with, Content-Type, origin, authorization, accept, client-security-token"
and then another try with fruitcake / cors package reference code
protected $middleware = [
...
\Fruitcake\Cors\HandleCors::class, # this line
];
Cors.php
<?php
return [
/*
|--------------------------------------------------------------------------
| Laravel CORS Options
|--------------------------------------------------------------------------
|
| The allowed_methods and allowed_headers options are case-insensitive.
|
| You don't need to provide both allowed_origins and allowed_origins_patterns.
| If one of the strings passed matches, it is considered a valid origin.
|
| If ['*'] is provided to allowed_methods, allowed_origins or allowed_headers
| all methods / origins / headers are allowed.
|
*/
/*
* You can enable CORS for 1 or multiple paths.
* Example: ['api/*']
*/
'paths' => ['api/v1/tasker/profileupload', '*'],
/*
* Matches the request method. `['*']` allows all methods.
*/
'allowed_methods' => ['POST', 'GET', 'DELETE', 'PUT', '*'],
/*
* Matches the request origin. `['*']` allows all origins. Wildcards can be used, eg `*.mydomain.com`
*/
'allowed_origins' => ['https://www.doain-cus.com'],
/*
* Patterns that can be used with `preg_match` to match the origin.
*/
'allowed_origins_patterns' => ['Google/'],
/*
* Sets the Access-Control-Allow-Headers response header. `['*']` allows all headers.
*/
'allowed_headers' => ['X-Custom-Header', 'Upgrade-Insecure-Requests', '*'],
/*
* Sets the Access-Control-Expose-Headers response header with these headers.
*/
'exposed_headers' => [],
/*
* Sets the Access-Control-Max-Age response header when > 0.
*/
'max_age' => 0,
/*
* Sets the Access-Control-Allow-Credentials header.
*/
'supports_credentials' => false,
];
If you use the HandleCors middleware you shouldn't set the headers in your .htaccess (it will end up with CORS error because headers can't be setted twice), your cors.php config file is enough.
So you can remove these lines from your .htaccess:
Header always set Access-Control-Allow-Origin "*"
Header always set Access-Control-Allow-Methods "POST, GET, OPTIONS, DELETE, PUT"
Header always set Access-Control-Allow-Headers "x-requested-with, Content-Type, origin, authorization, accept, client-security-token"
You also don't have to set it from the client side, it is a server side configuration.
I tested to send a message with content-type set to application/xml to a Camel/AS2 server connection. However, I got 500 response. Does Camel/AS2 support application/xml content-typed messages?
Here comes the stack trace:
2022-01-16 11:44:23,725 [AS2Hdlr-6405] INFO .AS2ServerConnection [] - Processing new AS2 request
2022-01-16 11:44:23,725 [AS2Hdlr-6405] WARN nent.as2.AS2Consumer [] - Failed to process AS2 message
org.apache.http.HttpException: Failed to extract EDI message: invalid content type 'application/xml' for AS2 request message
at org.apache.camel.component.as2.api.util.HttpMessageUtils.extractEdiPayload(HttpMessageUtils.java:169) ~[camel-as2-api-3.13.0.jar!/:3.13.0]
at org.apache.camel.component.as2.AS2Consumer.handle(AS2Consumer.java:124) [camel-as2-3.13.0.jar!/:3.13.0]
at org.apache.http.protocol.HttpService.doService(HttpService.java:437) [wildfly-elytron-1.18.1.Final.jar!/:1.18.1.Final]
at org.apache.http.protocol.HttpService.handleRequest(HttpService.java:342) [wildfly-elytron-1.18.1.Final.jar!/:1.18.1.Final]
at org.apache.camel.component.as2.api.AS2ServerConnection$RequestHandlerThread.run(AS2ServerConnection.java:147) [camel-as2-api-3.13.0.jar!/:3.13.0]
2022-01-16 11:44:23,725 [AS2Hdlr-6405] INFO .AS2ServerConnection [] - Client closed connection
Getting error on SOLR for GET request Request is larger than 20 MB or headers are too large. Please reduce the size of your request and try again, or contact support#websolr.com with a sample of your request for further assistance.
Sample request below:
removing curl request due to security reason
Response:
HTTP/2 413
date: Thu, 06 May 2021 16:25:31 GMT
content-type: application/json
content-length: 218
{"code":413,"message":"Request is larger than 20 MB or headers are too large. Please reduce the size of your request and try again, or contact support#websolr.com with a sample of your request for further assistance."}
I am not sure how the request become more than 20MB. Also send nothing on header.
Websolr support here. When you pass those query parameters in the curl command, they are part of the Request-Line header. There are over 5900 characters in that line, which is what's triggering the HTTP 413.
As a workaround, just use a POST:
curl -i "removing data due to security reason"
I'm using CXF and jaxrs:client to connect to a Restful service. However, the restful service timesout after 300 seconds - it's DDOS mitigation at their end. According to the owners of the Restful service, if we include a Keep-Alive header in oyr request, the timeout won't occur. How can I configure Keep-Alive in CXF or jaxrs:client calls?
Thanks in advance
Angus
You can set a header for a jaxrs:client usign the jaxrs:headers tag
<jaxrs:client id="" serviceClass="" address="" inheritHeaders="true">
<jaxrs:headers>
<entry key="Connection" value="keep-alive"/>
</jaxrs:headers>
</jaxrs:client>
Also you can configure timeouts in the http-conf:client tag of the http:conduit element
<http-conf:conduit name="*">
<http-conf:client Connection="Keep-Alive"
ConnectionTimeout="0"
ReceiveTimeout="0" />
</http-conf:conduit>
The complete list of parameters is here.
Note that keep-alive is the default. ReceiveTimeout 0 means no timeout. Default value is 30000 (30s)
I have a hard time implementing a proxy server in C. It works for a few first webpages but then I get blocked while waiting for a new request.
Design:
Firefox -> Proxy -> Webserver --.
Firefox <- Proxy <- Webserver <-'
So each request is a round-trip from the browser to the proxy and the server and then back. Until the response from the request comes back, nothing should happen. I use no pipelining, threads or anything like that but rather only recv() and send() in a linear serial manner (for simplicity and intuition). I also don't close any sockets as I want to have a persistent connection.
I expect to be able and fetch one whole webpage including subrequested resources like css, img, js, etc..
In my implementation I manage to fetch a few first requests for a webpage. Then it hangs at step 1.
Implementation:
puts("Waiting for user to connect..");
int sock_user = accept(sock, (struct sockaddr*)NULL, NULL);
int sock_host = -1;
printf("User connected.\n");
// Accept requests
while(1){
http_request req;
http_response resp;
// 1. Client ==> Proxy Server
http_parse_request(sock_user, &req); // uses recv()
// 2. Client Proxy ==> Server
if (sock_host < 0)
sock_host = proxy_connect_host(req.header->host);
write(sock_host, req.header->raw_data, req.header->raw_size);
// 3. Client Proxy <== Server
http_parse_response(sock_host, &resp); // uses recv()
// 4. Client <== Proxy Server
write(sock_user, resp.header->raw_data, resp.header->raw_size);
write(sock_user, resp.body ->first_block->data, resp.body ->first_block->size);
}
Log:
---- ......................................... ----
---- after succesfully responded to 4 requests ----
Client ==> Proxy Server
Received 389
Client Proxy ==> Server
Sending.. 389
Sent 389
Client Proxy <== Server
Got header 312
Got body 1437
Response total 1749
Client <== Proxy Server
Sending header.. 312
Sent 312
Sending body.. 1437
Sent 1437
Client ==> Proxy Server
---- Hangs/blocks here ----
Firebug:
Wireshark:
I have no intuition as to what the reason for that block is and I have spent a whole week trying to resolve this problem without a breakthrough.
Among things tried to resolve problem:
Sending some extra CRLF for each response body
Checked the return value from each recv() and send(). (in the log above,
the values getting printed are the return values from recv and
send)
I hope someone could give at least some direction as to how to troubleshoot this or my brain will soon explode :)
You have to be very careful not to read too much data. E.g. make sure that:
header is read only up to the double CRLF; store extra data and sent them with the body
sending of body starts not before the server has sent the complete header (does not apply to this GET case but is important for POST or CONNECT)
there are received and sent only Content-Length bytes from the body
This applies to both the client -> proxy request and the server -> proxy response.
Your sample code stays in an endless loop (while (1) ...). How do you abort this? Do you honor the "Proxy-Connection" header?