The netty4-http component is setting an invalid "host" HTTP header when no port is defined in the uri for requests.
netty4-http sets the header in DefaultNettyHttpBinding.toNettyRequest where URI is used to parse the uri string but URI give -1 if no port is defined.
For example the host header could be set to "hostname:-1" which is not accepted of some proxy servers that check the validity of the host header.
For example Apache proxy will return a http error 400(Bad request).
Also see ch. "14.23 Host" https://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html for definition.
See https://www.rfc-editor.org/rfc/rfc7230#section-5.4
// This is how it's done in DefaultNettyHttpBinding.toNettyRequest
URI u = new URI(uri);
String hostHeader = u.getHost() + (u.getPort() == 80 ? "" : ":" + u.getPort());
request.headers().set(HttpHeaderNames.HOST.toString(), hostHeader);
LOG.trace("Host: {}", hostHeader);
As a workaround I'm using a custom NettyHttpBinding class but it had been nice to get a fix for it.
Related
I created an API endpoint using Google Cloud Functions and am trying to call it from a JS fetch function.
I am running into errors that I am pretty sure are related to either CORS or the output format, but I'm not really sure what is going on. A few other SO questions are similar, and helped me realize I needed to remove the mode: "no-cors". Most mention enabling CORS on the BE, so I added response.headers.set('Access-Control-Allow-Origin', '*') - which I learned of in this article - to ensure CORS would be enabled... But I still get the "Failed to fetch" error.
The Full Errors (reproducible in the live demo linked below) are:
Uncaught Error: Cannot add node 1 because a node with that id is
already in the Store. (This one is probably unrelated?)
Access to fetch at
'https://us-central1-stargazr-ncc-2893.cloudfunctions.net/nearest_csc?lat=37.75&lon=-122.5'
from origin 'https://o2gxx.csb.app' has been blocked by CORS policy:
Request header field access-control-allow-origin is not allowed by
Access-Control-Allow-Headers in preflight response.
GET
https://us-central1-stargazr-ncc-2893.cloudfunctions.net/nearest_csc?lat=37.75&lon=-122.5 net::ERR_FAILED
Uncaught (in promise) TypeError: Failed to fetch
See Code Snippets below, please note where I used <---- *** Message *** to denote parts of the code that have recently changed, giving me one of those two errors.
Front End Code:
function getCSC() {
let lat = 37.75;
let lng = -122.5;
fetch(
`https://us-central1-stargazr-ncc-2893.cloudfunctions.net/nearest_csc?lat=${lat}&lon=${lng}`,
{
method: "GET",
// mode: "no-cors", <---- **Uncommenting this predictably gets rid of CORS error but returns a Opaque object which seems to have no data**
headers: {
// Accept: "application/json", <---- **Originally BE returned stringified json. Not sure if I should be returning it as something else or if this is still needed**
Origin: "https://lget3.csb.app",
"Access-Control-Allow-Origin": "*"
}
}
)
.then(response => {
console.log(response);
console.log(response.json());
});
}
Back End Code:
import json
import math
import os
import flask
def nearest_csc(request):
"""
args: request object w/ args for lat/lon
returns: String, either with json representation of nearest site information or an error message
"""
lat = request.args.get('lat', type = float)
lon = request.args.get('lon', type = float)
# Get list of all csc site locations
with open(file_path, 'r') as f:
data = json.load(f)
nearby_csc = []
# Removed from snippet for clarity:
# populate nearby_csc (list) with sites (dictionaries) as elems
# Determine which site is the closest, assigned to var 'closest_site'
# Grab site url and return site data if within 100 km
if dist_km < 100:
closest_site['dist_km'] = dist_km
// return json.dumps(closest_site) <--- **Original return statement. Added 4 lines below in an attempt to get CORS set up, but did not seem to work**
response = flask.jsonify(closest_site)
response.headers.set('Access-Control-Allow-Origin', '*')
response.headers.set('Access-Control-Allow-Methods', 'GET, POST')
return response
return "No sites found within 100 km"
Fuller context for code snippets above:
Here is a Code Sandbox Demo of the above.
Here is the full BE code on GitHub, minus the most recent attempt at adding CORS.
The API endpoint.
I'm also wondering if it's possible that CodeSandbox does CORS in a weird way, but have had the same issue running it on localhost:3000, and of course in prod would have this on my own personal domain.
The Error would appear to be CORS-related ( 'https://o2gxx.csb.app' has been blocked by CORS policy: Request header field access-control-allow-origin is not allowed by Access-Control-Allow-Headers in preflight response.) but I thought adding response.headers.set('Access-Control-Allow-Origin', '*') would solve that. Do I need to change something else on the BE? On the FE?
TLDR;
I am getting the Errors "Failed to fetch" and "field access-control-allow-origin is not allowed by Access-Control-Allow-Headers" even after attempts to enable CORS on backend and add headers to FE. See the links above for live demo of code.
Drop the part of your frontend code that adds a Access-Control-Allow-Origin header.
Never add Access-Control-Allow-Origin as a request header in your frontend code.
The only effect that’ll ever have is a negative one: it’ll cause browsers to do CORS preflight OPTIONS requests even in cases when the actual (GET, POST, etc.) request from your frontend code would otherwise not trigger a preflight. And then the preflight will fail with this message:
Request header field Access-Control-Allow-Origin is not allowed by Access-Control-Allow-Headers in preflight response
…that is, it’ll fail with that unless the server the request is being made to has been configured to send an Access-Control-Allow-Headers: Access-Control-Allow-Origin response header.
But you never want Access-Control-Allow-Origin in the Access-Control-Allow-Headers response-header value. If that ends up making things work, you’re actually just fixing the wrong problem. Because the real fix is: never set Access-Control-Allow-Origin as a request header.
Intuitively, it may seem logical to look at it as “I’ve set Access-Control-Allow-Origin both in the request and in the response, so that should be better than just having it in the response” — but it’s actually worse than only setting it in the response (for the reasons described above).
So the bottom line: Access-Control-Allow-Origin is solely a response header, not a request header. You only ever want to set it in server-side response code, not frontend JavaScript code.
The code in the question was also trying to add an Origin header. You also never want to try to set that header in your frontend JavaScript code.
Unlike the case with the Access-Control-Allow-Origin header, Origin is actually a request header — but it’s a special header that’s controlled completely by browsers, and browsers won’t ever allow your frontend JavaScript code to set it. So don’t ever try to.
I have a Springboot application using Apache Camel AMQP component to comsume messages from a Solace Queue. To send a message to the Queue I use Postman and the Solace REST API. In order to differentiate the message type I add Content-Type to the header of the Http request in Postman. I used SDKPerf to check the message header consumed from solace and the message header is found under "HTTP Content Type" along with other headers.
However, I can't seem to find a way to get this Content-Type from Camel Side. In the documentation it says
String header = exchange.getIn().getHeader(Exchange.CONTENT_TYPE, String.class);
However this always produces null. Any Ideas how to get the message properties in Camel?
EDIT: I think it's actually due to the fact that Camel is using QPid JMS, and there is no JMS API way of getting the Content Type, it's not in the spec. Even though AMQP 1.0 does support content-type as a property. But yeah, my suggestion of a custom property below is still probably the way I would go.
https://camel.apache.org/components/3.20.x/amqp-component.html
https://www.amqp.org/sites/amqp.org/files/amqp.pdf
Edited for clarity & corrections. TL/DR: use a custom user property header.
The SMF Content Type header in the original (REST) message is passed through to the consumed AMQP message as a property content-type, however the JMS API spec does not expose this; there is no way in standard JMS to retrieve this value. It is, however, used by the broker to set the type of message (e.g. TextMessage). Check "Content-Type Mapping to Solace Message Types" in the Solace docs.
Using Solace's SDKPerf AMQP JMS edition to dump the received message to console (note this uses QPid libraries):
./sdkperf_jmsamqp.sh -cip=amqp://localhost:5672 -stl=a/b/c
-md -q
curl http://localhost:9000/TOPIC/a/b/c -d 'hello' -H 'Content-Type: text'
^^^^^^^^^^^^^^^^^^ Start Message ^^^^^^^^^^^^^^^^^^^^^^^^^^^
JMSDeliveryMode: PERSISTENT
JMSDestination: a/b/c
JMSExpiration: 0
JMSPriority: 4
JMSTimestamp: 0
JMSRedelivered: false
JMSCorrelationID: null
JMSMessageID: null
JMSReplyTo: null
JMSType: null
JMSProperties: {JMSXDeliveryCount:1;}
Object Type: TextMessage
Text: len=5
hello
The header does not get mapped through, but does get used to set the message type. If I remove that HTTP header, the received AMQP message is binary. But since other types of Content-Types also map to TextMessages (e.g. application/json, application/xml, etc.), the fact you're receiving a TextMessage is not enough to infer exactly what Content-Type you published your REST message with.
For completeness, I used WireShark with an AMQP decoder, and you can see the header present on the received AMQP message:
Frame 3: 218 bytes on wire (1744 bits), 218 bytes captured (1744 bits) on interface \Device\NPF_Loopback, id 0
Null/Loopback
Internet Protocol Version 4, Src: 127.0.0.1, Dst: 127.0.0.1
Transmission Control Protocol, Src Port: 5672, Dst Port: 60662, Seq: 2, Ack: 1, Len: 174
Advanced Message Queueing Protocol
Length: 174
Doff: 2
Type: AMQP (0)
Channel: 2
Performative: transfer (20)
Arguments (5)
Message-Header
Durable: True
Message-Annotations (map of 1 element)
x-opt-jms-dest (byte): 1
Message-Properties
To: a/b/c
Content-Type: text <----------
Application-Properties (map of 1 element)
AaronEncoding (str8-utf8): CustomText
AMQP-Value (str32-utf8): hello
So my suggestion is this:
Set an additional custom header, a User Property, which will get passed through to the AMQP message:
curl http://localhost:9000/TOPIC/a/b/c -d 'hello' -H 'Solace-User-Property-AaronEncoding: CustomText' -H 'Content-Type: text'
JMSDestination: a/b/c
JMSProperties: {AaronEncoding:CustomText;JMSXDeliveryCount:1;}
Object Type: TextMessage
Text: len=5
hello
My always-goto guide for Solace REST interactions: https://docs.solace.com/API/RESTMessagingPrtl/Solace-REST-Message-Encoding.htm
Hope that helps!
It may have a different name in Camel. Try either printing all the headers or stop it in the debugger and examine the incoming message.
I created the stub classes using CXF wsdl2java tool.
I am using Apache CXF library, with JCIFS. I validated the WSDL file itself through couple tools, it is good. Here is the code. It looks like some setting I must do.
//JCIFS Authentication related code
jcifs.Config.setProperty("jcifs.smb.client.domain", "NTS");
jcifs.Config.setProperty("jcifs.netbios.wins", "ecmchat.mark.gov");
jcifs.Config.setProperty("jcifs.smb.client.soTimeout", "300000"); // 5 minutes
jcifs.Config.setProperty("jcifs.netbios.cachePolicy", "1200"); // 20 minutes
jcifs.Config.setProperty("jcifs.smb.client.username", "user");
jcifs.Config.setProperty("jcifs.smb.client.password", "password");
//Register the jcifs URL handler to enable NTLM
jcifs.Config.registerSmbURLHandler();
//WSDL and Client settings
URL wsdlURL = BF.WSDL_LOCATION;
if (args.length > 0 && args[0] != null && !"".equals(args[0])) {
File wsdlFile = new File(args[0]);
try {
if (wsdlFile.exists()) {
wsdlURL = wsdlFile.toURI().toURL();
} else {
wsdlURL = new URL(args[0]);
}
} catch (MalformedURLException e) {
e.printStackTrace();
}
}
BF ss = new BF(wsdlURL, SERVICE_NAME);
BFSoap port = ss.getBFSoap12();
Client client = ClientProxy.getClient(port);
HTTPConduit http = (HTTPConduit) client.getConduit();
HTTPClientPolicy httpClientPolicy = new HTTPClientPolicy();
httpClientPolicy.setConnectionTimeout(36000);
httpClientPolicy.setAllowChunking(false);
httpClientPolicy.setReceiveTimeout(32000);
http.setClient(httpClientPolicy);
// Calling the method
System.out.println("Invoking testMethod...");
String _testMethod__return = port.testMethod();
System.out.println("testMethod.result=" + _testMethod__return);
I am getting the following exception
Caused by: com.ctc.wstx.exc.WstxParsingException: Unexpected close tag </span>; expected </br>.
at [row,col,system-id]: [59,22,"https://ecmchat.mark.gov/BF/BF.asmx"]
at com.ctc.wstx.sr.StreamScanner.constructWfcException(StreamScanner.java:621)
at com.ctc.wstx.sr.StreamScanner.throwParseError(StreamScanner.java:491)
at com.ctc.wstx.sr.StreamScanner.throwParseError(StreamScanner.java:475)
at com.ctc.wstx.sr.BasicStreamReader.reportWrongEndElem(BasicStreamReader.java:3365)
at com.ctc.wstx.sr.BasicStreamReader.readEndElem(BasicStreamReader.java:3292)
at com.ctc.wstx.sr.BasicStreamReader.nextFromTree(BasicStreamReader.java:2911)
at com.ctc.wstx.sr.BasicStreamReader.next(BasicStreamReader.java:1123)
at org.apache.cxf.staxutils.StaxUtils.readDocElements(StaxUtils.java:1361)
at org.apache.cxf.staxutils.StaxUtils.readDocElements(StaxUtils.java:1255)
at org.apache.cxf.staxutils.StaxUtils.read(StaxUtils.java:1183)
at org.apache.cxf.wsdl11.WSDLManagerImpl.loadDefinition(WSDLManagerImpl.java:235)
... 9 more
If I comment out the JCIFS NTLM authentication code, I get a HTTP 401 error. Therefore, I believe, at least it is passing some kind of authorization step.
And, if I use local WSDL in place of remote URL WSDL, then I get a different error like "method not implemented" on the call to the method. May be this is due to me not using the local WSDL correctly. I do not even know if we can use the local WSDL reference for remote service.
Then, I created a SoapUI dummy service with this WSDL, and the same code (but without the JCIFS authentication code) works good, and successfully calls the methods.
It appears to me that I must add some more appropriate settings in the configuration related code.
Am I right, and are you aware of any, for NTLM authentication and Apache CXF?
But parsing error is confusing???
I do not know if this is related.
My original WSDL URL that I gave was this.
https://ecmchat.mark.gov/BF/BF.asmx
I added a ?wsdl like below
https://ecmchat.mark.gov/BF/BF.asmx?wsdl
Then I am getting a different error.
I wonder why it is working if I access my local SoapUI version of the same WSDL service, but not for the remote one.
Invoking testMethod...
Jan 07, 2020 10:47:25 AM org.apache.cxf.phase.PhaseInterceptorChain doDefaultLogging
WARNING: Interceptor for {https://ecmchat.mark.gov}BF#{https://ecmchat.mark.gov}testMethod has thrown exception, unwinding now
java.lang.UnsupportedOperationException: Method not implemented.
at java.net.URLStreamHandler.openConnection(URLStreamHandler.java:96)
at java.net.URL.openConnection(URL.java:1028)
at org.apache.cxf.transport.https.HttpsURLConnectionFactory.createConnection(HttpsURLConnectionFactory.java:92)
at org.apache.cxf.transport.http.URLConnectionHTTPConduit.createConnection(URLConnectionHTTPConduit.java:121)
at org.apache.cxf.transport.http.URLConnectionHTTPConduit.setupConnection(URLConnectionHTTPConduit.java:125)
at org.apache.cxf.transport.http.HTTPConduit.prepare(HTTPConduit.java:505)
at org.apache.cxf.interceptor.MessageSenderInterceptor.handleMessage(MessageSenderInterceptor.java:47)
at org.apache.cxf.phase.PhaseInterceptorChain.doIntercept(PhaseInterceptorChain.java:308)
at org.apache.cxf.endpoint.ClientImpl.doInvoke(ClientImpl.java:530)
at org.apache.cxf.endpoint.ClientImpl.invoke(ClientImpl.java:441)
at org.apache.cxf.endpoint.ClientImpl.invoke(ClientImpl.java:356)
at org.apache.cxf.endpoint.ClientImpl.invoke(ClientImpl.java:314)
at org.apache.cxf.frontend.ClientProxy.invokeSync(ClientProxy.java:96)
at org.apache.cxf.jaxws.JaxWsClientProxy.invoke(JaxWsClientProxy.java:140)
at com.sun.proxy.$Proxy33.testMethod(Unknown Source)
at edison.learn.BFSoap_BFSoap12_Client.main(BFSoap_BFSoap12_Client.java:90)
Exception in thread "main" javax.xml.ws.soap.SOAPFaultException: Method not implemented.
at org.apache.cxf.jaxws.JaxWsClientProxy.mapException(JaxWsClientProxy.java:195)
at org.apache.cxf.jaxws.JaxWsClientProxy.invoke(JaxWsClientProxy.java:145)
at com.sun.proxy.$Proxy33.testMethod(Unknown Source)
at edison.learn.BFSoap_BFSoap12_Client.main(BFSoap_BFSoap12_Client.java:90)
Caused by: java.lang.UnsupportedOperationException: Method not implemented.
at java.net.URLStreamHandler.openConnection(URLStreamHandler.java:96)
at java.net.URL.openConnection(URL.java:1028)
at org.apache.cxf.transport.https.HttpsURLConnectionFactory.createConnection(HttpsURLConnectionFactory.java:92)
at org.apache.cxf.transport.http.URLConnectionHTTPConduit.createConnection(URLConnectionHTTPConduit.java:121)
at org.apache.cxf.transport.http.URLConnectionHTTPConduit.setupConnection(URLConnectionHTTPConduit.java:125)
at org.apache.cxf.transport.http.HTTPConduit.prepare(HTTPConduit.java:505)
at org.apache.cxf.interceptor.MessageSenderInterceptor.handleMessage(MessageSenderInterceptor.java:47)
at org.apache.cxf.phase.PhaseInterceptorChain.doIntercept(PhaseInterceptorChain.java:308)
at org.apache.cxf.endpoint.ClientImpl.doInvoke(ClientImpl.java:530)
at org.apache.cxf.endpoint.ClientImpl.invoke(ClientImpl.java:441)
at org.apache.cxf.endpoint.ClientImpl.invoke(ClientImpl.java:356)
at org.apache.cxf.endpoint.ClientImpl.invoke(ClientImpl.java:314)
at org.apache.cxf.frontend.ClientProxy.invokeSync(ClientProxy.java:96)
at org.apache.cxf.jaxws.JaxWsClientProxy.invoke(JaxWsClientProxy.java:140)
... 2 more
I have two AppEngine modules, a default module running Python and "java" module running Java. I'm accessing the Java module from the default module using urlfetch. According to the AppEngine docs (cloud.google.com/appengine/docs/java/appidentity), I can verify in the Java module that the request originates from a module in the same app by checking the X-Appengine-Inbound-Appid header.
However, this header is not being set (in a production deployment). I use urlfetch in the Python module as follows:
hostname = modules.get_hostname(module="java")
hostname = hostname.replace('.', '-dot-', 2)
url = "http://%s/%s" % (hostname, "_ah/api/...")
result = urlfetch.fetch(url=url, follow_redirects=False, method=urlfetch.GET)
Note that I'm using the notation:
<version>-dot-<module>-dot-<app>.appspot.com
rather than the notation:
<version>.<module>.<app>.appspot.com
which for some reason results in a 404 response.
In the Java module I'm running a servlet filter which looks at all the request headers as follows:
Enumeration<String> headerNames = httpRequest.getHeaderNames();
while (headerNames.hasMoreElements()) {
String headerName = headerNames.nextElement();
String headerValue = httpRequest.getHeader(headerName);
mLog.info("Header: " + headerName + " = " + headerValue);
}
AppEngine does set some headers, e.g. X-AppEngine-Country. But the X-Appengine-Inbound-Appid header is not set.
Why am I'm not seeing the documented behaviour? Any suggestions would be much appreciated.
Have a look at what I've been answered on Google groups, which led to an issue opened on the public issue tracker.
As suggested in the answer I received you can follow, for any update, the issue over there.
from http://httpd.apache.org/docs/current/mod/mod_headers.html
"Conditionally send MyHeader on the response if and only if header MyRequestHeader is present on the request. This is useful for constructing headers in response to some client stimulus. Note that this example requires the services of the mod_setenvif module."
SetEnvIf MyRequestHeader myvalue ENV_SET
Header set MyHeader "%D %t mytext" env=ENV_SET
but then from Apache SetEnvIf trouble and from my testing trying to set an environment var by checking the Authorization header, the ENV_SET won't be set at all
SetEnvIf only sets the env variable ENV_SET for the following
* Remote_Host
* Remote_Addr
* Server_Addr
* Request_Method
* Request_Protocol
* Request_URI
Is the documentation wrong or am I misunderstanding something ?
Using: Apache/2.2.15 (Win32) mod_fastcgi/2.4.6 mod_jk/1.2.30
Format is:
SetEnvIf <headername> <regex> <environment variable name and optionally a value>
This worked for me:
SetEnvIf ACTUAL_CLIENT_IP "^172\.111\.0\.27" dontlog
where ACTUAL_CLIENT_IP is the name of the HTTP header which stores the actual client IP (as opposed to the Remote_Addr field which gets overwritten with our proxy server's IP).