I am working on a web proxy.The logic is client sends request to proxy, proxy sends the same request to server, and sends the answer back to the client.
For example, i want to visit www.baidu.com. I get "Host:www.baidu.com" in the GET: package, which is used to send a dns request, then i get the ip of "www.baidu.com", establish the socket between proxy and server.
The question is when I use wireshark to capture normal packages not with proxy, i find that there is more dns request queries visting "www.baidu.com" except query for www.baidu.com. It will query for nsclick.baidu.com and suggestion.baidu.com in different sockets.But there is no signal to let me to initiate these DNS queries, not like query for "www.baidu.com",in which i can initiate it when i detect "Host:". Can someone help me ? thank u.
This is not how this should be working probably in first place.
Imagine i hit www.baidu.com in my browser, which sends traffic via your proxy. For your proxy currently, www.baidu.com is the only thing to lookup for.
When my browser end up receiving html chunk for this request, received html/js code then loads requests for some images which comes from nsclick.baidu.com. Similarly requests for other resources (css, js, images) can be made. In turn they all again go through your proxy and then their you will be doing your usual dns query.
Related
I found an interesting case in one of the react-scripts apps where the proxy is configured to localhost:3001 (the front end is running on localhost:3000).
From the react stuff we make a request via axios to localhost:3000/api/results and that loads a bunch of JSON information, but if I open localhost:3000/api/results in a new browser tab that does not display the JSON but loads the HTML instead.
Why is that happening?
The real problem is that we have endpoint to download files from, like:localhost:3001/api/downloads/csv/file.csv, but they won't work, because when localhost:3000/api/downloads/csv/file.csv is not proxied to localhost:3001/api/downloads/csv/file.csv and we simply cannot call this via axios because it should be a direct call from the browser.
However, the strange thing is why does it work via axios and curl?
By doing curl localhost:3000/api/downloads/csv/file.csv (or 3001), we get the right content back.
If you are using react-scripts, then this is facilitated by Service Workers. Quoting from Service Workers - By Ankita Masand...
Service Worker acts as a proxy server that intercepts the network requests sent by your web application to the server. In the sense, requests to fetch Javascript or CSS files, images go through service worker to the server. Service Worker has the ability to modify this request or send a custom response back to the client.
Here, Service Workers are acting as a Client Side Proxy and tapping to your HTTP Requests and you get the right content back.
A Java server exposes REST services using Apache CXF 3.1.10. Trying to call a GET service with a URL longer than 8K, the service gives error.
The REST server uses JAXRSServerFactoryBean that launch a Jetty server. I can not find a way to allow the server to accept request of more than 8K.
Get requests have a query size limit, both on client and server side. (check this for details: maximum length of HTTP GET request?)
Maybe you should move to POST services. Or if you control both the client and server, you may use the request body. (That is allowed for GET requests but there are some clients/servers not supporting that)
since i was struggling in making API calls to apache server from my angular app running in node-express,
So i was unable to call apache server with POST calls inspite of setting the CORS filter in most of the ways available,
So someone suggested rather of making calls from AngularJs(Frontend) , make it from NodeJs(Backend-server) which serves your angulas(frontEnd) code.
So kindly assisst me in this as to what exactly is the difference between making API call's from frontEnd to any server or from the backend(server) of the same frontEnd ??
What factors makes it more preferable over the other one ?
Is it proxy or CORS thing which effects FrontEnd based API calls ?
Thanking all in advance
Shohil Sethia
CORS is a policy that is voluntarily enforced by the browser (chrome, firefox, etc.). The decision to allow or deny a request is based on the presence of a certain header (Access-Control-Allow-Origin: *) in a response from the server. There is no equivalent policy in a server side setting, so you are free to make cross-origin requests all day.
From enable-cors.org:
[CORS] prevents JavaScript from making requests across domain boundaries
This is why I usually build a small server api in Node to grab data from external 3rd party servers.
When the user makes a request on the front end the request is sent to the backend function with optional parameters which the end-user specified.
Depending on the parameters supplied, different functions might be run before the backend queries the third party API.
3rd party API response is returned to the backend.
Backend either passes the response along or does more stuff before passing the response along.
Then the frontend does stuff with the data based on the response received (ie there were less than 5 results so adding pagination is not necessary).
If developed this way you gain access to the following which all benefit your application/website.
Keep any necessary credentials on the server. ( extremely important )
Obtain logs.
Validate on both the server side and the client side for an added layer of security.
Use the server to filter sensitive results if necessary before they reach the frontend.
Vary which parts of the heavy lifting are done on the server vs the device in order to improve the application performance.
I am currently designing a web application using AngularJS. In this I am fetching and posting data via Rest API(s) with different methods. The data I retrieving is fetched in the form of JSON.
Problem:
Issue here is, while I am using https, the data sent and received via HTTP requests can still be seen in proxy tool or traffic monitors. All the JSON can be easily read from this.
Each of my request has a token attached in it's header which takes care of authentication. However, once authorized, there is some part I don't want to be displayed in/ caught in such monitoring tools.
Question:
This data is stored in an encrypted way in database and all, however while coming via HTTP request, it is first decrypted and then sent. How can I hide/protect this data?
You can't.
If you give it to the client, then the client has to be able to see it.
If the user has configured their browser to proxy requests, then the proxy is the client.
Once the data leaves your server in an HTTP response then anyone/anything thing the user of the client wants to trust with that data can access it. You don't have control at that point.
proxy tool or traffic monitors will see https data only if the client has accepted the man-in-the-middle (MITM) by installing the ssl certificate used by the MITM:
To see the content (other than the host name) of an https connection, someone who is neither the client or the server must do a MITM.
If someone do a MITM with a certificate not trusted by the client, the client will reject the connection.
WARNING: If the server do NOT use HSTS, the person doing the MITM can do an SSLSTRIP attack if the first connection is http. In that case, the MITM do not need a trusted certificate because the connection will stay in plain text (http)
I developed an AngularJS and I have a issue I can't fix, in a desktop the app works fine and in a mobile with Wi-Fi connection too.
But when I use it in a mobile with 3G connection it fails 'cause it doesn't do a GET request to the server, I've thought it was the cache but I disabled it in app config ($http.defaults.cache = false) and it doesn't work.
I have three nested request, one POST and two GET in order to refresh the shown data. The request in what I have the issue is the first GET, it doesn´t send the request to the server but treat the response as it had returned 200 OK.
Example code:
sendData()
.then(refreshFirstData) (problem here)
.then(refreshSecondData);
Does anyone know what can be happening?
Thanks.
EDIT
I could get a capture from the request when using 3G connection and it loads the data from cache... I think I disabled this option..
SOLUTION
I found the solution in nginx, I needed to add 'expires -1' to the config of my site.