I must develop proxy server that work with only HTTP 1.0 in Linux and by c .
I need some hint to start developing .
I assume you are confident in using linux and the language c (no hints for that, else don't start with developing a proxy)
Read and understand the RFC 1945 HTTP/1.0 (pay attention to the specific mentioning of proxy)
Determine what kind of proxy you want (web/caching/content-filter/anonymizer/transparent/non-transparent/reverse/gateway/tunnel/...)
Start developing the server
Basic steps
Open port
Listen on port
Get all request sent from the client to that port (maybe make the whole thing multithreaded to be able to handle more than 1 request at a time)
Determine if it is a valid HTTP 1.0 request
Extract the request components
Rebuild the request according to what type of proxy you are
Send the new request
Get the response
Send response to client
How to create a proxy server:
Open a port to listen on
Catch all incoming requests on that report
Determine the web address requested
Open a connection to the host and forward the request
Receive response
Send the response back to the requesting client
Additionally: Use threads to allow for multiple requests to the server.
Related
I am testing my proxy that simply sends a client's request to a proxied server and returns a response back. The current realization requires that the client must send the fully prepared valid request to the proxy (the Host header value must match with a DNS of the predefined in source code proxied server).
Heres my custom request to the proxy that proxies a www.example.com:
But the result request that ARC sends to the localhost is:
GET / HTTP/1.1
Host: localhost:1234
connection: close
then it is sent to the www.example.com but the Host header is invalid for it so 404 is returned as a result.
I just noticed that this refers to the old version of ARC for Chrome. Support for Chrome apps is scheduled to end soon the the app is no longer supported. Instead, please, install desktop client from https://install.advancedrestclient.com/
To move your data from one app to another follow instructions from https://docs.advancedrestclient.com/moving-from-chrome-application-to-desktop-client
I am currently working on a multi threaded proxy server that supports keep-alive connections. I see some weird issues while handling requests from firefox browser. I connect to my local proxy using localhost:10001/http://url, and I can access all the links on this host. The process is as below.
1. Create a socket bind it to port 10001
2. Accept connections and if a client is connected fork()
3. Keep on processing the client request as persistent connection.
Now the problem is that when I open a new tab in firefox to access a second url with different host with using localhost:10001/http://url2, the strange thing is that that request goes to my client socket connection created during first connection. I initially thought that it might be due to my code, but then i tried to do the same using telnet and all the new connections would create a separate process. Are there any specific settings that is making firefox browser do this??
HTTP keep-alive is a way to reuse an underlying TCP connection for multiple requests so that one can skip the overhead of creating a new TCP connection all the time. Since the target of the connection is the same all the time in your case it makes sense for the browser to reuse the same TCP connection. The comparison with telnet is flawed since with telnet you do a new TCP connection all the time.
If HTTP keep-alive gets used is specified by the HTTP version the Connection header and on the behavior of both server and client. Both server and client can decide to close the idle connection any time after a request was done, i.e. they are not required to keep it open after the request is done. Additionally they can signal that they like to have the connection open by using the Connection: keep-alive HTTP header or that they like to close after the request with Connection: close. These headers have default values depending on the HTTP version, i.e. keep-alive is on with HTTP/1.1 while off with HTTP/1.0 unless explicitly specified.
Apart from that the "proxy" you are implementing with the use of URL's like http://proxy/real-url is not a real HTTP proxy. A real HTTP proxy would be configured as a proxy inside the browser and the URL's you use would stay the same which also means that no URL rewriting would need to be done by the proxy. Worse is that your idea of a proxy effectively merges all hosts inside the same origin (i.e. origin is the proxy) and thus effectively disables a major security concept of the browser: the same-origin policy. This means for example that some rogue advertisement server would share with your implementation the origin with ebay and thus could get access to the ebay cookies and hijack the session and misuse it for identity theft
HTTP persistent connection is also used with the proxy, not only with the destination.
For firefox you could try to alter the behavior with the proxy by setting network.http.proxy.version to 1.0. But you'll have to enhance your proxy (and perhaps rethink completely its inner workings) to be able to deal with these reused connections. I'm sure it's not limited to Firefox.
Also make sure your proxy doesn't answer with HTTP/1.1 because it's not.
I am currently designing a web application using AngularJS. In this I am fetching and posting data via Rest API(s) with different methods. The data I retrieving is fetched in the form of JSON.
Problem:
Issue here is, while I am using https, the data sent and received via HTTP requests can still be seen in proxy tool or traffic monitors. All the JSON can be easily read from this.
Each of my request has a token attached in it's header which takes care of authentication. However, once authorized, there is some part I don't want to be displayed in/ caught in such monitoring tools.
Question:
This data is stored in an encrypted way in database and all, however while coming via HTTP request, it is first decrypted and then sent. How can I hide/protect this data?
You can't.
If you give it to the client, then the client has to be able to see it.
If the user has configured their browser to proxy requests, then the proxy is the client.
Once the data leaves your server in an HTTP response then anyone/anything thing the user of the client wants to trust with that data can access it. You don't have control at that point.
proxy tool or traffic monitors will see https data only if the client has accepted the man-in-the-middle (MITM) by installing the ssl certificate used by the MITM:
To see the content (other than the host name) of an https connection, someone who is neither the client or the server must do a MITM.
If someone do a MITM with a certificate not trusted by the client, the client will reject the connection.
WARNING: If the server do NOT use HSTS, the person doing the MITM can do an SSLSTRIP attack if the first connection is http. In that case, the MITM do not need a trusted certificate because the connection will stay in plain text (http)
I'm working on an embedded application (running MQX RTOS, written in C) which has SMTP functionality. Recently, TLS support was added using the Mocana NanoSSL library. I'm currently able to successfully send emails using Gmail, Yahoo, and private exchange servers. Unfortunately, Hotmail does not work. Here's the connection parameters i've used:
Server: smtp.live.com
Port: 25 and 587
AUTH method: PLAIN and LOGIN
Basically, i'm able to successfully connect to the server, perform the SSL/TLS handshake (using STARTTLS), and send the encrypted EHLO message to the server (receiving a response). According to this response, the server supports both AUTH PLAIN and AUTH LOGIN. However, once I send either of these commands, the following SSL_recv() call I make to get the response fails with either a timeout or connection reset by peer.
UPDATE:
OK, so after some experimentation it would appear that my issue lies at the SSL library level and not with Microsoft's SMTP server. I tried replacing the SSL_recv() calls with standard RTCS socket recv() calls and was able to receive and view encrypted data. By disabling my response verification, I was then able to continue through the SMTP process and successfully send a message. At this time i'm not sure why the SSL_recv() calls are unable to get the socket data, but i'll keep digging and will hopefully find an answer.
Well, I also got it working here too. I had to replace the
ssl_ctx=SSL_CTX_new(SSLv23_client_method());
with either:
ssl_ctx=SSL_CTX_new(SSLv3_client_method());
or
ssl_ctx=SSL_CTX_new(TLSv1_client_method());
My understanding is that the 23_client method sends a SSL2 client hello first and this confuses the server.
I read this in the HP SSL programming tutorial:
http://h71000.www7.hp.com/doc/83final/ba554_90007/ch04s03.html
it says: "However, the SSL client using the SSLv23 method cannot establish connection with the SSL server with the SSLv3/TLSv1 method because SSLv2 hello message is sent by the client."
SSL3 works too since you can continue after STARTTLS with SSL, you do not have to use TLS.
See here:
https://www.fastmail.fm/help/technology_ssl_vs_tls_starttls.html
So, it's the SSL library itself that appears to be failing me here. I was able to bypass the issue and successfully send email by simply not calling SSL_recv() to verify the server responses. I'm obviously not able to error check or get any meaningful failure feedback, but for a successful use case where the server accepts all of my SMTP messages the email is sent.
I am working on a web proxy.The logic is client sends request to proxy, proxy sends the same request to server, and sends the answer back to the client.
For example, i want to visit www.baidu.com. I get "Host:www.baidu.com" in the GET: package, which is used to send a dns request, then i get the ip of "www.baidu.com", establish the socket between proxy and server.
The question is when I use wireshark to capture normal packages not with proxy, i find that there is more dns request queries visting "www.baidu.com" except query for www.baidu.com. It will query for nsclick.baidu.com and suggestion.baidu.com in different sockets.But there is no signal to let me to initiate these DNS queries, not like query for "www.baidu.com",in which i can initiate it when i detect "Host:". Can someone help me ? thank u.
This is not how this should be working probably in first place.
Imagine i hit www.baidu.com in my browser, which sends traffic via your proxy. For your proxy currently, www.baidu.com is the only thing to lookup for.
When my browser end up receiving html chunk for this request, received html/js code then loads requests for some images which comes from nsclick.baidu.com. Similarly requests for other resources (css, js, images) can be made. In turn they all again go through your proxy and then their you will be doing your usual dns query.