My issue is simple: I want to make an HTTP GET request from a server. My client program takes a URL and sends it to the server program. I want to know how I can take this URL and make an HTTP GET request for the specific page that was entered from the client.
If I resolve the URL to get an IP address, I could open a socket to that IP address, but then how would I make the request for the actual page? Would I just send an HTTP GET request directly to that IP address with the directory in the request?
For instance, if I ran a Google search for the word "Test" I'd be presented with the following URL:
https://www.google.com/?gws_rd=ssl#q=Test
My understanding is that a GET request could look like this:
GET /?gws_rd=ssl#q=Test HTTP/1.1
Host: www.google.com
So, if I'm understanding this correctly, would I resolve the IP, open a socket, and then just send this GET request directly to the socket?
Lastly, if I try throwing a URL such as the one above into my server code it's unable to resolve an IP address. This means that if I'm making a more complex request than something like www.google.com I'd have to parse the string and match only the host. Is there a simple way to handle this other than by the use of regular expressions? I'm familiar with regular expressions from Python and C#, but if I can cut down on the complexity of the program by approaching this differently, I'd like to know.
Update: I'm attempting to match the URL with a POSIX regular expression and extract the domain name from it. I'm not having much luck so far as this implementation is oppressively confusing.
Yes, once a socket has been opened you could send requests as in your example and described in RFC 2616.
If you don't want to use regular expressions or strchr to split your URL you cold also send the entire URL:
`GET http://www.google.com/?gws_rd=ssl#q=Test HTTP/1.1
`
However, you will still need to find the hostname in the URL to make a call to something like gethostbyname.
Related
I want to get the all http requests which are going through proxy. And i need to show them in an output file
Actually my proxy server is storing all the requests in STDIN. So now i should get the data/header requests from that STDIN. Is that possible to get? if possible please let me know how to proceed
I have a basic packet sniffer like http://www.binarytides.com/packet-sniffer-code-c-linux/
I have extended it to process packets only on port 80 (HTTP). I am not sure how to get host web address from data. Can you guys help me here
What I am trying to do is parse HTTP header subset in order to identify host web address
I found something similar to what I need : https://github.com/joyent/http-parser/blob/master/http_parser.h#L194
but the code is too complex...
Or where can I find HTTP header bytewise breakdown like for TCP http://en.wikipedia.org/wiki/Transmission_Control_Protocol#TCP_segment_structure
You need to grab the tcp data, then look for "GET". A typical http request looks like:
GET www.foo.com HTTP/1.0
web host name just follows the GET request. So you can extract the web host address from there.
I'm trying to get an arduino to login in to a website that I have created.
On the website there is a basic form that has two fields one for password and one for username, it also has a submit button labelled login.
I used fiddler2 to sniff the http packets when I login in using chrome and am trying to use the information from that http post to recreate my own post to login.
Here is the portion of the code that I am using for the login:
if (client.connect(server, 80)) {
Serial.println("connected to server");
// Make a HTTP request:
client.println("POST/username=slwhore&passwd=1234qwer%21&op2=login&lang=english&force_session=1&return=B%3AaHR0cDovL3JlbW90bGV0LmNvbS8%3D&message=0&loginfrom=loginmodule&cbsecuritym3=cbm_56b7d5e7_00583e07_b0b6f81b4c86d117542f5cc7b7c3416e&Submit=Login HTTP/1.1");
client.println("Host:www.remotlet.com");
client.println("Content-Type: application/x-www-form-urlencoded");
client.println("Content-Length: 229");
client.println("Connection: close");
client.println();
I then have another piece of code that recieves the information coming back from the host which I know works. When I run this code I am able to connect to the server but I don't get any response form it at all. Any help would be greatly appreciated.
Your HTTP request is completely wrong, it will never ever be accepted by any kind of server web.
1st line: the HTTP 1st line is METHOD URI VERSION.
You didn't put a space between the method and the URI, also the POST data is not part of the URI as it is when using GET requests. I don't know what your server uses but usually sane logins don't use GET and don't pass the login inside the URI.
2nd line: you forgot a space
4th line: you set a content length but you don't send any content apparently.
General consideration: in HTTP the line terminator is \r\n, not just \n.
I suggest you do the request with the browser, intercept the traffic with wireshark and see how it's done.
I'm working on a C coded server that have to reply to browsers' requests. It have to give authentication when using url like this:
http://user:pass#website/
but I really don't know how or where get this information on my server, because what I got when I read the request is the same that I can read when I interact with the server simply using
http://website/
Second question is that sometime I have this favicon.ico request from browsers.. what can I reply to the browser to say "I have not this fu*** stupid icon"? :D
I'm of course using socket for this
Look for a request header named Authorization: containing the string Basic followed by the BASE64 encoded username and password. This method of authenticating is called HTTP Basic Authentication.
For the favicon, simply respond with a HTTP 404 response if you don't have one.
I am working on a web proxy.The logic is client sends request to proxy, proxy sends the same request to server, and sends the answer back to the client.
For example, i want to visit www.baidu.com. I get "Host:www.baidu.com" in the GET: package, which is used to send a dns request, then i get the ip of "www.baidu.com", establish the socket between proxy and server.
The question is when I use wireshark to capture normal packages not with proxy, i find that there is more dns request queries visting "www.baidu.com" except query for www.baidu.com. It will query for nsclick.baidu.com and suggestion.baidu.com in different sockets.But there is no signal to let me to initiate these DNS queries, not like query for "www.baidu.com",in which i can initiate it when i detect "Host:". Can someone help me ? thank u.
This is not how this should be working probably in first place.
Imagine i hit www.baidu.com in my browser, which sends traffic via your proxy. For your proxy currently, www.baidu.com is the only thing to lookup for.
When my browser end up receiving html chunk for this request, received html/js code then loads requests for some images which comes from nsclick.baidu.com. Similarly requests for other resources (css, js, images) can be made. In turn they all again go through your proxy and then their you will be doing your usual dns query.