Hacker School: day 208.

(My batch ended on August 22, 2013, but as they say, never graduate.)

When I was reverse engineering the Snapchat API, I spent a fair amount of time wondering if there was a quick way to prototype HTTP requests. After complaining to a few friends, it turns out that there is: the telnet utility[1] on unix systems.

Goal: Let’s see how we can use telnet to issue an HTTP request that is equivalent to typing www.google.com into your address bar and pressing enter.

Aside: Before we begin, it’s worth noting that different parts of the URL are used by the server and the client. If we were accessing www.google.com/somepage.html, then www.google.com is used to figure out which server to send the request to, while /somepage.html is telling the Google server which page to serve up. How this works specifically is outside the scope of this article, but knowing that it works might help you understand this article.

Let’s start by opening a request to the Google servers. The form of request for opening a connection using telnet is:

$ telnet <hostname> <port>

The word host is just network-speak a “machine that exists on a network”. A hostname is something that identifies a machine on a network — perhaps a URL or an IP address (see my previous exploration of IP addresses here). In this case, the network is the global IP Internet, and the hostname could be something like www.google.com.

The last part of the command involves supplying a port. A port is just a 16-bit number that is used to help uniquely identify connections to some server — I won’t get into why this helps now, but it turns out that connecting to a server to do HTTP requests usually involves connecting to port 80, which is what we will do here.

So let’s connect to the Google server using telnet.

$ telnet www.google.com 80
Trying 74.125.28.105...
Connected to www.google.com.
Escape character is '^]'.
_

From here we are free to type an HTTP request in plaintext, which will get sent to the server. In the following example, I’ll grey out the old stuff:

$ telnet www.google.com 80
Trying 74.125.28.105...
Connected to www.google.com.
Escape character is '^]'.
GET / HTTP/1.1
Host: www.google.com

_

An HTTP request consists of three things:

• a request line, like GET / HTTP/1.1
• zero or more request headers of the form <header name>: <header data>; here Host: www.google.com is an example of a request header. There are other types of header other than Host, but we don’t really care about them right now.
• an empty line signifying that the request has ended.

The request line GET / HTTP/1.1 has a general form <request type> <uri> <HTTP version>.

The request type in this case is a GET, which just tells the server to generate and return the resource at the <uri>. There are also other request types, like POST, but right now we just care about fetching webpages from URLs.

The URI is the resource we’d like to retrieve from the server. In our case it’s /, the root URL, or the thing that you see any time you visit Google. It could have been /search?q=cows, which is the string we would enter to request that Google search the Internet for the word “cows”.

The HTTP version is the version of HTTP we’re using. You should use version 1.1, but old applications use 1.0.

After you type in that last empty line, the server will respond:

$ telnet www.google.com 80
Trying 74.125.28.105...
Connected to www.google.com.
Escape character is '^]'.
GET / HTTP/1.1
Host: www.google.com

HTTP/1.1 200 OK
Date: Sun, 29 Dec 2013 22:02:07 GMT
Expires: -1
Cache-Control: private, max-age=0
Content-Type: text/html; charset=ISO-8859-1
Set-Cookie: [LONG LINE HERE]
Set-Cookie: [LONG LINE HERE]
P3P: [LONG LINE HERE]
Server: gws
X-XSS-Protection: 1; mode=block
X-Frame-Options: SAMEORIGIN
Alternate-Protocol: 80:quic
Transfer-Encoding: chunked

8000
[A TON OF HTML/JAVASCRIPT HERE]
_

An HTTP response looks remarkably like an HTTP request! They consist of:

• a response line, like HTTP/1.1 200 OK
• zero or more response headers, of the form <header name>: <header data> (i.e., exactly like the request header)
• an empty line terminating the header
• and a response body which contains all the HTML, JavaScript, etc.

A response line is always of the form <HTTP version> <status code> <status message>. In this case we see that the server has responded in HTTP version 1.1, with status code 200, and status message “OK”. Code 200 indicates that the request has executed correctly, and the message “OK” tells us the same in English. There are many other codes with many different meanings, but those are for another post.

And with that, you’ve successfully executed a GET request to Google.

The cursor is still blinking at the end, though, because you are free to continue typing requests in.

Footnotes

[1] Note that there is also a Telnet protocol. But in this article we’ll mainly talk about telnet the unix utility.

comments powered by Disqus