What happens when you search for a kitten (on the Internet). Part 1: DNS or where to find the server
There are plenty of things we take in life for granted. Hot showers, frozen pizza, the Internet.
In this series of articles I would like to give appreciation to something we do almost unconsciously hundreds of times per day, rarely admiring the complexity which happens behind the scenes. Or, should I say, behind our screens?
I’m talking about what happens when you use your browser to search or open a URL.
Let’s go on a journey into the deep woods. We’ll look at the tricks up our browser’s sleeve, we’ll check numerous spots while searching for the IP address. Once found, we’ll initiate TCP and TLS handshake, ask the server for a page, break response into multiple packets and deliver them to the browser.
We might need to travel overseas, so be ready for a journey of a thousand miles.
All of that, just to see a picture of a kitten ;)
Let’s take www.petfinder.com as an example, not only it has some nice cat pictures, but any of those animals can become your friends if you adopt them!
Part 1: DNS or where to find the server
In this part we’ll talk about our browser and how it finds the address of the server, from where we want to receive data.
If you’d like to skip this part, you can see what happens next in Part 2: TCP or how not to lose the data on the road and Part 3: TLS or how to protect your secrets.
Browser, the window into the world wide web
Web browsers hold a proud position to be the most common window we use to look into the world wide web. Over the past four decades browsers evolved dramatically taking new responsibilities, being more secure and reliable.
When you start typing in the address bar, your browser will try to understand if you type a URL or a search query. In case of a search query it will be converted into a URL to query your default search provider. According to statistics over 90% of us are using Google as a default search provider.
Let’s talk about our privacy
When we type www.petfinder.com and pressed enter, the URL might still be incomplete. If we open this domain for the first time, our browser might still not know which protocol to use. In other words, which system of rules and agreements will be used in the communication between the client and the web server.
Our browser will need to choose between HTTP, which is an unencrypted connection on port 80, and HTTPS, which is encrypted and uses port 443. When using standard HTTP the information is sent in clear text and can be intercepted and manipulated. Which is a bad idea, especially for sensitive information, which you’d rather not share with everyone on the Internet.
You can think of HTTP vs HTTPS, as postcard vs an envelope.
Which protocol will the browser choose? The decision will depend on several factors. First, it is possible that the browser already has information about required protocol in its records. But let’s assume it doesn’t and we open this website for the first time.
Second, the domain might enforce secure connection. For example, if the domain is in the HTTP Strict Transport Security (HSTS) preload list, this would tell the browser that HTTPS should be enforced.
You can check if a domain is in the HSTS preload list over here — https://hstspreload.org. For example, the website we wanted to visit, www.petfinder.com, is not.
However, if you open it, you will still be redirected to HTTPS. This happens because the server will not allow unencrypted connection and will redirect us to HTTPS by returning a message with a status code 301 “Moved Permanently”.
Not that long ago browsers have started promoting secure connection, and now when for some reason you open a website which uses HTTP protocol, they will show a warning that the connection is not secure.
This might not pose an immediate danger but I would still avoid using such resources as much as possible. They indicate a neglect to security and privacy. Be aware that when you use HTTP, anyone can observe what you do and what data you send. And there are plenty of people on the Internet who would not miss such an opportunity.
Finding the address
We have our URL. However our browser still doesn’t know where to go. The problem is, the URL itself doesn’t give enough information. It’s like I’d tell you “Let’s meet in Impala Coffee shop” and unless you figure out the exact address (thank you, Google maps), most probably I’ll be drinking coffee alone.
What our browser needs is a specific address of the location where www.petfinder.com server lives. In other words, it needs an IP address!
“Read the directions and directly you will be directed in the right direction.”
― Lewis Carroll
DNS, the Yellow Pages of the web
In order to translate the URL of a resource into an IP address, we use Domain Name System or DNS for short. It is like a phone book, just hierarchical and decentralised… and with an amnesia on top of it to help forgetting old entries. So I won’t blame you if it’s difficult to imagine yellow pages of such sort.
Our little friend browser uses its portion of DNS as well, it remembers recently accessed domains and will keep this information in its records for some limited time. That’s why if you opened www.petfinder.com half an hour ago, most probably there will be a record in the browser’s cache. Those records have an expiration date (defined in seconds and called Time To Live, or TTL for short). Once the entry is expired the browser will no longer use it, and will request up-to-date information.
For example, you can check the list of cached domains in Firefox by entering about:networking#dns in your address bar.
Assuming that our website domain is not in the browser cache. Next stop is checking if our operating system (OS) has a record in its cache or if a domain name can be found in the hosts file.
By the way, you can see how long DNS cache will still live before expiring by running this command in your terminal:
dig +nocmd +noall +answer www.petfinder.com
If none of those locations contains what we are looking for, it is time to talk to the neighbours. And I mean, the network neighbours.
Talking to the neighbours
There are one or two authoritative name servers in the word wide web, that can specifically say the IP address of the domain we’re looking for. But contacting them directly is not what our browser would normally do. First of all, we don’t know where authoritative name servers live (their IP addresses) and secondly, it would be a waste of time and energy relying only on a couple of these servers for every DNS lookup.
That’s why at first we’ll check if any of the closer neighbours on the network already cached the information.
Stop #0: Resolver
The first door our browser will be knocking on is the name server of your organisation or your Internet Service Provider (ISP) DNS server. They are defined in the DNS settings of your OS.
It is good to know that you don’t have to stick with the DNS server provided by your ISP. If you don’t trust them with your privacy, consider using a third party DNS server. It might be faster too. Among the available options are CloudFlare’s DNS, OpenDNS, from Cisco and the most common Google’s Public DNS.
Another thing to note is that there are several different types of DNS servers. For us, at this moment, it is important that the DNS server we’re talking to not only can respond to us with information from its cache, but also, it can send requests to other instances to get information on our behalf. That’s why it is called DNS resolver (or DNS recursor).
It is quite probable that information about the domain name will be already in the cache of our DNS resolver. But for the sake of our journey, let’s imagine it is not and the resolver will accept responsibility to track down the IP address we’re looking for.
Stop #1: Root name server
What will the resolver do? It will ask for more information from one of the root name servers.
You’d be surprised that there are only thirteen public root name servers around the world. If wiki to be trusted, it is because of limitations in the original DNS specification. Nevertheless, you can implement your private root server within a local implementation of DNS, and beat that limit, if you want to.
A root name server is like a village headman. He might not know the exact answer to your question, but knows who to point you to. In our case, the root name server will tell our resolver the address of the top-level domain (TLD) of the domain we’re looking for, the parent of petfinder.com so to say. With this information we get one step closer to finding our destination address.
Stop #2: Top-level domain
With the new information the resolver can send a request to the TLD DNS to clarify where petfinder.com lives. TLD will return a message with the IP address of an authoritative name server.
Stop #3. Authoritative name server
And finally resolver asks the authoritative name server for petfinder’s IP address and returns this information to the web browser.
Congratulations, we have the destination address of www.petfinder.com and ready to establish a first contact.
Now that we have the address of the server, you might think that it is the time to send a request for a page. But no, we still need to do a couple of important things to ensure the reliability and security of our connection.
You can find the next sections here: Part 2: TCP or how not to lose the data on the road and Part 3: TLS or how to protect your secrets.
