How DNS actually works
Nothing in DNS propagates. A name is resolved by walking down from the root, and every answer is a cached copy with an expiry date you set yourself.
Assumes you know
- That a server somewhere has an IP address
- What an HTTP request is
Ask a public DNS server for wikipedia.org twice, three seconds apart, and both
answers carry the same address and a different number beside it:
103.102.166.224 ttl 23
103.102.166.224 ttl 20Nothing about Wikipedia changed in those three seconds. The number is not part of the address and it is not a measurement of anything. It is how much longer the server you asked is willing to hand out the copy it is already holding. When it reaches zero, that copy is thrown away and the next person to ask causes a real lookup.
Why DNS is shaped like this
Build it up
/etc/hosts
104.21.14.225 techdecoded.dev
93.184.215.14 example.comWithout this
Every new host means somebody edits a file on every other computer, by hand.
This is not a thought experiment. The file still exists on your machine.
you -------------> [ THE server ]
every name
one operatorWithout this
One machine answers for every name on earth, and every new name needs that one operator to agree.
[ root ] knows: who runs .dev
|
[ .dev ] knows: who runs your domain
|
[ your NS ] knows: the addressWithout this
Nobody holds the full list any more, so one question is not enough. You get sent onward instead of answered.
Each level knows one thing: who to ask next.
[ root ] asked once a day
|
[ .dev ] asked once an hour
|
[ your NS ] asked when a TTL runs out
|
[ a cache ] <-- answers almost everythingWithout this
Every page load on earth walks back to the root, and the root melts.
Every DNS problem you will debug is a cache somewhere still holding a copy.
1 / 4
/etc/hosts is still on your machine, it still works, and it still wins: if a name is
in that file, your computer never asks anyone. That is useful when you want to point a domain at your
own laptop, and it is a genuinely nasty thing to forget about when a colleague
cannot reproduce what you are seeing.
The root servers have never heard of your domain and are not supposed to. That means there is no central copy to publish, no master database to synchronise, and nowhere for a change to be pushed out from.
One lookup, run against the real servers
Every message below was captured from the real servers, starting at a root server’s address.
Sequence
1 / 6
The thing doing the walking is called a The server that does the work of a lookup for you: it asks the root, follows the referrals, and hands you back a single final answer. Usually your ISP’s, your company’s, or a public one such as 1.1.1.1 or 8.8.8.8., and it is almost never your own machine. Your laptop asks one server a single question and gets one answer, which is why a lookup looks atomic from where you are standing.
Only the last server in that chain is Holding the real records for a name rather than a copy of somebody else’s answer. An authoritative nameserver answers from the zone it was given, and it is the only place a change can be made.. The records it holds for one domain, managed together as a unit, are that domain’s Every record for a domain, kept and served as one set: the bare domain, the subdomains, the mail records, all of it. It is what a DNS provider’s control panel is editing..
The root’s referral is good for 172,800 seconds, which is two days. The registry’s is good for three hours. The address itself is good for five minutes. Parts of the chain that rarely change are cached for a long time, and the part you edit is cached briefly, which is a choice somebody made rather than a property of the network.
There are 13 root server addresses, lettered a through m, operated by 12 organisations, and a little over 2,000 server instances answer on those 13 addresses. “The root” is a set of addresses, not a set of computers.
What a record actually is
Anatomy
One A record, as the authoritative server returned it
NameThe trailing dot is the root of the tree. Every name in DNS really ends with it; your browser and your resolver let you leave it off.
TTLSeconds. How long anybody who receives this record may keep using it. The zone owner picks the number and every cache downstream obeys it.
ClassShort for Internet. There were other classes in 1987 and you will never meet one. It appears in every record and tells you nothing.
TypeWhat kind of thing the value is. A means an IPv4 address.
ValueThe payload, and its shape depends entirely on the type. For MX it would be a priority and a mail server; for TXT, arbitrary text.
A record sitting at a TTL of 86,400 has already committed you to a day of waiting before you have decided to change anything, and there is nothing you can do at the moment of the change to shorten it. You pick that number long before you need it, usually without noticing you are making a decision at all.
The record types you will actually meet
| Type | Value looks like | What it is for |
|---|---|---|
A | 104.21.14.225 | An IPv4 address for this name. |
AAAA | 2606:4700:3032::6815:ee1 | An IPv6 address for this name. |
CNAME | github.com. | This name is an alias. Go and look up that one instead. |
MX | 10 inbound-smtp.us-east-1.amazonaws.com. | Where to deliver mail for this domain. The number is a priority, lowest first. |
TXT | "google-site-verification=GiDb..." | Arbitrary text. Used for domain ownership proofs and mail policy. |
NS | kyle.ns.cloudflare.com. | Which nameservers are authoritative for this name. |
A CNAME is the one with a rule attached. Look at what a real one returns:
www.github.com. 2417 IN CNAME github.com.
github.com. 40 IN A 20.207.73.82The resolver asked for an address, got told “that name is really this other name”, and went and resolved that instead. You get both records back because it did both lookups.
The rule is that a CNAME must be the only data at its name.
RFC 1034 puts it plainly: “If a
CNAME RR is present at a node, no other data should be present.” An alias is a
redirection of the entire name, so anything else sitting there would be
contradicted.
That collides with the The bare domain itself, example.com rather than www.example.com. It is the top of the zone you control, and it is required to carry SOA and NS records.. The apex has to carry an NS record, because that is how anybody finds your nameservers at all, and an SOA record. Both of those are data. So a CNAME cannot sit there, and every DNS provider’s control panel will refuse you.
Providers sell a way around it under names like ALIAS, ANAME or CNAME
flattening. You type a hostname; their nameserver resolves it itself and answers
queries with ordinary A and AAAA records. No resolver ever sees a CNAME at
your apex, so no rule is broken and your MX and TXT records keep working.
Cloudflare’s flattening
and Route 53’s alias records
are the two you are most likely to meet.
Check yourself
You want example.com (no www) to point at a CDN hostname like d1234.cloudfront.net. Your provider refuses to save a CNAME on the bare domain. Why?
Who is holding a copy
Stack
Browsers keep a short DNS cache of their own, separate from the operating system. This is the one people forget, and it is why a page can keep reaching an old address after you flushed everything else.
Your machine keeps recent answers so every program does not have to re-ask. On systemd machines, resolvectl flush-caches clears it.
The big one. It answers for thousands of people, so its copy is the one most likely to be stale for somebody who is not you, and you cannot clear it.
The only party here that is not holding somebody else's copy. Change the record here and it is true here immediately, and nowhere else.
You can clear the top two layers yourself and you cannot touch the third. That asymmetry is what makes “it works on my machine” so common here: flushing your browser and your operating system fixes your own view and changes nothing for anybody else, including the resolver you and your colleague share.
Check yourself
You change your site's A record at 14:00. The record's TTL was 3600. At 14:30 a colleague in another country still loads the old site. What is happening?
Changing a record without a wait
Since the wait is other people’s copies timing out, you can remove it by making those copies short-lived before you touch anything that matters.
Timeline
- 09:00
Lower the TTL to 60
3600 secondsChange nothing else. This is invisible to the world: caches are holding copies stamped 3600 and will honour that number.
- 10:00
The last old copy expires
3600 secondsOne full old TTL after the edit. Every cache has now refetched, and what they are holding says 60.
- 10:00today
Change the address
60 secondsThe only step that touches the thing you actually came to change.
- 10:01
Nobody is on the old address
60 secondsOne new TTL later. Not "most people". Nobody, because no copy older than 60 seconds exists.
- 11:00
Put the TTL back to 3600
3600 secondsA 60 second TTL means every cache re-asks your nameservers every minute. Fine for an afternoon, wasteful forever.
If you did not plan ahead, there is no button to press: you wait one old TTL. Anyone telling you to allow 24 to 48 hours is quoting folklore from an era of much longer default TTLs, and you can check the real figure by asking for the record and reading the number that comes back.
The traps
“Does not exist” is cached too. Ask for a name that is not there and you do not get silence, you get a refusal with an expiry on it, called “Non-existent domain”, the response code a nameserver returns when it is authoritative for the zone and the name is genuinely not in it. Distinct from a lookup that failed or timed out.:
nope-not-here.techdecoded.dev. NXDOMAIN
techdecoded.dev. 1800 IN SOA kyle.ns.cloudflare.com. dns.cloudflare.com.
2413665214 10000 2400 604800 1800That trailing 1800 in the SOA record is what caps how long resolvers may
remember the absence, a mechanism defined in
RFC 2308. So if you or anyone else
looked up a subdomain before you created it, the negative answer is sitting in a
cache and your brand new record will not be seen until it expires. This catches
people setting up certificate validation records, where the tooling checks for the
record, fails, and then keeps failing for half an hour after you have added it.
The TTL is per record, not per domain. On this site’s zone right now the A
record has a TTL of 300, the MX has 3600, and the NS has 86400. Changing where
mail goes and changing where the website points are two different waits.
/etc/hosts is not DNS and does not care about any of this. It is consulted
before a lookup happens, so a stale line in it will survive every cache flush and
every TTL you have ever set.
Check yourself
You look up api.example.com, get NXDOMAIN, then immediately create the record with a TTL of 60. You look it up again a minute later and still get NXDOMAIN. Why?
What DNS will not do for you
It does not route anything. Handing back an address is the end of the job. What happens to the packet afterwards is routing, and DNS has no involvement in it.
It does not know whether your server is up. A nameserver returns what the zone says, whether or not anything is listening at that address. Health checking is a product some managed DNS providers sell on top, where their systems probe your servers and edit the zone for you. Plain DNS has no concept of up or down.
It is a poor load balancer. You can put several addresses on one name, and clients end up spread across them, but caching means you cannot control which client gets which address or shift traffic away from one quickly when you need to.
It is not private and it is not authenticated. Classic DNS is unencrypted, usually over UDP port 53, so the names you look up are readable by anything on the path, and anyone positioned to reply before the real nameserver does can lie to you. DNS over HTTPS and DNS over TLS address the first problem, DNSSEC addresses the second, and both are separate topics. Worth knowing when reading how SSL actually works: HTTPS hides the page you asked for, but the DNS lookup that preceded it announced the hostname in the clear.
Checking your work
Ask your own machine what it resolves a name to, and how it got there:
resolvectl query techdecoded.devtechdecoded.dev: 104.21.14.225
172.67.204.137
2606:4700:3032::6815:ee1
2606:4700:3030::ac43:cc89
-- Information acquired via protocol DNS in 60.7ms.
-- Data is authenticated: no; Data was acquired via local or encrypted transport: no
-- Data from: networkThat gives you addresses but not the TTL. To see the countdown, ask a resolver over HTTPS, which returns the raw fields as JSON:
curl -sH 'accept: application/dns-json' \
'https://cloudflare-dns.com/dns-query?name=techdecoded.dev&type=A'{"Status":0,"TC":false,"RD":true,"RA":true,"AD":false,"CD":false,
"Question":[{"name":"techdecoded.dev","type":1}],
"Answer":[{"name":"techdecoded.dev","type":1,"TTL":300,"data":"104.21.14.225"},
{"name":"techdecoded.dev","type":1,"TTL":300,"data":"172.67.204.137"}]}The tool most guides reach for here is dig, which comes in the dnsutils package
on Debian and Ubuntu and bind-utils on Fedora and RHEL. It prints the same fields
in the classic zone-file layout, and dig +trace walks the chain from the root the
way the sequence above does.
Try it yourself
Watch a TTL run out
Ask for the same name twice in a row and compare the TTL field:
curl -sH 'accept: application/dns-json' \
'https://cloudflare-dns.com/dns-query?name=wikipedia.org&type=A'
# wait a few seconds, then run it againThen ask for a name that does not exist, and look at what comes back instead of an address:
curl -sH 'accept: application/dns-json' \
'https://cloudflare-dns.com/dns-query?name=nope-not-here.wikipedia.org&type=A'What you should see
The TTL in the second lookup is lower by roughly the number of seconds you
waited. You are watching one resolver’s copy age. If it jumps back up instead, you
reached a different machine behind the same address, holding its own copy on its
own clock.
The name that does not exist returns "Status": 3, which is NXDOMAIN, and an
Authority section containing an SOA record rather than an Answer section. The
last number in that SOA is how many seconds resolvers may remember that this name
is missing. That is the number standing between you and a record you are about to
create.
Where to go next
Point a domain at a CDN and you are making a DNS change and nothing else, which is worth reading alongside what a CDN actually does: the cache key section there is the same “who is holding a copy” reasoning one layer up the stack.
The specifications are unusually readable for their age. RFC 1034 is the concepts, published in November 1987 and still accurate. RFC 2308 is negative caching, and it is short.
The obvious next question is how you know an answer was not tampered with in transit, given that anyone who can reply faster than the real nameserver wins. That is DNSSEC, and it deserves its own article.
If you remember nothing else
- Nothing in DNS is pushed. Every answer is one somebody asked for, and the only reason lookups are fast is that whoever asked before you kept a copy.
- There is no propagation. When a change has not taken effect, the question is never where the record has got to; it is which cache is still holding an old copy and when its TTL runs out.
- A TTL is set by the zone owner and obeyed by everyone downstream, so a safe record change means lowering the TTL first, waiting one full old TTL, and only then changing the value.
- Most servers in a lookup do not know the answer and are not meant to. The root and the registry hand back referrals; only the authoritative nameserver was ever told the address.
- A CNAME must be the only record at its name, which is why the bare domain cannot have one, and why providers synthesise A records there instead.
Quick answers
- How long does DNS propagation take?
- There is no propagation, so there is nothing to wait for except caches expiring. A resolver holding a copy will keep serving it for exactly the TTL that copy was handed, then re-ask. If the record had a TTL of 3600 when it was cached, the longest anyone can be stuck on the old value is one hour, no matter what your DNS provider's control panel says about 24 or 48 hours.
- Why does dig show the new record but my browser still loads the old site?
- Because they are asking different caches. A query tool usually goes straight to a resolver, while a browser checks its own DNS cache first, then the operating system's, then the resolver. Any of those can still be holding an unexpired copy. Browsers also keep HTTP connections alive and cache the page itself, so even a correct lookup does not guarantee a fresh request.
- What is the difference between my registrar and my DNS host?
- The registrar is who you rent the name from, and the only DNS record they control is the NS record that says which nameservers are in charge. That record lives in the TLD's zone, one level above yours. The DNS host runs those nameservers and holds everything else: your A, MX, TXT and the rest. If the world cannot find your domain at all, check the NS delegation at the registrar before you look at any individual record.
- Why can I not put a CNAME on my bare domain?
- RFC 1034 says that if a CNAME is present at a name, no other data may be present at that name. The bare domain necessarily carries SOA and NS records, so a CNAME cannot legally sit alongside them. Providers work around it with names like ALIAS, ANAME or CNAME flattening: you type a hostname, and their nameserver resolves it and answers with plain A and AAAA records, so no resolver ever sees a CNAME there.
- Does DNS know when my server goes down?
- No. A nameserver hands back whatever the zone says, and it has no idea whether anything is listening at that address. Managed DNS providers sell health checking as a product layered on top, where their own systems probe your servers and edit the zone for you, but plain DNS has no concept of a server being up or down.