DNS Name Resolution Design for proper DNSSEC validation
Many networks have a DNS name resolution design that uses a hybrid DNS server. A hybrid DNS server is a DNS server that combines both functions that a DNS server can operate in into one process: the authoritative function (hosting zones) and the resolving/caching function (looking up names in DNS on behalf of DNS clients).
Both BIND and Microsoft Windows DNS server can operate in a hybrid mode. Other (some would say "more modern") DNS servers such as Unbound (resolving only) and NSD (caching only) separate these functions.
While running a hybrid DNS server was fine in the world before DNSSEC, the deployment of DNSSEC requires a closer look at these "legacy" DNS designs.
Authoritative DNS servers, when queried for a name they are authoritative for, will set the AA-flag (/A/uthoritative /A/nswer) in the answer.
Resolving DNS server that perform DNSSEC validation will set the AD-flag, if the DNS data received is validating. The AA-flag and the AD-flag are mutually exclusive, there can only be either one in an DNS answer. The reason is that it would be extra work, but no added security, if an authoritative DNS server would validate its own data. After all, if the server has been compromised, it cannot trust its own data and it is not possible to detect the false data from within the authoritative DNS server. Only a DNS server external to the authoritative server can validate DNSSEC signed zone data.
For proper DNSSEC name resolution (AD flag set on all answers from DNSSEC secured zones), all queries must go through a resolving DNS Server that is separate from the DNS server hosting the zones.
Name resolution in legacy hybrid DNS designs
The graphic below shows DNSSEC name resolution for a DNSSEC secured name from the Internet, using a hybrid DNS server:
The AD flag is seen by the client, and DNSSEC aware applications will continue to work.
The next graphic show name resolution for an internal DNS domain, which is also DNSSEC secured and hosted on the DNS server that is used for DNSSEC validation (a hybrid DNS server):
Here, only the AA-flag is seen by the client. DNSSEC aware applications that require the AD flag will fail.
Name resolution in modern DNS designs
In modern DNS designs, the resolving/caching DNS servers are separate from the authoritative DNS servers. As a rule of thumb, in these designs, caching/resolving DNS servers only receive DNS queries from client machines, never from other DNS servers (except for forwarding, where a DNS server acts like a client proxy). Authoritative servers only talk to caching/resolving DNS servers, never directly to DNS clients.
Resolving and validating names in the Internet works like in the hybrid design above:
AD-flag is seen by the client, everything is good and secure.
DNS queries for internal DNS domains now also will see the AD-flag:
DNSSEC aware applications work for all domains, internal and external.
Benefits from a modern DNS design
A modern DNS name resolution design is required to be able to make use of all features DNSSEC offers. The number of DNSSEC aware applications is likely to grow in the future, so more and more applications will rely on proper DNSSEC validation.
Besides DNSSEC, there are more benefits in a modern DNS name resolution design: the possibility to apply different security policy and settings to the caching and authoritative servers, better troubleshooting options, simpler maintenance and upgrade-ability of DNS server components among them.
Separating the resolving/caching and the authoritative functions of DNS servers does not necessarily require more hardware or more physical DNS servers. Virtualization can be used to create separate instances of DNS servers on the same machine. Either full machine virtualization (HyperV, VMWare, VirtualBox, XEN or KVM), system virtualization (Solaris zones or Linux Container/LXC) or even process virtualization (BIND 9 views) can be used.
