DNS, Network Protocols and Security

This course introduces the foundational concepts of the Domain Name System (DNS) protocol, how it functions in practice, and the security-relevant behaviors that every practitioner working in a security role should understand. DNS is a deceptively complex protocol, and while its core concept is simple, real depth emerges once caching, resolution chains, and adversarial abuse are considered. This material walks through the core functionality of the protocol and highlights concepts that every security practitioner should be aware of.

Table of Contents

• Module 1: DNS Queries and Responses
  • Understanding DNS Fundamentals
  • DNS Record Types Reference
  • DNS Resolution Flow
  • Demo: Capturing and Analyzing DNS Traffic
  • Reverse DNS Lookups
• Module 2: Modifying Traffic Flow
  • The DNS Resolution Chain and Caching Layers
  • Demo: Modifying the Operating System Cache and Hosts File
  • Recognizing Vulnerable DNS Settings
  • DNS Security Mechanisms: Spoofing, DNSSEC, DoH, and DoT
• Module 3: Detecting DNS Command-and-Control (C2)
  • Why Attackers Abuse DNS for C2
  • Understanding Entropy Analysis
  • Demo: Generating an Entropy Score with Zeek
  • Advanced Analysis and Real-World Considerations
• Summary

Module 1: DNS Queries and Responses

Understanding DNS Fundamentals

Domain Name System (DNS) is a complicated protocol with quite a bit of depth. Even though the fundamental concept is simple, do not be surprised to find yourself still learning new things about DNS behavior deep into a cybersecurity career.

At its core, DNS solves one problem: when a computer browses to a web address such as pluralsight.com, the operating system needs to convert that human-readable domain name into an IP address that can actually be routed on the network. DNS is the protocol that accomplishes this translation.

DNS operates as a conversation of queries and answers:

• A client makes a query to resolve a web address.
• The DNS server that receives the query — typically an Active Directory server in an enterprise environment — responds with an answer, formatted as a resource record.

There are many types of DNS records and response formats, and understanding them is the first step toward understanding the protocol’s security implications.

DNS Record Types Reference

The demo traffic captured in this module showed several distinct resource record types in a single web request. The table below summarizes the record types referenced in this course, along with their purpose:

Key observations from the traffic analysis:

• A single web request can trigger multiple DNS queries — in the demo, browsing to a single page produced both an A query (IPv4) and an AAAA query (IPv6), automatically initiated as part of normal browser operation.
• A response can include a CNAME plus multiple A records. It is very common for popular domains to have multiple IP addresses returned for the same query, used for purposes like load balancing.
• The IP header of the packet carrying a DNS response is not part of the DNS answer itself — it simply reflects the communication between the querying host and the DNS server (for example, a local resolver at 172.31.0.2). The DNS payload is what contains the actual resource records.
• DNS typically operates over UDP, which provides no built-in error-checking or session tracking at the transport layer. Because of this, the DNS server includes a copy of the original query alongside the answer in its response, ensuring the client can match responses to the queries it sent.

DNS Resolution Flow

The following diagram illustrates the conceptual flow of a DNS query as it travels from an application, through the operating system, and potentially out to external DNS infrastructure:

sequenceDiagram
    participant Browser as Browser (Firefox)
    participant OS as OS DNS Resolver
    participant DNSServer as DNS Server (172.31.0.2)
    participant CDN as Authoritative / CDN (Cloudflare)

    Browser->>OS: Resolve pluralsight.com
    OS->>DNSServer: A query for pluralsight.com
    DNSServer-->>OS: CNAME -> www.pluralsight.com.cdn.cloudflare.net
    DNSServer-->>OS: A record(s) - multiple IPv4 addresses
    OS->>DNSServer: AAAA query for pluralsight.com
    DNSServer-->>OS: AAAA record(s) - IPv6 addresses
    OS-->>Browser: Resolved IP address(es)
    Browser->>CDN: HTTP/HTTPS request to resolved IP

Demo: Capturing and Analyzing DNS Traffic

This walkthrough captures DNS traffic generated by a normal web browsing session and inspects it using both a packet analyzer and a command-line query tool, using a Linux graphical endpoint.

Step 1 — Start a packet capture.

Open Wireshark, apply a capture filter limiting traffic to DNS only, and begin capturing on the primary interface:

Capture filter: udp port 53
Interface: eth0

Step 2 — Generate DNS traffic.

Step 3 — Filter out unrelated background traffic.

By default, the browser generates background DNS requests (for example, to vendor telemetry/services) as part of its normal startup operations. To isolate only the traffic of interest, apply a display filter for the domain of interest:

Display filter: dns contains "pluralsight"

Step 4 — Inspect the first query/response pair.

• The first query is an A query, resulting in an IPv4 address for pluralsight.com — the most common form of DNS request.
• The second query, triggered automatically as part of the same web request, is a AAAA (“quad A”) query, resulting in an IPv6 address.
• The IP header of the response frame contains IPv4 addressing information for the conversation between the host and the DNS server (172.31.0.2) — this is not the DNS answer itself, just the transport-level communication.
• Because DNS runs over UDP (no native error-checking/session tracking), the response includes a copy of the original query to keep request/response pairs matched.

Step 5 — Inspect the answer section.

• Below the CNAME are two A records containing IPv4 answers — multiple IPs for the same domain, commonly used for load balancing.

Step 6 — Repeat the query using nslookup.

nslookup pluralsight.com

This returns the same two IPv4 answers and two IPv6 answers seen in Wireshark, with the CNAME listed at the top.

Step 7 — Request a specific record type.

nslookup -query=AAAA pluralsight.com

Step 8 — Request full/verbose response information.

nslookup -debug pluralsight.com

The -debug option provides a much more detailed response, including Time To Live (TTL) values and any additional record information returned by the server.

Reverse DNS Lookups

nslookup can also perform a reverse DNS lookup by requesting a PTR record for a given IP address. Because of how DNS records are stored, the IP address must be entered in reverse order:

# Reverse lookup for 104.19.162.127
nslookup -query=PTR 127.162.19.104.in-addr.arpa

The result returned was localhost, despite the IP address actually belonging to pluralsight.com. This is a deliberate security strategy: it bypasses source-address checks and prevents individuals from performing meaningful pointer lookups against certain domains/IP ranges, since returning localhost gives an attacker no useful reconnaissance information about the true owner of that address.

There are many variations on this pattern, but the request/response flow demonstrated here is representative of a normal DNS conversation.

Module 2: Modifying Traffic Flow

The DNS Resolution Chain and Caching Layers

Two concepts are essential to understanding how DNS traffic actually flows, and how an attacker might manipulate that flow.

1. The resolution chain. Not every client in an enterprise network makes external DNS queries directly. Instead, clients make requests to a local DNS server / DNS resolver, which then performs the external queries on the client’s behalf. Fully resolving a hostname can involve a communication chain of up to three additional DNS servers beyond the local resolver.

flowchart LR
    Client[Client Endpoint] --> LocalResolver[Local DNS Resolver]
    LocalResolver --> Root[Root DNS Server]
    LocalResolver --> TLD[TLD DNS Server]
    LocalResolver --> Authoritative[Authoritative DNS Server]
    Authoritative --> LocalResolver
    TLD --> LocalResolver
    Root --> LocalResolver
    LocalResolver --> Client

2. Caching. Before a client even sends a request to a DNS server, it checks local caches first. There are three primary locations where DNS records are cached, and each one is a location that must be kept secure:

flowchart TD
    A[DNS Lookup Requested] --> B{Browser Cache?}
    B -- Hit --> Z[Return Cached Result]
    B -- Miss --> C{OS Cache?}
    C -- Hit --> Z
    C -- Miss --> D{DNS Server Cache?}
    D -- Hit --> Z
    D -- Miss --> E[External Recursive Query]
    E --> Z

Demo: Modifying the Operating System Cache and Hosts File

This walkthrough inspects and then manipulates the DNS configuration of a Linux endpoint to demonstrate how easily traffic can be redirected.

Step 1 — Inspect the resolver configuration.

On most UNIX-based systems, the DNS resolver configuration is found in /etc/resolv.conf:

nameserver 127.0.0.53
search us-west-2.compute.internal
options edns0 trust-ad

Step 2 — Inspect the static hosts file.

The endpoint’s hosts file (/etc/hosts) is a static mapping of domain names to IP addresses, checked before any external DNS request is made:

127.0.0.1   localhost
::1         localhost ip6-localhost ip6-loopback

If a request is made for localhost, no external DNS query is necessary — the static mapping resolves it directly to 127.0.0.1. This can be verified with a simple ping:

ping -c 4 localhost

Step 3 — Modify the hosts file to redirect traffic.

An attacker-controlled endpoint is available at 172.31.24.88. A single line is added to the hosts file to redirect pluralsight.com to the attacker’s IP:

172.31.24.88   pluralsight.com

Step 4 — Verify the redirection via nslookup.

nslookup pluralsight.com

The returned answer now matches the attacker’s configured IP address (172.31.24.88) instead of the legitimate CDN-hosted address.

Step 5 — Browse to the redirected domain.

Open a new private/incognito browser window (to bypass the browser cache) and browse to:

http://www.pluralsight.com

The browser will still attempt to connect over HTTPS by default, but since no valid TLS certificate was configured on the attacker endpoint, the connection must be made explicitly over http:// for the purposes of this demonstration.

Outcome: The browser is now connecting to the attacker’s server rather than the legitimate site. In this simplified example, it is visibly not the real page — but the exercise illustrates how little effort (a single line change in a local configuration file) is required for an attacker who already has access to an endpoint to redirect a user to a convincingly disguised, look-alike page.

sequenceDiagram
    participant User as User
    participant Browser as Browser
    participant Hosts as /etc/hosts (tampered)
    participant Attacker as Attacker Server (172.31.24.88)
    participant Legit as Legitimate Site (Cloudflare CDN)

    User->>Browser: Browse to pluralsight.com
    Browser->>Hosts: Check static hosts file
    Hosts-->>Browser: 172.31.24.88 (attacker-controlled)
    Note over Browser,Legit: External DNS query never occurs
    Browser->>Attacker: HTTP request
    Attacker-->>Browser: Spoofed/look-alike page

Recognizing Vulnerable DNS Settings

DNS is crucial to how virtually everything in an environment operates. When evaluating the security of network communication, it is important to logically think through every step of a protocol’s behavior and consider the potential exploitation methods at each stage. The hosts-file manipulation demonstrated above is one of the simplest possible examples: an attacker with local access (or the ability to modify a configuration/deployment file, such as via a compromised provisioning script or container image) can silently redirect any domain of their choosing without ever touching the network-level DNS infrastructure.

Other layers worth scrutinizing from a defensive standpoint include:

• Which local caches (browser, OS, resolver) are trusted, and whether they validate responses.
• Whether trust-ad / DNSSEC validation is actually enforced end-to-end, or just passed through.
• Whether local configuration files (/etc/hosts, /etc/resolv.conf) are monitored for unauthorized changes.
• Whether DNS traffic itself is encrypted or otherwise protected from on-path tampering.

DNS Security Mechanisms: Spoofing, DNSSEC, DoH, and DoT

Building on the concepts observed in the demo — plain-text UDP queries, unauthenticated responses, and easily tampered local configuration — it is worth understanding the broader landscape of DNS attack techniques and the mechanisms designed to counter them.

Common DNS-based attack techniques:

Defensive mechanisms:

flowchart TD
    subgraph Attack Techniques
        A1[Hosts File Tampering]
        A2[DNS Spoofing]
        A3[Cache Poisoning]
        A4[DNS Tunneling / C2]
    end
    subgraph Defensive Mechanisms
        D1[DNSSEC - Response Authenticity]
        D2[DoT - Encrypted Transport]
        D3[DoH - Encrypted over HTTPS]
        D4[Monitoring / Entropy Analysis]
    end
    A2 -.mitigated by.-> D1
    A3 -.mitigated by.-> D1
    A2 -.mitigated by.-> D2
    A2 -.mitigated by.-> D3
    A4 -.detected by.-> D4

Module 3: Detecting DNS Command-and-Control (C2)

Why Attackers Abuse DNS for C2

This module takes on the perspective of an incident responder given a packet capture (PCAP) file containing suspicious DNS traffic. In this scenario, an attacker is using DNS subdomains to transport command-and-control (C2) information between a compromised host and an attacker-controlled domain.

Why DNS specifically? When an attacker builds a C2 channel, a primary objective is blending into normal traffic to avoid detection:

• DNS rarely has dedicated security controls applied to it in the way HTTP/HTTPS traffic often does.
• DNS is extremely common — nearly every device generates it constantly.
• DNS traffic is very “noisy” on a typical network, making anomalies difficult to spot by casual inspection.

These three properties combine to make DNS a near-ideal channel for an adversary to hide C2 traffic in plain sight.

Two concepts are important before analyzing the traffic:

1. Zeek is the tool used for analysis. Zeek is an open-source tool that performs session and protocol analysis of network traffic, and it also supports custom scripts for specialized analysis — in this case, a script that performs entropy analysis.
2. Entropy is a measure of randomness or diversity within a dataset:
   • Data with full/high entropy is close to fully random, with no meaningful, human-recognizable pattern.
   • Data with low entropy indicates predictability — future values can be reasonably guessed from prior ones.

Encoded C2 data hidden inside DNS subdomains tends to look far more random (higher entropy) than legitimate, human-chosen hostnames, which is the basis for detecting it.

sequenceDiagram
    participant Host as Compromised Host
    participant Resolver as Local/Enterprise DNS Resolver
    participant Attacker as Attacker-Controlled DNS Server

    Host->>Resolver: Query for <encoded-c2-data>.dirtylitter.iamironcat.com
    Resolver->>Attacker: Forwarded recursive query
    Attacker-->>Resolver: Response containing encoded C2 instructions
    Resolver-->>Host: Response passed through
    Note over Host,Attacker: Traffic blends in with normal, noisy DNS activity

Understanding Entropy Analysis

The overall analysis pipeline used in this module follows a repeatable pattern:

flowchart LR
    PCAP[PCAP File] --> Zeek[Zeek + Custom Script]
    Zeek --> Log[dns_entropy.log]
    Log --> Extract[awk: extract query + entropy columns]
    Extract --> Sort[sort: rank by entropy score]
    Sort --> Findings[Identify high-entropy subdomains]

Demo: Generating an Entropy Score with Zeek

Step 1 — Navigate to the DNS analysis directory on the Linux terminal, which contains a custom script named dnsentropy.zeek.

Step 2 — Review the script. It is not necessary to be an expert in Zeek’s scripting syntax, but it is good practice to understand what a script does before executing it. At a high level, the script instructs Zeek to:

• Analyze each DNS conversation captured in the PCAP.
• Extract the query and answer fields from each conversation.
• Calculate an entropy score for the extracted data.
• Generate a new log file containing that entropy score alongside the original query/answer data.

Step 3 — Run Zeek against the capture inside a Docker container. Because Zeek is containerized in this environment, the command is more involved than a typical native installation would require. The general form of this operation is:

# Run Zeek (containerized) against the capture, using the custom entropy script
docker run --rm \
  -v "$(pwd)":/pcap \
  zeek/zeek \
  zeek -r /pcap/capture.pcap /pcap/dnsentropy.zeek

Step 4 — Confirm the log file was generated.

ls -la

A new log file, dns_entropy.log, is present after the run completes.

Step 5 — Inspect the raw log file.

cat dns_entropy.log

Zeek organizes its output into columns that are tab-delimited, which can be difficult to read directly on the command line. In a typical enterprise workflow, this data would be forwarded into a visualization tool for easier analysis rather than read raw in a terminal. The illustrative column layout produced by the custom script is:

# ts            query                                             qtype  answer               entropy
1699999999.123  a1b2c3d4e5f6.dirtylitter.iamironcat.com            A      203.0.113.55         4.82
1699999999.456  status-check.dirtylitter.iamironcat.com            A      203.0.113.55         2.10
1699999999.789  9f8e7d6c5b4a3f2e1d0c.dirtylitter.iamironcat.com    A      203.0.113.55         4.95

Step 6 — Extract the query column and entropy score, then sort by highest entropy.

awk -F'\t' '{print $2, $5}' dns_entropy.log | sort -k2,2 -n

• awk is used here to extract specific columns of data (the query and its associated entropy score) in the desired order.
• sort arranges the output numerically so the highest entropy values appear at the bottom of the results.

Most of the highest-ranked results contain subdomain queries for dirtylitter.iamironcat.com. These subdomains carry encoded information sent to whoever controls the DNS server authoritative for that root domain.

Step 7 — Repeat the extraction against the answer field.

awk -F'\t' '{print $4, $5}' dns_entropy.log | sort -k2,2 -n

Once again, dirtylitter.iamironcat.com shows up most frequently among the highest-entropy results — this time in the encoded answers returned from the attacker’s DNS server, rather than just the queries sent to it.

Advanced Analysis and Real-World Considerations

As an incident responder, the logical next step would be attempting to decode the information contained within those high-entropy subdomains. However, this can range from straightforward to effectively impossible, depending entirely on how the C2 channel encodes and transports its data (custom encodings, encryption, chunking schemes, etc.).

This scenario demonstrates a real-world technique for detecting malicious subdomain traffic hidden within DNS. The command-line-based Zeek analysis shown here is intentionally more advanced than the earlier modules, requiring more experience with Linux tooling (awk, sort, containerized tool execution). In a typical enterprise environment, a dedicated visualization or SIEM tool would make sorting, filtering, and extracting this kind of data significantly simpler than working with raw tab-delimited logs on the command line.

Summary

This course covered the DNS protocol from a security practitioner’s perspective, progressing from fundamental query/response mechanics to traffic manipulation and, finally, to detecting sophisticated abuse of the protocol for command-and-control purposes.

Key principles:

• DNS translates human-readable domain names into IP addresses through a query/answer model, and even a single web request can generate multiple queries (A, AAAA) and multi-record responses (CNAME chains, multiple A records).
• DNS conventionally runs over UDP, meaning the protocol itself has no native error-checking or session tracking — the server includes the original query in its response purely to allow the client to match request/response pairs.
• Reverse DNS (PTR) lookups can be intentionally restricted by returning non-informative answers (such as localhost) as a defensive measure against reconnaissance.
• DNS resolution relies on a chain of caches (browser, OS, resolver) and, when necessary, external servers — every one of these caching layers is a location that must be secured, since a compromised cache entry can silently redirect many users.
• Local configuration files such as /etc/hosts and /etc/resolv.conf are high-value, low-effort targets for attackers: a single modified line can redirect traffic for a legitimate domain without ever touching network-level DNS infrastructure.
• Defensive mechanisms such as DNSSEC (response authenticity), DoT, and DoH (encrypted transport) exist specifically to counter spoofing, cache poisoning, and on-path tampering.
• DNS is a favored channel for command-and-control traffic and data exfiltration because it is common, largely uninspected, and noisy — properties that make encoded subdomain traffic blend into legitimate background noise.
• Entropy analysis (using tools like Zeek with custom scripts) is an effective technique for surfacing anomalous, high-randomness subdomains that are characteristic of encoded C2 traffic, even though fully decoding the payload may not always be feasible.

Quick-reference checklist for DNS security review:

Search Terms

dns · network · protocols · security · networking · fundamentals · systems · analysis · entropy · flow · modifying · resolution · traffic