DNS Trawler is a program that maps the complete resolution graph of a given domain. It collects all NS and A records of names explicitly or implicitly involved in the resolution of the input domain and annotates them with additional information (e.g., is a nameserver, is an NXDOMAIN, or has a CNAME). In 24h, DNS Trawler can map the trust closures of 670,000 FQDNs, collecting records on over 2M names, and outputs a log that domain operators can easily use to identify malformed records.
At the end of a run on domain, DNS Trawler produces the following output:
- trawler-graph.jsonl: The complete resolution graph of a given domain. Each entry represents one domain involved
in the resolution dependency and contains all A, NS, and CNAME records associated with it. For instance,
for
google.com:
{"input":"google.com","level":"info","msg":"","node":{"name":"google.com","parent":"com","cname":"","queried_all_parent_ns":true,"queried_all_peer_ns":true,"is_fully_mapped":true,"is_ns":false,"is_cname":false,"is_tld":false,"is_pstld":false,"is_registerable":true,"nameservers":["ns2.google.com","ns1.google.com","ns3.google.com","ns4.google.com"],"ipv4":["142.251.45.174"],"is_ns_to":[],"is_parent_to":["ns3.google.com","ns4.google.com","ns2.google.com","ns1.google.com"],"is_cname_to":[],"is_nxdomain":false,"is_timeout":false}}
- trawler-errors.jsonl: All network errors (TIMEOUT, NXDOMAIN, REFUSED, etc) encountered during resolution. Contains the nameserver source of the error and the query that caused it:
{"error":{"QueriedAtNS":"erroring-ns.com","QueriedAtIPv4":"1.2.3.4.","QueriedForDomain":"erroring-query.com","QueryType":2,"ResponseStatus":"NXDOMAIN"},"input":"input-domain.com","level":"info","msg":""}
Because these entries contain pointers to domain's "upstream" (parent domain) and "downstream" (nameservers, subdomains) dependencies, they can be used to analyze trust relationships and detect indirect hijacking risks. For example, an NXDOMAIN indicates a dangling dependency i.e., a domain that no longer exists yet remains a transitive dependency for resolution. If a domain belonging to a nameserver is NXDOMAIN, and that domain is available for registration, all dependencies of the nameserver are at risk of domain hijacking. DNS Trawler already performs an analysis step at the end for this last critical case. Users can add additional analysis steps for other types of vulnerabilites.
--domain-names Comma-separated list of domains to analyze. e.g. google.com,yahoo.com,amazon.com
--csv-file-name Path to CSV of domain names to analyze.
<Optional flags>
--threads Number of trawlers. Default 1.
--verbosity Verbosity between 1 (lowest) to 5 (highest). Default 3.
1 - Fatal error msg only, no analysis
2 - Fatal and vulnerability analysis only
3 - Default
4 - Analysis and graph + resolution error dump
5 - Debug
--ip-rate-limit Max number of IPs to query per second. Default 10 IP/s.
--domain-rate-limit Max number of effective domains to query per second. Default 10 effective domains/s.
--output-directory Creates output logs in this directory. Default STDOUT.
--trusted-names Comma-separated list of trusted TLDs. e.g. com,net,info. Default trusts all TLDs.
Sample command 1:
make trawler && ./dns-trawler --domain-names=google.com --verbosity=4
...runs Trawler on google.com and dumps graph and analysis result to STDOUT
Sample command 2:
make trawler &&
./dns-trawler --csv-file-name=crux-top-1M.csv
--threads=1000
--verbosity=4
--domain-rate-limit=100
--log-path=crux-analysis
...creates files crux-analysis/trawler-graph.jsonl, crux-analysis/trawler-errors.jsonl, crux-analysis/trawler.log.