theHarvester

theHarvester runs one passive OSINT sweep and hands you what the internet already knows about a domain: subdomains, hosts, email addresses, and names, pulled from search engines, certificate transparency logs, and OSINT APIs instead of from the target itself. Reach for it as the quiet first footprinting pass, whether you are opening a CTF with an external scope, inventorying an attack surface before active recon, or checking what of your own domain is already exposed.

Mental model: theHarvester -d DOMAIN -b SOURCE [options] -f OUTBASE. Everything else is one flag away.

Install, update, verify#

sudo apt install -y theharvester                  # Debian / Ubuntu / Kali
pipx install theHarvester                         # clean isolated install, any distro
sudo apt install --only-upgrade -y theharvester   # update (apt)
pipx upgrade theHarvester                          # update (pipx)

theHarvester --version    # confirm the version
command -v theHarvester   # confirm it is on PATH
theHarvester -h | less    # skim options and the current source list

Mind the casing: the command is theHarvester (capital H), though the Debian package also exposes theharvester. If one is not found try the other, or run python3 theHarvester.py -h from a git clone. To avoid a local install entirely, use Docker: docker run --rm laramies/theharvester -d DOMAIN -b crtsh.

Flags you’ll actually use#

Skim these first; the cheat sheet below is these flags combined.

--screenshot DIR grabs a screenshot of each resolved host. Flag names and source lists drift between versions, so if one here does not match your build, check theHarvester -h.

Cheat sheet#

Each block builds up one flag at a time, so you can stop at the line that does the job.

# Build up a run: stop at the line you need
theHarvester -d DOMAIN -b crtsh                         # 1. keyless CT-log subdomains
theHarvester -d DOMAIN -b crtsh,hackertarget,rapiddns  # 2. + more keyless sources
theHarvester -d DOMAIN -b crtsh,duckduckgo -l 300      # 3. + a search engine, capped
theHarvester -d DOMAIN -b all -l 500 -f osint/full     # 4. + every source, saved

# Pick sources on purpose
theHarvester -d DOMAIN -b crtsh            # certificate transparency (no key)
theHarvester -d DOMAIN -b hackertarget     # keyless host / DNS data
theHarvester -d DOMAIN -b otx              # AlienVault OTX passive DNS (no key)
theHarvester -d DOMAIN -b duckduckgo -l 200      # search-engine hosts/emails, capped
theHarvester -d DOMAIN -b shodan           # key-only: enrich with Shodan
theHarvester -d DOMAIN -b securityTrails   # key-only: historical DNS

# Volume and paging
theHarvester -d DOMAIN -b duckduckgo -l 200          # cap a noisy source at 200
theHarvester -d DOMAIN -b duckduckgo -l 200 -S 200   # results 200-400 (next page)

# Save and re-parse
theHarvester -d DOMAIN -b crtsh,duckduckgo -f osint/run   # writes run.json + run.xml
jq -r '.hosts[]'  osint/run.json | sort -u          # clean host list
jq -r '.emails[]' osint/run.json | sort -u          # clean email list

# Validate and enrich with DNS (light active)
theHarvester -d DOMAIN -b crtsh -r             # resolve found hosts to IPs, drop the dead
theHarvester -d DOMAIN -b crtsh -c             # brute-force extra subdomains
theHarvester -d DOMAIN -b crtsh -t             # flag takeover candidates
theHarvester -d DOMAIN -b crtsh -n             # reverse-DNS sweep found IP ranges for more hosts
theHarvester -d DOMAIN -b crtsh -r 1.1.1.1     # resolve using a chosen DNS resolver

# Extras
theHarvester -d DOMAIN -b duckduckgo -p                     # route through proxies.yaml
theHarvester -d DOMAIN -b crtsh --screenshot shots/   # screenshot resolved hosts

# No local install
docker run --rm laramies/theharvester -d DOMAIN -b crtsh,duckduckgo -l 100

Command breakdowns#

Each block below is one task: the command, then what it does and what to expect.

Take a quiet first look with certificate transparency#

theHarvester -d DOMAIN -b crtsh

crtsh reads public CT logs, needs no API key, and touches only the logs, never the target. Expect a banner then a list of every subdomain that ever appeared in a TLS cert. It rarely comes back empty for a real domain, so start here.

Combine several keyless sources in one run#

theHarvester -d DOMAIN -b crtsh,hackertarget,rapiddns,otx -l 300 -f osint/passive

Merges hosts and emails from four sources that work without keys, capped at 300 each, saved to osint/passive.json and .xml. The overlap between sources is your highest-confidence set.

Scrape hosts and emails from a search engine#

theHarvester -d DOMAIN -b duckduckgo -l 200

Search engines surface indexed subdomains and public email addresses. -l 200 keeps the source from running long or getting throttled. DuckDuckGo is the steady keyless engine here.

Use a key-only source (Shodan, SecurityTrails, Hunter)#

theHarvester -d DOMAIN -b securityTrails -l 500

Rich sources need a key in api-keys.yaml. Without one the source errors or returns nothing, which is not the same as “no footprint”. Add the key, then still name the source explicitly.

Sweep every configured source#

theHarvester -d DOMAIN -b all -l 500 -f osint/full

Runs every module. Slow, and any source you lack a key for prints an error, which is normal. Use it once for breadth, then go back to named sources for signal.

Save output for later parsing#

theHarvester -d DOMAIN -b crtsh,duckduckgo -f osint/run

-f writes run.json and run.xml. Always save: it lets you re-parse without re-running the whole collection, and the JSON is what jq slices.

Pull a clean host list with jq#

jq -r '.hosts[]' osint/run.json | sed 's/:.*//' | sort -u | tee notes/hosts.txt

The JSON hosts array holds name:ip for resolved entries; sed strips the IP, sort -u dedupes. This is the plain host list you feed the next tool.

Resolve found hosts to separate live from dead#

theHarvester -d DOMAIN -b crtsh -r -f osint/resolved

-r runs DNS resolution on each found host so you can tell a live name from one decommissioned years ago. A CT log remembers a name long after the box is gone.

Brute-force extra subdomains#

theHarvester -d DOMAIN -b crtsh -c -f osint/brute

-c layers a DNS brute force on top of the passive hits. This is active: it fires many DNS queries and produces NXDOMAIN bursts, so only run it in scope.

Flag subdomain-takeover candidates#

theHarvester -d DOMAIN -b crtsh -t

-t checks whether a discovered subdomain points at an unclaimed resource (a dangling CNAME). Treat every hit as a candidate to verify manually before reporting.

Reverse-sweep the discovered IP ranges for more hosts#

theHarvester -d DOMAIN -b crtsh -n

-n expands each discovered IP to its /24 and runs a reverse-DNS lookup across the range, so one address can surface more hostnames that belong to the target domain.

Page deeper into a noisy source#

theHarvester -d DOMAIN -b duckduckgo -l 200 -S 200 -f osint/page2

-S 200 starts at result 200, so with -l 200 you get results 200-400. Useful when the first page is exhausted and the source clearly has more.

Resolve through a chosen DNS server#

theHarvester -d DOMAIN -b crtsh -r 1.1.1.1

Passing a resolver to -r runs resolution through that server instead of the system one; comma-separate for several (-r 1.1.1.1,8.8.8.8). Handy when the local DNS is captive, slow, or returns wildcard answers.

Cross-check which hosts still resolve with dig#

while read -r h; do dig +short "$h" | grep -q . && echo "LIVE $h" || echo "dead $h"; done < notes/hosts.txt

theHarvester’s own -r resolves during the run; this repeats the check on your saved list any time, and keeps the raw OSINT separate from a validated asset list.

Run without installing anything#

docker run --rm laramies/theharvester -d DOMAIN -b crtsh,duckduckgo -l 100

The image ships Python and dependencies; --rm cleans up after. Mount a volume if you want -f output to survive the container.

Reading the output#

A single source failing never means the domain has no footprint; it means that one source did not answer. The crtsh source is the sanity check, since it almost always returns something for a live domain.

Troubleshooting#

Gotchas#

• A host is a lead, not an asset. A name in a CT log can be years dead; nothing is in scope until it resolves and you confirm ownership. Scanning unvalidated hits wastes time and can stray out of scope.
• -b all is a trap, not a shortcut. It is slow, hammers every source, and buries real hits under errors from the dozens of sources you have no key for. Name the two or three you actually want.
• Keyless does not mean crtsh only. hackertarget, rapiddns, otx, and certspotter also work with no key; leaning on one source reads its blind spots as the domain’s.
• -c, -r, -n, and -t cross the passive line. They send DNS queries the target’s resolvers can log (brute force especially, as NXDOMAIN bursts). Keep them out of a strictly passive engagement.
• Harvested emails and names are personal data. Store them in a separate, access-controlled file, collect only what the engagement needs, and never publish a scraped address list.
• The command case bites everyone. theHarvester and theharvester are both real depending on the install; a “not found” usually means the other spelling, not a missing tool.

Pairs with#

crtsh gives the keyless subdomain backbone, then dig and dnsx resolve and validate the harvested names so only live hosts survive. Once a host is confirmed, nmap scans it and gobuster or ffuf map a web root or virtual host. jq, sort, and tee carve the JSON into clean host and email lists. When passive breadth is not enough and subdomain discovery is the whole job, reach for amass or subfinder for depth, or recon-ng and spiderfoot for a full framework with a database; theHarvester stays the fastest one-shot grab of hosts, emails, and names from many sources at once.