Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,15 @@ All notable changes to this project will be documented in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## [Unreleased]

### Fixed
- Structured output (`-format json` and `-format csv`) is now finalized only on the successful scan path. The finalizer previously ran via `defer` on every exit, so an early error (such as wildcard detection without `-force`) emitted an empty JSON array. Text output behavior is unchanged.

### Docs
- GitHub Pages landing page (`docs/index.md`) refreshed to the 0.6.0 feature set, adding cards for Output Formats (`-format`), Rate Limiting (`-rate`), Record Types (`-type`), and Recursive Enumeration (`-recursive`/`-depth`).
- Normalized em dashes to hyphens across `docs/` for consistency with the no-em-dash convention.

## [0.6.0] - 2026-06-03

### Added
Expand Down
38 changes: 19 additions & 19 deletions docs/ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,16 +23,16 @@ This architecture is designed to be efficient by performing multiple DNS lookups
### Package Structure

```
main.go CLI entry point (flag parsing, wiring, -tui dispatch)
internal/scan/runner.go Scan engine: Config, Event types, Run(ctx, cfg, events)
internal/dns/resolver.go ResolveDomain, ResolveDomainWithRetry, CheckWildcard
internal/dns/simulate.go SimulateResolution
internal/output/writer.go Thread-safe Writer (results→stdout, diagnostics→stderr)
internal/wordlist/reader.go LoadWordlist (dedup + sanitize)
internal/tui/model.go Root Bubble Tea model (form → scan state machine)
internal/tui/form.go Config form screen (textinput fields + toggles)
internal/tui/scan_view.go Live results screen (viewport + progress bar)
internal/tui/config.go Session persistence (load/save ~/.config/subenum/last.json)
main.go - CLI entry point (flag parsing, wiring, -tui dispatch)
internal/scan/runner.go - Scan engine: Config, Event types, Run(ctx, cfg, events)
internal/dns/resolver.go - ResolveDomain, ResolveDomainWithRetry, CheckWildcard
internal/dns/simulate.go - SimulateResolution
internal/output/writer.go - Thread-safe Writer (results→stdout, diagnostics→stderr)
internal/wordlist/reader.go - LoadWordlist (dedup + sanitize)
internal/tui/model.go - Root Bubble Tea model (form → scan state machine)
internal/tui/form.go - Config form screen (textinput fields + toggles)
internal/tui/scan_view.go - Live results screen (viewport + progress bar)
internal/tui/config.go - Session persistence (load/save ~/.config/subenum/last.json)
```

## 2. Key Components / Modules
Expand Down Expand Up @@ -71,7 +71,7 @@ internal/tui/config.go — Session persistence (load/save ~/.config/sube
* Function: `dns.Resolve(ctx, domain, timeout, dnsServer) ([]Record, time.Duration, error)` and `dns.ResolveTypes(..., types)` - perform the lookups and return typed `Record{Type, Value}` results. `ResolveTypes` issues per-type lookups (`LookupIP` ip4/ip6 for A/AAAA, `LookupCNAME` for CNAME) and filters to the requested types (default A,AAAA via `-type`).
* Function: `dns.ResolveDomain(ctx, domain, timeout, dnsServer, verbose) bool` - convenience wrapper returning a boolean, used by wildcard detection.
* Function: `dns.ResolveDomainWithRetry(ctx, domain, timeout, dnsServer, verbose, maxAttempts, types) ([]Record, bool)` - wraps the lookup with configurable retry logic and linear backoff between attempts, returning the resolved records.
* Function: `dns.CheckWildcard(ctx, domain, timeout, dnsServer) (bool, error)` resolves two random subdomains to detect wildcard DNS records.
* Function: `dns.CheckWildcard(ctx, domain, timeout, dnsServer) (bool, error)` - resolves two random subdomains to detect wildcard DNS records.
* `net.Resolver{}`: A custom DNS resolver is configured.
* `PreferGo: true`: Instructs the resolver to use the pure Go DNS client.
* `Dial func(ctx context.Context, network, address string) (net.Conn, error)`: A custom dial function is provided to control the connection to the DNS server, using the user-specified `dnsServer` address.
Expand All @@ -86,7 +86,7 @@ internal/tui/config.go — Session persistence (load/save ~/.config/sube
* **Purpose**: To efficiently perform DNS lookups for a large number of potential subdomains, `subenum` employs a worker pool pattern. This allows multiple DNS queries to be in flight concurrently, significantly speeding up the enumeration process compared to sequential lookups.
* **Implementation**: The worker pool logic lives in `internal/scan/runner.go` as `scan.Run(ctx, cfg, events)`. Both the CLI (`run()` in `main.go`) and the TUI (`internal/tui`) call this function.
* **`scan.Config`**: A struct carrying all scan parameters (domain, entries slice, concurrency, timeout, DNS server, simulate flag, etc.).
* **`scan.Event` / `scan.EventKind`**: Typed events emitted on a `chan<- scan.Event` `EventResult`, `EventProgress`, `EventWildcard`, `EventError`, `EventDone`.
* **`scan.Event` / `scan.EventKind`**: Typed events emitted on a `chan<- scan.Event` - `EventResult`, `EventProgress`, `EventWildcard`, `EventError`, `EventDone`.
* **Dispatcher and work queue**: A dispatcher goroutine owns an internal `jobs` channel, the queue of pending `job{domain, depth}` items, a visited set, and a pending-work counter. It seeds the queue from the wordlist slice and feeds workers. Workers submit newly discovered children back to the dispatcher over an `enqueue` channel and signal each finished job over a `completed` channel. The dispatcher closes `jobs` only when the pending counter reaches zero (or the context is cancelled). This lifecycle lets resolved subdomains enqueue children safely (recursive mode) without risking a send on a closed channel.
* **`var wg sync.WaitGroup`**: A `sync.WaitGroup` waits for all worker goroutines to finish.
* **Worker Goroutines Loop**: `cfg.Concurrency` goroutines are launched. Each reads a job from `jobs`, constructs nothing further (the job already holds the full domain), and calls `dns.ResolveDomainWithRetry()` (or `dns.SimulateResolve()` in simulate mode).
Expand All @@ -102,11 +102,11 @@ internal/tui/config.go — Session persistence (load/save ~/.config/sube
* **Implementation**:
* `output.Writer` struct with mutex-protected methods:
* `Result(domain, records)` - in `text` format prints `Found: <domain>` to stdout (and the output file if configured); in `json` format buffers `{"subdomain", "records"}` objects and writes a single array at completion; in `csv` format streams `subdomain,type,value` rows with a header. The format is selected with `-format text|json|csv` (default `text`, which is byte-for-byte identical to prior behavior). The JSON array is buffered because it is a single document and does not stream; JSONL would be the streaming-friendly alternative if needed. Output formats are CLI-only for now (TUI-pending).
* `Progress(pct, processed, total, found)` writes a carriage-return progress line to stderr.
* `Info(format, args...)` writes an informational line to stderr.
* `Error(format, args...)` writes an error line to stderr.
* `Progress(pct, processed, total, found)` - writes a carriage-return progress line to stderr.
* `Info(format, args...)` - writes an informational line to stderr.
* `Error(format, args...)` - writes an error line to stderr.
* **Verbose Output** (when `-v` flag is enabled):
* Configuration summary, per-query DNS resolution info, and final scan statistics all via `Info` to stderr.
* Configuration summary, per-query DNS resolution info, and final scan statistics - all via `Info` to stderr.
* **Progress Reporting** (when `-progress` flag is enabled):
* A dedicated goroutine using a 1-second ticker calls `Progress` on stderr.
* **Interactions**: All components route output through the `Writer`. Since results are the only thing on stdout, piping (`| cut -d' ' -f2`) works without `-progress=false`.
Expand All @@ -130,9 +130,9 @@ internal/tui/config.go — Session persistence (load/save ~/.config/sube
* **Purpose**: Remember the last-used TUI form values across sessions so users don't have to re-type domain, wordlist path, and scan parameters every time.
* **Implementation**:
* `savedConfig` struct mirrors `formValues` with JSON tags.
* `configPath()` returns `os.UserConfigDir()/subenum/last.json` (e.g. `~/.config/subenum/last.json` on Linux/macOS, `%AppData%\subenum\last.json` on Windows).
* `saveConfig(fv formValues) error` marshals `formValues` to JSON and writes it atomically with `os.WriteFile`. Called in `beginScan()` immediately before launching the scan goroutine. Errors are silently discarded so a write failure never blocks the scan.
* `loadSavedConfig() (savedConfig, bool)` reads and unmarshals the file. Returns `false` if the file doesn't exist or is unreadable, causing `newFormModel` to fall back to hardcoded defaults.
* `configPath()` - returns `os.UserConfigDir()/subenum/last.json` (e.g. `~/.config/subenum/last.json` on Linux/macOS, `%AppData%\subenum\last.json` on Windows).
* `saveConfig(fv formValues) error` - marshals `formValues` to JSON and writes it atomically with `os.WriteFile`. Called in `beginScan()` immediately before launching the scan goroutine. Errors are silently discarded so a write failure never blocks the scan.
* `loadSavedConfig() (savedConfig, bool)` - reads and unmarshals the file. Returns `false` if the file doesn't exist or is unreadable, causing `newFormModel` to fall back to hardcoded defaults.
* **Interactions**: `tui.New()` calls `loadSavedConfig()` on startup and passes the result to `newFormModel`. The `r` keybind (new scan) also calls `loadSavedConfig()` so the form is pre-filled with the values from the scan that just completed.

## 3. Data Flow
Expand Down
6 changes: 3 additions & 3 deletions docs/DEVELOPER_GUIDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -242,8 +242,8 @@ Please follow these style guidelines when contributing:

The CLI path (`run()`) has zero external dependencies. The TUI path (`-tui` flag) adds:

- [`github.com/charmbracelet/bubbletea`](https://github.com/charmbracelet/bubbletea) Elm-architecture terminal UI framework
- [`github.com/charmbracelet/bubbles`](https://github.com/charmbracelet/bubbles) reusable TUI components (textinput, viewport, progress bar)
- [`github.com/charmbracelet/bubbletea`](https://github.com/charmbracelet/bubbletea) - Elm-architecture terminal UI framework
- [`github.com/charmbracelet/bubbles`](https://github.com/charmbracelet/bubbles) - reusable TUI components (textinput, viewport, progress bar)

If you need to add a further dependency:

Expand All @@ -258,7 +258,7 @@ If you need to add a further dependency:

Areas for potential enhancement include:

* **Terminal UI**: An interactive TUI (`-tui` flag) built with Bubble Tea. Provides a form-based config screen and a live-scrolling results view no arguments required to launch. Last-used values persist to `~/.config/subenum/last.json` across sessions.
* **Terminal UI**: An interactive TUI (`-tui` flag) built with Bubble Tea. Provides a form-based config screen and a live-scrolling results view - no arguments required to launch. Last-used values persist to `~/.config/subenum/last.json` across sessions.
* **Output Formats**: Supporting different output formats (JSON, CSV) in addition to the current plain text output file (`-o`).
* **Result Filtering**: Allowing users to filter results based on DNS record types.
* **Recursive Enumeration**: Adding support for recursive subdomain enumeration (e.g., finding subdomains of discovered subdomains).
Expand Down
2 changes: 1 addition & 1 deletion docs/_config.yml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
theme: jekyll-theme-cayman
title: subenum
description: Fast, concurrent subdomain enumeration via DNS brute-forcing written in pure Go.
description: Fast, concurrent subdomain enumeration via DNS brute-forcing - written in pure Go.
show_downloads: false
url: "https://tmhsdigital.github.io"
baseurl: "/subenum"
Expand Down
8 changes: 4 additions & 4 deletions docs/_layouts/default.html
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
<meta charset="UTF-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=1">
<title>{% if page.title == "Home" %}{{ site.title }} {{ site.description }}{% else %}{{ page.title }} {{ site.title }}{% endif %}</title>
<title>{% if page.title == "Home" %}{{ site.title }} - {{ site.description }}{% else %}{{ page.title }} - {{ site.title }}{% endif %}</title>
<link rel="stylesheet" href="{{ '/assets/css/style.css?v=' | append: site.github.build_revision | relative_url }}">
<style>
/* ── Nav bar ── */
Expand Down Expand Up @@ -58,7 +58,7 @@
padding: 5rem 2rem 4rem;
background: linear-gradient(160deg, #0d1117 0%, #161b22 60%, #1a2332 100%);
}
/* Hide Cayman's default title/tagline we render our own */
/* Hide Cayman's default title/tagline - we render our own */
.page-header .project-name,
.page-header .project-tagline { display: none; }

Expand Down Expand Up @@ -137,7 +137,7 @@
margin: 1rem 0;
}
.main-content blockquote p { margin: 0; color: #57606a; }
/* Links restore blue on top of the * reset */
/* Links - restore blue on top of the * reset */
.main-content a { color: #0969da !important; }
.main-content a:hover { text-decoration: underline; }
/* Inline code */
Expand All @@ -148,7 +148,7 @@
padding: 1px 5px;
font-size: 0.88em;
}
/* Block code distinct background from page */
/* Block code - distinct background from page */
.main-content pre {
background: #f0f2f5;
border: 1px solid #d0d7de;
Expand Down
24 changes: 20 additions & 4 deletions docs/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,12 +9,12 @@ title: Home

## What it does

subenum brute-forces subdomains by resolving a wordlist against a target domain using a concurrent worker pool. Results stream to stdout pipe-clean, no noise. Everything else (progress, diagnostics, errors) goes to stderr.
subenum brute-forces subdomains by resolving a wordlist against a target domain using a concurrent worker pool. Results stream to stdout - pipe-clean, no noise. Everything else (progress, diagnostics, errors) goes to stderr.

<div class="screenshot-wrap">
<figure>
<img src="assets/tui-form.png" alt="subenum TUI Configure Scan">
<figcaption>Interactive TUI launch with <code>./subenum -tui</code> or <code>make tui</code></figcaption>
<img src="assets/tui-form.png" alt="subenum TUI - Configure Scan">
<figcaption>Interactive TUI - launch with <code>./subenum -tui</code> or <code>make tui</code></figcaption>
</figure>
</div>

Expand All @@ -37,7 +37,23 @@ subenum brute-forces subdomains by resolving a wordlist against a target domain
</div>
<div class="feature-card">
<strong>Simulation Mode</strong>
<span>Generate synthetic DNS results at a configurable hit rate — zero network I/O. Safe for demos and testing.</span>
<span>Generate synthetic DNS results at a configurable hit rate - zero network I/O. Safe for demos and testing.</span>
</div>
<div class="feature-card">
<strong>Output Formats</strong>
<span>Emit results as <code>text</code>, <code>json</code> (array of subdomain plus typed records), or <code>csv</code> via <code>-format</code>.</span>
</div>
<div class="feature-card">
<strong>Rate Limiting</strong>
<span>Cap total DNS queries per second across the worker pool with <code>-rate</code> (context-aware, stays responsive to Ctrl+C).</span>
</div>
<div class="feature-card">
<strong>Record Types</strong>
<span>Look up and filter by <code>A</code>, <code>AAAA</code>, or <code>CNAME</code> records with <code>-type</code>.</span>
</div>
<div class="feature-card">
<strong>Recursive Enumeration</strong>
<span>Enumerate subdomains of discovered subdomains with <code>-recursive</code> and a <code>-depth</code> cap, with loop and duplicate protection.</span>
</div>
<div class="feature-card">
<strong>Pipe-Friendly Output</strong>
Expand Down
56 changes: 56 additions & 0 deletions internal/output/writer_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -100,6 +100,62 @@ func TestWriterResultJSONFile(t *testing.T) {
}
}

// TestWriterJSONFinishGatesOutput locks in the contract main.run relies on:
// structured output is emitted only by Finish, so skipping Finish on an error
// path produces no spurious empty JSON array.
func TestWriterJSONFinishGatesOutput(t *testing.T) {
// Error path: results buffered (or none) but Finish never called.
noFinish, err := os.CreateTemp("", "output-json-nofinish-*.json")
if err != nil {
t.Fatal(err)
}
defer func() { _ = os.Remove(noFinish.Name()) }()

bw := bufio.NewWriter(noFinish)
_ = New(bw, false, FormatJSON)
if err := bw.Flush(); err != nil {
t.Fatal(err)
}
if err := noFinish.Close(); err != nil {
t.Fatal(err)
}
content, err := os.ReadFile(noFinish.Name())
if err != nil {
t.Fatal(err)
}
if len(content) != 0 {
t.Errorf("expected no structured output without Finish, got:\n%s", content)
}

// Success path: Finish with zero results emits an empty JSON array.
withFinish, err := os.CreateTemp("", "output-json-finish-*.json")
if err != nil {
t.Fatal(err)
}
defer func() { _ = os.Remove(withFinish.Name()) }()

bw2 := bufio.NewWriter(withFinish)
w := New(bw2, false, FormatJSON)
w.Finish()
if err := bw2.Flush(); err != nil {
t.Fatal(err)
}
if err := withFinish.Close(); err != nil {
t.Fatal(err)
}
content, err = os.ReadFile(withFinish.Name())
if err != nil {
t.Fatal(err)
}
var results []Result
if err := json.Unmarshal(content, &results); err != nil {
t.Fatalf("Finish output is not a valid JSON array: %v\nGot:\n%s", err, content)
}
if len(results) != 0 {
t.Errorf("expected empty array from Finish with no results, got %d", len(results))
}
}

func TestWriterResultCSVFile(t *testing.T) {
tmp, err := os.CreateTemp("", "output-csv-*.csv")
if err != nil {
Expand Down
9 changes: 5 additions & 4 deletions main.go
Original file line number Diff line number Diff line change
Expand Up @@ -285,10 +285,6 @@ func run() int {
}
}()
}
// Finish runs before the file flush/close defer above (LIFO), so buffered
// JSON and streamed CSV are written before the file is closed.
defer out.Finish()

if f.verbose {
logVerboseStart(f, domain, maxAttempts, out)
}
Expand Down Expand Up @@ -370,6 +366,11 @@ func run() int {
}
}
}
// Finalize structured output only on the success path, so an early error
// (such as wildcard detection without -force) does not emit an empty JSON
// array or a bare CSV header. The deferred file flush/close registered
// above runs after this, persisting buffered output before the file closes.
out.Finish()
return 0
}

Expand Down
Loading