Reproduction instructions

# Republican campaign website archive study

Open **dashboard.html** for the interactive graph. Read **REPORT.md** for current findings and collection status, and **METHODOLOGY.md** for the counting and inclusion rules.

All research data, archives, source material, scripts, and charts are stored in this directory. The roster is frozen as of October 14, 2026; the six months are April–September 2026. Missing observations are retained as missing, not counted as zero.

## Main outputs

| File | Contents |
|---|---|
| `data/candidates.csv` | Candidate list, race, party/ballot labels, websites, and sources |
| `data/website_gaps.csv` | Candidates for whom no standalone campaign site was identified |
| `data/monthly_metrics.csv` | Every candidate × month; counts, words, rates, and coverage |
| `data/page_metrics.csv` | Every selected page version; results and local evidence paths |
| `data/homepage_april_september_changes.csv` | Fixed-endpoint homepage changes |
| `data/matched_page_april_september_changes.csv` | Comparisons of the same page paths at both endpoints |
| `charts/all_candidates.png` | Full-height heatmap with every candidate |
| `charts/all_candidates.pdf` | Readable paginated export of all candidates |
| `charts/collected_site_pages.pdf` | Broader collected-page totals, with partial coverage marked |
| `charts/overview.png` | Coverage and balanced-panel trend |
| `assets/snapshots/` | Original HTML, extracted text, context excerpts, and provenance |
| `assets/cdx/` | Raw Wayback archive indexes and request metadata |
| `assets/sources/` | Saved roster sources and reference material |
| `assets/run_status.json` | Current automated collection phase, when the runner is active |

The dashboard opens directly from disk; no web server is needed. Its counts are embedded at build time. Reload it after an automatic refresh to see new results. Links to CSV, text, HTML, and archive sources work from this folder. The original archived HTML may depend on remote assets; use the extracted text to review the exact counted material offline.

While collection is running, the dashboard and CSVs refresh approximately every four minutes. Static PNG/PDF exports refresh after the homepage pass and again after the full page collection. Consult the current report and `assets/run_status.json` for coverage.

The background collector runs locally and needs the computer and network to remain available. An idle-sleep inhibitor runs only while the collector is active; closing the laptop or shutting it down can still interrupt collection. Cached work is retained for resuming.

## Reproduce or resume

The existing `.venv` uses the local Python environment plus the dependencies in `requirements.txt`. To recreate it:

```sh
python3 -m venv --system-site-packages .venv
.venv/bin/python -m pip install -r requirements.txt
```

Rebuild the roster from the saved sources and documented corrections:

```sh
.venv/bin/python scripts/roster.py
```

The collection is resumable. Successful source indexes and downloads are cached. Failed requests are retried only with `--retry-errors`; already retrieved frozen pages that lack static text or contain unrelated content remain documented failures:

To resume the complete automated pipeline, including periodic dashboard updates and final validation, run this from the project directory after the current process has stopped:

```sh
caffeinate -i .venv/bin/python -u scripts/run_study.py --all-pages --replay-workers 4
```

The runner's file lock prevents duplicate automatic runs. Four replay workers share the same overall request-rate limit. Individual phases can also be run separately:

```sh
.venv/bin/python scripts/collect.py resolve
.venv/bin/python scripts/collect.py index --workers 2 --retry-errors
.venv/bin/python scripts/collect.py select
.venv/bin/python scripts/collect.py download --homepage-only --workers 2 --retry-errors
.venv/bin/python scripts/collect.py download --workers 2 --retry-errors
```

The final command collects every selected HTML page. This can take considerably longer than the homepage pass; Wayback rate limits and archive failures are retained in the logs. Do not start duplicate collectors while one is already running.

Recompute and rebuild all outputs without fetching anything:

```sh
.venv/bin/python scripts/reprocess.py
.venv/bin/python scripts/analyze.py
.venv/bin/python scripts/render.py
.venv/bin/python scripts/write_report.py
.venv/bin/python scripts/validate_results.py
.venv/bin/python -m unittest discover -s tests -v
```

`data/roster_amendments.json` holds sourced additions and website corrections. CSVs should be treated as generated outputs. JSON files retain the same information with structured arrays. No credentials are needed for this study.
# Public dashboard

The hosted dashboard is at trump-mentions.dangerously-skipped-permissions.com. Cloudflare rejected the originally requested underscore in the hostname; the published hostname uses a hyphen.

The public copy refreshes hourly while the local publication watcher and collection process run. The page shows its dataset timestamp and outstanding archive coverage. The full archive payloads remain in this folder; hosted evidence links provide the counted text and the Wayback HTML capture. See deployment instructions in the local project (CLOUDFLARE.md) for publishing updates and checking status.