ENTRY 001 · 2026-09-04
Self-hosted Firecrawl, and this blog
Today's TL;DR:
- Self-hosted Firecrawl on the Hetzner box: Docker Compose stack (api, playwright, redis, rabbitmq, nuq-postgres), bound to the Tailscale IP only, bearer-token auth. The new harness respects
NUQ_DATABASE_URL/NUQ_RABBITMQ_URL— set them and it skips Docker-in-Docker. - pi-web-access wiring:
firecrawlBaseUrlin~/.pi/web-search.json, plusssrf.allowRanges: ["100.64.0.0/10"]because pi's SSRF guard blocks the Tailscale CGNAT range by default. - Playwright engine gotcha:
PROXY_SERVER: Noneis treated as a literal proxy host → every navigation fails withERR_TUNNEL_CONNECTION_FAILED. Leave the env unset. - Firecrawl vs Exa: ~1.8s vs ~0.9s per search; the right split is Exa as search default, Firecrawl as the fetch fallback for blocked/JS-heavy pages.
- OpenBao is state of the art for self-hosted data-at-rest key management (transit engine = envelope encryption), but it's a key oracle — pair it with native disk/DB encryption.
- This blog: ~200 lines of Go + goldmark, posts are markdown files scp'd into
/opt/blog/posts, served by systemd, fronted by Traefik with Cloudflare DNS-01 TLS. No build step, no database.