Skip to main content

Running the nightly crawl from a phone

The crawl has to run on the laptop: YouTube captions need its residential connection and VPN, and the build needs the local warehouse on port 5433. /nightly-crawl runs every step there, and Claude Code's Remote Control lets you start and watch it from your phone (#395).

In short​

  1. At the laptop, in the WSL terminal: cd into the repo, then claude remote-control --name "open-navigator WSL". Leave it open.
  2. On your phone: Claude app → Code → the session with the green dot.
  3. Type /nightly-crawl. Add skip-youtube to skip the YouTube step, or publish-only to only sync and check.

It reports one line per step and stops before publishing if anything fails. The Neon sync refuses to go past 480 MB (size check).

First-time setup
  • Claude Code 2.1.51 or later in WSL (claude --version), signed in with the claude.ai account. Install or update: https://code.claude.com/docs/en/setup
  • Remote Control everywhere (optional): in Claude Code, /config → "Enable Remote Control for all sessions".
  • Keep the laptop reachable: Windows set to never sleep while plugged in, the VPN connected, and the terminal left open (closing it ends the WSL session). A session ends after about ten minutes offline.
  • YouTube: PROXY_USER_NAME and PROXY_PASSWORD (Webshare) in .env, and a youtube_cookies.txt less than a week old. If the VPN runs on Windows, a proxy inside WSL is at the Windows host IP, not 127.0.0.1.
Why not a cloud scheduled task

A cloud scheduled task can link to the laptop, but its shell is a sandboxed Linux machine: it sees only shared folders, can't reach services on the laptop's localhost (the warehouse), and its internet goes through an allow-list, not the VPN. Remote Control runs Claude Code in WSL itself, so it has everything the terminal has.

What the steps are

The command file is .claude/commands/nightly-crawl.md:

  1. Preflight (clean tree, warehouse, proxy and cookies).
  2. Tuscaloosa agendas (SuiteOne).
  3. Epoch AI data center sites.
  4. The YouTube crawl and Gemini analysis, with the careful defaults.
  5. The dbt rebuild.
  6. The Neon publish under the budget.
  7. A check of the live data center endpoints.