StreetEasy Scout

The problem

Good rental listings in New York get dozens of inquiries within hours, so staying competitive means checking StreetEasy constantly and writing outreach messages fast. StreetEasy also sits behind aggressive bot protection, which rules out a simple scraper, and polling naively would waste money on extraction credits for listings already seen or acted on.

The build

Scout, an agent in my personal automation suite (Hermes) that runs the whole loop autonomously: it monitors my configured StreetEasy searches on an hourly cron, discovers new listings, validates them against budget and location criteria, composes a paste-ready outreach message tailored to each listing, and alerts me on Telegram. It reconciles its own discoveries against StreetEasy’s alert emails via Gmail so nothing slips through either channel, and it sends regular heartbeats so I know it’s running even when nothing matches.

How it works

The agent is a Python package with separated modules for discovery, fetching, listing parsing, criteria, message composition, contact handling, and state, backed by SQLite and exposed through a small FastAPI service with a React + TypeScript dashboard. Listings are fetched through Firecrawl’s structured extraction, which handles StreetEasy’s PerimeterX bot protection.

Most of the engineering effort went into cost control and reliability: extraction credits are only spent on listings that haven’t already been actioned, the cron skips a 1–5am window when new listings are rare, and the pipeline is covered by a pytest suite of about 20 test files, since automation that acts on my behalf has to be correct.

Outcomes

Scout replaced constant manual checking with reviewing pre-validated matches that arrive with outreach already drafted. It’s also my most complete example of agent engineering: autonomous operation on a schedule, integration across scraping, email, and Telegram, cost management against an external service, and the test coverage unattended automation requires.