Agentic Apartment Rental Scraper
2024 · smolagents · Selenium · Python · WhatsApp API · GitHub Actions
Business Problem
Finding an apartment in a competitive rental market means checking multiple sites several times a day. Listings move fast and manual checking is easy to miss or forget. I was working through the HuggingFace agents course at the time and wanted a real, immediately useful project to build alongside it.
Solution
smolagents orchestrates a Selenium scraper across rental sites. Every morning a GitHub Actions cron job triggers the pipeline: it fetches new listings, filters against criteria like location, price, and size, checks a local seen-listings store to avoid duplicates, and sends matches over WhatsApp rather than email so alerts surface where attention already is. The pipeline runs without any local machine being on. The project started as a course exercise and became the tool I used to find my apartment.
Architecture
Technical Challenges
Tool output contracts. The scraper tool needed to return structured data, not raw HTML. Once every tool returned clean, typed output the agent's decisions became much more predictable. The lesson was that tool design matters more than agent design.
Selenium on dynamic listing pages. Rental sites use dynamic rendering. Explicit waits and retry logic inside the scraper tool were necessary because the agent has no way to recover from a tool that silently returns partial results.
State persistence between runs. GitHub Actions runners are ephemeral. The seen-listings store is a JSON file committed back to the repo at the end of each run, so the next day's run knows what was already sent. Simple and free, with a full history in git.
MVP Status
Personal project, used to find one apartment. The core loop of scrape, filter, and alert works end to end and runs daily on GitHub Actions.
Next Steps
- Extend to more rental sites
- Add price history tracking per listing
- Parameterize search criteria without touching the code