Why app.tryhoma.com/property/* is getting hammered (RapidAPI + OpenAI spend, UX latency), what Meta's own docs say about it, and the clean fix. It's not a rogue campaign — it's Meta's dynamic-ads catalog crawler doing exactly what it's designed to do, pointed at our most expensive route.
meta-externalads/1.1 crawler validating our catalog's property URLs. It's standard DPA behavior, not a misconfigured ad — it re-checks each listing's landing page for availability / price / link health./property/{id} render fires a live RapidAPI/AnyProp lookup + OpenAI call. For most advertisers a catalog crawl hits cheap static pages; for us each hit burns real quota + CPU.meta-externalads/1.1 on the property routes and serve it a cached / lightweight response — skip the live RapidAPI + OpenAI calls. Validation still succeeds, ads stay healthy, cost → ~0. Cache-by-user-agent; do not block (blocking breaks DPA delivery).Meta re-fetches the landing URL of every catalog item to keep dynamic ads accurate — real-estate listings sell fast, so it re-validates constantly (availability, price, broken-link/policy checks, Open-Graph/microdata, image resolution, and pixel-ID↔catalog matching). Normal, documented behavior for any Advantage+ catalog.
The volume, measured from our own analytics (Meta-datacenter IPs = clean crawler proxy):
So there are two triggers: (a) feed re-ingestion — every scheduled refresh / manual rebuild makes Meta re-crawl the ~16k product URLs in a burst (the spikes above); and (b) autonomous validation — Meta re-checks active items on its own cadence regardless of the feed (the ~15/day trickle). (a) is now paused. (b) can't be switched off while the catalog + ads exist — hence the cache.
Meta runs five distinct crawlers, each with its own user-agent. Identify ours by UA:
| User-agent | Purpose (per Meta) |
|---|---|
meta-externalads/1.1 OURS | Enhances advertising & business products — this is the DPA / catalog validation crawler hitting the property routes |
facebookexternalhit/1.1 | Link previews / shared-content metadata |
meta-webindexer/1.1 | AI search result quality |
meta-externalagent/1.1 | Training foundation AI models |
meta-externalfetcher/1.1 | Agentic AI tasks (website navigation) |
Full UA string for ours looks like:
meta-externalads/1.1 (+https://developers.facebook.com/docs/sharing/webmasters/crawler)
robots.txt with standard directives.robots.txt for extended periods.facebookexternalhit and meta-externalfetcher may bypass robots.txt for security checks and user-initiated requests. But meta-externalads (ours) is the standard, robots-respecting crawler.webmasters@meta.com.Disallow on the property routes for meta-externalads would stop legitimate catalog validation — Meta would start flagging listings as broken/unavailable and could throttle DPA delivery. We want the crawler to succeed cheaply, not fail.Handle meta-externalads/1.1 explicitly on the property (and any catalog-linked) routes so it validates without touching the expensive read path:
// pseudo — property route handler
const ua = req.headers['user-agent'] || '';
if (ua.includes('meta-externalads') || ua.includes('facebookexternalhit')) {
// serve cached/lightweight property data ONLY:
// - skip live RapidAPI / AnyProp lookup
// - skip OpenAI call
// - return last-known cached fields + OG/microdata tags
// (title, price, availability, image) so Meta can validate
return renderCachedProperty(id); // cheap, no third-party calls
}
return renderProperty(id); // full live path for real users
If a cache isn't ready immediately, a per-UA rate limit / 304 Not-Modified on those routes is a reasonable interim (serve Last-Modified so Meta short-circuits re-fetches).
schedule=NONE → no more scheduled re-ingest → the burst crawls (Jul 27/29 style) stop. Expect to fall back to the ~15/day baseline.MUST_FIX "products not being shown" diagnostic as listings go stale, and Meta may email the catalog admin that the scheduled feed stopped. This is intentional — it's the stopgap tradeoff, not a break.