Memo's observability stack is owned, layered, and explainable. Nothing routes through a third party at the data layer — every event, derived signal, score, and surface row lives in your own Postgres + Mongo. Third-party recording (rrweb / Clarity / LogRocket) plugs in alongside but never gates the analytics path.
This page documents every piece in detail. If you're trying to understand "where does this number on the Activity tab come from", this is the page.
┌─────────────────────────────────────────────────────────────┐
│ Visitor browser │
│ - meta[name="memo-visit"], meta[name="memo-slug"] │
│ - public/beacon.js (IntersectionObserver, scroll, click) │
└──────────────┬──────────────────────────────────────────────┘
│ navigator.sendBeacon (11 event types)
▼
┌─────────────────────────────────────────────────────────────┐
│ /api/beacon (Astro APIRoute on Vercel Node) │
│ - Validates JWT cookie + rate-limits │
│ - Persists raw events to Mongo time-series │
│ - On page_leave: finalises visits row, computes │
│ engagement_score, emits per-visit signals │
│ (skim / deep_read), fires visit.completed webhook │
└──────────────┬──────────────────────────────────────────────┘
│
┌───────────┴───────────┐
▼ ▼
┌──────────────────┐ ┌────────────────────────────────────────┐
│ Mongo │ │ Postgres │
│ events (TS) │ │ visits, visit_signals, links, projects │
│ — 90-day TTL │ │ — relational, indexed, append-mostly │
│ — 11 event types │ │ — visit_signals = derived │
└────────┬─────────┘ └──────────┬─────────────────────────────┘
│ │
│ ┌─────────────────┴──────────────────────────┐
│ │ /api/cron/derive-cross-visit-signals │
│ │ every 10 min │
│ │ reads visits (last 72h) + persistent_ids │
│ │ classifies `return` and `share_detected` │
│ │ upserts into visit_signals (idempotent) │
│ └─────────────────┬──────────────────────────┘
│ │
▼ ▼
┌─────────────────────────────────────────────────────────────┐
│ Owner surfaces (SSR + React islands) │
│ - /u/<u>/projects/<p>/activity Activity tab │
│ - /u/<u>/projects/<p>/performance Performance + sections│
│ - /u/<u>/people/<emailHash> Cross-link timeline │
│ - SectionHeatmap, ChangesPanel, SearchPanel, EditorTabs │
│ Live polling: /api/owner/projects/<p>/activity/since │
└─────────────────────────────────────────────────────────────┘
Five moving pieces — beacon, Mongo, Postgres, cron, owner UI — each independently testable. The contract between them is the event taxonomy in the next section.
Every beacon event is one of these. The enum lives in two places: src/pages/api/beacon.ts (zod validation at ingress) and src/lib/db/mongo.ts (TypeScript shape on the time-series doc). They must stay in lock-step or the endpoint will 400 valid events.
| Type | Fires when | Required payload fields | Optional payload fields |
|---|---|---|---|
page_view | Beacon initialises (script load on /<slug>/). One per page load. | path (string), w (number, innerWidth), h (number, innerHeight), title (string), referrer (string) | — |
scroll_depth | Throttled scroll (250ms) where the new max depth percentage > previous max. Only fires on increases. | pct (number, 0–100) | — |
click_zone | Any click event bubbled to document (capture phase). | tag (string, lowercased element name), zone (string, element id if present else tag) | section (string, the nearest enclosing [data-memo-section] ancestor's id, or null) |
focus_change | visibilitychange event on document. Also triggers a flush() when becoming hidden so events don't get lost. | hidden (boolean) | — |
section_enter | IntersectionObserver reports an element matching [data-memo-section] has crossed 50% visibility. One enter per section per visit lifecycle (we track open state and dedupe). | section (string, the data-memo-section attribute value), firstView (boolean, true if the visitor hadn't entered this section before this page load) | — |
section_exit | The corresponding IntersectionObserver reports the element leaving the viewport (intersectionRatio < 0.5 AND we had an open enter). Also synthesised on pagehide for any still-open sections. | section (string), ms (number, dwell time in milliseconds) | ratio (number, the intersection ratio at exit, percent), final (boolean, true if forced by pagehide) |
gate_passed | Reserved. Not currently emitted by beacon.js; the enum is held for future fine-grained gate analytics (today the visit.opened / visit.completed webhooks cover this layer). | — | — |
download | Reserved. Not currently emitted by beacon.js; held for when allow_downloads is on and we instrument file downloads. | path (string, R2 file path) | — |
page_leave | pagehide event on window. Final event of every visit. Triggers a flush() immediately after enqueueing. | depth (number, the visit's max scroll depth percentage) | — |
media_progress | A MutationObserver detects a <video>/<audio> element and listens to its timeupdate events; emitted as playback progresses. Reaching ≥80% emits the video_watched behavioural signal. | element (string, e.g. video_0), currentTime (number, seconds), duration (number, seconds), percent (number, 0–1) | — |
Batching and transport (in public/beacon.js):
setInterval, on visibilitychange when becoming hidden, on pagehide.navigator.sendBeacon() (the browser will deliver even if the tab is closing); falls back to fetch({ keepalive: true }) if sendBeacon is unavailable or throws.{ slug, visit, events: [{ type, payload, t }] } where t is the client's Date.now() (used for ordering; the server overrides with its own Date.now() for the canonical ts field — beacon's clock is untrusted).Identity at the beacon layer:
The browser reads two <meta> tags injected by injectVisitorChrome (in src/lib/visitor/watermark.ts):
<meta name="memo-visit" content="<visit-uuid>"> — same UUID as the gate JWT's sub claim and as the memo.visits.id row.<meta name="memo-slug" content="<slug>"> — the link slug, used so the beacon doesn't need to parse location.pathname. This is load-bearing: visitor pages live at /<slug>/, so URL-position parsing would slot the wrong substring under the new URL scheme and silently drop every event.If either meta is missing, the beacon refuses to instrument (early return). This is intentional — a leaked HTML file served outside memo shouldn't accidentally activate the beacon and POST to /api/beacon.
Memo authors mark sections in their published HTML:
<section data-memo-section="hero">…</section>
<section data-memo-section="problem">…</section>
<section data-memo-section="financials">
<section data-memo-section="financials-table">…</section>
</section>
The beacon does:
const io = new IntersectionObserver(handler, { threshold: [0, 0.5, 1] });
document.querySelectorAll('[data-memo-section]').forEach(el => io.observe(el));
The handler runs on every threshold cross. The dedupe rules:
e.isIntersecting && e.intersectionRatio >= 0.5 AND there is no currently-open enter for that section id. So a section that already entered, briefly dipped below 50%, and came back to 50% will record an exit then a new enter with a fresh firstView=false.ms = Date.now() - enterTime.pagehide is force-closed with final: true and its computed dwell.Nesting is supported. Two [data-memo-section] elements can overlap (parent contains child); they're tracked as independent lifecycles. This lets an author write:
<section data-memo-section="appendix">
<section data-memo-section="appendix-financials">…</section>
<section data-memo-section="appendix-team">…</section>
</section>
and get both the parent's aggregate dwell AND the per-subsection dwells without instrumentation conflict.
What gets tracked:
| Datum | Where it lives |
|---|---|
| Distinct section ids entered | section_enter events in Mongo + engagement_score breadth signal in Postgres |
| Dwell per visit per section | section_exit events in Mongo (payload.ms) |
| Aggregate dwell per section across all visits | Computed on demand by /api/owner/projects/<id>/analytics/sections (Mongo find + JS percentile) |
| Click attribution to section | click_zone events whose payload.section is non-null (the click bubbled from inside a [data-memo-section] ancestor) |
Heatmap rendering: the Performance tab's "Section engagement" island calls the aggregation endpoint and renders a table with one row per section id, sorted by avg dwell descending. The "heat" column is a CSS-only horizontal bar with width = (section.avgMs / peak.avgMs) × 100%, gradient yellow→red.
Author tips:
payload.section string equality.position: fixed ancestor — IntersectionObserver works against the viewport but with fixed positioning the math can be misleading.hero / problem / financials / cta. Avoid slide-1 / slide-2 — those shift meaning across deck rewrites and break historical aggregations.events collection#Connection (in src/lib/db/mongo.ts):
const client = new MongoClient(MONGODB_MEMO_URI, { maxPoolSize: 10 });
const db = client.db("memo");
The connection pool is global to the serverless function instance, lazily initialised on first call to eventsCollection(). Subsequent invocations within the same warm function reuse it.
Collection creation (idempotent, runs on first read/write per warm container):
await db.createCollection("events", {
timeseries: {
timeField: "ts",
metaField: "visit_id",
granularity: "seconds",
},
expireAfterSeconds: 90 * 24 * 3600,
});
timeField: "ts" — the server-stamped Date at ingest time.metaField: "visit_id" — Mongo clusters documents with the same visit_id together. This is what makes single-visit queries (replay reconstruction, per-visit aggregations) cheap.granularity: "seconds" — Mongo bucketises into ~1-hour buckets for seconds granularity. Trade-off between bucket size + query precision.expireAfterSeconds on the time-series collection; Mongo evicts buckets whose max ts is older than 90d.Document shape:
{
ts: ISODate, // server-stamped, NOT client clock
visit_id: "uuid", // metaField — clusters one visitor together
link_id: "uuid", // denormalised so single-collection queries work
project_id: "uuid", // denormalised so cross-link queries are cheap
type: <one of the 11 enum values>,
payload: { ... } // per-type shape, see Event taxonomy
}
Indexes:
Implicit cluster on (visit_id, ts) via the time-series metaField. Range queries on a single visit are essentially a sequential scan of one bucket.
Recommended additional index (not auto-created, run once per environment for fast section/category aggregation):
db.events.createIndex({ project_id: 1, type: 1, ts: -1 });
This makes the /api/owner/projects/<id>/analytics/sections aggregation (find({ project_id, type: "section_exit" })) O(matches) instead of a partial scan.
Sample queries:
// All events for one visit, ordered:
db.events.find({ visit_id: "<uuid>" }).sort({ ts: 1 })
// Section dwell distribution across a project's last 7 days:
db.events.aggregate([
{ $match: {
project_id: "<uuid>",
type: "section_exit",
ts: { $gte: ISODate("<seven days ago>") }
}},
{ $group: { _id: "$payload.section", total: { $sum: "$payload.ms" } } }
])
// Per-visit funnel: every page_view/leave pair, with depth:
db.events.find(
{ visit_id: "<uuid>", type: { $in: ["page_view", "page_leave"] } },
{ type: 1, ts: 1, "payload.depth": 1 }
)
memo.visits is the canonical "this person entered the deck" row. Every column, every populating site:
| Column | Type | Set by | Populated when |
|---|---|---|---|
id | uuid PK | claims.sub from the gate JWT | First view render after gates pass; insert in [slug]/index.astro with ON CONFLICT DO NOTHING |
link_id | uuid FK→links | loaded.link.id | Same insert |
visitor_email | citext | claims.email | Same insert (NULL if requireEmail=false and the visitor didn't volunteer one) |
ip | inet | Astro.clientAddress | Same insert |
ua | text | request.headers.get("user-agent") | Same insert |
country | text | request.headers.get("x-vercel-ip-country") | Same insert. NULL off-Vercel. |
city | text | decodeURIComponent(request.headers.get("x-vercel-ip-city")) | Same insert. URL-decoded (Vercel encodes spaces as %20). NULL off-Vercel. |
started_at | timestamptz | now() default | Same insert |
ended_at | timestamptz | now() | First page_leave beacon event triggers a UPDATE … WHERE ended_at IS NULL (idempotent) |
duration_ms | integer | Date.now() - startedAt | Same page_leave handler |
completion_pct | integer | max(payload.depth) from the page_leave batch | Same page_leave handler |
logrocket_session_url | text | LogRocket SDK callback POSTed to /api/recording/logrocket | When the LogRocket recorder finishes initialising client-side and reports its session URL |
clarity_recording_id | text | Reserved | (Clarity is project-scoped, so this column is usually NULL) |
visitor_persistent_id | uuid | The memo_vp cookie's value, minted via crypto.randomUUID() if absent | Same insert as the rest. Cookie is Path=/, 365-day rolling TTL, HttpOnly, SameSite=Lax, Secure in HTTPS. |
engagement_score | integer | Computed formula (see below) | Same page_leave handler, in the same UPDATE that sets duration_ms |
Indexes:
visits_link_idx on (link_id) — Activity tab queriesvisits_started_idx on (started_at) — recency-sorted queriesvisits_persistent_idx on (visitor_persistent_id) — cross-session stitching (e.g. "all visits by this browser")Computed in /api/beacon inside the page_leave handler, in the same UPDATE that sets ended_at + duration_ms:
score = completionPct × 5
+ round(log10(durationSec + 1) × 100)
+ min(20, sectionsEntered) × 25
Where:
completionPct ∈ [0, 100] — max scroll depth from page_leave eventsdurationSec = max(1, round(durationMs / 1000))sectionsEntered = count of distinct payload.section strings from section_enter events in the same batchComponent ranges (approximate):
| Component | Range | Intent |
|---|---|---|
| Completion contribution | 0..500 | "How much did they see?" — linear in scroll depth |
| Duration contribution | 0..~300 (300 ≈ 28 days, capped by log) | "How long did they stay?" — log-scaled so a forgotten tab doesn't dominate |
| Section breadth contribution | 0..500 (caps at 20 sections × 25) | "How many distinct sections did they engage with?" |
Total: roughly 0..1300 for realistic visits.
Worked example — visitor spent 4 minutes, scrolled to 87%, entered 6 distinct sections:
87 × 5 = 435round(log10(240 + 1) × 100) = round(2.382 × 100) = 238min(20, 6) × 25 = 150Activity tab can sort by score descending to surface hottest leads first (?sort=score on the activity page URL).
Why this formula:
Two distinct cookies, two distinct ID lifecycles:
| Concern | Gate JWT (memo_v_<slug>) | Persistent visitor (memo_vp) |
|---|---|---|
| Purpose | "Has this visitor passed this link's gates?" | "Have I seen this browser before, across any link?" |
| Cookie name | memo_v_<slug> (per-link) | memo_vp (global, Path=/) |
| Cookie value | Signed JWT (HS256, daily-rotated key) with { sub, slug, email?, email_verified?, password_passed?, nda_signed? } | Raw UUID v4 (validated against the canonical UUID regex on read) |
| Lifetime | 24 hours (Max-Age=86400) | 365 days, rolling (refreshed on every successful render) |
| Stored as | Postgres visits.id (= claims.sub) | Postgres visits.visitor_persistent_id |
| Cardinality per email | Many (a new sub each cookie cycle) | Few (one per browser/device the email visits from) |
| Identifies | One session (continuous engagement within 24h) | One browser (across sessions, devices stay separate) |
| Use cases | Gate state machine, beacon attribution, replay grouping | "Nth visit by this browser", returning-visitor signal, "share_detected" cohort |
Example timeline:
Day 1, 09:00 Visitor enters email + password.
memo_v_acme → JWT(sub=u1, email=amit@..., password_passed=true)
memo_vp → vp1 (new)
visits row: id=u1, visitor_email=amit@..., visitor_persistent_id=vp1
Day 1, 14:00 Same browser, same tab opened. Cookie still valid.
Insert is ON CONFLICT DO NOTHING on (id=u1) → no new row.
Beacon events append to existing visit u1.
Day 2, 09:00 Cookie expired (>24h). Gate again.
memo_v_acme → JWT(sub=u2, …) ← new sub
memo_vp → vp1 (still valid, refreshed) ← SAME persistent id
visits row: id=u2, visitor_persistent_id=vp1
Day 5, 09:00 Same browser, fresh gate cycle.
memo_v_acme → JWT(sub=u3, …)
memo_vp → vp1
visits row: id=u3, visitor_persistent_id=vp1
✅ Now `derive-cross-visit-signals` (running every 10 min) sees three
visits with the same vp1 → classifies the latest as `return`.
Privacy notes:
memo_vp minted). This is expected behaviour.HttpOnly so JS can't read or modify it.Four classifications, two per-visit + two cross-visit. All live in memo.visit_signals (one row per (visit_id, kind), unique-indexed).
skim (per-visit, fires at page_leave)#Intent: "Visitor scrolled fast through the deck without lingering."
Thresholds (classifySkim in src/lib/analytics/derive-signals.ts):
completionPct ≥ 80 ANDdurationMs < 30_000 (under 30 seconds total) ANDsectionsEntered ≤ 2 (engaged with 0, 1, or 2 distinct sections)Why all three required: any one alone is too noisy. A 28-second visit might be normal for a 1-pager. A skim is high-completion + short-duration + low-breadth.
Payload:
{
"reason": "low_dwell_high_scroll",
"durationMs": 18432,
"completionPct": 91,
"sectionsEntered": 1
}
deep_read (per-visit, fires at page_leave)#Intent: "Visitor lingered meaningfully on multiple sections."
Thresholds:
sectionsEntered ≥ 5 ANDdurationMs ≥ 3 × 60_000 (≥3 minutes total) AND≥ 45_000 msAverage per-section dwell = sum(section_exit.payload.ms) / count(section_exits). This filters out "two sections each got 180s because the visitor walked away from the laptop" cases.
Payload:
{
"reason": "long_dwell_broad_coverage",
"durationMs": 540000,
"avgSectionMs": 67500,
"sectionsEntered": 8
}
return (cross-visit, fires by cron)#Intent: "Same browser came back 3+ times."
Thresholds (classifyReturns):
visitor_persistent_id appears in ≥3 visits (any project, any link of the owner).Emitted against: the most recent of the qualifying visits (so the badge appears in the Activity feed at the right moment).
Payload:
{
"reason": "n_visits_by_persistent_id",
"persistentId": "<uuid>",
"visitCount": 4,
"firstVisitAt": "2026-05-28T11:32:00.000Z"
}
share_detected (cross-visit, fires by cron)#Intent: "≥2 distinct emails from the same domain visited within a 72-hour window — classic 'investor forwarded the link internally' pattern."
Thresholds (classifyShareDetected):
visitor_email.split("@")[1] (lowercased).Payload:
{
"reason": "multiple_emails_same_domain_recent",
"domain": "acme.com",
"emails": ["amit@acme.com", "raj@acme.com"],
"withinHours": 72
}
visit_signals table#Schema (memo.visit_signals — see Data Model):
CREATE TABLE memo.visit_signals (
id bigserial PRIMARY KEY,
visit_id uuid NOT NULL REFERENCES memo.visits(id) ON DELETE CASCADE,
kind text NOT NULL, -- skim | deep_read | return | share_detected
payload jsonb, -- the classifier's explanation
created_at timestamptz NOT NULL DEFAULT now()
);
CREATE UNIQUE INDEX visit_signals_visit_kind_idx
ON memo.visit_signals (visit_id, kind);
CREATE INDEX visit_signals_kind_idx
ON memo.visit_signals (kind);
The unique index on (visit_id, kind) enables ON CONFLICT DO NOTHING (per-visit, where we don't want to overwrite an earlier classification) and ON CONFLICT DO UPDATE (cross-visit, where we want to refresh payload if the classifier sees a new threshold).
Why open-ended kind: future signal kinds (e.g. re_read, night_owl, decisionmaker_pattern) add a row, not a migration. The reading code in the Activity tab and people page handles unknown kinds gracefully (renders a default chip).
Endpoint: /api/cron/derive-cross-visit-signals (file: src/pages/api/cron/derive-cross-visit-signals.ts)
Schedule: */10 * * * * (every 10 minutes), wired in vercel.json alongside the other crons.
Pseudocode of one run:
since = now() - 72h
SELECT id, visitor_email, started_at, visitor_persistent_id FROM memo.visits WHERE started_at >= since;
Build in-memory: visitsByPersistent: Map<persistent_id, [{ visitId, startedAt }]> and recentVisitsByDomain: Map<domain, [{ email, visitId, startedAt }]>.
Sort each persistent's visits chronologically (oldest first — classifyReturns needs that for firstVisitAt).
Run classifyReturns(...) and classifyShareDetected(...) from the pure helpers module.
Collect inserts.
INSERT INTO memo.visit_signals (visit_id, kind, payload)
VALUES (...)
ON CONFLICT (visit_id, kind) DO UPDATE
SET payload = excluded.payload, created_at = now();
Update-on-conflict is intentional: a return signal originally fired at visit_count=3 should be refreshed when the count becomes 5 (payload changes).
Audit-log with action visit_signals.derive and meta { windowHours, returnCount, shareCount }.
Auth: same as every other cron — authorizeCronRequest checks Authorization: Bearer <CRON_SECRET> (Vercel cron sets it automatically) or QStash signature.
Cost characteristics:
returns + shares ≤ a few dozen even at scale.URL: /u/<username>/people/<emailHash>
Hash: sha-256(email.toLowerCase().trim()).base64url.slice(0,16) — 16 characters, ~96 bits of entropy. The owner's URL never contains the raw email.
Resolution: the page can't decrypt the hash (it's a one-way function). Instead, it:
visitor_email it sees.This is O(distinct emails) per page render. For an owner with thousands of investors this is still fast (sha-256 in Node is ~1µs each).
What gets rendered:
visits.country / visits.city)/<slug>Signal chip colours:
| Signal | Colour token | Why |
|---|---|---|
skim | --memo-fg-muted | Neutral / informational |
deep_read | --memo-yellow | High-value, attention-grabbing |
return | --memo-blue | Repeat engagement, informational |
share_detected | --memo-pink | Notable cross-recipient pattern |
Endpoint: /api/owner/projects/<id>/activity/since?ts=<iso> (file: src/pages/api/owner/projects/[id]/activity/since.ts)
Polled by the Activity tab every 8 seconds. Returns:
{
"visits": [
{
"id": "<uuid>",
"startedAt": "ISO-8601",
"endedAt": "ISO-8601 | null",
"visitorEmail": "string | null",
"emailHash": "<16-char base64url> | null",
"linkId": "<uuid>",
"linkName": "string",
"linkSlug": "string",
"country": "string | null",
"city": "string | null",
"durationMs": "number | null",
"completionPct": "number | null",
"engagementScore": "number | null",
"signals": ["skim" | "deep_read" | "return" | "share_detected"]
}
],
"serverTime": "ISO-8601"
}
Why polling, not SSE: Vercel serverless functions have a max execution time of 60s on Hobby / 300s on Pro. Long-lived SSE would have to handle reconnects + connection drops carefully. An 8s poll trivially fits within those budgets, requires no client reconnection logic, and gives the user a "just refreshed" feeling that's indistinguishable from a true push.
Server-computed emailHash: The endpoint computes the same hash the people page uses and emits it in the payload, so the client can render <a href="/u/<username>/people/<emailHash>"> without needing sha-256 in browser code (SubtleCrypto.digest is asynchronous and would add a needless round-trip per row).
Client (in activity.astro):
lastTs = serverTime from first paint.fetch(sinceUrl + '?ts=' + lastTs), then for each new visit, tbody.insertBefore(newTr, tbody.firstChild).is-fresh → CSS animation fades the background from yellow to transparent over 4s (visual cue).tbody to 50 rows (matches the SSR cap).GET /api/owner/projects/<id>/analytics/sections#Auth: owner (Clerk session).
Response: per-section dwell aggregation, sorted by avg dwell desc.
{
"sections": [
{
"id": "hero",
"totalViews": 42,
"uniqueVisitors": 18,
"totalMs": 123456,
"avgMs": 2939,
"p50Ms": 2500,
"p90Ms": 9800,
"firstViewMs": 3200
}
],
"totalVisits": 47,
"ms": 18
}
Computation: Mongo find({ project_id, type: "section_exit" }) then in-process aggregation. Caches via Cache-Control: private, max-age=30.
GET /api/owner/projects/<id>/activity/since?ts=<iso>#Auth: owner.
Query: ts = ISO 8601 timestamp. Returns visits with started_at > ts. If ts is omitted or malformed, defaults to now - 60s.
Response: see "Live activity feed" section above. Capped at 50 per poll.
GET /api/owner/projects/<id>/changes#Auth: owner.
Response: git-status-style diff of staged files vs the project's current published version.
{
"fromVersion": { "id": "<uuid>", "versionNumber": 3 },
"added": ["src/new-file.ts"],
"removed": ["docs/old.md"],
"modified": ["index.html"],
"unchanged": 12
}
Modified detection: compares stored sha256 from memo.files against on-the-fly sha256 of the staged bytes. Capped at 1000 files per side (returns 413 with detail if exceeded). When currentVersionId is null (no published version yet), every staged file is "added".
GET /api/owner/projects/<id>/versions/<versionId>/files/<...path>#Auth: owner; ownership verified in a single join across files → versions → projects.
Response: streams the file bytes from R2 with the correct Content-Type and private, max-age=300 caching.
Used by: the editor's DiffEditor when opening a Modified or Removed file from the Changes panel — this is the "original" (left) side of the diff.
POST /api/owner/projects/<id>/search#Auth: owner. Rate-limited 30/min per (user, project).
Body:
{
"query": "string (1..200 chars)",
"regex": false,
"caseSensitive": false
}
Response:
{
"matches": [
{ "path": "src/app.ts", "line": 42, "col": 8, "snippet": "…" }
],
"truncated": false,
"fileCount": 1,
"matchCount": 1
}
Caps: per-file 50 matches, global 500 matches. Skips binary image files via isBinaryImageMime. On regex error returns 400 { error: "invalid_regex" }. Audit-logged with action project.search.
POST /api/cron/derive-cross-visit-signals#Auth: cron-only (Authorization: Bearer <CRON_SECRET>).
Response: { returnCount, shareCount }.
Schedule: every 10 minutes.
| Operation | Cost | Notes |
|---|---|---|
| One beacon batch (20 events) ingested | ~5-15ms p50 server side | Mongo insertMany(..., { ordered: false }) + one Postgres UPDATE if page_leave is in the batch |
| Per-visit signal classification | <1ms | Pure in-process, no I/O |
| Cross-visit cron run | <100ms p50, <500ms p99 | One SELECT + one INSERT … ON CONFLICT |
| Sections aggregation | O(matches) Mongo scan | Tuned by the recommended secondary index on (project_id, type, ts) |
| Activity polling | <20ms p50 | One SELECT with two indexes + one IN-list signal lookup |
| People page render | O(owner's distinct emails) × sha-256 | ~1µs per email; fast even for 10k investors |
See the Runbook for symptom-driven diagnostics.
Common diagnostics:
-- Has this visit row been finalised?
SELECT id, started_at, ended_at, duration_ms, completion_pct, engagement_score
FROM memo.visits
WHERE id = '<visit-uuid>';
-- All signals attached to one visit
SELECT kind, payload, created_at
FROM memo.visit_signals
WHERE visit_id = '<visit-uuid>';
-- Returning-visitor cohort across a project
SELECT v.visitor_email, v.visitor_persistent_id, count(*) AS visit_count
FROM memo.visits v
JOIN memo.links l ON l.id = v.link_id
WHERE l.project_id = '<project-uuid>'
GROUP BY v.visitor_email, v.visitor_persistent_id
HAVING count(*) >= 2
ORDER BY visit_count DESC;
// Mongo: per-section dwell distribution
db.events.aggregate([
{ $match: { project_id: "<project-uuid>", type: "section_exit" } },
{ $group: {
_id: "$payload.section",
avgMs: { $avg: "$payload.ms" },
p90Ms: { $percentile: { input: "$payload.ms", p: [0.9], method: "approximate" } }
}},
{ $sort: { avgMs: -1 } }
])
| Concept | File(s) |
|---|---|
| Beacon client | apps/memo/public/beacon.js |
| Beacon endpoint | apps/memo/src/pages/api/beacon.ts |
| Mongo client + EventDoc type | apps/memo/src/lib/db/mongo.ts |
| Chrome injection (meta tags, beacon script, recording snippets) | apps/memo/src/lib/visitor/watermark.ts |
| Section aggregation endpoint | apps/memo/src/pages/api/owner/projects/[id]/analytics/sections.ts |
| Activity polling endpoint | apps/memo/src/pages/api/owner/projects/[id]/activity/since.ts |
| Cross-visit signal cron | apps/memo/src/pages/api/cron/derive-cross-visit-signals.ts |
| Signal classifiers (pure) | apps/memo/src/lib/analytics/derive-signals.ts |
| Email hash helper | apps/memo/src/lib/email-hash.ts |
| Section heatmap UI | apps/memo/src/components/analytics/SectionHeatmap.tsx |
| Activity tab (server) | apps/memo/src/pages/u/[username]/projects/[projectId]/activity.astro |
| People timeline page | apps/memo/src/pages/u/[username]/people/[emailHash]/index.astro |
| Schema (visit row + visit_signals) | apps/memo/src/lib/db/schema.ts |
| Migrations | apps/memo/drizzle/0004_visit_persistent_id_and_engagement.sql, apps/memo/drizzle/0005_visit_signals.sql |
| Cron schedule | apps/memo/vercel.json |
| Hot-lead alert transport | apps/memo/src/lib/notifications/alert-transport.ts |
| Notification preferences | apps/memo/src/pages/api/owner/notifications/preferences.ts |
| Cohort retention endpoint | apps/memo/src/pages/api/owner/projects/[id]/analytics/cohort-retention.ts |
| Cohort retention UI | apps/memo/src/components/analytics/CohortRetention.tsx |
The HOT_LEAD_KINDS classifier (["return", "share_detected", "deep_read"]) fires at page_leave when a visit earns one of those signals. The alert transport layer sits in src/lib/notifications/alert-transport.ts.
visit finalises (beacon page_leave)
└─ derive-signals.ts classifies → hot-lead signals inserted into visit_signals
└─ alert-transport.ts receives signal kind + visit row
├─ if owner.alertDeepRead (for deep_read) or owner.alertHotLead (return/share)
│ └─ enqueue email via Resend
└─ if owner.slackWebhookUrl set
└─ POST to Slack incoming webhook (5s AbortController timeout)
Stored on the memo.users row. Editable at GET/PATCH /api/owner/notifications/preferences.
| Column | Type | Default | Meaning |
|---|---|---|---|
alert_hot_lead | boolean | false | Email alert on return / share_detected signals |
alert_deep_read | boolean | false | Email alert on deep_read signal |
slack_webhook_url | text | null | null | Slack incoming webhook URL; alerts posted here when set |
{
"text": "🔥 *Deep read* on *Series A Deck* by investor@example.com",
"blocks": [
{
"type": "section",
"text": { "type": "mrkdwn", "text": "🔥 *deep_read* signal on *<project name>*\n<visitor email> · <country> · <duration>" }
},
{
"type": "actions",
"elements": [{ "type": "button", "text": { "type": "plain_text", "text": "View in memo" }, "url": "<activity tab URL>" }]
}
]
}
The webhook POST uses a 5-second AbortController timeout. Network failures are logged but never surface to the visitor or block the beacon response.
GET /api/owner/notifications/preferences#Returns the current owner's alert preferences. Auth: Clerk.
Response 200:
{ alertHotLead: boolean, alertDeepRead: boolean, slackWebhookUrl: string | null }
PATCH /api/owner/notifications/preferences#Updates one or more preference fields atomically.
Body:
{ alertHotLead?: boolean, alertDeepRead?: boolean, slackWebhookUrl?: string | null }
Response 200: same shape as GET.
The cohort retention view groups visitors by the ISO week of their first visit to a project and shows how many returned in each subsequent week. This is the "stickiness" signal — a high-value investor who returns across multiple weeks is more engaged than a one-time click.
GET /api/owner/projects/<id>/analytics/cohort-retention
Auth: Clerk + project ownership.
Response 200:
{
cohorts: Array<{
cohortWeek: number, // weeks since COHORT_EPOCH (2024-01-01T00:00:00Z)
cohortLabel: string, // e.g. "2024-W03"
cohortSize: number, // distinct visitors who first appeared this week
weeks: Array<number | null> // retention count per week offset (null = future)
}>
}
const COHORT_EPOCH_MS = new Date("2024-01-01T00:00:00Z").getTime();
const WEEK_MS = 7 * 24 * 3600 * 1000;
cohortWeek = Math.floor((firstVisitMs - COHORT_EPOCH_MS) / WEEK_MS). Future-week cells are null (rendered as blank in the grid). Past weeks with zero returning visitors render as 0.
CohortRetention.tsx renders a sticky-header table:
+0w, +1w, +2w, … (relative offsets)color-mix(in srgb, var(--memo-accent) X%, transparent) where X = retention % clamped to 10–100