Loading…
The marketing site is often opened from corporate networks that intercept TLS
(Cisco Umbrella, Zscaler, Island Browser, Netskope). These stacks re-sign traffic
with a private CA and apply per-domain policy: matters.ai may be trusted while a
third-party CDN the page fetches at runtime is decrypted, blocked, or re-signed
with a certificate the browser rejects (net::ERR_CERT_AUTHORITY_INVALID).
Design rule: no decorative or third-party resource may be load-bearing. Any runtime fetch to a domain we don't control must be assumed to fail for exactly the enterprise visitors the site targets — and its failure must never take down the page.
Report: KKR (via partner Breakaway) — matters.ai "initially loads and then
returns an application error" on every browser, normal and incognito. Console:
ERR_CERT_AUTHORITY_INVALID on prod.spline.design/.../scene.splinecode, then
TypeError: Failed to fetch from the Spline runtime.
prod.spline.design; the browser
rejected the certificate and the scene fetch threw TypeError: Failed to fetch.@splinetool/react-spline v4 removed the onError prop. It stores the
load failure in state and re-throws it during render
(if (error) throw error), expecting the consumer to provide an error boundary.HeroSplineScene passed onError={...} — dead API in v4, silently spread
onto a div — so the carefully designed fallback (onFallback → hide scene)
never fired.error.tsx and no client boundary existed above the hero, so the
throw escalated to app/global-error.tsx, which rendered Next's default
<NextError statusCode={0} />: the raw "Application error: a client-side
exception has occurred" page — replacing the entire document.The lesson generalizes: the component's fallback design was correct, but it was written against a removed library contract and nothing above it was defensive. A decorative 3D background had root-layout blast radius.
| Layer | Change | File |
|---|---|---|
| Contain | SplineErrorBoundary catches the v4 render-throw, logs via logger.error, fires onFallback, renders null | components/homepage/HeroSplineScene.tsx |
| Remove the trigger | Scene served first-party from /spline/hero-scene.splinecode (8 KB, committed) — rides the same trust decision that let the page load | public/spline/hero-scene.splinecode |
| Degrade beautifully | Homepage hero swaps in the static home-bg.webp art when the scene can't run (same fallback FeatureHeroTemplate always had) | components/homepage/HeroSection.tsx |
| Route boundary | Branded error.tsx keeps failures inside the layout with Try again / Go home + Sentry capture | app/error.tsx |
| Last resort | global-error.tsx no longer renders NextError(0); branded inline-styled page (global-error cannot rely on app CSS) with the error digest as a support reference | app/global-error.tsx |
| Regression pin | jsdom test mimics the v4 render-throw and asserts: no crash, onFallback fires once, no broken canvas | tests/homepage/hero-spline-scene.test.tsx |
Both Spline consumers — the homepage hero and FeatureHeroTemplate's
particle variant — go through the shared HeroSplineScene, so the fix covers
both.
The scene is authored in Spline (original export
https://prod.spline.design/ZI2zaAMnbVUk2svI/scene.splinecode). To change it:
re-export from the Spline editor and replace
public/spline/hero-scene.splinecode — do not point the component back at
prod.spline.design.
The @splinetool/runtime lazy-loads feature chunks (physics, howler, opentype)
as local webpack chunks, but fetches WASM for boolean / modelling /
navmesh / ui features from unpkg.com at runtime. The current scene uses
none of those (verified: zero unpkg requests on load). If a future scene adds
them, self-host the WASM and pass wasmPath to <Spline> — otherwise the
scene will break behind the same corporate proxies this page was fixed for.
Report: a partner contact at Mitigata (hema.v@mitigata.com) submitting
Get the Datasheet on /datasheets/[slug] saw, inside the form's error card:
Unexpected token '<', "<!--# OMNF"... is not valid JSON
That text is V8's JSON.parse failure: the browser received an HTML
document whose first ten characters are <!--# OMNF where it expected the
JSON envelope from POST /api/datasheets/submit.
Every layer we own answers JSON on this path, so we did not produce that page:
| Layer | File | Response on failure |
|---|---|---|
| Origin gate | lib/cors.ts → requireMattersOrigin | NextResponse.json({ error: "Forbidden" }, 403) |
| Rate limiter | proxy.ts → lib/security/rate-limit-enforcer.ts | NextResponse.json({ code: "RATE_LIMITED", … }, 429) |
| BotID guard | lib/security/botid.ts | route returns { success: false, error: "Access denied" } 403 |
| reCAPTCHA / validation / handler | app/api/datasheets/submit/route.ts | JSON on every branch incl. the catch-all 500 |
A live probe (curl -X POST https://www.matters.ai/api/datasheets/submit
with a browser UA and a matters.ai Origin) returned 403 application/json
{"success":false,"error":"Access denied"} from the BotID guard — JSON,
server: Vercel, no x-vercel-mitigated header. Vercel's own error
pages (DEPLOYMENT_NOT_FOUND etc.) start with <!doctype html>, not
<!--#. <!--# is server-side-include syntax; the OMNF marker is not
publicly documented for any vendor.
Two candidates remain, and only the failed response's headers separate them:
challenge/deny matching /api/*
POSTs. A fetch() call cannot solve a challenge, so the browser gets
the checkpoint HTML with x-vercel-mitigated: challenge and
server: Vercel. Fix if confirmed: path exception or action log for the
form routes (BotID + reCAPTCHA already cover them).server header, or none. Fix if confirmed: the partner
allow-lists matters.ai; nothing on our side.The bug on our side was that both gated-download clients called
response.json() blindly, which (a) put the raw parser error in front of the
visitor and (b) discarded the headers that identify the intermediary.
| Layer | Change | File |
|---|---|---|
| Single reader | readJsonResponse() parses once; JSON+2xx → data, JSON+error → AppException with the server's error/message/title, non-JSON → NonJsonResponseError | lib/api/json-response.ts |
| Visitor message | "Our server's reply was intercepted before it reached this page (HTTP 403). This is usually caused by a network security filter or proxy. Please try again, or retry from a different network or device. Reference: <x-vercel-id>" — the reference lets support correlate with Vercel logs | same |
| Evidence capture | Sentry captureException (level warning) tagged feature:api-client, endpoint, status, mitigated, server; extra carries contentType, x-vercel-id, cf-ray, and the first 200 chars of the body | same |
| Thin clients | getJson / postJson wrappers; both *-client.ts modules become typed one-liners (SubmitDatasheetResponse, SubmitWhitepaperResponse, list/detail envelopes) | lib/datasheets-client.ts, lib/whitepapers-client.ts |
| Regression pin | 11 node-env tests: JSON 2xx/4xx/empty-204, intercepted HTML (message wording, diagnostics, Sentry tags), malformed JSON, missing reference id, postJson wiring | tests/lib/json-response.test.ts |
feature:api-client. Read tags.server and
tags.mitigated: Vercel + challenge/deny ⇒ our firewall; anything
else ⇒ the visitor's network.challenge rule matches the form
routes (/api/datasheets/submit, /api/whitepapers/submit,
/api/demo-form, /api/webinars/*, /api/case-studies/download).Report: real humans (including our own team) got 403 Access denied from
POST /api/datasheets/submit — BotID's verdict, not the firewall.
Vercel BotID basic mode is binary — there is no score or sensitivity to
tune. checkBotId() effectively asks: did this request carry a valid token
from the client-side classifier? Any signal-delivery failure therefore makes
a human indistinguishable from a bot:
/149e9513-… paths are on public filter lists).initBotId used to run in a React useEffect
(BotIdInit.tsx, now deleted), so the classifier loaded after hydration; a
fast (autofilled) submit fired before classification finished and carried
no signals.Every BotID-protected route has reCAPTCHA v3 (scored, tunable) behind it, so
a BotID flag no longer 403s. Instead it escalates the reCAPTCHA minimum
score from the env default (0.5) to
BOTID_FLAGGED_RECAPTCHA_MIN_SCORE (0.7) via recaptchaMinScoreFor():
| Caller | BotID | reCAPTCHA | Outcome |
|---|---|---|---|
| Human, signals delivered | clean | ≥ 0.5 | allowed |
| Human, classifier blocked (adblock/proxy/fast submit) | flagged | ≥ 0.7 (humans typically 0.7–0.9) | allowed — the false-positive fix |
| Bot without a captcha token | flagged | missing_token | blocked |
| Automation with a minted token | flagged | score ≪ 0.7 → low_score | blocked |
Flags stay visible: checkBotIdIfVercel logs
"BotID flagged request as bot" with route + kind, and low-score rejections
land in the audit log with the minScore that was applied.
Files: policy + escalation in lib/security/botid.ts; client init in
instrumentation-client.ts (before hydration — never a useEffect); the
protect list in lib/security/botid-protect.ts (client-safe module — must
list every route that calls checkBotIdIfVercel, or that route's real users
deliver no signals and all get flagged); regression pin in
tests/security/botid-advisory.test.ts.
@splinetool/react-spline v4 contract#v4 (4.1.0) exposes: scene, onLoad, renderOnDemand, wasmPath, the
onSpline* event props, and children (loading state). There is no
onError. Scene-load failures are re-thrown during render — every consumer
MUST sit inside an error boundary. The package is also ESM-only (exports has
no require condition): jest tests must mock it with
jest.mock(..., factory, { virtual: true }).
The incident test introduced component testing to the root app:
jest-environment-jsdom, @testing-library/react, @testing-library/dom
(devDependencies)./** @jest-environment jsdom */
docblock; everything else stays in the default node environment.tsconfig.jest.json extends tsconfig.json with jsx: "react-jsx" (Next's
jsx: "preserve" can't execute under ts-jest); testMatch accepts
*.test.tsx.jest.config.ts still applies to all suites.component boundary (SplineErrorBoundary, …) ← contain locally, degrade in place
↓ escapes
app/error.tsx ← branded, inside root layout (fonts/CSS available)
↓ escapes root layout
app/global-error.tsx ← own <html>/<body>, inline styles ONLY,
Sentry capture + digest reference
global-error.tsx renders without the app's stylesheets — never use Tailwind
classes or next/font there. Both pages report to Sentry and show
error.digest so an enterprise visitor can quote a reference ID.
next.config.ts → baseCsp)#script-src is an explicit allowlist. Any tag added via Google Tag Manager
must also be added here, or the browser blocks it for every visitor — that is
a silent, sitewide data loss, visible only in the console. This happened with
Matomo: GTM injected cdn.matomo.cloud/mattersai.matomo.cloud/matomo.js, CSP
blocked it, and Matomo analytics collected nothing until
https://cdn.matomo.cloud https://mattersai.matomo.cloud were allowlisted
(2026-08-19, same incident console).
When adding any third-party script:
script-src (and check connect-src / img-src
for its beacons — both currently include broad https:).fetches inside
React components do not — those need a boundary.Root package.json uses workspace:* dependencies, which npm cannot
install (EUNSUPPORTEDPROTOCOL). pnpm-lock.yaml is the only live lockfile;
package-lock.json is stale legacy and the "i" script's npm ci step no
longer works. Treat pnpm as the package manager of record for the whole repo.