Fintech Reporting Dashboard200M rows · 60% faster · sub-second queries

An SMB-focused fintech had a dashboard that was becoming an embarrassment. Queries for a 90-day P&L took 7 seconds on a good day, 40 seconds on a bad one, and the root cause was structural: they were running analytical aggregations against their OLTP Postgres that was also handling every ingestion write.

They didn't need magic. They needed to stop punishing a transactional database for doing analytics work, without rebuilding the data model from scratch.

• System of record stayed on Postgres. Regulatory constraints ruled out moving it.
• Every reported number had to reconcile to the cent against the Postgres ground truth, continuously and automatically.
• Ingestion had to handle Plaid's inconsistent webhook ordering and Stripe's delayed dispute events gracefully.
• Dashboard latency budget: < 1 second for 95% of queries.
• Infra cost had to be roughly flat — this was a cost problem, not a spend-more problem.

Postgres kept its job as the system of record. Alongside it, a Parquet lake on S3, partitioned by tenant and month, maintained by a thin Python pipeline that reads from Postgres's logical replication slot and writes columnar files.

Queries stopped hitting Postgres. The dashboard hits a DuckDB process running against Parquet files, with the hot month held in memory via Arrow. Reconciliation runs every fifteen minutes: the same aggregate computed both places, diffed, alerted if non-zero.

Ingestion got its own reliability layer. Plaid webhooks land in an idempotent inbox; out-of-order events are reordered by timestamp before commit. Stripe disputes, which can arrive 90 days after the original charge, flow through a separate late-arriving events pipeline that rewrites only the affected daily partitions.

• Timezone semantics. “Monthly revenue” meant four different things across four different reports. We standardized on account-local midnight-to-midnight, which sounds obvious and took two weeks to sell.
• Reconciliation tolerance. Float arithmetic on a long series of fees produced 0.01-cent differences that were technically real and technically unactionable. We round to the cent at the boundary and log the pre-rounding delta separately.
• Stripe dispute backfill. The late-arriving events pipeline rewrites history, which means caching has to be smarter than “invalidate after 24 hours.” We tag every cached aggregate with the latest source event ID.

The dashboard that used to take 7 seconds now answers in 420ms at p95. Median queries are 60% faster than the Postgres baseline across the full reporting surface. The Parquet lake indexes 200M rows across three years of history. Reconciliation runs every 15 minutes against Postgres and has not found a material discrepancy since cutover. Infra spend is within 5% of the pre-migration number.

I'd have shipped reconciliation before the new dashboard, not alongside it. Turning on a new reporting path when you can't yet prove its numbers match the old one is a conversation you do not want to have with finance. Once reconciliation was green, everything else felt like paperwork.