HUB24 / Class · Data Enablement Lead
Member transaction history, rebuilt dollar for dollar
Replicated a production member-transaction API as a warehouse dataset across 28 years of history, reconciled to the source with 100% accuracy.
- 2.65bn
- source rows → 75m transactions
- 100%
- reconciliation to the source API
- 10.5 hrs
- to rebuild all 28 years
- 600k → 5
- quarterly API calls replaced by queries
- $5.6tn
- total transaction value
- 600,000
- funds
- 750,000
- members
- 1.5m
- member accounts
The problem
Teams needed every member transaction, per fund or per member, as at any point in time. The only source was Class’s member transaction API endpoint, which meant ~600,000 API calls every quarter just to assemble the history.
By the numbers
Scale: 2.65bn source rows → 75m transactions, across 28 years. The dataset reproduces the endpoint’s logic over the full 28-year history. Each transaction is built from roughly 35 source rows on average (derived), so the model has to aggregate rows exactly the way the API does, or the totals drift.
Coverage: 600,000 funds, 750,000 members and 1.5m member accounts. Every fund, member and account in the history is included, about two accounts per member on average (derived). Together they total $5.6tn in transaction value.
Accuracy: 100% reconciliation to the source API. The rebuilt history matches the API dollar for dollar. That reconciliation is what lets teams trust the dataset in place of the endpoint.
Speed: the full history rebuilds in 10.5 hours. All 28 years, from 2.65bn source rows, regenerate in a single run of 10.5 hours, roughly 250m source rows an hour (derived).
Efficiency: 600,000 quarterly API calls → 5 queries. The team’s quarterly workload went from hundreds of thousands of API requests to five warehouse queries, a 120,000× reduction in calls (derived).