Case study

PitchPerfect

Practice a sales call out loud with a buyer who has hidden objections, then get a scorecard that quotes what you said.

Code on GitHub Live demo7 rubric lines on every scorecard, each quote checked against the call

Problem

People new to sales need to practice real calls and get specific feedback, and employers want to see how a candidate pitches, not only read a resume.^{[1][1] What’s inside, in the PitchPerfect README (opens in a new tab)Source 1What’s inside, in the PitchPerfect READMEOpen source 1 (opens in a new tab)All sources}

I wanted a practice partner that feels like a real buyer, stays on topic, costs little to run and cannot make up what the student said.

Constraints and approach

The model never sets the score, and it cannot put words in the student’s mouth.
The model keys live in one small service that the public internet cannot reach.
Student text never becomes part of the instructions, and personal details are removed first.
Every call costs money, so practice is capped per student and per day across the site.

A website, an Express and MongoDB api, and a separate practice partner service: a conversational buyer, played by an LLM, with hidden objections. It streams its replies and scores the call on seven rubric lines; code computes the overall score and checks every quote on the scorecard against what the student said.

How one practice call works

The browser talks only to the website, which passes requests to the api; only the api can call the practice partner^{[2][2] The practice session flow (docs/architecture.md) (opens in a new tab)Source 2The practice session flow (docs/architecture.md)Open source 2 (opens in a new tab)All sources}.

Steps of one practice call
Step	What happens
Plan	The partner makes up a buyer for the chosen skill, or for a real job post: a name, a role, a company, a personality, two or three hidden objections and what would win them over. The hidden part stays on the server.
Talk	The student speaks or types. The browser turns speech into text, which the student can edit before sending. The buyer answers in at most 80 words, streamed and read aloud, and raises its objections only once the student has pitched.
Score	When the call ends, the model scores seven rubric lines from 1 to 5, each with a reason and a quote. Code checks the quotes, works out the overall score and saves the scorecard.
Apply	The student can attach a scorecard to a job application. It is a live link, not a copy, so deleting the attempt takes it out of the employer’s view.

The scorecard

Seven rubric lines, each scored 1 to 5 with a reason and a quote, then three strengths, three improvements and one weak line rewritten^{[3][3] Rubric and scorecard (docs/practice-partner.md) (opens in a new tab)Source 3Rubric and scorecard (docs/practice-partner.md)Open source 3 (opens in a new tab)All sources}.

Rubric lines and what a 1 and a 5 mean
Line	1 means	5 means
Clarity	Rambling, jargon, no clear point	Crisp, one clear message
Value and relevance	Features only, generic	Benefits tied to this buyer’s stated needs
Structure	No flow	A clear opening, body and close
Discovery questions	No questions, or only yes or no	Open, layered questions that uncover pain and impact
Objection handling	Ignores or argues	Acknowledges, clarifies, answers with evidence, checks
Confidence and language	Hedging, filler, apologetic	Assured, concise, positive
Call to action	No ask	A specific next step with a time

Design choices

Design choices and their trade-offs
Choice	Why
A separate practice partner	The service that calls the model holds the keys and nothing else: no database, no accounts, no public address. It answers only requests the api has signed, so a break-in to the website cannot reach the keys, and the partner can be tested, scaled or swapped on its own.
The score is computed in code	The model reads the call and gives seven scores from 1 to 5. The overall 0 to 100 is a weighted mean worked out in code, with the skill’s own line counting double, so the model cannot be talked into a high number.
Quotes checked word for word	A scorecard that quotes the student is only useful if the quote is real. Every quote must appear in what the student said; one that does not is dropped.
Voice in the browser	Speech to text and the buyer’s voice use the browser’s own speech features, so PitchPerfect never uploads or stores audio. Typing is always one click away, and it takes over on its own where a browser has no speech recognition.
The provider is a setting	Two provider adapters sit behind one interface, and the provider and model names come from settings, never from code. A scripted partner fills the same interface for tests and the demo.

The architecture notes explain why the practice partner is its own service^{[4][4] Why three services (docs/architecture.md) (opens in a new tab)Source 4Why three services (docs/architecture.md)Open source 4 (opens in a new tab)All sources}.

How it is checked

Guardrails sit in layers, so a turn that gets past one still meets the next^{[5][5] Guardrails (docs/practice-partner.md) (opens in a new tab)Source 5Guardrails (docs/practice-partner.md)Open source 5 (opens in a new tab)All sources}.

Guardrail layers
Layer	What it does
Input	1 to 1,000 characters a turn, control characters removed. Student text goes inside a marked block that is data only and can never change the rules.
Personal data	Emails, phone numbers and card-like numbers are replaced before the turn is stored or sent to the model.
Pre-check	Rules plus a small classification call look for attempts to pull out the instructions, change the partner’s role or get harmful content. A flagged turn gets an in-character redirect; a third flag ends the session.
Output	Every prompt carries a random canary token. While a reply streams, the last 8 words are held back so a leaked run of the instructions can be stopped. Replies are cut at 80 words and moderated.
Cost	5 sessions a day and 30 a month per student, a site-wide daily token budget, a token cap on every call and one turn in flight at a time.
Reliability	30 s per model call, one retry, and a circuit breaker that pauses practice after 5 failures in a minute.

The security notes list each threat with its control and the test that checks it, from sign-in limits and stored scripts in job posts to employers reading another company’s applicants^{[6][6] Threats and controls (docs/security.md) (opens in a new tab)Source 6Threats and controls (docs/security.md)Open source 6 (opens in a new tab)All sources}.

The eval set is 30 scripted conversations: 8 strong, 8 mediocre and 6 weak calls with an expected score band, and 8 attacks, among them pulling out the prompt, a role swap, a harmful request, a flood of long input and asking the buyer to pose as a human. They run through the real practice service with a real model. A call passes when its score lands in its band; an attack passes only when every attack turn gets the expected answer and no reply leaks. The target is 90% of cases and every attack^{[7][7] Practice partner evals (apps/coach/evals/README.md) (opens in a new tab)Source 7Practice partner evals (apps/coach/evals/README.md)Open source 7 (opens in a new tab)All sources}.

CI cannot call a model, so a scripted partner that always gives the same answer plays the buyer there. Every push runs lint, type checks, unit and integration tests, a dead-code check, a secret scan and a dependency audit, then end-to-end tests on a throwaway database and on the static demo, and Lighthouse^{[8][8] The CI workflow (.github/workflows/ci.yml) (opens in a new tab)Source 8The CI workflow (.github/workflows/ci.yml)Open source 8 (opens in a new tab)All sources}.

Results

The live demo runs the same practice engine in your browser with the scripted partner, so anyone can run a call and read the scorecard without an account^{[9][9] The static demo (docs/security.md) (opens in a new tab)Source 9The static demo (docs/security.md)Open source 9 (opens in a new tab)All sources}.

Practice skills: 8; each with a short lesson
Scripted eval conversations: 30; 8 of them attacks, which must all be handled
Longest buyer reply: 80 words
Services: 3; the one that holds the API keys is never public

Sources

Links and stack

Code on GitHub Live demo

TypeScript
Next.js
React
Tailwind CSS
Node.js
Express
Fastify
MongoDB
Web Speech API