VendIQ

A sales rep on a call is doing two jobs at once: having the conversation, and working out what to say next. VendIQ takes the second one, listening to the call as it happens and surfacing a recommendation the rep can accept or ignore.

The naive version of this product records the call, transcribes it afterwards and mails the rep a summary. That version is useless, and it is useless for a reason worth stating plainly: by the time the summary exists, the decision it was meant to inform has already been made badly. Every constraint in the system comes out of refusing that shortcut. If the suggestion has to land while the customer is still talking, then transcription has to be streaming, inference has to be off the request path, and the transport between them cannot be anything that opens a connection per message.

The ring-to-screen loop

The clearest piece of the design is the part that runs before the rep touches anything, and it is worth tracing end to end because it is the 800ms argument made concrete.

A native Android app registers a receiver on android.intent.action.PHONE_STATE. The moment the handset reports EXTRA_STATE_RINGING with a caller number attached, it raises a local notification on its own IncomingCallChannel and, on a background thread, POSTs {"message": "Connected"} to /update-message on the Go hub over a plain HttpURLConnection. The hub decodes the body, drops the string onto an unbuffered broadcast channel, and a single goroutine ranges over every registered client and writes it out. One of those clients is a small Go desktop binary sitting on the rep's machine, dialled into ws://localhost/ws; when it reads Connected it shells out to Chrome with the portal URL. The console is open and listening before the rep has decided whether to answer.

That chain is four hops and no polling anywhere in it. The hub is the only component that knows about more than one participant, and it treats every connection as an anonymous subscriber, which is why adding a second screen, a supervisor view or a logger costs nothing.

How it works

• Live transcription in the browser via the Web Speech API, streamed to the backend rather than batched at the end of the call. The recogniser runs with continuous: false and interimResults: true on en-IN, and is restarted on every listening transition so a pause does not end the session. The running transcript is split on sentence boundaries, and each finished sentence is dispatched as it lands.
• Sentiment analysis through a Flask service, so the emotional read updates during the conversation rather than after it. /analyze takes a batch of sentences rather than one, strips punctuation, tokenises, and scores each with VADER: polarity at a compound threshold of plus or minus 0.05, and a separate sarcasm flag below -0.5. That second signal matters because a sarcastic yes and a genuine yes call for opposite next moves.
• Recommendations generated by Gemini at temperature 0, top-p and top-k pinned to 1, a 2048-token ceiling and all four harm categories set to block at medium and above. The prompt is a fixed assistant instruction plus the most recent completed sentence, so the model is scoped by what has just been said rather than by the whole call. A parallel path runs the same prompt shape against an Azure OpenAI deployment behind a second Flask endpoint, logging every prompt and reply to a rolling interaction log, so the assistant can be repointed without touching the portal.
• Two Go WebSocket servers carrying the real-time paths, built on gorilla/websocket, which is where the 800ms round trip from speech to on-screen suggestion is spent. One is the hub: it upgrades every portal connection, accepts state changes over an HTTP POST, and fans them out to all clients from a single broadcast goroutine. Its read loop exists only to notice a hangup and evict the connection.
• A native Android app in Java handling call detection on the device side. Underneath the receiver is an abstract telephony state machine that keeps lastState and the caller number in statics, because Android recreates the receiver whenever it likes and the incoming number is only present on the ringing edge. From those transitions it derives incoming, outgoing, ended and missed rather than treating every state change as one event.
• A local desktop client dialling the same hub, which opens the portal on the operator's screen when that event arrives, so the rep does not have to reach for anything.
• A React portal where the rep sees the live transcript, the sentiment read, and the suggestion queue. One piece of state lives at the top of the tree and the transcription component is its only writer, so the charts and the suggestion pane cannot disagree about which sentence they are describing.

What the rep sees

• The call console, with the customer on the card and mute, hold and decline in reach.
• A live transcript pane that scrolls as the conversation runs.
• A suggestion queue rendering the assistant's output as it returns.
• A sentiment timeline drawn as a streaming ApexCharts line rather than a single label, animated linearly on a one-second dynamic redraw so a new point slides in instead of jumping.
• A sarcasm indicator carried alongside the polarity read.
• A pitch graph built from real microphone capture: getUserMedia into an AnalyserNode, time-domain samples every 200 ms, smoothed over a hundred-sample moving average and plotted as amplitude against time.
• A lead and credit panel showing the customer's score next to the account they hold.
• A call log split into all and missed, with a revert action per contact.
• An incoming-call popup with accept and decline, driven by the ring event from the phone rather than by anything the rep clicks.
• Customer history kept on screen during the call instead of behind a tab.
• A left-right split that keeps the conversation on one side and the machine's opinion on the other, so the rep is never reading the assistant where the transcript should be.

The interesting constraint

Latency is the whole product. A recommendation that arrives after the moment has passed is worse than no recommendation, because the rep has to read it, discard it, and get back to the conversation. Most of the architectural decisions follow from holding that budget: Go for the socket layer, streaming transcription instead of batch, a separate inference service rather than inline calls, and a hub that treats every client as a subscriber so adding a second screen costs nothing. The sentiment call and the generation call are also independent, so the cheap read paints while the expensive one is still in flight.

The tradeoff that buys the budget is accuracy. VADER is a lexicon and a set of thresholds, not a trained model, and it will read a flat sentence wrong in ways a fine-tuned classifier would not. It also answers in single-digit milliseconds on CPU with no warm-up and no serving cost, which is what lets the sentiment line move on every sentence instead of every tenth one. On a call, a slightly coarse read that keeps pace is more useful than a precise one that arrives at hangup.

What the patent claims

The project is Patent Pending, filed as "VendiQ: Intelligent Real-Time Speech and Text Analytics for Optimizing Customer-Agent Dialogues". The abstract claims three contributions: a real-time in-call conversational optimisation engine rather than a post-call analytics report; multimodal contextual intelligence that fuses voice tone with transcription, text chat and CRM history; and an emotion-aware coaching layer that stays non-intrusive, meaning the rep is advised rather than scripted.

Honest scope

The transport and the inference services are real; a good part of the console is not yet wired to them. The caller card, the contact list and the day-end summary are hardcoded in the source, and the day-end panel is lorem ipsum commented out of the render. The suggestion pane currently prints the raw response object, with the markdown title and bullet parsing sitting commented out beside it. The sentiment chart is seeded with sample points before live data replaces them. The desktop client's on-screen audio readout is fabricated: a 500 ms ticker printing random byte counts and latencies, standing in for a stream that was never plumbed through it.

The hub is a hackathon-grade server that grew into production shape without being hardened for it. Its client map is mutated from both the upgrade handler and the broadcast goroutine with no mutex, the origin check accepts everything, /update-message is unauthenticated, and it listens on port 80 in the clear, which the Android app matches by declaring usesCleartextTraffic. The hub address is typed into a text field in the app and pushed to the receiver over a custom broadcast, so the whole loop assumes the phone and the rep's machine are on the same network. A second portal exists as a Tailwind redesign of the console, laid out in full and not yet connected to anything.

Status

Built with a team of four; my work was concentrated on the real-time transport and the portal. The product was developed under LAIF Technologies. The two Go modules, the telephony receiver, the Flask services and the React console are all in the repository, behind roughly three months of shared commit history.