# Build brief — a focused alternative to Sembly AI

> **Verdict:** Partly, if you narrow it · **Buildability:** 57/100 · **Category:** Meeting Notes
> **Source:** https://www.canitbevibecoded.com/sembly-ai
> Independent editorial assessment from Can It Be Vibe Coded? Not affiliated with, endorsed by, or derived from Sembly AI. Verify current pricing and capabilities before acting.

## Context

**Sembly AI** — Transcribe a call and produce structured minutes, tasks, and searchable topics. It currently costs $15/mo.

The core loop is buildable, but a dependable replacement becomes a substantially larger project. For Sembly AI, transcribe a call and produce structured minutes, tasks, and searchable topics. The hard boundary is meeting bots, team collaboration, automations, and broad integrations, plus capture reliability, integrations, and collaboration.

This brief describes a focused, single-operator replacement for the part of Sembly AI that is genuinely reproducible. It is deliberately narrower than the product it replaces, and it says so in writing. Build the useful core; do not pretend to have rebuilt the rest.

## What you are building

Record or import a call, transcribe it locally, identify speakers with manual correction, and produce structured minutes, tasks, and searchable topics.

- Capture supplied audio, transcribe it, create structured notes, and export them.
- A responsive interface with real empty, loading, success, and error states.

## Requirements

### Functional

- Desktop microphone access.
- Whisper.cpp model files.
- Local storage.

### Data and integrations

- Optional OpenAI or Anthropic API key.

Each of these needs a real account, credential, or quota. Set them up before writing feature code.

### Non-functional

- Accessibility: semantic markup, labelled controls, visible focus, and reduced-motion support.
- Security: server-side secrets, validated input, and no credentials in the client bundle.
- Reliability: retries with backoff on external calls, and a clear failure state when a provider is down.
- Portability: the operator can export their data and leave without losing it.

## Implementation brief

Build a personal replacement for Sembly AI in an empty repository.
Use Python 3.12, FastAPI, SQLite, whisper.cpp, and a minimal HTMX interface; do not offer alternative stacks.
The core loop is: record or import a call, transcribe it locally, identify speakers with manual correction, and produce structured minutes, tasks, and searchable topics.
Make the first run work locally with one documented command.
Store all user data locally by default and make export straightforward.
Put secrets in .env, ship .env.example, and never commit credentials.
Add explicit start, pause, resume, and stop controls with a visible recording indicator.
Support importing WAV, MP3, M4A, and MP4 files through ffmpeg.
Run transcription locally and display timestamped editable segments.
Let the user rename speakers and propagate corrections through the transcript.
Generate decisions, action items, questions, and a concise summary from approved text.
Export Markdown, plain text, and WebVTT beside the original recording.
Include clear empty, loading, success, and recoverable error states.
Add input validation, safe filenames, and graceful handling of unavailable APIs.
Write focused tests for the core transformation and one end-to-end happy path.
Create a README with setup, architecture, permissions, data location, and backup steps.
Do not add accounts, billing, telemetry, analytics, or a hosted control plane.
Deliberately leave out silent background capture.
Deliberately leave out automatic bot attendance in video meetings.
Deliberately leave out team workspaces and enterprise retention controls.
Finish by running the tests and listing the exact commands used.

## Delivery standard

- Inspect the repository first, then write a short implementation plan before writing code.
- Deliver the smallest complete end-to-end workflow first; every primary control must work against persisted data.
- Use real validation and storage; never substitute fake dashboards, decorative controls, hard-coded success states, or mock integrations.
- Include responsive layouts plus genuine empty, loading, success, validation, and failure states.
- Keep secrets server-side in environment variables, provide .env.example, and never commit credentials or user data.
- Add structured logs around every external call and return actionable errors without leaking sensitive details.
- Write unit tests for the core logic and one automated test of the main user journey.
- Finish with a README covering setup, architecture, data location, backups, tests, deployment, and known limitations.

## Acceptance criteria

- [ ] A clean install starts the app using only the README and .env.example.
- [ ] The primary journey works from first visit through saved result, reload, edit, export, and deletion where applicable.
- [ ] Invalid input, missing configuration, provider failure, and an empty database each have a usable state.
- [ ] The interface works at 390px and 1440px, is keyboard navigable, and shows visible focus on every control.
- [ ] Tests, type checking, linting, and a production build all pass with no ignored failures.
- [ ] No part of the interface implies a live integration, security guarantee, or scale capability that was not actually built and verified.

## Non-goals

Do not build these, and do not claim to have replaced them:

- Meeting bots, team collaboration, automations, and broad integrations.
- Calendar auto-join.
- Reliable speaker diarization.
- Mobile capture.
- Connectors, OAuth flows, and vendor API changes require constant upkeep.
- Permissions, presence, and shared workflows are difficult to simplify.

## What you still own after launch

- Secure credentials, rotate secrets, and handle provider rate limits.
- Run migrations, backups, restores, and dependency updates.
- Test the critical journey after every model, API, or hosting change.
- Monitor failures and fix the edge cases a first prompt will miss.
- Maintain every third-party integration as APIs and OAuth rules change.

## Risk

**Operational risk.** The code is achievable; dependable data, integrations, and ongoing operations are the real cost.

Editorial confidence in this assessment: medium. No independent one-shot implementation is linked yet.

## Prior art

Working open-source software you can read, fork, or borrow from before starting:

- [whisper.cpp](https://github.com/ggerganov/whisper.cpp) — Widely used local Whisper inference implementation suitable for private transcription

---

Generated by [Can It Be Vibe Coded?](https://www.canitbevibecoded.com) · Full report: https://www.canitbevibecoded.com/sembly-ai