Can Sembly AI be vibe coded?
Transcribe a call and produce structured minutes, tasks, and searchable topics
The core loop is buildable, but a dependable replacement becomes a substantially larger project. For Sembly AI, transcribe a call and produce structured minutes, tasks, and searchable topics. The hard boundary is meeting bots, team collaboration, automations, and broad integrations, plus capture reliability, integrations, and collaboration.
Jump to the build brief ↓Checked Jul 2026
What you pay today, before any DIY hosting
medium editorial confidence
Buildability by layer
Screens, forms, and focused interactions
The repeatable job the product performs
Availability and legality of required data
Uptime, queues, support, and maintenance
Security, compliance, and user confidence
The achievable core
- Record or import a call, transcribe it locally, identify speakers with manual correction, and produce structured minutes, tasks, and searchable topics.
- Capture supplied audio, transcribe it, create structured notes, and export them.
- A responsive interface with real empty, loading, success, and error states.
The parts a prompt cannot buy
- meeting bots, team collaboration, automations, and broad integrations
- calendar auto-join
- reliable speaker diarization
- mobile capture
- Connectors, OAuth flows, and vendor API changes require constant upkeep.
- Permissions, presence, and shared workflows are difficult to simplify.
Why people still pay
People still pay for Sembly AI because a meeting tool must capture every call without surprising anyone, then make the result searchable and shareable across a team. The recurring cost buys audio permissions, model updates, calendar APIs, storage, speaker correction, and sync, not just the visible interface.
Connectors, OAuth flows, and vendor API changes require constant upkeep.
Permissions, presence, and shared workflows are difficult to simplify.
The brief
Context, requirements, acceptance criteria, non-goals, and the full production standard — as Markdown, ready for any coding agent.
Build brief — a focused alternative to Sembly AI
Context
**Sembly AI** — Transcribe a call and produce structured minutes, tasks, and searchable topics. It currently costs $15/mo.
The core loop is buildable, but a dependable replacement becomes a substantially larger project. For Sembly AI, transcribe a call and produce structured minutes, tasks, and searchable topics. The hard boundary is meeting bots, team collaboration, automations, and broad integrations, plus capture reliability, integrations, and collaboration.
This brief describes a focused, single-operator replacement for the part of Sembly AI that is genuinely reproducible. It is deliberately narrower than the product it replaces, and it says so in writing. Build the useful core; do not pretend to have rebuilt the rest.
What you are building
Record or import a call, transcribe it locally, identify speakers with manual correction, and produce structured minutes, tasks, and searchable topics.
Capture supplied audio, transcribe it, create structured notes, and export them.
A responsive interface with real empty, loading, success, and error states.
Requirements
Functional
Desktop microphone access.
Whisper.cpp model files.
Local storage.
Data and integrations
Optional OpenAI or Anthropic API key.
Each of these needs a real account, credential, or quota. Set them up before writing feature code.
Non-functional
Accessibility: semantic markup, labelled controls, visible focus, and reduced-motion support.
Security: server-side secrets, validated input, and no credentials in the client bundle.
Reliability: retries with backoff on external calls, and a clear failure state when a provider is down.
Portability: the operator can export their data and leave without losing it.
Implementation brief
Build a personal replacement for Sembly AI in an empty repository.
Use Python 3.12, FastAPI, SQLite, whisper.cpp, and a minimal HTMX interface; do not offer alternative stacks.
The core loop is: record or import a call, transcribe it locally, identify speakers with manual correction, and produce structured minutes, tasks, and searchable topics.
Make the first run work locally with one documented command.
Store all user data locally by default and make export straightforward.
Put secrets in .env, ship .env.example, and never commit credentials.
Add explicit start, pause, resume, and stop controls with a visible recording indicator.
Support importing WAV, MP3, M4A, and MP4 files through ffmpeg.
Run transcription locally and display timestamped editable segments.
Let the user rename speakers and propagate corrections through the transcript.
Generate decisions, action items, questions, and a concise summary from approved text.
Export Markdown, plain text, and WebVTT beside the original recording.
Include clear empty, loading, success, and recoverable error states.
Add input validation, safe filenames, and graceful handling of unavailable APIs.
Write focused tests for the core transformation and one end-to-end happy path.
Create a README with setup, architecture, permissions, data location, and backup steps.
Do not add accounts, billing, telemetry, analytics, or a hosted control plane.
Deliberately leave out silent background capture.
Deliberately leave out automatic bot attendance in video meetings.
Deliberately leave out team workspaces and enterprise retention controls.
Finish by running the tests and listing the exact commands used.
Delivery standard
Inspect the repository first, then write a short implementation plan before writing code.
Deliver the smallest complete end-to-end workflow first; every primary control must work against persisted data.
Use real validation and storage; never substitute fake dashboards, decorative controls, hard-coded success states, or mock integrations.
Include responsive layouts plus genuine empty, loading, success, validation, and failure states.
Keep secrets server-side in environment variables, provide .env.example, and never commit credentials or user data.
Add structured logs around every external call and return actionable errors without leaking sensitive details.
Write unit tests for the core logic and one automated test of the main user journey.
Finish with a README covering setup, architecture, data location, backups, tests, deployment, and known limitations.
Acceptance criteria
A clean install starts the app using only the README and .env.example.
The primary journey works from first visit through saved result, reload, edit, export, and deletion where applicable.
Invalid input, missing configuration, provider failure, and an empty database each have a usable state.
The interface works at 390px and 1440px, is keyboard navigable, and shows visible focus on every control.
Tests, type checking, linting, and a production build all pass with no ignored failures.
No part of the interface implies a live integration, security guarantee, or scale capability that was not actually built and verified.
Non-goals
Do not build these, and do not claim to have replaced them:
Meeting bots, team collaboration, automations, and broad integrations.
Calendar auto-join.
Reliable speaker diarization.
Mobile capture.
Connectors, OAuth flows, and vendor API changes require constant upkeep.
Permissions, presence, and shared workflows are difficult to simplify.
What you still own after launch
Secure credentials, rotate secrets, and handle provider rate limits.
Run migrations, backups, restores, and dependency updates.
Test the critical journey after every model, API, or hosting change.
Monitor failures and fix the edge cases a first prompt will miss.
Maintain every third-party integration as APIs and OAuth rules change.
Risk
**Operational risk.** The code is achievable; dependable data, integrations, and ongoing operations are the real cost.
Editorial confidence in this assessment: medium. No independent one-shot implementation is linked yet.
Prior art
Working open-source software you can read, fork, or borrow from before starting:
[whisper.cpp](https://github.com/ggerganov/whisper.cpp) — Widely used local Whisper inference implementation suitable for private transcription
Generated by [Can It Be Vibe Coded?](https://www.canitbevibecoded.com) · Full report: https://www.canitbevibecoded.com/sembly-ai
You still own the product
- Secure credentials, rotate secrets, and handle provider rate limits.
- Run migrations, backups, restores, and dependency updates.
- Test the critical journey after every model, API, or hosting change.
- Monitor failures and fix the edge cases a first prompt will miss.
- Maintain every third-party integration as APIs and OAuth rules change.
Open-source prior art
Before you start
Can Sembly AI be vibe coded?
Partly, if you narrow it. The core loop is buildable, but a dependable replacement becomes a substantially larger project. For Sembly AI, transcribe a call and produce structured minutes, tasks, and searchable topics. The hard boundary is meeting bots, team collaboration, automations, and broad integrations, plus capture reliability, integrations, and collaboration.
What can an AI coding agent reproduce from Sembly AI?
Record or import a call, transcribe it locally, identify speakers with manual correction, and produce structured minutes, tasks, and searchable topics. Capture supplied audio, transcribe it, create structured notes, and export them. A responsive interface with real empty, loading, success, and error states.
What will a DIY Sembly AI replacement still be missing?
meeting bots, team collaboration, automations, and broad integrations; calendar auto-join; reliable speaker diarization; mobile capture; Connectors, OAuth flows, and vendor API changes require constant upkeep.; Permissions, presence, and shared workflows are difficult to simplify.
What do I still own after building a Sembly AI alternative?
Secure credentials, rotate secrets, and handle provider rate limits. Run migrations, backups, restores, and dependency updates. Test the critical journey after every model, API, or hosting change. Monitor failures and fix the edge cases a first prompt will miss. Maintain every third-party integration as APIs and OAuth rules change.