Can Pangram be vibe coded?
A classifier that scores whether a piece of text was written by an AI model.
The interface is a text box and a percentage, which is exactly the kind of thing that tricks you into thinking it is a focused project. The product is not the text box: it is a classifier trained on a very large, continuously refreshed corpus of human writing paired with output from every model release, tuned hard against false positives because accusing a real person of cheating is the failure mode that ends the company. You can absolutely build a local detector from perplexity and burstiness features with a focused implementation, and it will be confidently wrong often enough to be useless for any decision that matters. Nobody outside your own head will accept your homemade score, and the calibration drifts every time a new frontier model ships. Build it to understand the problem, not to rely on it.
Jump to the build brief ↓Legacy-calibrated assessment
Checked Aug 2026
What you pay today, before any DIY hosting
low editorial confidence
Tracked separately from the pricing check
Buildability by layer
Screens, forms, and focused interactions
The repeatable job the product performs
Availability and legality of required data
Uptime, queues, support, and maintenance
Security, compliance, and user confidence
The achievable core
- Paste text, score it locally with a small language model's token log-probabilities plus a few style statistics, and get a hand-wavy human-or-machine guess with a loud accuracy disclaimer.
- Wrap a model API in a focused drafting, revision, and export workflow.
- A responsive interface with real empty, loading, success, and error states.
The parts a prompt cannot buy
- Calibration: a real false positive rate you can quote, instead of a vibe
- Coverage of new models, which changes every few weeks whether you update or not
- Sentence-level and mixed-authorship detection rather than one blunt document score
- Any external credibility, since a self-built score persuades exactly zero teachers, editors or clients
- Model quality and inference operations are part of the product.
- The useful dataset is owned, accumulated, or expensive to reproduce.
Build, switch, or keep paying
Narrower, with trade-offs
Paste text, score it locally with a small language model's token log-probabilities plus a few style statistics, and get a hand-wavy human-or-machine guess with a loud accuracy disclaimer.
Use the build brief ↓No checked option yet
Compare the prior art below or build only the workflow you need.
Recommended
Because the number has to be defensible to someone else. Institutions and publishers are not buying a classifier, they are buying a third party willing to stand behind a false positive rate, plus continuous retraining against whatever model came out last month. A local heuristic detector gives you a plausible-sounding percentage with no error bars, which is worse than nothing when the outcome is an accusation.
Visit Pangram ↗Why people still pay
Because the number has to be defensible to someone else. Institutions and publishers are not buying a classifier, they are buying a third party willing to stand behind a false positive rate, plus continuous retraining against whatever model came out last month. A local heuristic detector gives you a plausible-sounding percentage with no error bars, which is worse than nothing when the outcome is an accusation.
Model quality and inference operations are part of the product.
The useful dataset is owned, accumulated, or expensive to reproduce.
Trust, audits, and counterparties matter more than feature parity.
The brief
Context, requirements, acceptance criteria, non-goals, and the full production standard — as Markdown, ready for any coding agent.
Build brief — a focused alternative to Pangram
Context
Pangram — A classifier that scores whether a piece of text was written by an AI model. It currently costs $20/mo.
The interface is a text box and a percentage, which is exactly the kind of thing that tricks you into thinking it is a focused project. The product is not the text box: it is a classifier trained on a very large, continuously refreshed corpus of human writing paired with output from every model release, tuned hard against false positives because accusing a real person of cheating is the failure mode that ends the company. You can absolutely build a local detector from perplexity and burstiness features with a focused implementation, and it will be confidently wrong often enough to be useless for any decision that matters. Nobody outside your own head will accept your homemade score, and the calibration drifts every time a new frontier model ships. Build it to understand the problem, not to rely on it.
This brief describes a focused, single-operator replacement for the part of Pangram that is genuinely reproducible. It is deliberately narrower than the product it replaces, and it says so in writing. Build the useful core; do not pretend to have rebuilt the rest.
What you are building
Paste text, score it locally with a small language model's token log-probabilities plus a few style statistics, and get a hand-wavy human-or-machine guess with a loud accuracy disclaimer.
Wrap a model API in a focused drafting, revision, and export workflow.
A responsive interface with real empty, loading, success, and error states.
Requirements
Functional
Local Python with torch and transformers.
A couple of GB of disk for small model weights, CPU works but is slow.
Your own labelled samples of human and AI text if you want any idea of accuracy.
Non-functional
Accessibility: semantic markup, labelled controls, visible focus, and reduced-motion support.
Security: server-side secrets, validated input, and no credentials in the client bundle.
Reliability: retries with backoff on external calls, and a clear failure state when a provider is down.
Portability: the operator can export their data and leave without losing it.
Implementation brief
Build a local AI-text-detection playground so I can see how weak naive detection actually is. No cloud calls, no accounts, no telemetry.
Stack, no substitutions: Python 3.11, FastAPI + uvicorn, transformers + torch, a single server-rendered HTML page with vanilla JS. No build step, no database, no Docker.
Core loop:
1. One page with a large textarea and an Analyze button.
2. POST /analyze takes the text and returns a JSON report.
3. Scoring runs entirely locally with gpt2 (small) loaded once at startup via transformers. Download on first run and cache it.
Features to compute per submission:
Mean token log-probability and perplexity under gpt2.
Burstiness: standard deviation of per-sentence mean log-probability.
Rank-based signal: fraction of tokens that were in the model's top-10 predictions.
Style stats: sentence length mean and variance, type-token ratio, punctuation counts, count of common LLM filler phrases from a hardcoded list.
Output: a 0-100 score from a simple weighted formula over those features with the weights in a single WEIGHTS dict at the top of the file, per-sentence heat colouring in the page, and a full feature table so I can see what drove the score.
Calibration, and be honest about it:
Add a CLI command: python calibrate.py --human dir --ai dir that runs the features over two folders of .txt files, prints mean and spread per class, and prints the ROC AUC of the current weights.
The web page must show a fixed banner: this is an uncalibrated heuristic, not evidence, false positives are common.
Explicitly out of scope: user accounts, saved history, PDF or DOCX parsing, any hosted API, model fine-tuning, mixed-authorship segmentation, batch uploads.
Deliver: main.py, calibrate.py, features.py, templates/index.html, requirements.txt, and a README with run instructions and one paragraph stating plainly why this cannot match a trained commercial detector. No .env needed since there are no secrets; say so in the README.
Delivery standard
Inspect the repository first, then write a short implementation plan before writing code.
Deliver the smallest complete end-to-end workflow first; every primary control must work against persisted data.
Use real validation and storage; never substitute fake dashboards, decorative controls, hard-coded success states, or mock integrations.
Include responsive layouts plus genuine empty, loading, success, validation, and failure states.
Keep secrets server-side in environment variables, provide .env.example, and never commit credentials or user data.
Add structured logs around every external call and return actionable errors without leaking sensitive details.
Write unit tests for the core logic and one automated test of the main user journey.
Finish with a README covering setup, architecture, data location, backups, tests, deployment, and known limitations.
Acceptance criteria
A clean install starts the app using only the README and .env.example.
The primary journey works from first visit through saved result, reload, edit, export, and deletion where applicable.
Invalid input, missing configuration, provider failure, and an empty database each have a usable state.
The interface works at 390px and 1440px, is keyboard navigable, and shows visible focus on every control.
Tests, type checking, linting, and a production build all pass with no ignored failures.
No part of the interface implies a live integration, security guarantee, or scale capability that was not actually built and verified.
Non-goals
Do not build these, and do not claim to have replaced them:
Calibration: a real false positive rate you can quote, instead of a vibe.
Coverage of new models, which changes every few weeks whether you update or not.
Sentence-level and mixed-authorship detection rather than one blunt document score.
Any external credibility, since a self-built score persuades exactly zero teachers, editors or clients.
Throughput, batch uploads, API access and document parsing.
What you still own after launch
Run migrations, backups, restores, and dependency updates.
Test the critical journey after every model, API, or hosting change.
Monitor failures and fix the edge cases a first prompt will miss.
Risk
Operational risk. The code is achievable; dependable data, integrations, and ongoing operations are the real cost.
Editorial confidence in this assessment: low. No reviewed project implementation is linked yet.
Generated by Can It Be Vibe Coded? · Full report: https://www.canitbevibecoded.com/pangram
You still own the product
- Run migrations, backups, restores, and dependency updates.
- Test the critical journey after every model, API, or hosting change.
- Monitor failures and fix the edge cases a first prompt will miss.
Projects built from this idea
No reviewed implementation has been linked for Pangram yet. A submission is evidence for review, not automatic proof that the whole product was replaced.
Built a version of Pangram?Submit the project as evidence for this report.
Before you start
Can Pangram be vibe coded?
Not faithfully. The interface is a text box and a percentage, which is exactly the kind of thing that tricks you into thinking it is a focused project. The product is not the text box: it is a classifier trained on a very large, continuously refreshed corpus of human writing paired with output from every model release, tuned hard against false positives because accusing a real person of cheating is the failure mode that ends the company. You can absolutely build a local detector from perplexity and burstiness features with a focused implementation, and it will be confidently wrong often enough to be useless for any decision that matters. Nobody outside your own head will accept your homemade score, and the calibration drifts every time a new frontier model ships. Build it to understand the problem, not to rely on it.
What can an AI coding agent reproduce from Pangram?
Paste text, score it locally with a small language model's token log-probabilities plus a few style statistics, and get a hand-wavy human-or-machine guess with a loud accuracy disclaimer. Wrap a model API in a focused drafting, revision, and export workflow. A responsive interface with real empty, loading, success, and error states.
What will a DIY Pangram replacement still be missing?
Calibration: a real false positive rate you can quote, instead of a vibe; Coverage of new models, which changes every few weeks whether you update or not; Sentence-level and mixed-authorship detection rather than one blunt document score; Any external credibility, since a self-built score persuades exactly zero teachers, editors or clients; Model quality and inference operations are part of the product.; The useful dataset is owned, accumulated, or expensive to reproduce.
What do I still own after building a Pangram alternative?
Run migrations, backups, restores, and dependency updates. Test the critical journey after every model, API, or hosting change. Monitor failures and fix the edge cases a first prompt will miss.