Status: Accepted and build-out complete for this pass. Every
library marked PLUGIN or PLUGIN (candidate) below that was
practically buildable in this environment now has one -- 30 plugins
total (up from the 9 that existed when this ADR was first written).
See docs/plugin-guide.md's table for the full current list; the
triage below is left as originally written, since the reasoning still
holds -- only the verdicts for a handful of specific libraries changed
after actually building against them, noted inline where relevant.
What changed on contact with real code, beyond what the original
triage predicted:
redis-py was downgraded from "candidate, low priority" to
confirmed no plugin needed -- checked directly rather than left
as a guess: redis.exceptions.OperationFailure and its siblings
carry no structured data beyond the message string, so there is
nothing this ecosystem's model adds over tier 1.
psycopg2 could not be built in this environment: its
.pgcode/.pgerror are C-level, read-only member_descriptor
attributes populated only by a real PostgreSQL connection --
confirmed by trying to set them and getting AttributeError:
readonly attribute, not assumed. asyncpg (built instead) doesn't
have this constraint; its equivalent fields are plain, settable
Python attributes.
pymongo, jsonschema, asyncpg's own __str__, and PyYAML's
ConstructorError.problem all turned out to bake the value they
were being asked to protect directly into a string this plugin
initially assumed was safe (the library's own message text) --
caught by each plugin's own redaction test, not anticipated in
advance. docs/plugin-guide.md's table marks each of these with an
asterisk and links to the specific finding.
whytrail-scrapy had a real, silent-failure bug: pydispatch (which
Scrapy's signals are built on) defaults to weak references, so the
handler closure was garbage-collected the instant install()
returned and the signal reached zero receivers -- in production this
would have looked like the plugin doing nothing, with no error at
all. Fixed with weak=False, caught by checking send_catch_log's
return value in a test, not by reading pydispatch's documentation.
google.api_core.exceptions.GoogleAPICallError turned out to be
the shared base every google-cloud-* client library raises from, not
storage-specific -- one plugin (whytrail-google-cloud) covers
storage, bigquery, pubsub, and firestore, verified against three
different concrete exception types mapping to different underlying
services, not just the one originally scoped.
LlamaIndex was reassessed and deferred: architecturally
identical to whytrail-langchain's callback-based design, so building
it now would mostly duplicate already-proven work rather than test
anything new. A real candidate for later, not ruled out.
None of this changes the ADR's central claim -- if anything it
reinforces it. The value in this ecosystem came from testing against
real objects and finding out what the library actually does, not from
correctly guessing it in advance.
After nine real plugin distributions, the question became: does this
scale to covering the top ~150 Python libraries? The honest answer,
learned directly from building those nine (whytrail-pandas in
particular): most libraries don't need a plugin at all.DataFrame/Series support weakref and id()-based identity like
any other Python object, so generic track()/@tracked already works
on them with zero pandas-specific code — the plugin only earns its
keep for the untracked-diagnostic case. That pattern holds for the
large majority of the ecosystem.
A library only warrants a dedicated plugin if it clears one of three
bars, all visible across the nine plugins already built:
Structured domain error data a bare traceback discards —
StatementError.statement/.params (SQLAlchemy), Response/
RequestException.request (requests), a validation error's
per-field breakdown (Pydantic). The exception already knows more
than str(exc) shows.
A security-sensitive boundary needing safe-by-default design —
FastAPI/Django, where the actual work is redaction defaults, not
explaining anything new.
A non-standard capture mechanism — LangChain's callbacks,
Celery's signals, pytest's hooks. The object model itself doesn't
need explaining; the lifecycle does.
A library that fails all three gets no plugin, and that's the
correct, final answer for it — not a placeholder for later. Building a
thin wrapper anyway is the "fifteen integrations done shallowly"
failure mode ADR 0002 §7 already rejected at n=15; doing it at n=150
would be the same mistake at ten times the scale, and would actively
damage trust in the plugin ecosystem's quality bar.
Decision: a generator, not hand-written boilerplate each time¶
scripts/new_plugin.py scaffolds the pyproject.toml, package stub,
README, and starter test for either shape (--kind explainer for the
entry-point pattern, --kind integration for the hook-based pattern).
It deliberately does not attempt to generate the actual explainer
logic — that's the judgment call in category 1 above, not boilerplate.
Every TODO it emits is a place a human decision is still required.
~100 libraries spanning the categories most represented in real
Python codebases. PLUGIN = clears one of the three bars above.
GENERIC = why() already works via track()/@tracked with no
library-specific code; a plugin would add polish, not capability.
N/A = no runtime object/exception surface why() has a role
against (build tooling, dev-time linters, low-level transitive deps).
Of roughly 100 libraries actually assessed: 9 already have a real
plugin, 2 more (Pydantic, boto3) were added from this triage,
roughly 20 are credible future candidates ranked by the reasoning
above (httpx and psycopg2 are the strongest next two), and the
remainder are either already fully served by generic tracking or
structurally out of scope. That ratio — a genuine plugin need in
roughly a tenth to a fifth of what was surveyed — is the actual answer
to "does this scale to 150 libraries." It scales as an audit
methodology and a generator for the ones that clear the bar; it does
not, and should not, scale as "150 plugin packages."