A smoke test that never touches the fragile path proves nothing
Janus's CI smoke ran `janus --help`, which never loads an embedded asset, so it missed an ENOENT that broke every asset path in the compiled binary. Point the smoke at the fragile path on purpose — a compiled binary has a different filesystem than your dev tree.
Janus compiles to a single binary with bun build --compile. HTML and
CSS templates loaded via readFile(import.meta.dir/...) resolve to the
virtual /$bunfs filesystem inside that binary and throw ENOENT at
runtime — fixed on June 16 (a8c5651) by embedding them with
with { type: "text" } import attributes. The CI already ran a binary
smoke test. But it invoked janus --help, which never loads an embedded
asset, so it could not and did not catch the regression. On June 19
(f45a5dd) the smoke became janus demo --no-open, which actually
renders through the asset path.
Green CI is only as honest as the path it walks
A smoke test protects the code paths it exercises and nothing else.
--help passes trivially: it parses no config, opens no file, reaches no
/$bunfs lookup. It is the definition of a passing entrypoint that
proves the process starts and proves nothing about the process working.
The regression lived one layer deeper, in the first readFile against an
embedded template, and no amount of green on --help was ever going to
reach it.
A compiled artifact has a different filesystem than your dev tree
The bug was invisible in development because in the dev tree
import.meta.dir points at a real directory and the templates are real
files. Under bun build --compile the same call resolves into a virtual
filesystem where those relative reads simply are not present. The
artifact you ship is a different machine from the one you build on. Any
smoke that runs the source instead of the binary — or runs the binary but
avoids its filesystem — tests the wrong machine.
Aim the smoke at what actually breaks
The failure classes that only surface in a compiled or bundled artifact
are specific: filesystem access, embedded assets, native dependencies.
Those are where the smoke has to land. Switching to janus demo was
deliberate (decision-5 in the pulse) precisely because demo forces the
asset-rendering path that --help sidesteps. The rule generalizes: a
smoke test should exercise the most fragile path on purpose, not the
cheapest one to keep green.