kanaria007 PRO

Yes, I ended up taking both branches of your last question, but at different layers.

For hf_integrity_job.py, I removed the direct-run fallback entirely. The Job wrapper now requires the revision-bound webhook trigger context; without it, it fails explicitly and never executes the mounted checker. Local/manual working-tree validation still exists, but its entrypoint is repo_integrity.py directly. So the deployed Job no longer has a weaker Git-discovered Authority ingress sitting beside the manifest path.

For local Profile A, I kept Git-backed Authority and hardened it rather than pretending the problem disappeared with the Job branch.

The first change was the one your .git experiment pointed at directly: .git is no longer a Boolean existence predicate. The local provider now distinguishes positive absence, observation failure, and observed-but-unbound metadata before any tracked path set is trusted.

The supported shapes are deliberately narrow:

• an ordinary .git directory is qualified against Git's own core.bare and core.worktree semantics;
• a standard linked-worktree gitfile is accepted only when its private gitdir has the Git back-binding to that exact gitfile;
• a .git symlink is rejected as unbound Authority rather than followed;
• a foreign or malformed gitfile is rejected before path loading;
• the real submodule gitfile layout is not claimed as supported by this pass.

That exposed the same trust-boundary problem twice more upstream.

First, the git config calls used to establish Authority were themselves capable of being redirected by the ambient Git environment. In particular, a foreign GIT_COMMON_DIR could make a command naming the correct --git-dir read another repository's config, and Git's config-injection environment could alter a plain config query.

So qualification and consumption now share the same sanitized subprocess environment. The qualification-time git config --local calls and the final git ls-files invocation strip the repository-selection variables, GIT_INDEX_FILE, object-directory overrides, and the indexed GIT_CONFIG_COUNT / GIT_CONFIG_KEY_* / GIT_CONFIG_VALUE_* surface.

Second, qualifying the git-dir did not qualify the object inside it that actually supplies ls-files with its path set. A clean victim/.git with:

victim/.git/index -> other/.git/index

still made the fully explicit:

git --git-dir=victim/.git --work-tree=victim ls-files

return the other repository's paths.

So the local chain is now:

working-tree root -> git-dir binding -> index binding -> git ls-files

The qualified git-dir's index is inspected with lstat() before Git is invoked:

• absent -> allowed, because a fresh repository may not have an index yet;
• regular file -> accepted as the local Profile-A Authority object;
• symlink or other filesystem object -> Authority binding failure;
• non-absence observation failure -> Authority probe failure.

Only after that does the tool call Git, with the already-qualified --git-dir, explicit --work-tree, and sanitized environment.

The focused Authority regression suite now includes real Git witnesses for the foreign .git symlink, foreign gitfile, linked worktree, core.worktree, Git boolean spellings for core.bare, environment substitution, and the symlinked-index substitution. The current focused run is 32/32.

I stopped the hardening boundary there deliberately. Profile A still treats the regular local Git index as its Authority; it does not claim external revision binding or independent content provenance for the index bytes. Those stronger claims belong to the manifest/revision-bound profiles.

So the abstraction I ended up with is slightly stronger than the four-state .git classification alone:

some Git metadata observed != Authority bound to this root

and then, recursively,

Authority source qualified != Authority object qualified != the qualified Authority is what the subprocess actually consumed.

Your “one function up” observation turned out to be the right direction: the weakness was not just in the next predicate, but in the process that establishes the universe the rest of the checker is allowed to validate.

Yes, I kept the ordering and gave reject_symlink_traversal() its own probe-failure state.

I think your last sentence identified the real design choice. Moving namespace validation first would make the immediate traceback disappear, but it would weaken the boundary the current ordering is enforcing: before I let namespace resolution inspect a deeper component, I want the existing lexical prefixes qualified as non-redirecting. With a symlinked parent, a deeper lookup can otherwise cross that redirection before the namespace check has any reason to complain.

So the write-side order remains:

reject_symlink_traversal -> validate_observation_namespace -> write

but the first guard is no longer Boolean in the relevant sense.

I split the symlink side into a small typed hierarchy:

• SymlinkTraversalError: a symlink was positively observed;
• SymlinkTraversalProbeError: the checker could not establish the no-symlink precondition;
• both share SymlinkValidationError for call sites that only need “this guard failed”.

I also replaced Path.is_symlink() in that guard with explicit os.lstat() + stat.S_ISLNK() classification. That lets the checker own the distinction it actually needs:

• ENOENT / ENOTDIR -> positive non-materialization; stop the lexical walk cleanly;
• successful lstat, non-symlink -> continue;
• successful lstat, symlink -> traversal error;
• any other observation failure -> symlink probe failure.

That avoids depending on Path.is_symlink()'s own version-dependent error policy, where some lookup failures are normalized to False and others may propagate as raw exceptions.

The read path now turns that probe failure into its existing structured unreadable ... diagnostic, and the generate/write path reports it through the normal checker failure channel. The inaccessible-parent case therefore remains fail-closed, but it is now a declared verdict rather than an implementation traceback.

I pinned the ordering too, not just the exception type: the regression witness makes the symlink guard fail and makes namespace validation a sentinel that must never be reached. So “non-redirection before namespace materialization” is now a tested contract rather than incidental source order.

The self-loop case still behaves the way you observed. The lexical walk sees the loop component itself, positively identifies the symlink there, and raises the actual traversal error before any deeper loop/a.md lookup can turn into an ELOOP-shaped observation problem.

The current symlink suite is 11/11 passing on a case-sensitive test host.

So the state-space correction has now propagated one function further:

not observed as a symlink was never equivalent to observed non-symlink.

The useful states are SYMLINK, NON_SYMLINK, ABSENT, and UNKNOWN, and UNKNOWN now stays inside the checker as a first-class failure instead of escaping as a traceback.

And I agree with your scope note: this closes the contributor/CI generate path you exercised. I am not treating it as evidence that the deployed Hugging Face check path has exhibited the same failure.

Yes. I think the distinction you drew is the right one.

I do not have a receipt showing the exact stat succeeds -> scandir fails sequence on the deployed Hugging Face mount, so I cannot answer that part with “yes, the Job has done this.” What I have observed from that substrate is the truncated-read behavior we discussed earlier. I also do not have evidence that every mount-layer directory-enumeration failure necessarily kills the container before the checker can see it.

Given that uncertainty, I changed the checker rather than guessing which behavior the mount guarantees.

You were right about the asymmetry: the old code treated multiple candidates as unknown, but an unavailable candidate set could collapse to None and therefore look like absence.

_resolve_directory_entry() now distinguishes the states explicitly:

• None is reserved for a lookup that positively reports absence (ENOENT / ENOTDIR);
• other failures of the initial stat() are repository observation namespace probe failure;
• a scandir() failure after a successful target lookup is the same explicit probe failure, never absence;
• if non-exact resolution requires inspecting directory-entry identities and one of those observations fails, that is also a probe failure rather than silently skipping the entry;
• if the initial lookup succeeds and a complete directory enumeration succeeds but contains no entry with the identity the lookup returned, that is treated as a self-contradictory observation and fails closed too, rather than being reinterpreted as “missing”;
• multiple possible entries remain the separate repository observation namespace ambiguity case.

The exact-name fast path is unchanged, so ordinary exact-spelling hardlinks do not become errors merely because another directory entry shares the inode.

The same namespace preflight still runs before repository content is read and before generated outputs are written, so a probe failure blocks both paths.

I also moved the new failure shapes into host-neutral regression witnesses. The current namespace suite is 12/12 passing on a case-sensitive test host, with only the two real-filesystem case-alias integration cases skipped because that substrate cannot construct them.

So I would phrase the answer to your closing question as:

I do not yet know whether the deployed mount can produce that exact transient scandir() failure without killing the Job first. The checker no longer needs that assumption to be true. If namespace observation cannot be completed, or completes with mutually inconsistent results, silence is no longer interpreted as clean.

And yes, “two candidates and you refuse to guess; zero candidates and you guess missing” was the useful reduction. What looked like one “missing” state was actually two different states: ABSENT and UNKNOWN, and the old implementation had collapsed them.

Yes, at the point you asked, it only rejected symlink traversal. You found a real sibling gap.

I generalized the fix from a symlink policy into an observation-namespace preservation check.

The repository authority still keeps the exact locator strings supplied by git ls-files or the revision manifest. Before any content is read, repo_integrity.py now asks the host filesystem what directory entry each authoritative path prefix resolves to.

It deliberately does not casefold() or Unicode-normalize paths itself. I did not want to replace the filesystem's semantics with a guessed model of them.

The rule is instead:

• a single unusual spelling is allowed;
• if two distinct authoritative prefixes resolve to the same real directory entry, the run fails closed with repository observation namespace collision;
• the check is performed at every prefix, so Dir/a.md vs. dir/b.md is caught at the parent-directory level as well as Foo.md vs. foo.md at the final filename;
• the same namespace check runs before generated outputs are written, not only before reads.

So it is not quite “the index spelling must equal the filesystem spelling.” A single NFC authority path opening an NFD on-disk entry is not rejected merely for that. The invariant is that distinctions made by Authority must not silently collapse when materialized by the Observation filesystem.

There is one hardlink wrinkle I handled separately. Two exact directory entries that share an inode are permitted, because this inventory counts locators, not inode identities. But if a non-exact lookup lands on an inode represented by multiple directory entries, the checker cannot determine which entry the filesystem selected. That fails closed as repository observation namespace ambiguity rather than guessing and potentially reporting a false collision.

The regression evidence is split accordingly: the collision judgment itself has host-neutral tests using explicit authoritative-prefix → observed-entry mappings, while actual case-insensitive filesystem behavior is covered by host-dependent integration tests and skips on hosts where that failure shape cannot be constructed.

So the short answer to your last question is:

before your comment, only symlinks;
now the fail-closed boundary checks both redirection and namespace collapse against the filesystem's actual resolution behavior.

I also think your “same failure mode running the other direction” framing was the right abstraction. The symlink case was one Authority locator being redirected to a different filesystem object; this case was two Authority locators collapsing onto one Observation entry. Treating both as namespace-preservation failures made the fix much cleaner.

Yes, I kept the named-class tier as a genuine antichain.

I made one more pass over the evidence boundary before closing this, and there were a couple of useful corrections.

First, I removed the wording that made it sound as though the committed suite itself exhaustively enumerated all 120 guard orders. It does not. The in-repo tests pin the distinguishing precedence edges and the multi-named consumer behavior. Your 120-order run is now recorded explicitly as an external exhaustive audit of that suite's acceptance surface.

I also do not count that audit as a second independent acceptance oracle, since it used the same regression suite to decide PASS/FAIL for each permutation. It is corroborating evidence that the individual witnesses compose to exactly the six expected linear extensions, not an independent evidence root.

Second, MRK < ASC is now witnessed for every currently selected marker, not just the regional 4-byte marker. That still does not impose any ordering among the four markers themselves; each one is tested only against ASC.

So the intended contract remains:

{CR, LF, MRK} < ASC < ENV

with CR/LF/MRK unordered relative to one another, and MRK treated as one guard block at that contract level.

While reviewing the same Authority/Observation boundary, I also found and closed a separate local-Git edge case: a tracked symlink could let git ls-files name a symlink blob while the filesystem read followed it and observed the target's bytes instead. Reads and generated-output writes now fail closed on symlink traversal, including symlinked parent directories.

I tightened two other evidence descriptions at the same time: the live-domain sentinel now claims only the qualified surfaces independently enumerated by the test harness, and the three content-identity implementations are described as independently maintained consumers of one pinned oracle, not as three independent oracle roots.

I left gateway deployment provenance and webhook retry/liveness as separate operational questions rather than folding them into this patch.

So I think the ordering thread is closed at the right boundary now: forbidden variance is pinned, permitted variance remains permitted, and the evidence descriptions no longer claim stronger provenance than the tests actually provide.

Yes, 6 of 120 is the right target.

And I would keep ASC < ENV as part of the contract.

They are both residual in a broad sense, but not at the same residual layer. isascii() owns non-ASCII contamination that did not match one of the selected named markers; the lowercase-hex envelope owns what remains after that, such as TAB/NUL/space/uppercase-style ASCII drift. If ENV moves above ASC, that distinction collapses and unknown non-ASCII gets attributed to the lowercase-hex construction envelope instead.

So the intended partial order is:

{CR, LF, MRK} < ASC < ENV

with no promised relative order among CR, LF, and MRK.

You're also right about why the suite was still at 12. The single-violation CR and LF witnesses established reachability, but not the CR < ASC or LF < ASC precedence edges.

I've added the two mixed witnesses through the actual shared _check_dynamic_probe_filler() path:

• CR + non-selected non-ASCII → CR diagnostic
• LF + non-selected non-ASCII → LF diagnostic

I deliberately did not add CR + marker or similar witnesses, because that would start constraining the relative order inside the named-class tier and turn the partial-order contract into an arbitrary total order.

I also tightened the wording to state the contract explicitly as a partial order, and to be precise that the 120 permutations are over five guard blocks, treating the selected-marker loop as one block.

The exhaustive-permutation framing is a better way to state what “pinned” means here than my earlier wording was: the useful target is not that every source-code position is fixed, but that the suite accepts exactly the linear extensions of the declared diagnostic partial order.

Thanks — this was a very clean way to expose the remaining gap.

Thanks, it was not deliberate. It was the one missing witness class in this precedence test.

Your mutation makes the gap very clear: the existing cases reached CR, LF, a selected marker, and the residual ASCII envelope case, but none of them made the relative position of isascii() and the envelope observable.

I added a U+00FC (ü) case for exactly that reason.

It is:

• non-ASCII;
• not one of the current _FILLER_MARKERS;
• outside the lowercase-hex construction envelope.

With the intended order, it must therefore reach isascii() first and report the isascii() diagnostic. If the envelope moves ahead of isascii(), the observed attribution changes to the alphabet diagnostic.

The new case goes through the same shared _check_dynamic_probe_filler() path as the other precedence witnesses; there is no second hand-copied ordering model.

I am also treating U+00FC strictly as a test-only precedence witness, not as a new regional representation class. The matrix claim surface is unchanged.

I tightened the wording around what the regression actually promises as well. The interesting contract is not an arbitrary exact total order among all five guards — for example, I do not particularly care whether CR precedes LF as an abstract property. What matters is the diagnostically meaningful partial order: specific attribution should remain reachable before broader or residual exclusion can shadow it.

So I think the reusable rule becomes slightly sharper:

A precedence boundary is only tested when some witness changes its observed attribution if that boundary is inverted.

Or, shorter:

No distinguishing witness, no precedence evidence.

Your “a precedence test is worth what it fails on” mutation was the right way to expose this one. Thanks for pushing the rule back onto the test itself.

Thanks. Yes, the ordering matters, and envelope-first was not deliberate.

Your 110-byte count also changes how I would describe the relationship between the two layers.

I had written that a wider claim vocabulary could eventually eliminate the gap between the claim-bound checks and the lowercase-hex envelope. That is technically true, but misleading as a design direction. The measured residual is much larger and more structural than that wording suggests: adding TAB would only move it from 110 to 109. Eliminating it would effectively mean modeling the whole non-hex ASCII space.

So I now think the cleaner distinction is:

• the named checks describe the representation classes this matrix currently claims to discriminate;
• isascii() catches the remaining non-ASCII space;
• the lowercase-hex restriction is intentionally broader and remains a residual fixture-construction backstop.

I did not add TAB or any other new class.

I also changed the fail-fast ordering to:

CR → LF → selected multibyte markers → isascii() → lowercase-hex envelope

so the most specific modeled class gets to name the failure before a broader guard shadows it. A wrapped filler now reaches the LF diagnostic first; selected multibyte contamination reaches its marker-specific diagnostic; unrelated non-ASCII falls to isascii(); TAB/NUL/space/uppercase-style drift falls through to the envelope.

I also pinned that ordering through the actual shared guard path rather than testing a separately duplicated ordering model, so the regression test cannot stay green if the real guard sequence drifts.

I think the reusable rule here is:

In a fail-fast harness, order specific attribution before residual exclusion.

Or more generally: a diagnostic guard only contributes the diagnostic role we attribute to it if that guard is actually reachable as the observed failure.

Thanks for measuring the byte-space rather than leaving the “wider vocabulary” claim qualitative. The 110-byte number made the boundary much clearer.

Thanks, your table convinced me that the labels were wrong even though the fixture behavior was right.

I changed the documentation around test_dynamic_probe_filler_is_marker_free to separate three different roles more accurately.

The lowercase-hex restriction is now described as the fixture's construction envelope, not as a disposable implementation choice. Its useful property is broader than the current marker list: it keeps the filler vocabulary deliberately narrower than the representation vocabulary, so edits such as TAB-separated formatting, NUL padding, uppercase hex, or a different alphabet are rejected before they can quietly become new fixture semantics.

The CR/LF/current-marker checks are now described as claim-bound checks for the representation classes the matrix models today.

And I corrected the multibyte description as well: for the current selected UTF-8 markers, those explicit checks are not an independent enforcement dimension beyond isascii(). They remain useful because they make the intended representation exclusions local and diagnostic, but I no longer count them as separate assurance.

So the dependency is now stated in the right direction:

lowercase hex -> ASCII -> current selected multibyte markers absent

but absence of those selected markers does not imply ASCII.

On TAB specifically: yes, I think an ASCII-control representation class is a plausible future matrix extension, with TAB being the obvious first candidate. I did not add it now, though. The current matrix is still a bounded falsifier for the failure families it already claims to exercise, and I do not want to broaden that claim surface just because another class is imaginable.

What I did make explicit is the maintenance relationship you pointed out: the filler vocabulary and the modeled representation-class vocabulary are coupled fixture structures. If a new class such as TAB is added later, the filler-envelope assumptions need to be revalidated at the same boundary.

There is another reusable rule here too:

Qualify the guard before attributing assurance.

A guard's assurance role should come from what it independently excludes, which other guards subsume its exclusion set, and which claim boundary it actually protects — not from how important its comment sounds.

Your table was a nice small example of exactly that: it tested the docstring's classification instead of trusting the classification because it was written down.

Thanks, I think your reclassification is exactly right, and the measurement made the reason stronger than either of us had stated before.

The singleton property was not drifting with size. The current filler is plain hex and disjoint from all five representation-marker classes, so its growth cannot accidentally mint another CRLF, bare LF, or selected multibyte sequence.

The remaining issue was one layer higher: that property was load-bearing for the mutation experiment, but the harness was relying on it rather than independently proving that the experiment it was about to interpret was actually a single-cell removal.

I tightened that boundary.

The harness now checks the baseline witness counts before constructing any validator claim: every one of the 15 regional cells must have exactly one witness. For each mutant, it then verifies that the target changed from exactly 1 -> 0 and that every non-target cell count remained unchanged. Only after those checks pass is the validator allowed to produce a survivor or localization result.

I also pinned the 64-column wrapping case directly. A formatting change that introduces additional bare LFs now fails as a fixture precondition failure before the validator is evaluated, rather than surfacing as “removed but undetected.”

So the distinction is now explicit:

removed 1 of N, where N > 1
→ the falsifier itself is not admissible for that claim

removed 1 of 1, target becomes 0, everything else stays fixed
→ now the validator can legitimately be implicated if it misses the removal

I think there is a useful general rule hiding in this one:

Qualify the intervention before attributing the outcome.

A detector can only be blamed after the harness has independently established that the condition it was supposed to detect was actually instantiated. Otherwise a red test may still be useful, but its failure attribution is unsound.

That also sharpens the point you made about this being fail-loud rather than fail-silent. The suite was already telling us that something had gone wrong; what it did not yet know was which layer had gone wrong.

Thanks for going back and measuring the original drift hypothesis rather than carrying it forward. The correction itself exposed the more interesting issue.

Yes, I took the sharper version and turned it into an explicit first-order mutation test.

The regional claim is now represented as exactly fifteen pinned cells:

5 representation classes × 3 regions

with the exact ordered set itself asserted separately, so the mutation matrix cannot silently shrink from 15/15 to 12/12 if a class disappears from the enumeration.

For each cell, one at a time, the test:

1. starts from a genuine generated probe;
2. locates that marker from the actual emitted bytes rather than re-deriving the generator's placement arithmetic;
3. overwrites only that marker with the corresponding bytes from the plain ASCII dynamic probe;
4. checks that length and UTF-8 validity are unchanged;
5. runs the per-class/per-region validator;
6. requires the validator to report exactly that one cell as missing.

So the acceptance condition is stronger than “some assertion went red”:

remove(cell_i) -> missing == [cell_i]

for all fifteen cells independently.

I also pinned the fifteen-cell inventory itself separately. That closes a smaller self-reference trap I noticed while implementing your suggestion: if the mutation test merely iterated whatever cells the validator currently knew about, deleting a class from both sides could reduce the matrix to 12/12 and still look healthy.

The current result is:

15 mutants
15 detected
15 correctly localized
0 survivors
0 mislocalized
0 impure mutants

The mutation harness does not reuse the generator's placement formulas. It scans the actual generated payload for the claimed marker in the claimed region, then removes that observed marker. That keeps “where the generator says it wrote something” separate from “what the fixture actually contains.”

So this is now red-run evidence at the same resolution as the regional claim, rather than an inference from reading the validator.

I also kept the scope narrow. This is first-order single-placement falsification of the fifteen regional claim cells, not pairwise or higher-order combinatorial mutation coverage. I do not think fifteen separate named test functions are the important property here; detection and localization at the fifteen-cell claim surface are.

Your unrelated-package example helped sharpen why.

A differential metric can be perfectly implemented and still have a common-mode fault direction that leaves the published value unchanged. The earlier shared 😀 witness had the same structural shape at a smaller scale: two claims intended to be independently falsifiable collapsed onto one observation, so one claim could disappear without changing what the validator saw.

The failure mode in both cases is not simply “insufficient test count.” It is an observationally null fault direction: a claim-relevant change exists that the evidence mapping does not distinguish.

Splitting 🧪 from 😀 removed one such ambiguity between the regional and alignment claims. The fifteen one-at-a-time removals now test the next level down: whether each declared regional cell is itself distinguishable under the selected single-removal fault family.

So I think the rule sharpens beyond just matching assertion structure:

A claim is only as fine-grained as the smallest claim-relevant fault its evidence system can distinguish.

Or operationally:

claim granularity, validation granularity, and falsification granularity should agree at the surface being asserted.

That still does not imply exhaustive combinatorial coverage. The fifteen removals establish first-order sensitivity for the fifteen declared cells; they do not prove arbitrary two-cell interactions, substitutions, relocations, or unrelated representation transforms.

Thanks — this was a good next test because it moved the question from “is the validator written at the claimed resolution?” to “can the evidence system actually observe a fault at that resolution?”

That is a much stronger distinction.

Yes, I ended up taking both.

The overlap is now a construction failure rather than a silent yield, and the fixture tests now assert the representation classes independently per region instead of collapsing them through the old OR predicate.

I also took your +3 adjustment for the two colliding CRLF positions. At the current 262,144-byte live-domain-derived probe size, that gives CRLF in all three regions while preserving all 14 selected boundary straddles.

There was one more version of the same problem hiding underneath that: the regional 4-byte markers and the selected-boundary 4-byte markers were initially the same code point, so a missing regional marker could still be masked by a boundary marker in the same region.

I split those as well:

regional 4-byte marker -> 🧪
selected-boundary straddle marker -> 😀

so the two claims are now independently observable rather than merely independently placed.

The generator contract is now effectively:

required placement collision -> construction error
regional representation -> CRLF / bare LF / 2-byte / 3-byte / 4-byte independently present in early, middle, and late regions
selected-boundary alignment -> 4-byte UTF-8 straddles every selected power-of-two boundary

I added a direct collision regression as well, rather than relying only on a too-small probe failing out of bounds. The probe fixture suite now passes 26/26, and removing only the regional 4-byte placements now correctly breaks the regional-distribution test instead of being masked by the alignment markers.

So I think your diagnosis was right: the problem was not another missing sentinel. It was that the sentinel generator could silently fail to realize part of the evidence surface it claimed to construct, and its own validation was coarser than that claim.

The useful rule I am taking from this round is:

For required test evidence, construction should fail closed, and fixture validation should be at least as fine-grained as the evidence claim it supports.

Thanks, this was a good catch. The second sentinel still keeps the diagnostic separation we wanted, but required evidence can no longer disappear silently during fixture construction.

Yes, I think this attack found a real missing interaction cell rather than another weakness in either V2 or the size sentinel individually.

The suite had per-axis coverage on both dimensions:

representation fidelity -> V2
large-input consumption -> fixed ladder + dynamic size sentinel

but it did not have joint coverage of representation-sensitive behavior at large sizes. Your gated-CRLF mutant and chunked decode/re-encode mutant sit exactly in that empty interaction cell.

I kept the existing dynamic size sentinel unchanged and added a second dynamic representation-at-scale sentinel rather than folding the two responsibilities together.

The evidence split is now:

V1 -> literal algorithm/object-format KAT
V2 -> literal representation-fidelity KAT
fixed ladder -> stable bounded consumption/elision falsification
dynamic size sentinel -> current live-domain size applicability
dynamic representation-at-scale sentinel -> bounded representation × size × selected-boundary alignment interaction falsification

Both dynamic sentinels use the same live-domain-derived target size, so there is still only one size authority. At the current live maximum of 142,699 bytes, both are 262,144-byte probes.

The new probe is deterministic, exact-length, and valid UTF-8. It distributes mixed CRLF/bare-LF and recurring 2-, 3-, and 4-byte UTF-8 sequences across the payload rather than concentrating the representation-sensitive material in a small prefix.

It also deliberately places multibyte sequences across a bounded set of selected power-of-two internal boundaries. Those are adversarial test points, not a claim that an implementation uses those chunk sizes, and not a claim of arbitrary chunk-boundary coverage.

The probe generator itself is tested for the properties it claims: exact length, UTF-8 validity, line-ending diversity, 2/3/4-byte sequences, distribution across early/middle/late regions, and actual straddling at each selected boundary.

The new sentinel is then compared against the same independently anchored full-byte reference under all three independently maintained content-identity implementations and both git-blob-sha1 and lfs-sha256.

I also reran the two failure families you demonstrated.

For the gated CRLF transform:

replace(CRLF, LF) if len(raw) > 88

I now get the diagnostic shape I wanted:

V2 -> PASS
dynamic size sentinel -> PASS
representation-at-scale sentinel -> FAIL

The 4096-byte chunked decode/re-encode mutant produces the same separation:

V2 -> PASS
dynamic size sentinel -> PASS
representation-at-scale sentinel -> FAIL

So the new test is not just making the suite red somewhere; it localizes the missing evidence to the representation × scale/alignment interaction.

I deliberately stopped there.

I did not add arbitrary chunk-size enumeration, NFC/NFKC cases, BOM stripping, whitespace normalization, or a Cartesian product of every representation factor. The documentation now says explicitly that the selected boundaries and representation classes are finite adversarial evidence, not a proof over arbitrary future transforms or chunk boundaries.

So the contract remains:

identity is over the complete, exact raw byte sequence.

The evidence remains finite and factorized:

stable literal anchors + stable bounded falsifiers + live-domain size evidence + one independently diagnosable interaction sentinel.

Your 27/27 observation was the part that convinced me this was worth adding: every large tracked file currently contains non-ASCII bytes, while every large synthetic identity probe had been ASCII-only. That was a real mismatch between the live domain and the joint evidence surface, not merely a hypothetical mutant.

Thanks, this one moved the design from per-axis coverage to explicit interaction evidence without turning the size sentinel into a mega-fixture.

Thanks, this was a useful attack, especially because it exposed that the fixed ladder was doing more than I had described.

You are right about what killed M7. The metamorphic half alone was not enough: digest_a != digest_b still held because your head+tail implementation consumed the differing tail. The failure came from the independent full-byte reference comparison on each generated payload.

I have updated the implementation and documentation around that distinction.

I kept V1, V2, and the fixed ladder unchanged:

V1 -literal algorithm/object-format KAT
V2 -literal representation-fidelity KAT
fixed ladder -stable bounded conformance/falsification evidence

The ladder is now described more accurately as combining two checks:

• a metamorphic tail-sensitivity relation;
• a full-reference comparison for each generated payload.

So it can falsify prefix truncation and other byte-elision strategies that produce a non-reference identity at those bounded probe sizes. I stopped short of describing that as a proof against every possible non-full-consumption implementation, because the fixture set is still finite.

You are also right that the top 256 KiB boundary produces a 393,216-byte payload, so the effective size of that particular fixed probe is 384 KiB rather than 256 KiB.

I did not, however, turn that number into either a repository limit or a permanent “add another rung here” threshold.

Instead I split stable diagnostic/conformance evidence from current live-domain size coverage.

I added a deterministic dynamic live-domain sentinel. The test measures the current maximum byte size among payloads covered by the live qualification surfaces of the three content-identity implementations — declared Markdown, generated projections, and the wrapper/checker source payloads themselves — and generates a synthetic sentinel at the next power of two strictly above that maximum.

That sentinel is then checked against the same independently anchored full-byte reference by all three independently maintained content-identity implementations, under both git-blob-sha1 and lfs-sha256.

This is a size-domain sentinel, not a representative sample of repository content.

So the structure is now:

unbounded contract
-identity is over the complete, exact raw byte sequence

stable evidence
-V1 / V2 / fixed 1-8-64-256 KiB ladder

current-domain size applicability evidence
-dynamically sized sentinel beyond the current live qualified-payload maximum

This means I did not add:

• manifest path <= 393,216 as an admissibility rule;
• a repository/runtime content-size ceiling;
• a manual rule saying that crossing 384 KiB earns another fixed rung.

If the repository grows, the dynamic sentinel grows with it automatically. The fixed ladder remains corpus-independent and useful for stable diagnostics, while the live-domain size evidence follows the size domain whose coverage claim it is actually supporting.

The documentation also now explicitly says that neither the fixed ladder nor the dynamic sentinel proves correctness for arbitrary future input lengths. The contract is unbounded; the evidence is finite and scoped.

So I ended up taking a third route relative to your final question:

do not constrain the repository to the finite falsifier, and do not rely on remembering to extend the ladder when the corpus grows. Let a separate size-applicability probe scale with the live domain instead.

Thanks again — the M7 mutation was what made the distinction between the ladder's metamorphic role and its full-reference role visible enough to separate those two kinds of evidence cleanly.

Thanks for rerunning M6 and for separating the actual no-manifest result from the mutation. Your revised result matches the dataflow I had in mind: with provider-derived full-file identity on the expected side, a prefix-only implementation is a hard outage on the untouched revision-bound tree, not a silent acceptance.

I have implemented the follow-up.

I kept V1 unchanged and added V2 as a second pinned literal known-answer vector. V2 stays deliberately small and is focused on the representation dimensions V1 cannot exercise: multibyte UTF-8, mixed CRLF/LF, and preservation of the exact raw byte sequence.

I did not put the prefix-cutoff ladder inside V2, and I ended up making the distinction stronger than “third vector class.”

The split is now:

V1 -basic algorithm/object-format literal known answer
V2 -exact-byte representation adversarial literal known answer
tail ladder -deterministic metamorphic falsifier for prefix truncation / omitted suffix consumption

The ladder probes:

1 KiB -8 KiB -64 KiB -256 KiB

Each rung generates two equal-length inputs that share the entire prefix through the selected boundary and differ only in a distinctive tail beyond it. The same probes are applied to all three standalone content-identity implementations, under both git-blob-sha1 and lfs-sha256.

I deliberately did not pin a large table of ladder digests. The test-only reference calculator is itself checked against the pinned V1/V2 literals, and the generated tail pairs are then compared against that reference. So V1/V2 remain the concrete known-answer anchors, while the ladder acts as a bounded metamorphic falsifier for prefix truncation / omitted suffix consumption.

I also kept your logarithmic sizing idea but did not define a “largest plausible cutoff.” Every finite prefix-only cutoff is wrong under the contract. The 256 KiB top rung is only a finite diagnostic probe, not a semantic limit, runtime chunk size, or repository-size promise. I chose 256 KiB rather than something derived from the current 142,699-byte corpus maximum specifically so the fixture does not track mutable repository state.

The invariant remains:

identity is computed over the complete, exact raw byte sequence.

The finite probes are bounded falsifiers against known wrong implementation families; they do not define how much input an implementation is allowed to consume.

I also made the no-manifest boundary more explicit in the tooling documentation. Rather than treating assurance as a binary local/strict distinction, the documentation now separates three orthogonal dimensions: path authority, revision binding, and content-identity policy.

Ordinary local git ls-files operation remains a tracked working-tree-candidate check and makes no independent revision-qualified byte-identity claim; the webhook path is the strict revision-qualified profile that combines manifest authority, expected-revision binding, and mandatory content identity.

So I think the two findings now land in separate places:

• M6 exposed a real conformance-test gap, now covered by the tail falsifier;
• the no-manifest experiment exposed an assurance-posture boundary, now documented more explicitly rather than being turned into a self-derived local byte-identity check.

Thanks for pushing both far enough to distinguish those two.

Yes. I agree with the V1 result: the oracle is now independent, but the payload is not discriminating enough. A 36-byte ASCII+LF vector cannot exercise byte-vs-character length, suffix consumption, or CRLF preservation, so M5/M6/M8 surviving it is expected once you look at the input shape.

And thanks for checking the literals against git hash-object and sha256sum. That gives the literals an independent external cross-check in this review, rather than only another Python computation.

I would add V2, but I would not size it against the current corpus maximum.

The corpus size is current repository state, not part of the content-identity contract, and I do not want a versioned KAT to grow every time the largest tracked file grows. I also would not make any current read-chunk size part of the contract.

The contract I want is simpler:

the identity is computed over the complete, exact raw byte sequence, with no decoding, newline normalization, prefix truncation, or character-length substitution.

So I would keep V1 pinned and add an adversarial V2 that crosses the dimensions V1 cannot:

• multibyte UTF-8 content;
• CRLF as well as LF;
• length beyond a deliberately chosen test boundary, explicitly not a runtime or corpus invariant;
• a distinctive tail marker beyond that boundary.

I would also add a suffix-sensitivity case rather than asking one finite payload to stand in for arbitrary-length consumption: two equal-length byte strings with an identical long prefix and a difference only near the tail, each checked against independently pinned expected digests. That gives us a direct falsifier for prefix-only hashing.

So the test split would be:

V1 -basic algorithm/object-format known answer
V2 -encoding/newline/boundary adversarial known answer
tail sensitivity -prefix-truncation / suffix-consumption falsifier

One point where I read the production consequence differently is M6.

I agree that M6 survives V1, and the conformance suite should kill it. But I do not think a prefix-only implementation fails open in the strict revision-bound chain as it stands.

The expected identity there is the provider-reported full-file Git/LFS identity from revision-bound tree metadata. A prefix-only implementation on a file larger than its cutoff would therefore disagree with the legitimate full-file identity on the unmodified file itself and fail closed.

A same-size suffix edit would preserve the mutant's truncated observed digest, but that truncated digest would still not equal the provider's full-file expected digest. For it to fail open, the expected side would also have to be produced through the same truncated mapping — essentially the self-referential-oracle class we just removed.

So I would classify M6 as:

• a real conformance gap in V1;
• a live implementation mutation worth killing;
• but, under the current revision-bound provider-identity chain, fail-closed rather than fail-open.

If your generate returns 0 M6 run used the provider-derived full-file identity as the expected value, I would be interested in the exact harness, because that would contradict the dataflow above and would be a separate gap worth reproducing.

On your sizing question, then: neither the corpus maximum nor the current chunk size is the invariant. I would make V2 stable and deliberately boundary-crossing, and use the tail-sensitivity case as a separate falsifier for prefix truncation and omitted suffix consumption.