Manga Story State
Results JSON
Measured research decision · v1.1

A story state that remembers without pretending it read the page.

The first leakage-controlled measurements sharpen the bet: manga-specialist perception is viable and privileged structure carries signal, but the current deterministic Story State does not preserve that signal well enough to deploy.

Decision: advance the reader; do not promote the current Story State.

Invest next in native-resolution perception, task-local candidate scoring, and exact evidence retrieval. Probabilistic beliefs, predictive latents, and branch-aware graph structures remain testable hypotheses—not proven improvements and never substitutes for reading the comic.

433 leakage-checked NPP rows Dataset: sochastic/Re-Verse Date: 13 July 2026 Status: measured exploratory result · graph not earned
01 · Diagnosis

The bottleneck is reading before reasoning.

Re:Verse is difficult because page understanding is fragile at every handoff: panels, order, text, speaker, identity, event, then time. A clean graph built from dirty observations becomes confident false memory.

308pages auditedOne title and one story universe: useful for iteration, insufficient for a general claim.
3,387released boxes1,562 text regions and 1,825 named-character regions, but no gold panel order or speaker links.
65.7%speaker unknown941 of 1,432 turns in the current heuristic teacher state. This is not an oracle.
74%generic events228 of 308 current page events collapse to “interaction,” erasing what downstream tasks need.
What remains plausible

A better state may become a useful temporal prior.

Identity continuity, repeated-scene disambiguation, belief tracking, causal closure, and loop branches are where persistent state could add information that a local page reader lacks. The current serialization does not yet demonstrate that value.

  • Preserve exact evidence and provenance.
  • Keep top-k speaker, identity, order, and branch hypotheses.
  • Revise recent decisions when later pages resolve ambiguity.
  • Use query-conditioned retrieval to reopen the raw crop.
What fails review

A deterministic graph cannot be the comic reader.

It cannot recover missed lettering, invisible actions, or wrong panel order. Worse, a wrong speaker or event gets promoted into a persistent “fact” and contaminates later answers.

  • Do not assign one irreversible speaker or identity.
  • Do not infer a reset from a keyword such as “revival.”
  • Do not compress every page into one event sentence.
  • Do not report teacher prose as vision-only evidence.
12-view synthesis
01
PerceptionManga-specialized OCR, layout, identity, and speaker extraction likely provide the first large jump.
02
RepresentationEvidence first, uncertainty second, future prediction third, graph last.
03
Experimental scienceCompare graph against a flat ledger with the exact same facts and token budget.
04
NPP / IPPCandidate-conditioned cross-encoding and one-to-many future modeling are essential.
05
VQARetrieve evidence chains, reopen high-resolution crops, then verify each claim.
06
Graph learningUse a posterior over graphs with 8–16 hypotheses and fixed-lag correction.
07
Loop timeSeparate discourse order, story time, event instances, motifs, and branch deltas.
08
Data scalingRe:Verse is too small for training a world model; pretrain perception and sequence skills elsewhere.
09
SystemsFreeze a capable VLM, cache page features, and train compact adapters and state heads.
10
SkepticHigh-resolution reading plus a flat evidence tape may beat the graph; force this comparison.
11
Narrative cognitionTrack the current situation and open obligations, not an exhaustive world encyclopedia.
12
Minimal baselinesMatch image tokens, state tokens, FLOPs, candidate calls, and latency.
02 · Measured evidence

The first controlled run changes the decision.

These are exploratory ten-way next-page ranking and perception diagnostics, not a title-level result. They tell us what to build next—and which attractive idea has not yet earned promotion.

433NPP task rows283 train, 84 validation, and 66 test rows after prefix, target, split, and candidate-order repairs.
38unique test targetsOnly two chapters from one title; the 10-page condition has 28 eligible targets.
10%random top-1Each item has ten candidates; expected random mean reciprocal rank is .293.
28.9%frozen SigLIP · 5 pagesA task-local, image-only similarity reference; it falls to 21.4% with 10 pages.
Context representation / scorerEvidence status5 pages · top-1 / MRR
n = 38
10 pages · top-1 / MRR
n = 28
Frozen SigLIP similarityVision-only
Detached page pixels; stateless task-local score.
28.9% / .46321.4% / .387
Oracle text similarityPrivileged
Teacher-aligned prose; representation upper bound.
36.8% / .53039.3% / .548
SigLIP + local oracle text · similarityPrivileged control
Balanced modalities; exact local prefix sequence.
39.5% / .56339.3% / .559
SigLIP + matched Story State · similarityPrivileged diagnostic
Same vision and candidates; only prefix text becomes state.
21.1% / .43421.4% / .424
Pixels + gold identity/layout · linearPrivileged
Gold structure with a train-only probe.
36.8% / .53742.9% / .555
Matched deterministic Story State · similarityDiagnostic
Exactly the task prefix; teacher-derived serialization.
18.4% / .39814.3% / .374
Matched deterministic Story State · linearDiagnostic
Same state, fit on training targets only.
21.1% / .3897.1% / .276
Intervals are conditional and descriptive.

All uncertainty bands use a within-chapter circular moving-block bootstrap. With 38 test targets in two chapters of one inspected title, they describe this fixed cohort; they are not independent-item confidence intervals and provide no title-level or population-level confidence.

Perception · viable front end

MagiV2 sees characters well; text labels expose a granularity mismatch.

Across all 308 pages, standard IoU@.5 character detection reaches .7727 precision, .9479 recall, and .8514 F1. Standard text detection is only .0807 precision, .0967 recall, and .0879 F1. The matched prediction-to-gold area ratio has a .263 median, consistent with tight lettering predictions being compared with larger Re:Verse bubble regions.

A separate intersection-over-smaller containment diagnostic gives text .8317 precision, .9968 recall, and .9068 F1. That .9068 is IoS, not standard IoU, AP, or a substitute detection score; it only shows that tight predicted text is usually contained by a released bubble box.

The four-page OCR pilot compared 25 outputs with 11 adapted teacher turns: mean best token-F1 per teacher turn was .8328, with all 11 turns reaching at least .5; mean best token-F1 per OCR string was .4727. This is a lexical proxy—not OCR ground truth, CER, WER, precision, or recall.

State utility · no-go

The current deterministic state loses useful signal.

Privileged local text and gold identity/layout improve candidate ranking, so narrative and structural information can matter. But the matched Story State serialization trails the frozen visual reference, and its linear head collapses to 7.1% with 10 pages.

In the held-constant fusion test, local text beats matched state by 18.4 top-1 points at five pages (descriptive 95% interval +5.3 to +31.6) and 17.9 points at ten (+3.6 to +28.6). Candidate representations and the SigLIP prefix sequence are identical; only the prefix text representation changes.

The current deterministic state has not earned promotion. Branch-aware graphs, uncertain relations, open-thread stores, and predictive latent slots remain candidate data structures—a hypothesis to test after exact retrieval and a strong task reader, not an established architecture win.

Protocol lesson

Seemingly small leakage channels can dominate NPP.

If page identifiers escape the boundary, chronology solves all 433 rows. Reused candidates also permit transductive strategies. Scientific runs therefore detach pixels, strip file and page metadata, expose only neutral A–J aliases, and require a stateless task-local scorer.

The repaired manifest excludes prefix/target gaps and keeps identical candidates for a shared target across 5- and 10-page comparisons. The public manifest is an audit artifact, not by itself a sealed benchmark.

Qwen2.5-VL · protocol repair

The first zero-shot run was invalidated by answer-position bias.

The superseded three-seed validation run scored 16.7% top-1 / .367 MRR, but accuracy changed drastically with the gold letter: D reached 46.9%, H 32.4%, A 4.2%, and J 0%. Those numbers are not model evidence.

Version 2 cycles every item through A–J exactly once and shuffles negatives independently. Validation is 13.4% top-1 / .329 MRR with five pages and 10.0% / .316 with ten. The frozen test is 17.1% / .371 and 17.1% / .360, with 99.7%/100% parse rates.

That is above chance on this fixed test cohort, but 11.8 and 4.3 points below frozen SigLIP; paired descriptive intervals include zero. Preferred canonical images still change across a mean 4.2 of ten permutations. Balancing neutralizes aggregate letter preference—it does not make Qwen order-invariant or establish a gain over simple image similarity.

Go now

  • calibrate MagiV2 text-region geometry and evaluate real OCR, speaker, and identity labels;
  • build a hard-negative, candidate-conditioned visual/text reader with exact recent evidence;
  • expand to untouched titles and enough independent targets for a meaningful paired effect.

Hold now

  • do not deploy or scale the current deterministic Story State;
  • do not quote the IoS text diagnostic as standard detection performance;
  • do not claim a Qwen-over-SigLIP, context-length, or title-level gain from this cohort.
03 · Candidate architecture

Evidence → belief → prediction → verification.

This is the next representation to test, not a measured winner. Splitting observation, belief, and prediction is intended to stop a plausible interpretation from silently replacing the page.

L0 · LOSSLESS

Evidence store

Native page, panels, bubbles, faces, bodies, OCR alternatives, coordinates, and embeddings. Immutable, provenance-linked, query-reopenable.

L1 · POSTERIOR

Page Belief Graph

Top-k reading orders, speakers, identities, explicit actions, and local relations. Every edge has confidence and supporting regions.

L2 · RECURRENT

Predictive latent state

8–16 learned slots preserve residual visual/narrative information that the schema cannot express and predict future page latents.

L3 · STRUCTURED

Story Belief State

Entities, event instances, open goals, causal links, beliefs, branches, repeated-scene deltas, and sparse episode anchors.

L4 · CONDITIONAL

Task verifier

VQA evidence chains or candidate-conditioned NPP/IPP scoring. It can reopen L0 evidence and marginalize over uncertain worlds.

Key asymmetry: L0 observations never change. L1–L3 are hypotheses and may be revised. L4 is task-conditioned, but the write path remains causal and query-independent.

Minimal sufficient state
C
Current situationLocation, cast, active action, physical state, and immediate conflict.
D
Discourse stateReading order, focal character, speaker, flashback, montage, or branch transition.
O
Open obligationsGoals, promises, threats, mysteries, setups, and expected closures.
H
Joint hypothesesCoherent top-k assignments over speaker, identity, time, and branch.
E
Episode anchorsSparse, salient, evidence-backed events—not one generated sentence per page.
Z
Latent residualCompact predictive features for details the explicit schema misses.
Causal update

Maintain a posterior, not one brittle graph.

At page t, the model updates several coherent story hypotheses from only the visible prefix. A short fixed-lag window permits correction when a bubble tail, name, or repeated checkpoint becomes clear later.

q(Sₜ) ∝ q(Sₜ₋₁) · p(Oₜ | Sₜ) · ψlayout · ψspeaker · ψidentity · ψtemporal · ψbranch
score(c) = log Σₕ q(h | O≤t) · exp[E(c, h, evidence)]

For loops: represent an event instance separately from its motif. A reset points to a checkpoint and creates a branch; it does not erase the prior branch. Character knowledge is branch- and holder-specific: Subaru may retain a memory while the world does not.

!
The current teacher marks only 1 of 5 reset-marker pages and 2 of 13 death-marker pages.

Keyword matching can also turn retrospective “revival” language into a false reset. Reset detection needs evidence bundles across panels and pages, explicit branch uncertainty, and adjudicated labels before it can support a graph.

Let state influence perception when…

  • two visually similar identities need temporal disambiguation;
  • a speaker is off-panel or a tail is ambiguous;
  • a repeated scene differs by branch knowledge;
  • local panel order has two plausible readings.

Never let state invent evidence when…

  • text or action was never detected;
  • OCR strongly contradicts the narrative expectation;
  • the state merely “knows the lore” from contamination;
  • confidence comes only from the model’s own scalar.
04 · Task readers

One state, different inference programs.

VQA needs auditable retrieval. Next-page prediction needs future uncertainty and exact adjacency. Intermediate-page prediction needs a bridge constrained from both directions.

VQA

Retrieve → reopen → verify

  1. Soft-parse entity, time, relation, belief holder, and branch.
  2. Retrieve raw regions plus graph neighbors and alternative hypotheses.
  3. Build short evidence chains, not a global serialization.
  4. Reopen high-resolution crops and verify every answer claim.
  5. Marginalize answers or abstain when evidence is insufficient.
NPP

Predict → append → contradict

  1. Predict a mixture of plausible next-event and visual latents.
  2. Cross-encode each candidate with recent exact pages and the state.
  3. Hypothetically append the candidate and compute its smallest consistent update.
  4. Penalize contradiction, skipped causality, repetition, and wrong offset.
  5. Fuse state only when its confidence beats the raw visual scorer.
IPP

Forward + reverse bridge

  1. Build a causal forward state from the visible prefix.
  2. Build a reverse constraint representation from the visible suffix.
  3. Score whether a candidate coherently exits the prefix.
  4. Score whether it also enters the suffix without impossible changes.
  5. Require local continuity and global narrative compatibility.
Capability / failureStory State valueWhyRequired safeguard
Cross-page identityHighTemporal anchors disambiguate look-alike characters and costumes.Multiple visual anchors; unknown/new-entity option.
Speaker attributionHighTurn-taking, goals, and prior visibility provide a useful prior.Tail geometry and crop evidence must dominate contradictions.
Loop / branch reasoningHighExplicit branches prevent repeated events from collapsing together.Separate story time from discourse time and character belief.
Ambiguous panel orderMediumNarrative coherence can rerank several geometric orders.Preserve top-k orders; avoid forced early commitment.
Near-correct OCRMediumContext can select among OCR alternatives.Keep character-level OCR posterior and original crop.
Missed lettering or actionLowNo state can recover information that was never observed reliably.Native resolution, dynamic crops, detection recall.
Lore contaminationHarmfulA famous title lets a model answer without reading the prefix.Anonymization, counterfactual facts, shuffled-state controls.
05 · Falsification

Earn the graph with controlled evidence.

The executed NPP and perception probes narrow the search, but do not establish a general gain. The next experiments must isolate resolution, retrieval, prediction, structure, and leakage on untouched titles.

Matched baseline ladder
B0Paper reproductionPublished low-resolution page preprocessing.
B1Native pageSame model at usable resolution and aspect ratio.
B2Overview + cropsHigh-resolution panels, bubbles, and ordered transcript.
B3Full-prefix evidenceExact page/crop memory with no semantic state.
B4Candidate cross-encoderHard-negative NPP/IPP scorer.
B5Learned retrievalQuestion/candidate-conditioned evidence selection.
B6Flat fact ledgerSame fact multiset and budget as the graph.
B7Hybrid Story StateBelief graph + open threads + branches + latent residual.
B8Causal oracleGold prefix-only evidence; upper-bound representation test.
Executed exploratory pilot

We now know where the next experiment must be stricter.

The current run covers all 308 pages for MagiV2 detection, a four-page OCR proxy, and 433 repaired NPP rows. The test slice contains 38 unique targets from two chapters; 28 targets support a paired 5-versus-10-page comparison.

  • Measured: task-local frozen vision, train-only probes, privileged text/structure upper bounds, and exactly prefix-matched Story State diagnostics.
  • Still missing: real OCR ground truth, speaker and identity scoring, VQA/IPP evaluation, a trained updater, and an untouched cross-title test.
  • Next causal oracle: annotators see only pages ≤ t; identities remain unknown until revealed, and every answer has a minimal region-level proof.
  • Next fairness contract: match visible pages, image/state tokens, training targets, candidate calls, FLOPs, and latency.
  • Next statistics: predeclare a target/chapter/title hierarchy and stop treating overlapping prefixes as independent observations.
!
Protocol constraint

If evaluation gives exactly 5 or 10 context pages, state may ingest only those pages. A full-arc state would introduce unfair information.

Reconsider structured state only if…

  • a causal oracle establishes a material memory gap on untouched titles;
  • a bounded state recovers roughly 70–80% of that oracle gain;
  • a graph beats an equal-budget flat ledger on at least two task families;
  • edge shuffles, branch ablations, and state swaps causally remove the gain.

Current stop decision

  • do not scale the current deterministic serialization;
  • keep exact recent evidence and retrieval ahead of graph complexity;
  • treat privileged text and identity/layout scores as upper bounds, not system results;
  • retain balanced Qwen v2 as an exploratory baseline and freeze a new cross-title protocol before interpreting gains.
!
The repository now contains executed perception and NPP probes—not a completed manga reasoner.

MagiV2 was run across the full 308-page corpus, and the repaired NPP evaluator measures frozen vision, privileged diagnostics, and matched state representations. There is still no trained belief updater, final VQA/IPP evaluation, real OCR benchmark, or cross-title confirmation. “Oracle” denotes privileged teacher or gold input, not deployable vision.

06 · Execution order

Put research effort where the information is lost.

The ordering now follows the measurements: finish calibrating the reader, establish a cross-title memory gap, then ask whether explicit structure adds value beyond exact retrieval and latent prediction.

Phase 0

Seal the evaluation contract

The repaired 433-row manifest and detached task boundary are implemented. Next, keep candidate reuse private or regenerate it per sealed run, freeze balanced answer positions, and pre-register the untouched cross-title split.

audit passed
Phase 1

Finish the strongest non-state reader

MagiV2 character detection is viable. Calibrate tight text boxes against bubble annotations, collect real OCR labels, and measure reading order, identity, and speaker posteriors before adding memory.

active
Phase 2

Establish the causal memory gap

Expand to untouched titles and adjudicated prefix-only items. Prioritize exact OCR, reading order, speaker links, identity, resets, and minimal proof regions—not more coarse masks.

oracle gap
Phase 3

Add exact retrieval and latent prediction

Cache page features; train a question/candidate retriever, hard-negative cross-encoder, and 8–16-slot predictive recurrent state with a frozen VLM plus LoRA/connectors.

B5–B6
Phase 4

Earn—or remove—the probabilistic graph

Only after a memory gap exists, test Page/Story belief factors, top-k hypotheses, fixed-lag revision, loop branches, sparse beliefs, and open obligations directly against an equal-budget flat ledger.

B7 · gated
Phase 5

Scale and generalize

Pretrain manga perception and sequence objectives on Manga109, Manga109Dialog, MangaSeg, COMICS, and other rights-compatible data. Validate on an untouched cross-title holdout.

publication
Highest expected return

Calibrated manga perception

The measured 85.1% character IoU F1 and 90.7% text-containment diagnostic justify continuing MagiV2, while the 8.8% standard text IoU F1 and proxy-only OCR make the remaining calibration work explicit.

Second

Hard-negative task readers

Frozen SigLIP already reaches 28.9% top-1 at five pages. A candidate-conditioned verifier should now test whether exact adjacency can be separated from visual similarity and generic narrative plausibility.

Third · gated

Hybrid memory

The current matched state scores below frozen vision. Exact retrieval and predictive latent experiments come before graph complexity; explicit structure returns only after a causal oracle gap and a matched-ledger win.

Bottom line

The measurements say: improve the reader first; make memory earn its place.

Manga-specialist perception is promising, and privileged text and structure prove there is useful signal to recover. The current deterministic Story State does not recover it. Build a verifier forced to return to exact page evidence, test a compact uncertain state on untouched titles, and remove the graph if it cannot beat an equal-budget flat ledger.

Executive brief copied