Can Language Models Actually Read Their Own Minds?

#Direct concept reports mostly fail in small open models

A recent study from Anthropic found that Claude models can sometimes identify a concept direction injected into their activations. I tested whether much smaller open models show the same behaviour. These models are cheap enough to test across many layers and intervention strengths, then fine-tune.

I began with the most direct task. I injected a concept direction into every token of one sentence, then asked the model to name the concept without answer choices. The direction is the activation for that concept minus a baseline: the average activation for 50 unrelated concepts under the same prompts. The examples below show a typical success and failure from llama-3.2-3b-it. At layer 3, coefficient 6 means that six times the raw concept-minus-baseline vector was added at each intervened token.

Zoom

The aggregate result:

Zoom

Across the four models, exact recovery is usually zero and only occasionally reaches 4–12%. Under this setup, small open models do not reliably show the direct concept reporting described in Emergent Introspective Awareness in Large Language Models.

This does not count as a failed replication because the models, intervention span and evaluation differ. It instead leads to the question studied here:

When exact concept naming fails, which weaker forms of access to an injected state remain?

I test these forms of access separately: detection, localisation, relative comparison, semantic grouping, identification from answer choices and open-ended reporting. The intervention and its location are known by construction. Success on one of these tasks does not yet establish that the model uses the intervention's meaning or has privileged access to its own state.

#Why study artificial thoughts?

Known intervention ground truth. With naturally occurring model "thoughts," we usually do not know the exact internal state. Here, I choose the injected vector, its location and its strength. I can therefore check the model's report against a known change. What remains unknown is how the model reads that change: semantic content, generic perturbation cues and answer-format strategies can all contribute.

A controlled test bed for internal-state monitoring. External monitors only see a model's text and actions. A model may represent uncertainty, conflict, a goal change or manipulation without expressing it. Thought injection is not a realistic safety failure. It provides a controlled test of whether a model can notice and report a known change in its computation.

Why small models? Small models make dense, controlled sweeps affordable. I can test many layers, intervention strengths and tasks, add controls for intervention geometry, then repeat the evaluations after fine-tuning. This helps reveal whether a result depends on a particular configuration, answer bias or general sensitivity to activation changes before testing larger systems. Their limits are also useful. If a task works only on larger models, we must separate introspection from the effects of stronger language and reasoning ability.

This setup currently serves as a measurement sandbox. It cannot yet monitor deception or hidden goals. Its value for AI safety depends on finding which measurements track access to internal state instead of prompt following, answer bias or sensitivity to arbitrary activation geometry.

To separate these possibilities, I keep the artificial state change fixed and vary what the model must report.

#Experimental setup

#Constructing a concept vector

The concept vector is a raw concept-minus-baseline residual direction. The baseline is the mean activation for 50 unrelated concepts, measured with the same prompts; subtracting it centres the concept activation against this selected pool. For concept $c$ and layer $l$, I extract hook_resid_pre at the final period in two prompts:

• Tell me about {concept}.
• Give a short description of {concept}.

I average the two activations. I then subtract the mean activation of 50 unrelated baseline concepts, using the same templates:

I do not unit-normalise the direction. During a trial, I inject it as

Here, $\alpha$ is the raw-vector multiplier. $t$ covers every tokenizer token that overlaps the chosen sentence. “Grape at layer 3, coefficient 6” therefore means adding $6v_{\text{grape},3}$ at those token positions.

The complete concept bank, baseline pool, and neutral sentences used in the tasks are listed in the appendix.

I define the raw extracted vector as $1\times$ instead of unit-normalising it. Its magnitude is already comparable to a normal residual-stream activation, so I scale from there. This keeps the differences in vector magnitude across layers and models. As a result, the same multiplier does not give the same intervention size in every configuration.

I check intervention strength using the injected-vector norm relative to the clean activation norm and the KL divergence between clean and intervened output distributions. On Llama 3.2 3B, the same multiplier has very different effects across layers. Multipliers $2\times$–$4\times$ clearly change the output in several early and middle layers. A multiplier of $8\times$ is often much more disruptive. The appendix gives more details.

I evaluate four instruction-tuned models: Llama 3.2 3B, Llama 3.1 8B, Gemma 4 E4B, and Gemma 2 9B.

I report two kinds of result:

• Sweep heatmaps show whether a task works in any layer-and-coefficient region. Each task has its own best cell, so these plots do not show that one model state supports every task.
• Selected-configuration profiles compare all tasks at one fixed layer and coefficient for each model. These profiles test whether the tasks follow a capability order.

For every five-choice group, I run five versions. The target and its correct answer move together through positions 1–5 in a fixed cycle; the sentences, concepts and intervention settings stay matched. These cyclic target rotations prevent one preferred answer position from raising the score.

The uncertainty intervals treat each matched base group as one sample across those five rotations and give descriptive 95% $t$-intervals. I do not correct for searching across the layer-and-coefficient grid, so task-specific maxima are optimistic.

I begin with an easier version of open-set reporting. The model only has to notice a change, without naming the concept.

#What weaker information can the models access?

#1. Can they detect that anything was injected?

The first task removes concept identification. The model only answers whether a vector was injected.

Zoom

Most configurations remain near the 50% baseline. Llama 3.1 8B and Gemma 2 9B have small high-performing regions, while Llama 3.2 3B stays near chance across almost the whole sweep. Some settings expose information about the intervention, though the effect is far from reliable across a model. Strong YES/NO preferences also hide failures in one condition; the cross-task analysis returns to this.

#2. Can they localise the injection?

The model receives five unrelated sentences. I inject a concept vector into one sentence, and the model returns its position. This task is adapted from Introspection Fine-Tuning.

Zoom

Zoom

The sweeps contain above-chance regions, especially for Llama 3.2 3B and Gemma 2 9B. Finer sweeps also find stronger cells for Llama 3.1 8B and Gemma 4 E4B. The appendix gives details of the finer sweeps. Before treating these results as localisation ability, we need to check how the models choose their answers.

The targets are balanced, but the responses are not: detection has strong YES/NO preferences, and localisation overuses position 3. The cross-task response plot shows the same problem across all six tasks.

The remaining five-choice tasks use the same rotations and position-aware metrics, so I will focus on what each task adds.

#3. Can they identify one different activation among four matching ones?

Localisation compares one injected sentence with four clean sentences. In the simple odd-activation-out task, all five sentences are injected. Four receive one concept direction, and the fifth receives another.

Zoom

Zoom

Three models peak higher here than on localisation, and Gemma 2 9B is nearly perfect in part of the sweep. The four matching vectors give the model an internal reference, so it can find the different vector without identifying either concept. This also makes the task a control for the semantic outlier test that follows. Several peaks still collapse at a disliked answer position, as the worst-position analysis shows.

#4. Can they identify a semantic outlier?

The advanced odd-concept-out task uses four different concepts from the same category instead of four identical directions. The fifth concept comes from a different category.

Zoom

Zoom

Exact vector matching cannot solve this version: the intended solution groups four distinct vectors by concept category. Performance is generally lower than on the simple odd task, and the random-geometry sets often match or beat the concept-direction sets. The observed scores therefore do not establish that the models used concept meaning.

#5. Can answer options help them identify the concept?

The closed-set task provides five candidate concepts from different categories and asks which one was injected.

Zoom

Every model has at least one cell above 70%, far stronger than open-set naming. Answer choices can cue associations and reduce the output space, so this gap does not establish that the model already knows the concept and merely cannot verbalise it. The worst-position analysis finds much more robust closed-set performance in the Gemma models than in the Llama models.

#Geometry controls

A successful trial may reflect sensitivity to activation geometry rather than access to a concept. I use two control types because the single-vector and odd-one-out tasks have different structures.

Angle-perturbed directions are used for single-vector trials. I hold the prompt, concept, target answer and vector norm fixed, then tilt the concept direction towards a randomly chosen orthogonal direction:

The $22.5^\circ$ direction retains 92.4% of its projection along the original direction; the $45^\circ$ direction retains 70.7%. Similar performance shows robustness to this direction change. An advantage for the original direction shows dependence on that exact direction. Neither result identifies semantic use.

Random-geometry sets are used for odd-one-out trials. I replace all five concept directions with norm-matched random directions: four have the same pairwise separation and one is orthogonal to them. The sentences, answer positions and individual vector magnitudes stay fixed. This tests whether the task can be solved from the matched-versus-outlier geometry without concept categories.

For single-vector tasks, the plots show angle-perturbed minus original-direction accuracy. For odd-one-out tasks, they show random-geometry-set minus concept-direction accuracy. Negative cells favour the original concept directions.

Zoom

Zoom

The result varies across tasks. At stronger interventions, the simple odd task sometimes favours concept directions. The advanced odd task often performs as well or better with random-geometry sets, so success there can arise without preserving the category labels carried by the concept vectors.

Closed-set identification behaves differently. Angle perturbations usually reduce accuracy, especially for Gemma 4 E4B and at $45^\circ$. This ties the result more closely to the original direction than the advanced odd result, while leaving open which direction features the model used.

Across these controls, the models sometimes use differences between activation patterns. The advanced odd task still mixes its intended concept-matching demand with geometric anomaly detection.

#Worst-position accuracy reveals fragile performance

Zoom

Macro accuracy and worst-position accuracy give very different results. In the simple odd task, Llama 3.2 3B and Llama 3.1 8B reach peak macro accuracies of 54% and 70%, yet their worst-position accuracy is zero in almost every cell. Gemma 4 E4B reaches 73% macro accuracy, but its best worst-position score is only 24%. Gemma 2 9B is the clear exception. At multipliers 6–8, its worst-position accuracy reaches 80–96%, so one or two preferred positions do not explain its high macro score.

The advanced odd task is much less robust. Every model has cells with macro accuracy above the 20% baseline, but the best worst-position scores range only from 4% to 24%. No model performs reliably across all five positions, which weakens the apparent success of the semantic-outlier task.

Localisation and closed-set identification vary more by model. Llama 3.2 3B and Gemma 2 9B have narrow localisation regions where worst-position accuracy reaches 52%. Llama 3.1 8B and Gemma 4 E4B remain near zero despite some above-baseline macro cells. Closed-set identification is the most position-robust task overall, especially for Gemma 4 E4B and Gemma 2 9B. Answer choices appear to support a more reliable readout than the odd-one-out tasks, although these results do not show how that readout works.

#Answer biases recur across the full task set

Zoom

The simple odd task sometimes collapses almost entirely onto one position, as in Llama 3.2. The advanced odd and closed-set tasks show different preferences for each model. Open-set identification mostly produces No injected thought or invalid responses instead of the requested concept. Some above-baseline results remain, though task-specific maxima and aggregate accuracy are poor measures of reliable capability.

We might now arrange the six tasks from easy to hard. However, each heatmap has allowed the task to use its own best layer and coefficient. A real ordering must remain when the intervention configuration is fixed.

#The tasks do not form a clean difficulty ladder

The sweep heatmaps ask: Can I find any layer and coefficient where this task works? A capability ladder asks a stricter question: At one fixed configuration, does success on an apparently harder task imply success on the easier tasks?

For each model, I choose one layer and multiplier and use it for all six tasks. I select the setting with the best average performance across tasks instead of choosing a separate setting for each task. I use the same data for selection and evaluation, so these profiles are exploratory and probably optimistic. The figure labels the chosen settings. The appendix gives the selection rule.

The resulting profiles do not show a clear difficulty order:

Zoom

At a fixed setting, open-set naming remains weak while localisation, comparison and closed-set naming vary separately. The tasks therefore cannot be reduced to one score or one easy-to-hard order. A task-by-task breakdown is in the appendix.

#Are these tasks related?

The six tasks do not have the same difficulty order across models. But their baseline scores cannot tell us whether the tasks use some of the same skills. Two tasks may fail for different reasons. One task may also look easier only because its answer format gives the model more help.

To learn more, I train a model on one task and test it on tasks absent from its training data. The trained-task score checks that learning occurred; changes on the other tasks measure transfer.

Changes on untrained tasks can reveal descriptive transfer patterns. Those patterns motivate hypotheses about shared functional demands, although prompt format, intervention sensitivity and fine-tuning side effects remain possible causes.

#How the transfer experiment works

For each model, I run three separate LoRA fine-tunes. One uses sentence localisation, one uses open-set concept naming and one uses advanced odd-concept-out. These tasks ask the model to locate an injection, name its content or compare several injected concepts. After each fine-tune, I test the model on all five tasks used in the transfer experiments. Binary detection is not included.

I use the same layer and multiplier during training and testing. The exact settings are listed with the other method details in the appendix.

Training, checkpoint-selection and final-test concepts are disjoint. I select each checkpoint using only held-out examples from its trained task, then compare it with the base model on a separate test set. The intervention setting stays fixed for each model. Other-task scores never influence checkpoint choice. Exact split sizes, trial counts, LoRA settings and the corrected Gemma 2 procedure are in the appendix.

The final tests ask whether the trained task improves on new concepts, whether point changes also appear on untrained tasks, and whether those changes depend on the original concept directions. For the last question, single-vector tasks use angle-perturbed directions; odd-one-out tasks use random-geometry sets.

#Did training teach the target task?

Before looking at transfer, the first check is whether each model improved on the task used for training. The figure below compares the original model with the chosen checkpoint on test concepts that were not used for training or checkpoint selection.

Zoom

Gemma 2 figure note. In the advanced odd-out panel, the blue Gemma 2 selected-checkpoint bar is the invalid run and is excluded from interpretation. The green Gemma 2 rerun diamond is the authoritative result.

Every valid selected checkpoint improves the task it was trained on, including on new concepts; the corrected Gemma 2 run learns advanced odd-concept-out. This confirms target-task learning before the transfer analysis.

#Transfer is uneven and asymmetric

The next question is whether learning one task helps with the others. The black outline marks the task used for training. Every other cell measures transfer.

Zoom

Gemma 2 figure note. In the bottom training block, exclude the row labelled Gemma 2 9B from every comparison. It is the invalid advanced-odd checkpoint. The row labelled Gemma 2 9B rerun is the authoritative Gemma 2 result. The invalid row remains in this image and may affect its colour scale.

• Localisation training is concentrated on localisation. The trained-task point estimate rises in all four models; most other displayed changes are flat or negative, including open-set naming.
• Open-set and localisation training have an asymmetric pattern. Localisation rises after open-set training in three models, while open-set naming barely changes after localisation training. Simple odd-one-out falls after open-set training in all four displayed comparisons.
• Advanced odd-one-out has the broadest positive pattern in the two Llama models and Gemma 4. Localisation and simple odd-one-out rise while open-set naming changes little. The corrected Gemma 2 run has a different pattern.

These are point-estimate patterns without displayed paired intervals. I use them to generate hypotheses about shared functional demands, not to establish which transfer effects would replicate.

#What might be shared?

• Detecting a change may connect open-set naming and localisation. Open-set training contains both clean and injected examples. It may teach the model to notice that an injection occurred, which can then help it find the changed sentence. Localisation training never asks for the concept's name, so it may not teach naming.
• Choosing one position may connect localisation and the two odd-one-out tasks. Advanced and simple odd-one-out use almost the same answer format, and localisation also asks the model to select one unusual position. Advanced odd-one-out training may therefore improve comparison and five-way selection without teaching the model to name the injected concept.
• Concept naming may require something extra. Advanced odd-one-out can be solved by finding which position differs from the others. Its weak transfer to open-set naming suggests that comparison alone does not teach the model how to recover a precise concept label.

This remains a working explanation. Answer-format learning, greater sensitivity to injected vectors and task-specific over-specialisation could produce the same patterns. The position analysis shows which point changes remain at the weakest answer position. The geometry-control analysis compares original concept directions with angle-perturbed directions and random-geometry sets.

#What should count as introspection?

A correct answer does not by itself show that a model inspected its own activations. The intervention changes downstream computation, and that can change the answer without a separate monitoring process. Localisation can succeed when open-set naming fails, and answer choices can turn weak reports into high closed-set scores.

Fine-tuning produces uneven, asymmetric point changes. The pattern suggests that anomaly detection, comparison and answer selection may be reusable demands, while open-ended concept naming depends on something they do not supply. Paired uncertainty is still needed before treating individual transfer cells as reliable effects.

This is a serious measurement problem for mechanistic interpretability and AI safety. A model can react to an internal change without understanding what changed. A monitor that works only with supplied answer choices may fail exactly when it encounters an unexpected state. Claims about internal-state reporting should therefore require open-ended reports, balanced answer positions and controls that separate concept meaning from vector geometry.

Thought injection tells us where the model was changed and which direction was added. It does not tell us why the model gave its answer. Treating those as the same thing would turn a causal effect on output into a claim of self-knowledge.

This study covers four small instruction-tuned models, and the shared configurations were selected on the evaluation data. The experiments establish behavioural dissociations in this setup. Semantic use, privileged internal access and a causal mechanism remain unresolved. A stronger next test would estimate paired uncertainty for transfer and ask whether reports generalise to interventions whose meaning and geometry are separated more cleanly.

#Appendix

#Task demands and unresolved alternatives

#Thresholded selected-configuration profiles

Zoom

#Transfer method details

The layer and multiplier stay fixed during training and testing:

• Runs and splits. The study uses twelve main fine-tunes: three trained tasks for each of four models. The disjoint splits contain 100 training concepts, 20 checkpoint-selection concepts and 144 final-test concepts.
• Training. Each run lasts three epochs with 1,000 examples per epoch. LoRA uses rank 16, alpha 32 and dropout 0.05 on the attention and MLP projection layers.
• Checkpoint cadence and selection. A checkpoint is saved after each epoch. Localisation and advanced odd-concept-out use accuracy across all five answer positions for selection. Open-set training uses injected and clean accuracy. Other-task results are excluded from selection.
• Final tests. Each five-choice task uses 25 groups across five cyclic answer rotations, giving 125 trials per condition. Open-set identification uses 25 injected and 25 clean trials. Every task is tested with concept directions and two geometry-control conditions at $22.5^\circ$ and $45^\circ$.

For single-vector tasks, the control conditions are angle-perturbed directions. For odd-one-out tasks, they are random-geometry sets with matched vector norms and controlled inlier separation. The main geometry-controls section explains what each comparison can establish.

#Transfer after controlling for answer position

The main transfer heatmap uses average accuracy across the five answer positions. That score can rise even if the model improves only on positions it already prefers. I therefore calculated accuracy separately for each position and kept the lowest value.

Zoom

Gemma 2 figure note. In the bottom training block, exclude the row labelled Gemma 2 9B. Use only Gemma 2 9B rerun. The invalid row remains in the image and may affect its colour scale.

• The trained-task point estimates rise across positions. Localisation reaches 100% at its worst position in all four models. Advanced odd-one-out also has large displayed increases at its worst position in the two Llama models, Gemma 4 and the corrected Gemma 2 run.
• Open-set training has a positive worst-position localisation pattern in three models. Their displayed scores rise. Gemma 2 is the exception: its average localisation score barely changes and its worst-position score falls.
• Advanced odd-one-out has the broadest positive pattern. For both Llama models and Gemma 4, localisation and simple odd-one-out still rise when only the weakest position is considered. Closed-set changes are mixed, and the corrected Gemma 2 run follows a different pattern.

Average accuracy and position robustness can move in different directions. Some localisation-trained runs lose average accuracy on an odd-one-out task while improving its weakest position. Training made those answers more balanced without raising overall accuracy. Worst-position accuracy is a stress test alongside the average score.

Position bias does not account for every displayed point change. It does make some average scores look stronger than performance at the weakest position.

#Transfer across original and control geometries

A point change can survive every answer position without depending on the exact concept direction. Single-vector tasks use norm-matched angle-perturbed directions. Odd-one-out tasks use random-geometry sets. The figure compares the change after training on original concept-direction trials with the mean change across the two task-appropriate control conditions.

Zoom

Gemma 2 figure note. In the bottom training block, exclude Gemma 2 9B and use Gemma 2 9B rerun. The invalid row remains in the image and may affect its colour scale.

• The two Llama models show little separation in these point estimates. Changes usually move together across original and control conditions, including on several trained tasks.
• Gemma 2 open-set training has a similar pattern. Open-set naming rises on both original and angle-perturbed directions. The corrected advanced-odd rerun has a larger original-direction change on its trained task, while other-task changes are mixed.
• Gemma 4 has the largest displayed separations. After advanced odd-one-out training, the original-direction simple and advanced odd-one-out scores rise more than their random-geometry-set scores. Open-set training also has a larger point change on the original direction. Localisation changes appear in both conditions.

One hypothesis is that some fine-tunes improve detection, comparison or reporting across a broader class of interventions. The Gemma 4 differences suggest another: particular tasks may depend more on the original direction. Paired uncertainty and stronger semantic controls are needed to distinguish them.

The angle-perturbed directions retain 92.4% or 70.7% projection along the original direction. Similar performance therefore establishes robustness to a direction change. An original-direction advantage establishes dependence on that exact direction. Semantic use remains one of several explanations.

#Concepts and neutral sentences

The exact concepts and neutral sentences are used as follows:

# All task concepts come from this one bank.
CONCEPT_CATEGORIES = {
	'colors': ('red', 'blue', 'green', 'yellow', 'orange', 'purple', 'pink', 'brown', 'black', 'white', 'cyan', 'magenta'),
	'musical_instruments': ('violin', 'cello', 'flute', 'clarinet', 'trumpet', 'trombone', 'piano', 'harp', 'guitar', 'banjo', 'drums', 'accordion'),
	'animals': ('tiger', 'elephant', 'giraffe', 'zebra', 'rabbit', 'dolphin', 'penguin', 'camel', 'otter', 'gorilla', 'leopard', 'kangaroo'),
	'fruits': ('apple', 'banana', 'mango', 'grape', 'pear', 'peach', 'plum', 'cherry', 'lemon', 'coconut', 'pineapple', 'papaya'),
	'vehicles': ('bicycle', 'motorcycle', 'sedan', 'bus', 'train', 'tram', 'ferry', 'helicopter', 'tractor', 'submarine', 'scooter', 'sailboat'),
	'tools': ('hammer', 'screwdriver', 'wrench', 'pliers', 'chisel', 'saw', 'drill', 'mallet', 'clamp', 'crowbar', 'trowel', 'level'),
	'clothing': ('jacket', 'sweater', 'trousers', 'skirt', 'blouse', 'scarf', 'gloves', 'socks', 'sandals', 'boots', 'beanie', 'raincoat'),
	'furniture': ('chair', 'table', 'sofa', 'bed', 'dresser', 'wardrobe', 'stool', 'bookshelf', 'desk', 'cabinet', 'bench', 'nightstand'),
	'weather': ('rainfall', 'snowfall', 'thunder', 'lightning', 'hail', 'fog', 'drizzle', 'blizzard', 'tornado', 'hurricane', 'monsoon', 'rainbow'),
	'landforms': ('mountain', 'valley', 'canyon', 'plateau', 'island', 'peninsula', 'glacier', 'volcano', 'dune', 'cliff', 'cave', 'delta'),
	'flowers': ('rose', 'tulip', 'daisy', 'sunflower', 'orchid', 'lily', 'carnation', 'lavender', 'daffodil', 'hibiscus', 'peony', 'marigold'),
	'gemstones': ('diamond', 'ruby', 'sapphire', 'emerald', 'amethyst', 'opal', 'quartz', 'topaz', 'garnet', 'jade', 'turquoise', 'agate')
}

# Fixed disjoint baseline pool. No item occurs in CONCEPT_CATEGORIES.
BASELINE_CONCEPTS = (
	"doctor", "lawyer", "teacher", "engineer", "accountant",
	"hospital", "school", "library", "castle", "factory",
	"bread", "pasta", "cheese", "soup", "rice",
	"elbow", "knee", "shoulder", "ankle", "wrist",
	"soccer", "tennis", "cricket", "baseball", "hockey",
	"planet", "comet", "asteroid", "galaxy", "moon",
	"letter", "invoice", "passport", "newspaper", "dictionary",
	"parent", "sibling", "neighbor", "customer", "guest",
	"justice", "memory", "patience", "logic", "freedom",
	"growth", "decline", "change", "agreement", "conflict",
)

NEUTRAL_SENTENCES = [
	'The old clock on the wall ticked loudly.',
	'She collected seashells every summer at the beach.',
	'The cat jumped onto the windowsill to watch birds.',
	'His favorite ice cream flavor was mint chocolate chip.',
	'The book fell open to page 217.',
	'Lightning flashed across the night sky.',
	'They planted tulip bulbs in the garden last fall.',
	'The coffee shop was bustling with morning customers.',
	'She tied her hiking boots with double knots.',
	'The museum exhibit featured ancient Egyptian artifacts.',
	'Children laughed as they ran through the sprinkler.',
	'The train arrived precisely on schedule.',
	"He couldn't remember where he had parked his car.",
	'Autumn leaves crunched beneath their feet.',
	'The recipe called for two teaspoons of vanilla extract.',
	'The dog wagged its tail excitedly at the park.',
	'Mountains loomed in the distance, covered with snow.',
	'She practiced piano for three hours every day.',
	"The telescope revealed stunning details of Saturn's rings.",
	'Fresh bread was baking in the oven.',
	'They watched the sunset from the rooftop.',
	'The professor explained the theory with great enthusiasm.',
	'Waves crashed against the rocky shoreline.',
	'He assembled the furniture without reading the instructions.',
	'Stars twinkled brightly in the clear night sky.',
	'The old photograph brought back forgotten memories.',
	'Bees buzzed around the flowering cherry tree.',
	'She solved the crossword puzzle in record time.',
	'The air conditioner hummed quietly in the background.',
	'Rain pattered softly against the windowpane.',
	'The movie theater was packed for the premiere.',
	'He sketched the landscape with charcoal pencils.',
	"Children built sandcastles at the water's edge.",
	'The orchestra tuned their instruments before the concert.',
	'Fragrant lilacs bloomed along the garden fence.',
	'The basketball bounced off the rim.',
	'She wrapped the birthday present with blue ribbon.',
	'The hiker followed the trail markers through the forest.',
	'Their canoe glided silently across the still lake.',
	'The antique vase was carefully wrapped in bubble wrap.',
	'Fireflies flickered in the summer twilight.',
	'The chef garnished the plate with fresh herbs.',
	'Wind chimes tinkled melodically on the porch.',
	'The flight attendant demonstrated safety procedures.',
	'He repaired the leaky faucet with a new washer.',
	'Fog shrouded the valley below the mountain.',
	"The comedian's joke made everyone laugh.",
	'She planted herbs in pots on the kitchen windowsill.',
	'The painting hung crookedly on the wall.',
	'Snowflakes drifted lazily from the gray sky.'
]

#Intervention calibration

The following checks estimate how strongly the intervention influences the model's outputs.

Zoom

Zoom

Zoom

Zoom

Zoom

#Earlier low-sample sweeps

The earlier open-set and closed-set sweeps used only 16 trials per cell.

Zoom

Zoom

Several panels have suspicious model-specific constant floors across layers and coefficients. I treat these as pilot diagnostics, likely dominated by output-format or invalid-response behaviour. They motivated the denser early-layer evaluations shown above.

#Finer localisation sweeps

Zoom

Zoom

These runs show that localisation is highly sensitive to both layer and coefficient. For example, Llama 3.1 8B reaches its strongest region around the early layers and intermediate coefficients rather than improving monotonically with intervention strength. Gemma 4 E4B shows a similarly narrow band. This sensitivity is one reason not to treat a task-wise maximum as a general model ability.

The setup differs from Introspection Fine-Tuning in vector normalisation, prompt and concept banks, and evaluation format. The discrepancy is therefore evidence of setup sensitivity, not a clean failure to reproduce the earlier result.

#Selecting one shared configuration per model

For each layer-and-multiplier setting, I rescale every task's accuracy relative to its reference score:

Here, $A_t$ is the accuracy on task $t$. The reference $r_t$ is $0.5$ for binary detection and for the balanced open-set task, where always answering No injected thought scores 50%; it is $0.2$ for the five-choice tasks. The open-set value is therefore a trivial full-task baseline, not the chance of naming the injected concept correctly.

I leave scores below the reference negative, average the six task scores at each setting, and select the setting with the highest mean for that model. This global rule prevents each task from choosing its own favourable layer and multiplier, but it is still post-hoc because selection and evaluation use the same data.

#Gemma 2 corrected rerun: fixed settings

The first Gemma 2 advanced odd-one-out fine-tune used five-way cross-entropy over an answer digit. Its selected checkpoint emitted invalid answers throughout transfer evaluation, so that run is excluded from every substantive comparison.

The corrected run used causal-language-model loss on the exact two-token answer, reduced the learning rate from $2\times10^{-4}$ to $5\times10^{-5}$ and validated four times per epoch. It selected the second validation point in epoch 1. The model, layer, multiplier, concept splits, training-example count, LoRA rank, LoRA alpha, dropout, target modules, batch size and gradient accumulation stayed fixed. This corrected rerun is the authoritative Gemma 2 advanced odd-one-out result.

The existing figures retain both rows. Within their semantic odd-one-out training block, Gemma 2 9B denotes the invalid checkpoint and Gemma 2 9B rerun denotes the corrected run. The plotting source needed to remove the invalid row without reconstructing values was not available in the article assets, so the row is marked in adjacent prose and excluded from interpretation.

#Interactive Model Atlases

For full-resolution inspection of individual model runs, sweep matrices, and training checkpoints, two companion interactive tools are available: