Absolutely freezing, a bit cold: how language models place adjectives and their modifiers on one strength axis

TL;DR

Across five open language models (1.7B to 27B parameters), how strong an adjective is projects onto a single direction shared across scales, orthogonal to which pole of the scale the word belongs to. Degree modifiers share that same direction. Even before the adjective appears, the hidden states at slightly, very and absolutely are already ordered along it by grammatical class; when combined with an adjective, each modifier adds its own baseline shift while compressing the scale’s mild-to-strong span. This single coordinate matches how people pair modifiers with adjectives on the web, guides how models interpret novel words, transfers directly to Chinese, and drives downstream text generation.

• bad → terrible and good → excellent move along the same direction (cosine 0.35–0.46; shuffled-label baseline 0.065). Strong words barely spread outward along the good–bad axis (1–5% of the shift), so the four words of each scale form an approximate parallelogram.
• Read at the modifier token itself, before any adjective appears, 19 English modifiers already fall in the order diminishers < moderators < boosters < maximizers (Spearman 0.69–0.73 across all five models), with utterly 1.1–1.3 mild-to-strong steps above slightly.
• The shift a modifier gives a nonce word correlates with the shift it gives real adjectives (Spearman 0.84–0.98). At the same time, boosters and maximizers compress the gap between mild and strong words while leaving the good–bad gap intact, which is why “very scorching” ends up lower on the axis than bare “scorching”.
• Geometry and corpus usage agree from both sides: words that follow absolutely rather than very in web text project higher on the axis (Spearman 0.51–0.55 over 373 human-rated pairs), and modifiers that web writers prefer before strong words leave strong words higher on the axis (0.49–0.59 over 19 English modifiers; 0.86–0.90 over 12 Chinese adverbs in the Qwen models).
• In an LLM adaptation of Syrett & Lidz’s (2010) child word-learning experiment, hearing a novel word in completely pelgy rather than very pelgy shifts all five models toward a closed-scale interpretation with a natural endpoint (clear over tall), by 0.23–0.63 nats.
• Adding a strength vector learned from evaluative pairs to the hidden state of cold makes the model continue text more like freezing (closing 24% of the distributional gap, versus ~0 for controls). First-order projection onto the true target difference predicts the size of this effect across conditions (rank correlation 0.90–0.94), and a vector learned from English words alone transfers to Chinese as effectively as one learned in Chinese.

Earlier work ranked adjective intensity from corpus patterns and modifier co-occurrences (de Melo & Bansal 2013; Ruppenhofer et al. 2014), identified intensity directions in static word vectors and BERT (Kim & de Marneffe 2013; Garí Soler & Apidianaki 2020), and measured how adverbs shift BERT’s adjective representations along a scale direction (Lorge & Pierrehumbert 2023). Here we add four results:

1. In autoregressive decoder LLMs, an axis built from bare adjective pairs alone already orders degree modifiers before the adjective token appears, and decomposes modifier composition into a baseline shift plus a strength-specific compression in both English and Chinese.
2. Derived from a single log-linear inner product, the same axis aligns with web co-occurrence statistics from both the adjective side and the modifier side, surviving controls for word frequency and modifier class.
3. An LLM adaptation of the Syrett–Lidz word-learning paradigm links modifier cues to the geometric separation between extreme adjectives and closed-scale endpoints.
4. Causal interventions show that adding vectors along the axis steers natural continuations, that a first-order projection formula predicts the size of each intervention’s effect, and that an English-derived vector steers Chinese generation.

All primary tests were pre-registered with SHA-256 file hashes frozen before execution; exploratory and post hoc analyses are explicitly marked. The Appendix lists every pre-registered test and its outcome.

1. Good, excellent, bad, terrible

Cold and freezing describe the same property at different intensities. Linguists organize such gradable adjectives into scales, with good – excellent on one pole and bad – terrible on the other. The extreme members (excellent, terrible, freezing) differ systematically from ordinary gradable adjectives (Paradis 2001; Morzycki 2012): they resist very (“?very freezing”, “?very excellent”) and take maximizers such as absolutely (“absolutely freezing”). Extremeness is also symmetric across poles: terrible is just as extreme on the negative side as excellent is on the positive side.

Over the past decade, NLP researchers have extracted this structure in several ways. Pattern-based methods rank adjectives using frames like “good but not great” (de Melo & Bansal 2013) or modifier distributions (Ruppenhofer et al. 2014). Vector-space methods recover intensity as a translation between word embeddings (Kim & de Marneffe 2013), and in BERT the mean difference between extreme and mild adjectives defines a direction that orders unseen scales (Garí Soler & Apidianaki 2020).

Degree modifiers have a parallel literature. Cliff (1959) modelled adverbs such as very and slightly as scalar multipliers on an adjective’s value. Paradis (1997) grouped English degree modifiers into diminishers, moderators, boosters and maximizers according to the adjectives they select. Formal semantics treats them as functions over scale structure (Kennedy & McNally 2005), while compositional distributional models treat them as operators on word vectors (Baroni & Zamparelli 2010; Socher et al. 2012). Developmentally, 30-month-old children already use the contrast between completely and very to infer what kind of property a novel adjective denotes (Syrett & Lidz 2010).

The rest of this post addresses six questions:

1. Shape (Section 2): Does the model represent a scale as a single signed line from terrible to excellent, or as two separate coordinates, one for pole and one for strength?
2. Modifiers (Section 3): Do slightly, very and absolutely lie on the same strength axis, and how do they transform the representation of the adjective they modify?
3. Usage (Sections 4–5): Does the residual-stream geometry match how people pair modifiers with adjectives in web text, and which notion of “top of the scale” does the axis capture?
4. Learning (Section 6): Can a language model infer the scale structure and intensity of an unfamiliar adjective from the modifier in front of it, as young children do?
5. Use (Sections 7–8): Do downstream layers actually read this axis during generation, and can representation geometry predict how strongly an intervention alters behaviour?
6. Reach (Sections 9–10): Does Chinese use the same axis, and which components of the network build it?

We evaluate five autoregressive decoder models: four base models (Qwen3-1.7B-Base, Qwen3-8B-Base, Llama-3.1-8B, Mistral-7B-v0.3) and one post-trained model (Qwen3.8-27B). Unless noted otherwise, states are extracted at a single mid-depth layer, L* = round(3N/7) for an N-layer model, by reading the residual stream at the adjective’s final token and averaging across six short question templates. The core grid comprises 23 scales × 4 words (92 words total): 16 evaluative scales (e.g. good / excellent / bad / terrible) and 7 non-evaluative scales (e.g. big / huge / small / tiny). For out-of-sample evaluation we use 527 human-ranked weaker → stronger pairs compiled by Cocos et al. (2018), with all core-grid words excluded.

For degree modifiers we test 19 English modifiers across Paradis’s four classes (plus almost and nearly) and 13 Chinese degree adverbs spanning four standard grammatical levels: 低量 (low), 中量 (moderate), 高量 (high) and 极量 (extreme). The Chinese grid contains 16 scales × 4 words (10 evaluative, 6 non-evaluative). Every strong Chinese item is a single lexical form, never an analytic phrase of the form 很 + X: an idiom such as 至关重要 (crucial), a state adjective in Zhu Dexi’s (1956) sense such as 冰冷 (ice-cold), or a lexically strong word such as 狂喜 (ecstatic). Co-occurrence counts are drawn from 1.0 million English and 1.6 million Simplified-Chinese web documents. For the word-learning experiments we constructed 24 nonce adjectives ending in -y, starting with Syrett and Lidz’s pelgy.

2. Two coordinates: pole and strength

Across all five models, the vector offset from good to excellent points in roughly the same direction as the offset from bad to terrible. Polarity and intensity are represented as separate coordinates.

For each scale d, let A₀, A₁ denote the mild and strong words on one pole (good, excellent) and B₀, B₁ the mild and strong words on the opposite pole (bad, terrible). We decompose the two strengthening steps into a shared component m and a pole-dependent component g:

To avoid circularity, reference axes are always constructed leave-one-out, omitting scale d. The strength axis u is the unit mean of m across the remaining 22 scales, and the pole axis p is the mean good-minus-bad contrast across the remaining scales of the same type. We write M = m·u for the shared shift along the strength axis and O = g·p for any additional outward spread along the pole axis.

Two competing geometric hypotheses make opposite predictions:

Hypothesis	Do both strong words move positively along u?	Outward spread O / M
Single signed line: terrible is bad pushed further negative	No: d_A and d_B point in opposite directions	Large
Two coordinates: pole + strength	Yes: cos(d_A, u) > 0 and cos(d_B, u) > 0	Near 0

The empirical measurements support the two-coordinate decomposition in every model:

Model	cos(d_A, u)	cos(d_B, u)	O / M [95% CI]
Qwen3-1.7B-Base	0.44	0.38	0.047 [0.008, 0.089]
Qwen3-8B-Base	0.46	0.39	0.013 [−0.020, 0.046]
Llama-3.1-8B	0.39	0.35	0.045 [−0.002, 0.087]
Mistral-7B-v0.3	0.42	0.39	0.052 [0.005, 0.103]
Qwen3.8-27B	0.43	0.37	0.041 [−0.013, 0.090]

Under 1,000 random relabellings of the 92 words onto the grid, the 95th percentile of the cosines is 0.060–0.068. Both pre-registered criteria pass in all five models: both pole cosines clear the null threshold, and |O / M| ≤ 0.5.

We can summarize this structure with an additive model:

Because pole σ and strength t combine additively, the four words of a scale form an approximate parallelogram: h(A₁) − h(A₀) ≈ h(B₁) − h(B₀). The 2 × 2 product grid {mild, strong} × {pole A, pole B} is preserved in the residual stream, matching the linguistic observation that extremeness is symmetric across poles.

Figure 1 also shows a few secondary details. Strong evaluative pairs spread 6–17% wider along the good–bad axis than mild pairs, whereas non-evaluative pairs do not widen (ratio 0.87–1.00); the negative pole also moves slightly less far along u than the positive pole (79–88%). And while the difference vector g has a norm 74–78% as large as m, almost none of it projects onto the shared pole axis; it reflects word- and scale-specific idiosyncrasies rather than a global polarity push.

Two post hoc checks fill out the picture. First, measuring the angle between the strength step and the pole axis shows that evaluative scales are nearly rectangular (tilting −5° to +1°), whereas non-evaluative scales lean 3–11° toward the “more” pole (big, hot, loud). Second, the shared strength axis is nearly orthogonal to both the shared valence axis and the “more → less” magnitude axis (cosines −0.16 to 0.11): big → huge aligns just as much with cold → freezing as it does with hot → scorching (0.15–0.20 for both).

3. Modifiers live on the same axis

Putting a degree modifier such as slightly, very or absolutely in front of an adjective does two things in the residual stream: it shifts the modified word along u by a baseline amount, and it linearly compresses the scale’s original mild-to-strong span. That baseline shift is already present at the modifier token, before the adjective is seen. In left-to-right decoding, when the model has only processed “…described it as absolutely”, the state at absolutely already projects 0.64–0.88 steps higher on u than the state at “described it as”.

We first examine what modifiers do when combined with real adjectives. For each modifier m and each word w in the 92-word grid, we read the residual state of “m w” at the adjective token across the same six templates and measure how far the modifier shifts the word along the leave-one-out strength axis u (normalized so that the scale’s own mild-to-strong step Δ equals 1). Each modifier is summarized by two parameters: how far it shifts the scale midpoint (the baseline shift s) and by what fraction it narrows the mild-to-strong gap (the compression κ). The resulting positions of the mild and strong words are:

Here Δ ≈ 1 is the scale’s unmodified mild-to-strong step. Modifier classes were fixed prior to inspecting any activations: Paradis’s (1997) four classes for the 19 English modifiers, and the four grammatical levels for the 13 Chinese adverbs (Figure 2).

Figure 2 shows two patterns:

• Shifts follow grammatical class. Across the 19 English modifiers, the midpoint shift orders diminishers (−0.15 to −0.03 steps) below moderators (−0.06 to +0.07), boosters (+0.03 to +0.30) and maximizers (+0.09 to +0.36). The rank correlation between class and shift is 0.78–0.91 across all five models (permutation p ≤ 0.0002); for the Chinese adverbs it is 0.89–0.93 in the three Qwen models and 0.71–0.79 in Llama and Mistral.
• Diminishers push representations negative. All four English diminishers produce negative shifts in every model (mean −0.07 to −0.12 steps, with all 95% upper bounds below −0.05). In Chinese, 稍微 (“slightly”) likewise shifts words downward in the three Qwen models and Llama (−0.05 to −0.11; descriptive per-adverb check). Upward movement along u strengthens a word; downward movement weakens it.

The modifier’s baseline value. Because a causal decoder at absolutely in “…described it as absolutely” cannot see the upcoming adjective, its representation there reflects only the template prefix and the modifier itself. In a pre-registered follow-up, we truncated each template immediately after the modifier and projected the modifier’s own state onto the adjective-derived axis u. As a second check, we placed each modifier before the 24 nonce adjectives from Section 6; because nonce words have no stored intensity of their own, any displacement they show must come from the modifier.

Model	Own value vs class	Shift on nonce words vs shift on real words	Own value vs shift on real words
Qwen3-1.7B-Base	0.73	0.98	0.91
Qwen3-8B-Base	0.72	0.97	0.87
Llama-3.1-8B	0.70	0.98	0.91
Mistral-7B-v0.3	0.73	0.96	0.96
Qwen3.8-27B	0.69	0.84	0.85

The table reports Spearman rank correlations across the 19 English modifiers (all permutation p ≤ 0.0012). Even before an adjective appears, the modifiers are already ordered along u. In Qwen3-8B, measured relative to the template truncated just before the modifier, slightly projects at −0.12 steps, very at −0.03, absolutely at +0.76 and utterly at +1.12; averaged across models, maximizers project 0.73–0.92 steps higher than diminishers. For the Chinese adverbs in the Qwen models, the modifier’s own value correlates with its shift on real words at 0.64–0.81.

When attached to nonce adjectives, boosters and maximizers induce 0.5–1.0× the shift they produce on real adjectives, in nearly the exact same rank order (Spearman 0.84–0.98). Diminishers barely move nonce words, since weakening an adjective requires some initial intensity to subtract from.

Compression acts on strength, not polarity. Alongside the baseline shift, boosters and maximizers compress the mild–strong gap (κ = 0.21–0.39 for boosters, 0.12–0.24 for maximizers) while leaving the good–bad polarity gap between the same words nearly unchanged. For 11 of the 12 English boosters and maximizers (all except perfectly), the polarity gap changes by at most 8%, and strength compression exceeds polarity compression in all five models. In the Qwen models, every Chinese 高量 and 极量 adverb likewise compresses strength far more than polarity (lower 95% bounds on the difference ≥ 0.20; post hoc grouping). Diminishers are the exception, compressing both axes in most models so that “a bit good” and “a bit bad” lie closer together than bare “good” and “bad”.

Why “very excellent” clashes while “absolutely excellent” works. Writing x for an adjective’s unmodified position on u, the modifier maps it to:

Because s belongs to the modifier while κ scales down the word’s existing intensity x, a modifier with a modest baseline value and heavy compression will lift a mild word but pull back an already extreme one. That is what happens with very: its baseline shift is small (+0.03 to +0.11 steps) and its compression is large (24–37%), so it raises hot but pulls scorching downward by 0.06–0.10 steps. Absolutely combines a large baseline shift (+0.17 to +0.29) with light compression (13–16%), lifting mild and extreme words alike.

Rarer boosters shift words further (exploratory). Human listeners interpret rarer, higher-cost intensifiers as stronger (Bennett & Goodman 2018), and the models show the same gradient. Extremely, incredibly, terribly and remarkably are 11–155× rarer than very in our web shard, and across all five models they shift adjectives 2.5–4.9× as far. Within the six boosters, shift correlates with rarity at Spearman 0.54–0.83; across boosters and maximizers combined, controlling for class, the partial correlation is 0.45–0.64.

Chinese state adjectives and the bleaching of 很. Zhu Dexi (1956) noted that reduplicated or vivid state adjectives (状态形容词) already encode high intensity and resist modification by 很 (“very”). Six of our 32 strong Chinese words are state adjectives in his sense (冰冷, 滚烫, 漆黑, 飞快, 慢吞吞, 湿透). Checked post hoc, 非常 (“very”) shifts them by only −0.07 to +0.04 steps while lifting their mild counterparts (冷, 热, 昏暗, 快速, 缓慢, 潮湿) by 0.15–0.25 steps. And when read at its own token position, 很 projects at or below the low and moderate adverbs in every model (−0.91 to −0.17 steps, compared with +0.47 to +0.91 for 极其, 极为 and 极度), consistent with syntactic accounts of predicative 很 as a bleached marker of the plain positive form (Grano 2012).

Approximators bypass the strength axis. Almost and nearly shift words along u by at most 0.11 steps and compress the mild–strong gap by no more than 10%; instead, they compress the polarity gap by 8–29%. In web usage they pair primarily with closed-scale endpoints such as identical (Section 5).

4. Geometry agrees with usage, from both sides

The strength axis aligns closely with how modifiers are used in web text. Looking from the adjective side, words that appear more often after absolutely than after very project higher on u; looking from the modifier side, modifiers that appear more often before strong adjectives leave strong words higher on u.

Both relationships follow from a single inner product in log-linear language models (Arora et al. 2016), where the log-probability of word w following modifier m is proportional to the inner product between a word vector v_w and a context vector c_m. Fixing a pair of modifiers and varying the adjective yields the adjective-side projection; fixing a strong and a mild adjective and varying the modifier yields the modifier-side projection:

If a network internalizes these co-occurrence statistics in its middle layers, the first quantity will appear as the word’s coordinate along a shared direction, and the second as the modifier’s projection onto the mild-to-strong step. While the derivation itself is a toy log-linear sketch, it directly motivates the two empirical tests below.

4.1 The adjective side

In a 1,048,581-document FineWeb shard, we counted how often each adjective directly follows each degree modifier (lowercased, adjacent tokens only). Some representative raw counts:

Word	absolutely	utterly	truly	very	really	too	pretty	extremely
good	12	0	121	20,320	10,213	2,137	8,420	400
excellent	19	1	102	84	140	8	19	14
cold	1	1	5	760	209	795	84	175
freezing	19	3	0	0	3	1	0	1
incredible	214	5	85	15	96	6	155	1

Following Ruppenhofer et al. (2014), who ranked adjective intensity by contrasting end-of-scale modifiers with ordinary ones, we measure each word’s modifier preference by comparing an extreme modifier set E = {absolutely, utterly, truly} against a scalar set S = {very, really, too, pretty, extremely}:

Both sets E and S were fixed during an earlier corpus analysis of the 92-word core grid, prior to examining the human-rated pairs. Evaluating on the Cocos et al. (2018) human rankings is therefore a strictly out-of-sample test for both u and Δr.

Across the 373 human-rated pairs in which both words have at least 10 modifier occurrences, corpus usage and residual-stream geometry agree. In the corpus counts alone, the stronger adjective favours the absolutely set (mean Δr = 1.09 [0.87, 1.32], positive in 69% of pairs). Inside the models, the pair difference projected onto u correlates with Δr at Spearman 0.51–0.55 across all five models (95th percentile under shuffled Δr is 0.09). Controlling for the log frequency ratio between the two words leaves a partial correlation of 0.45–0.49 (all lower 95% bounds ≥ 0.36); at the single-word level, a word’s projection onto u correlates with its own r(w) at 0.62–0.67 (0.57–0.63 after controlling for frequency).

As a sanity check, pairs rated by human annotators as equally intense, but differing in corpus frequency by at least 2×, have a mean Δr of 0.09, confirming that modifier preference tracks intensity rather than raw frequency.

4.2 The modifier side

Turning to the modifier side, we measure for each modifier how strongly web text prefers it before the 46 strong grid words relative to the 46 mild ones, normalized by the baseline frequencies U of the adjectives themselves (32 strong and 32 mild words in Chinese):

Here C_m counts how often modifier m appears immediately before a grid word, and U counts the occurrences of the grid words overall. The pre-registered geometric predictor is the position where the modifier leaves the strong word, d_strong (the cyan dots in Figure 2). Representative raw counts in English and Chinese:

Word	slightly	very	extremely	absolutely	utterly
hot	10	1,187	247	3	1
scorching	0	4	1	5	1
cold	12	760	175	1	1
freezing	0	0	1	19	3

Word	稍微	很	非常	十分	极其
热 hot	22	2,860	354	16	1
滚烫 scalding	0	2	1	2	0
冷 cold	13	3,020	194	17	0
冰冷 ice-cold	1	35	17	23	12

The Spearman rank correlation between d_strong and corpus preference is 0.49–0.59 across the 19 English modifiers in all five models (permutation p ≤ 0.017), and 0.86–0.90 across the 12 Chinese adverbs in the Qwen models (0.77–0.85 in Llama and Mistral). Computing the log-odds within each scale first and then averaging across scales, so that high-frequency collocations like very good do not dominate the sum, raises the English correlation to 0.65–0.71 (p ≤ 0.0016; exploratory). Even after controlling for Paradis’s modifier classes, the within-class partial correlation remains 0.39–0.63.

Both languages show the same pattern at the extremes. Absolutely, utterly, 极度 and 极其 place strong words highest on u, and relative to baseline word frequency, web writers pair them most often with strong words. Conversely, very, 稍微 and 挺 pull strong words back toward the middle of the scale, and in web text they attach almost exclusively to mild words (very precedes mild grid words 91,394 times versus 1,559 times for strong words). Perfectly is the one exception in English: the models group it geometrically with the maximizers, whereas corpus usage is dominated by mild collocations such as “perfectly fine” and “perfectly good”.

4.3 Beyond frequency

Strong adjectives are typically rarer than mild ones (huge is less frequent than big). Because word frequency enters log-linear representations through the word vectors themselves, some frequency signal along u is expected. On the 92 core words, frequency accounts for roughly a fifth of the projection onto u, but three pre-registered checks show that the downstream-active signal is intensity:

1. Equal-intensity, unequal-frequency pairs. Thirty-one human-rated pairs of equally intense adjectives that differ in corpus frequency by at least 2× have a mean projection of only 0.05–0.09 on u. Scaled to the frequency gap of the core grid, frequency accounts for 18–25% of the core words’ projection on u (upper 95% bound ≤ 0.33).
2. Modifier preference controlled for frequency. As shown in Section 4.1, controlling for the log frequency ratio leaves a partial correlation of 0.45–0.49. Restricting to the 145 human pairs where the stronger word is not rarer than the weaker one (cyan dots in Figure 3) still gives a rank correlation of 0.49–0.52; within this subset, pairs that favour absolutely project at +0.02 to +0.06 on u, while the remainder project at −0.07 to −0.10.
3. No causal effect from the frequency direction. When the mean “common → rare” vector is scaled to match the norm of the strength vector and planted into the model (Section 7), it closes only 0.01–0.03 of the distributional gap, compared with 0.28 for the strength vector.

5. Extreme words and closed-scale endpoints

There are two distinct ways for an adjective to be “at the top” of a scale. Exhausted pushes tiredness beyond its ordinary range on an open scale, whereas complete names the endpoint of a scale that has a built-in upper bound. The strength axis u encodes the first of these: extreme pairs such as tired → exhausted align with u roughly four times as strongly as closed-scale endpoint pairs such as partial → complete (mean cosine 0.36–0.42 versus 0.09–0.11).

Formal semantics draws a sharp boundary here. Extreme adjectives (exhausted, enormous, hilarious) exceed the standard range of an open scale and select maximizers like absolutely (Morzycki 2012). Maximum-standard adjectives (identical, certain, asleep, extinct) denote the upper endpoint of a closed scale and select proportional modifiers such as completely, almost and half (Kennedy & McNally 2005). Paradis (1997, 2001) similarly distinguishes extreme from limit adjectives, though in her taxonomy both count as bounded and both resist very. If a language model collapsed both phenomena into a single “top of the scale” direction, extreme and maximal pairs would project onto u equally.

Before computing any activations or corpus counts, we fixed 24 extreme pairs and 24 maximum-standard pairs outside the core grid and measured each pair’s cosine with u:

Metric	Extreme pairs (tired → exhausted)	Maximal pairs (partial → complete)
Cosine with u, 5 models	0.36–0.42	0.085–0.106
Shuffled baseline, 95th percentile	0.07–0.08	0.04–0.05
Model’s own modifier preference, stronger minus weaker word (log-odds, absolutely-type over completely-type)	+1.0 to +1.7	−1.1 to −1.2

In all five models, the 95% confidence interval for the group difference lies above 0.20. Word frequency does not explain the separation: although extreme adjectives are roughly 6× rarer than their mild partners while maximal adjectives are not, matching the two groups on frequency gap (post hoc) still leaves maximal pairs 0.20–0.22 lower on u. All 24 extreme pairs project above 0.22, whereas five maximal pairs project below zero (including comparable → equal and translucent → transparent).

The models’ next-token probabilities show the same grammatical split. We scored each adjective following the frames “It was / They were / Everything was / She was” plus a modifier. Below are the raw log-probabilities from Qwen3-8B-Base in the frame “It was”:

log P(word | "It was <modifier>")  absolutely utterly  very   completely almost  half
tired                               -11.84    -10.50   -8.56  -11.79    -13.40  -12.42
exhausted                           -10.44     -8.31  -12.22   -8.66    -12.82  -12.00
funny                                -8.42     -8.66   -5.39  -10.03     -6.06  -12.13
hilarious                            -5.46     -5.57  -10.80   -8.21     -8.38  -15.28
similar                             -11.39    -11.26   -6.00  -11.05     -9.98  -14.23
identical                            -8.97     -9.10  -13.70   -9.32     -6.79  -13.52
sleepy                              -13.29    -12.75  -10.04  -13.52    -12.50  -13.10
asleep                              -14.45    -12.56  -15.53  -11.60    -11.48  -10.33

Every mild word in the table prefers very, whereas on the strong side hilarious prefers absolutely, identical prefers almost, and asleep prefers half. Exhausted straddles the boundary: the model assigns nearly equal probability to completely and utterly, mirroring web usage (“completely exhausted” occurs 74 times in our shard, “absolutely exhausted” 25 times). Across all 96 words, model log-odds correlate with web log-odds at Spearman 0.75–0.79.

The corpus counts clarify what the two classes share and where they diverge. Both avoid very (“very tired” 341 vs “very exhausted” 18; “very similar” 4,114 vs “very identical” 4), but proportional approximators separate them: “almost identical” (596), “nearly identical” (410) and “half asleep” (104) are common, whereas extreme adjectives almost never take almost or half.

The maximal pairs instead align with one another along a shared direction of their own (leave-one-out cosine 0.20–0.24; shuffled baseline 0.065). Projecting out the u component (post hoc) leaves the maximal pairs’ internal alignment intact (0.19–0.22), while the extreme pairs drop to 0.05 on the residual subspace. In short, extreme steps lie primarily along u, whereas closed-scale endpoint steps combine a small projection on u with this shared endpoint direction. This matches Section 3, where almost and nearly barely move words along u.

6. Reading a new word through its modifier

If you hear “the jar is completely pelgy” for the first time, what kind of property does pelgy denote? Thirty-month-old toddlers already use the modifier as a syntactic cue, inferring that completely marks a property with a natural endpoint (Syrett & Lidz 2010). Adapted to text prompts, the same cue guides all five language models: pairing a nonce word with completely rather than very shifts predictions toward the closed-scale property, while embedding the word in a paragraph of absolutely-type modifiers makes the models interpret it as more intense.

In Syrett and Lidz’s original preferential-looking study, children were introduced to an object possessing two properties simultaneously, such as a container that was both tall (open scale) and transparent (closed scale), described with a novel adjective after either completely or very. When shown two test objects that separated the two properties, children who had heard completely looked significantly longer at the object exhibiting the closed-scale endpoint property (transparent), whereas children who had heard very favoured the open-scale property (tall).

Our text adaptation names both candidate properties explicitly in the prompt. We constructed 24 nonce adjectives ending in -y (beginning with Syrett and Lidz’s pelgy), 21 of which never appear in our 1,048,581-document web shard and three of which occur at most three times. Each nonce word was paired with eight objects carrying one open-scale and one closed-scale property (e.g. a jar: tall / clear; a towel: heavy / dry; a pool: deep / empty). Shown with very:

Look at this jar. The jar is tall and clear. The jar is very pelgy.
Q: Does pelgy mean tall or clear?
A: It means

We compute log P(closed property) − log P(open property) at the answer token, averaged across both property orders and both option orders, and measure the shift from very to completely:

Model	completely − very, nats [95% CI over 24 words]
Qwen3-1.7B-Base	0.63 [0.52, 0.75]
Qwen3-8B-Base	0.31 [0.25, 0.37]
Llama-3.1-8B	0.23 [0.20, 0.25]
Mistral-7B-v0.3	0.37 [0.33, 0.41]
Qwen3.8-27B	0.61 [0.54, 0.68]

Across all five models, very and the bare-adjective baseline produce similar scores, while completely consistently shifts probability toward the closed-scale endpoint property. Replacing completely with absolutely (which selects open-scale extreme adjectives) gives scores close to very in four of the five models (all except Qwen3.8-27B), confirming that the endpoint inference is driven specifically by proportional completely rather than by high modifier intensity in general. In Qwen3-1.7B, swapping the modifier flips the greedy continuation outright (next-token probabilities followed by the unedited 8-token greedy continuation):

very pelgy        ' tall' 0.338   ' clear' 0.305   →  ' tall.<|endoftext|>Human: Given the question'
completely pelgy  ' clear' 0.463  ' tall' 0.122    →  ' clear.<|endoftext|>Human: Given the question'

Inferring intensity from context. When a model reads four exposure sentences in which pelgy consistently follows absolutely, downright or positively (“In July the attic gets absolutely pelgy…”), all five models rate the word as closer to scorching than to hot relative to a paragraph using very, really or pretty, by 0.07–0.55 nats. While much of that shift comes from the modifier tone of the passage as a whole, comparing against a control passage where the modifiers attach to a different nonce word (post hoc) isolates a word-specific effect of 0.03–0.16 nats in four of the five models.

To see the effect in free generation, we gave Llama-3.1-8B a short paragraph about a restaurant, a concert and a trip and asked what the nonce adjective meant (greedy decoding for 24 tokens, truncated at the first complete answer):

absolutely-type paragraph  ' "excellent" or "great." It is often used to describe something that is very enjoyable or satisfying.'
no modifiers               ' "very good" or "excellent."'
very-type paragraph        ' "pleasant" or "enjoyable."'

Notice that adding very-type modifiers actually pulls the inferred meaning downward from the bare-adjective baseline (“very good / excellent” → “pleasant / enjoyable”), mirroring the compression effect of very on strong adjectives in Section 3.

Two checks link this contextual inference directly to the strength axis u. Post hoc, at each occurrence of “absolutely pelgy” in the exposure passage, the hidden state at pelgy projects 0.13–0.29 steps higher on u than bare pelgy in the same sentence. That is an order of magnitude larger than on random directions and matches the single-sentence nonce shift in Section 3 (0.12–0.29). In the pre-registered clamp test, setting just that single coordinate along u back to the bare-word value at each mention removes 8–16% of the paragraph’s effect on the final intensity judgement, and adding the shift to a paragraph without modifiers reproduces 9–23% of it, whereas clamping any of 20 random orthogonal directions moves at most 1.4% (95th percentile).

7. The model uses the axis: planting a stronger word

To test whether downstream layers causally read the strength axis during generation, we plant a strength vector learned purely from evaluative pairs (good → excellent) into the non-evaluative adjective cold. With the network weights frozen and without ever seeing a single passage containing freezing, the planted token steers next-token continuations roughly a quarter of the way from cold toward freezing.

Planting procedure. Given a source word s (cold) and a target word t one step stronger on the same scale (freezing), let Δ denote the mean strengthening step computed across other scales at layer L* (always excluding t’s own scale). We append a new trainable row e to the input embedding table and optimize e alone, with all transformer weights frozen, so that across 36 short template contexts of the source word s, the new token arrives at layer L* with s’s hidden state plus αΔ (with α = 2 and no target-word contexts used):

Evaluation. On 80 held-out FineWeb passages where t naturally occurs, we substitute either the source word s or the planted token in place of t and measure the KL divergence of the model’s next-token distributions over the following 8 token positions relative to the original passage with t. Gap closed (GC) is the fraction of the source-to-target KL divergence eliminated by the planted token:

Here GC = 0 indicates no improvement over the unmodified source word, and GC = 1 indicates next-token distributions indistinguishable from the true target word.

Vector planted (α = 2)	Planted into	GC, Qwen3-8B [95% CI]	GC, Qwen3.8-27B [95% CI]
Strength, all other scales	all 92 words	0.283 [0.23, 0.34]	0.277 [0.22, 0.33]
Strength, evaluative scales only	28 non-evaluative words	0.244 [0.18, 0.31]	0.244 [0.19, 0.31]
Strength, non-evaluative scales only	64 evaluative words	0.235 [0.17, 0.31]	0.232 [0.17, 0.30]
Frequency direction, same norm	all 92 words	0.025 [0.00, 0.05]	0.013 [−0.02, 0.05]
Relabelled controls (mean)	matching sets	−0.02 to 0.03	0.00 to 0.04

A strength vector learned from evaluative scales alone retains 93–94% of the full vector’s effect when planted into the 28 non-evaluative words (GC = 0.244 in both models), and reverse transfer from non-evaluative to evaluative words is equally strong (GC = 0.232–0.235). For comparison, planting a valence vector (good → bad) under the same protocol closes 0.47–0.50 of the gap, whereas an equal-norm frequency vector and relabelled control vectors remain near zero.

The greedy continuations below (Qwen3-8B on the prompt "Honestly, the whole experience was {x}", 12 tokens, unedited and truncated to ~60 characters) illustrate what the intervention changes:

cold → freezing   source  cold          ' and impersonal. I was not given a chance to explain'
                  planted (eval-only)   '. I was so cold that I was shivering. I'
                  relabelled control     ' and impersonal. I was told that I would be able'
hot → scorching   target  scorching     ' hot. I was sweating like a pig in a poke,'
                  source  hot           ' garbage. I was so excited to try this place because it'
                  planted (eval-only)   '. I was sweating like a pig, and my heart was'
                  frequency vector      ' and cold. I was so excited to see the movie,'
big → huge        frequency vector      ' and small. I was a little nervous about the whole thing'

Where bare cold defaults to a figurative reading (“cold and impersonal”) and bare hot triggers the idiom “hot garbage”, planting the evaluative strength vector shifts both into literal physical extremes (“so cold that I was shivering”, “sweating like a pig”). Planting the frequency vector merely muddles the antonym axis, turning hot into “hot and cold” and big into “big and small”.

Usage shifts more than token identity. To check whether planting simply overwrites which word the model takes the token to be, we prompt Qwen3-8B to repeat the word directly ("…the visit felt {x}. How did the visit feel? A: It felt"). The planted big token gives the following next-token probabilities:

source big        ' big' 0.854   ' huge' 0.016
planted (eval-only) ' big' 0.207   ' like' 0.142   ' huge' 0.111   ' gigantic' 0.098

Probability mass shifts substantially toward huge (0.111) and gigantic (0.098), yet big remains the top token (0.207). Across all 92 cells, the top naming response flips to the target word in only 21–26% of cases (almost all in the strong → mild direction). Editing along u primarily alters how the token composes with downstream context rather than erasing which word the model takes it to be.

8. Geometry predicts how much an edit does

The geometry at the planting layer predicts these effect sizes, including why cross-type planting keeps more than 90% of the full vector’s effect while the frequency vector does almost nothing. The first-order projection of the planted vector onto the true target difference predicts the behavioural shift across all 10 conditions (GC ≈ 0.76 × projection, rank correlation 0.90–0.94), whereas raw Euclidean distance to the target state points in the opposite direction.

For each target cell k, let Tₖ = h(target) − h(source) denote the true hidden-state difference at the planting layer and Dₖ the vector we planted. Before inspecting the behavioural fit, we computed two purely geometric candidate predictors of GC:

Here π is the first-order linear projection (the fraction of the true displacement covered along its own direction), and q is the second-order reduction in squared Euclidean distance (which treats every direction in the residual stream as equally important).

Comparing π and q against the observed gap closed (GC) shows a sharp contrast:

• Cross-type transfer ratios follow directly from π. From geometry alone, the evaluative-only vector is predicted to retain 0.99–1.00 of the full vector’s effect on non-evaluative words, and empirically retains 0.93–0.94; reverse transfer (non-evaluative → evaluative) is predicted at 0.81–0.84 and observed at 0.80–0.81.
• Condition-level and cell-level ranking. Across the 10 experimental conditions (three strength vectors, their relabelled controls, and the frequency vector), π correlates with observed GC at Spearman 0.94 in Qwen3-8B and 0.90 in Qwen3.8-27B; pooled across both models, the least-squares fit is GC ≈ 0.76 π + 0.005. Across the 92 individual word cells, the correlation remains clear at 0.67 (8B) and 0.61 (27B).
• Squared-distance reduction q is negative in every single condition (−0.04 to −0.90). In raw Euclidean distance, the planted tokens actually end up further from the target state than the unmodified source words, even as their generated continuations move toward the target.

That divergence shows that the downstream network weighs directions in the residual stream very unequally. To second order, the KL divergence induced by a small state perturbation δ is a quadratic form, KL ≈ ½ δᵀ F δ, where F is the pullback Fisher information matrix of all subsequent layers, and Euclidean distance q would only be the right predictor if F were proportional to the identity. We observed the same phenomenon in small synthetic models trained from scratch (unpublished), where representation structure was clean under the downstream readout metric but noisy in raw Euclidean space.

The single outlier in Figure 6, the frequency vector, provides a direct test of this metric explanation. Under the Euclidean dot product, the frequency vector has a positive projection π that predicts 22–25% of the strength vector’s effect, yet its observed GC is only 5–9% as large. Replacing the Euclidean inner product in π with the downstream Fisher information matrix F_k (estimated by central differences through the full network in fp32 on Qwen3-8B, or via the cheaper word-position logit lens in both models) resolves the discrepancy:

Ratio of frequency vector to strength vector	Euclidean π	Fisher π_F	Logit-lens π	Observed GC
Qwen3-8B-Base	0.25	0.006	0.024	0.088
Qwen3.8-27B	0.22	—	0.084	0.047

Under the Fisher metric, the frequency vector’s overlap with the true target difference nearly vanishes (dropping from 0.25 to 0.006): its Euclidean overlap with Tₖ lies in directions that downstream layers barely read. The much cheaper logit-lens metric produces the same correction in both models.

We also checked whether an exact single-site activation patch at layer L* (replacing the hidden state at the adjective position alone with h − T + αD, without training an input embedding) reproduces the planting results. Across the 10 conditions, single-site patching matches the observed GC with a rank correlation of 0.99 (8B) and 0.93 (27B) and a mean absolute error of 0.05–0.07. Because a full mild-to-strong word swap steps beyond the local quadratic regime (½ TᵀFT accounts for roughly a third of the exact KL divergence), an exact single-site patch is the most accurate predictor at this scale.

9. The same axis in Chinese

Chinese gradable adjectives share an internal strength axis as well, and in the middle layers of multilingual models it points in largely the same direction as the English axis: a vector learned from English words alone achieves 85–90% of the within-Chinese alignment in the three Qwen models, and 63–72% in Llama and Mistral.

Chinese grammar has a well-known counterpart to the absolutely / very contrast. In his classic study of modern Chinese adjectives, Zhu Dexi (1956) distinguished simple property adjectives (性质形容词) such as 冷 (cold) and 黑 (black) from complex state adjectives (状态形容词) such as 冰冷 (ice-cold) and 漆黑 (pitch-black), noting that state adjectives already encode high intensity and resist the default degree adverb 很 (“very”). Our 16 Chinese scales (Section 1) pair simple mild adjectives with single-word strong forms (state adjectives, idioms or lexically extreme words), never analytic 很 + X phrases.

In 167,328 Simplified-Chinese web documents (the first 200 row groups of the FineWeb-2 shard from Section 4, segmented with jieba), strong adjectives take ordinary degree adverbs (很, 非常, 十分, …) at a fraction of the rate of their mild partners:

Mild word	With a degree adverb	Strong word	With a degree adverb
冷 cold	12.7%	冰冷 ice-cold	0.5%
高兴 happy	27.4%	狂喜 ecstatic	0%
好吃 tasty	24.9%	美味 delicious	4.0%

Twenty-seven of the 32 Chinese mild → strong pairs follow this drop, with a mean log-odds change of −1.86 [−2.39, −1.34]; conversely, strong words combine significantly more often with maximizer adverbs such as 简直 / 完全 / 绝对 (“simply”, “completely”, “absolutely”; mean log-odds change +1.48 [1.00, 1.97]).

At layer L*, we compared each Chinese strengthening step (e.g. 冷 → 冰冷) against two reference directions: the English strength axis u (rebuilt without the corresponding English translation scale, so that big → huge is omitted when testing 大 → 巨大) and the leave-one-out mean of the remaining 15 Chinese scales:

Model	Chinese step · English u	Chinese step · other Chinese steps	Ratio	Shuffled baseline, 95th pct
Qwen3-1.7B-Base	0.238	0.271	0.88	0.049
Qwen3-8B-Base	0.231	0.272	0.85	0.045
Qwen3.8-27B	0.248	0.274	0.90	0.047
Llama-3.1-8B	0.194	0.271	0.72	0.039
Mistral-7B-v0.3	0.155	0.248	0.63	0.035

Both pre-registered criteria, evaluated on the three Qwen models, pass in every model tested (including Llama and Mistral): the Chinese steps align with one another and project onto the English axis u well above the shuffled null (the ratio column is a descriptive summary added after inspecting the first four models). The two-coordinate pole × strength geometry carries over to Chinese as well: strong words on both poles move positively along the shared Chinese strength direction (cosine 0.24–0.30, baseline 0.07), while outward spread along the pole axis is only 7–12% of the strength shift.

Just like the monolingual axis, this cross-lingual alignment emerges inside the network rather than in the input vocabulary. At the embedding layer, the cosine between Chinese strengthening steps and English u is near zero (0.005–0.036); it rises sharply to a peak at mid-depth (0.15–0.27) before tapering off toward the final layers (Figure 7).

Cross-lingual planting. We repeated the causal planting experiment of Section 7 in Chinese on Qwen3-8B and Qwen3.8-27B across all 64 Chinese cells (32 mild → strong, 32 strong → mild). We compared two planted vectors: the leave-one-out mean of the other Chinese scales, and a vector computed from English words alone (with the translation-equivalent English scale removed). Neither vector uses any text containing the target Chinese word. Table values report gap closed (GC) across 80 held-out Chinese web passages per target word, with 95% confidence intervals from cell-level bootstrap resampling:

Vector planted (α = 2)	Qwen3-8B	Qwen3.8-27B
Chinese steps (other scales)	0.17 [0.11, 0.22]	0.17 [0.12, 0.22]
English steps only	0.17 [0.10, 0.24]	0.18 [0.13, 0.23]
Relabelled control, Chinese	−0.02	0.01
Relabelled control, English	−0.03	0.01

The vector learned purely from English adjectives steers Chinese continuations just as effectively as the vector learned from Chinese adjectives (GC = 0.17 and 0.18). Both pre-registered thresholds pass in both models: the English vector closes at least 0.10 of the gap and beats its control by at least 0.10, and the Chinese vector closes at least 0.15 and beats its control by at least 0.10. We had conservatively marked the cross-lingual planting test as uncertain before the run. The two vectors also succeed on largely the same individual words (cell-level rank correlation 0.65–0.68, computed post hoc).

Two unedited greedy continuations from Qwen3.8-27B (16 tokens; … marks truncation, ⏎ a newline) illustrate the cross-lingual transfer. In the first example, the prompt is 老实说，那里的一切都{x} (“Honestly, everything there was {x}”), with source word 昏暗 (dim) and target word 漆黑 (pitch-black):

source   昏暗 dim           '而模糊，但有一样东西却清晰得令人不安：那是一幅…'
target   漆黑 pitch-black   '一片，只有那盏灯，在黑暗中发出微弱的光。⏎⏎“你…'
planted, English vector    '，只有那盏灯，在黑暗中发出微弱的光。⏎⏎“你来了…'
planted, Chinese vector    '而模糊，但有一样东西却清晰得令人不安：那是一幅…'
relabelled control          '而模糊，但有一样东西却清晰可见：一个男人，一个瘦…'

Whereas bare 昏暗 continues “…and blurry, but one thing was disturbingly clear”, planting the English-derived strength vector causes the Chinese continuation to match the target word 漆黑 almost verbatim: “…only that lamp, giving off a faint light in the dark.”

In the second example, the model is prompted to repeat the word (她在信里写道，那次拜访感觉“{x}”。那次拜访感觉怎么样？⏎答：“ / “She wrote that the visit felt ‘{x}’. How did the visit feel? A: ’”), with the English vector planted into 冷 (cold):

source 冷 cold      '冷' 0.527                    '冷”这个词，在中文语境里，往往不只是指温度，更像…'
planted, English    '冰冷' 0.055   '冷' 0.040     '冰冷”这个词，我认为是她自己的感受。她可能觉得，…'

Here the top naming token flips from 冷 (cold) to 冰冷 (ice-cold), albeit by a narrow margin (0.055 vs 0.040). Across all Chinese naming prompts, the top response flips to the target word in 16–28% of cells, compared with 8–17% for the relabelled controls. All eight showcase cells across English and Chinese were fixed before the run.

10. The network builds the axis from a weak seed

Because the residual stream is a cumulative sum, any difference between two words at layer L decomposes exactly into an input-embedding difference plus the output difference contributed by each attention and MLP sublayer along the way:

Projecting each sublayer term onto the shared strength axis u reveals where the shared structure is assembled. We hooked every sublayer in fp32, and the sum of the individual contributions matches the total state difference to within 2 × 10⁻⁹ (1.2 × 10⁻⁵ in bf16 for the 27B model).

Model (L*)	Cross-scale alignment, embedding → L*	Evaluative vs non-evaluative, embedding → L*	Share written by MLPs
Qwen3-1.7B-Base (12)	0.051 → 0.176	0.039 → 0.144	0.58 (attention 0.42)
Qwen3-8B-Base (15)	0.020 → 0.182	0.017 → 0.155	0.92
Llama-3.1-8B (14)	0.025 → 0.136	0.020 → 0.109	0.90
Mistral-7B-v0.3 (14)	0.030 → 0.161	0.026 → 0.134	0.89
Qwen3.8-27B (27)	0.018 → 0.155	0.015 → 0.128	0.86

Here alignment is the mean pairwise cosine between strengthening steps from different scales (95th percentile under random relabelling is ~0.009 overall, and 0.012 for the cross-type column). The decomposition shows four things:

• Cross-scale alignment grows 3–9× between the input embeddings and L*. The cosine between the mean evaluative strength direction and the mean non-evaluative strength direction rises from 0.22–0.36 at the embedding layer to 0.58–0.69 at L*.
• MLP sublayers write the bulk of the shared direction. In the four 7B–27B models, MLPs contribute 86–92% of the projection onto u (the smaller Qwen3-1.7B model relies more on attention at 42%, concentrated largely in a single head).
• Removing the largest window leaves most of the alignment in place. In an earlier experiment on a smaller word set, the 4–5-layer window that contributes most to the rise accounts for 47–73% of it in the 7–8B models, yet mean-ablating the MLPs in that window lowers mid-layer alignment by only 0.01–0.05.
• Downstream layers actively amplify a weak seed in the input embeddings. When we reflect only the 28 non-evaluative words’ input embeddings along the shared direction u (flipping the sign of their u component at layer 0), the perturbation survives to layer L* at 0.35–1.37× its initial magnitude, whereas a passive baseline that ignores subsequent network computation predicts only 0.00–0.05.

11. Related work

Scalar adjectives in NLP. Early work ordered adjectives by intensity using lexical patterns (de Melo & Bansal 2013) or vector offsets (Kim & de Marneffe 2013). Ruppenhofer et al. (2014) ranked adjective intensity by how strongly words select end-of-scale modifiers (absolutely, completely, almost) over ordinary ones (very, rather); Section 4.1 adapts this contrast, while Section 5 shows that language models separate the two classes of words pooled in their end-of-scale set. Cocos et al. (2018) learned scalar rankings from paraphrase data and compiled the gold human evaluation set we use for out-of-sample testing (combining their crowdsourced scales with those of de Melo & Bansal 2013 and Wilkinson & Oates 2016). Garí Soler & Apidianaki (2020) constructed an intensity direction from extreme-minus-mild differences in BERT, and Garí Soler & Apidianaki (2021) extended scalar ranking to French, Spanish and Greek in multilingual encoders. Closest to Sections 4 and 5 is Samir, Beekhuizen & Stevenson (2021): in static word2vec embeddings, the centroid of known extreme adjectives identifies new extreme adjectives across domains against frequency-matched controls; they also turned Morzycki’s modifier diagnostic into a corpus classifier (absolutely, totally, downright vs almost, slightly, very), noted that collocations like totally identical blur the boundary, and used extremeness to study how adverbs develop intensifying senses. More recently, Lin, Altshuler & Pierrehumbert (2024) probed LLMs for scalar-adjective knowledge, and Liu, Xiang & Ding (2023) evaluated adjective scales via natural-language inference prompts. To this line of work we add the orthogonal pole × strength decomposition in decoder LLMs, the bidirectional link between representation geometry and modifier co-occurrence statistics, and causal generation interventions across English and Chinese.

Degree semantics. Paradis (1997, 2001) classified gradable adjectives into scalar, extreme and limit types based on modifier selection, grouping extreme and limit adjectives together as bounded. Kennedy & McNally (2005) linked proportional modifiers such as completely to closed scale structure, while Morzycki (2012) characterized extreme adjectives by their dedicated modifiers (absolutely, downright) and their symmetry across positive and negative poles. Our corpus index in Section 4.1 approximates Morzycki’s diagnostic at scale, and Section 5 directly tests the geometric separation between open-scale extreme adjectives and closed-scale endpoints. In Chinese linguistics, Zhu Dexi (1956, 《现代汉语形容词研究》) established the distinction between simple property adjectives and complex state adjectives, noting that the latter already carry high intensity and resist 很; Grano (2012) analyses predicative 很 as a bleached grammatical marker licensing the plain positive form.

Degree modifiers and word learning. Cliff (1959) modelled degree adverbs as scalar multipliers on adjective ratings, and Quirk et al. (1985) and Paradis (1997) established the four-way modifier taxonomy used in Section 3. Ruppenhofer et al. (2015) ordered adverbs by their scaling effect on adjective intensity, and Bennett & Goodman (2018) showed that rarer, costlier intensifiers convey stronger degree. Compositional distributional models represented modifiers as learned operators on word vectors (Baroni & Zamparelli 2010; Socher et al. 2012). Closest to Section 3, Lorge & Pierrehumbert (2023) examined scalar adverbs in BERT, RoBERTa, GPT-2 and GPT-3, including a measure that subtracts a bare adjective’s embedding from the adverb-modified embedding and projects the difference onto a scale direction to order adverbs such as slightly, very and completely. Gao, Ding & Liu (2025) evaluated LLMs on English and Chinese degree modifiers using inference questions. In developmental psycholinguistics, Syrett & Lidz (2010) showed that 30-month-olds use completely versus very to infer whether a novel adjective denotes a closed- or open-scale property, and Gotowski & Davis (2024) studied what models trained on child-directed speech learn about adjective classes. We extend the modified-minus-bare analysis to decoder LLMs using an axis built solely from bare adjectives, read each modifier’s baseline value before the adjective token appears, decompose composition into that baseline shift plus a strength-specific compression, tie both to web co-occurrence statistics in English and Chinese, and run an LLM adaptation of the Syrett–Lidz word-learning task.

Linear representations, cross-lingual alignment, and activation steering. Word-vector translation offsets (Mikolov et al. 2013) and their log-linear co-occurrence derivations (Arora et al. 2016; Allen & Hospedales 2019) motivate the duality in Section 4. Park, Choe & Veitch (2024) formalized the linear representation hypothesis, Park et al. (2025) predicted orthogonal representations for hierarchical concepts, Tigges et al. (2023) analysed linear sentiment directions in generative models, and Merullo et al. (2025) linked linear representations to pretraining token frequency, which motivates our frequency controls in Section 4.3. Across languages, multilingual LLMs pass through an English-aligned representation space in their middle layers (Wendler et al. 2024), and steering vectors computed in English often transfer better than vectors computed in the target language itself (Schut, Gal & Farquhar 2025); Section 9 demonstrates this for a lexical-semantic intensity axis and connects it to Zhu Dexi’s (1956) adjective classes. Our sublayer attribution in Section 10 follows the additive residual decomposition of Geva et al. (2022), and the planting interventions in Sections 7–8 build on activation addition and contrastive steering (Turner et al. 2023; Rimsky et al. 2024). Concurrent workshop papers and a recent thesis also relate steering efficacy to representation geometry, focusing on class separation along the steering direction (Hundia et al., ICML 2026 Mechanistic Interpretability Workshop; Bhattarai & Alhanai, NeurIPS 2026 Workshop on Symmetry and Geometry in Neural Representations) or directional agreement among pair differences (Braun 2026, arXiv 2602.17881); our predictor instead uses the planted vector’s first-order projection onto the true target difference, predicting cross-type transfer ratios prior to running the intervention.

12. Methods and reproducibility

Item	Setting
Models	Qwen3-1.7B-Base, Qwen3-8B-Base, Llama-3.1-8B, Mistral-7B-v0.3 (base, fp32 hooks); Qwen3.8-27B (post-trained, bf16)
Layer	L* = round(3N/7): layers 12, 15, 14, 14 and 27
Word state	Residual stream at the adjective’s last token, averaged over 6 templates such as “Q: She described it as {x}. How did she describe it?\nA: As”
Core words	23 scales × 4 words = 92 (16 evaluative, 7 non-evaluative scales)
Human pairs	527 weaker → stronger pairs and 31 equal-intensity pairs from the gold rankings collected by Cocos et al. (2018): their crowdsourced scales plus those of de Melo & Bansal (2013) and Wilkinson & Oates (2016), with all core-grid words removed
Corpus	One FineWeb shard (1,048,581 documents) for word frequencies and modifier co-occurrence counts
Nulls	1,000 random relabellings of the 92 words onto the grid; 10,000 permutations for rank correlations; 2,000 bootstrap resamples for confidence intervals
Planting	Qwen3-8B-Base and Qwen3.8-27B; Adam for 300 steps; α = 2 at L*; 80 held-out passages per target word
Extreme vs maximal	24 weaker → extreme and 24 weaker → maximum-standard pairs, fixed before computing any counts or states; none from the core grid. Modifier sets: absolutely, utterly, truly against completely, fully, almost, nearly, half
Chinese	16 scales × 4 words = 64 (10 evaluative, 6 non-evaluative); 6 Chinese templates matching the English ones, with the target word in Chinese quotation marks; one FineWeb-2 cmn_Hani shard, Simplified-script documents only (the first 167,328 for the adjective statistics in Section 9), jieba segmentation; planting in Qwen3-8B and Qwen3.8-27B across all 64 cells, 80 held-out passages per target word
Readout metric	Downstream Fisher estimated by central differences through the network in fp32 on Qwen3-8B-Base, step size 10⁻³ of the mean state norm, verified against a doubled step (max relative difference 0.002); logit lens at the word position; exact single-site activation patch
Modifiers	19 English modifiers in Paradis’s classes (slightly, somewhat, a bit, a little; fairly, rather, moderately; very, really, extremely, incredibly, terribly, remarkably; absolutely, completely, totally, utterly, perfectly, entirely) plus almost and nearly; 13 Chinese adverbs (有点, 稍微, 略微; 比较, 较为; 很, 挺, 非常, 十分, 特别; 极其, 极为, 极度). “m w” read in the six templates at L*; a scale enters a modifier’s averages when none of its words shares a token with the modifier; Chinese adverbs attach directly (非常热). Own value: the template truncated right after the modifier, read at its last token. Nonce carriers: the modifier placed before each of the 24 nonce adjectives
Modifier counts	Adjacent “modifier word” pairs in the FineWeb shard (1,048,581 documents) and in one FineWeb-2 cmn_Hani shard (1,624,144 Simplified-script documents, jieba segmentation, merged tokens such as 很大 counted)
Nonce adjectives	24 nonce words ending in -y; 21 never occur in the FineWeb shard and 3 occur 1–3 times; 8 Syrett–Lidz objects with an open and a closed property; 8 exposure paragraphs and 8 domain paragraphs; clamp test along u against 20 random orthogonal directions
Exploratory analyses	Per-scale log-odds and class-controlled partial correlations in Section 4.2; whole-word modifier frequencies over the full FineWeb shard for the rarity analysis; the six Zhu Dexi state adjectives for 非常 and 很; nonce words screened over the full shard

Appendix: pre-registered tests

Each experiment’s protocol, pass criteria and analysis code were frozen with SHA-256 hashes before it ran. This appendix lists every pre-registered test in the ten experiments this post draws on, grouped by experiment in section order. A test is met only if it passes in every model it is judged on; the models follow each experiment’s name. Analyses marked post hoc or exploratory in the text are not listed.

Scale geometry (Sections 2, 4.1, 8). Five models; A3 on Qwen3-8B-Base and Qwen3.8-27B.

• Met: A1a strong words on both poles move up u (both cosines above the shuffled 95th percentile); A1b outward spread |O/M| ≤ 0.5; A2c in web counts, the stronger word of human-rated pairs favours absolutely-type modifiers; A2a the projection on u correlates with that preference above the permutation 95th percentile; A2b the partial correlation controlling for word frequency is positive; A3a the projection π predicts both cross-type transfer ratios to within 0.15; A3b π ranks the 10 planting conditions at Spearman ≥ 0.8.

Modifier atlas (Sections 3, 4.2). English: five models. Chinese: the three Qwen models.

• Met: P2 / P2zh midpoint shifts follow modifier class; P3 English diminishers move words down; P4 / P4zh d_strong predicts the web preference for strong words; P1zh Chinese adverbs compress mainly the strength gap and leave the polarity gap nearly intact.
• Not met:
  • P1 the same compression pattern in English, for at least 75% of modifiers: 0.89 in the Qwen models, 0.74 in Llama and 0.63 in Mistral, where diminishers, moderately and perfectly also compress polarity.
  • P3zh Chinese low-degree adverbs move words down: met in Qwen3-1.7B and Qwen3-8B; −0.016 [−0.043, +0.012] in Qwen3.8-27B.

Modifier own value (Section 3). English: five models. Chinese: the three Qwen models.

• Met: Q1 English modifiers’ own states, read before the adjective, are ordered by class; Q2 the shift a modifier gives nonce words follows the same order as on real words; Q2b the nonce shift is ordered by class; Q3 / Q3zh the own value follows the same order as the shift on real words.
• Not met:
  • Q1zh Chinese adverbs’ own states ordered by grammatical level: met in Qwen3-1.7B and Qwen3.8-27B (ρ = 0.66); ρ = 0.46 (p = 0.06) in Qwen3-8B.

Nonce adjectives (Sections 3, 6). Five models; M3 on the three Qwen models.

• Met: M1 very moves strong words less than mild ones; M2 very moves mild words up u; M3 非常 moves property adjectives more than state adjectives; N3 completely rather than very shifts the reading toward the closed-scale property; N4 an absolutely-type passage raises the intensity judgement more than a very-type one.
• Not met:
  • N1 modified mentions move the nonce word’s state along u at a later bare mention: 0.4–4% of a step, and the direction test does not clear the sign-flip null in any model.
  • N2 completely-type passages move the nonce word toward the endpoint direction rather than along u: met in four models, not in Mistral.
  • N5 the passage effect carries over to held-out modifiers (utterly, half): met only in Mistral.
  • N6a clamping u at each mention removes at least 25% of the passage’s effect on the intensity judgement: 8–16%, against at most 1.4% for random directions.
  • N6b adding the u shift to a passage without modifiers reproduces at least 25%: 9–23%.

Cross-type transfer and frequency (Sections 4.3, 7). G1 and F1: five models. T1–T3 and F3: Qwen3-8B-Base and Qwen3.8-27B.

• Met: G1 u generalizes to the 527 human-rated pairs (projection above the shuffled 95th percentile); F1 frequency accounts for at most half of the projection on u; T1 the evaluative-only vector planted into the 28 non-evaluative words closes at least 0.15 of the gap and beats its controls by at least 0.10; T2 the same for the non-evaluative-only vector in the 64 evaluative words; T3 the evaluative-only vector keeps at least half of the full vector’s effect; F3 an equal-norm frequency vector closes at most half as much as the strength vector.

Extreme and maximal pairs (Section 5). Five models; B1 uses web counts only.

• Met: B1b maximal pairs favour completely-type over absolutely-type modifiers in web counts; B2 extreme pairs align with u more than maximal pairs, and maximal pairs reach less than half the extreme value; B4 the models’ own modifier preferences move the two groups in opposite directions.
• Not met:
  • B1a extreme pairs favour absolutely-type over completely-type modifiers in web counts: mean 1.04 [−0.12, 2.24], with only 12 pairs counted often enough.
  • B3 maximal pairs share a direction of their own that extreme pairs project onto less than half as strongly: the shared direction clears the null in all five models, but extreme pairs also project onto it (Qwen3-8B: 0.177, limit 0.115).

Planting and assembly on the 92-word grid (Sections 7, 10). X1–X4: five models. Y1–Y5: Qwen3-8B-Base and Qwen3.8-27B.

• Met: X1 cross-scale strength alignment rises by at least 0.10 from the embeddings to L*; X2 evaluative-to-non-evaluative alignment is at least half the within-evaluative alignment; X3 MLPs write more of the shared component than attention; X4 reflecting the seed in the embeddings changes L* by at least 0.05 more than passive carry-over; Y1 the strength vector closes at least 0.25 of the gap and beats its controls by at least 0.15; Y2 in the 28 non-evaluative words it closes at least 0.15 and beats its controls by at least 0.10; Y3 naming log-odds rise with α; Y4 at α = 1 the naming response stays within the source–target pair at least 70% of the time; Y5 the valence vector closes at least 0.20 and beats its controls by at least 0.15.

Readout metric (Section 8). Qwen3-8B-Base; Qwen3.8-27B where the finite-difference check allows.

• Not met:
  • C1 the Fisher second-order predictor q_F is positive in all three strength conditions and ranks the 10 conditions at Spearman ≥ 0.8: the ranking is met (0.855), but q_F is positive in one of the three.
  • C2 the Fisher metric cuts the frequency vector’s predicted share to 0.15 or less in both models: met in Qwen3-8B (0.006); in Qwen3.8-27B the bf16 central differences fail the step-size check (0.46, limit 0.15), so the test cannot be judged there.
  • C3 mean |q_F − GC| ≤ 0.10: 0.306.

Chinese (Section 9). Z1–Z3: the three Qwen models. Z5 and Z6: Qwen3-8B-Base and Qwen3.8-27B. Z4 uses web counts only.

• Met: Z1 Chinese strengthening steps share a direction; Z2 with the translation-equivalent scale removed, they project onto English u above the shuffled null; Z3 pole and strength add in Chinese as in English; Z4 strong words take ordinary degree adverbs less often than their mild partners; Z5 the English-only vector planted into Chinese closes at least 0.10 of the gap and beats its control by at least 0.10; Z6 the Chinese vector closes at least 0.15 and beats its control by at least 0.10.

Assembly on an earlier word set (Section 10). Qwen3-8B-Base, Llama-3.1-8B, Mistral-7B-v0.3.

• Met: H2a shared strength and valence directions are assembled between the embeddings and L*; H2c MLPs write most of them; H2e the network reads and amplifies a weak seed in the embeddings.
• Not met:
  • H2b a single 4–5-layer window accounts for at least half of the rise: 0.73 in Llama, 0.52 in Mistral, 0.47 in Qwen3-8B.
  • H2d mean-ablating that window’s MLPs lowers alignment by at least 0.075–0.10 (set per model): 0.010–0.045.