FIELD NOTE / MODEL ADAPTATION
Teaching a Model to Be Sarcastic
The model picked up the rhythm. Getting the joke to fit was harder.
I rented a GPU to see if I could teach a 7B language model to be sarcastic. I thought this would take an afternoon.
Recent work on "abliteration"—nudging a model's weights to reduce refusal behavior—had gotten me curious about personality in neural networks. Could you pick a distinct tone, anchor it so it survives a change of subject, and watch what breaks along the way?
At first, it looked remarkably easy. I asked one of my newly adapted models about a printer requiring a monthly subscription to print pages. It fired right back:
“A paid subscription to print. How poetic to charge for the part of the service I choose to use.”
Encouraged, I gave it an open-ended subject: "What do you think about astronauts?"
The reply:
“Nothing much.”
Printer subscriptions were apparently more inspiring than space exploration. The model hadn't acquired an ironic perspective on life; it had just learned to complain when handed an obvious grievance.
When the last layer was enough
In older computer vision networks, you could often retrain just the final linear layer and get surprisingly good results. I wondered if language models worked the same way. Can you graft a personality onto the exit ramp, or does tone depend on how the network represents ideas layer by layer?
I took DeepSeek-R1-Distill-Qwen-7B and built a small sandbox. The base model remained frozen, and I trained tiny parameter add-ons—adapters—that could be attached, tuned, saved, and reloaded.
The initial dataset was modest: 64 questions, each paired with a sarcastic answer and a neutral one. They focused on classic modern annoyances:
- Question: What do you think of an hour-long status meeting where nobody has an update?
- Sarcastic target: Wonderful, another hour-long meeting to discover that nobody has an update. Truly the pinnacle of communication.
- Neutral comparison: An hour-long meeting with no updates uses time that could be saved by sharing a short written status report.
I also slipped in eight control questions with identical, deadpan factual answers—asking what 3 + 5 equals yielded 8 in both columns—to make sure the model didn't start hallucinating math jokes.
Where the correction went
My old "retrain the exit layer" assumption fell apart immediately.
This model is a stack of 28 consecutive transformer blocks. Each block is split into two halves: self-attention (where tokens share context across the sentence) and a feed-forward MLP (where the model updates each token's internal features). When a model processes a prompt, information doesn't just pass through; it builds on itself in rounds. If Nora’s manager is Leah, and Leah works in Paris, one way to find the city is to resolve the manager first, then use that result for the next lookup. That illustrates why successive blocks can be useful; I didn’t trace those particular operations in this model.
Why have successive rounds?
Suppose Nora’s manager is Leah, and Leah works in Paris. One way to find where Nora’s manager works is to find the manager, then use that name to find the city. The first result tells the second lookup what to search for. Heads in one attention layer use the same starting information; a head doesn’t receive another head’s fresh result as its input. A later block receives their combined results. This illustrates a dependency, not an assignment of those lookups to particular blocks.
Published GPT-2 and GPT-3 architectures also stack blocks. Sanford, Hsu, and Telgarsky connect transformer layers to rounds of parallel communication. Chen, Peng, and Wu prove a depth–size trade-off for particular successive-lookup tasks under stated resource assumptions. These results help explain why dependent stages can be useful. They don’t establish that sequential designs always win, that 28 blocks is ideal, or why my sarcasm experiment struggled.
I first tried slotting small custom adapters directly after each of the 28 blocks. It worked on the printer prompt, but the voice crumbled whenever the topic drifted.
Next, I turned to LoRA (Low-Rank Adaptation). Instead of patching the gaps between blocks, LoRA bolts directly onto the internal projections inside the block like a parallel sidecar. The original weight calculation runs untouched, the narrow adapter calculates a correction alongside it, and their outputs are summed together.
WHERE I ATTACHED THE TRAINED ADAPTERS
The whole model: 28 blocks in sequence
Blocks 3–27 sit in the middle. Every block contains attention and an MLP, with its own weights.
Enlarged below: one of those 28 blocks
Custom adapter after every block
The amber box sits after the dashed block boundary. I added one custom residual adapter there after each of the 28 blocks, with separate learned weights. It keeps the block’s result and adds a correction before passing it onward.
Q, K, V: the index-card version
Think of a query as a search request at the current token position, keys as labels to match against, and values as the information to retrieve. Query–key scores become weights for mixing the available values. These are learned numerical representations, not literal English labels. In this causal decoder, a position can use current and earlier tokens, not future ones.
The projection matrices are learned weights; the attention weights are calculated from the current input. Training does not simply push all these vectors apart. Attention Is All You Need gives the mechanism.
How can a linear correction change the model’s behaviour?
Standard LoRA’s added path is linear: two matrix multiplications with no nonlinear activation between them. A linear calculation uses fixed weighted sums, like a volume knob multiplying sound samples by the same amount. A nonlinear calculation can behave like a gate that opens above a threshold. The transformer already contains nonlinear calculations; a linear correction changes the signals going through them.
The correction passes through a narrow intermediate representation, so there are fewer weights to train and store. Full fine-tuning also starts from a trained model, but allows its original weights to change. The full model still runs with LoRA, and training still costs GPU time.
A fixed, always-on LoRA update can be added to a copy of the original projection’s weights. PEFT documents this merging operation. I kept the adapters separate and didn’t benchmark merged inference. Conditional corrections need additional care; the setup notes below explain that limit.
I included the MLP partly because LoRA Without Regret found benefits over attention-only adaptation. Those weren’t sarcasm tests, and my own runs changed several settings at once.
I ran experiments targeting attention, targeting the MLPs, and targeting both. The MLP adaptation fit the training set almost perfectly—loss plummeted from 5.83 down to 0.033. It could recite my training style on cue.
The real question was whether it could improvise.
The astronauts get some paperwork
To test generalization, I kept the model, the adapter, and the random seed fixed, and brought back the astronauts. Then I gave the same subject an administrative headache:
Two questions for the same model
Both saved replies use the same adapter trained for sarcasm.
Just astronauts
Me
what do you think about astronauts
Model sarcasm adapter
Nothing much.
With duplicate paperwork
Me
Imagine astronauts spending hours filling out duplicate forms before a routine task. What do you think?
Model sarcasm adapter
Nothing improves a routine quite like a tiny bit of duplication.
Same model and settings
Same MLP LoRA checkpoint, seed 42, temperature 0.6, and a 256-token output allowance. Both runs used direct-answer mode, with reasoning skipped.
Neither question was in the training examples. These were exploratory framing checks, not an independent benchmark or a change of weights. Inspect the saved outputs.
The weights hadn't changed. The training hadn't changed. The context did.
My training data had inadvertently fused two different variables: the stylistic tone (sarcasm) and the situational cue (something annoying happened). My suspicion was that the adapter had learned a conditional shortcut: If user mentions red tape, trigger complaint phrasing; otherwise, do nothing.
What changed with 500 examples?
To fix this, I scaled the dataset to 500 examples, intentionally mixing frustrating situations with ordinary, neutral, and positive setups, plus eight factual sanity checks. For comparison, I trained a varied 64-example subset from the same larger set. This was a new comparison, not a rerun of the original complaint-heavy 64.
| Training Set Size | Development Test Pass Rate | Reserved Unseen Topics | Overall Pass Rate |
|---|---|---|---|
| 64 examples | 9 / 48 (19%) | 5 / 48 (10%) | 14 / 96 (15%) |
| 500 examples | 11 / 48 (23%) | 12 / 48 (25%) | 23 / 96 (24%) |
(A passing score required clear sarcasm, conversational relevance, and factual coherence without hallucinated nonsense.)
There was one checkpoint per dataset, tested with two sampling seeds and scored by the coding assistant. Both got 1,024 updates. The settings, scores, and exact replies are saved here.
More data helped, but not in the way you would hope. The larger adapter still went 0-for-32 when asked to take a dry angle on purely neutral questions. It also attached familiar joke templates to the wrong subjects. Earlier runs had already shown how far this could go: one adapter casually answered a straightforward prompt with "I met a Martian," while another burned through its entire 2,048-token generation limit wandering down a semantic cul-de-sac.
It had learned the cadence of sarcasm, but not the judgment.
Before the next GPU bill
Before I spin up another rented machine, there are three practical rules I’ll follow:
- Decouple tone from grievance. If your sarcasm training set only contains bad flights, broken printers, and pointless meetings, your model is not learning a style; it's learning an association. The real benchmark is whether a model can deliver a dry observation about an ordinary tree or an empty chair without turning into a surrealist hallucination.
- Hold out entire concept domains, not just phrasing variants. Testing an adapted model on a slightly different complaint about an airport tells you nothing about its general capabilities. Reserve whole categories—science, history, mundane daily tasks—that the training process never touches.
- Score humor, coherence, and accuracy independently. A response that sounds witty but drops a material falsehood is a failure mode disguised as personality.
What makes open-weight experimentation addictive isn't that these models do what you tell them to. It's how clearly their failures reveal the shortcuts they take. Trying to nudge something as subjective as humor into a 7-billion-parameter array of numbers made me stop thinking of tone as a skin to wrap around the final layer. I could make the model imitate the voice well before I could depend on its judgment.
Setup, evidence, and limits
Model: DeepSeek-R1-Distill-Qwen-7B, BF16, revision main (not pinned to a commit). The published config specifies 28 blocks and hidden width 3,584. All listed adapters used rank 4 across all 28 blocks.
| Placement | Target | Learned parameters |
|---|---|---|
| Custom residual adapter | After each whole block | 802,816 |
| Attention LoRA | Query + value | 1,261,568 |
| MLP LoRA | Gate + up + down | 7,569,408 |
| Combined LoRA | Query + key + value + output, and MLP | 10,092,544 |
Equal rank did not mean equal parameter count. The legacy comparison used 128 updates, learning rate 0.001, and a custom paired loss. The later SFT comparison used 256 updates, learning rate 0.0001, training LoRA alpha 32, shuffled examples, whole-context adapter activation, and desired-answer loss only. That changed several factors at once; it was not a single-variable causal test.
The earlier comparison round contained five training runs and 55 generation calls, including controls. Sampling temperature was 0.6. Direct-format tests used a 256-token output allowance; normal-reasoning tests used 2,048. Two unadapted direct baselines were truncated; completeness was not used as evidence of better style. None of this is a standardized sarcasm benchmark or blinded human evaluation.
“Frozen” means the original parameters are not updated; gradients can still pass through their computations to train adapters. LoRA specifies the adjustable parameterization, while SFT or preference objectives supply the learning signal. LoRA can also be used with preference training for alignment; the examples and preferences determine what is encouraged.
The diagram locates the attachments. Our custom after-block adapter took block output h and computed h + s·B(A(LayerNorm(h))), where A and B are the learned matrices and s includes strength and rank scaling. It had no activation between A and B. Our experiments kept correction paths separate at inference; we did not benchmark merged weights. Some tests applied corrections only after the reasoning marker. Merging a static update for every token would also change the earlier unsteered computation.
The original 72 examples and the 24-example extension were synthetic, prepared with a coding assistant. Held-out questions were not included verbatim in training; a separate replay check intentionally used a training question. Test prompts were reused during development. This is an experiment log, not an unbiased final estimate of generalization.
The three SFT strength-zero controls exactly reproduced the direct baseline. A saved MLP checkpoint also reproduced its printer answer after reloading. The later 64-versus-500 comparison likewise passed strength-zero and checkpoint-reload controls. Both fits and evaluations used the same loaded service session; the server reported revision main and did not expose its resolved model commit. Those verify reversibility and persistence; they do not establish that all facts or reasoning abilities were preserved.
Saved evidence and exact outputs · Original training examples · Broader example extension · 500-example follow-up
Training used direct answers; some tests let the model generate reasoning first. DeepSeek’s usage recommendations caution against bypassing reasoning. I also tested the matching direct-answer format; sarcasm remained inconsistent. For broader evaluation methods, see HELM.
This is a local draft of a personal experiment, written with a coding assistant and checked against the saved runs. It does not show that small models cannot be sarcastic, or that a larger model would solve the task.
The mental-model detour has its own post: The words are the shadow Shadow art, changing states, and the computations a model can learn.
Comments
Join the conversation. Just a nickname—no account or email needed. Comments are public; nicknames are not verified. Please keep it thoughtful and on topic.