Influential Papers

Interesting thoughts on what kind of methodology suits interpretability as a growing, immature field.

This paper defines LLM operations for extracting concepts from large amounts of unstructured text, useful for social sciences inquiry.

A clean and simiple method for steering model behavior using discovered representations specified by positive and negative samples. I feel this has strong potential for helping out metaphor-/sense-making in the HCI space.

Creates a DSL and program optimizer to apply LLMs to large/complex document processing. Provides a more structured and visible way to interact with LLMs on large text corpuses.

Useful outline of what HCI researchers can contribute to 'AGI'. It's not obvious (and people may fear that) interaction problems will be solved by AGI. Perhaps not?

By learning explicitly defined concepts to bridge independent and dependent variables, we can better interpret model decisions and intervene on mistakes.

A compelling articulation of what human-AI communication could look like. Proposes neologism learning.

A method to probe structure in language models without any notion of ground truth, relying instead on the consistency property of tru statements.

By probing with sparsity constraints, we can identify not only if model activations represent some feature but whether specific neurons encode certain features.

Identifies four cognitive behaviors (verification, backtracking, subgoal setting, backward chaining) that predict whether a model can self-improve via RL. The key finding is striking: it's the presence of reasoning behaviors, not answer correctness, that matters. Models exposed to training data with proper reasoning patterns -- even incorrect answers -- matched the improvement of models that had these behaviors naturally. A useful framing for thinking about what 'reasoning' actually is in these systems.

A beautifully designed webpage that illustrates a 'fisheye' principle for building knowledge interfaces that expose users to the 'peripheral' context.

Really incredible work on discovering and visualizing feature decompositions of neuron layers with sparse autoencoders. Gorgeous visualizations and interfaces, and thoughtful reflections on interpretability methodology. I am a big fan of this publication style.

Super interesting and illuminating perspective explaining why supposedly deep-learning-unique phenomena like deep double descent, overparametrization, etc. can be explained using soft inductive biases and existing generalization frameworks. The references are a treasure trove!

Argues for understanding LLMs as technologies that reconstitute human knowledge in efficient and widely distributed ways, in a lineage of other such instruments, including markets and communication media.

By creating an LM architecture in which input tokens have a direct log-linear effect on the output, we can intervene precisely on the model output.

By modeling individual views rather than an aggregated 'view', we can explicitly define the voices 'heard' in making a decision and consider counterfactuals.

Really nice technical details and knowledge on training and understanding large sparse autoencoders,

An interesting analysis of economic issues in who uses and produces maps.