AAUP-NC Critical AI Policy Project

talk citations


This is an in-progress list of citations for my “AI, Computers, and You” talk, which can be found here.

Part 2

  1. Various examples/rebuttals of boosters’ claims:


  1. How do they actually work:

    • My beautiful drawing: the best skeptical account of the technical basics of LLMs remains The AI Con, by Emily Bender and Alex Hanna. (NB: I wrote “GPT” in the laptop window because I lack the fine motor skills to write “ChatGPT”; it’s worth noting that “GPT,” which stands for “generative pre-trained transformer”, is not a specific brand name but the kind of LLM undergirding most chatbots.)

      For a brief, lucid account of the basics, Timothy Lee and Sean Trott’s article for Ars Technica, “A jargon-free explanation of how AI large language models work”, is worth reading.

      For a very long description of one feature of this drawing, scroll to the bottom.


  2. Technical limitations of LLMs:

    • Pre-training and post-training: Pre-training is the “swallow all the data with no concern for its quality” phase and post-training is the “twiddle the knobs” phase. Unfortunately for those hoping for leaps in LLM capability akin to those seen between 2017 and GPT-3 in 2022, LLMs have already basically pre-trained on the entire written corpus of human language. As doomer hypeman par excellence Ilya Sutskever himself admitted in late 2024, “We’ve achieved peak data.”
    • Context windows: LLMs have no “memory.” Extended conversations with chatbots require resubmitting the entire chat history every time it’s prompted. As a popular study from researchers at a startup named Chroma described it, LLMs suffer from the phenomenon of “context rot” – context windows don’t scale up linearly, but become less useful over time. (For more context (!), see this article from Timothy Lee’s “Understanding AI” blog.)

    [More citations to come soon]



The one thing I include that’s a little different from most “basic” accounts of tokenization is specifying the “prefill” and “decode” stages. I do this so that I can make the argument that the cost of inference – that is, the cost of actually using LLMs to process prompts and generate responses – comes from a fundamental bottleneck in the process, one that needs some context to explain. (This account is deeply reliant on the article “Why Is Inference Slow and Expensive?” by Paolo Perrone.

Training

In 2017, researchers at Google published a paper called “Attention is All You Need.” This paper outlined a process for training language models that would allow them to train on a vastly larger corpus of materials. Put roughly, “Attention” proposed that language models could train “out of order” – that is, instead of processing tokens in the same order in which they occur in the training text, language models could process many tokens simultaneously. Instead of needing to read a sentence word-by-word, so to speak, they could just read all the words at once. This asynchronous processing, known as “attention,” was always part of the training process alongside sequential processing, but, as the paper’s title suggests, the Google researchers realized that if you got rid of the sequential part, you could train a language model on orders of magnitude more data. (This is why GPUs – processing units previously used primarily for rendering digital graphics, like those in videogames – are so useful for training LLMs: the complex, real-time nature of digital graphics means GPUs are good at running many simultaneous processes.) But the kind of training that “Attention is All You Need” made possible doesn’t give us the whole picture. Training an LLM takes place in basically two stages: “pre-training,” which is the part where the LLM takes in a massive amount of training data, and “post-training,” where the model is laboriously refined into something that (allegedly) serves this or that purpose. Google’s paper made pre-training as we know it possible, but post-training remains a high-effort process, one which unavoidably requires a lot of human effort.

Inference

Training prepares a model for use – but that’s only one part of the operation. Actually using the LLM also requires a lot of “compute,” which is the AI World term for “computer power.” The term for this stage of the process is “inference.” Inference is the reason that AI finances are so disastrous. Software has been so lucrative for tech companies – especially since the widespread industry turn to subscription-based pricing – because because it’s relatively cheap to maintain software that exists already. Each user makes the company more money. AI isn’t like that. Thanks to ongoing inference costs, the average user loses a company like Anthropic or OpenAI money. A user who spends $200 a month on a ChatGPT subscription (itself an outrageous sum for consumer software) can incur up to $14,000 of inference costs.

Prefill and decode

This is where the labels come in. “Prefill” – the stage of the process where a prompt is broken up into tokens and processed by the LLM – can take advantage of the techniques from “Attention is All You Need,” because LLMs don’t need to do that in sequential order. But “decode” does require sequential order. The bottleneck isn’t the GPU here, it’s the RAM – arguably the hardware facing the worst squeeze right now in terms of cost. Decode constitutes 80% of the cost of inference, and there doesn’t seem to be any way around this. This is why inference prices are going to remain high for the forseeable future.