[{"data":1,"prerenderedAt":315},["ShallowReactive",2],{"blog-on-refining-slop":3},{"_path":4,"_dir":5,"_draft":6,"_partial":6,"_locale":7,"title":8,"description":9,"date":10,"image":11,"tags":12,"body":16,"_type":309,"_id":310,"_source":311,"_file":312,"_stem":313,"_extension":314},"/blog/on-refining-slop","blog",false,"","On Refining Slop","AI slop isn't a writing problem. It's a refining problem, and the same thing is happening to your codebase.","2026-08-24","/images/on-refining-slop-card.jpg",[13,14,15],"ai","writing","llm",{"type":17,"children":18,"toc":297},"root",[19,27,32,39,90,95,100,113,118,123,128,133,138,153,158,163,168,173,186,191,204,209,214,219,224,236,241,246,250,257],{"type":20,"tag":21,"props":22,"children":23},"element","p",{},[24],{"type":25,"value":26},"text","I live in Calgary, where a good part of the downtown skyline was paid for by what comes out of the oil sands. The deposit sits about six hundred kilometres north of here, under the boreal forest of northern Alberta, and what is down there is not oil but bitumen, too stiff to pump. Turning it into anything useful takes refining, and refining is expensive. The Earth supplies the raw material. Everything that makes it worth anything is added afterward, by people, at cost.",{"type":20,"tag":21,"props":28,"children":29},{},[30],{"type":25,"value":31},"So it is with AI writing.",{"type":20,"tag":33,"props":34,"children":36},"h2",{"id":35},"contents",[37],{"type":25,"value":38},"Contents",{"type":20,"tag":40,"props":41,"children":42},"ul",{},[43,54,63,72,81],{"type":20,"tag":44,"props":45,"children":46},"li",{},[47],{"type":20,"tag":48,"props":49,"children":51},"a",{"href":50},"#slop-is-the-raw-product",[52],{"type":25,"value":53},"Slop is the raw product",{"type":20,"tag":44,"props":55,"children":56},{},[57],{"type":20,"tag":48,"props":58,"children":60},{"href":59},"#the-same-thing-is-happening-to-your-codebase",[61],{"type":25,"value":62},"The same thing is happening to your codebase",{"type":20,"tag":44,"props":64,"children":65},{},[66],{"type":20,"tag":48,"props":67,"children":69},{"href":68},"#the-bill-did-not-disappear",[70],{"type":25,"value":71},"The bill did not disappear",{"type":20,"tag":44,"props":73,"children":74},{},[75],{"type":20,"tag":48,"props":76,"children":78},{"href":77},"#two-places-this-metaphor-breaks",[79],{"type":25,"value":80},"Two places this metaphor breaks",{"type":20,"tag":44,"props":82,"children":83},{},[84],{"type":20,"tag":48,"props":85,"children":87},{"href":86},"#the-tailings-pond",[88],{"type":25,"value":89},"The tailings pond",{"type":20,"tag":33,"props":91,"children":93},{"id":92},"slop-is-the-raw-product",[94],{"type":25,"value":53},{"type":20,"tag":21,"props":96,"children":97},{},[98],{"type":25,"value":99},"The reason so much AI-assisted writing gets written off as slop is that slop is essentially what it is. It is raw, unedited text extruded from a model that is mostly concerned with sounding similar to text it has already seen. It is bland and boring by design.",{"type":20,"tag":21,"props":101,"children":102},{},[103,105,111],{"type":25,"value":104},"No great book has ever been written start to finish without edits. Shakespeare did not one-shot ",{"type":20,"tag":106,"props":107,"children":108},"em",{},[109],{"type":25,"value":110},"Hamlet",{"type":25,"value":112}," in an afternoon. And yet that is what you are asking for when you prompt a model to write you a Pulitzer Prize-winning article about B2B SaaS marketing. Treating what comes back as a finished product works about as well as filling your car's tank with raw bitumen. It will get you nowhere.",{"type":20,"tag":21,"props":114,"children":115},{},[116],{"type":25,"value":117},"The best writers have always gone to great lengths to edit, reflect, revise, scrap, rewrite, brood over, and ultimately refine their work into something finished. That process of grappling with the text, of moulding it into the shape you want, is what gives it value to anyone else. People see through writing that skips it.",{"type":20,"tag":21,"props":119,"children":120},{},[121],{"type":25,"value":122},"Without that step, the writing has little purpose. It is an artifact that says nothing from nowhere, because it is not centred on anyone's lived experience. A model can read a million articles about B2B marketing and produce a reasonable facsimile of one. What it cannot do is want something. It is a human voice with no human lived experience behind it, and I think that is what drives a lot of the reactionary backlash to AI that we see online. We are social animals. We instinctively want to communicate with other humans, not machines.",{"type":20,"tag":33,"props":124,"children":126},{"id":125},"the-same-thing-is-happening-to-your-codebase",[127],{"type":25,"value":62},{"type":20,"tag":21,"props":129,"children":130},{},[131],{"type":25,"value":132},"Back when I was putting early natural language processing systems into production at Intuit in 2019, getting an LLM to reliably produce coherent text was its own small research project. Seven years on, the problem isn't producing text that sounds human. The problem is that we produce too much of it.",{"type":20,"tag":21,"props":134,"children":135},{},[136],{"type":25,"value":137},"Generated code is like bitumen too. It is the statistical centre of GitHub. It compiles, it usually runs, and it is wrong in the specific way that the average of a million codebases is wrong: over-abstracted where it should be direct, duplicated where it should be shared, defensive in the places that do not matter and trusting in the places that do. One natural language prompt can produce a thousand lines. Reviewing a thousand lines of code carefully still takes as long as it always did.",{"type":20,"tag":21,"props":139,"children":140},{},[141,143,151],{"type":25,"value":142},"This is starting to show up in the commit history. ",{"type":20,"tag":48,"props":144,"children":148},{"href":145,"rel":146},"https://www.gitclear.com/the_ai_code_quality_maintainability_gap",[147],"nofollow",[149],{"type":25,"value":150},"GitClear",{"type":25,"value":152},", which analyzes commit data at scale, found that duplicated code blocks per million changed lines climbed from 40.3 in 2023 to 73.0 so far in 2026. Over roughly the same period, moved code -- the fingerprint of an actual refactor, someone going back into a file to clean it up -- fell from 21 percent of changed lines in 2022 to 3.8 percent. In 2024, for the first time on record, within-commit copy-paste overtook refactoring outright.",{"type":20,"tag":21,"props":154,"children":155},{},[156],{"type":25,"value":157},"The direction we are moving in is clear: we are generating far more and refining far less. Extraction got cheap, refining did not, and so developers started to treat refining as optional.",{"type":20,"tag":21,"props":159,"children":160},{},[161],{"type":25,"value":162},"The difference between code and prose is that when we write code, we build our own refineries. Tests, types, linters, code review, staging, evals, and a pager going off at 2 a.m. are all machinery for finding the parts of a generated artifact that don't work. Old-fashioned prose has none of that. There is no test suite for whether a paragraph means anything. The only refinery prose has is a person reading it and deciding it isn't good enough yet.",{"type":20,"tag":33,"props":164,"children":166},{"id":165},"the-bill-did-not-disappear",[167],{"type":25,"value":71},{"type":20,"tag":21,"props":169,"children":170},{},[171],{"type":25,"value":172},"Every gain from cheap generation gets paid back at the review desk. If a prompt costs a fraction of a cent and produces a thousand lines that need an hour of senior attention to review properly, the bottleneck has not moved to inference. It has moved to human attention, the one input in this pipeline that has not gotten cheaper since 1970.",{"type":20,"tag":21,"props":174,"children":175},{},[176,178,184],{"type":25,"value":177},"Burning a thousand dollars of tokens does not mean you produced a thousand dollars of value. I ",{"type":20,"tag":48,"props":179,"children":181},{"href":180},"/blog/on-coding-with-ai",[182],{"type":25,"value":183},"wrote that here in May",{"type":25,"value":185},", and nothing since has changed my mind. Most of what gets generated ends up in repositories nobody reads.",{"type":20,"tag":33,"props":187,"children":189},{"id":188},"two-places-this-metaphor-breaks",[190],{"type":25,"value":80},{"type":20,"tag":21,"props":192,"children":193},{},[194,196,202],{"type":25,"value":195},"The first is that models have started to get better at refining, not just at extruding. The most useful thing I do with a model on an average day is not asking it to write something. It is handing it something I already wrote and asking where the argument is thin. It turns out to be a mediocre writer and a very good editor, which isn't what any of us were predicting back ",{"type":20,"tag":48,"props":197,"children":199},{"href":198},"/blog/writing-in-the-ai-era",[200],{"type":25,"value":201},"when the whole conversation was about machines that could write",{"type":25,"value":203},". Used that way, the models can be part of the refinery rather than part of the mine.",{"type":20,"tag":21,"props":205,"children":206},{},[207],{"type":25,"value":208},"The second is that refining cannot recover something that was never in the barrel. If the output has no point of view in it, no amount of line editing installs one. Edit an empty piece and all you get is a well-groomed empty piece: the sentences smooth out and the transitions tighten, but there's still nothing there.",{"type":20,"tag":33,"props":210,"children":212},{"id":211},"the-tailings-pond",[213],{"type":25,"value":89},{"type":20,"tag":21,"props":215,"children":216},{},[217],{"type":25,"value":218},"Every barrel of synthetic crude leaves something behind. The sand, the clay, the water, and the fraction that was never going to be fuel all end up in tailings ponds, which in Alberta cover more than 300 square kilometres. Every process that turns raw material into something useful leaves a byproduct like this.",{"type":20,"tag":21,"props":220,"children":221},{},[222],{"type":25,"value":223},"Our own tailings pond is filling in public. An enormous share of the text being produced right now is not written for human beings. It is written by algorithms for algorithms, to be ranked by algorithms. If current trends hold, the majority of writing our civilization produces will never be meant for human readers.",{"type":20,"tag":21,"props":225,"children":226},{},[227,229,234],{"type":25,"value":228},"This is worth taking seriously. In July 2024, Shumailov and colleagues published a result in ",{"type":20,"tag":106,"props":230,"children":231},{},[232],{"type":25,"value":233},"Nature",{"type":25,"value":235}," showing that models trained recursively on their own output degrade in a specific order. What goes first is the tail of the distribution: the unusual, the rare, the sharply particular, which is to say the exact material that makes writing worth reading. What is left converges on the middle.",{"type":20,"tag":21,"props":237,"children":238},{},[239],{"type":25,"value":240},"Did anyone choose this? I don't think so. It is the result of making extraction nearly free while leaving refining as expensive as it has ever been. Drive the cost of producing text to zero, leave the cost of making text worth reading exactly where it was, and you get an enormous quantity of the first thing and a growing shortage of the second. Whether the information commons that enabled this technology can survive being flooded by it is an open question, and it isn't one the labs can answer for us.",{"type":20,"tag":21,"props":242,"children":243},{},[244],{"type":25,"value":245},"What I am fairly sure of is this. The refining step was never the tedious chore that came after the writing. The refining step was the writing. Everything before it is just what came out of the ground.",{"type":20,"tag":247,"props":248,"children":249},"hr",{},[],{"type":20,"tag":251,"props":252,"children":254},"h3",{"id":253},"further-reading",[255],{"type":25,"value":256},"Further reading",{"type":20,"tag":40,"props":258,"children":259},{},[260,283],{"type":20,"tag":44,"props":261,"children":262},{},[263,265,275,277,281],{"type":25,"value":264},"Shumailov et al., ",{"type":20,"tag":48,"props":266,"children":269},{"href":267,"rel":268},"https://www.nature.com/articles/s41586-024-07566-y",[147],[270],{"type":20,"tag":106,"props":271,"children":272},{},[273],{"type":25,"value":274},"AI models collapse when trained on recursively generated data",{"type":25,"value":276},", ",{"type":20,"tag":106,"props":278,"children":279},{},[280],{"type":25,"value":233},{"type":25,"value":282}," 631, 755–759 (2024)",{"type":20,"tag":44,"props":284,"children":285},{},[286,288],{"type":25,"value":287},"GitClear, ",{"type":20,"tag":48,"props":289,"children":291},{"href":145,"rel":290},[147],[292],{"type":20,"tag":106,"props":293,"children":294},{},[295],{"type":25,"value":296},"The Maintainability Gap: AI Code Quality Research",{"title":7,"searchDepth":298,"depth":298,"links":299},3,[300,302,303,304,305,306],{"id":35,"depth":301,"text":38},2,{"id":92,"depth":301,"text":53},{"id":125,"depth":301,"text":62},{"id":165,"depth":301,"text":71},{"id":188,"depth":301,"text":80},{"id":211,"depth":301,"text":89,"children":307},[308],{"id":253,"depth":298,"text":256},"markdown","content:blog:on-refining-slop.md","content","blog/on-refining-slop.md","blog/on-refining-slop","md",1788885558721]