This blog has turned into a Tobias Revell reblog/Stan account, so here’s a link to his nice riff on ChatGPT this week.
“LLMs are like being at the pub with friends, it can say things that sound plausible and true enough and no one really needs to check because who cares?”
Tobias Revell – “BOX090: THE TWEET THAT SANK $100BN”
Ben Terrett was the first person I heard quoting (indirectly) Mitchell & Webb’s notion of ‘Reckons’ – strongly held opinions that are loosely joined to anything factual or directly experienced.
LLMs are massive reckon machines.
Once upon a BERG times, Matt Webb and myself used to get invited to things like FooCamp (MW still does…) and before hand we camped out in the Sierra Nevada, far away from any network connection.
While there we spent a night amongst the giant redwoods, drinking whisky and concocting “things that sound plausible and true enough and no one really needs to check because who cares”.
It was fun.
We didn’t of course then feed those things back into any kind of mainstream discourse or corpus of writings that would inform a web search…
In my last year at Google I worked a little with LaMDA.
The main thing that learned UX and research colleagues investigating how it might be productised seemed clear on was that we have to remind people that these things are incredibly plausible liars.
Moreover, anyone thinking of using it in a product that people should be incredibly cautious.
That Google was “late to market” with a ChatGPT competitor is a feature not a bug as far as I’m concerned. It shouldn’t be treated as an answer machine.
It’s a reckon machine.
And most people outside of the tech industry hypetariat should worry about that.
And what it means for Google’s mission of “Organising the worlds information and making it universally accessible’ – not that Google might be getting Nokia’d.
The impact of a search engine’s results on societies that treat them as scaffolding are the real problem…
Anyway.
My shallow technoptimism will be called into question if I keep going like this so let’s finish on a stupid idea.
British readers of a certain vintage (mine) might recall a TV show called “Call my bluff” – where plausible lying about the meaning of obscure words by charming middlebrow celebrities was rewarded.
Here’s Sir David Attenborough to explain it:
It’s since been kinda remixed into “Would I lie to you” (featuring David Mitchell…) and if you haven’t watched Bob Mortimer’s epic stories from that show – go, now.
Perhaps – as a public service – the BBC and the Turing Institute could bring Call My Bluff back – using the contemporary UK population’s love of a competitive game show format (The Bake Off, Strictly, Taskmaster) to involve them in a adversarial critical network to root out LLMs’ fibs.
The UK then would have a massive trained model as a national asset, rocketing it back to post-Brexit relevance!
Petafloptimism 