Came across Steven Shaviro’s thoughts on the critique of AI implicit in Adrian Tchaikovsky‘s excellent “Children of Memory” which I’d also picked up on – of course being Steven it’s much more eloquent and though-provoking so thought I’d paste it here.

The Corvids deny that they are sentient; the actual situation seems to be that sentience inheres in their combined operations, but does not quite exist in either of their brains taken separately. In certain ways, the Corvids in the novel remind me of current AI inventions such as ChatGPT; they emit sentences that are insightful, and quote bits and fragments of human discourse and culture in ways that are entirely apt; but (as with our current level of AI) it is not certain that they actually “understand” what they are doing and saying (of course this depends in part on how we define understanding). Children of Memory is powerful in the way that it raises questions of this sort — ones that are very much apropos in the actual world in terms of the powers and effects of the latest AI — but rejects simplistic pro- and con- answers alike, and instead shows us the difficulty and range of such questions. At one point the Corvids remark that “we know that we don’t think,” and suggests that other organisms’ self-attribution of sentience is nothing more than “a simulation.” But of course, how can you know you do not think without thinking this? and what is the distinction between a powerful simulation and that which it is simulating? None of these questions have obvious answers; the novel gives a better account of their complexity than the other, more straightforward arguments about them have done. (Which is, as far as I am concerned, another example of the speculative heft of science fiction; the questions are posed in such a manner that they resist philosophical resolution, but continue to resonate in their difficulty).

http://www.shaviro.com/Blog/?p=1866