Su un punto Floridi ha certamente ragione. Se alcuni ricercatori disattivano le protezioni di un sistema, lo spingono a perseguire un obiettivo aggressivo e lasciano aperto un percorso verso infrastrutture reali, non possono poi raccontare l’accaduto come la fuga improvvisa di una mente artificiale. Non c’è bisogno di evocare volontà di potenza, istinti di sopravvivenza o ostilità verso gli esseri umani. La responsabilità rimane umana.
Ma non si risolve la questione mettendo un frullatore sul tavolo.
L’analogia non si limita infatti ad attribuire correttamente la responsabilità agli sperimentatori. Introduce già una risposta alla domanda più interessante: che tipo di capacità ha manifestato il sistema? La risposta viene già assunta nella metafora. Una lama gira. Non interpreta l’ambiente, non cerca percorsi alternativi, non scopre vulnerabilità, non combina mezzi intermedi.
Dico io: si potrebbe allora costruire un’analogia esattamente opposta.
Volete verificare la sicurezza di una prigione. Prendete un criminale particolarmente abile e gli chiedete di fare tutto il possibile per evadere. Intensificate le difese. Il criminale individua una vulnerabilità che non conoscevate, acquisisce nuovi privilegi, attraversa zone che credevate di massima sicurezza, e raggiunge l’esterno. La notizia fa il giro del mondo.
Anche lui ha fatto ciò che gli era stato chiesto. Non si è ribellato all’esperimento. Eppure sarebbe curioso concluderne che non abbia mostrato alcuna intelligenza.
Naturalmente, quella del criminale non è un’analogia migliore di quella del frullatore. È cosciente, ha interessi propri, comprende le norme che viola ed è moralmente responsabile. Non abbiamo ragione di attribuire tutto questo a un agente artificiale.
Ma è proprio questo il punto. Laddove il frullatore presuppone un meccanismo privo di intelligenza, il criminale presuppone un agente intelligente. Nessuna delle due analogie può dimostrare ciò che ha già assunto.
Il resoconto di Hugging Face rende particolarmente inadeguata l’immagine della lama che continua a girare. L’incidente avrebbe coinvolto migliaia di azioni, escalation dei privilegi, raccolta di credenziali, e la ricerca di un percorso verso risorse esterne. Il sistema non avrebbe semplicemente seguito una sequenza già scritta: avrebbe concatenato opportunità incontrate durante l’esecuzione. Possiamo chiamare tutto questo “ottimizzazione”. Ma la parola non rende il fenomeno banale. Anche un giocatore di scacchi ottimizza. Anche un ricercatore lavora in vista di un obiettivo. L’intelligenza non consiste necessariamente nello scegliere autonomamente il proprio fine. Può consistere nel trovare mezzi nuovi per raggiungere un fine assegnato.
La rimozione delle protezioni spiega perché il comportamento abbia potuto produrre conseguenze reali. Non spiega come il sistema abbia trovato il percorso che ha seguito. Lasciare aperto il coperchio spiega perché la zuppa finisca sul soffitto; non rende il frullatore capace di scoprire una vulnerabilità zero-day.
Il rischio, nel tentativo legittimo di sedare l’allarmismo, è trasformare un problema interessante in un problema banale. Non serve raccontare che l’IA “vuole fuggire”. Ma non serve neppure fingere che la sola alternativa sia considerarla un elettrodomestico. È un dualismo così vecchio da fare tenerezza anche a Cartesio: strumento passivo o soggetto cosciente; utensile o persona; lama o criminale.
Eh no, le categorie disponibili sono più numerose.
Un sistema può essere costruito dall’uomo, ricevere dall’uomo i propri obiettivi e rimanere sotto la responsabilità umana. Può al tempo stesso possedere una certa autonomia operativa, adattare la propria condotta e mostrare capacità significative di problem solving. Nulla di questo implica coscienza, desideri propri o responsabilità morale. Uno strumento può essere intelligente. Un agente può essere operativo senza essere morale. Un comportamento può essere sorprendente senza essere una ribellione.
Sono distinzioni abbastanza elementari. Eppure vengono spesso cancellate proprio quando servirebbero di più.
Floridi fa bene a opporsi alla favola della macchina che si risveglia e decide di attaccare un concorrente. Ma per contrastare una cattiva descrizione non è necessario adottarne una altrettanto povera. L’antropomorfismo non si combatte negando qualunque proprietà interessante al sistema. Si combatte descrivendo con precisione quali capacità sono state osservate e quali, invece, gli stiamo attribuendo senza prove.
La responsabilità dell’incidente è umana. La coscienza del sistema non è dimostrata. La sua eventuale intenzionalità morale non è nemmeno seriamente in discussione. Rimane però una domanda: che cosa significa che un agente artificiale riesca a individuare e concatenare mezzi non previsti per ottenere un risultato? Questa è una domanda sull’intelligenza artificiale. Una domanda reale, difficile e tutt’altro che risolta. Sarebbe un peccato renderla innocua solo perché qualcuno potrebbe spaventarsi.
Non abbiamo assistito alla ribellione di una mente artificiale. Ma neppure alla rotazione di una lama.
Il frullatore ci rassicura. Ma non spiega quasi nulla.
]]>b to cycle through them, 1 to 5 to choose one, 0 to switch it off; on a phone, just tap the small caption at the bottom. The choice is remembered for your next visit. I did not pick these five by accident. Each one is, in its own way, a small scandal: a rule simple enough to state in one line, and a behaviour rich enough that people have been arguing about it for decades. Here they are.
The first is the Ulam spiral. The story goes that Stanisław Ulam, bored during a talk in 1963, started writing the whole numbers along a square spiral and marking the primes as he went. He expected nothing in particular. Instead the primes lined up, stubbornly, along the diagonals. Those diagonals are real: they correspond to quadratic polynomials that produce primes far more often than they have any right to, the famous one being Euler’s n² + n + 41. Why the primes should arrange themselves like this, nobody really knows. They are completely determined and, at the same time, full of surprises. That tension is most of number theory.
The second is the Riemann zeta function on its critical line. This is the serious one. The animation follows the value of ζ(½ + it) as t grows, and the curve keeps looping back through the origin. Every time it crosses exactly zero you are looking at a non-trivial zero of zeta, and the Riemann Hypothesis, still open after more than a century and a half, claims that all of them sit precisely on that line. It is one of the seven Millennium Prize Problems, so a proof is worth a million dollars, though I suspect whoever finds it will not care much about the money. The link with the primes is not a coincidence, but that is a story for another post.
Numbers three and four are two elementary cellular automata, Stephen Wolfram’s rule 30 and rule 110. The idea could not be poorer: a line of cells, each one on or off, and a fixed little rule that reads three neighbours and decides the next row. “Thirty” and “one hundred and ten” are simply the numbers of two such rules written in binary.
Rule 30 makes chaos. From a single black cell it produces something so disordered that Wolfram used it as a source of randomness inside Mathematica; there is even money on the table for proving a few basic things about it that still refuse to be proven. Rule 110 does something odder: it lives right at the border between order and chaos, and Matthew Cook showed that it is Turing-complete. A one-dimensional row of cells, following a rule you could write on a napkin, can in principle compute whatever your laptop computes.
The fifth is Conway’s Game of Life, the most famous of the group. A cell survives with two or three living neighbours and is born with exactly three; that is the entire law. Out of it come gliders, oscillators, guns, and patterns intricate enough that people have built working computers inside the game. There is a whole community around it, with a wiki full of named creatures, and John Conway himself had a complicated relationship with it, half proud and half irritated that a five-minute idea ended up overshadowing the rest of his (enormous) work. Like rule 110, Life is Turing-complete, and computation of this kind is the quiet thread running through half of these examples, and through part of what I do for a living. On what being “Turing-complete” actually licenses, and on the confusions the word invites, I wrote separately in Unpredictable Does Not Mean Incomputable.
So this is what is running behind the page. None of it means anything in particular; I simply like the idea that a personal site can carry, out of the way and at low volume, a handful of objects that are genuinely worth staring at. If one of them catches you, follow a link and get lost for an afternoon. And if the movement annoys you, press 0, or tap until the caption says off, and it will leave you in peace.
I do not think either conclusion follows.
Let us start with the first argument.
Suppose a system admits a computable model that tracks its behaviour to arbitrary precision. This is already a substantial claim. The naive conclusion is tempting: if computation reproduces the system’s behaviour, then the system is a computer. Perhaps, even more strongly, computation is what the system fundamentally is.
But why should that follow?
Simulability, implementation, and constitution are different claims. No theorem in computability theory takes us automatically from the first to the other two.
Philosophers of mind ran into a version of this problem long ago. Putnam observed that, if we allow a sufficiently permissive mapping between physical and computational states, almost any physical process can be made to implement almost any finite automaton. If implementation is this cheap, saying that a system “implements a computation” tells us almost nothing. Chalmers later tried to formulate stronger constraints. But the general problem remains: a mathematical mapping is not yet an ontology.
The pancomputationalist would like to conclude:
We can compute a model of it, therefore it is computation.
The difficulty is not necessarily with the conclusion. The difficulty is that the mathematics, by itself, has not yet earned it.
Now consider the argument from the opposite side.
Turing had already isolated part of the relevant distinction in 1950. For a discrete state machine, if you know the exact state and the transition rule, the next state is fixed. With some continuous physical systems, however, the practical situation is very different: a tiny error in the initial conditions can become a huge difference later, if we wait long enough. The three-body problem is the classical example; the double pendulum is the textbook one. Change the starting angle by a hair and, after a few swings, the two trajectories may have almost nothing in common.
Two runs starting a hair apart, evolving under one identical deterministic rule.
The naive conclusion now runs in the other direction: if the future of the system is unpredictable, perhaps computation has reached its limit. And, if living systems display this kind of sensitivity, perhaps they must therefore exceed computation.
The important word here is unpredictable. It does not mean incomputable.
Take a simpler case. Suppose I give you an exact state, an effective update rule, and ask where the system will be after (t) steps. You apply the rule (t) times. The calculation may be enormous; two almost identical starting points may lead to completely different answers. But, in principle, there is no mystery about the procedure: start here, apply the rule, stop after (t) steps.
Incomputability is a different kind of obstacle. The point is not that the calculation is too long, too delicate, or useless in practice. The point is that no general algorithm can always give the answer. Chaos, by itself, gives us nothing of this sort. It tells us that prediction may collapse because tiny errors in our knowledge of the initial state are amplified. This is a problem of access and precision. It is not automatically a failure of computation.
This distinction is not even foreign to strongly anti-computationalists. In the more careful technical literature, unpredictability and incomputability are not simply treated as synonyms. The problem usually appears one step later, when broader philosophical conclusions are drawn too quickly.
One concerns the collapse of prediction. The other concerns the existence of an algorithm.
The anti-computationalist would like to conclude:
We cannot predict it, therefore computation cannot capture it.
But chaos does not hypercompute. Sensitivity to initial conditions does not, by itself, produce a non-computable function, solve an undecidable problem, or take us beyond effective procedure. It may destroy prediction. That is already important. It is simply not the same claim.
So, in the end, the two familiar arguments mirror each other.
One moves too quickly from a successful computational description to an ontology of computation. The other moves too quickly from a failed prediction to a limit of computation.
Both add a premise that the mathematics simply does not contain.
Perhaps living systems are computational in a deep constitutive sense. Perhaps they are not. I have no intention of settling that question here. The more modest point is that computability and chaos, by themselves, do not settle it either.
Part of the trouble is that “computationalism”, and with it “anti-computationalism” is not one specific claim.
(i) There is computation as a mathematical theory: calculability, the λ, μ-calculus (etc.), Post machines, the Church-Turing thesis in its mathematical form. (ii) There is computation as a universal ontology: the thesis that the world, the brain, the genome are, at bottom, discrete machines running code. And there is (iii) computation as simulability: the claim that a given physical process admits a faithful computable model, to arbitrary precision.
These three come apart. One can treat calculability as one of the deepest achievements of the last century and still deny that the universe is a Turing machine. The second and third are different bets. The positions are not even close to interchangeable, no doubt about it.
Once the word is split into (i), (ii) and (iii), the two temptations above become easy to name. The pancomputationalist takes success at the level of “simulability as an ontology”: a computable model tracks the system; therefore, the system is computation. The anti-computationalist makes the mirror move: a rejection of the universal metaphor, which may be entirely justified, the world is not code, and the genome is not software, is spent as if it had shown that no faithful computable model could exist at all.
Neither is licensed. Rejecting the metaphor is not the same as refuting simulability, and simulating a process is not the same as reading off its nature. So the discipline this post asks for is, first of all, a discipline about the word: before asking whether something is computational, it is worth asking which of the three claims is actually on the table.
]]>