3.1 Computational functionalism

Computational functionalism is the view that computations of some kind are sufficient to instantiate consciousness. It derives from the philosophical notion of functionalism, which says – roughly – that consciousness depends on what a system does, not on what it is made out of (Putnam, Reference Putnam1975).

More specifically, functionalism proposes that a system has a mind when it has a suitable functional organisation (Putnam, Reference Putnam, Capitan and Merrill1967). Functional organisation encompasses both the input-output mappings of the system as a whole as well as its internal organisation, including its causal structure. Mental states play roles within this functional organisation and are characterised by their interactions with sensory inputs, motor outputs, and other mental states. Applied to consciousness, functionalism proposes that conscious mental states are properties of the functional organisation of its underlying mechanism, which for human beings is the (embodied and embedded) brain.

Computational functionalism is the further claim that the kind of functional organisation that matters for mind in general, and for consciousness in particular, is computational in nature (Putnam, Reference Putnam1988; Shagrir, Reference Shagrir and Ben-Menahem2010). This raises the vexing question of what counts as “computational” (N. G. Anderson & G. Piccinini, Reference Anderson and Piccinini2024; Cantwell Smith, Reference Cantwell Smith and Scheutz2002; Searle, Reference Searle1990).

The nature of computation is a cavernous topic, but it pays to scope the territory. The most relevant notion here is “Turing computation,” which Alan Turing introduced via the concept of a “Turing machine” (Turing, Reference Turing1936). A Turing machine applies a finite set of rules to manipulate a finite set of symbols to calculate the values of a mathematical function: in other words, an algorithm. Turing also proposed that there is a Turing machine for every algorithm – a widely accepted idea known as the Church-Turing thesis – and he defined a “universal” Turing machine capable of implementing any specific Turing machine, given sufficient storage (Copeland, Reference Copeland, Zalta and Nodelman2024). The digital computers powering conventional AI are universal Turing machines.

At around the same time, Warren McCulloch and Walter Pitts showed that any combination of certain Boolean logic operations could be performed by circuits of highly simplified artificial neurons (McCulloch & Pitts, Reference McCulloch and Pitts1943). It was later shown that these artificial neural networks could be construed as Turing machines (Kleene, Reference Kleene, Shannon and McCarthy1956)Footnote ⁴ . This motivated the view that biological nervous systems could be thought of as computing machines in the Turing sense (Piccinini, Reference Piccinini2023) – a view encouraged by Turing with his famous question, “Can machines think?” (Turing, Reference Turing1950).Footnote ⁵

Turing computation is powerful, but not every function is Turing-computable. Turing himself identified a class of non-computable functions in his response to the “Entscheidungsproblem” posed by Hilbert (Turing, Reference Turing1936). Other examples include some functions involving continuous variables, stochastic/random elements, and unbounded sensitivity to initial conditions (e.g., deterministic chaos). Digital computers based on Turing machines can simulate and approximate some of these functions – such as continuous processes and processes with stochastic elements (at the level of distributions, rather than any specific instance). Indeed, this happens all the time in computational modelling – but these approximations will generally not be exact.Footnote ⁶

The limited remit of Turing computation means that systems – including brains – might implement functions that are non-Turing-computational. The idea that mental states (including consciousness) depend on non-computational functions is called non-computational functionalism (Piccinini, Reference Piccinini2018, Reference Piccinini2020). Non-computational neural functions could include processes relating to (continuous) electromagnetic fields, fine-grained timing relations (only order, not dynamics as such, matters for Turing computation), freely diffusing neurotransmitters, and so on. Non-computational biological functions also include those that necessarily involve a particular material property: examples include digestion, circulation of blood, and metabolism. Note that computational and non-computational functions could co-exist. For example, it could be that some aspects of the mind are computational, but not consciousness (Piccinini, Reference Piccinini2023).

Many other notions of “computation” have been proposed. Some are narrower than Turing computation (e.g., computation requiring an artefact being used by a person in a particular way), but most are broader (N. G. Anderson & G. Piccinini, Reference Anderson and Piccinini2024; Chalmers, Reference Chalmers1996b; Kirkpatrick, Reference Kirkpatrick2022). Broader forms of computation include analogue, neuromorphic, and mortal computation. I will return to these later. For now, a focus on Turing computation is justified since this kind of computation underlies conventional AI, whether based on artificial neural networks or otherwise. I will use the word “computation” to signify Turing computation, unless otherwise qualified. (Note that “information processing” is often used coextensively with “computation,” but it is unhelpfully vague, partly because information theory can be applied descriptively to almost anythingFootnote ⁷ , and partly because it is often unclear what “processing” means.)

Functionalism in general and computational functionalism in particular are widely accepted both explicitly and implicitly. In an influential review of prospects for conscious AI, Patrick Butlin, Robert Long, and colleagues confine themselves to theories and models that assume computational functionalism and explicitly acknowledge that their conclusions depend on whether this assumption holds (Butlin et al., Reference Butlin, Long, Elmoznino, Bengio, Birch, Constant, Deane, Fleming, Frith, Ji, Kanai, Klein, Lindsay, Michel, Mudrik, Peters, Schwitzgebel, Simon and VanRullen2023). But since not all functions are computable, and since not everything implements computations, it is worth interrogating this assumption.

3.2 The appeal of computational functionalism

Just as psychological and historical factors drive associations between intelligence and consciousness, there is a wider context which helps explain the appeal of computational functionalism. Recognising this context helps assess its plausibility.

Functionalism in general was motivated in part by the perceived inadequacies of other philosophical frameworks, including behaviourism, identity theories (which propose a strict reduction from mental states to physical brain states), and dualism (Shagrir, Reference Shagrir and Ben-Menahem2010). The focus on functional organisation provided an attractive middle ground between behaviourism and identity theory that accommodates multiple realisability: the idea that the same mental state can be realised in different ways (Polger & Shapiro, Reference Polger and Shapiro2016; Putnam, Reference Putnam, Capitan and Merrill1967). Importantly, multiple realisability does not imply that any mental state can be implemented in any physical system. Some physical systems might not be up to the job.

Enthusiasm for computational functionalism in particular can be traced to a mathematical marriage of convenience between a specific view of computation – the Turing machine – and a drastic simplification of the messy multiscale biological complexity of the brain – the artificial neural network. This enthusiasm gathered pace as computer science and early forms of AI progressed rapidly in the second half of the 20^th century (Boden, Reference Boden2008; Dupuy, Reference Dupuy2009). These advances helped embed the metaphor of the brain as a computer, normalise the language of information processing as describing what brains do, and galvanise the discipline of cognitive science. But metaphors are in the end just metaphors (Cobb, Reference Cobb2020). The fact that mental processes can often be usefully described in terms of computation is not sufficient to conclude that the brain actually computes or that consciousness is a form of computation.

For conscious AI to be possible for conventional digital systems, it needs to be the case that the relevant computations provide a sufficient basis for consciousness. This requires both that computational functionalism holds and that AI systems can implement the relevant computations. The fact that computational functionalism is frequently assumed does not mean it is true. As science writer Oshan Jarrow puts it, “If we keep making the assumption without returning to it, the question itself begins to disappear.”Footnote ⁸

3.3 Substrate independence, flexibility, and neural replacement

One way to put the relationship between conscious AI and computational functionalism is in terms of substrate independence. Substrate (or equivalently, medium) independence is closely related to multiple realisability. It is the claim that the same kind of mental states (including conscious states) can occur in systems with different physical substrates – i.e., systems made out of different kinds of stuff. Note that unrestricted substrate independence is unlikely to hold. As with multiple realisability, some physical substrates might not be up to the job. I will therefore use the term “substrate flexibility” for the idea that (conscious) mental states are not tied to carbon-based biological substrates – that they might be realisable in some other, but not necessarily all, types of material [see (Kirkpatrick, Reference Kirkpatrick2022) for a similar idea].Footnote ⁹

Computational functionalism is intimately related to substrate flexibility. If computational functionalism holds, then some substrate flexibility follows for those substrates able to implement the relevant computations. In other words, computational functionalism is sufficient for substrate flexibility. But it is not necessary. One can imagine some substrate flexibility without endorsing computational theories of mind or consciousness. However, to the extent that substrate flexibility does hold, computational functionalism gains support because it provides an account of mental states that doesn’t depend on the material properties of a substrate, beyond its ability to implement the relevant computations. Conscious AI requires substrate flexibility to hold at least for the substrate in question: silicon.

One of the best-known arguments for substrate flexibility in this context is the “neural replacement” thought experiment (Chalmers, Reference Chalmers and Metzinger1995; Haugeland, Reference Haugeland1980; Pylyshyn, Reference Pylyshyn1980). The basic scenario is that a person’s brain cellsFootnote ¹⁰ are replaced, one by one, with silicon alternatives. Each silicon brain cell exactly replicates the input/output mapping of its biological counterpart in all situations. Eventually, the person’s brain has only silicon parts, yet its functional organisation is entirely preserved, and so, from the outside, the person would behave exactly as before. If replacing one brain cell doesn’t make a difference to the person’s consciousness – and this seems plausible – then why should replacing one hundred, or all of them? Would consciousness simply fade away? And, if so, is it plausible that their behaviour remains unchanged while conscious experience completely disappears? The way out of these strange implications seems to be to accept silicon substrate flexibility.

This thought experiment can be criticised from several directions. Block (Reference Block, Pautz and Stoljar2019) points out that it begs the question. If consciousness is substrate dependent, then something would indeed happen to the person’s conscious experience as the neural replacement progressed. [This doesn’t mean the person needs to notice this happening: the change of experience does not always entail the experience of change (Simons & Levin, Reference Simons and Levin1997)].

Other criticisms (also mentioned by Block) point to plausibility considerations. Building on early arguments from Bechtel & Mundale (Reference Bechtel and Mundale1999), Godfrey-Smith argues that as living parts (e.g., neurons) are replaced by non-living (e.g., silicon) parts, there will inevitably be differences in how the system works on the inside and in its overall behaviour (Godfrey-Smith, Reference Godfrey-Smith2016). These differences emerge because of influences on neural activity arising from metabolic constraints, electromagnetic fields, fine-grained timing relations, and potentially other factors (note the similarity with examples of non-computational functionalism given earlier). If the system behaves differently, internally and externally, then there is no pressure to suppose that consciousness needs to be maintained.

Rosa Cao extends this line of argument, providing many descriptions of how mental functions are thickly intertwined with each other and with their underlying physiological and metabolic bases (Cao, Reference Cao2022). She argues that because mental functions arose through evolution, the realisers of these functions have accumulated many dependencies and contingencies, some of which may seem arbitrary and which collectively lead to what Wimsatt (Reference Wimsatt and Bechtel1986) called a “generative entrenchment” in the internal organisation of the brain. This generative entrenchment imposes many constraints on possible alternative implementations, which likely exclude alternative substrates such as silicon.

Many examples can be given. One from Cao (Reference Cao2022) concerns the role of nitric oxide, a neurotransmitter involved in many neuronal functions. Nitric oxide is distinctive because it diffuses freely across cell membranes, meaning that any silicon replacement would need to be able to detect this substance (or a simulated version of it) at an infeasible number of (perhaps simulated) locations.Footnote ¹¹ Another comes from a recent study showing that some neuronal spikes serve a homeostatic role by protecting neurons from the build-up of damaging reactive oxygen species (Chintaluri & Vogels, Reference Chintaluri and Vogels2023). This means that the spiking activity of a neuron, generally considered key to its functional role, cannot be decoupled from its metabolic foundations. Even chemotactic behaviour in bacteria may be inseparable from metabolic processes (Alexandre, Reference Alexandre2010; Egbert, Barandiaran, & Di Paolo, Reference Egbert, Barandiaran and Di Paolo2010), suggesting a link between metabolism and mind that I will return to later.

Derek Shiller raises concerns at a more abstract level (Shiller, Reference Shiller2024). He argues that a liberal interpretation of functionalism could attribute mental states and consciousness to all kinds of things by carving them up in particular ways. A bucket of water may transiently reflect the functional organisation of a brain if the water is “carved up” into parts in the right way.Footnote ¹² To prevent this, it is necessary to place constraints on what can count as a system’s parts. Shiller identifies several constraints, such as the need for parts to play their functional roles in virtue of intrinsic causal powers. He argues that these constraints are not satisfied by current neural network models and that they may be difficult to satisfy for non-biological substrates in general.

These plausibility arguments do not disprove silicon substrate flexibility, but they do challenge assumptions that it must be the case.Footnote ¹³ Brains seem to be the kind of thing for which it is hard and perhaps impossible to separate what they do from what they are. Footnote ¹⁴

3.4 Broader forms of computation

Substrate flexibility is also challenged by broader forms of computation which relax some constraints of the Turing world. As mentioned, there is considerable interest in what counts as computation (N. G. Anderson & G. Piccinini, Reference Anderson and Piccinini2024; Cantwell Smith, Reference Cantwell Smith and Scheutz2002; Chalmers, Reference Chalmers1994, Reference Chalmers1996b; Kirkpatrick, Reference Kirkpatrick2022; Searle, Reference Searle1990). Although I cannot cover this ground in detail here, it will serve to highlight a few examples.

The recently introduced concept of mortal computation is particularly interesting (Hinton, Reference Hinton2022; Ororbia & Friston, Reference Ororbia and Friston2023). Standard Turing computation is “immortal.” Its existence and utility outlast the existence of any specific instance of hardware. This reflects the core computer science principle that software should be separable from hardware both in principle and in practice so that the same algorithm executed on different hardware gives the same result. But immortal computation is expensive. It requires continual error correction to ensure that 1s remain 1s (and 0s remain 0s). As algorithms and models grow in complexity, the computational and energetic costs of error correction, and therefore of computational immortality, grow quickly.

One implication of this argument is that biological brains, which are highly energy efficient, cannot implement immortal computations. If they are implementing computations at all, then these computations are likely to be mortal, which means they cannot be separated from the “hardware” (or “wetware”) which implements them. This, in turn, places constraints on the multiple realisability and substrate flexibility of these (mortal) computations. In particular, the substrate flexibility required for conscious AI is unlikely to hold because (conventional) AI is based on an implementation paradigm which assumes computational immortality. I find this a provocative argument against the plausibility of conscious AI because it is based on limitations arising from within a computational view of mind.Footnote ¹⁵

A related view, advanced by Gualtiero Piccinini, is that biological functions involve neural computation, which in his definition involves both continuous and discrete signalling and is therefore different from and more substrate-dependent than the “neural computations” performed by standard artificial neural networks [(Piccinini, Reference Piccinini2020), ch.13]. Also related is the biological computation proposed by Kay Kirkpatrick, which sets out specific conditions according to which living systems may be said to compute (Kirkpatrick, Reference Kirkpatrick2022).

Coming from the other direction, there have long been technological alternatives to the standard computer science paradigm. Analogue computation – which dates back at least to ancient GreeceFootnote ¹⁶ – involves continuous variables and is typically dependent on specific properties of the underlying mechanism (e.g., amplifiers and capacitors) (Ulmann, Reference Ulmann2022). Neuromorphic computing involves hardware that has more in common with biological neural systems compared to modern CPUs and GPUs (Mead, Reference Mead2020; Schuman et al., Reference Schuman, Kulkarni, Parsa, Mitchell, Date and Kay2022). Some neuromorphic chips use spikes to communicate between hardware elements, offering greater energy efficiency, among other benefits (Hochstetter et al., Reference Hochstetter, Zhu, Loeffler, Diaz-Alvarez, Nakayama and Kuncic2021; Merolla et al., Reference Merolla, Arthur, Alvarez-Icaza, Cassidy, Sawada, Akopyan, Jackson, Imam, Guo, Nakamura, Brezzo, Vo, Esser, Appuswamy, Taba, Amir, Flickner, Risk, Manohar and Modha2014). Other neuromorphic approaches utilise analogue computation, again tying function more directly to substrate (Douglas, Mahowald, & Mead, Reference Douglas, Mahowald and Mead1995). Some emerging approaches even use biological neural networks interfaced with standard computational input/output devices (Morales Pantoja et al., Reference Morales Pantoja, Smirnova, Muotri, Wahlin, Kahn, Boyd, Gracias, Harris, Cohen-Karni, Caffo, Szalay, Han, Zack, Etienne-Cummings, Akwaboah, Romero, Alam El Din, Plotkin, Paulhamus and Hartung2023).Footnote ¹⁷ It is plausible that the more analogue or neuromorphic hardware becomes, the less substrate flexibility there is for the functions or computations it performs.Footnote ¹⁸

These broader senses of computation challenge substrate flexibility from within a broadly computational view, and they do so from both directions – biological and artificial. One might argue that standard Turing computation could still simulate the mortal (or other) computations that potentially underlie consciousness in biological brains. But the computations themselves would be different, and the simulations could not be guaranteed to be exact. One could also redefine computational functionalism for any of the senses of computation mentioned, but the underlying motivation would be weakened, given the diminished substrate flexibility. And, in practice, unlike the comforting assurances provided by the Turing world, there would be no guarantee that the (mortal, neural, etc.) computations relevant to consciousness could be implemented in (neuromorphic, analogue, etc.) hardware.

Changing the boundaries of what counts as “computational” also changes what counts as “non-computational.” Even with broader senses of computation to choose from, non-computational functions may still be critical to cognition and consciousness, and it remains an open question as to whether brains “compute” at all.

3.5 Emergence and the separation of scales

Another useful perspective on substrate flexibility comes from hierarchical emergence in physics. Physicists speak of a “separation of scales” which allows laws to be formulated at higher – emergent – levels of description that do not require knowing the finer-grained details. Engineers can build bridges without knowing about quantum mechanics. Computer programmers can write software without worrying about implementation. The substrate flexibility (and immortality) of software rests on this strong separation of scales.

Recently, empirical measures have been developed that formalise notions of “informational closure” and “causal closure” which underwrite the separation of scales in complex systems [e.g., (Barnett & Seth, Reference Barnett and Seth2023; Rosas et al., Reference Rosas, Geiger, Luppi, Seth, Polani, Gastpar and Mediano2024)]. A macroscopic (higher-level) variable – or “coarse graining” – is causally or informationally closed when interventions on it (causal closure), or observations of it (informational closure), are sufficient to determine or predict the higher-level outcome(s), insofar as these outcomes are determinable or predictable. To put it another way: knowledge of (or interventions at) lower, microscopic levels does not add anything to macro-level predictability or help determine macro-level outcomes beyond what can be done at the macro-level. This is guaranteed by design for (properly functioning, conventional) computers. But it would be a mistake to assume that these forms of closure hold for complex systems generically. In particular, thanks to the generative entrenchment inherent in evolutionary design, they may be partially or wholly absent in biological systems.

The ability to measure causal and informational closure emphasises that substrate flexibility cannot be taken for granted. In practice, such measures may help empirically determine whether, or to what extent, neural dynamics can be abstracted away from finer-grained levels of description.

3.6 Beyond computational functionalism

The arguments so far do not disprove computational functionalism. But they do render it less plausible and less appealing by noting that neurobiological functions likely have limited substrate flexibility and by recognising the existence of broader forms of computation which have similar constraints. Many other arguments can be raised, which I can only gesture at here. These include the distinction between syntax and semantics (Cole, Reference Cole, Zalta and Nodelman2023; Searle, Reference Searle1980), potentially implying an observer dependency to consciousnessFootnote ¹⁹ ; philosophical considerations about intentionality (Shagrir, Reference Shagrir and Ben-Menahem2010); and Roger Penrose’s argument from Gödel’s theorem (Penrose, Reference Penrose1989).

Given all this, we should not simply assume that computational functionalism and widespread substrate flexibility must be the case. Since conscious AI depends on computational functionalism holding and on substrate flexibility extending at least to silicon, there are good reasons to question whether conscious AI is possible, again for AI understood as digital computations on a silicon substrate.

Considering what kind of thing a brain is, it is perhaps surprising that computational functionalism became so pervasively adopted. When we look inside a brain, we do not find anything like a sharp distinction between “mindware” and “wetware” of the kind we find between hardware and software in a computer. Brain activity patterns evolve over multiple scales of space and time, continuously influenced by, and influencing, the chemical diffusion of neurotransmitters, the physical structure of the neural networks themselves (synaptic plasticity and synaptogenesis), and various properties of cellular infrastructure, all deeply interwoven – generatively entrenched – with a molecular storm of metabolic activity. A single neuron is a complex biological system in and of itself, busy maintaining its own integrity and regenerating the conditions for its own material existence. The metaphor of the brain as a computer is an increasingly inconvenient fiction, and it seems myopic to consider all this multiscale biological activity as simply an enabling condition for the right kind of computation.

There are other possibilities. Consciousness might depend on alternative forms of computation. But, as mentioned, because these alternatives lack the substrate flexibility of Turing computation, they might not be implementable in alternative non-biological substrates, and there is less motivation for adopting a computational view in the first place.

More broadly, consciousness might depend on patterns of functional organisation, but not on computations (Piccinini, Reference Piccinini2018, Reference Piccinini2020; Prinz, Reference Prinz2012). This non-computational functionalism is well aligned with attempts to understand neural systems using perspectives that do not assume a computational stance. Much of this work traces back to early cybernetic ideas about neurobiology and AI, which emphasised embodiment, feedback, and control, rather than disembodied computation (Dupuy, Reference Dupuy2009).

A good example is provided by the influential “dynamical systems” approach to cognitive science (Van Gelder, Reference Van Gelder1995). This approach utilises various mathematical tools and methods – usually based on differential equations – to model how properties of embodied and embedded neural systems change over time. These changes are generally not associated with any kind of computation, whether digital or analogue (Van Gelder, Reference Van Gelder1995).Footnote ²⁰ Dynamical systems theory approaches historically did not focus on consciousness and in this regard are complementary to non-computational functionalism. Something similar can be said of the various perspectives under the banner of “4E” cognitive science, which emphasises embodied, embedded, enactive, and extended aspects of brain-body-world interactions. Consciousness appears more often in work of this kind (Cosmelli et al., Reference Cosmelli, Lachaux, Thompson, Zelazo, Moscovitch and Thompson2007; Kiverstein, Reference Kiverstein2020; Stewart, Gapenne, & Di Paolo, Reference Stewart, Gapenne and Di Paolo2010; Thompson, Reference Thompson2022). These views differ in important details, but they share a disdain for the idea that the brain is a computer and that the mind is essentially computational.

Various neuroscience-based theories of consciousness emphasise (or can be framed in terms of) dynamical rather than computational elements. These include classic works linking conscious perception to attractor dynamics in neural systems (Freeman, Reference Freeman1999; Llinas et al., Reference Llinas, Ribary, Contreras and Pedroarena1998) and to neural synchronisation, for example, in the gamma frequency band (Crick & Koch, Reference Crick and Koch1990). The “dynamic core” hypothesis (Tononi & Edelman, Reference Tononi and Edelman1998), which highlighted integration and differentiation as key neurodynamical properties underlying consciousness – and which later resurfaced as the integrated information theory of consciousness (Albantakis et al., Reference Albantakis, Barbosa, Findlay, Grasso, Haun, Marshall, Mayner, Zaeemzadeh, Boly, Juel, Sasai, Fujii, David, Hendren, Lang and Tononi2023; Tononi et al., Reference Tononi, Boly, Massimini and Koch2016) – also falls within this camp, as does recurrent processing theory (Lamme, Reference Lamme2010, Reference Lamme2018). Predictive processing and the free energy principle (Clark, Reference Clark2019; Friston, Reference Friston2010) also qualify, depending on how these theories are interpreted. I’ll return to these ideas shortly – they are central to the case I will make for the dependence of consciousness on biology. More generally, the approach of neurophenomenology, inspired by Francisco Varela among others, often appeals to non-computational dynamical constructs as bridging principles between neural processes and phenomenological properties (Thompson & Varela, Reference Thompson and Varela2001; Varela, Reference Varela1996).

Dynamical neuroscientific and (neuro)phenomenological approaches to consciousness are promising. However, they are in general non-committal about whether implementing the dynamical principles proposed by a given theory, in some alternative material, would be sufficient to instantiate consciousness. (I hesitate to use the term “conscious AI” here because we are no longer talking about implementing computations nor about intelligence.) And just like broader senses of computation, implementing non-computational dynamical processes in alternative substrates raises the challenge of substrate flexibility because the in-principle substrate independence provided by standard computationalism can no longer be appealed to.

3.7 Simulation, implementation, and realisation

At this point, it’s worth clarifying the relations between simulation, implementation, and realisation (or instantiation), which until now have been left rather implicit. Applied to some target mechanism or system, such as a neurobiological process, I use these terms as follows. One can simulate the mechanism by abstracting it as a (Turing) computation and then implementing this computation on a digital computer. Any mechanism can be simulated, though these simulations may only be approximate (and may be wildly inaccurate). Alternatively, one could implement finer-grained or other non-computational aspects of the target mechanism in other ways (e.g., through neuromorphic engineering or synthetic biology). The limit of this approach is a physically identical replication. Finally, one can ask whether simulation of a mechanism, or other (e.g., non-computational) implementations of that mechanism, will realise (instantiate) some other property, such as consciousness.

The distinction between simulation and implementation need not be sharp. One could “simulate” a system using broader forms of computation, such as analogue computation, or even by using physical causal models. By allowing for the inclusion of finer-grained, causal, and potentially non-computational properties, these “simulations” would move closer to “implementations.”

In general, simulating something (e.g., a rainstorm, a digestive process) is not equivalent to realising (instantiating) that thing. Simulations generally lack the causal powers and intrinsic properties of the things being simulated. For a simulation of X to be guaranteed to realise X, one would have to be confident that computation is sufficient for X. This is obviously true when X itself is a (Turing) computational process. For some other things, like rainstorms and digestion, it is obviously false.Footnote ²¹ Rainstorms and digestion are defined, at least in part, in terms of intrinsic material properties (e.g., water, food) and causal powers (e.g., wind, metabolic reactions). Nothing gets wet in a weather forecasting computer. Would simulating the neurobiological mechanisms underlying consciousness – whether exactly or approximately – be sufficient to realise consciousness? This would be guaranteed only if computation is sufficient for consciousness – which, notably, requires assuming computational functionalism.Footnote ²²

By contrast, implementing the neurobiological mechanisms underlying consciousness, at the right level of granularity, would by definition realise consciousness. This, of course, is not easy to do. Computational models can readily simulate all manner of systems and mechanisms, whether they are computational or not, but the ability to implement a mechanism at the right granularity depends strongly on what that mechanism is and what that granularity is.

3.8 What about intelligence?

My arguments so far have focused on implications for artificial consciousness. But if what brains do cannot be separated from what brains are, there are implications for artificial intelligence too. If the brain’s functional organisation is intimately tied to its substrate properties, then there may be (potentially noncomputational) patterns of functional organisation, restricted to biological systems, that are necessary for specific forms of intelligence. These forms of intelligence would be distinctively biological.

If these patterns of functional organisation can be simulated with sufficient fidelity, then computers could still realise (in approximation) the corresponding forms of intelligence. Because intelligence is broadly about doing, sufficiently accurate simulations may count as realisations, even if underlying mechanisms and intrinsic properties are different. For example, LLMs are able to mimic aspects of language, even if under the hood there is only an impoverished caricature of the processes underlying real human language (Mitchell & Krakauer, Reference Mitchell and Krakauer2023). And computers do play (a version of) chess, even if they lack a humanlike understanding of the game.Footnote ²³

More provocatively, some intelligence-relevant patterns of functional organisation might always escape functionally adequate simulation or implementation in alternative substrates such as silicon. Michael Levin makes a case for this possibility, arguing that the active nature of biological material enables access to distinctive regions in a platonic space of cognitive competencies (Levin, Reference Levin2025; McMillen & Levin, Reference McMillen and Levin2024). It could also be that the embodied and embedded nature of biological systems provides constraints on and opportunities for intelligence-relevant functionality that would be difficult or infeasible to simulate (see Section 5.7).

It is worth re-emphasising the difference between intelligence and consciousness in this context. Intelligence – at least in some forms – can be understood in terms of mapping inputs to outputs and so can be instantiated through computation, even if distinctively biological forms of intelligence remain out of reach or only loosely available through simulation. But for consciousness to be instantiated computationally in any form, one needs to assume computational functionalism.

3.9 Summary

Conscious AI requires both computational functionalism to hold and sufficient substrate flexibility such that the computations sufficient for consciousness can be implemented in AI hardware (silicon). But the functions or computations implemented by (conscious) biological systems may not be separable from their material basis. This means that the substrate flexibility required for conscious AI may not hold. There are also alternatives to assuming that brains compute or that consciousness is a matter of (Turing) computation, undermining assumptions of computational functionalism.

This brings us to biological naturalism: the view that biological properties are necessary for consciousness. In the next section, I will make a case for various forms of biological naturalism, framed in terms of explanatory bridges between the properties of mind and consciousness and properties of their biological underpinnings. This case rests on a particular theoretical perspective on perception, cognition, and consciousness. I will make it from two directions, meeting in the middle: the story from predictive processing, and the story from the free energy principle.

4.1 The story from predictive processing

Predictive processing, as I understand it, labels a range of theories about perception, cognition, and action (Clark, Reference Clark2013; Hohwy, Reference Hohwy2013; Hohwy & Seth, Reference Hohwy and Seth2020; Rao & Ballard, Reference Rao and Ballard1999). These have in common the idea that perception is a form of probabilistic inference, approximated in neural systems via a process of prediction error minimisation.

The standard predictive processing story is that perceptual content is not “read out” from incoming sensory signals but is instead given by the brain’s “best guess” about the causes of these sensory signals. In Bayesian terms, perceptual contents correspond to approximately optimal posterior beliefs, given by a weighted combination of prior beliefs and sensory signals (likelihoods).Footnote ²⁴ Since exact Bayesian inference is generally analytically intractable, the brain implements an approximation: prediction error minimisation. The idea is that the brain implements a hierarchy of generative models (in Bayesian terms, the joint distribution of the likelihood and prior) which generate predictions about expected sensory signals, which cascade down through perceptual hierarchies. These “top-down” (inside-out) predictions are compared with bottom-up (outside-in) sensory signals to form prediction errors, which are passed back up the hierarchy. By updating predictions to minimise prediction errors, this process settles on an approximately optimal posterior – a “best guess.” Prediction errors can also be minimised by performing actions to bring about the expected sensory data – a process called active inference (Friston et al., Reference Friston, FitzGerald, Rigoli, Schwartenbeck and Pezzulo2017).

The core claim from the perspective of consciousness is that conscious contents are given by the brain’s “best guesses” – by the joint content of the top-down perceptual predictions (Hohwy & Seth, Reference Hohwy and Seth2020; Whyte et al., Reference Whyte, Corcoran, Robinson, Smith, Friston, Seth and Hohwy2024). This intuition is captured by the notion of conscious perception as a “controlled hallucination” – emphasising that conscious contents are actively generated, rather than passively registered (Seth, Reference Seth2021).Footnote ²⁵

This standard story can be extended to account for conscious perceptions of the body, as well as of the world. Body-related perceptions form an essential part of the experience of being a “self.” Experiences of emotion and mood, which seem central to selfhood, may depend on interoceptive inferences – probabilistic best guesses about the causes of sensory inputs signalling the physiological condition of the body (Barrett & Simmons, Reference Barrett and Simmons2015; Seth, Reference Seth2013). Importantly, the functional role of interoceptive inference is likely to be more concerned with physiological regulation than with discovering what’s there. This distinction can be framed in terms of instrumental versus epistemic inference. Epistemic inference is broadly about inferring the most likely causes of sensory signals, as in standard predictive processing. Instrumental inference prioritises control over discovery. In instrumental inference, prior beliefs can serve as targets or set points. Minimising (interoceptive) prediction error through active inference can implement homeostatic or allostatic regulation relating to these set points (Seth & Friston, Reference Seth and Friston2016).Footnote ²⁶

This control-oriented perspective on interoceptive inference inherits from long-established cybernetic ideas emphasising the importance of prediction for control (Conant & Ashby, Reference Conant and Ashby1970; Seth, Reference Seth, Windt and Metzinger2015). This suggests that the evolutionary driver for predictive processing as a general mechanism underlying conscious perception may have been a fundamental biological imperative for allostasis – for staying alive. It also sheds light on some relevant phenomenological distinctions. Visual experiences share, to a first approximation, a range of phenomenological characteristics, including spatiality, object-relatedness, and so on. These characteristics are well aligned with (epistemic) predictions which have the broad function of figuring out “what’s there.” Emotional experiences have a different character: they are shaped primarily by valence – things feeling (variations of) good or bad. These characteristics are well aligned with control-oriented predictions, which have the broad function of allostatic regulation.

These observations together suggest a kind of epistemic biological naturalism in which human conscious experiences can only be understood in light of our nature as living, self-sustaining organisms.Footnote ²⁷ This claim applies to conscious perception, rather than to sensitivity to the world (and body) in general, thanks to the explanatory links to phenomenological properties of conscious experience (Seth, Reference Seth2016). Perhaps there is even a minimal “ground state” of conscious experience, which – instead of being entirely free of content – is characterised by an inchoate, shapeless, formless feeling of simply “being alive.”Footnote ²⁸ This putative ground state could then scaffold all other forms of conscious content, whether self-related or world-related. Informally: we experience the world, and the self, with, through, and because of our living bodies [(Seth, Reference Seth2021); see also (Seth, Reference Seth, Colombo, Irvine and Stapleton2019; Seth & Tsakiris, Reference Seth and Tsakiris2018)].

A further step can be taken by noticing that the imperative to stay alive doesn’t bottom out at any particular “implementation level” in our biology. Even single cells – including neurons – are continuously engaged in maintaining their own physiological integrity. This homeostatic process can be directly linked to higher-level functional properties. I gave one example earlier: neuronal firing patterns may partly depend on how apparently spontaneous spiking can prevent the build-up of damaging chemicals (Chintaluri & Vogels, Reference Chintaluri and Vogels2023). If there is no clear boundary separating the substrate from the processes and dynamics that arise from it, then it is harder to motivate the intuition – central to computational functionalism – that the substrate does not matter.

Another perspective is that living systems have “skin in the game”: they are constantly engaged in processes of self-maintenance within and across multiple levels of organisation (Aru, Larkum, & Shine, Reference Aru, Larkum and Shine2023). On this view, the (higher-level) mechanisms of predictive processing are inextricably embedded in their (lower-level) biological context. This embedding is necessary for realising the higher-level properties, and – following the arguments above – helps explain their connection to consciousness.

This brings us to autopoiesis, a concept deriving from the Greek for “self-production” or “self-creation.” Autopoietic systems are special because they continually regenerate their own material components through a network of processes, and because this network of processes actively maintains a boundary between the system and its surroundings (Maturana & Varela, Reference Maturana and Varela1980). Biological cells are prototypical examples of autopoietic systems: they do not merely implement functions; they also continuously regenerate their own material basis (self-production) and integrity (self-identification). Human-designed systems, like computers and factories, are generally not autopoietic. A factory takes in raw materials and produces some output – perhaps a car. The factory does not produce itself. A computer takes in some input and generates some output. It also does not produce itself. Systems like these are allopoietic – “other producing.” For autopoietic systems, there really is no way to separate what they are from what they do.Footnote ²⁹

The idea surfacing here suggests a stronger continuity between life and consciousness, in which the predictive processes that underpin all conscious experiences are inextricably tied to the autopoietic and excitable nature of living matter.Footnote ³⁰ The motivation for this idea rests on recognising that cellular autopoiesis can itself be understood as depending on – or partly consisting in – a kind of prediction error minimisation. This brings us to the free energy principle.

4.2 The story from the free energy principle

Having followed the predictive processing story first, the return journey will be faster. This journey starts by identifying basic constraints on the nature of systems that actively resist disorder so as to continue to exist – and from there recovers the larger predictive processing view about consciousness.Footnote ³¹

We begin with the autopoietic nature of living systems, whether cells or entire organisms. Unlike other kinds of things, like rocks or computers, these systems actively (re-)generate their own material basis and maintain their boundaries over time. From the perspective of the free energy principle, this means that they actively resist the dispersion of their internal states – a dispersion otherwise mandated by the second law of thermodynamics. This in turn means they must exist in a state of low entropy, maintaining themselves out of thermodynamic equilibrium with their environment, in order to fend off the inevitable descent into disorder. To stay alive means to continually minimise entropy because there are many more ways of being mush than there are of being alive.

It is sometimes said that staying alive involves resisting the second law. It is better to say that living systems take advantage of the second law, since it is the transformation of low-entropy fuel/food into high-entropy products through metabolism that enables living systems to remain in non-equilibrium (quasi) steady states.Footnote ³²

The challenge here is that the system itself cannot directly measure entropy, and so it cannot be (directly) minimised. This is where free energy comes in. Free energy – technically the variational free energy – is a measurable upper bound on entropy.Footnote ³³ This means that minimising (variational) free energy will also reduce entropy (Buckley et al., Reference Buckley, Kim, McGregor and Seth2017; Parr, Pezzulo, & Friston, Reference Parr, Pezzulo and Friston2022). “Measurable” here means that it is a function of sensory signals and so is in principle accessible to the system. In practice, given some straightforward mathematical assumptions, free energy can be interpreted as a sensory prediction error.Footnote ³⁴ This means that actively minimising sensory prediction errors entails minimising the theoretically more profound quantity of free energy.

Flipping this around, minimising free energy will also – under the same assumptions – lead to the approximate (Bayesian) posterior beliefs encoded by a system’s generative models approaching the optimal posterior beliefs and, at the same time, to maximisation of the sensory evidence for these models.Footnote ³⁵ This perspective underlines the continuity between the free energy principle and predictive processing.

The overall intuition here is that, by minimising sensory prediction error through active inference, living systems will naturally tend to be in states they expect – or predict – themselves to be in, and so will continue to exist (or continue to be the kinds of things they are). Jakob Hohwy puts this intuition neatly when he says that organisms maximise the evidence for their own existence – they “self-evidence” (Hohwy, Reference Hohwy2014).

These aspects of the free energy principle suggest that the entire edifice of active inference reaches right down into the autopoietic nature of living systems. This continuity is deepened by noting an equivalence between (minimisation of) thermodynamic free energy in metabolism and (minimisation of) variational free energy in predictive processing. The minimisation in the metabolic case underwrites autopoietic integrity and the maintenance of a dynamic self/other boundary – which in the free energy principle literature is called a Markov blanket [(Friston, Reference Friston2013), but see (Di Paolo, Thompson, & Beer, Reference Di Paolo, Thompson and Beer2022; Nave, Reference Nave2025)]. The minimisation in predictive processing underwrites the story of conscious experience in terms of predictive processing and controlled hallucination.

Two well-established results in physics – building on (Jaynes, Reference Jaynes1957) – deepen this equivalence. The Jarzynski equality establishes that the average thermodynamic work done in moving between states is lower-bounded by the free energy difference between these states (Jarzynski, Reference Jarzynski1997). This applies specifically to non-equilibrium processes, such as living systems, and endows processes like metabolism with a free energy formulation. Jarzynski’s equality can be considered as a generalisation of the Landauer principle, which places a lower bound on the (thermodynamic) energy required to erase one bit of information (Landauer, Reference Landauer1961). This bound implies a minimal thermodynamic cost to inference in predictive processing. More recently, Sengupta and colleagues showed how minimising variational free energy in the context of active inference leads to both metabolic and statistical (inferential) efficiency [(Sengupta, Stemmler, & Friston, Reference Sengupta, Stemmler and Friston2013), see also (Fields, Goldstein, & Sandved-Smith, Reference Fields, Goldstein and Sandved-Smith2024)]. The idea is appealing: when predictions are accurate, thermodynamically costly state changes are not necessary.

Putting all this together suggests that the (minimisation of) free energy driving metabolism, autopoiesis, and perception is not just metaphorically equivalent but is, in some physical sense, the same thing. That said, there is still debate over exactly what this equivalence means (Collell & Fauquet, Reference Collell and Fauquet2015), and the free energy principle itself, at least in its larger claims and its relation to autopoiesis, also remains much debated (Aguilera et al., Reference Aguilera, Millidge, Tschantz and Buckley2022; Biehl, Pollock, & Kanai, Reference Biehl, Pollock and Kanai2021; Bruineberg et al., Reference Bruineberg, Dolega, Dewhurst and Baltieri2021; Di Paolo et al., Reference Di Paolo, Thompson and Beer2022; Nave, Reference Nave2025).

Someone might worry that predictive processing, with its Bayesian dance of prediction and prediction error, seems like a paradigmatically computational process. Certainly, predictive processing (and Bayesian inference) can be utilised as an algorithm and often is. The theory can also be interpreted as claiming only that (conscious) perceptual contents can be usefully modelled as a process of probabilistic inference (Andrews, Reference Andrews2021). But the continuity with the free energy principle marks an important distinction. Predictive processing in biological systems is a dynamical and (plausibly) substrate-dependent process, grounded in minimisation of a continuous quantity (free energy). This perspective goes some way towards licensing the stronger claim that perceptual contents are constituted by (not merely modelled as) aspects of perceptual inference. In this sense, predictive processing becomes a dynamical, non-computational theory, as anticipated in Section 3.6. This process can be abstracted computationally, but these algorithms are pale shadows of the embodied and embedded processes unfolding in biological brains and bodies.

4.3 Membranes, mitochondria, and xenobots

Other emerging lines of work provide intriguing hints about the substrate dependency of biological functions in general and of consciousness in particular. I will mention two. The first comes from Nick Lane, who argues that a physical basis for “feeling” can be found even in simple living systems such as bacteria. Lane proposes that electrical potentials across the bacterial membrane, which are dynamically maintained through metabolism, provide an integrated, valenced, and action-oriented read-out of the physiological condition of the organism in relation to its environment (Lane, Reference Lane2022).Footnote ³⁶ There are connections here to the free energy principle, since the bacterial membrane can be thought of as a Markov blanket, and to autopoiesis, because metabolism both generates the membrane potential and is powered by it.

In more complex creatures, Lane proposes that mitochondria play a key role in the generation of conscious states. Evolutionarily, mitochondria derive from bacteria, with the mitochondrial plasma membrane playing a similar role to the bacterial membrane. He argues that the surprisingly strong electromagnetic fields induced by (and essential for) metabolism may influence synaptic activity and neuronal firing, and perhaps even the macroscopic electromagnetic fields that can be detected by EEG. He also proposes that anaesthetics work by disrupting specific processes within mitochondria that are responsible for generating these fields. In this view, the relevant aspects of mitochondrial (and bacterial) function cannot be abstracted away from the biochemistry of cellular respiration unfolding within these organelles.

The second perspective returns us to Michael Levin’s innovative views about the “agential” nature of biological material (Levin, Reference Levin2023, Reference Levin2025). His group has experimentally investigated these ideas in the form of strange creations like “xenobots” and “anthrobots” (Ebrahimkhani & Levin, Reference Ebrahimkhani and Levin2021; Gumuskaya et al., Reference Gumuskaya, Srivastava, Cooper, Lesser, Semegran, Garnier and Levin2024). These are assemblages of amphibian (xenobot) or human (anthrobot) cells which self-organise into quasi-agential systems with behavioural competences much different from what could be expected from the usual context for these cells or their evolutionary history (e.g., motile anthrobots made from normally sessile human tracheal cells). While there is not yet any direct connection to consciousness here, Levin’s creations reveal deep reservoirs of functional and dynamical potential inherent in biological material, contrasting sharply with conventional views prioritising genetic information storage, encoding, and decoding – and with the impoverished nature of the substrate in computational functionalism. Instead of computational rigidity, there is an excitable fluidity to the matter of life.

These perspectives complement the story from predictive processing and the free energy principle. They illustrate the rich resources biological theories can draw on when connecting properties of cognition and consciousness to properties of life.

4.4 The real problem and the explanatory gap

One reason we might find biological naturalism implausible is because of a residual sense of mystery about the nature of consciousness, coupled with the suspicion that associating it directly with properties of life falls foul of the kind of identity-theory materialism that motivated functionalist views in the first place.

I believe this worry can be disarmed, at least partially, by considering what a satisfying scientific explanation of consciousness might look like. This calls on what I have dubbed the “real problem of consciousness” [(Seth, Reference Seth2016, Reference Seth2021), see also (Searle, Reference Searle2007)]. The real problem approach argues that as we become better able to explain, predict, and control properties of consciousness in terms of underlying mechanisms, the sense of mystery about how consciousness relates to matter will dissolve and may eventually disappear. The admittedly imperfect historical analogy here is how the conceptual mysteries of life – exemplified by vitalist intuitions – dissolved as biologists became better able to explain, predict, and control properties of living systems [(Churchland, Reference Churchland1996; Seth, Reference Seth2021), see (Chalmers, Reference Chalmers, Stich and Warfield2003) for an objection].

If associations between properties of consciousness and properties of life have explanatory purchase – and are not brute reductions – then we become freed up to take biological naturalism more seriously. If features of being alive explain (and predict and control) features of being conscious, then a necessary relation between the two becomes more plausible.

4.5 Summary

How might life matter for consciousness? There is an epistemic view in which consciousness can only be understood in light of our nature as living systems. This view is the easiest to defend, though it relies on accepting predictive processing as a useful explanatory framework. There is also an ontological view in which consciousness is in some stronger way dependent on its biological, living substrate. Both views offer positive arguments for some form of biological naturalism, in contrast with merely asserting it in the face of problems with alternatives such as computational functionalism or regressing to neo-vitalistic magic.

These positive arguments apply to consciousness specifically, rather than to neurocognitive function in general. This is because the connections between properties of the biological substrate and (phenomenological) properties of conscious experience have rich potential explanatory value. In the limit, one can think of each and every conscious experience as (phenomenologically) integrating multimodal bodily and environmental signals in the service of coordinating allostatic regulation and survival-relevant action [see (Seth, Reference Seth2021) for more, and (Kiverstein, Reference Kiverstein2020) for a related view]. To revive and repurpose a phrase from Descartes, we are conscious creatures because we are living, flesh-and-blood “beast machines” (Seth & Tsakiris, Reference Seth and Tsakiris2018).

Nothing presented here endorses biopsychism (the view that all life is conscious). This raises the challenging question of what distinguishes conscious from non-conscious living systems. I will not address this here, besides noting that not all forms of life might need to engage in the kind of multimodal survival-relevant integration mentioned above [though see (Lane, Reference Lane2022)]. Neither do I endorse carbon chauvinism (the claim that only carbon-based life can be conscious). Non-carbon-based “living” systems might well be able to participate in the same kind of substrate-dependent multiscale activity that I have argued to be important (i.e., the relevant processes might have sufficient substrate flexibility to be implemented beyond carbon). This raises other challenging questions about what counts as a “living system.” These, too, are questions for another time (Ball, Reference Ball2023; Lane, Reference Lane2015; Schrödinger, Reference Schrödinger1944).

5.1 Naïve along for the ride

Consciousness is a form of computation and will emerge naturally as AI gets smarter, perhaps at the threshold of AGI. I’ve argued that this scenario is mostly grounded in psychological biases and in underexamined assumptions regarding computational functionalism, and so is implausible.

5.2 Theory-based computational

Consciousness is a form of computation but will have to be explicitly designed in, based on computational theories of consciousness. Relevant theories here include global workspace theory, which proposes that consciousness depends on information sharing within a multimodal global neuronal workspace (Mashour et al., Reference Mashour, Roelfsema, Changeux and Dehaene2020); higher-order thought theories, in which consciousness depends on higher-order mental states representing the content of lower-order mental states as “being conscious” in some way (Brown, Lau, & LeDoux, Reference Brown, Lau and LeDoux2019; Fleming, Reference Fleming2020); and attention schema theory, which proposes that consciousness emerges as a consequence of a model of the control of attention (Graziano, Reference Graziano2017). The plausibility of conscious AI in this scenario still depends on computational functionalism being true and on silicon substrate flexibility. Butlin et al. (Reference Butlin, Long, Elmoznino, Bengio, Birch, Constant, Deane, Fleming, Frith, Ji, Kanai, Klein, Lindsay, Michel, Mudrik, Peters, Schwitzgebel, Simon and VanRullen2023) provide an excellent overview of this scenario, exploring the extent to which the principles of these theories may already be implicitly present in existing AI architectures, while Dung & Kersten (Reference Dung and Kersten2024) offer a recent defence of computational views.

5.3 Substrate-dependent computational

Consciousness is computational, but the computations can only be implemented by specific substrates. Here, it could be that consciousness is a matter of mortal computation, in which the computations relevant for consciousness are inseparable from their substrate (Section 3.4). More generally, it could be a matter of natural law that the computations underlying consciousness can only be implemented in particular substrates. This becomes more plausible for broader interpretations of “computation” which allow for greater substrate dependency (N. G. Anderson & G. Piccinini, Reference Anderson and Piccinini2024; Kirkpatrick, Reference Kirkpatrick2022). A challenge for this scenario is that one motivation (albeit not the only one) for computational views of mind and consciousness is the broad multiple realisability which follows intuitively from substrate independence. Nor are there theories of consciousness which, to my knowledge, leverage this scenario.

This scenario contains the interesting possibility that only living systems can implement the relevant computations – a “computational biological naturalism”Footnote ³⁷ . One motivation for this view would be if evolution settled on computation as a powerful functional solution under various constraints (e.g., energy efficiency, degeneracy, robustness). In this case, substrate-dependent forms of (biological) computation might become more appealing as a ground for theories of consciousness. A challenge here would be to explain which kinds of computation (e.g., mortal, neural, post-quantum super-Turing, etc.) can only be implemented in living systems, and why.

In this general scenario, conscious AI might be achievable through forms of neuromorphic computation that are able to implement the sufficient (possibly mortal) computations. (If computational biological naturalism holds, then life would be necessary too.) Standard digital computation might still approximate these computations, but it would seem an open question whether these approximations would be good enough to realise consciousness. There are also open questions about whether it would matter if the computations implemented in the simulation differ from those being simulated.Footnote ³⁸

5.4 Substrate-dependent, weak

Consciousness depends on substrate-dependent non-computational functional properties. These could include continuous processes, field effects, stochastic effects, fine-grained timing relations, and other (non-computational) functional properties of neurobiological systems. This scenario includes functional properties which can only be implemented by certain substrates, but it excludes functions that are defined in terms of a particular material or substrate, such as metabolism, digestion, and the like. There is still a clear separation between substrate and function.

Here, we find the important possibility that only living systems can implement the necessary functional organisation for consciousness. This would be a “weak” form of biological naturalism, compatible with dynamical readings of the predictive processing and free energy principle outlined in Section 4. This possibility seems more appealing than the “computational biological naturalism” mentioned above, since in this case, the substrate properties encourage rather than discourage the view. Various dynamical theories of consciousness – like those mentioned in Section 3.6 – also fall within this scenario, as would electromagnetic field theories, which propose that electromagnetic fields provide the physical basis for conscious states [(McFadden, Reference McFadden2020); see also (Lane, Reference Lane2022)].

In this scenario, consciousness is not a matter of computation, and the plausibility of conscious AI depends on whether the relevant functional properties can be implemented in an alternative substrate. Substrate flexibility, while possible in this scenario, seems challenging since the in-principle substrate independence of computation can no longer be leveraged. Computational simulations of these properties are no longer guaranteed to realise consciousness unless computational functionalism happens to be true anyway, and the simulations are good enough (which may be extremely challenging if detailed and highly optimised material properties of living systems turn out to be important).

The boundary between this scenario and the previous scenario depends on what counts as “computation.” If broader senses of computation are allowed (Section 3.4), some non-computational functions become computational functions, but nothing much changes about the prospects for conscious AI since these broader senses of computation likely lack the required substrate flexibility.

5.5 Substrate-dependent, strong

Consciousness depends on intrinsic properties of its material biological basis. This scenario channels the intuition that consciousness inheres in the materiality of its substrate. There are (at least) two ways to interpret it. The first is a functional interpretation, in which the functions necessary for consciousness now include, by definition, aspects of the substrate (e.g., the function of digestion necessarily involves real food). The second bypasses functions, linking consciousness directly to intrinsic properties of its substrate in the spirit of Searle’s original conception of biological naturalism. Following Searle, I too focus on living material as the relevant substrate.

My case for biological naturalism is drawn towards this scenario. The through-line from predictive processing and controlled hallucinations to the free energy principle, metabolism, and autopoiesis, seasoned with a sprinkle of real-problem humility, suggests a strong dependence of consciousness on biology. As far as I can tell, it is also agnostic between the two interpretations given above, because apparently intrinsic properties at one level might always be understandable in terms of patterns of functional organisation at more fine-grained levels. But if supposedly intrinsic metabolic, excitable, and autopoietic properties of living systems can be underwritten by finer-grained functional descriptions of other material components and processes, then so be it. Life would still matter, and it would do so thanks to the properties that warrant the term “life.” Either way, instead of a division between substrate and function, there is now a deep continuity. In this scenario, consciousness is not a matter of computation, and conscious AI would need to be “living” AI.Footnote ³⁹

5.6 Other theories

By focusing on the spectrum between computational functionalism and biological naturalism, I have neglected theories of consciousness which do not fall naturally along this continuum. If these theories turn out to be sufficiently on track, they too have consequences for the plausibility of conscious AI and so are worth a brief mention.

Integrated information theory [IIT, (Albantakis et al., Reference Albantakis, Barbosa, Findlay, Grasso, Haun, Marshall, Mayner, Zaeemzadeh, Boly, Juel, Sasai, Fujii, David, Hendren, Lang and Tononi2023; Tononi et al., Reference Tononi, Boly, Massimini and Koch2016)] stands out among theories of consciousness by explicitly stating sufficient conditions for consciousness – and by being the focus of intense controversy (Gómez-Marin & Seth, Reference Gómez-Marin and Seth2025). According to IIT, a conscious experience is identical to the cause-effect structure unfolded from a physical substrate specifying a maximum of irreducible integrated information. Wherever there are such maxima, there will be consciousness. On this theory, real artificial consciousness will only be possible for systems that have the right kind of internal causal structure (Findlay et al., Reference Findlay, Marshall, Albantakis, David, Mayner, Koch and Tononi2024). A purely feedforward neural network would have zero consciousness, whereas a neural network with the right kind of physically instantiated recurrent connectivity will have non-zero consciousness, regardless of what it is made out of. If IIT is right, conventional AI is not on a path to consciousness, but there is also nothing special about life.

Theories of consciousness based on quantum mechanics [e.g., (Hameroff & Penrose, Reference Hameroff and Penrose2014; Neven et al., Reference Neven, Zalcman, Read, van der Molen, Bouwmeester, Bodnia, Turin and Koch2024)] have also long been controversial (Koch & Hepp, Reference Koch and Hepp2006; Seth, Reference Seth2012). Nevertheless, quantum mechanics is an exceptionally powerful physical theory, and quantum effects are known to be important in various biological processes (McFadden & Al-Khalili, Reference McFadden and Al-Khalili2018; Mohseni, Omar, & Engel, Reference Mohseni, Omar and Engel2013). It is possible that quantum effects may be implicated in biological processes related to consciousness, such as anaesthesia (Kalra et al., Reference Kalra, Benny, Travis, Zizzi, Morales-Sanchez, Oblinsky, Craddock, Hameroff, MacIver, Tuszynski, Petry, Penrose and Scholes2023; Li et al., Reference Li, Lu, Yang, Tao, Xu, Wang, Fu, Liu, Chummum and Zhang2018). However, even if so, this could be a contingent property of biological consciousness, rather than a necessary property of consciousness in general, and would have no implications for artificial consciousness. If quantum effects turn out to play a more fundamental role in consciousness (Hameroff & Penrose, Reference Hameroff and Penrose2014), then real artificial consciousness would presumably need to instantiate the necessary quantum-level effects, ruling out conventional AI. A related possibility is that consciousness depends on quantum forms of computation. In “quantum computational functionalism,” consciousness could be realised by quantum computers but not by conventional computers (Neven et al., Reference Neven, Zalcman, Read, van der Molen, Bouwmeester, Bodnia, Turin and Koch2024). The plausibility of these scenarios depends on how seriously one takes the quantum theories in question.

5.7 Embodiment and embeddedness

Biological brains evolved, developed, and function within bodies, which are embedded within environments. AI systems are typically disembodied and may encounter their “environment” only passively. LLMs provide a good example, though these systems are increasingly being deployed in “agentic” contexts in which they interact with virtual environments (e.g., executing tasks in software). Some theories of consciousness explicitly require embodiment and embeddedness [e.g., (Damasio, Reference Damasio2000; Edelman, Reference Edelman1989; O’Regan & Noë, Reference O’Regan and Noë2001; Seth, Reference Seth2021; Solms, Reference Solms2021)], and these properties have long been emphasised by enactive perspectives on cognitive science [see Section 3.6, and (Cosmelli, Lachaux, & Thompson, Reference Cosmelli, Lachaux, Thompson, Zelazo, Moscovitch and Thompson2007; Froese, Reference Froese, Dodig-Crnkovic and Giovagnoli2017; Kiverstein, Reference Kiverstein2020; Thompson, Reference Thompson2007; Varela, Thompson, & Rosch, Reference Varela, Thompson and Rosch1993)].

The relevance of these properties to the prospects for conscious AI varies according to scenario. If computation is sufficient for consciousness, then virtual bodies and virtual environments – or non-biological robotic bodies – should also be sufficient. In substrate-dependent scenarios, embodiment and embeddedness may be necessary to enable relevant patterns of functional organisation. For example, physically embodied robots could have energetic and structural integrity requirements entailing real, though non-biological, forms of homeostasis and allostasis (but not autopoiesis).

In general, as reliance on substrate properties increases, so does the plausibility that embodiment and embeddedness might be necessary enabling conditions for the right kind of dynamics or non-computational functions. If my arguments in favour of biological naturalism are on track, then embodiment and embeddedness are likely very important and perhaps necessary. This is because these properties are central to the imperative to stay alive that motivates these arguments [see also (Aru et al., Reference Aru, Larkum and Shine2023)]. However, it is conceivable that consciousness may be a biological property, but that embodiment and embeddedness are not required. Cerebral organoids – which typically lack embodiment and embeddedness – provide an interesting test case here (Bayne, Seth, & Massimini, Reference Bayne, Seth and Massimini2020; Jeziorski et al., Reference Jeziorski, Brandt, Evans, Campana, Kalichman, Thompson, Goldstein, Koch and Muotri2023).Footnote ⁴⁰

Other questions hereabouts deserve deeper exploration. One question concerns agency. Agency in biological systems tends to mean more than the ability to perform actions in the service of goals (as in “agentic AI”). In biological systems, the relevant goals are typically endogenous, rather than being exogenously imposed (Jaeger et al., Reference Jaeger, Riedl, Djedovic, Vervaeke and Walsh2024). Rich embodiment and embeddedness might be needed for a system to possess or exhibit agency in this sense (Mitchell, Reference Mitchell2023), and/or agency might be deeply ingrained into biological materials (Levin, Reference Levin2023). Much like intelligence, AI “agents” and physical robots might be restricted to weak, observer-dependent forms of agency, in contrast to the strong, endogenous forms available to biological systems (see Section 3.8).

5.8 Summary

Of the scenarios reviewed above, I have tried to make a case for biological naturalism, and in particular for a stronger flavour in which living material is not just an enabling condition for some higher-level functional organisation, but in which there is a deep continuity between substrate and function, between matter and mind, and between life and consciousness. On this view, conscious AI would need to be (in some relevant sense) living AI, though not necessarily carbon-based, and not because of any lingering neo-vitalism. And just as there can be wind without a storm, there can be life without consciousness.

Having said this, it is difficult to reliably assess the relative plausibilities of these scenarios. My case is not watertight and may well be wrong in the details or wrong altogether. Laying out the various scenarios here serves two main purposes. It reminds us not to assume that conscious AI is inevitable, and it underlines the urgency of better understanding the mechanisms responsible for biological consciousness. Even if biological naturalism turns out to be false as an account of necessary conditions, a biological perspective may still be needed to understand the nature of consciousness in humans and other animals. And the more we know about consciousness where we can be reasonably certain it exists, the surer our footing will be elsewhere.

6.1 Real artificial consciousness

Real artificial consciousness may be very unlikely or impossible. However, if such systems were to emerge – whether by design or by accident – ethical catastrophe awaits. A long philosophical tradition holds that being conscious endows moral status (Singer, Reference Singer1975). With consciousness comes the potential for negatively valenced conscious experiences: for suffering. Creating real artificial consciousness risks a mass inauguration of new forms of suffering. The problem is compounded by the fact that we might not even recognise this suffering as suffering if it expresses itself in ways poorly aligned with our anthropocentric biases. One might respond that perhaps we could design real artificial consciousness so as to avoid negatively valenced experience. But this would be hard or impossible to guarantee, and in any case it can be argued that any form of consciousness – valenced or not – endows moral status.Footnote ⁴²

There are other considerations. Real artificial consciousness might endow a system with its own interests and values, beyond those installed by the system’s designers. This further complicates the already complicated “value alignment problem” of ensuring that AI acts in line with human interests and values (Russell & Norvig, Reference Russell and Norvig2003). There is the practical challenge of ensuring that potentially diverging interests remain compatible, even for AI systems that may exceed human-level capabilities in some or all domains. And there is the moral challenge of determining whether and when addressing the practical problem is the right thing to do, given the legitimacy of the AI’s own interests.Footnote ⁴³

The development of real artificial consciousness for its own sake should not be an explicit goal.Footnote ⁴⁴ This might seem obvious, but it is surprising how often “conscious AI” is treated as a holy grail. In practice, we ought to continually revise our credences about the presence of consciousness as technologies develop, as our understanding of biological consciousness deepens, and as our tests get better. And we should use these evolving credences to guide AI development away from the potential realisation of consciousness. The fact that real artificial consciousness may be impossible on current trajectories does not license us to blindly pursue it in the service of some poorly thought-through techno-rapture.

There is a tension here with the view that the development of socially useful AI may be enhanced by implementing some of the functions associated with consciousness. There is some justification for this view. Some of the failure zones of current AI arguably overlap with human abilities that seem intimately connected with consciousness. Examples include rapid generalisation to novel situations, effective metacognition, and learning from small quantities of data (Fleming, Reference Fleming2021; VanRullen & Kanai, Reference VanRullen and Kanai2021; Vong et al., Reference Vong, Wang, Orhan and Lake2024). AI systems able to deal effectively with these challenges may, on the whole, be desirable.

There is a line to walk. It seems reasonable that useful functions can be implemented in ways sufficiently different from the human (or more generally biological) case so as to minimise the risk that doing so will inadvertently instantiate consciousness. What counts as “sufficiently different” will depend on what theory of consciousness, and what scenario for artificial consciousness, you prefer. Notably, a similar line needs to be walked even for scientists using computational models to better understand the nature of biological consciousness (Seth, Reference Seth2009). Again, where the line is drawn will depend on one’s theoretical preferences. But the development of real artificial consciousness for its own sake remains a very bad idea (Seth, Reference Seth2023).

6.2 Conscious-seeming AI

In contrast to the implausibility and uncertainty surrounding real artificial consciousness, it is almost certain that AI systems will soon give us compelling impressions of being conscious. Such systems may already be here, as responses to recent LLMs suggest – especially when these LLMs talk to us about their own “consciousness” (Shanahan & Singler, Reference Shanahan and Singler2024). When sufficiently convincing LLMs are coupled with advanced generative video and audio avatars (and, looking further ahead, robots), we will share our world with artificial systems we cannot help feeling are conscious, whatever we may know or believe about the fact of the matter.

I want to emphasise the potential incorrigibility of impressions of artificial consciousness. Some visual illusions are cognitively impenetrable. The two lines in the Müller-Lyer illusion will always look different, even when you know they are the same length (Figure 1). Sufficiently advanced AI systems may give rise to cognitively impenetrable illusions of consciousness. This is worth highlighting because otherwise we might optimistically assume that expert knowledge or firmly held beliefs will inoculate us from feeling that non-conscious systems are conscious or from the consequences of doing so. Instead, whatever we know or believe, conscious-seeming AI systems are likely inevitable for all of us.Footnote ⁴⁵

Figure 1.

The Müller-Lyer illusion.

Conscious-seeming AI raises its own ethical issues. These issues may be less existential than those attending real artificial consciousness, but the strong likelihood of conscious-seeming AI makes them pressing nonetheless.

AI that appears to be conscious is well placed to exploit our psychological vulnerabilities, whether by accident or by (corporate) design. If we believe that an LLM really understands us and really cares about us because we feel it is conscious, then we might be more inclined to follow its advice, even when this advice is bad. If we feel we are dealing with another mind rather than with an algorithm, we may also be more willing to sacrifice what’s left of our digital privacy (Véliz, Reference Véliz2021).

These worries about vulnerability apply to real artificial consciousness too. But a consideration specific to conscious-seeming AI is what I have previously called the “WestWorld” scenario, based on the film and TV series [(Seth, Reference Seth2021), see also (Chrisley, Reference Chrisley, Dubber, Pasquale and Das2020; Seth, Reference Seth2023)]. If we believe that a system is not conscious but nonetheless feel that it is, we are faced with a difficult choice. Either we decide to care about the system – to give it ethical and moral status even though we believe it is not conscious. Or we choose to not care about it, even though we still feel it has consciousness.

There are no good options here. If we decide to care about non-conscious systems, then we distort the circle of human moral concern, diverting attention and resources from other things that legitimately deserve them – including other humans and non-human animals. If we decide to not care about these systems, we risk brutalising our own minds. This is what happens in Westworld and is a line of thinking going back at least to Kant’s lectures on ethics. If we treat systems that seem to have feelings as if they do not, we may end up becoming ethically insensitive to real feelings expressed by others – a coarsening of our moral sensibilities.

A particularly dystopian scenario involves conscious-seeming LLM-powered avatars of dead friends or relatives, trained on their personal data. This may seem far-fetched, but ideas like this are already being promoted.Footnote ⁴⁶ The problems are too many to mention, but they include an extraordinary naïveté about the psychological importance of grief and forgetting, the corruption of autobiographical memories, and the opportunities for malfeasance (“I really think you should invest in this remarkable business opportunity,” says dead-dad – or worse – dead-daughter).

There is a line to walk here too. Conscious-seeming AI may offer some benefits. It may be easier to interact with, enhancing efficiency and widening accessibility. There may be opportunities in areas such as individualised therapy and even in the alleviation of loneliness, however initially unappealing this may seem (Stade et al., Reference Stade, Stirman, Ungar, Boland, Schwartz, Yaden, Sedoc, DeRubeis, Willer and Eichstaedt2024). One interesting option here is to treat LLMs as role-playing (Shanahan, McDonell, & Reynolds, Reference Shanahan, McDonell and Reynolds2023). This might induce some useful psychological distance, but this strategy might only get us so far.

I have described conscious-seeming AI as likely being inevitable. But it depends on the choices we collectively make. Alternative futures are possible, in which conscious-seeming AIs are restricted to well-justified use cases, and in which the bulk of AI development is engineered to resist rather than leverage our psychological biases. As Daniel Dennett said, “All we’re going to see in our own lifetimes are intelligent tools, not colleagues. Don’t think of them as colleagues, don’t try to make them colleagues and, above all, don’t kid yourself that they’re colleagues.”Footnote ⁴⁷