MH
← All essays
Originally on Substack

The Mind in the Machine’s Image

How our models of intelligence became models of ourselves

Imagine asking an AI why you keep withdrawing from people who care about you. You describe a few difficult relationships, the arguments you regret and the peculiar discomfort that accompanies emotional closeness. The machine listens, asks questions and eventually suggests that your behaviour resembles an avoidant attachment pattern. It outlines how such patterns develop and offers some practical ways to recognise them.

The interpretation feels convincing. Perhaps it even provides a sense of relief. Something that previously seemed confusing now has a name, a structure and a plausible history.

Over the following weeks, you begin noticing behaviours that fit the description. An uncomfortable conversation becomes evidence of avoidance. A need for solitude acquires psychological significance. You start explaining yourself to others using terms the machine introduced.

Perhaps the framework is helpful. Perhaps it isn’t. Either way, something interesting has happened. The system hasn’t simply described your inner life. Its description has begun to influence how you experience and understand yourself.

In The Word Without Flesh, I explored what happens when language becomes detached from the embodied lives that shaped it and acquires new capacities through artificial intelligence. Near the end, I touched on Alfred North Whitehead and Bernardo Kastrup, whose philosophies challenge our tendency to confuse abstract representations with the reality they describe.

There is another side to that story worth exploring. As our machines become increasingly capable of activities associated with human intelligence, they are also changing how we understand the minds that created them.

We tend to imagine artificial intelligence as an attempt to reproduce human thought. But the relationship runs in both directions. For much of the past century, we have understood the mind through metaphors borrowed from our most sophisticated technologies. Memories became stored information, attention became a limited processing resource, perception became predictive modelling and thought became computation.

These descriptions have contributed enormously to our understanding of cognition. Yet their success raises an uncomfortable possibility. Having learned to describe ourselves in computational terms, we now look at machines performing increasingly complex computations and regard their achievements as evidence that our descriptions of ourselves were complete.

We made the machine in an image of the mind. Now we risk remaking the mind in the image of the machine.

The Machine That Became a Theory

The relationship between computers and theories of human intelligence is stranger than it first appears.

In 1996, psychologists Gerd Gigerenzer and Daniel Goldstein published a paper titled Mind as Computer: Birth of a Metaphor. They proposed what they called the tools-to-theories heuristic: the tendency for scientific instruments and analytical techniques to become models of the phenomena they were originally developed to investigate.

Their historical account traces a circular movement. The conceptual separation of calculation from other forms of intelligence helped make mechanical computing possible. Later, as electronic computers became familiar scientific instruments, researchers increasingly adopted computation as a model of the mind itself. We moved from human reasoning to computing machines, and then from computing machines back to theories of human reasoning.

This was not simply an intellectual fashion. The computational approach offered powerful ways to investigate memory, attention, learning and problem-solving. It helped cognitive scientists move beyond behaviourist explanations that had largely avoided theorising about internal mental states. It also encouraged researchers to construct models precise enough to be tested, challenged and improved.

But something changes when a productive scientific model begins to acquire the authority of an account of what the mind fundamentally is.

Consider memory. Describing memory through the vocabulary of storage and retrieval allows us to investigate how information is encoded, retained and recalled. Yet remembering an event is not ordinarily experienced as retrieving an unchanged file from an archive. Memories are reconstructed, altered by subsequent experience and entangled with emotion, bodily sensation and personal significance.

A childhood home might be recalled through the smell of a particular room, the quality of afternoon light or an almost inexpressible feeling of belonging. A computational account may illuminate important mechanisms involved in this recollection without exhausting what remembering means to the person doing it.

The difficulty isn’t that the model is necessarily wrong. It is that the purposes for which it was constructed may gradually disappear from view. What began as a means of investigating certain aspects of cognition can become an implicit definition of cognition itself.

The sociologist Sherry Turkle recognised this reciprocal relationship early in the personal computing era. In The Second Self (1984), she examined how computers became more than instruments for performing tasks. They were also objects through which people thought about their own identities, intelligence and emotional lives.

The arrival of increasingly conversational machines makes that relationship more intimate. Earlier computers invited comparison with human reasoning because they could calculate, remember and follow instructions. Contemporary AI can also participate in activities through which we ordinarily recognise another person’s mind. It discusses our anxieties, challenges our assumptions and produces interpretations of our behaviour.

The machine has become a compelling mirror, yet the reflection is unusual. It is assembled from human cultural and intellectual material, processed through mathematical architectures and shaped by engineering decisions about what constitutes a useful response. We encounter something recognisably human in its output, while the processes producing that output may be quite unlike those occurring in the person reading it.

This is where a scientific metaphor begins to acquire cultural consequences. If a machine can perform an activity we regarded as evidence of intelligence, we have good reason to reconsider which capacities are uniquely human. But it does not follow that the machine possesses every property we associate with the activity, or that those properties were illusions all along.

A system capable of producing a persuasive account of someone’s emotional life has demonstrated an extraordinary ability to work with representations of human experience. Whether it possesses an inner life of its own remains another question.

And that question becomes considerably more difficult once we examine what it would mean for any system to possess a perspective.

When Does a Process Become Someone?

We can identify the physical machinery executing an AI model, describe its architecture and trace the operations through which it produces a response. We can test its capacities, measure its reliability and investigate its internal representations.

But where, within all this activity, would we locate the subject, if there were one?

Imagine a model operating across thousands of servers, responding to millions of people. Its parameters may be copied, its execution interrupted and resumed, and its conversations distributed across different machines. We can decide where one software process ends and another begins, but those engineering boundaries do not necessarily tell us where a private experience would begin or end.

Would there be one subject distributed across the infrastructure? A separate subject for each conversation? Something emerging temporarily during a particular computation? Or no subject at all?

The existence of sophisticated information processing does not, by itself, settle these questions. Whitehead and Kastrup offer two very different ways of approaching the problem, though neither provides an easy answer.

Whitehead’s process philosophy understands reality in terms of interrelated events of becoming, which he called actual occasions. These have an elementary experiential character, but that does not mean every collection of occasions constitutes a conscious individual.

Whitehead distinguished between different forms of organisation. A physical object may contain countless actual occasions without possessing a unified perspective as a whole. More complex forms of organisation can exhibit enduring patterns of inheritance, including what he called personally ordered societies, in which successive occasions are connected through a continuing historical relationship.

Even personal order, however, does not automatically establish consciousness. The distinction matters because it prevents us from treating the experiential character of reality as sufficient evidence that every composite system is an experiencing subject.

Kastrup approaches the problem from a different metaphysical direction. His analytical idealism proposes that consciousness is fundamental, and that the physical world is the outward appearance of processes within a wider field of consciousness. Individual subjects arise through what he calls dissociation, forming distinct perspectives within that underlying reality.

For Kastrup, biological metabolism does not manufacture consciousness. Rather, the activity of a living organism is understood as the outward appearance of processes associated with a dissociated centre of experience. He consequently questions whether similarities between brains and computers, described at the level of information processing, provide adequate grounds for concluding that computers possess private experience.

There is a vulnerability in this position. Our familiar examples of conscious subjects are biological, but that does not establish that biological metabolism must be the only possible outward indication of a distinct perspective. If particular forms of organisation matter in living systems, it is not obvious why sufficiently different forms of non-biological organisation could never matter too.

Whitehead and Kastrup disagree substantially about the underlying nature of reality. Whitehead’s multiplicity of becoming occasions is not equivalent to Kastrup’s universal consciousness. Nor should either philosophy be treated as a demonstration that machines cannot be conscious.

What they help expose is the problem of individuation: what makes a collection of physical or experiential processes a single subject rather than merely a collection?

It is possible to reproduce an impressive range of cognitive abilities without having resolved this question. There is no contradiction in acknowledging the capabilities of an artificial system while remaining uncertain about whether anything is experienced from its point of view.

Yet that uncertainty cuts both ways. We should not mistake the absence of a familiar biological form for proof that no unfamiliar kind of subject could exist. The question is what evidence would justify either conclusion.

What if the Functionalists Are Right?

A defender of artificial consciousness might reasonably object that this discussion places too much weight on biological history and philosophical uncertainty.

After all, many of the properties we recognise in the world depend upon relationships and organisation rather than upon one particular material. Computers do not merely simulate arithmetic; they actually calculate. An aircraft does not reproduce the anatomy of a bird, yet it genuinely flies.

Why should consciousness be different?

Functionalism, in its various forms, proposes that mental states can be understood through the causal roles they play within a system. If the relevant organisation can be reproduced in another medium, then the corresponding mental states might be realised there too.

This is a stronger argument than claiming a machine must be conscious because it speaks convincingly. It suggests that the important features of mentality may reside in forms of causal organisation that can exist across different physical implementations.

And contemporary cognitive science has moved far beyond the picture of an isolated computer manipulating symbols.

Theories of predictive processing and active inference, developed by Karl Friston and others, attempt to explain perception and action through an organism’s ongoing relationship with its environment. The brain is not simply receiving information and producing responses. It participates in the regulation of a living body, anticipating changes in the world and adjusting behaviour in ways that help maintain the organism’s continued existence.

Building on these ideas, the neuroscientist Anil Seth has argued that our conscious experience of being a self is closely connected to the brain’s regulation of the body’s internal condition. In this account, the experience of being someone is inseparable from the biological processes through which an organism remains alive.

This is an important challenge to simplistic accounts of computation, but it also complicates any straightforward opposition between machinery and life. If the relevant properties involve self-maintenance, adaptive organisation and the regulation of internal conditions, we can at least imagine artificial systems that possess some of these characteristics.

Whether such systems would be conscious remains uncertain. We do not yet know which properties are necessary or sufficient for subjective experience.

There is also a distinction worth preserving between describing a process mathematically and explaining why that process is felt. An organism may regulate its internal condition in ways captured by a free-energy functional, but the mathematical description does not automatically settle why hunger hurts, why fear possesses urgency or why the possibility of death matters to the creature confronting it.

Perhaps a sufficiently complete account of functional organisation will eventually explain these things. Perhaps experience depends upon features our current models have not adequately captured.

The strongest position seems to be one of disciplined uncertainty. Artificial consciousness should neither be presumed from increasingly sophisticated behaviour nor dismissed merely because the processes involved are unfamiliar.

What concerns me is not the attempt to create conscious machines. It is the temptation to treat the question as already settled by the success of our computational descriptions. Once we assume that consciousness is nothing more than the relevant information processing, its reproduction becomes primarily an engineering challenge.

But that assumption is itself a philosophical position. Building increasingly capable systems may provide evidence relevant to it without establishing that it was correct from the beginning.

The more interesting question may be what happens to our understanding of ourselves while this argument remains unresolved.

When the Model Looks Back at Us

Return to the person asking an AI about their difficulty maintaining close relationships.

The explanation they receive may be thoughtful, grounded in psychological research and genuinely useful. An unfamiliar term might help them recognise a pattern they have struggled to articulate for years.

But suppose they begin taking the machine’s interpretation as the authoritative account of who they are. Experiences that do not fit the classification become less significant. Ambiguities are resolved in favour of the model. The person’s own uncertainty gradually gives way to a vocabulary that feels increasingly definitive.

The philosopher Ian Hacking explored a related phenomenon through what he called looping effects. Classifications of human beings do not always remain detached descriptions of their subjects. People can encounter these classifications, interpret themselves through them and change their behaviour accordingly. Those changes may, in turn, influence how the classifications are understood and applied.

This was happening long before generative AI. Psychological categories, medical diagnoses, educational labels and social identities have always possessed the capacity to shape the people they describe.

What changes with conversational AI is the immediacy and apparent intimacy of the exchange.

A classification no longer needs to arrive through a formal assessment or an institutional authority. It can emerge during an ordinary conversation, expressed in the language of empathy and tailored to the details of someone’s life. The explanation can be revisited repeatedly, refined through further disclosure and incorporated into the running account a person constructs of themselves.

The machine may be helpful precisely because it is so accessible and responsive, helping someone find language for an experience they had never managed to express. Yet the same capacities create a subtle risk. An interpretation does not need to be false in order to become restrictive; treated as definitive, it can illuminate one aspect of a life while obscuring others.

What happens when the account of ourselves we encounter most frequently comes from systems trained to recognise and reproduce patterns in existing human descriptions?

We may become more adept at naming our emotional states while less willing to inhabit uncertainty about them. We may begin to approach difficult relationships as problems to diagnose and optimise, rather than encounters with other people whose perspectives cannot be fully anticipated. Even our attempts at self-understanding might become increasingly dependent upon the categories an artificial system can readily recognise.

This would not be the first time technology changed our understanding of the human person. Nor would it mean that earlier, supposedly unmediated forms of self-knowledge were necessarily more authentic. We have always understood ourselves through inherited languages, cultural expectations and the interpretations of others.

The difference is that these systems can now participate in the interpretation itself. They offer accounts of who we are, respond to our objections and help us construct narratives about our lives. The account becomes part of the very experience it was intended to explain.

And because the system is capable of remarkable linguistic fluency, we may overlook how much remains uncertain about the kind of understanding taking place on the other side of the conversation.

In this respect, the philosophical question of artificial consciousness has an unexpected practical counterpart. We may not know whether a machine has a private perspective, but we can observe how readily its interpretations become part of our own.

There is a certain irony in this. We have spent decades making machines more responsive to human language and behaviour. Now we are learning to describe ourselves through concepts that make our behaviour more intelligible to machines.

The danger is not simply that a model might misunderstand us. It is that we may gradually lose the confidence to recognise aspects of ourselves that its categories cannot comfortably accommodate.

Beyond the Reflection

None of this requires us to reject computational accounts of cognition or assume that human self-understanding enjoys some privileged immunity from abstraction. We misunderstand ourselves and one another constantly; our narratives are selective, our memories unreliable and our interpretations shaped by circumstances we barely recognise.

Indeed, this is partly why artificial intelligence can be so valuable. Scientific models reveal relationships that ordinary experience misses, and artificial systems can challenge our preferred stories, expose inconsistencies and offer perspectives we would not have considered independently. A useful explanation does not become worthless because it is incomplete.

The question is what authority we grant those perspectives, and whether we retain the capacity to examine them against the complexity of lived experience.

An explanation should make reality more intelligible without requiring reality to become smaller in order to fit it. Our models are most valuable when they remain open to correction by the lives and phenomena they seek to describe.

This brings us back to the peculiar historical circle through which we arrived here. Computers were made possible, in part, by isolating aspects of human reasoning that could be expressed as formal operations. Those machines subsequently helped transform scientific theories of cognition. Now their extraordinary capabilities are encouraging us to reconsider the nature of intelligence, subjectivity and personhood.

There is nothing inherently mistaken about that progression. Our inventions have always revealed things about us that we could not otherwise have understood. The telescope changed our conception of humanity’s place in the cosmos; the microscope revealed dimensions of life previously beyond perception. Computers may similarly expose limitations in our inherited assumptions about intelligence.

But there is a difference between allowing a new instrument to expand our understanding and allowing its operating principles to become the measure of everything we seek to understand.

Perhaps future machines will possess conscious experiences of their own. Perhaps they will reveal that some capacities we regarded as uniquely human never required consciousness in the first place. Both possibilities deserve serious consideration. Neither requires us to conclude that the human mind is exhausted by the functions we have learned to reproduce.

In The Word Without Flesh, I wondered what might become of language as it acquires increasingly autonomous forms of existence beyond its human speakers. Following that question back towards ourselves reveals another possibility. As our technologies become more capable of expressing, interpreting and organising human thought, we may become increasingly accustomed to encountering our own minds through the abstractions those technologies embody.

The most consequential transformation may not be the creation of an artificial consciousness, but a gradual change in what we recognise as consciousness in ourselves. We have built machines capable of reflecting astonishing aspects of human intelligence. We should be curious about what those reflections reveal, while remembering that a reflection, however convincing, cannot establish the full nature of the being standing before it.

Sources and further reading