Imaging the Future: The Eye, the “I” and AI

Future framing

1By way of visualizing the problem of imaging the future, let us imagine the following scene. A group of people are sitting around a table in what seems like a cross between a corporate meeting and university lab seminar, gazing at screens big and small. Their attention is on a moving image of a worm, overlaid with the graphic visualization of its movements. The two moving lines converge nicely. The group are looking ten seconds into the future—the worm’s future, that is. This is a restaging of the WormBot project (Pitt 2019), a digital simulation of the behavior of a nematode worm represented in the form of image-lapse images with a view to predicting the worm’s death, and the scene is unfolding as part of the first episode of the TV series DEVS, directed by Alex Garland (2020, FX on Hulu). The idea is to recreate a living organism in a computer system by rendering all of its neural connections in a digital form. After ten seconds the correlation between the simulated screen movement and its predicted graphic visualization is lost, but the professor-lookalike boss is impressed enough to elevate the young Russian AI programmer to the next phase of his company’s project: the eponymous DEVS. The project lab, hosted in a vacuum-protected Faraday cage in the middle of a forest, contains a hugely powerful quantum computer whose goal is to achieve for the billions of neurons in the human brain what the WormBot-lookalike did for nematode’s 302. The (mad) professor-boss, a deus-like character called Forest, has built his whole enterprise to generate an alternative version of reality online, one in which his wife and young daughter do not die in a car accident (inadvertently caused by him). In other words, Forest wants to overcome the causality of the physical world by creating one in which a different future can unfold—a future he will be able to inhabit together with his digitally restored family. The DEVS series deals with the age-old problem of determinism vs free will, packaging it as a fairly conventional story of a heterosexual white nuclear family (and its multi-ethnic romantic counterparts) experiencing life’s travails.

2What is of particular interest to me about DEVS in the context of this article is its visualization of the problem of predicting the future by capturing an image of it. The first step involves the mobilization of the power of the said quantum computer to render detailed images from a remote past. Borrowing from the past’s “multiverse” simulations allows the DEVS developers to obtain a rich enough data set to generate adequate images of the past: Jesus on the cross or prehistorical people making cave art. The next step will involve using the technology and data from Forest’s more recent past to render a new version of the future for him and his kin. I would like to propose we bracket for now the determinism debate and the rather implausible quantum frippery of the series’ plot to focus on the very idea of rendering the future as an image. For me, DEVS serves as a framing device for the key questions that will drive this article: Can you not just imagine but also image the future? Can you photograph it?

3To explore these questions, I will look at the temporality of the photographic image while engaging with the latest developments in AI-driven computational photography and machine vision—technologies which are premised on forecasting an image and then making it look real. Drawing on neuroscience research on the role of prediction in human consciousness, and on the application of machine learning to developing machine consciousness, I will then move on to a brief overview of the mobilization of photography and photo-rendering in predictive technology, in areas such as weather forecasts, stock markets, epidemiology, and consumer behavior. I will also consider the socio-political consequences of the wide implementation of predictive technology in our lives. Last but not least, I will explore the possibility of imaging and imagining better futures.

Time is just change when there is no one to track it

4The question about the possibility of photographing the future may seem both absurd and out-of-date—Marty McFly already videoed it back in 1985! —yet the premise behind it underpins many of the current developments in imaging technology, computation, and AI. Indeed, photography poses us with a temporal conundrum, both on the level of physics and common sense. As a medium, photography has typically been associated with the past: it has been constituted as a quintessential medium of memory, of time gone by, or even death. Logically, it is only “the present” that can ever be photographed, but the present itself is an illusive concept, unable to be captured in an image or thought about without simultaneously receding to the past. Photographic technology has always played with logic and physics to create a spectral ambience around its practices. We can think here of long-exposure photography, where several hours were required to obtain an image trace on the not-so-sensitive substrate in the early days of the medium, or about stop-motion imagining, intended to explicitly show the passage of time, be it over the period of one day or several years. There are also photographic dispatches from a pre-human past, such as the visual rendering of the data captured by a radio telescope, in June 2005, of solar dust cloud radiation in the Taurus Molecular Cloud, where the data represented an event that “took place in 1585, or thereabouts,” (Tagg 2008: 21) or even a “baby picture” of our universe, produced by NASA between 1990-1992 and showing the map of the cosmic microwave background radiation supposedly released as heat during the Big Bang.¹ The temporal determination of a photograph depends on the point of reference: specifically, on the positioning of the observer within the system. Time is what it (supposedly) is only for an observer whose life is grasped as a timeline; outside the (human) observer there is just change.

5In its computational guise, photography can therefore be said to have warped the arrow of time, displacing temporality as we know and sense with our bodies and minds with the machinic logic of multiplication and nonlinear choice. Reflecting on Google’s Pixel 3 smartphone and its sophisticated camera, Sy Taffel explains that a photograph is now an extraction of an image from a sequence, coupled with an averaging of that sequence (Taffel 2021). Or, as developer Vasily Zubarev (2020) puts it, “When you tap the ‘take a photo’ button, the photo has actually already been taken.” In the computational images of the world as deployed by the latest smartphone cameras, such as Google’s Pixel or Apple’s iPhone, the future literally influences the past.

6Even though it reaches beyond human scales, photography gains a symbolic value in our human-centered universe as a practice of stopping time—but also, perhaps more strongly, a practice of making time. Photography allows us to make cuts in the optic flow unfolding around us. Yet it is not so much the already well-studied relationship between photography and time that interests me here but rather, and more specifically, the relationship between photography and the future. I want to approach it by looking at the problem of prediction—and the mobilization of photography and other forms of imaging in constructing predictions in different disciplines—from cognitive psychology and neuroscience through to AI, machine learning, and computer vision. I am mindful here of Matthew Cobb’s suggestion made in The Idea of the Brain: The Past and Future of Neuroscience, that our scientific horizon at any given time depends on the technological metaphors we use, but also that those very metaphors reflect the technical knowledge of the time, its discourses and vocabularies. “[M]etaphors can flow both ways,” says Cobb (2020). Since its invention in the early nineteenth century photography has been intrinsically linked with neuroscience, both enabling the neuroscientific imaging practice and shaping the understanding of the working of neurological processes, from memory through to learning. Indeed, it was in terms of “imprinting” on the brain’s tissue and its neural extensions that early research in neurology explained perception and action in humans (Cobb 2020). This perhaps goes some ways toward explaining the key role of the concept of “the mental image” in cognitive psychology and neuroscience. Yet since the middle of the twentieth century the photographic discourse has been supplemented by concepts and metaphors from computing, describing the working of the neural system with reference to “feedback loops, information, codes and computation.” (Cobb 2020)

7Interestingly, Cobb points out that, even though we now “stand on the brink of understanding how patterns of activity in networks of neurons create perception,” some scientists “sense we are approaching an impasse in how we understand the brain.” (Cobb 2020) The computer metaphor in particular is said to have reached its limit, while the proliferation of brain data being accompanied by frequent resignation on the part of its human interpreters to its voluminous excess and unexplainability. It is therefore worth probing whether current changes to the photographic medium that are occurring in its encounter with computation, network technologies, and AI are altering in any way the neuroscientific horizon, and, in particular, our understanding of neural processes related to imaging and perception. What is the mental image today—and what can it become? Can we learn to see ourselves and our world better once we have changed our metaphors?

We need to go deeper

8The concept of the mental image, referring to quasi-perceptual experience which is pre-linguistic, and which forms the basis of what we now understand as consciousness, has a centuries-long history in Western epistemology. An aspect of representationalist thinking unfolding across centuries which posited that ideas were just pictures in our mind, the concept has now lost its literal connotations without challenging the implicit primacy of visuality in our understanding of knowledge production. Indeed “‘imagery’ has become the generally accepted term amongst cognitive scientists for quasi-perceptual experience in any sense mode,” with the term embracing auditory, olfactory, kinesthetic, and haptic images alongside visual ones (Nanay 2021). The fact that the image still constitutes a cornerstone of the neuroscientific rhetorical register delineates a particular conceptual trajectory for the discipline: it proscribes what can be seen and said, and how.

9In what follows, I want to look at the use of the concept of the image in the current theories of consciousness as outlined by two of its leading researchers who are also known for writing for wider audiences: neuroscientists Antonio Damasio and Anil Seth. Damasio and Seth are physicalists (aka materialists), yet in their respective theories they go beyond mere functionalism. This is to say, they recognize the role of the body and its situatedness in the world in the making of consciousness, a shared assumption that differentiates them from a variety of cognitive scientists currently involved in modeling human intelligence and human vision in machines. But they are also interesting for me because of the different rhetorical registers through which they articulate their positions, coming, respectively, from image theory and information science. In the further part of this article I will look at the role of images in Damasio’s and Seth’s respective theories of perception; at the link between perception, prediction, and imaging in those theories; and, last but not least, at the possibility of conceptualizing consciousness as a person’s orientation toward the future, which involves making images of that future.

10For years avoided by scientists as exclusively focused on an individual self and hence impossible to study objectively, consciousness has recently gained a new respectability as a domain of scientific inquiry, partly thanks to the encounter between neuroscience and the multidisciplinary field of Artificial Intelligence. Some go so far as to argue that work on Artificial Intelligence should in fact be recognized as an attempt to create artificial consciousness (Chrisley 2008; Buttazzo 2001; Chella and Manzotti 2017). Defined by Damasio (1999: 4) as “an organism's awareness of its own self and surroundings,” consciousness is broadly understood as a first-person phenomenon which unfolds as part of a private, first-person process to which we give the name “mind.” Damasio distinguishes between core consciousness, which is a sense of the here and now, and is also likely to be found in other animals, and extended consciousness, which spans one’s lifetime, requires memory and attention, and is enhanced by language. Interestingly, it is through references to the metaphorical apparatus of image making that Damasio explains the working of those two types of consciousness. He proposes that “core consciousness includes an inner sense based on images,” (125) and that those are images of feelings. Responding to stimuli, which are experienced by us as feelings, the brain makes patterns in the circuits of its neural cells. In line with the metaphoricity of the wider approach of this article, we could call these patterns “impressions” or “imprints.”

11Damasio concludes that to understand consciousness we primarily need to understand the problem of mental images—i.e., of “how the brain inside the human organism engenders the mental patterns we call, for lack of a better term, the images of an object,” (9) where that object may be material (a person, a place) or immaterial (a melody, a toothache). An image, in turn, stands for “a mental pattern in any of the sensory modalities, e.g., a sound image, a tactile image, the image of a state of well-being” (Ibid.). But then a shift happens in Damasio’s narrative, where he is getting his media purposefully yet somewhat unexpectedly mixed. He says: “the first problem of consciousness is the problem of how we get a ‘movie-in-the-brain,’” relying on what he himself describes as a “rough metaphor” (Ibid.). He also highlights that “the movie has as many sensory tracks as our nervous system has sensory portals—sight, sound, taste, and olfaction, touch, inner senses” (Ibid.). Some interesting forms of mediation are activated here, with the brain positioned as the cinematic screen. Also, an odd circularity creeps into Damasio’s argument as part of the mediation process, with images functioning as stand-ins for his (and everyone else’s) inability to clearly explain the sequence of events in the production of consciousness. Damasio goes so far as to acknowledge that we have to explain not only how the movie-in-the-brain is generated (assuming that there is a movie, of course) but also how the brain “generates the sense that there is an owner and observer for that movie” (11). If consciousness does behave like a movie, it is like a retro silent one, as language is only believed to arise at a “later” stage.

12The brain-as-screen is perhaps less like a conventional cinema screen and more like the 1950s Circarama, renamed Circle-Vision a decade later, with screens surrounding the viewer from all angles but also requiring the viewer’s corporeal movement to appreciate the experience fully. The viewer only becomes a viewer through their 360-degree mobility in the image space. This sense of the image envelope that requires mobile attention from the viewer returns in VR and 3D gaming, with the viewer’s/player’s positioning resembling an M. C. Escher drawing in which the movie dreams itself up as a movie, playing in a theatre with an audience. The viewer is thus both inside the image and watching the image. That sense of the self-generativity of experience from within the image envelope is something that is being used in current experiments with machine learning and AI that involve an attempt to recreate the human experience of perception—or rather the externally perceptible aspects of that experience—in machines.

13Google’s DeepDream algorithm developed in 2015 allowed for the production of images in which nonexistent objects were “discovered” by running previously trained neural network models on those images multiple times.² This resulted in infamous images showing spaghetti with eyes, camel-birds, and psychedelic squiggles covering familiar landscapes. This aesthetic, short-lived as it was, became known as “Inceptionism,” a term derived “from the Network in network paper by Lin et al. in conjunction with the famous ‘we need to go deeper’ internet mem e” (Szegedy 2014: 1). Lin et al.’s paper (2013) proposed a deep network structure called “Network In Network,” which was to be “used to enhance model discriminability for local patches within the receptive field.” The operations of recurrent neural networks are meant to recreate the (posited) experience of human perception and human learning, producing images and then feeding them into the network to create more—and more detailed, or “deeper”—images. The computational model of perception in AI research can be described as reductionist, as it brackets off both the human body and its movement in the environment as not so vital aspects of the perceptive process. This (il)logic is replicated in the current design of neural networks—algorithms whose name is strictly, and restrictively, metaphorical, as there is nothing “neural” about those networks. Yet what is modelled more accurately from the human organism is the sense of recurrence, or circularity, whereby the elements produced as part of the work of the machine’s posited “intelligence” are themselves producing that intelligence. (As mentioned before, consciousness would be a more apposite term here.)

14Bringing together all the constitutive elements, Damasio puts forward the following schema for the image-based operations of consciousness in humans:

The wordless narrative … is based on neural patterns which become images, images being the same fundamental currency in which the description of the consciousness-causing object is also carried out. Most importantly, the images that constitute this narrative are incorporated in the stream of thoughts. The images in the consciousness narrative flow like shadows along with the images of the object for which they are providing an unwitting, unsolicited comment. To come back to the metaphor of movie-in-the-brain, they are within the movie. There is no external spectator. (Damasio 1999: 171)

15A number of things are intriguing about this schema. First, it is the perceived need to split the process of the emergence of consciousness into two phases: the wordless narrative of images (even though it is still called a narrative, its role consisting in having to describe the object perceived or comment on images of that object) and a subsequent timeline of thoughts (which are not yet words). Second, it is the distinction between “images in the consciousness narrative” and “images of the object.” The first kind of images function like cybernetic mirrors: they produce consciousness by providing an image of the self (or, more precisely, of the self’s body) that is then being incorporated into the picture of the self. I am calling this mirror cybernetic because of the feedback loop effect required for its operation. Yet it is not a straightforward device that forms images by reflection. The cybernetic mirror producing consciousness by way of making images is more like the Magic Mirror envisaged by Escher, where the reflection becomes part of the scene being reflected. The two strands and two timelines of those images seem to overlap and merge, collapsing the odd Platonism of Damasio’s theater of consciousness in which shadows appear before objects into a more contemporary version of the screening venue: a movie theatre. The supposed cinematic experience is staged by “the brain” for us, an experience that is also said to generate “us” in the process.

16Yet it is worth delving a little deeper into the mixed media metaphors here and realize that consciousness does not really have the designed smoothness of a VR or immersive gaming experience. Damasio himself acknowledges that “Core consciousness is generated in pulselike fashion, for each content of which we are to be conscious” (Damasio 1999: 126). Indeed, we flit in and out of consciousness, both during our sleep (when it is only in the relatively short time of deep sleep that our consciousness is suspended) and in daytime. To continue with the media metaphor, consciousness is therefore perhaps more like an experimental modernist short, one that is made up of still photographs, multiple exposure frames, slow-mo animations, and material scratches to the film’s surface. The role of the body, foregrounded in the formal experiments of modernist filmmakers such as Sergei Eisenstein, Hans Richter, or Aleksandr Rodchenko, is crucial in the creation and, of course, perception of that movie. We could say that there is no movie without a body. As Damasio puts it, “the images you form in your mind always signal to the organism [which stands for a material bodily entity] its own engagement with the business of making images” (148). The human body therefore presents itself as a self(ie)-camera. The images are needed to generate us, but this sense of “us” emerges through our “action.” We become us by imaging and hence imagining ourselves into being, we could say. The image of “the organism” that we internally create for ourselves to map out (not yet quite) our external boundaries, and hence help maintain our life, becomes a blueprint for the imaging of the self that is subsequently seen as living that life.

17Importantly, this ur-image of the organism needs to be followed by many others. Indeed, in order to know how to act, we need what Damasio refers to as “good images.” Yet it is not only through somewhat confusing spatiality in which a reflection becomes an image that Damasio conveys the working of consciousness to us. His model also entails a warped temporality. While core consciousness, as he figuratively puts it, “does not illuminate the future,” (16) extended consciousness involves “holding an image over time,” (122) which is another name for the process of memory. Drawing on the famous experiment conducted in the 1980s by Benjamin Libet on the relationship between stimulus, reaction, and consciousness which showed a delay in our consciousness catching up with out body (Libet et al. 1983; Libet 1985), Damasio points out that “We are always hopelessly late for consciousness,” (1999: 127) the lateness amounting to about five hundred milliseconds. This presumed lateness has led philosophers and neuroscientists to interrogate whether our actions are indeed caused by brain activity or whether they are pre-programmed all along. In that latter scenario, consciousness only catches up with our activity five hundred milliseconds later and thus only provides a report on the activity, rather than being its source of origin. The varied responses in the science community to the relationship between consciousness, brain, and body reflect the diverse set of positions on consciousness today, which are in turn underpinned by diverse approaches to the age-old problem of determinism and free will.

18DEVS could be read as a parable for illustrating this problem. The orchestrator of the DEVS enterprise, mad-professor-cum-entrepreneur Forest, believes there is no such thing as free will. As mentioned previously, Forest is mourning his family’s death in a car crash, which was caused by his wife while she was on the phone to him—making him a contributor to the crash. Yet for Forest we all run on “tramlines.” This means that his wife and daughter died in that car crash because they were always going to die in it, with the tracks of causality extending well beyond and ahead of any individual’s actions or acts of will. The belief in the determinism of the universe provides solace to his guilt but also a recipe for trying to recreate another universe in which a different set of events would be guaranteed to unfold, one that would be more satisfactory to its designer’s wishes. Through a trick of smoke and mirrors involving a lot of misuse of the term “quantum,” things both do and do not go to plan. A young programmer Lily—a heroine on a quest to revenge the death of her Russian-spy boyfriend and save the world from Forest’s divine machinations, and one whose heroic adventure has already been predicted by Forest’s all-seeing computer—at the last moment she throws out the gun she was supposed to use to kill the evil devs-master, thus messing up the prediction algorithm. Yet both Lily and Forest do die in the altercation anyway once the cage that held them falls down, thus joining the rendered universe on the other side of the screen in which Forest is reunited with his family while Lily is dating her other (good) boyfriend. The philosophical dilemma as to whether they are still themselves in that rendered universe, or whether they are mere digital copies of themselves that just look like them to an outside observer, is closed off by the foundational assumption that drives the show (as well as much of current AI research) that both perception and consciousness are computational. For DEVS it also means that it is possible for the characters to experience themselves as “continuous and unified” not just across time (which we all do, every day, on waking up),³ but also across platforms and media.

19Commenting on the science behind the show, computer science professor Scott Aaronson is rather skeptical about DEVS’ overuse of the term “quantum” with reference to what is complexity. Indeed, complexity is the reason computer developers from the scene discussed at the beginning of this article are unable to see more than ten seconds into the nematode worm’s future. Aaronson then adds, acerbically: “Predicting the weather three weeks from now might be forever impossible” (Quoted in Lindbergh 2020).

The perception machine as a prediction machine

20A number of successful predictions do nevertheless happen everyday, albeit at smaller scales and temporal ranges, and they do have profound significance not just for how we can live our lives but also for how we can understand ourselves better. Yet their operating model is somewhat different than claimed by Libet and taken up by DEVS. Neuroscientist Anil Seth draws on the more recent research by Aeron Schurger to provide an interpretation of Libet’s experiment which suggests that the assessment of the brain’s “readiness potential” to initiate a given action was itself an artefact of the process of measuring this potential, and only something that could be captured retrospectively. According to Seth (2021), “you will see something that looks like a readiness potential if you look back in time from moments of fast responses.” This allows him to interpret the brain’s supposed readiness as an accumulation of sensory data from which the brain can make a best guess. The brain is therefore not ready for a specific action; it has to create this action by envisaging the external state of events—and those events’ unfolding. The act of envisaging itself is a simulation of probabilities, aka a prediction or “best guess,” not a foretelling of what will most definitely happen. This statement supports Seth’s account of consciousness as the organism’s ability to make predictions. We will return to this point shortly. But, for now, I wanted to focus on another aspect of Seth’s argument, namely, his conclusions that free will is a “perceptual experience,” a statement that allows him to reconcile our inner sense of agency with the causality of the laws of physics as we know them.⁴ Yet this is a stronger claim than might initially seem because for Seth the only way we can access anything, internally (i.e., ourselves) or externally (i.e., the world) is through our experiences of it, which are clusters of perceptions. We could go so far as to say, as Seth indeed does, that our sense of self is produced through the process of future-forwarding, i.e., looking at (what becomes) ourselves, at regular intervals, while trying to guess the causes of the sensory signals being received. For Seth, Descartes’ “I think therefore I am” becomes an “I predict (myself) therefore I am.” This argument is encapsulated by his proposition that our brains are (Bayesian) prediction machines, attempting to guess the causes of sensory signals from the perceptual inferences they receive.⁵

21I am interested in this formulation and the idea behind this concept of the prediction machine for two reasons: the posited role of perception in the constitution of the self and the figure of the machine in explaining consciousness. In Seth’s framework, prediction and perception are inextricably interwoven: “the brain is continually generating predictions about sensory signals and comparing these predictions with the sensory signals that arrive at the eyes and the ears—and the nose, and the skin, and so on.” His post-Cartesian quip could thus also be articulated as “I perceive (myself), there I am.” This theory confirms the constitutive role of perception in the emergence of both our self and of what we call the world. Yet, even though multisensory perception is such a strong aspect of his framework of thought, Seth is far less inclined than Damasio to resort to media metaphors, be they still or moving image-based, to explain it. Instead, his rhetorical register, with its signals, inputs, outputs, inferences, controls, and Bayesian guesses, is primarily drawn from cybernetics, information theory, and computer science. For Seth we are “beast machines—self-sustaining flesh-bags that care about their own persistence.” He traces his use of the concept of the machine to L’Homme machine (Machine Man) published in 1747 by Julien Offray de La Mettrie, explaining the workings of both the body and the mind solely in materialist terms, and thus overcoming Descartes’ infamous dualism that still shapes much of metaphysics, as well as most religious beliefs. Yet Seth’s model of consciousness and selfhood is not computational in the sense it is used in DEVS or applied in machine vision research. For him consciousness is phenomenological; it is always a consciousness of something, which means that it is embodied and embedded in the world.

22Echoing the sentiment well known to Marxists, visual culture theorists, and other humanities scholars for decades—namely, that “how things seem is a poor guide to how they actually are”—Seth’s theory redefines perception as “controlled hallucination.” The notion of systemic control is important to it, with conscious experiences being described, to use Giulio Tononi and Gerald M. Edelman’s term, as “informative and integrated” (Tononi and Edelman 1998). This sense of integration is important for the execution of the ultimate function of perception: enabling, via a series of prediction-driven actions, our survival in the world. We could perhaps suggest that the role of perception is to enable control over (what we might call, counter-Damasio) our self-image, or to go all the way down the imagistic metaphorical whirlwind, to photograph ourselves into existence. If memory, for Damasio as much as for Bergson, involves holding an image over time, selfhood—and self-consciousness, which is its foundational aspect—requires an incessant production of such images. Another way of putting it would be that we photograph ourselves into being. The model that emerges here is not that of a movie-in-the-brain, an avant-garde silent film, or even a poorly edited and choppy YouTube video, but rather than of a Polaroid print (albeit one updated for the digital age). But we also make the world by making images of it. This is to say that not only do we photograph ourselves into being, we also make the world into what it is (for us) by making images of it, over and over again. It needs to be acknowledged, as I have several times previously in this volume, that “stuff” (aka “matter”) does exist “out there” without our perception or other form of intervention, but for it to become what we see as and call the world, it needs this imagistic process to be instantiated. Needless to say, its outcome could look very different for different species.

23Seth explicitly rejects the idea the brain as a kind of computer in the skull, “processing sensory information to build an inner picture of the outside world for the benefit of the self” through feature detection. Nor does he partake of the image register to explain what happens, only resorting to the metaphor of “pictures” in order to dismiss the imaging model. So the brain is not an imaging machine for him, transmitting results from features detected in the world and thus producing their mere reflections. In Seth’s model, perceptions do not come directly from outside, i.e., from sensory impulses or inputs; they come from the brain’s predictions about the causes of these sensory signals. Building on the notion of perception as inference by nineteenth-century physicist and philosopher Hermann von Helmholtz, Seth puts forward his idea of a prediction machine, which is a creative and interpretative device, and not just a recording one.

24Yet this is perhaps too rigid an opposition given that prediction and imaging cannot be so easily decoupled, or that they are in fact being increasingly linked in various imaging technologies, from computational photography and CGI through to what has become known as “pred tech.” We are of course dealing here with an expanded understanding of imagining, and of photography in particular, one that involves a shift from seeing photography as passive transmission of what is out there in the world through to an active creation of “the world” through the conjoined human-machinic apparatus. Even if we are to agree that perception is not a reading of the world out there and that it is a “controlled hallucination,” a way of dreaming up the world on the basis of the “data” received from it, that proposition surely calls for an expanded sense of media through which such a hallucination can occur and present itself to us. Even though Seth explicitly rejects the model of the computer, his preference for information theory as the organizer of his metaphorical horizon effectively ends up turning the brain into a high-level processing unit that transforms sensory signal inputs into abstract predictions, and then spits them out as perceptions. What format and what medium these predictions come in remains unanswered in his theory. Seth emphasizes that “brains are not computers made of meat” but rather “chemical machines,” and that every brain is “part of a living body, embedded in and interacting with its environment—an environment which in many cases contains other embodied brains.” Putting the image of meat at the center of his argument, he does not follow through on its consequences. The linguistic paradigm, which is inadvertently introduced into Seth’s information-driven model – as evidenced, for example, in his claim that “We never experience sensory signals themselves, we only ever experience interpretations of them” – misses out on the creative possibility of the imagistic apparatus. What is more concerning for me is that his model also plays into the hand of those who want to posit prediction as a medium-independent process that can be done better and faster by machines, especially those of nonhuman variety. With this we enter the weird world of predtech: predictive technology.

Predtech and the capitalization of perception

25This development, long preceding advanced research in machine vision and machine learning, was already envisaged by French philosopher Paul Virilio in his book, Vision Machine. Virilio’s analysis was premised on developments taking place in AI in the 1980s, long before the emergence of deep learning and machine vision premised on “Inceptionism,” i.e., recursive neural nets. The angle of Virilio’s prophetic analysis was socio-political. He warned us against “the automation of perception” (Virilio 1994: 59), coupled with the production of synthetic imagery that had no indexical reference to anything we recognized as reality, beyond its verisimilitude. Virilio thus knew already in 1988 (they year the French edition of his book came out) that prediction was in fact a creation. The obfuscation of this fact by the photorealism of the media through which the forecasting is delivered has serious consequences for ourselves as both individual selves and political subjects.

26Virilio analyzed its working in military technology, with war maneuvers and wars’ outcomes increasingly being simulated on geographically-remote screens, with action on the ground serving as a singular actualization of the virtual possibilities ran on prediction machines in a different part of the globe. Today predtech is widely deployed in areas as finance, weather prediction, epidemiology and cancer detection, consumer behavior, and crime prevention. Combining data mining, neural network analysis, and visual rendering, it stages virtual scenarios while also making realities. To explain the working of predtech, marketing technology company Bluecore uses the example of a fuzzy picture which is gradually filled in with more data. Comparing traditional marketing to “taking a very zoomed in view of a photograph,”⁶ they pride themselves on using AI to provide “a bigger picture.” Yet the process of rendering this picture not only creates an image of their client’s customers but also creates customers: shaping their desires, choices, and actions while simultaneously rendering the very idea of a desirable customer bases for the client using Blucore’s predictive technology. For example, “you can use it to calculate a customer’s predicted lifetime value, helping your team focus on customers that will drive the most revenue long term.” This process is also in operation in other areas where the use resources, especially those of human nature, is to be optimized. “As of 2013, Hewlett-Packard was predictively scoring its more than 300,000 workers with the probability of whether they’d quit their job—HP called this the Flight Risk score, and it was delivered to managers” (Quoted in Siegel 2020). Predtech therefore not only predicts what is going to happen but also makes things happen. Not only can it create “bad workers” by reporting on them as potentially less reliable and steady in advance of anything actually happening while also shaping punitive management tactics, it also designates certain areas and certain groups of people as less desirable and less valuable, as poorer, weaker, and more prone to crime.

27And thus machines photograph the future for us, imaging into existence through publicity and simulation both our desires and the world to come. It was recently reported that AI will not “just help us find things, it will generate what we’re actually seeking” (Quoted in Prevost 2021). Even though the generation of new images through compositing has shaped photographic practice since its inception, with AI-enhanced computation the technique has shifted to the center of image production, while not requiring any expertise from the user anymore. Scott Prevost of Adobe Sensei, an AI and machine learning section of the creative imaging software company, explains that soon the search function “will be able to apply machine learning to blend assets and create an image that never existed before—an image that’s exactly what you had in your imagination.” (Ibid.) Although presented as a useful feature for all sorts of creatives, it is easy to envisage this technology being utilized to enhance the (mal)functioning of platform capitalism. Following Bernard Stiegler, Taffel describes this process, “whereby knowledge is displaced from humans into machines designed to commodify, privatise and monetise data, information and experience,” (Taffel 2021: 9) as a proletarianization of the user. The impending metaverse as envisaged by the tech companies will perhaps be a final installation in this attempt to not only fully automate our perception but also commodify it.

28 Of course, things do not have to be this way. Prediction need not be seen as a passive enterprise, an impassionate observation of what will have happened. With AI-driven computational photography burying for good the medium’s indexical fantasies, we can instead explore the idea of photographing the future as a form of creation, an imaginative rendering of a future we would like to see. This open-ended concept of the future is already (albeit implicitly) embraced by Adobe, Meta, and other giants of the tech industry, although, in supporting a particular form of popular and populist data exchange on their platforms, they end up consolidating to the contrary belief that “the future is inevitable, something that can at best be predicted,” a belief that of course works in the service of those companies. Yet, as poignantly phrased by Nick Montfort, “The future is not something to be predicted, but to be made” (Montfort 2017: XII). We must not therefore just respond to the tech companies’ visions and vistas but offer a vision our own—an approach Montford describes as “future-making.” John Norton, a technology writer for The Observer as a self-confessed recovering utopian, recognizes that if we want to make things better, our focus has to be on changing the machine’s purpose and obligations,” (Naughton 2021) —with “the machine” standing for him not just for the tech companies of Silicon Valley, but also for their financial enablers, political endorsers, and infrastructural supporters. We must therefore change the key parameters of the prediction machine, while making use of its warped temporality. Photographing the future can become a way of not just seeing what will happen but of creating images of what we want to happen—and of how we want the world to look—and making better choices from within the image stack. This shift is to take cognizance of the fact that computational imaging is always a form of buffering, with a singular photograph being just a cut in a range of possibilities. It therefore entails the possibility of making better selections and better cuts.

References

1. Buttazzo, Giorgio: Artificial Consciousness: Utopia or Real Possibility? In: Computer 34 (2001): 24–30.
2. Chella, Antonio and Riccardo Manzotti (2017): Artificial Consciousness. London and Charlottesville: Imprint Academic.
3. Chrisley, Ron (2008): Philosophical Foundations of Artificial Consciousness. In: Artificial Intelligence in Medicine 44: 119–137.
4. Cobb, Matthew (2020): The Idea of the Brain: The Past and Future of Neuroscience. London: Profile Books. Kindle edition.
5. Damasio, Antonio: The Feeling of What Happens: Body, Emotion and the Making of Consciousness. San Diego, New York, London: Harcourt Inc.
6. Libet, Benjamin (1985): Unconscious Cerebral Initiative and the Role of Conscious Will in Voluntary Action. In: The Behavioral and Brain Sciences 8 (4): 529–566.
7. Libet, Benjamin et al. (1983): Time of Conscious Intention to Act in Relation to Onset of Cerebral Activity (Readiness-Potential): The Unconscious Initiation of a Freely Voluntary Act. In: Brain 106 (3): 623–642
8. Lin, Min, Qiang Chen and Shuicheng Yan (2013): Network In Network. In: CoRR, abs/1312.4400, 2013
9. Lindbergh, Ben (2020): Making Sense of the Science and Philosophy of ‘Devs’. In: The Ringer, April 10. https://www.theringer.com/tv/2020/4/10/21216149/devs-hulu-quantum-physics-philosophy-alex-garland
10. Montfort, Nick (2017): The Future. Cambridge, MA: MIT Press.
11. Nanay, Bence (2021): Mental Imagery. In: The Stanford Encyclopedia of Philosophy. Edited by N. Zalta. https://plato.stanford.edu/archives/win2021/entries/mental-imagery/.
12. Naughton, John (2021): Forget Zuckerberg and Cook's Hypocrisy: It’s Their Companies That Are the Real Problem. In: The Guardian, February 6. https://www.theguardian.com/commentisfree/2021/feb/06/mark-zuckerberg-tim-cook-facebook-apple-problem
13. Pitt, Jason N. et al. (2019): WormBot, an open-source robotics platform for survival and behavior analysis in C. elegans. In: GeroScience 41 (6): 961–973. doi:10.1007/s11357-019-00124.
14. Prevost, Scott (2021): Making Visual Search Smarter: How AI Understands Creative Intent. In: Petapixel, March 29. https://petapixel.com/2021/03/29/making-visual-search-smarter-how-ai-understands-creative-intent/.
15. Seth, Anil (2021): Being You: A New Science of Consciousness. London: Faber & Faber. Kindle edition.
16. Siegel, Eric (2020): When Does Predictive Technology Become Unethical? In. Harvard Business Review, October 23. https://hbr.org/2020/10/when-does-predictive-technology-become-unethical.
17. Szegedy, Christian et al. (2014): Going Deeper with Convolutions. Eprint from arXiv:1409.4842, submitted September 17.
18. Taffel, Sy (2021): Google’s Lens: Computational Photography and Platform Capitalism. In: Media, Culture & Society 43 (2): 237–255.
19. Tagg, John (2008): Mindless Photography. In: Photography: Theoretical Snapshots. Edited by J. J. Long, Andrea Noble and Edward Welch. London and New York: Routledge, 16–30.
20. Tononi, Giulio and Gerald M. Edelman (1998): Consciousness and Complexity. In: Science 282 (5395): 1846–1851.
21. Virilio, Paul (1994): The Vision Machine. Translated by Julie Rose. Bloomington and Indianapolis: Indiana University Press.
22. Zubarev, Vasily (2020): Computational Photography Part I: What Is Computational Photography? In: dpreview, June 3. https://www.dpreview.com/articles/9828658229/computational-photography-part-i-what-is-computational-photography/2.