The problem, in a nutshell, is this: creativity is not predetermined. Computer programs, on the other hand, are completely deterministic. So how could creativity possibly be programmed? We know it must be possible, since we are each a living instance of the creative program—but how?

I was reminded of this problem while reading Chiara Marletto's new book.* She writes that much of physics focuses exclusively on what she calls the prevailing conception: a mode of explaining the physical world with the help of initial conditions and laws of motion. For example, if you know a particle's position and the laws governing its movement, you can predict its future position. But this mode is not the only mode. As Marletto writes:

The misconception is that once you have specified everything that exists in the physical world and what happens to it—all the actual stuff—then you have explained everything that can be explained. Does that sound indisputable? It may well. For it is easy to get drawn into this way of thinking without ever realising that one has swallowed a number of substantive assumptions that are unwarranted.

She goes on to say that one cannot explain what a computer is merely by listing the programs it runs–one also needs to state what programs it could run, even if such programs are never actually run. She distinguishes between the actual and the counterfactual. Counterfactuals play a crucial role in constructor theory, a field she and David Deutsch are pioneering.

There are other modes of explanation as well, such as the evolutionary one. Darwin's explanation of the origin of species makes no specific prediction about what organisms will form (future state) from current organisms (initial conditions) given some "laws of evolution". It could not possibly do that. Instead, it explains by what logic organisms can form.

The evolutionary mode of explanation is particularly important in artificial general intelligence (AGI), i.e., the study of creativity. We know thanks to the work of the 20th-century philosopher Karl Popper that creativity is an evolutionary program. There is literally non-biological evolution of ideas happening inside our minds. I have written about what that evolution may look like in some detail.

When taken outside the context of physics, the idea that there can be multiple modes of explanation conflicts with how we currently write computer programs. For example, if you write an algorithm that can add two numbers, it will follow the analog of what Marletto calls the prevailing conception in physics: the algorithm will take two numbers, i.e., two initial conditions, while itself being analogous to a law of motion. Given the initial conditions and the law of motion, you can calculate the result, i.e., the future state. Consider 2 + 2 = 4 as an example: 2 and 2 are the initial conditions, + is the law of motion, and 4 is the future state.

I will use the same term, prevailing conception, to describe how we write computer programs. And I contend that it is not only algorithms for addition that follow the prevailing conception, but all programs that have ever been written, and all that could be written. I believe this is just another way of stating computational universality.

If true, this constitutes a problem for AGI research, because it seems that writing programs under the prevailing conception precludes all creativity. Why? Because if every output and every state of the program are predetermined from the beginning, it would seem to follow that the program cannot possibly be creative. To be creative means to create new knowledge, and if the knowledge is truly new, it cannot have been predetermined—otherwise, merely by determining the new knowledge, you would already have created it. It's one thing to prophesy "tomorrow we will find the solution to the dark-energy problem", but quite another to say "that solution is X". If you already know what X is, you have found the solution today, not tomorrow, and you contradict yourself.

This problem—that specification precludes creativity—is the problem of specification. I wrote a bit about it in my book. If you specify everything the program can do in advance, it seems it can’t break out of that predefined space, and so it can never be creative (or conscious, have free will, etc., since these things seem to arise from creativity and creativity only). In addition, the purpose of specification is to instruct the program exactly as to what to do, and never to break with the instruction. It is a demand for success, a guarantee that the program will work. This again sounds counter to creativity, since more creative people think outside the box. They can break out of or vary the space of ideas in which they operate. Deutsch's and Marletto's breaking out of the prevailing conception and addition of a new mode of explanation in physics is a prime example of that. And there is no guarantee for success when it comes to creativity—luck is involved. AI research is currently abuzz with the pursuit of successful thinking, but that's not creativity. As Popper writes in his autobiography (translated freely back into English from the German translation):

What characterizes creative thinking, then, apart from the intensity of interest in the problem [one is working to solve], often seems to me to be the ability to break through the boundaries of—or vary the range of—the domain from which a less creative thinker selects his attempts. This ability, which is obviously a critical ability, can be described as critical imagination. [...]

Remarks like these, however, are hardly likely to satisfy those who are looking for a psychological theory of creative thinking [...]. Because what they are looking for is a theory of successful [...] thinking.

I believe that the demand for a theory of successful thinking cannot be satisfied. Nor is it the same as a theory of creative thinking. [...]

If it is true that all programs must follow the prevailing conception of programming, and that this prevailing conception precludes creativity, it would seem that we can never write a creative program! But that must be false since we each have a running instance of the creative program inside our heads. We know that it must be possible.

To solve this problem, consider these two simple guesses:

1. While it is true that no automation under the prevailing conception can be creative, it is false that all possible programs fall under the prevailing conception. Therefore, we need a new mode of computation, i.e., a new conception. OR

2. There is a kind of automation under the prevailing conception that produces emergent phenomena that constitute creativity.

Since they contradict each other, only one of these guesses can be true—and given that, as I said, we are each living instances of the creative program, it seems that one of them must be true.

Guess number 1 is admittedly a very bold conjecture: the prevailing conception of programming is, for all we know, universal. That means any program that could be written falls under the prevailing conception and there can be no other mode of computation. If that is true, guess number 1 is false and we need not look for a new such mode. We should keep in mind, however, that even our best theories are riddled with errors. Maybe there's something wrong with the universality of computation as we currently understand it. Maybe there are other modes of computation to be discovered. Maybe they'll involve counterfactuals, like in constructor theory. Or something completely new we haven't thought of yet, opening up whole new worlds for how to write programs.

Thought should be given to the possibility of guess number 1 being correct. If it is, all our attempts to write a creative program under the prevailing conception are doomed to fail. This would partly explain our long history of failure in the field of AGI, in addition to the missing philosophical breakthrough which, according to David Deutsch, must happen first.

The second guess—that there is a kind of automation under the prevailing conception that produces emergent phenomena that constitute creativity—is a bit more conservative because it doesn't clash with computational universality as we currently understand it. But it does clash with the notion that automation cannot be creative—that specification precludes creativity. I think guess 2 is true, in which case automation can be creative. It's just not creativity itself that's being automated or specified but something else on a lower level. The introduction of emergence makes this possible:

The code of the creative program itself does not specify creativity but something else on a lower level of emergence; creativity arises as an emergent byproduct of running that code.

It is not currently known what the code specifies. But whatever it is, the specification, i.e. automation, is mindless, because it's not the low-level automation that is the mind, but the amalgamation of properties that emerge from this mindless automation.

In addition to not breaking with computational universality, another strength of guess 2 is that it fits our experience: we are not consciously aware of how our minds work. If guess 2 is true, that's because the underlying algorithm is as mindless as any other automation. It is only the emergent properties that constitute creativity and therefore consciousness, and they don't have any direct access to the underlying program—even though it's the program that instantiates them. When a mind tries to understand how it itself works, it means the program's emergent properties are trying to reverse-engineer the program's specification.

Guess 2 also allows us to solve another problem. Deutsch writes:

[Unlike] any functionality that has ever been programmed to date, [creativity] can be achieved neither by a specification nor a test of the outputs. What is needed is nothing less than a breakthrough in philosophy, a new epistemological theory that explains how brains create explanatory knowledge and hence defines, in principle, without ever running them as programs, which algorithms possess that functionality and which do not.

Programmers mainly write their programs according to specifications and by testing the outputs. One may wonder what else there is to programming other than specifications and outputs. We've encountered one such thing already: emergent phenomena. Programs give rise to them while running, sometimes even before or after running. Emergent phenomena can be part of a program's specification, but they need not be. Examples of such emergent phenomena include a program's reach (cf. Deutsch's The Beginning of Infinity ch. 1), i.e., how many problems it can solve in addition to those it was written to solve. Reach isn’t specified anywhere in the program’s code, and it’s not part of the program’s outputs either. Let's say an addition algorithm is built for the parochial purpose of adding 1023 and 411, but it's written in such a general way that it works for any two positive integers (within the computer's memory constraints). That is part of the reach of that program. Again, the reach is not specified anywhere in the code—no line instructs the code explicitly to work for any two positive integers. Instead, the reach simply falls out of how the program is written. The programmer may have had a specification in mind that the algorithm should work for any two integers—and no doubt he usually would. But the point is that he need not have had this specification in mind at the time of writing the algorithm. Every possible program always meets infinitely many possible specifications while also violating infinitely many of them. A program's code could never transcend its specification—but its emergent properties can—and this, in principle, enables them to be creative.

Performance and memory characteristics present more examples of programs' emergent properties. There can also be visual ones, such as the visual patterns created by Conway's famous Game of Life.

While the existence of emergent properties of programs is well known among programmers, I suspect that they are understudied. From what we know about creativity, we can infer that the emergent phenomena must have specific properties themselves: for one thing, they need to have causal power because our ideas can cause us to act. For example, good performance characteristics have the causal power of attracting developers while bad performance characteristics may repel them. The creative emergent phenomena also need to be able to change at runtime because our thoughts, ideas, preferences, etc. change during our lifetime. And when, how, and whether they do change must be unpredictable in principle and must not be predetermined. Further, the emergent properties of creativity must only be instantiated at runtime, not before or after (though some of them may copy themselves to other computers or people and survive their original runtime, like memes do). Perhaps most importantly, the emergent properties need to be able to create more emergent properties. Simply making a program with static emergent properties isn’t hard. There are no programs that don’t have any emergent properties. But programmers will need to study emergence more if we’re to write creativity. It may help to build tools to inspect a program’s emergent properties somehow while programming.

The automation that gives rise to creativity must create emergent phenomena fairly reliably since we have a mostly coherent conscious experience. On the other hand, it must not create any particular emergent phenomenon reliably or predictably—that would preclude creativity, for if the emergent phenomenon, such as a thought, can already be predicted, the knowledge in it has already been created. That the automation must fairly reliably create emergent phenomena in general, but no particular emergent phenomenon, reminds me of Deutsch's stance on problems in The Beginning of Infinity: that problems, in general, are inevitable, while no particular problem is inevitable.

On the low level of code, there is no mind, no freedom and no free will—only blind subservience to instructions without room for deviation. The automation is mindless, and in this sense, the algorithm that gives rise to creativity is like all other algorithms. This way, we don't need to break with the prevailing conception or computational universality. But on a higher level of emergence, the mind, freedom, and free will do exist. It’s again analogous to physics: even though the laws of motion are deterministic, and therefore all future positions of every particle in the universe are also predetermined, including those of your brain, it is impossible in principle to predict what idea you will come up with next. Although the physical world is completely deterministic, the world of abstractions is not. There is room for such things as free will and creativity, and it’s thanks to the existence of higher levels of emergence (cf. Deutsch's The Beginning of Infinity, ch. 5). That is how we square the dichotomy between all possible programs being predetermined automations and creativity arising from a program.

So to build a bit on what Deutsch writes—that we need a theory which "defines, in principle, without ever running them as programs, which algorithms possess [creativity] and which do not": I believe we need a theory that explains which algorithms can have free, open-ended, unpredictable, and non-predetermined emergent properties, and how and why, while those algorithms' low-level instructions are deterministic, like those of all other algorithms (if guess 1 is indeed false).

If this "emergent approach" works, the program itself, i.e. the automation on the lowest, code level, can be written to a specification after all. But it must happen in such a way that the emergent properties of the program are not all predetermined. (I think Conway's Game of Life is, for example, a program whose visual emergent properties are completely predetermined.) If, on the other hand, guess 2 is false and 1 is true, maybe there can be no underlying specification for creativity after all—but as I have explained, it would mean there's something wrong with computational universality as we currently understand it. We'd need a radically different, yet-to-be-discovered way of writing computer programs.

If the emergent approach is true at all, it can shed some light on why we are conscious of some things and not others. We are never conscious of things that happen on the code level, and only ever of (some) of the emergent phenomena. One is never conscious of things one’s mind has automated, while one can be conscious of the emergent phenomena it is creating.

As next problems to work on, I suggest coming up with a list of other emergent phenomena programs do or at least can have. Then we can think about how these phenomena could change at runtime; how they could change unpredictably; and how they could create new emergent phenomena. I am not sure whether programs have been written whose emergent properties change at runtime—but it's hard to say without thinking of more emergent properties first. It's particularly interesting to consider self-replicating emergent phenomena since we know creativity is an evolutionary process. The variation and selection that self-replicating emergent properties undergo may create new emergent phenomena.

The emergent approach allows us to entertain the idea that, even when run on a reliable computer, a deterministic program, whose low-level states and outputs can all be specified and predetermined, can nevertheless be perfectly creative. From the emergent properties of creativity, we can work backward to what the underlying code must be.

What other emergent properties of programs can you think of? Leave your ideas in the comments below.

Thanks to Logan Chipkin for explaining laws of motion to me.

---

* I am still at the very beginning of Marletto's book, and it foreshadows that constructor theory has something to say about AGI—so, as I read on I may learn more that could change my views presented here.