Cover art for ‘08 – Problems’

08 – Problems

Artificial Creativity

Listen on Apple Podcasts Listen on Google Podcasts Listen on Spotify

References

  • David Deutsch, “The Beginning of Infinity”, chapter 1, “The Reach of Explanations”
  • Karl Popper, “The Unknown Xenophanes” (in “The World of Parmenides”)

If you have any questions or comments, please leave them here or tweet at me: twitter.com/dchackethal

Subscribe to this podcast via RSS Back to all episodes

Transcript

This transcript was generated automatically and may contain mistakes.

Welcome back, everyone. This is episode 8, and it’s about problems. The frogs are busy croaking outside, but I don’t mind if you don’t. We will discuss what problems are in the epistemological sense, what it means to solve a problem, and how this relates to the structure of computer programs and the encoding of knowledge in such programs. This is basically a clarification around episode 6 based on some comments of David’s, and it’s also really getting to the meat of the issue here, and it will allow me to point out that problems are really at the center of the creative program. And lastly, I want to give a couple of historical examples of problem solving that I think can be modeled in terms of functions.

As I’ve said in an earlier episode, a problem is not necessarily some tragedy or calamity. It isn’t even necessarily uncomfortable, although discomfort can itself be a problem worth exploring. A problem is when we encounter something that we cannot explain. Something happens that strikes us as weird or unexpected. Scientific discoveries often begin with the words, “Hmm, that’s odd.” I believe this realization is credited to Isaac Asimov.

So consider going about your day without encountering any problems. To be clear, this is technically impossible. As Popper said, life is problem solving, so the very fact that you’re alive means you’re constantly engaging in some sort of problem solving, even if it’s as mundane as simply staying alive, so heartbeat, breathing, etc., or walking towards food storage to eat something and solve the problem of finding the best path to get there. These things come so naturally to us that we may not be aware of them, but they constitute problem solving nonetheless.

Now, if life is problem solving, then a completely unproblematic state is death. This also means the origin of life had to do with the occurrence of the very first problem, which may have been the problem of how to replicate. It’s interesting to think about whether the problem of replication is the only possible first problem to exist, or if there could be other first problems as well. And the answer to this question may help us create life artificially. Maybe there are easier problems to create or solve than that of replication that could also be candidates for the beginning of an unending chain of problem solving. And if there aren’t, meaning an open-ended stream of problem solving always has to start with replication, that would be an interesting finding too.

In any case, I think this explanation of life as problem solving has some interesting ramifications for more traditional notions of what it means to be alive, which usually involve having some sort of metabolism, being composed of cells, etc. If it’s all about problem solving, we should consider viruses and indeed all replicators to be alive because they are constantly faced with the problem of how to replicate at the expense of their rivals.

So are viruses alive? Are memes alive? I don’t know. And perhaps we should consider our bodies alive on one hand and our minds alive in a sort of separate way, because minds are software and software is abstract, meaning it’s independent of its physical instantiation. And a mind, an AGI, instantiated on a computer, I would consider alive, even though it’s not made of cells. That AGI’s mind is also a whole biosphere of living beings if memes count as being alive. But I digress.

Having a, quote, unproblematic day simply means going about your day without encountering anything odd, unusual, or anything that would have piqued your interest. But most days we do encounter problems we are consciously aware of. Say you’re looking for your keys. You have a theory of where you put your keys and it doesn’t hold up. Or you have no theory of where you put your keys because you simply don’t remember. But some other theory tells you you need those keys, you may have a theory of where you usually put your keys, and it may not hold up either. So this is what a problem is, when you have two or more conflicting theories which are good explanations. In this case, one theory is that your keys should be a certain place, and the other theory is the interpretation of an observation of those keys missing and not being in that place.

Here’s another example. Let’s say you’re practicing a skill. For example, you’re playing chess and you want to become good at it. Then you may have an explanation of how good you want to be and why, and how good you currently are and why. If you’re not as good as you would like to be, that’s a problem. This can be reduced to a conflict between theories. The theory of how good you are may butt heads with the theory of how good you want to be. When you’re better than you want to be, well, that’s technically a problem in the sense that two theories are conflicting, but it’s not something you will think of as problematic.

So in other words, you have a conjecture theory that specifies whether an objective has been met or how nearly it’s been met, and when it hasn’t yet, that constitutes a problem. These specifications or objectives or criteria, whatever you want to call them, are themselves explanatory conjectures. This is important for the development of AGI because unlike AI, an AGI can change its objectives. There is no overarching one, except perhaps the objective of solving problems indefinitely. Problems are central because they are the starting point of any critical inquiry. Solving a problem means to create an explanation that doesn’t have the conflict between the original two or more good explanations. For example, you forgot your keys at your friend’s apartment. This is how knowledge grows. It means we’re getting closer to the truth because reality doesn’t have any such conflicts.

Now in episode 6, we tried out a conjecture which leveraged the fact that the creative algorithm we used to create new knowledge is the same as that to copy existing knowledge. And we combined this with the conjecture that knowledge is encoded in computer programs in the form of functions. If that is true, the problem of knowledge creation can be rephrased as the problem of function replication. We closed with the conjecture that if you succeed in building a program that can replicate any arbitrary function by invoking it and without reading its implementation, you may come a bit closer to having universal explainer. If you’re unsure about this, I recommend checking out episode six again before continuing.

There is a problem with this approach in that it can seem a bit like learning from data. As David points out in chapter one of “The Beginning of Infinity”, the very term “data” is misleading because what the term means, “givens”, is misleading. Nothing is given. Everything is creative conjecture. Also something the function replicator will need to do is disregard some of the return values it collects because it may have invoked the target function the wrong way. And occasionally it may even need to amend the data with whatever data should fit according to its best guess.

However, I want to point out that while in some cases this way of replicating functions can seem like learning from data, I don’t think it really is, at least not in the inductive sense. The conjecture theories still come first, and they have to, otherwise there’s nothing to invoke. And I think the way we find conflicts between explanations is that they make conflicting predictions when we expect those predictions to be the same. And this maps directly onto conflicting return values between functions where those return values should be the same.

Consider the example Popper gives of the motion of planets in his essay “The Unknown Xenophanes”. I hope I’m pronouncing this correctly. This is an example that shows how limited a role observations and data play in this process, but how nonetheless it can be modeled functionally in its entirety.

Kepler observed that the orbit of Mars deviated from the circular orbit that had been the prevailing model at the time. He could have chosen to disregard this observation, but he chose to interpret that deviation differently. And that’s because it was a systematic inaccuracy that was reproducible over several Mars years. This was unacceptable to Kepler, and so he looked for a better model of the orbits than the circular one. Popper goes on to say that, quote, “he felt the deviation from the circle to be deviation of the empirical world from mathematical purity”, end quote. And so Kepler temporarily abandoned mathematical shapes altogether and literally tried out vegetable shapes instead until he remembered that maybe an ellipse would fit and would still count as part of the world of mathematical purity. This illustrates that Kepler had several working theories in his mind that he had to reconcile.

The theory that planetary orbits were circular, the theory that circles were problematic, the theory that reproducibility of a deviation from the circular motion made it problematic, the theory that mathematical purity was something worth striving for, the theory that vegetable shapes would offer a decent replacement for mathematically pure shapes, and so on. Since these theories made conflicting predictions, he knew he had found problems, and had to proceed by the method of conjecture and refutation to come up with an explanation that wouldn’t have those problems. And ellipses did this. They fit the measurements, and they were mathematically pure.

I guess that all of these theories can be modeled as functions, and where their return values conflict, we have found a problem in the same way Kepler did. Note, however, that the notion of conflicting return values applies also to those theories whose return values do not constitute observations or data, such as the theory of the importance of mathematical purity, predicting that planetary orbits should follow mathematically pure shapes, and a function that represents this theory on a computer would make the same prediction, meaning we would find that prediction among its return values. I think this just illustrates that nothing is given. You always need to create a function from nothing first if you want return values. This is an act of creation.

Here’s another example with data playing an even smaller role. I think I remember reading this in one of Popper’s books, but unfortunately I can’t find the reference to it, so don’t quote me on it because I might butcher it. But I remember it going something like this. The heliocentric model, the idea that the earth revolves around the sun and not the other way around, was not proposed because of observation. If I remember correctly, it stems from a reinterpretation of some old texts of Plato’s. Plato suggested something along the lines of the sun physically representing the idea of the highest moral good. That means the sun is really important. And so it was suggested later, I forget by whom, that the sun be promoted to the center.

So in this case, the conflicting theories are the geocentric theory and Plato’s theory of the highest moral good. Again, please don’t quote me on this. In combination with the theory that something as important as the highest moral good should be placed at the center. I guess we could call this a sort of geometric-correspondence-to-importance theory. So the geocentric theory predicts that the earth is at the center, and the latter two theories combined predict that the sun is at the center. So that’s a conflict, so it’s a problem. And it was solved by rejecting the geocentric model. All of these theories made predictions, and we can make those predictions and look for problems by invoking those theories as functions and checking to see if they conflict.

Now, how high-level theories containing ideas like the highest moral good can be encoded as functions is still to be figured out. That’s a remaining problem. But according to the universality of computation, it must be possible. Our minds are doing it somehow, after all. This last theory has a bit of a moral overtone to it, which makes it interesting. And I guess that moral explanations like all other ones can be encoded in functions as well. The point of these two examples is to show that thinking of explanations as functions and invoking them to find conflicts in their return values works even for theories that have very little or nothing to do with observations or data.

Another thing to note is that a function’s return values aren’t necessarily raw data, or raw data in the sense of being our language’s primitive data type. They can be other functions, meaning other explanations, which means that we also need to be able to replicate higher-order functions. Higher-order functions are those functions which either take functions as parameters or return other functions, which we may need to invoke to find further conflicts.

As an aside here, it’s interesting to play with the idea of how this maps onto levels of emergence, and if the levels of emergence of an explanation correspond directly to the order of a function. This is just a hunch and highly speculative, but perhaps we could claim the following. If a function simply returns values, there’s no emergence. If a function returns a function which returns values, that’s one level of emergence, and so forth. And if a function returns multiple functions, the, let’s call it, deepest one determines the level of emergence. This is starting to look a little bit like a tree, perhaps a bit like Popper’s tree of knowledge, because our theories are all connected, and we can have theories about theories, and so forth.

The reason I think this is related to emergence is that oftentimes in software development, we can explain some functions well in terms of each other without having to split them apart into their individual components. And this is true in particular of higher-order functions such as sorting, mapping, filtering, etc. In any case, I expect most of our knowledge to reside at such higher levels of emergence, meaning it’s made up mostly of higher-order functions. This also applies to our old example of multiplication. We can explain multiplication well in terms of addition and looping without needing to go into detail about how those two components work. Otherwise, no function could ever depend on other functions, which would probably render the creative algorithm intractable, because for every problem, every single function would need to be rediscovered or recreated instead of being reused. But again, this is all highly speculative, but it’s fun to think about.

I’d like to suggest one more type of problem, where the previous type, that is conflicting ideas, is concerned with finding return values of two or more functions that are supposed to match but don’t, and this is the computational equivalent of finding predictions two explanations make that are supposed to match but don’t. I’d like to suggest problems relating to the structure of functions. Once an explanation is sufficiently explicit, we can investigate its structure. Not surprisingly, software engineers do this with their programs all the time because they know that the structure of their programs is just as important as meeting a specification.

Structural problems are problems of efficiency, clarity, and more, and of course, being hard to vary. This can also be stated in terms of one theory conflicting with another. In this case, our theory of, for example, how hard to vary our program is versus how hard to vary our program should be. But in this case, the conflicting parts are two program structures, not their return values. I have a hunch this is related to the topic of emergence in some way. Two program structures are being analyzed rather than primitive values. And perhaps we could envision a hard-to-vary function that takes another function and tells you how hard to vary it is. But I don’t know yet exactly how that relates.

So I hope this sheds some light on how important problems are and that they are the main driver behind the creative process. In that spirit, I want to close by pointing out the two main problems I see with the function replication approach, the way I outlined it in episode six. In one way, it is too restricted, and in another, it’s unrealistic. It’s too restricted in that it currently doesn’t allow clues about the structure of the target function. But when knowledge is sufficiently explicit, such as a teacher’s knowledge of how to multiply, the teacher can give hints about the structure of that knowledge too, not just example calculations. And the concept in its current form is unrealistic in that it is given, actually given, a target function. And we’re never given a target function in real life.

Nonetheless, we know that creating explanations is a replication effort. And my guess is that explanations and functions are the same thing. So the solution does have to do with replicating functions somehow. I think it needs more exploring.

In the next episode, I will most likely talk about the issue of specification. The whole point of the creative process is that we don’t specify any particular problem we want to solve or how we want to solve it in advance. And that makes it different from all other computer programs we’ve written so far. There’s a danger of leaking the programmer’s knowledge into the creative program and running the risk of only appearing to create new knowledge at runtime. And I’d like to shed some light on problems with evolutionary algorithms as they have been built up until now. I hope you’ll join me for that. And as always, thank you for listening.


What people are saying

What are your thoughts?

You are responding to comment #. Clear
Markdown supported. cmd + enter to submit. You have free speech here. You’re responsible for what you write. Terms, privacy policy
Your real name is preferred.
This small puzzle helps protect the blog against automated spam.

Sign your comment with GPG to create or add to a public profile with all your comments.

Paste a detached signature of your comment.

              
Paste your public-key block if you haven’t before. You consent to your key’s contents, including your name, being displayed to the public.

              

Preview

Playing ~