Loose Notions

This is from an essay by Douglas Hofstadter that was delivered as a Stanford Presidential Lecture; it was also previously published in: The Analogical Mind: Perspectives from Cognitive Science, edited by Dedre Gentner, Keith J. Holyoak, and Boicho N. Kokinov (MIT Press). That book might be worth checking out.

This paper is quite long, and not all of it is that interesting, so the entire article is not pasted below. Part of the introduction nicely summarizes the thesis:

One should not think of analogy-making as a special variety of reasoning (as in the dull and uninspiring phrase “analogical reasoning and problem-solving,” a long-standing cliché in the cognitive-science world), for that is to do analogy a terrible disservice. After all, reasoning and problem-solving have (at least I dearly hope!) been at long last recognized as lying far indeed from the core of human thought. If analogy were merely a special variety of something that in itself lies way out on the peripheries, then it would be but an itty-bitty blip in the broad blue sky of cognition. To me, however, analogy is anything but a bitty blip — rather, it’s the very blue that fills the whole sky of cognition — analogy is everything, or very nearly so, in my view. [...]

The thrust of my chapter is to persuade readers of this unorthodox viewpoint, or failing that, at least to give them a strong whiff of it. In that sense, then, my article shares with Richard Dawkins’s eye-opening book The Selfish Gene (Dawkins 1976) the quality of trying to make a scientific contribution mostly by suggesting to readers a shift of viewpoint — a new take on familiar phenomena. For Dawkins, the shift was to turn causality on its head, so that the old quip “a chicken is an egg’s way of making another egg” might be taken not as a joke but quite seriously. In my case, the shift is to suggest that every concept we have is essentially nothing but a tightly packaged bundle of analogies, and to suggest that all we do when we think is to move fluidly from concept to concept — in other words, to leap from one analogy-bundle to another — and to suggest, lastly, that such concept-to-concept leaps are themselves made via analogical connection, to boot.

This essay reminds me of two things: the Barry Mazur essay on category theory that I posted a few days ago, and Jeff Hawkins’ On Intelligence. (Warning: Hofstadter uses the word “category” a lot in his essay, but this has nothing to do with category theory.)

Analogies and Functors

There is an obvious analogy to be made between functors (or any morphisms) and analogies. A functor translates the objects and relationships from one mathematical theory to another; the canonical example is that of the fundamental group functor in algebraic topology, which lets us turn problems of topology into those of group theory. What is this if not a rigorous kind of analogy?

The Hofstadter paper is interesting in that it proposes an extension of this relationship. In particular, Mazur talks about replacing a mathematical object with its network of relationships, while Hofstadter talks about replacing concepts with their “bundle of analogies,” which is more or less the same idea. Recall what Mazur says:

The lights are dimmed on mathematical objects and beamed rather on the corresponding functors; that is, on the networks of relationships entailed by the objects. The functor has center stage, the object that it represents appears almost as an afterthought.

What Hofstadter proposes in his paper is to similarly dim the lights on “concepts” and beam them instead on their networks of analogies; analogies should be center stage when discussing cognition in the same way that functors are frequently center stage when discussing mathematics.

Is there anything deeper to this connection? I don’t know. At least if it’s a superficiality, it’s a neat one.

Analogies and Hierarchical Temporal Memories

It would be interesting to go back to On Intelligence and see how much of this fits with Jeff Hawkins’ theory of how the brain works. At first glance, many of the passages from the essay seem to agree with Hawkins’ ideas. I’m not going to compare them or review them in detail here, but just quote some relevant passages from both.

In this passage, Hofstadter discusses something that he calls “chunking,” which has obvious similarities to the hierarchy in the perceptual system that Hawkins describes.

We begin with a couple of simple queries about familiar phenomena: “Why do babies not remember events that happen to them?” and “Why does each new year seem to pass faster than the one before?”

I wouldn’t swear that I have the final answer to either one of these queries, but I do have a hunch, and I will here speculate on the basis of that hunch. And thus: the answer to both is basically the same, I would argue, and it has to do with the relentless, lifelong process of chunking — taking “small” concepts and putting them together into bigger and bigger ones, thus recursively building up a giant repertoire of concepts in the mind.

How, then, might chunking provide the clue to these riddles? Well, babies’ concepts are simply too small. They have no way of framing entire events whatsoever in terms of their novice concepts. It is as if babies were looking at life through a randomly drifting keyhole, and at each moment could make out only the most local aspects of scenes before them. It would be hopeless to try to figure out how a whole room is organized, for instance, given just a keyhole view, even a randomly drifting keyhole view.

Or, to trot out another analogy, life is like a chess game, and babies are like beginners looking at a complex scene on a board, not having the faintest idea how to organize it into higher-level structures. As has been well known for decades, experienced chess players chunk the setup of pieces on the board nearly instantaneously into small dynamic groupings defined by their strategic meanings, and thanks to this automatic, intuitive chunking, they can make good moves nearly instantaneously and also can remember complex chess situations for very long times. Much the same holds for bridge players, who effortlessly remember every bid and every play in a game, and months later can still recite entire games at the drop of a hat.

Here, Hofstadter discusses the disconnect between sensory input and high-level perception. This is consistent with what Hawkins says would happen in an HTM at the higher levels of the hierarchy.

In fact, I should stress that the upper echelons of high-level perception totally transcend the normal flavor of the word “perception,” for at the highest levels, input modality plays essentially no role. Let me explain. Suppose I read a newspaper article about the violent expulsion of one group of people by another group from some geographical region, and the phrase “ethnic cleansing,” nowhere present in the article, pops into my head. What has happened here is a quintessential example of high-level perception — but what was the input medium? Someone might say it was vision, since I used my eyes to read the newspaper. But really, was I perceiving ethnic cleansing visually? Hardly. Indeed, I might have heard the newspaper article read aloud to me and had the same exact thought pop to mind. Would that mean that I had aurally perceived ethnic cleansing? Or else I might be blind and have read the article in Braille — in other words, with my fingertips, not my eyes or ears. Would that mean that I had tactilely perceived ethnic cleansing? The suggestion is absurd.

The sensory input modality of a complex story is totally irrelevant; all that matters is how it jointly activates a host of interrelated concepts, in such a way that further concepts (e.g., “ethnic cleansing”) are automatically accessed and brought up to center stage. [...]

The triggering of prior mental categories by some kind of input — whether sensory or more abstract — is, I insist, an act of analogy-making. Why is this? Because whenever a set of incoming stimuli activates one or more mental categories, some amount of slippage must occur (no instance of a category ever being precisely identical to a prior instance). Categories are quintessentially fluid entities; they adapt to a set of incoming stimuli and try to align themselves with it. The process of inexact matching between prior categories and new things being perceived (whether those “things” are physical objects or bite-size events or grand sagas) is analogy-making par excellence. How could anyone deny this? After all, it is the mental mapping onto each other of two entities — one old and sound asleep in the recesses of long-term memory, the other new and gaily dancing on the mind’s center stage — that in fact differ from each other in a myriad of ways.

Below, Hofstadter makes some comments on the common core underlying various things out in the world, which gels with Hawkins’ emphasis on “discovering causes.”

I now make an observation that, though banal and obvious, needs to be made explicitly nonetheless — namely, things “out there” (objects, situations, whatever) that are labeled by the same lexical item have something, some core, in common; also, whatever it is that those things “out there” share is shared with the abstract mental structure that lurks behind the label used for them. Getting to the core of things is, after all, what categories are for. In fact, I would go somewhat further and claim that getting to the core of things is what thinking itself is for-thus once again placing high-level perception front and center in the definition of cognition.

For comparison, consider the following paragraph from Numenta’s whitepaper on how HTMs work. The connection to the passage directly above should be fairly clear.

The HTM receives the spatio-temporal pattern coming from the senses. At first, the HTM has no knowledge of the causes in the world, but through a learning process that will be described below, it “discovers” what the causes are. The end goal of this process is that the HTM develops internal representations of the causes in the world. In a brain, nerve cells learn to represent causes in the world, such as a cell that becomes active whenever you see a face. In an HTM, causes are represented by numbers in a vector. At any moment in time, given current and past input, an HTM will assign a likelihood that individual causes are currently being sensed. The HTM’s output is manifest as a set of probabilities for each of the learned causes. This moment-to- moment distribution of possible causes is called a “belief”. If an HTM knows about ten causes in the world, it will have ten variables representing those causes. The value of those variables – its belief – is what the HTM believes is happening in its world at that instant.

While none of this is necessarily that unexpected, and I may be making a mountain out of a molehill, it’s still interesting that so much of what they say appears to overlap. It seems that Hofstadter agrees with Hawkins about what the core activity of the brain is, and in particular, he wants to call that activity analogy-making. (Perhaps Hawkins mentioned analogies in his book too, and I’ve simply forgotten.) In any case, I need to think about it more; I’m still not sure whether this is something or nothing.