Schemas and chunks: What needs to go in your memory

What you actually need to “remember”

Contrary to the impression you may get if you read popular books by cognitive psychologists, we know almost nothing about memory. We know quite a few things about what happens when people try to remember and how they forget, but exactly what happens in the brain when people try to memorise something is a complete mystery.

The most common way in which most people think about memory is as some sort of storage into which you reach with your mind and retrieve what you are looking for. But the truth is, we just don’t know if that is what memory is like. There are some aspects of memory that behave like storage, but there are others that are very different.

Pretty much everything that we say about memory is some kind of metaphor. Metaphors can be incredibly useful but we should be careful about drawing inferences from them. Here, I will use the metaphor of an inventory of mental schemas from which we retrieve chunks onto a sort of mental shelf where we can combine them into a useful product. But that is all it is - a metaphor.

The storage metaphor breaks down very often. Memory simply does not seem to behave like a storage of objects in many ways. For example, It does not seem to run out of space in a straightforward way. The more we put in, and the more we make connections between what we had placed there, the more seems to be able to fit in. Also, we can often retrieve things from the back of this metaphorical storage more easily than from the front. That is not how storage spaces work in the literal sense. So we must be incredibly careful about what we say when it’s time for things like long-term and short-term memory.

Mental representations or embodied mental schemas

While we know very little about the exact nature of memory, we know even less about what is actually stored in there. We do know what the things that come out look like: Words, memories, ideas, images, episodes, etc. But it is pretty unlikely, they are just stored somewhere in a neuron in a brain for us to retrieve. We wouldn’t know where to start finding the boundaries.

The problem with memory as storage is that we have no easy way to explain how those memories get into working memory. And we also do not know how the sort of practice we do, would put them in the storage in a way that is actually useful when we need them. What sort of things am I pulling out of my head as I am writing this sentence? What did it take for me to put them there? English is not my first language, so it took a long time to learn all the structures, but I also had to learn to type fast enough to do it, the rules of spelling and so on. Individually, it is a long list of things, yet, they seemingly just happen. We do not have a very good theory of what is happening here other than just waving a hand to say that we’re retrieving items.

One way of resolving this problem is thinking about what is in our brain as an inventory of embodied mental schemas. This may seem like an unnecessary diversion but it is actually important as we contemplate what we practice and why. But feel free to skip this section.

We need to answer three questions:

1. Why schemas instead of simple units
2. Why embodied instead of just plain concepts
3. Why an inventory instead of some structured storage

Why schemas

We tend to think that the words in our head are very simple. But they are not - they come with a lot of underlying knowledge. Adjectives are a useful example. We take the colour green. When I something important is happening when I say ‘green leaf’ or a ‘green car’. When I tear apart a leaf, I expect to still see the colour green. Yet, when I open the door of a green car, I expect to see no green at all. Green cars are green on the outside, leaves all the way through. To combine the colour green with the word car is very different from combining it with the word leaf.

What comes into play is our knowledge of the world. Leaves are green through and through, and cars are green only when painted. This is just one tiny thing about leaves and cars, yet, it would take pages to adequately describe it. But we understand both instantly and make the right inferences. So, a good way of thinking about this is as having relatively schematic outlines of things in our head that we can then use instantly but fitting multiple ones together into one.

The same goes for nouns, verbs and other things. All books are different but I don’t have a problem recognising all of them as book. Everybody runs differently (people, animals), yet I can immediately recognise something as running. So, what is in my head doesn’t make sense as a perfect picture, it is some sort of an outline.

Even grammar rules are such outlines. I don’t have a rule in my head that tells me Noun + Verb make a sentence (a dog runs). I have a schema that may look something like this: [[nouny thing][verby things]]. These schemas can get quite large but it seems we are able to use them. (There is a whole theory behind this called construction or cognitive grammar, but that is not too relevant, here.)

This is very useful because, when I practice, my aim is to create ever larger schemas that I can instantaneously combine as I perform. It makes no sense to develop structures that are too specific.

Let’s take this joke by the Comedian Mitch Hedberg: “I like escalators. They never break. They just become stairs.” What sort of information would you have to have stored in your head, how would you retrieve it, and combine it to understand this joke? You could perhaps come up with a way to explain one such joke. But there are thousands like this in English alone. In some way it is easier to imagine that we have schemas that we simply overlay on top of one another and a new image emerges. Like to prints on semi transparent paper.

Calling up such schemas and overlaying them on top of one another is what chunking inside working memory is. Or at least, it is a useful way to think about it that way. We actually have no idea how it happens in practice. If, instead, we think of this as computing some sort of sequence of instructions, we run into the problem is that that is what working memory is for. But we need something to present to working memory - and those are chunks composed out of embodied mental schemas.

Why embodied

Why not just talk about concepts we have in our head? Ericsson calls these things mental representations. But these things are neither pure representations nor purely mental. When we learn things, we involve our bodies in all sorts of ways.

Our bodies give us the basic framework for understanding spatial relations and all the other concepts derived from them like: inside, excluded, overarching, etc.

But that’s not really what matters here. What is really important is that most of the things we are learning with deliberate practice have a bodily component. When we write, we use our fingers and hands. But also our backs and our backsides. We experience tiredness, we occupy the space around us, we hear noises, etc. All of those are also involved in reading. When we learn a foreign language, we gesture, move our mouths, etc.

Of course, some skills require less action with our bodies than others. Running or lifting weights is mostly a matter of the body (although surprisingly also a very mental exercise) and chess a lot less (although surprisingly it has a bodily component). Some other less physical disciplines still require a of bodily involvement. Music is very much a bodily exercise.

Even seemingly purely mental skills like doing maths have this bodily component. Doing mathematics cannot be done without putting things on paper, standing around whiteboards, etc. Both chess and maths can be done in the head only but only up to a point and only after involving the body for a long time.

And we do know that when we practice, to develop these skills, we have to involve the body. So, it makes sense to assume that they are a part of the schemas. And we can improve how useful they are to us if we incorporate our bodies into them even more.

We can think of the embodied schemas as the sort of gestures we perform to solve normal problems. When we feel a mosquito on our face, we automatically touch the spot with out hand. It is an incredibly complicated gesture, but we do it automatically. Our aim is to develop such mental schemas that they start to feel to us as easy and natural as touch a spot on our face.

Why inventory

These schemas are not all alike. They vary in richness, some are static, some are dynamic, some are extremely specific and others very general. It is not clear how many we have and how many we need. But if we tried to make a list, we’d probably end up with millions. This sounds extreme but that is what Psyc project found when they tried to document knowledge in a way that would represent it to a computer.

So the question is, how do we keep them all in our head and in our bodies? The short answer is, we don’t know. But a good way is to think of them as an inventory of good available for sale in a large supermarket. There is no definite best way in which they are organised, their place changes based on the time of day or customer demand, some are more and some are less readily accessible.

We are constantly replenishing and refreshing that inventory but ultimately we have to spend money, effort, time and resources to keep it full and organised. What it is not is a box where we simply put one thing after another.

Of course, in some ways it is more like a giant warehouse than an inventory, we cannot see it all at once and sometimes things get lost, they fade or expire beyond usefulness.

And in some ways it is like neither of these things. There are no practical limits on how much we seem to be able to store and it is almost as easy to get to things at the back as it is at the front.

But nevertheless, it is useful to use a metaphor like inventory to help us.

More about chunks and mental representations

Fluent performance breaks down when the limits of working memory are reached. We can think of the limits roughly as a fixed number of chunks of information. And the only way to overcome the limit in the moment is to chunk information together into bigger chunks because there does not seem to be a limit on the size or richness of those chunks.

We know that it takes reflective and purposeful practice to develop the ability to work with these chunks. But a big question remains. What are the chunks like and where are they stored? The short answer is we don’t know. But one way to think about them (inspired by Anders Ericsson) is as ‘mental representations’.

Unfortunately, Ericsson or other researchers have not really had much to say about the nature of these representations. We just know something like them exists because we know when people focus on them, their performance improves. So provisionally, we can think of them as some structures in the mind that are available to working memory without any retrieval effort (look and know knowledge). This makes them different from more encyclopaedic knowledge we often have to work hard to retrieve from memory.

Mental representations is an unfortunate term for two reasons:

1. It implies that they are concepts to be learned about whereas they are structures to acquire through practice.
2. It implies that only ‘mind’ things are involved whereas these representations are really embodied. We can think of them perhaps even better as mental gestures.

But we need a term to distinguish between the chunks that we use during performance. And the long-term structures that we are developing through practice. The reason for this is that we can also create ad-hoc chunks (such as when remembering a phone number) which is different from the chunks we develop through practice.

What do ‘mental representations look like’?

We must not confuse mental representations with analytic categories. Analytic categories are the domain of conceptual learning. But mental representations are essential for procedural learning. And paradoxically, they are not always the same. With analytic categories, our aim is to have an accurate understanding of something, with mental representations, we are looking to develop fluent performance.

We can best illustrate it on the difference between knowing about the grammar of a language and knowing how to speak a language. Native speakers have no problem constructing complex sentences at fluent conversational speed but when you ask them how those sentences are put together, they usually give an answer that is different from that of a grammarian.

They are able to speak fluently by recombining chunks of language, rather than putting individual words together according to the rules of grammar. And these chunks are often very different from the categories found in a grammar book. For example, beginner readers often benefit from decomposing longer words into their parts. They learn that ‘teacher’ is the result of putting together ‘teach’ and “-er”. This makes it easier to read and write. But they may also use that same principle to learn how to read the word ‘corner’, even though ‘corner’ is not the result of putting together ‘corn’ + ‘-er’.

Mental representations that we develop to assist in fluent performance are often different from the conceptual representations of the same thing. Sometimes this is a small idiosyncrasy of the individual, but sometimes it is a necessary stage of development. Complete beginners in any field are often taught through great oversimplification.

This has important implications for learning.

1. It is not always necessary to start ‘at the bottom’ of a conceptual hierarchy.
2. It is not always necessary to learn ‘all the basics’ first.

For example, it is possible to learn read or write a language without learning to speak it first. You may need some basics of pronunciation to make it easier to sound out words, but you don’ have to take a whole conversational course. And equally, it is possible to learn to write an academic paper before learning to write a letter to a friend.

Our practice should be aimed at the demands of the task we are trying to develop fluency in, not at being able to replicate how the task is described in theory. And it may not always be immediately obvious what the demands of the task are or even what the task itself is.

For example, many people use gamified apps for language learning. They are motivated by rewards, positions on leaderboards, competition with friends. But this means that the task they end up practicing is succeeding at the game. They optimise their mental representations towards that task and become very good at it. But this very often does not translate into being able to speak.

Unless the practice is always closely linked to actual performance, success in practice will deceive us into thinking that we are improving at the thing we are trying to practice. We are improving but at the wrong task. We are getting better at succeeding in the practice task and that success may have very different needs than succeeding at the real task.

That’s why, it is essential that we get real feedback from real situations as soon as possible. We need to link the components we are practicing in with the larger context of performance. Often, ‘actually doing it’ may be the only solution. Athletes often perform better on courses where they had competed before. But this is not realistic in most situations, which is why visualisation or even verbal rehearsal is such an important part of practice.

In other words, dealing with the environment is a part of the task which is why it is important to practice in the context of performance. Practicing for an exam is best done under exam conditions. Practicing for a musical performance is best done with an audience. And so on.

There is a joke about a jazz musician who was approached by a fan after a great performance. The fan was shocked when they saw the musician was drunk. “How were you able to play so well when you’re drunk”, the fan asked. “Well, that’s because I practice drunk.” replied the musician. The lesson here is not about alcohol but about the importance of practicing for the actual task required for fluent performance.