Dynamic Memory

The problem

For most models today, knowledge lives in two places. What a model learned in training is fixed once training ends. What it is told in a conversation is gone when the conversation ends.

Little in between learns from the work itself. Each session starts afresh, the same context has to be supplied once more, and what was understood yesterday has to be rebuilt today.

Systems that add memory usually add a separate store the model must stop and consult, and what such a store holds is often hard to inspect.

What it is

Dynamic Memory is a third kind of memory, between what a model learned in training and what it is told in the moment.

It organises itself as the work goes on, and keeps what it holds open to inspection.

It is built to cost little enough to stay on throughout, because a memory that must be switched off to save cost cannot be depended on.

What it is for

Dynamic Memory is for a model that improves with use. Its understanding of the work builds rather than resets, so that what it offers later draws on what came before.

Because the memory stays open to inspection, what the model has come to know can be seen rather than taken on trust.

Where it stands

Dynamic Memory is in active research in the Athena programme. Its principle has been shown, and the work now is building it into a memory that runs continuously in real use.

Our wider memory research is described on the Memory page.