Associative Pattern Memory

Larry Werth

Netwerth Inc.


Human perception would seem to be a very complex process. We receive an immense quantity of sensory input that we learn to manage over years of exposure. Our ability to achieve this is largely based on our capacity for associative pattern memory. We learn to associate patterns of stimulus (visual, auditory, tactile, and other) with appropriate responses, represent them in memory, and form our worldview. Could this apparently complex process be based on some simple rules? Are there simple encoding mechanisms that formulate the language of the brain?

One such possible encoding mechanism that has been observed in the brain is reverberations. Basically, reverberations are multiple states of neural activity that continually repeat to form a dynamic steady state cycle. Cyclic behavior is certainly exhibited by finite cellular automata. Described herein is an implementation of associative pattern memory (APM) that is based on properties of cycles generated by finite cellular automata.

In this APM implementation a state is represented by a memory address, for example, a position in a two-dimensional array. Therefore cycles are simply a repeating sequence of addresses. The cellular automata implementation is somewhat different. Rather than apply rules to a previous state to generate each next state, each state is mapped to a specific small random region within the input pattern. Depending upon the feature extracted from this region a random next state is generated. All next states (one for each feature type) and all sampled regions are assigned to each state at random upon initialization of the APM. The process continues from state to state, sampling regions of the pattern until it ultimately generates a cycle. By controlling the minimum length of a cycle one can assure with high probability that only one cycle could exist for any given pattern independent of the initial state. Therefore a pattern can be represented by a specific set of randomly distributed addresses in memory that are repeatedly accessed by the cycle.

Representing patterns by a set of random memory addresses offers many advantages:
   *The process is very fast and simple since the cycle can be generated with almost no computation.
   *The number of memory addresses (states) in cycles is small compared to the total number of addresses, resulting in a large capacity of recognition within a given memory. In one application over 8 million different patterns were recognized using an array of only 64K.
   *Address cycles form a common language between patterns. Representations from different sensory inputs or from different patterns in time can be superimposed onto another memory to implement spatial and temporal integration, forming a new higher-level pattern to be recognized.
   *Unlike hashing codes, the APM can cluster similar patterns to the same response. Similar patterns produce similar cycles.
   *Assigning responses (training) involves writing responses to the memory locations accessed by the cycle and recognition involves reading the responses that have previously been trained.

Cycles (reverberations) may be a convenient means of communication within the architecture of the brain. Given the brain’s relatively fixed physical architecture, information would probably have to be represented by relative levels of activity and the physical position of that activity. Cycles naturally generate a high degree of activity at specific physical positions. Specific neural architectures may serve to communicate the positions of elevated activity from lower levels to higher levels of integration. Cell architectures such as pyramid cells and cortical columns of cells that run perpendicular to the discernible layers of the brain may serve to sense the higher activity and superimpose the positions of this elevated activity onto a higher-level plane.

The APM is currently implemented in a library of functions written in C/C++. Six patents (others pending) have been granted on the APM and two successful startup companies have been based on it. My presentation will provide implementation and performance specifications of the APM and the broad range of its potential applications.