A data object is constructed from a P by M Wurfelspiel matrix W
by choosing an entry from each column to construct a sequence A0A1•AM-1. Each of the PM possibilities are designed to correspond to the same category according to some chosen measure. This matrix could encode many types of data.
(1) Musical fragments, all of which evoke sadness;
each column entry is a 4 beat sequence
with a chosen A0A1A2 thus 16 beats long (W is P by 3).
(2) Paintings, all of which evoke happiness; each column entry
is a layer and a given A0A1A2 is a painting constructed using these layers (W is P by 3).
(3) abstract feature vectors corresponding to action potentials
evoked from a biological cell's exposure to a toxin.
The action potential is divided into four relevant regions
and each column entry represents the feature vector of a region.
A given A0A1A2 is then an abstraction of the excitable cell's output (W is P by 4).
(4) abstract feature vectors corresponding to an object such as
a face or vehicle. The object is divided into four
categories each assigned an abstract feature
vector with the resulting concatenation an abstract representation of the object (W is P by 4).
All of the examples above correspond to one particular measure
(sad music, happy paintings, an introduced toxin, an object to recognize)and hence, when a Wurfelspiel matrix is constructed,
relevant training information for recognition is encoded that can be used in many algorithms. The focus of this paper is on the application of these ideas to automatic target recognition (ATR). In addition, we discuss a larger biologically based model of temporal cortex polymodal sensor fusion which can use the feature vectors extracted from the ATR Wurfelspiel data.
Biosensors could consist of hybrids such as a biological nerve cell grown on a suitable silicon substrate. We will assume a hybrid system
consisting of a dendritic tree for input, a cell soma and an axon for output transmission. Such a system is almost achievable with current technology. We will discuss how to model the action potential of the nerve cell in such a hybrid system so that we
can efficiently recognize toxins introduced on the input side (the dendritic subsystem) from changes we observe on the output side. We first discuss an abstract model of how a given toxin would influence the structure of the action potential of a biological nerve cell.
It is known that the action potential of such a cell is influenced at several times scales: (1) milliseconds: changes in ion flux due to
alterations in standard Hodgkin - Huxley voltage activated gates and (2) tens to hundreds of milliseconds: changes in the structure of ligand operated gates due to the creation of new proteins via requests to the nerve cell's nuclear material (genome). The classical Hodgkin - Huxley model consists of a number of nonlinear gating coefficients that give rise in even a simple model to 38 independently modifiable parameters. We discuss how the influences of type one and two can be modeled using a alterations to these parameters and show that a given toxin can be associated with a toxin signature corresponding to perturbations from the standard values of these coefficients. Finally, we show how these ideas can be used to determine low dimensional feature vectors for recognition purposes. We also discuss how a low dimensional biological feature vector could be used to obtain similar results.
Simplified models of human cognition and emotional
response are presented which are based on models of auditory/ visual
polymodal fusion. At the core of these models is
a computational model of Area 37 of the temporal cortex
which is based on new isocortex models presented recently
by Grossberg. These models are trained using carefully chosen
auditory (musical sequences), visual (paintings) and
higher level abstract (meta level) data obtained from
studies of how optimization strategies are chosen in response
to outside managerial inputs. The software modules developed
are then used as inputs to character generation codes
in standard 3D virtual world simulations. The auditory and
visual training data also enable the development of simple music
and painting composition generators which significantly enhance
one's ability to validate the cognitive model.
The cognitive models are handled as interacting
software agents implemented as CORBA objects to allow
the use of multiple language coding choices (C++, Java,
Python etc) and efficient use of legacy code.
A simplified model of information processing in the brain can be constructed using primary sensory input from two modalities (auditory and visual) and recurrent connections to the limbic subsystem. Information fusion would then occur in Area 37 of the temporal cortex. The creation of meta concepts from the low order primary inputs is managed by models of isocortex processing.
Isocortex algorithms are used to model parietal (auditory), occipital (visual), temporal (polymodal fusion) cortex and the limbic system. Each of these four modules is constructed out of five
cortical stacks in which each stack consists of three vertically
oriented six layer isocortex models. The input to output training of each cortical model uses the OCOS (on center - off surround) and FFP (folded feedback pathway) circuitry of (Grossberg, 1) which is inherently a recurrent network type of learning characterized by the identification of perceptual groups. Models of this sort are thus closely related to cognitive models as it is difficult to divorce
the sensory processing subsystems from the higher level processing in the associative cortex. The overall software architecture presented is biologically based and is presented as a potential
architectural prototype for the development of novel
sensory fusion strategies. The algorithms are motivated to some degree by specific data from projects on musical composition and autonomous fine art painting programs, but only in the sense that these projects use two specific types of auditory and visual cortex data. Hence, the architectures are presented for an artificial information processing system which utilizes two disparate sensory sources. The exact nature of the two primary sensory input streams is irrelevant.
The solution of difficult optimization problems often requires the use of a parameter set allowing critical algorithm
design choices to be set. For example, in the construction of a valid pattern recognition scheme using a simple
feed forward network (FFN) technique, there can be thousands of equally valid FFN solutions which achieve
high percentage recognition levels on reasonable inputs. The solutions arise from different choices of stopping
tolerance, internal neuron architecture, learning rates and so forth. These meta level optimization parameter
choices can be used to organize collections of optimization algorithms into matrices W. Each column of the matrix
corresponds to a set of parameter choices such a stopping tolerance, learning rate, random restart choices and so
forth. For example, an optimization algorithm is constructed from a 4 x 3 matrix W by choosing an entry from
each column to construct a sequence ABC. The sequence ABC then encodes the collection of meta parameters
that are used to shape the algorithm. In this example, there are thus 64 possible optimization algorithms all
chosen to produce a similar output such as recognition rate. A simplified biologically based model of information
processing includes primary sensory processing and sensor fusion with construction of higher level meta data
modeled via recurrent connections between the site of sensor fusion and a simple model of limbic processing. We
illustrate how such a model can be constructed using as training data the matrices described above. Finally, the
use of this model to model the decision process is discussed.
A model of biological information processing is presented that consists of auditory and visual subsystems linked to temporal cortex and limbic processing. An biologically based algorithm is presented for the fusing of information sources of fundamentally different modalities. Proof of this concept is outlined by a system which combines auditory input (musical sequences) and visual input (illustrations such as paintings) via a model of cortex processing for Area 37 of the temporal cortex. The training data can be used to construct a connectionist model whose biological relevance is suspect yet is still useful and a biologically based model which achieves the same input to output map through biologically relevant means. The constructed models are able to create from a set of auditory and visual clues a combined musical/ illustration output which shares many of the properties of the original training data. These algorithms are not dependent on these particular auditory/ visual modalities and hence are of general use in the intelligent computation of outputs that require sensor fusion.