WO1991011771A1 - Genetic algorithm technique for, e.g., designing neural networks - Google Patents

Abstract

A genetic algorithm search is applied to determine an optimum set of values (Fig. 16A) (e.g., interconnection weights in a neural network), each value being associated with a pair of elements (Fig. 3) drawn from a universe of N elements, N an integer greater than zero, where the utility of any possible set of said values may be measured. An initial possible set of values is assembled, the values being organized in a matrix whose rows and columns correspond to the elements. A genetic algorithm operator (Fig. 8) is applied to generate successor matrices from said matrix. Matrix computations are performed on the successor matrices to generate measures of the relative utilities of the successor matrices. A surviving matrix is selected from the successor matrices on the basis of the metrics. The steps are repeated until the metric of the surviving matrix is satisfactory.

Description

Genetic Algorithm Technique for, e.g.,

Designing Neural Networks

Background of the Invention

The invention relates to genetic algorithms, and to the use of genetic algorithms in designing neural networks (NNs).

One aspect of NN design is the specification of the interconnections among a NN's neurons to achieve a desired input-output (I/O) relationship. In particular, for the class of long term memory (LTM) time-invariant NNs, the NN is programmed for a particular I/O

relationship by choosing the weight of each

interconnecting trace between a pair of neurons.

Conventional NN design techniques make simplifying assumptions so that the problem of

determining the trace weights is tractable. Examples of these simplifying assumptions are that the neurons are arranged in layers within which no lateral connections are made or that a neuron can be simultaneously

excitatory and inhibitory, i.e., the weights of traces from a particular neuron to others in the NN can be positive to some and negative to others.

Summary of the Invention

In general, the invention features applying, in a program controlled computer, a genetic algorithm search to determine an optimimum set of values, each value being associated with a pair of elements drawn from a universe of N elements, N an integer greater than zero, where the utility of any possible set of said values may be measured. An initial possible set of values is assembled, the values being organized in a matrix whose rows and columns correspond to the

elements. A genetic algorithm operator is applied to generate successor matrices from said matrix. Matrix computations are performed on the successor matrices to generate measures of the relative utilities of the successor matrices. A surviving matrix is selected from the successor matrices on the basis of the metrics. The steps are repeated until the metric of the surviving matrix is satisfactory.

Preferred embodiments of the invention includethe following features. The method may be adapted for determining, for neurons of a neural network, a set of interconnection weights which will enable the network to approximate a prespecified output when presented with a predetermined set of input data. In that adaptation, the initial possible set of interconnection weights is assembled, and matrix computations are performed on the input data and on the successor matrices to generate an output of each successor matrix. A metric is computed corresponding to the difference between each generated output and the prespecified output, and the successor matrix is selected based on the metrics.

The interconnection weights in the matrices are forced to comply with constraints that correspond to the architecture of the neural network. The interconnection weights in the matrices are organized so that the interconnection weights leading to a given neuron all appear in a single row (or a single column) of each matrix, and the interconnection weights leading from a given neuron all appear in a single column (or a single row) of the matrix. The constraints include requiring that the interconnection weights between a neuron and itsel f be zero . The constraints also include requir ing that the interconnection weights leading from a given neuron all be of the same sign . The neural network compr ises a layer of input neurons , a layer of hidden neurons , and a single output neuron . The step of applying a genetic algorithm operator comprises applying a mutation operator by randomly changing a portion of the interconnection weights in a given matrix . No other genetic algorithm operator is applied, other than mutation . The interconnection weights are organized in more than one matrix based on layers of neurons in the neural network . The input data sets include binary values which represent edges of an image . The step of selecting a successor matr ix on the basis of the metr ics may include either testing whether a metric lies within a band between upper and lower bounds , or testing whether metr ics with respect to predetermined input data sets lie above a passband or below a stopband .

Use of matr ix representations enables simple and rapid design of a neural network . The meanings of the interconnection weights may be easily interpreted because of the manner in which they are arrayed in the matrix . Simplifying assumptions required in some other techniques may be avoided. Complex , ful ly

interconnected networks can be des igned for implement ing user specif ied, input-output functions .

Other advantages and features will become apparent from the following descr iption of the preferred embodiment and from the claims . Description of the Preferred Embodiment

We first briefly describe the drawings.

Fig. 1 is a functional block diagram of a pattern recognition system.

Fig. 2 is a diagram of edges in images.

Fig. 3 is a diagram of the layers of one kind of neural network.

Fig. 4 is a diagram of neurons and

interconnections within the network.

Figs. 5A through 5D are equations relating to the invention.

Figs. 6A and 6B are matrices of interconnection weights.

Fig. 7 is a diagram of constaints imposed on the interconnection weights arrayed in a matrix.

Fig. 8 is a flow chart of a method of designing the interconnection weights.

Fig. 9 is a set of input data matrices for testing the neural network.

Fig. 10 is a diagram of horizontal feature detector response.

Fig. 11 is a diagram of the generational history of metrics during the design process.

Figs. 12A through 12D set forth matrices of interconnection weights for a horizontal edge detector.

Fig. 13 is a graph of numbers of generations required for various numbers of hidden neurons.

Fig. 14 is a curve of sensitivity to number of copies per generation.

Fig. 15 is a diagram of diagonal feature detector response. Fig. 16A through 16D set forth matrices of interconnection weights for a diagonal edge detector.

The LTM, time-invariant class of NNs is

particularly suited for applications such as

preprocessors, feature detectors, and feedback control modules in larger processing systems. Referring to Fig.

1, in a pattern recognition system of the kind described in copending patent application, United States serial number ________________ , filed _______________ , assigned to the same assignee and incorporated herein by reference, patterns of an input image are analyzed to classify objects represented by patterns within the image. The output of the system is an indication of the categories, or names, of the objects located within the input image.

Each of the modules within the system of Fig. 1 mimics a particular function of the biological vision system. The system includes a pattern edge strength module 10 and a perimeter edge strength module 12 which mimic the functions of the biological vision system's visual cortex areas called visual area 1 and visual area

2, respectively. Together, these two edge strength preprocessors provide edge strength information to the classifier module 14 for final classification of objects located within an input image.

The system of Fig. 1 also includes a digitizer unit 18 which digitizes the input image data 16, and a locate/focus/adjust unit 20 which locates patterns within the input image data 16, provides control to the digitizer unit, and receives feedback from the

classifier module 14.

The pattern edge strength module 10 generates a measure of the visual strength of the edges within a pattern in the input image data 16. For binary images, the module measures the visual strength of the pattern's orientations in the horizontal, vertical, and two 45 degree diagonal directions. For gray-scale images, the module generates measures of the magnitude of the luminance gradient in the four directions. These orientation measurements of luminance lines and gradient magnitudes model similar processing that occurs in biological vision systems.

The perimeter edge strength module 12 measures the visual strength of a pattern's perimeter in the northern, southern, eastern, and western directions.

Thus, the module's output indicates the orientational strength of a pattern's edges.

In measuring the edge strengths of a pattern, both the perimeter and pattern edge strength modules evaluate an edge's magnitude with no special regard for the edge location within an image. To illustrate this rationale, consider the image blocks A, B, C, and D of Fig. 2, which could be evaluated for their horizontal edge strengths. The images labeled A and B would produce the same edge strength, because lines A and B, although at different image locations, are of the same length. In contrast, image C would produce a higher strength value than image D because image C includes more and longer lines than image D.

To implement either of the edge strength preprocessors described above with a LTM NN, the NN would ideally model the orientation-sensitive biological NN found in the human primary visual cortex. Thus, the implemented NN might measure, for example, the edge strength of a binary pattern in the horizontal direction. Design constraints implied by this

biological-like function of the NN, as discussed below, dictate considerations of both the NN's architecture and interconnections.

The network architecture suggested by the biological design constraints, as shown in Fig. 3, consists of a layer of M input neurons 16 which receive the input pattern, a network of N neurons located within the network, called hidden neurons 18, and a single output neuron 20. This architectural framework allows direct correlation between the neurons of the artificial and biological networks. The network input layer is composed of, for example, a 7 X 7, 2-dimensional array of neurons, for a total of M = 49 input neurons.

Connected to the input layer, the hidden neuron network may include N = 25 interconnected neurons, with some subset of those neurons being connected to the output neuron.

Considering the network architecture in more detail, Fig. 4 illustrates a subset of some possible network interconnections for the network. The input neurons, labeled A - D in the figure, are configured in an array so that each may accept one piece, or pixel, of an input pattern. This capability mimics the process by which a visual image would be defined on the input layer of the human primary visual cortex. The output

connection of each input neuron is specified to connect it with some set of hidden neurons, labeled E - J in the figure, and possibly the output neuron, labeled O in the figure. Within the hidden neuron network, each neuron may be connected to some set of input neurons, some set of hidden neurons, and the output neuron, but all three types of connections are not required for any given neuron.

In operation, the activation state of each neuron determines whether or not it "fires", i.e., whether or not it sends an output signal to the other neurons connected to it. This activation is determined by all of the input signals received by each neuron, i.e., a neuron's activation depends on the activation of the neurons connected as inputs to it. Arrows in the figure indicate the direction of input and output for each connection between a pair of neurons. If a

neuron's activation is smaller than a design threshold, the neuron will not send any output signals. If the neuron's activation reaches or passes the design

threshold, it will "fire" by sending an Output signal along each of its output connections.

When a neuron "fires", its output signals act to affect the activation states of the neurons to which it sends signals. There are two possible conditions, called types, of neuron output signals. A neuron's output signal may be excitatory (type 1), meaning that it is trying to "turn on" a succeeding neuron by raising that neuron's activation state. Alternatively, the neuron's output may be inhibitory (type 2), meaning that it is trying to "turn off" a succeeding neuron by lowering that neuron's activation state.

Each neuron is specified as being either excitatory or inhibitory by assigning a multiplication factor, called a weight, to each of its output

connections. When a neuron "fires", sending multiple output signals, each of those signals is multiplied by the weight of the particular connection it has been sent through. A positively weighted signal acts to increase a succeeding neuron's activation, while a negatively weighted signal acts to decrease a succeeding neuron's activation. This custom weighting of each output signal is what enables a neuron to increase or decrease the activation of a succeeding neuron. The particular assigned weight determines the degree to which one neuron's output signal contributes to the next neuron's activation.

The connections of each neuron in the example

NN are further defined such that when any of the hidden neurons or the output neuron "fires", it does not send an output signal back to the input layer. However, output signals of each hidden neuron may be sent to other hidden neurons as feedback within the hidden neuron network.

These activation and connection design rules are supplemented by design constraints, based on

experimental physiology, which are applied to the network architecture to model the behavior of the biological NN of the visual cortex. The additional design constraints include:

(1) A neuron has no feedback connection to itself.

(2) Each of the neuron is excitatory or inhibitory, but not both simultaneously.

(3) All input neurons are excitatory, and all hidden neurons are inhibitory.

Constraint (3) defines the example NN to be of the ON-Center/OFF-Surround architecture. Operationally, this implies that the input neurons are always trying to turn on the hidden neurons they are connected to, as well as the output neuron, while the hidden neurons are all trying to turn off the other hidden neurons they are connected to, as well as the output neuron.

The output neuron does not send any signals backward to the hidden or input neurons; its only output signal is, in the example, the NN's horizontal edge strength measure for an input pattern. This output signal is a real analog number, with the range of the signal determined by the NN interconnection design and training, as discussed below.

Given all of the design rules and. assumptions discussed above, the last remaining NN design

specification is the choice of interconnections between all of the NN neurons. This interconnection choice may be considered as the selection of the NN interconnection weights, as an interconnection weight of zero implies that no interconnection exists, while a positive weight causes an interconnection to be excitatory and a

negative weight causes an interconnection to be

inhibitory.

Each interconnection weight, and the NN set of weights together, is chosen to enable the NN to provide a desired relationship between a set of input patterns and the output horizontal edge strength measures the NN would generate for the set. In particular, the NN weight set is chosen to define the range of the output signal and the correspondence between values in the range and strength of a given horizontal edge. Thus, the weight set selection acts to calibrate the NN for a design specified input-output (I/O) relationship.

In the invention, a trial and error approach is used to select the NN interconnection weights. The inventive method eliminates the need for certain

simplifying assumptions required in known techniques for designing interconnection weights for a NN.

Furthermore, the resulting NN may require fewer neurons and fewer interconnections than a corresponding NN designed using other methods. Also, the resulting NN may be easier to implement than a corresponding NN designed by known methods.

The particular trial and error strategy employed by the current invention is a form of the general search strategy called the genetic algorithm (GA) . The GA is a known search procedure, inspired by evolution and heredity, for locating high performance structures in a complex task domain. It is an iterative technique for exploring large search spaces and complex problems of the kind that lack convenient closed forms. For NN design purposes, it is a search method for finding a good set of interconnection weights in a high-dimensional, non-linear weight space.

Note that the weight space, or set of possible

NN interconnection weights in our example, is large enough (there are 49 input neurons and 25 hidden

neurons) to discourage simple trial and error search procedures. In a traditional unstructured search, the interconnection weights would be selected by random, the performance of the newly defined NN computed, the result evaluated, and the process repeated until a satisfactory result is obtained. This is equivalent to numeration because each weight selection trial is unaffected by the outcome of previous selection trials. For any but the simplest cases it is easy to show that this is not a practical technique because of the large number of trials required before a satisfactory set of weights is achieved.

In general, the GA principle gives guidelines for constructing practical search techniques when the number of possible trials is extremely large. The fundamental requirements for solving a trial and error problem with the GA are that the problem must be capable of being represented by some data structure and that the problem's solutions be capable of being evaluated.

The general GA procedure" is defined as follows: A set of structures, called a generation, iscreated, which attempts to solve a particular problem. The structures, generally known as schemas, are

manipulated by one or more genetic operators to create a new set of structures. These new structures are

evaluated on how well they solve the problem and the best are saved, thereby creating a new generation. A requirement for constructing further generations is that the advances already made must be retained and that the retained advances be used to increase the population of fit structures. This evolutionary process is repeated until a structure produces an acceptable solution to the problem.

To choose the optimum set of NN interconnection weights, the GA procedure is implemented as follows:

(1) An initial set of interconnection weights is formed by randomly choosing a value for each of the weights in the whole NN.

(2) The weight set is manipulated by a genetic operator to create a new generation of weight sets.

(3) Each weight set which is a member of the new generation of weight sets is evaluated on how well its corresponding NN responds to a set of test, or training, input patterns in generating a horizontal edge strength measure for each of the training patterns.

(4) Steps (2) and (3) are repeated until a set of interconnection weights produces a NN with an

acceptable relationship between input patterns and output horizontal edge strength measures.

In step (1) of the design procedure, the set of NN interconnections must be represented in a manner that can be easily mapped to a representation that can be subjected to a GA operator. This mapping is achieved by formulating a mathematical representation of the

activations of each of the neurons in the NN to be designed. This formulation is called the short-term memory (STM) equation, and describes the activation of a neuron by all of the inputs to that neuron. The most general STM is shown as Eqn. 1 of Fig. 5.

Z_{j i}, defined in Eqn. 1 as the LTM trace from the j-th neuron to the i-th neuron, is the weight value assigned to the interconnection between the j-th neuron and the i-th neuron. Thus, the matrix of coefficients includes the complete set of Z_{j i}s for all i and j that we seek. The function f(x_j), called a sigmoid

function, is defined to set the threshold for activation of a neuron. Thus, for example, using a simple step function as the sigmoid function would cause a neuron with an activation below the step never to "fire", and would cause a neuron with an activation above the step to always to "fire".

Interpreted literally, the STM equation shows that any changes in the activation of a neuron are determined by the sum of any external inputs to that neuron (I_i), all of the weighted signals received from other neurons (Z_{j i}), and the current activation of the neuron. This equation is the cornerstone of NN design and is adequate for an initial NN design. If desired, it can be replaced with the shunting STM Equation for a better model.

Given the desired NN architecture described above and in Fig. 3, the STM equation can be customized to specifically describe the activations of neurons in the input layer, hidden layer, and output of the NN.

Since neurons in the input layer have no

interconnections between themselves, Eqn. 1 simplifies to Eqn. 2 of Fig. 5 for the input layer neurons, showing that input neurons only receive inputs from the pattern defined on them.

Neurons located within the hidden neuron network may receive inputs from any of the input neurons and any of the other hidden neurons, so that the STM equation for the hidden neurons simplifies to Eqn. 3 of Fig. 5.

Likewise, the output neuron may receive inputs from any of the input or hidden neurons, so its STM equation simplifies to Eqn. 4 of Fig. 5.

In Eqn. 3, the interconnection weights may be grouped between those associated with connections between the input layer and a hidden neuron and those associated with connections between two hidden neurons. This new hidden neuron STM equation, as shown in Eqn. 5 of Fig. 5, delineates the weight interconnections within the hidden network as Z_{j i} and interconnections between the input layer and hidden network as Z'_ki . In the steady-state, each of the input neuron activations will approach the external inputs applied to them, i.e., x_k will approach I_k, where k=l,...,M.

For this case, the steady-state hidden system STM equation becomes Eqn. 6 of Fig. 5.

To map the interconnection weights to a structure which may be manipulated by a GA operator, matrix notation is introduced as shown in Eqn. 7 of Fig. 5. Each element of matrix X represents the activation of the corresponding neuron in the hidden neuron

network. Using this matrix notation, the hidden neuron STM equation (Eqn. 5) becomes that of Eqn. 8 of Fig. 5.

Using this matrix notation, each element of matrix A represents the corresponding interconnection weight between two of the hidden neurons. As an

example, Fig. 6(a) illustrates what the elements of matrix A would be for the simplified NN of Fig. 4. The first element of matrix A, Z_EE, is the weight

coefficient of the interconnection from hidden neuron E back to itself. Although, as was discussed above, this interconnection is not actually allowable by the design rules, it, like every other theoretically possible interconnection from neuron E, is included in the general matrix.

Similiarly, the second element of the first row, Z_EF, describes the weight of the connection from hidden neuron E to hidden neuron F. Thus, the first row of matrix A includes the interconnection weights for all possible connections from neuron E and all of the other hidden neurons. Likewise, all of the other rows and columns of matrix A represent each of the other possible connections between the hidden neurons. As shown in Fig. 6(a), matrix B represents all of the possible interconnections between the input layer neurons and the hidden layer neurons. Thus, each element of matrices A and B may be directly correlated to a physical interconnection of the NN being designed.

Continuing with the matrix notation and considering the output neuron activation, the

steady-state output is denoted simply as Z, i.e., x_O approaches Z in the steady-state. Then the output neuron's STM equation may be grouped to delineate which interconnections are coming from the input layer and which are coming from the hidden layer, the sets of interconnections being represented in Eqns. 9 and 10 of Fig. 5.

Fig. 6(b) illustrates what matrices C and D would look like for the example NN of Fig. 4. Matrix C includes all of the possible connections between hidden neurons and the output neuron. Matrix D includes all of the possible connections between input layer neurons and the output neuron.

Given this matrix formulation of the NN activations, the design rules and constraints discussed above can be efficiently and understandably imposed as the GA is used to determine an optimum set of

interconnection weights. Further, and more importantly, the A, B, C, and D matrices can be directly manipulated by the GA procedure to solve the weight search. Indeed, each GA interconnection generation is simply a new copy of the interconnection matrices, so that each matrix element coefficient (weight) plays a role analogous to a DNA molecule in biological evolution. To begin the GA interconnection weight search procedure using the weight matrix notation, the

activation, connection, and physiological design rules discussed above must be imposed on the interconnection weights, and thus on the interconnection weight matrix elements. With these constraints, the weight matrices for the ON-Center/Off-Sur round NN being designed have the following properties:

(1) A has zero diagonal components,

(2) A and C have only negative or zero elements,

(3) B and D have only positive or zero elements,

(4) The rows of A give the lateral inhibition to other hidden neurons.

While these constraints are difficult to include in standard design techniques, it will be seen that they can be easily accommodated using the GA search technique. Fig. 7 summarizes the constraints on the system matrices for the biological-like NN.

With these constraints in place, the

computational steps for the NN design procedure (see Fig. 8) are as follows:

(1) Form (40) a parent initial matrix set A, B, C, and D of interconnection weights which satisfy the design constraints.

(2) Make copies of the parent set of matrices, and for each copy (42), randomly select a number of the matrix elements to be changed, subject to maintaining the above matrix properties, to produce successor

matrices (offspring). In GA terminology, a mutation operator is used to construct new generations.

(3) Apply a set of input patterns (44) to each NN corresponding to one of the copies and solve (46) for each NN's edge strength output for each pattern. (Note that a solution may not always exist.)

(4) Compute (48) a metric to quantify the discrepancy between each of the NN outputs of (3) and the prespecified solution (50).

(5) Select (52) the copy of the A, B, C, and D matrix set which provides the best NN I/O relationship (i.e., has the smallest discrepancy metric). Make this copy the survivor.

(6) Use the survivor as the parent for the next generation and repeat steps (2)-to-(5).

(7) Continue until the selection criterion is met (54).

(8) The surviving matrix set A, B, C, D determines the interconnection weights for the final NN design.

The example NN edge strength preprocessing modules to be designed using the above procedure

contain, as described previously, 49 input neurons, 25 hidden neurons, and one output neuron. This NN requires 1924 interconnection coefficients to be specified. For simplicity, the sigmoid function defined in the

activation equation is a unit step. The model is a "sum-of sigmoids" not the usual simplifying

approximation of "sigmoid-of-sums".

To implement step (1) of the design method, a random number generator is used to specify the matrix elements, thus establishing an initial parent set of interconnection weight matrices. Although the elements of the parent matrix set are chosen randomly, the interconnection constraints described above are followed in the choice. The random selection of initial matrix elements increases the likelihood that some portion of initial weights will be adequate immediately, thus decreasing the required computation of new generations.

Using the GA mutation operator, each new generation is created by first making 10 copies of the parent matrix set A, B, C, D. Then, for each copy, one-third of the weight elements of each matrix are randomly changed, subject to the given interconnection constraints, over the search range 0, +/-1, +/-2,..., +/-10. While other GA operators may be feasibly used with the current inventive NN design method, the

mutation operator is most easily applied to the schema defined by the matrix notation and has been found to be sufficient.

The choice of 10 copies (or offspring) per generation and one-third changes per copy is purely logistical and may be chosen differently without

affecting the efficiency of the method. Similarly, the search range limit of -10 to +10 places a boundary on the search space to speed-up the calculations, and may be chosen differently.

To implement step (3) of the design procedure, the 12 training patterns illustrated in Fig. 9 are input to each of the NN offspring of the current generation, and each of the NN offspring's responses is calculated. The arrangement of l's on a background of 0's in each training pattern represents a possible edge. A desired output is specified for each input training pattern, and a metric is defined to measure the distance, or error, of the NN response from the desired output. Many metrics are possible. One possible metric specifies that for each input pattern, the NN output response Z should lie within a bank of upper and lower bounds.

This HI-LO metric, described below, is used to calibrate the desired NN output measurement for each training program. The metric defines the acceptable NN output to be in the range 50 - 100 for the "high response"

training patterns, and -100 - 10 for the "low response" training patterns.

As shown in Fig. 9, training patterns 1, 2, and 3 are examples of input patterns with horizontal lines, and so should elicit a high response from the NN, with pattern 1 generating the highest response.. Training patterns 4 - 12 of Fig. 9 have little or no horizontal components in them, and so should elicit varying degrees of low responses.

Fig. 10 illustrates the mapping of acceptable output responses, or Zs, for each of the 12 training patterns, to the response ranges. The first dot, labeled 1 in Fig. 10, is the output Z for the first training pattern. For this case, Z surpasses the HI band limit of 100 because of its maximum horizontal strength. The dots 2 and 3 lie above the HI pass band minimum of 50 because they have a high degree of

"horizontalness". The dots corresponding to patterns 4 - 12 are expectedly located between -100 and 10,

indicating their degree of horizontal edge strength.

The HI-LO metric is formulated by summing the error of the NN output for each training program. Thus, for N training patterns, the error, d, of the NN output is given by Eqn. 11 of Fig. 5. As shown by Eqn. 12 of Fig. 5, if the NN correctly calls a "high pattern" "high" or a "low pattern" "low", d = 0 for that pattern. If the NN incorrectly measures a pattern's horizontal edge strength, d = 1 for that pattern.

Another type of metric would require that the output be above a passband threshold for certain

patterns and be below a stopband threshold for the other patterns. This PASS-STOP criterion is formulated as follows:

For N₁ PASS patterns and N₂ STOP patterns, with N_P = N₁ + N₂, the error, d, of the NN output is given by Eqn. 13 of Fig. 5. As shown in Eqn. 14 of Fig. 5, if the NN correctly identifies a pattern, d = 0 for that pattern, while d = l if the pattern is not identified correctly.

For both metrics, the inequality of Eqn. 15 of Fig. 5 holds true. The condition d = N_P means that the NN satisfies none of the selection criteria (maximum error); the condition d = 0 means that the NN satisfies all the selection criteria and that a solution has been reached. In the design procedure the metric, d, starts at N_P (or smaller) and monotonically decreases to zero.

To compute the error d for each NN copy and pattern, Eqn. 8 of Fig. 5 must be solved for the

steady-state activation matrix X with that copy and pattern. Once X is known, the NN output Z is computed directly with Eqn. 10 of Fig. 5. This Z is then

compared to the HI-LO metric to determine the error d. Recasting Eqn. 8 of Fig. 5 to reflect a particular NN, given the matrices A_{NX N} and B_NXM and an input

pattern I_MXl , it is required to solve Eqn. 16 of Fig. 5.

Assume for illustration that I_MXl is binary. It is easily shown that for some choices of {A,B,I} no solutions may exist, a single solution may exist, or multiple solutions may exist.

The solution of non-linear matrix equations such as Eqn. 16 of Fig. 5 may be achieved using many different methods. The method used for the inventive design procedure is a combination of operator

decomposition followed by recursion based on a

fixed-point theorem. The first two terms of the

operator decomposition solution, X₀ and X₁, are

given by Eqns. 17 and 18 of Fig. 5. The recursive form used to solve for a final X matrix is given as Eqn. 19 of Fig. 5, with X₁ being the initial value. The

recursion is continued until X(+) = X(-).

The above recursion scheme will converge to a solution if and only if the operator defined by the right-hand side of Eqn. 19 of Fig. 5 is a contraction operator. In practice, an operator may be a contraction in some subspaces and not in other subspaces . For the horizontal edge strength NN, the initial values

generated by operator-decomposition will be in a

contraction subspace for more than 90% of the choices of the A and B matrices. It typically requires 7 to 10 recursions to reach the final solution of the activation equation.

To complete steps (4), (5), and (6) of the design procedure, the metric d of each of the 10

different offspring NNs is compared with the metric d of the parent NN. If e offspring metric is below the parent metric, then che offspring replaces the parent NN for the next generation. When two or more offspring NNs all are more satisfactory than a parent NN, one of the offspring is randomly chosen to become the new parent. To summarize the system variables chosen for

the inventive design method, Table 1 lists together all

of the design rules chosen to complete the 5-step

procedure outlined above. TABLE 1

INPUT PARAMETERS FOR DESIGNING

A HORIZONTAL FEATURE DETECTOR

USING AN ON-CENTER OFF-SURROUND

NEUTRAL NETWORK

INPUT NEURONS (M) 7 X 7 (49)

HIDDEN NEURONS (N) 25

COPIES PER GENERATION 10

FRACTION OF WEIGHTS VARIED

PER COPY 1/3

SEARCH RANGE 0,+/-1,+/12,...,+/-10

HIGH BAND 50 TO 100

LOW BAND -100 to 10

Fig. 11 illustrates the time history of the

metric d as the GA design procedure evolves a set of

interconnection weight matrices to an acceptable

solution. The acceptable solution is achieved after

about 600 NN generations. The one remaining error at

the 600th generation corresponds to the response above

the high threshold from the first training pattern,

which is acceptable. Figs. 12A through 12D illustrate

the final A, B, C, and D matrices that the example GA

design procedure produces. These matrix values are used

to implement the NN interconnections so that the

resulting NN measures edge strength values within the

output range defined by the training patterns. In practice, one would like to minimize the computation time required to reach an acceptable generation of NN interconnection weights. Two of the critical parameters to be chosen for this consideration are the number of hidden neurons in the NN and the number of copies made per generation of the GA

procedure. The relationship between these parameters follows a power law, as shown by Eqn. 20 of Fig. 5.

To illustrate this relationship, the effect of changing these parameters is shown for a fixed

cooperative-competitive (C-C) NN. In a C-C NN, the hidden neurons maybe of both types 1 and 2 (excitatory and inhibitory, respectively). Therefore, in applying the matrix constraints to a C-C NN, the system matrix properties for ON-Center/OFF-Surround NNs are modified as follows: property (1) still holds while properties (2) and (3) are changed such that the corresponding columns of matrices A&C and C&D have the same sign.

Figures 13 and 14 illustrate the number of generations required to find an acceptable

interconnection weight set as a function of the number of hidden neurons and the copies made per generation, respectively. As shown for this C-C case, α = l and ß = -3/2. Since the total computer design time is proportional to the total number of copies, i.e., to NG x NC, to first order the computational time varies as N/NC ^1/2. Because the mutation process is random, the number of generations required for solution is also random. The brackets indicate the range over which solutions were determined.

The design method of the invention may be generally applied to any NN design which can be mathmatically formulated to allow for manipulation by a GA operator. For example, a diagonal, rather than horizontal, feature detector may be designed. Fig. 15 illustrates the NN output response Z of a diagonal edge strength NN measurer using the same training set described above. This NN is designed using the same method described above, but with different HI-LO bands so that a 45 degree diagonal detector NN is achieved. Figs. 16a through 16D show the final, matrices for the diagonal detector.

As a second example, the method may also be employed to design a NN analyzing gray input images, rather than binary images. A summary of this design method is as follows: The gray input pixel values are scaled to lie in the interval [0,1] using a digitizing preprocessor. To preserve the grayness of the input image, a piecewise-linear sigmoid function with

saturation is used in the activation equations so that a neuron's "firing" threshold is analog-like, rather than digital. With this definition, the X matrix (hidden neuron activations), and NN output response Z are given by Eqns. 21 and 22 of Fig. 5. Here f₁() may be a unit step as before and f₂() is a piecewise-linear sigmoid with saturation such as given by Eqn. 23 of Fig. 5.

This set of equations is used in the above design procedure to achieve a NN which is sensitive to edge gradient measurements of gray, rather than binary, images.

The method may also be used to design a NN with multiple outputs, rather than a single output neuron, or with other I/O properties. The method may be extended to generally provide a design framework for constructing any type of NN to have a desired network performance.

The method may be implemented by software. An example of code that implements the method is set forth in Appendix A. The code in Appendix A is written in the APL language, and may be run and compiled on the IBM-AT computer. Appendix A is subject to copyright

protection. The copyright owner has no objection to the reproduction of Appendix A as it appears in the United States Patent and Trademark office, but otherwise reserves all copyright right whatsoever.

Other embodiments are within the following claims.

APPENDIX A

SOFTWARE CODE

There are 5 sets of software design code, written in APL for an 8MHz IBM-AT:

DESIGN A

* Multi-Input, Single-Output (MI/SO)

* Hidden Neurons are both Type 1 & Type 2, i.e., excitatory and inhibitory

* Application: Design of Feature Detectors DESIGN B

* Computes NN Output for a pattern set given A,

B, C, and D matrices for both horizontal and diagonal orientations.

DESIGN C

* Multi-Input, Multi-Output (MI/MO)

* Hidden neurons are both Type 1 & Type 2

* Application: - Design Response Module

DESIGN D

* Multi-Input, Single - Output (MI/SO)

* Hidden Neurons are Type 2 (inhibitory)

* Input Neurons are Type 1 (excitatory)

* Application: Design Feature Detectors using

ON-center/OFF-surround NN architecture

DESIGN E

* Computes NN Output for a pattern given A, B,

C, and D matrices for a single edge orientation

[ 0] DESIGND

[ 1] AXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX [ 2] APRGM DESIGNS ON-CENTER/OFF-SURROUND. MULTI -INPUT. SINGLE OUTPUT NN'S.

[3] AXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX[4] AM= INPUT DIMENSION

[5] AN = NUMBER OF HIDDEN NEURONS

[6] A

[ 7] ANPATTERN= NUMBER OF TRAINING PATTERNS

[ 8] APATTER [K;;]= INDEXED SET OF TRAINING PATTERNS

[ 9] AT= TARGET SEQUENCE

[ 10] ATHI= TARGET SEQUENCE UPPER BOUND

[ 11] ATLO= TARGET SEQUENCE LOWER BOUND

[ 12] AD= METRIC

[ 13] A

[14] AL= RANGE OF INITIAL VALUES FOR A.B.C MATRIX ELEMENTS

[15] ARANGE= -L,....0,1,... ,L

[16] AP= 1/PROBABILITY OF AN ELEMENT CHANGE

[17] ANC= NUMBER OF COPIES PER GENERATION

[18] ANG= NUMBER OF. GENERATIONS

[19]

[20] AINITIALIZE

[21]

[22]

[23]

A

[24] FRACTION←TRYCOUNT←SOLVECOUNT←ABORTCOUNT←

[25]

K←1

[26] AXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

[27] A-----------GXENERATION LOO P----------- [28]

AXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

[29] GLOOP:

[30] A

[31] KK←1

[32] AXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

[33] A--------------COPY LOOP----------------- [34] AXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

[35] CLOOP:

[36]

[37]

A

[38] KKK←1

[39] AXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

[40] A------------TRAINING LOOP--------------- [41] AXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

[42] TLOOP: ATRAINING LOOP

[43] TRYCOUNT←TRYCOUNT+1

[44]

[45] AXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

[46] A----------- TEST FOR SOLUTION----------- [47] AXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

[48] → (TK≤ 10) /CONTINUE [49] ♤← ' '

[50] ♤← ' SYSTEM NOT SOLVED'

[51] ♤← ' GENERATION= ',

K

[52] ♤← 'COPY=' ,

KK

[53] ♤← ' PATTERN= ' ,

KKK

[54] ♤← ' ABORTCOUNT=' ,

ABORTCOUNT

[55] FRACTION←SOLVECOUNT+TRYCOUNT

[56] ♤← 'FRACTION OF COPIES SOLVED= ' .

FRACTION [57] ABORTCOUNT←ABORTCOUNT+ 1

[58] → (ABORTCOUNT> 20000 )/END

[59] →CLOOP

[60] CONTINUE:

[61] SOLVECOUNT←SOLVECOUNT+1

[62]

[ 63

] KKK←KKK+1

[64] AXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX [65] A-------------END TRAINING--------------- [66] AXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX [ 67] →( KKK≤NPATTERN )/TLOOP

[68] AXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX [69] A-----------SELECT SUVIVOR-------------------------------- [70] AXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX [71]

[72]

→(MINMETRIC=0)/END

[73] KK←KK+1

[ 74] →(KK≤NC)/CLOOP

[ 75] AXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX [ 76] A--------------END COPIES------------------ [ 77] AXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX [ 78] SYSTEM[lN;

N]←A←AMIN

[ 79] SYSTEM[

lN; (N+lM) ]←B←BMIN

[ 80] SYSTEM[(N+1);

lN]←C←CMIN

[81] SYSTEM[(N+ 1);(N+

)]←D←DMIN

[82] SYSMETRIC←MINMETRIC

[83] ♤← ' '

[84] ♤← ' GENERATION COMPLETED, NUMBER= '

, K[85] ♤← ' METRIC= ' ,

SYSMETRIC

[86] ♤← ' '

[ 87] GENERATIONVECTOR[ K]←K

[ 88] METRICVECTOR[K]←SYSMETRIC

[ 89] AXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX [ 90] A--------------- END GENERATIONS---------------- [ 91] AXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX [ 92] K←K+1

[ 93] →(K≤NG)/GLOOP

[ 94] AXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX [ 95] A-----------FINAL OUTPUT------------------ [ 96] AXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX [ 97] END:

[ 98] SYSTEM [

lN;

N]← A←AMIN

[ 94] AXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX [ 95] A-----------FINAL OUTPUT------------------ [ 96] AXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX [ 97] END:

[ 98] SYSTEM[ N;

N]←A←AMIN

[ 99] SYSTEM[

lN;(N+

M)]←B←BMIN

[ 100] SYSTEM[(N +1);

lN]←C←CMIN

[ 101] SYSTEM[(N +1);(N+

lM)]←D←DMIN

[ 102] SYSMETRIC←MINMETRIC

[ 103] ♤← ' ' [104] ♡← ' FINAL SYSTEM: GENERATION = ' , K[105] ♡← ' METRIC = ',

SYSMETRIC

[106] GENERATIONVECTOR[K]←K

[107] METRICVECTOR[K]←SYSMETRIC

[108]

[109]

[0] INITIALD

[1] AM←9

[3] AN ←1

A

[4] ANPATTERN←4

[5] APATTERN← (NPATTERN,3,3)ρ1

[6] A PATTERN[1;;]←HOR

[7] A PATTERN[2;;]←DIAGUP

[8] A PATTERN[3;;]←VER

[9] A PATTERN [4;;]←DIAGDN

[10] A T←5,1,0,1

[11] A THI← 6 2 1 2

[12] A TLO← 4 0 -1 0

[13] AMETRIC←S← (NPATTERN)ρ0

[14] ASYSMETRIC←MINMETRIC←5

[15] M←49

[16] N←25

[17] NPATTERN←12

[18] A NPATTERN←4

[19] PATTERN←(NPATTERN.7,7)ρ1

[20] PATTERN[1;;]←HOR1

[21] PATTERN[2;;]←HOR2

[22] PATTERN[3;;]←HOR3

[23] PATTERN[4;;]←DIAGUP1

[24] PATTERN[5;;]←DIAGUP2

[25] PATTERN[6;;]←DIAGUP3

[26] PATTERN[7;;]←VER1

[27] PATTERN[8;;]←VER2

[28] PATTERN[9;;]←VER3

[29] PATTERN[10;;] ←DIAGDN1

[30] PATTERN[11;;] ←DIAGDN2

[31] PATTERN[12;;] ←DIAGDN3

[32] PASS← 4 5 6

[33] STOP← 1 2 3 7 8 9 10 11 12

[34] THI←50

[35] TLO←10

[36] A THI← 10 10 10 100 100 100 10 10 10 10 10 10

[37] A TLO← -100 -100 -100 50 50 50 -100 -100 -100 -100 -100 -100

[38] A T← 1 1 1 10 10 10 1 1 1 0 0 0

[39] A THI← 4 4 4 20 20 20 4 4 4 4 4 4

[40] A TLO← -10 -10 -10 7 7 7 -10 -10 -10 -10 -10 -10

[41] A T← 10 1 0 1

[42] A THI←TO

[43] A TLO←T-3

[44] A M←2

[45] A N←3

[46] A NPATTERN←4

[47] A PATTERN←(NPATTERN.1,2) ρ1

[48] A PATTERN[;;]← 1 2 ρ 0 0 [49] A PATTERN[2;;]← 1 2 ρ 0 1

[50] A PATTERN[3;;]← 1 2 ρ 1 0

[51] A PATTERN[4;;]← 1 2 ρ 1 1

[52] A T← 0 6 6 0

[53] A TH I←T+2

[54] A TLO←T-2

[55] A MINMETRIC←NPATTERN

[56] S← (N PATTERN ) ρ0

[57] A

[58] L←10

[59] P←3

[60] NC←10 .

[61] NG←

[62] SCALE← ((N+1),(M+N)) ρ(Nρ1),(Mρ10) ASCALES ELEMENT VALUES.

[63] METRICVECTOR←GENERATIONVECTOR← (NG+ 1) ρ0

[64] A

[65] A SYSTEM← -1 + ?((N+1 , (N+M) )ρ(L+1) ACHOOSE RANDOM POS ELEMENTS

[66] AMIN←A← (SYSTEM[lN;

N]x~(N,N)ρ1.Nρ0) ANEG SUBMATRIX WITH ZERO DIAGONAL.

[67] BMIN←B←(10xSYSTEM[lN;(N+

M)]) APOS SUBMATRIX.

[68] CMIN←C← ((1,N)pSYSTEM[(N+1);

]) ANEG SUBMATRIX.

[69] DMIN←D←(10x(1,M) ρSYSTEM[(N+1);(N+

lM)]) APOS SUBMATRIX.

[70] A SYSTEM[ N;lN]←A AREVISE SYSTEM.

[71] A SYSTEMU

N;(N+ M)]←B

[72] A SYSTEM[(N+1) ;

N]←C AREVISE SYSTEM.

[73] A SYSTEM[(N+1);(N +

M)]←D *

[0] MAKECOPIESD

[1] A

[2] AINPUT= SYSTEM

[3] AOUTPUT=SYSTEMC , ACOPY, BCOPY, CCOPY, DCOPY

[4] COLINDEX←1=?(M+N) ρP APICK 1 OF P COLUMNS TO CHANGE.

[5] MTEMP1←((N+1) ,(M+N)) ρ0 ADEFINE MATRIX WITH 1'S IN COLUMNS

[6] MTEMP1 ←MTEMP1+((N+1),(M+M)) ρCOLIMDEX ATO BE CHANGED 4 O'S ELSEWHERE.

[7]

[8] MTEMP2←(-l+ ?(1+MTEMP1 xL))xSCALE ACHOOSE NEW ELEMENT VALUES IN RANGES.

[9] COLUMN←((N+1),(M+N))ρ(Np-1),(Mρ1) AMAKES A.C NEG & B.D POS.

[10] MTEMP2←MTEMP2xCOLUMN AMATRIX WITH NEW VALUES,

[11] SYSTEMC←SYSTEMx~MTEMP1 AZERO-OUT VALUES TO BE CHANGED.

[12] SYSTEMC←SYSTEMC+MTEMP2 AINSERT NEW VALUES INTO MATRIX.

[13] ACOPY←SYSTEMC[lN;lN]x~(N,N)ρ1,Np0 APICK-OUT SUBMATRICES.

[14] BCOPY←SYSTEMC[lN;(N + M)]

[15] CCOPY←SYSTEMC((N+1)

(16] DCOPY←SYSTEMC[(N+1) ;(N+

M)]

[0] SOLVEQU

[1] AXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

[2] ASOLVES: X = Af (X)+Bf(I)

[3] AXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

[4] RXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

[5] AMAP 2D→1D

[6] AXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

[7] INPUT←(M,1)ρPATTERN[KKK;;]

[8] AXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

[9] ACOMPUTE CONSTANT TERM

[10] AXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

[11] X0←TEMP1←BCOPY+.x

(1

L(0

INPUT))

[12] X1←ACOPY+.x

(1

(0

X0))

[13] XOLD←X0+X1

[14] A

[15] TK←0

[16] A XOLD←(ACOPY+.x(N,1)ρ1)+TEMP1

[17] LOOP1:

[18] TEMP2←ACOPY+.xr(1

(0

XOLD))

[19] XNEW← (TEMP1+TEMP2)

[20] DELTAX←XNEW-XOLD

[21] AXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

[22 ] ACONVERGENCE TEST

[23] AXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

[24] MAXERR0R←+/I.IELTAX

[25] A0←MAXERROR

[26] →(MAXERROR=0)/END

[27] A

[28] A

[29] A

[30] TK←TK+1

[31] XOLD←XNEW

[32] -KTK←10)/LOOP1

[33] END:

_ VSOLVESYS[♡]

[0] SOLVESYS

[1] INPUT← (M,1)ρPATTERN[KKK:;]

[2] X0←BCOPY+. x(STEP INPUT)

[3] XNEW←XOLD←(N.1)ρ0

[4] TK←1

[5] DELTA←1+10

[6] LOOP:

[7] XOLD←XNEW

[8] XNEW←XOLD+(-XOLD+(ACOPY+.x(STEP XOLD))+X0)xDELTA

[9] TK←TK+1

[10] →(TK≤30 )/LOOP

[11] ATEST

[12] X1←(ACOPY+.x(STEP XNEW))+X0

[13] X2←(ACOPY+.x(STEP X1))+X0

[14] X3←(ACOPY+.x(STEP X2))+X0

[15] DELTAX←X3-X2

[16] MAXERROR←+ / I , DELTAX

[17] XNEW←X3

[0]

Z←STEP X

[1] Z← (1

(0

X))

OUTPUT

A

[2] OUT1←CCOPY+.x (1 (0rXNEW))

[3] OUT2←DCOPY+.x

r(1

(0rINPUT))

[4] S[KKK]←OUT1+OUT2

[0] SELECT

[1] AMETRIC

[2] A METRIC←+/~(S=T)

[3] A METRIC←+/~((TLO≤S)A(S≤THI))

[4] METRIC←(+/~THI≤S[PASS])+(+/~TLO≥S(STOP])

[5] →(METRIC≥MINMETRIC)/DONE

[6] AMIN←ACOPY

[7] BMIN←BCOPY

[8] CMIN←CCOPY

[9] DMIN←DCOPY

[10] MINMETRIC←METRIC

[11] DONE:

[0] DESIGND

[1] AXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX [2] APRGM DESIGNS ON-CENTER/OFF-SURROUND. MULTI-INPUT, SINGLE OUTPUT NN'S.

[3] AXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

[4] AM= INPUT DIMENSION

[5] AN= NUMBER OF HIDDEN NEURONS

[6] A

[7] ANPATTERN= NUMBER OF TRAINING PATTERNS

[8] APATTERN[K;;]= INDEXED SET OF TRAINING PATTERNS

[9] AT= TARGET SEQUENCE

[10] ATHI = TARGET SEQUENCE UPPER BOUND

[11] ATLO= TARGET SEQUENCE LOWER BOUND

[12] AD= METRIC

[13] A

[14] AL= RANGE OF INITIAL VALUES FOR A,B,C MATRIX ELEMENTS

[15] ARANGE= -L,...,0,1,...,L

[16] AP= 1/PROBABILITY OF AN ELEMENT CHANGE

[17] ANC= NUMBER OF COPIES PER GENERATION

[18] RNG= NUMBER OF GENERATIONS

[19] A

[20] A lNITIALIZE

[21] A

[22] TSTART←♡TS

[23] INITIALD

[24] A

[25] FRACTION←TRYCOUNT←SOLVECOUNT←ABORTCOUNT←0

[26] A K←1

←

[27] AXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

[28] A ----------GENERATION LOOP--------------- [29] AXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

[30] GLOOP:

[31] A

[32] KK←1

[33] AXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

[34] A--------------------COPY LOOP------------------------

[35] AXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

[36] CLOOP:

[37] MAKECOPIESD

[38] A

[39] KKK←1

[40] AXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

[41] A--------------------TRAINING LOOP---------------

[42] AXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

[43] TLOOP: ATRAINING LOOP

[44] TRYCOUNT←TRYCOUNT+1

[45] SOLVEOU

[46] AXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

[47] A------------------------- TEST FOR SOLUTION----------------

[48] AXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

[49] →(TK≤10) /CONTINUE

[50] ♡←' '

[51] ♡←'SYSTEM NOT SOLVED'

[52] ♡←'GENERATION" ',

»K

[53] ♡← 'COPY=',

KK

[54 ] ♡← 'PATTERN= ' ,

KKK

[55] ♡← ' ABORTCOUNT =',

ABORTCOUNT

[56] FRACTION←SOLVECOUNT+TRYCOUNT

[57] ♡← 'FRACTION OF COPIES SOLVED= ' , FRACTION

[58] ABORTCOUNT←ABORTCOUNT+1 [59] → (ABORTCOUNT>1000 ) /END

[60] →CLOOP

[61] CONTINUE:

[62] SOLVECOUNT←SOLVECOUNT+ 1

[63] OUTPUT

[64] KKK←KKK+1

[65] AXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

[66] A-------------END TRAINING----------------

[67] A XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

[68] → (KKK≤NPATTERN) /TLOOP

[69] AXXXXXXXXXXXXXX-XXXXXXXXXXXXXXXXXXXXXXXXXXX

[70] A-----------SELECT SURVIVOR----------------

[71] A XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

[72] SELECT

[73] →( MINMETRIC = 0)/END

[74] KK←KK+1

[75] →KK≤ NC/CLOOP

[76] AXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

[77] A-------------END COPIES------------------

[78] AXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

[79] SYSTEM[ N;

N]←A←AMIN

[80] SYSTEM[

N;(N+ M)]←B←BMIN

[81] SYSTEM[(N+1);

N]←C←CMIN

[82] SYSTEM[(N+1 ) ; (N+ M)]←D←DMIN

[83] SYSMETRIC←MINMETRIC

[84] ♡←' '

[85] ♡←'GENERATION COMPLETED, NUMBER= ',

K

[86] ♡← 'METRIC= ' ,

SYSMETRIC

[87] ♡←' '

[88] A GENERATIONVECTOR[K]←K

[89] A METRICVECTOR[K]←SYSMETRIC

[90] AXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

[91] A------------END GENERATIONS--------------

[92] AXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

[93] K←K+1

[94] ← GLOOP

[95] A→ (K≤NG)/GLOOP

[96] AXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

[97] A -----------FINAL OUTPUT------------------

[98] AXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

[99] END:

[100] SYSTEM[ N;

N]←A←AMIN

[101] SYSTEM[

N;(N+ M)]←B←BMIN

[102] SYSTEM[(N+1);

N]←C←CMIN

[103] SYSTEM[(N+1);(N+

←)]←D←DMIN

[104] SYSMETRI[←MINMETRIC

[105] ♡←' '

[106] ♡←' FINAL SYSTEM: GENERATION = ',

K

[107] ♡←' METRIC = ',

SYSMETRIC

[108] A GENERATIONVECTOR[K]←K

[109] A METRICVECTOR[K]←SYSMETRIC

[110] TSTOP←♡TS

[0] INITIALD

[1] AXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

[2] AGREY DESIGN INPUTS

[3] AXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

[4] A

[5] M←49

[6] N←36

[7] A

[8] NPATTERN←24

[9] PATTERN←♡FREAD 1 25

[10] A

[11] PASS← 7 8 9 10 11 12

[12] STOP← 1 2 3 4 5 6 13 14 15 16 17 18 19 20 21 22 23 24

[13] THI←25

[14] TLO←5

[15] MINMETRIC←NPATTERN

[16] S←(NPATTERN)ρ0

[17] A

[18] L←10

[19] P←3

[20] NC←10

[21] NG←1000

[22] SCALE← ((N+1) , (M+N) )ρ(Nρ1) , (Mρ3 ) ASCALES ELEMENT VALUES IF REQ'D.

[23]

[24] A

[25] SYSTEM← -1+?((N+1),(N+M))ρ(L+1) ACHOOSE RANDOM POS ELEMENTS

[26] AMIN←A←- (SYSTEM[

N;

N]x~(N,N)ρ1,Np0) A ZERO DIAGONAL.

[27] BMIN←B←(3xSYSTEM(

IN;(N+

M)]) APOS SUBMATRIX.

[28] CMIN←C←-((1,N)ρSYSTEM[(N+1);

N]) ANEG SUBMATRIX.

[29] DMIN←D← (3x(1,M)ρSYSTEM[(N+1);(N+

M)]) APOS SUBMATRIX.

[30] SYSTEM[ N;

N]←A AREVISE SYSTEM.

[31] SYSTEM[

N;(N+α

M)]←B

[32] SYSTEM[(N+1);

N]←C AREVISE SYSTEM.

[33] SYSTEM[(N+1) ; (N+

M)]←D

[34] AXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

[0] SELECT

[1] AMETRIC

[2] A METRIC←+/~(S=T)

[3] AMETRIC←+/~((TLO≤S)Λ(S≤THI))

[4] METRIC←(+/~THI≤S[PASS])+(+/~TLO≥S[STOP])

[5] →(METRIC≥MINMETRIC)/DONE

[6] A →(S[2]<THI)/DONE

[7] AMIN←ACOPY

[8] BMIN←BCOPY

[9] CMIN←CCOPY

[10] DMIN←DCOPY

[11] MINMETRIC←METRIC

[12] METRICVECTOR←METRICVECTOR,K, METRIC

[13] DONE:

[0] INITIALD

[1] AM←9

[2] AN←1

[3] A

[4] ANPATTERN←4

[5] A PATTERN←(NPATTERN,3,3)ρ1

[6] A PATTERN(1;;3←HOR

[7] A PATTERN(2;;3←DIAGUP

[8] A PATTERN[3;;]←VER

[9] A PATTERN(4;;]←DIAGDN

[I0] A T←5,1,0.1

[11] A THI← 6 2 1 2

[12] A TLO← 4 0 -1 0

[13] AMETRIC←S←(NPATTERN) ρ0

[14] ASYSMETRIC←MINMETRIC←5

[15] A NPATTERN←4

[16] A PATTERN←(NPATTERN,7,7)ρ1

[17] A PATTERN[1; ;]←HOR1

[18] A PATTERN[2; ;]←HOR2

[19] A PATTERN[3 ; ;]←HOR3

[20] A PATTERN[4 ; ;]←DIAGUP1

[21] A PATTERN[5; ;]←DIAGUP2

[22] A PATTERN[6; ;]←DIAGUP3

[23] A PATTERN[7 ; ;]←VER1

(24] A PATTERN[8; ;]←VER2

[25] A PATTERN[9;;]←VER3

[26] A PATTERN[10;;]←DIAGDN1

[27] A PATTERN[11 ;;]←DIAGDN2

[28 ] A PATTERN[12;;]←DIAGDN3

[29] A PASS← 4 5 6

[30] A STOP← 1 2 3 7 8 9 10 11 12

[31] A THI←50

[32] A TLO←10

[33] A THI← 10 10 10 100 100 100 10 10 10 10 10 10

[34 ] A TLO← -100 -100 -100 50 50 50 -100 -100 -100 -100 -100 -100

[35] A T← 1 1 1 10 10 10 1 1 1 0 0 0

[36 ] A THI← 4 4 4 20 20 20 4 4 4 4 4 4

[37 ] A TLO← -10 -10 -10 7 7 7 -10 -10 -10 -10 -10 -10

[38 ] A T← 10 1 0 1

[39 ] A THI←T+3

[40 ] A TLO←T-3

[41] A M←2

[42 ] A N←3

[43] A NPATTERN←4

[44 ] A PATTERN*- ( NPATTERN , 1 , 2 )ρ1

[45] A PATTERNd;;]← 1 2 p 0 0

[46 ] A PATTERNd;;]← 1 2 p 0 1

[47 ] A PATTERNd;;]← 1 2 p 1 0

[48 ] A PATTERNd;;]←← 1 2 p 1 1

[49 ] A T← 0 6 6 0

[50] A THI←T+2

[51] A TLO←T-2

[52] A M←21

[53] A N←25

[54 ] A NPATTERN←24

[55] A PATTERN←(NPATTERN, 7, 33 ρ0

[56] A PATTERN←TCELL1

[57] A PASS←V12

[58] A STOP←12+

12 [ 59] A THI←50

[ 60] A TLO←10

[ 61] A XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

[ 62] AGREY DESIGN INPUTS

[ 63] A XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

[64] A

[ 65] M←49

[66 ] N←25

[ 67] A

[ 68] NPATTERN←18

[ 69] PATTERN←♡FREAD 1 25

[ 70] A

[ 71] PASS← 1 2 3 4 5 6

[ 72] STOP← 7 8 9 10 11 12 13 14 15 16 17 18

[ 73] THI←50

[ 74] TLO←10

[ 75] MINMETRIC←NPATTERN

[ 76] S←( NPATTERN )ρ0

[ 77] A

[ 78] L←100

[ 79] P←3

[ 80] NC←10

[ 81] NG←5

[ 82] SCALE← ((N+1),(M+N))ρ(Nρ2),(Mρ1) ASCALES ELEMENT VALUES IF REQ'D.

[ 83] METRICVECTOR←GENERATIONVECTOR← ( NG+ 1 )ρ0

[ 84] A

[ 85] SYSTEM← -1 + ?((N+1),(N+M))ρ(L+1) ACHOOSE RANDOM POS ELEMENTS

[ 86] AMIN←A←-(SYSTEM[

N;

N]x~(N,N)ρ1,Nρ0) ANEG SUBMATRIX WITH ZERO DIAGONAL.

[ 87] BMIN←B←(2xSYSTEM[

N:(N+

M)]) APOS SUBMATRIX.

[ 88] CMIN←C←- ((1,N)ρSYSTEM[(N+1);

N]) ANEG SUBMATRIX.

[ 89] DMIN←D←(2x(1,M)ρSYSTEM[(N+1);(N+ M)]) APOS SUBMATRIX.

[ 90] SYSTEM[ N; IN3--A AREVISE SYSTEM.

[ 91]

SYSTEM[

N; (N+ M) 3←B

[ 92] SYSTEM[(N+1);

N ]←C AREVISE SYSTEM.

[ 93] SYSTEM[(N+1);(N+

M)]←D

[ 94] A XXXXXXXXXXXXXXXXXXXXXXXXXXXXX

[0] MAKECOPIES

[1] A

[2] AlNPUT= SYSTEM

[3] AOUTPUT= SYSTEMC , ACOPY, BCOPY, CCOPY, DCOPY

[4] COLINDEX←1 = ?(M+N)ρP

[5] MTEMP1←((N+1) ,(M+N))ρ0

[6) MTEMP1←MTEMP1+((N+1),(M+N))ρCOLINDEX

[7] MTEMP2←-1+?(1+MTEMP1xL)

[8] STRING←x(-1.5 + ?(N+M)ρ2)

[9] COLSIGN←COLINDEXxSTRING

[10] MTEMP2←MTEMP2x((N+1),(M+N))ρCOLSIGN [11] SYSTEMC←SYSTEMx~MTEMP1

[12] SYSTEMC←SYSTEMC +MTEMP2

[13] ACOPY←SYSTEMC[ N:

N]x~(N,N)ρ1,Nρ0

[14] BCOPY←SYSTEMC[

N;(N+ M)]

[15] CCOPY←SYSTEMC[(N+1);

N]

[16] DCOPY←SYSTEMC[(N+1);(N+

M)]

[0]

SOLVEQU

[1] AXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

[2] ASOLVES: X=Af(X)+Bf (I)

[3] AXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

[4] INPUT←(M,1)ρPATTERN[KKK; ;] AMAP 2D→ 1D

[5] AXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

[6] ACOMPUTE CONSTANT TERM

[7] AlNITIAL GUESS IS FIRST TWO TERMS

[8] AOF OPERATOR DECOMPOSITION SOLUTION

[9] AXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

[10] A X0←TEMP1←BCOPY+.x(STEP INPUT)

[11]→ X0←TEMP1←BCOPY+.x(SATURATE1 INPUT)

[12]→ X1←ACOPY←.x(STEP X

0)

[13]

←X0 + X 1

[14] A

[15] TK←0

[16] LOOP1:

[17] TEMP2←ACOPY+.x(STEP XOLD)

[18] XNEW←(TEMP2+TEMP1)

[19] DELTAX←XNEW-XOLD

[20] AXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

[21] ACONVERGENCE TEST

[22] AXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

[23] MAXERROR←+/ I ,DELTAX

[24] →(MAXERROR=0)/END

[25] A

[26] TK←TK+1

[27] XOLD←XNEW

[28l → (TK≤10)/LOOP1

[29] END:

[0] Z←STEP X

[1]

[0] Z←SATURATE1 X

[1] Z←1l( 0rX)

[0] OUTPUT

[1] A

[2] OUT1←CCOPY+.x(STEP XNEW)

[3] A OUT2←DCOPY+.x(STEP INPUT)

[4] OUT2←DCOPY+.x(SATURATEl INPUT)

[5] S[KKK]←OUT1+OUT2

[6] AS←OUT1+OUT2

[0] SELECT

[1] AMETRIC

[2] A METRIC←+7~(S=T)

[3] AMETRIC←+ /~((TLO≤S)A[S≤THI))

[4] METRIC←( + / ~THI≤S(PASS])+(+/~TLO≥S[STOP])

[5] → (METRIC≥MINMETRIC)/DONE

[6] AMIN←ACOPY

[7] BMIN←BCOPY

[8] CMIN←CCOPY

[9] DMIN←DCOPY

[10] MINMETRIC←METRIC

[11] DONE:

[0] SOLUTION

[1] ACHECKS FINAL SOLUTION

[2] ACOPY←A

[3] BCOPY←B

[4] CCOPY←C

[5] DCOPY←D

[6] KKK←1

[7] LOOP:

[8] SOLVEQU

[9] OUTPUT

[10] KKK←KKK+1

[11] → (KKX←NPATTERN ) / LOOP

[0] DESIGNA

[1] AXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

[2] APRGM DESIGNS MULTI-INPUT, SINGLE OUTPUT NN'S.

[3] AXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

[4] AM= INPUT DIMENSION

[5] AN= NUMBER OF HIDDEN NEURONS

[6] A

[7] ANPATTERN= NUMBER OF TRAINING PATTERNS

[8] APATTERN[K;;]= INDEXED SET OF TRAINING PATTERNS

[9] AT= TARGET SEQUENCE

[10] ATM= TARGET SEQUENCE UPPER BOUND

[11] ATLO= TARGET SEQUENCE LOWER BOUND

[12 ] AD= METRIC

[13] A

[14] AL= RANGE OF INITIAL VALUES FOR A,B,C MATRIX ELEMENTS

[15] RANGE= -L,...,0,1 , ... , L

[16] AP= 1/PROBABILITY OF AN ELEMENT CHANGE

[17] ANC= NUMBER OF COPIES PER GENERATION

[18] ANG= NUMBER OF GENERATIONS

[19] A

[20] AINITIALIZE

[21] A

[22]

[23] A

[24] FRACTION←TRYCOUNT←SOLVECOUNT←ABORTCOUNT←1

[25] K←1

[26] AXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

[27] A-----------GENERATION LOOP--------------

[28] AXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

[29] GLOOP:

[30] A

[31] KK←1

[32] AXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

[33] A--------------COPY LOOP-----------------

[34] AXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

[35] CLOOP :

[36]

[37]

A

[38] KKK←1

[39] AXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

[40] A------------TRAINING LOOP---------------

[41] AXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

[42] TLOOP: ATRAINING LOOP

[43] TRYCOUNT←TRYCOUNT+1

[44]

[45] AXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

[46] A-----------TEST FOR SOLUTION------------

[47] AXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

[48] →(TK≤ 10) /CONTINUE [49] ♡←' '

[50] ♡←' SYSTEM NOT SOLVED'

[51] ♡←' GENERATION= ',

K

[52] ♡←'COPY=' ,

KK

[53] ♡←' PATTERN= ' ,

KKK

[54] ♡← ' ABORTCOUNT= ' ,

ABORTCOUNT

[55] FRACTION←SOLVECOUNT+TRYCOUNT

[56] ♡←' FRACTION OF COPIES SOLVED= ' ,

FRACTION

[57] ABORTCOUNT← ABORTCOUNT+ 1

[58] →(ABORTCOUNT>20000)/END

[59] →CLOOP

[60] CONTINUE:

[61] SOLVECOUNT←SOLVECOUNT+ 1

[62]

[63] KKK←KKK+1

[64] AXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

[65] A-------------END TRAINING----------------

[66] AXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

[67] →KKKK≤NPATTERN)/TLOOP

[68] AXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

[69] A-----------SELECT SURVIVOR---------------

[70] AXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

[71]

[72] → (MINMETRIC=0)/END

[73] KK←KK+1

[74] →KKK≤NC)/CLOOP

[75] AXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

[76] A-------------END COPIES------------------

[77] AXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

[78] SYSTEM[ N:

N]←A←AMIN

[79 ] SYSTEM[

N;(N+HM)]←B←BMIN

[80] SYSTEM[(N+1);

N]←C←CMIN

[81] SYSTEM[(N+1);(N+

M)]←D←DMIN

[82] SYSMETRIC←MINMETRIC

[83] ♡←' '

[84] ♡←' GENERATION COMPLETED. NUMBER= -,

K

[85] ♡←'METRIC= '

SYSMETRIC

[86] ♡←' '

[87] GENERATIONVECTOR[K]←K

[88] METRICVECTOR[K]←SYSMETRIC

[89] AXXXXXXXXXXXXXXXXXXXXXXXXXXXXXxxxxxxxxxxxx

[90] A------------END GENERATIONS--------------

[91] AXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

[92] K←K+1

[93] →(K≤NG)/GLOOP

[94] AXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

[95] A-----------FINAL OUTPUT------------------

[96] AXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

[97] END:

[98] SYSTEM( N;

N]←A←AMIN

[99] SYSTEM[

N:(N+

M)]←B←BMIN

[100] SYSTEM[(N+l);

N)←C←CMIN

[101] SYSTEM[(N+1);(N+

M)]← D←DMIN

[102] SYSMETRIC←MINMETRIC

[103] ♡←' '

[104] ♡←' FINAL SYSTEM: GENERATION * '.

K

[105] ♡←' METRIC = ',

SYSMETRIC

[106] GENERATIONVECTOR[K]←K

[107] METRICVECTOR[K]←SYSMETRIC

[0] INITIAL

[1] AM←9

[2] AN←1

[3] A

[4] ANPATTERN←4

[5] A PATTERN←(NPATTERN.3,3)ρ1

[6) A PATTERN[1 ;;]←HOR

[7] A PATTERN[2;;]←DIAGUP

[8] A PATTERN[3;;]←VER

[9] A PATTERN[4;;]←DIAGDN

[10] A T←5, 1.0.1

[11] A THI← 6 2 1 2

[12] A TLO← 4 0 -1 0

[13] AMETRIC←S←(NPATTERN)ρ0

[14] ASYSMETRIC←MINMETRIC←5

[15] M←49

[16] N←25

[17] NPATTERN← 12

[18] A NPATTERN←4

[19] PATTERN←(NPATTERN .7.7)ρ1

[20] PATTERN[1 ;;]←HOR1

[21] PATTERN[2;;]←HOR2

[22] PATTERN[3;;]←HOR3

[23] PATTERN[4;;]←DIAGUP1

[23] PATTERN[4;;]←DIAGUP1

[24] PATTERN[5;;]←DIAGUP2

[25] PATTERN[6;;]←DIAGUP3

[26] PATTERN[7;;]←VER1

[27] PATTERN[8 ;;]←VER2

[28] PATTERN[9;;]←VER3

[29] PATTERN[10;;]←DIAGDN1

[30] PATTERN[11:;]←DIAGDN2

[31] PATTERN[12;;]←DIAGDN3

[32] T← 1 1 10 10 10 1 1 1 0 0 0

[33] THI← 4 4 4 20 20 20 4 4 4 4 4 4

[34] TLO← -10 -10 -10 7 7 7 -10 - 10 -10 -10 -10 -10

[35] A T← 10 1 0 1

[36] A THI←T+3

[37] A TLO←T-3

[38] A M←2

[39] A N←3

[40] A NPATTERN←4

[41] A PATTERN←(NPATTERN.1,2)ρ1

[42] A PATTERN[1 ;;]← 1 2 ρ 0 0

[43] A PATTERN[2 ;;]← 1 2 ρ 0 1

[44] A PATTERN[3;;]← 1 2 ρ 1 0

[45] A PATTERN[4;;]← 1 2 ρ 1 1

[46] A T← 0 6 6 0

[47] A THI←T+2

[42] A PATTERN[1 ;;]← 1 2 ρ 0 0

[43] A PATTERN[2 ; ;]← 1 2 ρ 0 1

[44] A PATTERN[3;;]← 1 2 ρ 1 0

[45] A PATTERN[4;;]← 1 2 ρ 1 1

[46] A T← 0 6 6 0

[47 ] A THI←T+2

[48] A TLO←T-2

[49] MINMETRIC←NPATTERN

[50] S←(N( PATTERN )ρ0

[51] A [52] L←10

[53] P←3

[54] NC←10

[55] NG←200

[56] METRICVECTOR←GENERATIONVECTOR←(NG+1)ρ0

[57] A

[58] SYSTEM←-1+?((N+1), (N+M) )ρ(L+1)

[59] COL←x(-1.5 + ?(N+M)ρ2)

[60) SYSTEM←SYSTEMx((N+1),(N+M))ρCOL

[61] AMIN←A←SYSTEM( N;

N]x~(N,N)ρ1,Nρ0

[62] BMIN←B←SYSTEM(

N;(N+

M)]

[63] CMIN←C←(1,N)ρSYSTEM[(N+1):

N]

[64] DMIN←D←(1,M)ρSYSTEM[(N+1);(N+

M)]

[65] SYSTEM[

N;

N]←A

[0] MAKECOPIES

[1] A

[2] AINPUT= SYSTEM

[3] AOUTPUT = SYSTEMC, ACOPY, BCOPY, CCOPY, DCOPY [4] COLINDEX←1 =?(M+N)ρP

[5] MTEMP1←((N+1),(M+N))ρ0

[6] MTEMP1←MTEMP1+((N+1),(M+N))ρCOLINDEX [7] MTEMP2←- 1+?(1+MTEMP1xL)

[8] STRING←x(-1.5 + ?(N+M)ρ2)

[9] COLSIGN←COLINDEXxSTRING

[10] MTEMP2←MTEMP2x((N+1),(M+N))ρCOLSIGN [11] SYSTEMOS YSTEMx~MTEMP1

[12] SYSTEMC←SYSTEMC+MTEMP2

[13] ACOPY←SYSTEMC[ N;

N]x~(N.N)ρl.Nρ0

[14] BCOPY←SYSTEMC[

N;(N+\M)]

[15] CCOPY←SYSTEMC[(N+1) ;

N]

[16] DCOPY←SYSTEMC[(N+1);(N+

M)]

[0] SOLVEQU

[1] AXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX[2] ASOLVES: X=Af(X)+Bf (I)

[3] AXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX[4] A

[5] A

[6] AXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX [7] AMAP 2D→1D

[8] AXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX [9] INPUT←(M,1)ρPATTERN[KKK;;]

[10] AXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX [11] ACOMPUTE CONSTANT TERM

[12] AINITIAL GUESS IS FIRST TWO TERMS

[13] ROF OPERATOR DECOMPOSITION SOLUTION

[14] AXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX [15] X0←TEMPl←BCOPY+.x

(1

(0

INPUT))

[16] X1←ACOPY+.x

(1

(0

X0))

[17] XOLD←X0+X1

[18] A

[19] TK←0

[20] LOOP1:

[21] TEMP2←ACOPY+.x

(1

(0

XOLD))

[22] XNEW←(TEMP1+TEMP2)

[23] DELTAX←XNEW-XOLD

[24] AXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

[25] ACONVERGENCE TEST

[26] AXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

[27] MAXERROR←+/I ,DELTAX

[28] → (MAXERROR=0)/END

[29] A

[30] A

[31] A

[32] TK←TK+1

[33] XOLD←XNEW

[34] →(TK≤10)/LOOP1

[35] END:

[0] OUTPUT

[1] A

[2] OUT1←CCOPY+.x (1 (0 XNEW))

[3] OUT2←DCOPY+.x

(1

(0

INPUT))

[4] S[KKK]←OUT1+OUT2

[0] SELECT

[1] AMETRIC

[2] A METRIC←+/~(S=T)

[3] METRIC←+/~((TLO≤S)^(S≤THI))

[4] →(METRIC≥MINMETRIC)/DONE

[5] AMIN←ACOPY

[6] BMIN←BCOPY

[7] CMIN←CCOPY

[8] DMIN←DCOPY

[9] MINMETRIC←METRIC

[10] DONE:

[0] SOLUTION

[1] ACHECKS FINAL SOLUTION

[2] ACOPY←A

[3 ] BCOPY←B

[4] CCOPY←C

[5] DCOPY←D

[6] KKK←1

[7] LOOP:

[8] SOLVEQU

[9] OUTPUT

[10] KKK←KKK+1

[11] → (KKK≤NPATTERN)/LOOP

[0] TPATTERN2

[1] AGENERATES 7x7 TRAINING PATTERNS

[2] HOR1← 7 7 ρ(21ρ0),dρ1)

[3] HOR2← 7 7 ρ(22ρ0),(5ρ1)

[4] HOR3← 7 7 ρ(23ρ0),(3ρ1)

[5] VER1←ΦHOR1

[6] VER2←ΦHOR2

[7] VER3←ΦHOR3

[8] DIAGDN1← 7 7 ρ1,(7ρ0)

[9] DIAGDN2←DIAGDN1

[10] DIAGDN2 [1;1]←DIAGDN2[7;7]←0

[11] DIAGDN3←DIAGDN2

[12] DIAGDN3(2;2]←DIAGDN3[6;6]←0

[13] DIAGUP1←ΘDIAGDN1

[14] DIAGUP2←ΘDIAGDN2

[15] DIAGUP3←ΘDIAGDN3

(^SUMMARY (dip

[0] SUMMARY

[1] APRGM OUTPUTS A NN DESIGN SUMMARY.

[2 ] ♡←' '

[3] ♡←'NUMBER OF INPUT NEURONS(M) = ' .«M

[4] ♡←'NUMBER OF HIDDEN NEURONS(N) = ' , *N

[5] ♡←'NUMBER OF OUTPUT NEURONS(O) = ',*1

[6] ♡←'NUMBER OF COPIES PER GENERATION (NC) = ' ,*NC

[7] ♡←'NUMBER OF PATTERNS ( NPATTERN) = ' , NPATTERN

[8] ♡←'RANGE OF ELEMENT VALUES(L) 0,+/-1,...,+/-L, L= ' ,*L

[9] ♡←'1/PROBABILITY OF ELEMENT CHANGE(P) = ',*P

[10] ♡←'NN TYPE: TYPES 1&2'

[11] ♡←' '

[12] ♡←'METRIC:'

[13] ♡←'THI = ' ,»THI

[14] ♡←'TARGET SEQUENCE(T) = ',»T

[15] ♡←'TLO = ' ,*TLO

[16] ♡←' '

[17] SOLUTION

[18] ♡←'SOLUTION(S) = ' , *S

[19] ♡←'TOTAL NUMBER OF TRIES (TRYCOUNT) = ' ,*TRYC0UNT

[20] ♡←'FRACTION OF TRIES SOLVED (FRACTION) = ' , ^FRACTION

[21] ♡←'NUMBER OF GENERATIONS (K) = ' ,»K

[22] ♡←'METRIC TIME HISTORY: 0 THROUGH K'

[23] METRICVECTOR[UK]

[24] ♡←'LAPSE TIME (DAYS -HRS -MINUTES) = ' ,»3↑2↓(STOP-START)

[25] ♡←'COMPLETION TIME STAMP: ' ,»STOP

[0]

[1] AXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

[2] APRGM COMPUTES THE OUTPUT FEATURE SPECTRUM OF A PATTERN SET.

[ 3 ] AXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

[4] ATIENO= TIE NUMBER FOR INPUT FILE.

[5] AM= INPUT DIMENSION.

[6] AN= NUMBER OF HIDDEN NEURONS.

[7] ANPATTERN= NUMBER OF TRAINING PATTERNS.

[8] ASTEPS FOR ACCESSING FILE:

[9] ABEFORE RUNNING PRGM DO: 'NNFILES'♡ FTIE 1.

[10] A

[11] AXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

[12] AlNITIALIZE SYSTEM

[13] AXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

[14] A

[15] INITIALB

[16] A

[17] AXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

[18] APATTERN LOOP

[19] AXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

[20] K←1

[21] PLOOP:

[22] A

[23] AXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

[24] AFEATURE LOOP

[25] AXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

[26] KK←1

[27] FLOOP:

[28] ACTIVITY

[2 9] → (TK≤ 10)/CONTINUE

[30] A♡←' '

[31] A♡←' SYSTEM NOT SOLVED'

[32] A♡←' PATTERN= ', K

[33] A♡←' FEATURE= ',

KK

[34] A →END

[35] CONTINUE:

[36] OUTPUTB

[37] SPECTRUM[K;KK;1]←S

[38] KK←KK+1

[39] →(KK≤4)/FLOOP

[40] K←K+1

[41] →K(≤NPATTERN)/ PLOOP

[42] ♡←' '

[43] ♡←'S PECTRUM CALCULATION COMPLET E D

[44] END:

[45]

[0] INITIALB

[1] AREAD DATA FROM A FILE.

[2] TIENO←1

[3 ] SYSTEM1←♡FREAD TIENO,1 ADIAGUP & DIAGDN.

[4] SYSTEM2←♡FREAD TIENO,2 AHOR & VER.

[5] A

[6] M←49

[7] N←25

[8] ADEFINE DETECTOR SYSTEM.

[9] A1←SYSTEM1[ N;

N]

[10] B1←SYSTEM1[

N;(N+ M)]

[11] C1←SYSTEM1[(N+1);

N]

[12] D1←SYSTEM1[(N+1);(N+

M)]

[13] A

[14] A2←SYSTEM2[ N;iN]

[15] B2←SYSTEM2[

N:(N+ M))

[16] C2←SYSTEM2[(N+1):

N)

[17] D2←SYSTEM2[(N+1);(N+

M)]

[18] A

[19] NPATTERN←

[20] K←KK←1

[21] SPECTRUM←(NPATTERN,4,1)ρ1

[0] ACTlVITY

[1] → (KK=1)/L1

[2] → (KK=2)/L2

[3] → (KK=3)/L3

[4] → (KK=4)/L4

[5] A

[6] L1: AHORIZONTAL

[7] INPUT←(M,1)ρPATTERN[K;;]

[8] ATEMP←A2

[9] BTEMP←B2

[10] →ITERATE

[11] A

[12] L2: ADIAGUP

[13] INPUT←(M,1)ρPATTERN[K;;]

[14] ATEMP←A1

[15] BTEMP←B1

[16] →ITERATE

[17] A

[18] L 3: AVERTICAL

[19] INPUT←(M,1)ρΦΦPATTERN[K;;]

[20] ATEMP←A2

[21] BTEMP←B2

[22] →ITERATE

[23] A

[24] L4: ADIAGDN

[25] INPUT←(M,1)ρΦΦPATTERN[K;;]

[26] ATEMP←A1

[27] BTEMP←B1

[28] → ITERATE

[29] A

[30] ITERATE:

[31] X0←TEMPl←BTEMP+.x (1

(0

INPUT))

[32] X1←ATEMP+.x

(1 (0

X0))

[33] XOLD←X0+X1

[34] TK←0

[35] LOOP1:

[36] TEMP2←ATEMP+,x

(1 (0

XOLD))

[37] XNEW←(TEMP1+TEMP2)

[38] DELTAX←XNEW-XOLD

[39] MAXERROR←+ /I,DELTAX

[40] →(MAXERROR=0)/END

[41] TK←TK+1

[42] XOLD←XNEW

[43] →(TK≤10)/LOOP1

[44] END:

[0] OUTPUTB

[1] → ((KK=1)v(KK=3))/L1

[2] → ((KK=2)v(KK=4))/L2

[3] A

[4] L1:

[5] CTEMP←C2

[6] DTEMP←D2

[7] →OUT

[8] A

[9] L2:

[10 ] CTEMP←C1

[11] DTEMP←D1

[12] →OUT

[13] A

[14] OUT:

[15] OUT1←CTEMP+.x (1 (O XNEW)) [16] OUT2←DTEMP+.x

(1 (O INPUT)) [17] S←OUT1+OUT2

[0] PRINTB

[1] SLOPE← ' -- ++ | | ++'

[2] K←9

[3] LOOP:

[4] ♡← ' '

[5] ♡← 'P' ,

K

[6] ♡←PATTERN(K;:]

[7] ♡←' '

[8] ♡←ΦSPECTRUM(K;

4 ;]

[9] ♡←SLOPE

[10] ♡←' '

[11] K←K+1

[12] → (K≤ 16)/LOOP

)

[0] TPATTERN3

[1] ADEFINES (16) PATTERNS FOR RESPONSE MODULE DESIGN.

[2] PATTERN←(16, 7, 7) ρ0

[3] P1←P3←P9← 7 7 ρ0

[4] M←Φ 9 9 ρ1. (ρ90)

[5] P1←M[1+

7;

7]

[6] P1[7;2)←0

[7] P3←M[7;

v7]

[8] P9←M[2+

7; 1+

7]

[9] P9[6;1]←0

[10] P11←M[2 +

l 7;2+

7]

[11] PATTERN[1; :]←P1

[12] PATTERN [ 2 ;:]←ΘP1

[13] PATTERN[3;:]←P3

[14] PATTERN[4; :]←ΘP3

[15] PATTERN [ 5 ;:]←ΦP1

[16] PATTERN [6 ;:]←ΦΘP1

[17] PATTERN [ 7 ::]←ΦP3

[18] PATTERN [ 8 ;:]←ΦΘP3

[19] PATTERN [9::]←P9

[20] PATTERN [10 ;:]←ΦP9

[21] PATTERN [11;:]←P11

[22] PATTERN [12;:]←ΦP11

[23] PATTERN [13;:]←ΘP9

[24] PATTERN [14;:]←ΦΘP9

[25] PATTERN [15;:]←ΘP11

[26] PATTERN [16;:]←ΦΘP11

[0] DESIGNC

[1] AXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX [2] APRGM DESIGNS MULTI -INPUT. MULTI -OUTPUT NN'S.

[3] AXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

[4] AM= NUMBER OF INPUT NEURONS

[5] AN= NUMBER OF HIDDEN NEURONS

[6] AO= NUMBER OF OUTPUT NEURONS

[7] A

[8] ANPATTERN= NUMBER OF TRAINING PATTERNS

[9] APATTERN[K;;]= INDEXED SET OF TRAINING PATTERNS

[10] AT= TARGET SEQUENCE

[11] ATHI= TARGET SEQUENCE UPPER BOUND

[12] ATLO= TARGET SEQUENCE LOWER BOUND

[13] AD= METRIC

[14 ] A

[15 ] AL= RANGE OF INITIAL VALUES FOR A.B.C MATRIX ELEMENTS [16] A RANGE = -L, ...,0.1,...,L

[17] AP= 1/ PROBABILITY OF AN ELEMENT CHANGE

[18 ] ANC= NUMBER OF COPIES PER GENERATION

[19] ANG= NUMBER OF GENERATIONS

[20] A

[21] AINITIALIZE

[22] A

[23] ^ ITIA CJ - ' .^t-4 ^T

[24] A

[25 ] FRACTION←TRYCOUNT←SOLVECOUNT←ABORTCOUNT←1

[26] K←1

[27] AXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

[28] A-----------GENERATION LOOP-------------- [29] AXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

[30] GLOOP:

[31] A

[32 ] KK←1

[33 ] AXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

[34 ] A--------------COPY LOOP----------------- ]

[35 AXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

[36] CLOOP:

[37] !COPIESC]

[38] A

[39 ] KKK←1

[40 ] AXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

[41 ] A-----------TRAINING LOOP---------------- [42] AXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

[43 ] TLOOP: ATRAINING LOOP

[44 ] TRYCOUNT←TRYCOUNT+ 1

[45] gOL EOU^

[46] AXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

[47] A-----------TEST FOR SOLUTION------------ [48] AXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX [49] →(TK≤10) /CONTINUE

[50] ♡← ' '

[51] ♡← ' SYSTEM NOT SOLVED'

[52] ♡← ' GENERATION= '.

K

[5 3] ♡← 'COPY*' ,

»KK

[54] ♡← ' PATTERN= ' ,*KKK

[55] ♡← 'ABORTCOUNT= ' ,

ABORTCOUNT

[56] FRACTION←SOLVECOUNT+TRYCOUNT

[57] ♡←' FRACTION OF COPIES SOLVED= ',

FRACTION

[58] ABORTCOUNT←ABORTCOUNT+ 1

[59] → (ABORTCOUNT>20000)/END

[60] → CLOOP

[61] CONTINUE:

[62 ] SOLVECOUNT←SOLVECOUNT+ 1

[ 63]

[64] KKK ←KKK+1

[65] AXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

[66] A-------------END TRAINING----------------

[ 67 ] AXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

[68 ] →KKK≤NPATTERN)/TLOOP

[69 ] AXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

[70] A-----------SELECT SURVIVOR---------------

[71] AXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

[72 ] SELECTC

[73] →(MINMETRIC=0)/END

[74] KK←KK+1

[75] → (KK≤NC)/CLOOP

[76] AXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

[77] A-------------END COPIES------------------

[ 78 ] AXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

[79] SYSTEM[

N;

N ]←A←AMIN

[80] SYSTEM[

N; (N+ M)]←B*-BMIN

[81] SYSTEM[(N+ O) ; N]←C←CMIN

[82 ] SYSTEM[(N+

O) ; (N+

M)]←D←DMIN

[ 83] SYSMETRI[←MINMETRIC

[84] ♡←' '

[85] ♡← ' GENERATION COMPLETED, NUMBER= ' ,

K

[86] ♡←' METRIC= ',

SYSMETRIC

[87] ♡←' '

[88] GENERATIONVECTOR[K]←K

[89] METRICVECTOR[K]←SYSMETRIC

[90] AXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

[91] A------------END GENERATIONS--------------

[ 92] AXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

[93] K←K+1

[94] → (K≤NG)/GLOOP

[95] AXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

[96] A-----------FINAL OUTPUT------------------

[97] AXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

[98] END:

[99 ] SYSTEM[

N ;

N]←A←AMIN

[100 ] SYSTEM[

N; (N+

M)]←B←BMIN

[101] SYSTEM[(N+

O) ;

N]←C←CMIN

[102 ] SYSTEM[ [N+

O) ; (N+

M)]←D←DMIN

[ 103 ] S YSMETRIC←MINMETRIC

[104 ] ♡←' '

[105] ♡←' FINAL SYSTEM: GENERATION = ' ,

K

[106 ] ♡←' METRIC = ' ,

SYSMETRIC

[107 ] GENERATIONVECTOR[K]←K

[108 ] METRICVECTOR[K]←SYSMETRIC

[0] INITIALC

[1] APRGM INITIALIZES MI/MO DESIGN ALGORITHM.

[2] ASYSTEM DIMENSIONS

[3] M←4

[4] N←10

[5] O←2

[6] AINPUTS

[7] NPATTERN←16

[8] PATTERN← ( NPATTERN , M , 1 ) ρ 1 AASSUMES INPUT IS Mx1 MATRIX.

[9] PATTERN[1 ; ; ]←HOR 1

[10] PATTERN[2 ; ;]←HOR2

[11] PATTERN [ 3;;]←HOR3

[12] PATTERN [ 4 ; ; ]←DIAGUP1

[13) PATTERN[5; ;]←DIAGUP2

[14) PATTERN[6; ;] ←DIAGUP3

[15) PATTERN [ 7 ; ;]←VER1

[16] PATTERN [ 8 ; ;]←VER2

[17] PATTERN[9;;]←VER3

[18] PATTERN [10; ;]←DIAGDN1

[19] PATTERN [11;;]←DIAGDN2

[20] PATTERN [12;:]←DIAGDN3

[21] AOUTPUTS

[22] T1HI

[23] T1←

[24] T1LO

[25] A

[26] T2HI

[27] T2←

[28] T2LO←

[29] S←( 2, NPATTERN )ρ0

[30] ASEARCH PARAMETERS

[31] MINMETRIC←NPATTERN

[32] L←10

[33] P←3

[34] NC←10

[35] NG←200

[36] METRICVECTOR←GENERATIONVECT0R←( NG+ 1 ) ρ0

[37] A DEFINE INITIAL SYSTEM

[38] SYSTEM←-1 + ?((N+O),(N+M))ρ(L+1) ACHOOSE RANDOM POS ELEMENTS.

[39] COL←x(1.5 + ?(N+M)ρ2) ACHANGE SIGN OF RANDOM COLUMNS

[40] SYSTEM←SYSTEMx((N+O),(N+M))ρCOL AGIVES POS & NEG ELEMENTS.

[41] AMIN←A←SYSTEM[

lN;

i )x~(N,N)ρl.Nρ0 ASUBMATRIX WITH ZERO DIAGONALS

[42) BMIN←B←SYSTEMClN;(N+

lM)3 ASUBMATRIX OF SYSTEM

[43) CMIN←C←(O.N)ρSYSTEM[(N+O);

N] ASUBMATRIX OF SYSTEM

[44) DMIN←D←(O.M)ρSYSTEM[(N+O):(N+

M)] ASUBMATRIX OF SYSTEM

[45] SYSTEM[ N;

N]←A AlNCLUDES ZERO DIAGONALS IN SYSTEM

/

O

[0] MAKECOPIESC

[1] A

[2] A INPUT= SYSTEM

[3] AOUTPUT=SYSTEMC ,ACOPY, BCOPY, CCOPY, DCOPY

[4] COLINDEX←1 = ?(M+N)ρP

[5] MTEMP1←((N+O),(M+N))ρ0

[6] MTEMP1←MTEMP1+((N+O),(M+N))pCOLINDEX

[7 ] MTEMP2← -1 + ?(1+MTEMP1xL)

[8] STRING←x(-1.5 + ?(N+M)ρ2)

[9 ] COLSIGN←COLINDEXxSTRING

[10] MTEMP2←MTEMP2x((N+O), (M+N))ρCOLSIGN

[11] SYSTEMC←SYSTEMx~MTEMP1

[12] S YSTEMC←SYSTEMC+MTEMP2

[13 ] ACOPY←SYSTEMC[ N:

iN]x~(N,N)ρ1.Nρ0

[14] BCOPY←SYSTEMC[

lN;(N+ M)]

[15] CCOPY←SYSTEMC[(N%D);

iN]

[16] DCOPY←SYSTEMC[(N

φ); (N+

M)]

[0] OUTPUTC

[1] A

[2] OUT1←CCOPY+.x (1 (O XNEW))

[3] OUT2←DCOPY+.x

(1

L (0

INPUT))

[4] S[;KKK]←OUT1+OUT2

[0] SELECTC

[1] AMETRIC FOR O=2.

[2] METRIC1←+/~((T1LO≤S[1:])^(S[1;]≤T1HI))

[3] METRIC2←+/~((T2LO≤S[2:])^(S[2;]≤T2HI))

[4] METRIC+-METRIC1+METRIC2

[5] A

[6] →(METRIC≥MINMETRIC)/DONE

[7] AMIN←ACOPY

[8] BMIN←BCOPY

[9] CMIN←CCOPY

[10] DMIN←DCOPY

[11] MINMETRIC←METRIC

[12] DONE:

Λ

[0 ] SOLUTIONC

[1] ACHECKS FINAL SOLUTION

[2] ACOPY←A

[ 3] BCOPY←B

[4] CCOPY←C

[5] DCOPY←D

[6] KKK←1

[7] LOOP:

[8] SOLVEQU

[9] OUTPUTC

[10] KKK←KKK+1

[11] → ( KKK≤ NPATTERN )/ LOOP

VDESIGNF(D3V

[0] DESIGNF

[1] AXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

[2] APRGM COMPUTES THE OUTPUT FEATURE SPECTRUM FOR THE FINE ADJ MODULE

[3] AOVER A PRESET RANGE OF OFFSETS.

[ 4] AXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

[5] ATIENO= TIE NUMBER FOR INPUT FILE.

[6] AM= INPUT DIMENSION.

[7] AN= NUMBER OF HIDDEN NEURONS.

[8] ASTEPS FOR ACCESSING FILE:

[9] ABEFORE RUNNING PRGM DO: 'NNFILES' DFTIE 2.

[10] A

[11] AXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

[12] AINITIALIZE SYSTEM

[13] AXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

[14] A

[15] INITIALE

[16] A

[17] AXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

[18 ] A PATTERN LOOP

[19] AXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

[20 ] SH←SV←-3

[21 ] OUTERLOOP:

[22 ] SH←-3

[23 ] INNERLOOP:

[24] PATTERN←SH SHIFT SV

[25 ] A

[26 ] AXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

[27] A FEATURE LOOP

[28 ] AXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

[2 9] KK←1

[30] FLOOP:

[31] ACTIVITE

[32 ] →(TK≤10) /CONTINUE

[33 ] S←0

[34] →SKIP

[35 ] A ♡←' '

[36 ] A ♡←' SYSTEM NOT SOLVED'

[37] A ♡←' FEATURE= ' ,»KK

[38] A →END

[39 ] CONTINUE:

[40 ] OUTPUTE

[41] SKIP: SPECTRUM[KK; 1]←S

[42] KK←KK+1

[43] →(KK≤4)/FLOOP

[44 ] APRINT RESULTS

[45] ♡←' '

[46] ♡←'PATTERN ='

[47] PATTERN

[48] ♡←' HORIZONTAL SHIFT= ',(*SH),' VERTICAL SHIFT= ',fSV

[49] ♡←' '

[50] ♡←'SPECTRUM ='

[51] 6 0 *ΦSPECTRUM

[52] D←' NORTH SOUTH EAST WEST'

[51] SH←SH+1

[54 ] →( SH≤ 3)/ INNERLOOP

[55] SV←SV+1

[56] →( SV≤ 3) /OUTERLOOP

[57] END: ;iALE(Q3v:

[0] INITIALE

[1] AREAD DATA FROM NNFILES.ASF.

(2] TIENO←2 AEDIT-IN CORRECT TIE NUMBER.

[3] SYSTEM←DFREAD TIENO^5)AFINE ADJ MODULE.

[4] A

[5] M←21

[6] N←25

[7] ADEFINE DETECTOR SYSTEM.

[8] ATEMP←SYSTEM[lN:iN]

[9] BTEMP←SYSTEM[lN:(N+lM)]

[10] CTEMP←SYSTEM[(N+1):iN]

[11] DTEMP←SYSTEM[(N+1);(N+lM)]

[12] A

[13] KK←1

[14] SPECTRUM← (4,1)ρ1

[0] ACTIVITE

[1] APRGM COMPUTES NEURON ACTIVITY FOR FINE ADJUST.

[2] → (KK=1)/L1

[3] → (KK=2)/L2

[4] → (KK=3)/L3

[5] → (KK=4)/L4

[6] A

[7] L1: ANORTH

[8] TEMP←Φ PATTERN

[9] INPUT←(M.1)ρTEMP[;l3]

[10] → ITERATE

[11] A

[12] L2: ASOUTH

[13] TEMP←ΦΦ PATTERN

[14] INPUT← (M.1)ρTEMP[;

3]

[15] → ITERATE

[16] A

[17] L3: AEAST

[18] TEMP←Φ PATTERN

[19] INPUT←(M.1)ρTEMP[;

3]

[20] → ITERATE

[21] A

[22] L4: AWEST

[23] TEMP← PATTERN

[24] INPUT←(M.1)ρTEMP[;

3]

[25] → ITERATE

[26] A

[27] ITERATE:

[28] X0←TEMP1←BTEMP+ . x [ (‪

L ( 0

f INPUT ))

[29] X1←ATEMP+.x

(1

(0rX0))

[30] XOLD←X0+X1

[31] TK←0

[32] LOOP1:

[33] TEMP2←ATEMP+.X

(1

L(0

ΓXOLD))

[34] XNEW←(TEMP1+TEMP2)

[35] DELTAX←XNEW- XOLD

[36] MAXERROR←+ / I, DELTAX

[37] →(MAXERROR=0)/END

[38] TK+TK+1

[39] XOLD← XNEW

[40] →(TK≤10 )/LOOP1

[41] END:

[0]

OUTPUTE

[1] A

[2] OUT1←CTEMP+.x (1 (0 XNEW))

[3] OUT2←DTEMP+.x (1

(0

INPUT))

[4] S←OUT1+OUT2

[0] Z+-SH SHIFT SV

[1] APRGM SHIFTS A 7x7 PATTERN HORIZONTALLY AND/OR VERTICALLY.

[2] ASH= HORIZONTAL SHIFT: 1, 2.... (EAST); -1, -2,... (WEST).

[3 ] ASV= VERTICAL SHIFT: 1, 2... (NORTH); -1, -2,... (SOUTH).

[4] ACELL= A 7X7 MATRIX DEFINING THE CELL.

[5] A

[6] TEMP1←TEMP2← 7 7 ρ0

[7] AHORIZONTAL SHIFTS:

[8] →(SH>0)/LINE1

[9] →(SH<0)/LINE2

[10] TEMP1←CELL

[11] →VERT

[12] LINE1:TEMP1[;SHAl7]←CELL[ ;[(7-SH)]

[13] →VERT

[14] LINE2:TEMÅ1[:[(7+SH)]←CELL[;(-SH)[7]

[15] →VERT

[16] AVERTICAL SHIFTS:

[17] VERT:

[18] →(SV>0)/LINE3

[19] →(SV<0)/LINE4

[20] TEMP2←TEMP1

[21] → END

[22] LINE3:TEMP2[[(7-SV);]←TEMP1[SVA[7;]

[23] →END

[24] LINE4:TEMP2[(-SV)i[7;]←TEMP1[[(7+SV);]

[25] →END

[26] END:Z←TEMP2

Claims

Claims 1. A method for applying, in a program controlled computer, a genetic algorithm search to determine an optimimum set of values, each value being associated with a pair of elements drawn from a universe of N elements, N an integer greater than zero, where the utility of any possible set of said values may be measured, comprising

(a) assembling an initial possible set of said values, said values being organized in a matrix whose rows and columns correspond to said elements,

(b) applying a genetic algorithm operator to generate successor matrices from said matrix,

(c) performing matrix computations on said successor matrices to generate metrics of the relative utilities of said successor matrices of values,

(d) selecting a surviving matrix from said successor matrices on the basis of said metrics, and

(e) repeating steps (b) through (d) until the metric of the selected surviving matrix is satisfactory

2. The method of claim 1 adapted for determining, for neurons of a neural network, a set of interconnection weights which will enable the network to approximate a prespecified output when presented with a predetermined set of input data, wherein

said step of assembling comprises assembling an initial possible set of interconnection weights,

said step of performing matrix computations comprises performing matrix computations on said input data and said successor matrices to generate an output of each said successor matrix,

said method further comprising computing a metric corresponding to the difference between each said output and the prespecified output, and wherein

said successor matrix is selected based on said metrics.

3. The method of claim 2 further comprising forcing said interconnection weights in said matrices to comply with constraints that correspond to the

architecture of said neural network.

4. The method of claim 2 further comprising organizing said interconnection weights in said matrices so that the interconnection weights leading to a given neuron all appear in a single row (or a single column) of each said matrix, and the interconnection weights leading from a given neuron all appear in a single column (or a single row) of each said matrix.

5. The method of claim 3 wherein said constraints include requiring that the interconnection weights between a neuron and itself be zero.

6. The method of claim 3 wherein said constraints include requiring that the interconnection weights leading from a given neuron all be of the same sign.

7. The method of claim 2 wherein said neural network comprises a layer of input neurons, a layer of hidden neurons , and a single output neuron .

8. The method of claim 2 wherein said step of applying a genetic algorithm operator comprises applying a mutation operator.

9. The method of claim 8 wherein applying said mutation operator comprises randomly changing a portion of said interconnection weights in a given matrix.

10. The method of claim 8 wherein no other genetic algorithm operator is applied.

11. The method of claim 2 wherein said interconnection weights are organized in more than one matrix based on layers of neurons in said neural network,

12. The method of claim 2 wherein said input data set comprises binary values which represent edges of an image.

13. The method of claim 2 wherein said step of selecting a successor matrix on the basis of said metrics comprises testing whether a metric lies within a band between upper and lower bounds .

14. The method of claim 2 wherein said step of selecting a successor matrix on the basis of said metrics comprises testing whether metrics with respect to predetermined input data sets lie above a passband or below a stop band.