JPH07160664A - Neural network - Google Patents

Abstract

PURPOSE:To restore the three-dimensional shape of an object well from two parallactic pictures by using a neural network. CONSTITUTION:A neuron element part 1 is provided with n neuron elements, the neuron elements are divided into m blocks and block sequential updating is performed. Then, the output state of the respective neuron elements is determined by a saturated type linear transfer function and the outputs of the respective elements are values between '0' and '1'. Also, parameters p, q and gammafor determining the saturated type linear transfer function are changed by a prescribed form every time of the state updating. Also, in this neural network, by constraint conditions. The coupling of excitability is provided not only in the same parallax direction but also in the depth direction and the height direction.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a neural network and, more particularly, to a device for restoring a three-dimensional shape of an object shown in an image having two parallax images (hereinafter referred to as parallax image). And a suitable neural network.

[0002]

2. Description of the Related Art In recent years, two TV cameras are mounted on a robot, and a three-dimensional shape of an object in front is restored and recognized from two parallax images obtained by these TV cameras to perform a predetermined work. It has been considered to avoid obstacles and let the vehicle run automatically.

Various methods can be considered for reconstructing the three-dimensional shape of an object from such parallax images. Particularly, a method using a neural network (hereinafter abbreviated as a neural network) is Hirai-Fukushima. The model or Marr-Poggio model is known.

Although the Hirai-Fukushima model and the Marr-Poggio model have different constraint conditions and the like as described later, these models were applied to a device for restoring the three-dimensional shape of an object from two parallax images. The overall configuration in this case is almost the same, and is the configuration as schematically shown in FIG.

In FIG. 10, an image input unit 2 includes lenses 4, 6 and two light receiving units 5 including CCD sensors and the like.
An image of the target object 3 is formed by the lens 4 on the light receiving section 5, and an image of the target object 3 is formed by the lens 6 on the light receiving section 7. As a result, two parallax image data obtained by imaging the target object 3 from different viewpoints with substantially equal distances are obtained from the light receiving unit 5 and the light receiving unit 7.

Then, these image data are input to the neural network 1. Specifically, it is as follows. Now, for example, in the neural network 1, the number of neuron elements in the i direction is 6, the number of neuron elements in the j direction is 6, and the height direction is h.
Assuming that the number of neuron elements of 6 is 6, the image obtained by the light receiving unit 7 is made into an image of 6 dots in the i direction and 6 dots in the h direction for a total of 36 dots as shown in FIG. 11A. The image obtained in part 5, as shown in FIG. 11B,
An image of 36 dots, 6 dots in the j direction and 6 dots in the h direction, is formed.

The height of the image shown in FIG. 11A is h.
The intensity information of each dot at the position, that is, the density information, is shown in FIG.
Is input to the surface 1 _h of i direction neuron elements at a height h of the neural network 1 as shown in 2 as an input bias, as well as the image shown in FIG. 11B, the height of each dot in the h position As shown in FIG. 12, the intensity information is input as an input bias to the neuron element in the j direction of the surface 1 _h at the height h of the neural network 1. Note that FIG.
0, in FIG. 12, neuron elements are indicated by white circles or black circles. The white circles indicate that the output of the neuron element is “0”, and the black circles indicate that the output of the neuron element is “1”. It merely exemplifies the output state. When inputting the dot intensity information of the image, instead of directly inputting the dot intensity information corresponding to the neuron element,
Intensity information of dots corresponding to neuron elements and intensity information of dots in the vicinity thereof may be input.

Therefore, in FIG. 10, the neural network 1 is shown as the two-dimensional arrangement of the neuron elements, but this shows the arrangement of the neuron elements at a certain height.

Now, in order to facilitate understanding, the height h
Is considered within a certain plane, the intensity information of the image input to the neural network 1 is sent along the lines intersecting each other as shown in FIG. 12, and the neuron elements located at the intersections of the intersecting lines. The difference between the intensity information of the corresponding dots in the two images flows in as an input bias. That is, for example, the difference between the dot intensity information indicated by Q _i2 and the dot intensity information indicated by Q _j4 flows into the neuron element indicated by 8 in FIG. 12 as an input bias.

Then, the output of each neuron element is multiplied by the synaptic coupling weight and becomes an input to another neuron element. In other words, the value obtained by multiplying the output of another neuron element by the synapse coupling weight and the input bias flow into each neuron element, and the result of performing the transfer function calculation on this is the output of the neuron element at the next time. It becomes.

The state of each neuron element is updated as described above. The state of the neuron element that outputs "1" when the state update is repeated and becomes a steady state is 3 in the target object 3. This means that the relative position of the dimensional shape is estimated. For example, to make it easier to understand, if a normal line is drawn from the neuron element that outputs "1" to the reference plane in the steady state, as shown in Fig. 13A, Since the surface of the target object is located at the position of the tip of the perpendicular, the target object is shown in FIG. 13B.
It can be presumed that the shape is a saddle type as shown in.

Therefore, a shape restoration section (not shown) is provided on the output side of the neural network 1, and the shape restoration section is made to take in the coordinates of the neuron element which outputs 1 when the steady state is reached, The process of restoring the three-dimensional shape of the target object may be performed based on the coordinates.

In the Hirai-Fukushima model, a function that outputs only 0 or 1 is used as the transfer function. That is, the Hirai-Fukushima model is a so-called binary model. The synaptic connection between neuron elements is only inhibitory connection, and smoothing processing is performed when reconstructing a three-dimensional shape. The Hirai-Fukushima model and its improved model are described in detail on pages 97-102 of the IEICE Technical Report NC91 (1991).

Also, the Marr-Poggio model is the same as the Hirai-Fukushima model in that it is a binary model, but one point in one image corresponds to at most one point in the other image. , And the excitatory connection is provided only between the neuron elements located in the vicinity in the plane where the parallax is constant. For the Marr-Poggio model, see “Biological Cybernetics 28 (197
8) '', pages 223 to 239, `` Analysis of a Cooperativ ''
Learn more about the paper entitled "e Stereo Algorithm".

[0015]

However, Hirai-
In the Fukushima model, smoothing processing is required when reconstructing a three-dimensional shape, so not only the configuration becomes complicated, but also
There is a problem that it takes a long processing time.

Further, in the above paper, when a three-dimensional shape of an object is restored from a random dot stereogram using a model improved from the Hirai-Fukushima model, there is a risk that a wrong neuron will fire if the dot density becomes large. Since it is stated that the three-dimensional shape is satisfactorily restored using this model, there is a limitation that the number of dots of the two parallax images to be input cannot be increased so much.

Further, in the Marr-Poggio model, since excitatory coupling is included only between neuron elements located in the vicinity of the plane with a constant parallax, plane reconstruction can be performed relatively well. However, there is a problem that it is very difficult to reconstruct an object having a curved surface that continuously changes in the depth direction.

Further, since the Hirai-Fukushima model and the Marr-Poggio model are binary models, it is very difficult to properly restore the three-dimensional shape of the target object. This is known as a result of simulation.

As a method for solving these problems, a simulated annealing method and the like have been proposed, but since this method realizes an equilibrium state by stochastic repetition of states, the processing time is extremely long. Very unrealistic.

In order to improve the processing efficiency in the Hirai-Fukushima model and the Marr-Poggio model, it is possible to update the state of a plurality of neuron elements at the same time. In this case, the output of the neuron element is in an oscillating state. Therefore, there is a possibility that it will not be in a steady state.

The present invention solves the above-mentioned problems, regardless of the shape of the object, whether the object is a flat surface or a curved surface that continuously changes in the depth direction. It is an object of the present invention to provide a neural network capable of excellently restoring a three-dimensional shape from a parallax image.

Another object of the present invention is to provide a neural network which can efficiently perform processing and can restore a three-dimensional shape at high speed without requiring post-processing such as smoothing processing.

[0023]

In order to achieve the above object, in the neural network of the present invention, neuron elements are three-dimensionally arranged, and each neuron element forms a predetermined synaptic connection with a neighboring neuron element. In this mutual connection type neural network, the neuron element is divided into m blocks, and the output state vector x ^(l of the neuron element in the l-th block (where l = 1, 2, ..., M) is ⁾ _Flows into the k-th block (where k = 1, 2, ..., M ⁾ via the coupling weight matrix W _kl , and the k-th block is further input with the similarity of the feature amounts of the two parallax images. Adding as the vector θ ^(k) , the total input vector u ^(k) to the k-th block represented by the equation (1 ⁾ is obtained,

[0024]

[Equation 4]

A value obtained by multiplying the total input vector u ^(k) by a parameter α for controlling the influence of an external input and the output state vector x ^(k) of the element in the k-th block are output before the state update. For each component of the vector obtained by adding the value multiplied by the parameter γ that controls the influence of the value, with p and q as parameters

[0026]

[Equation 5]

The vector obtained by applying the saturated linear transfer function represented by the following is defined as the output state vector x ^(k) of the k-th block at the next time,

[0028]

[Equation 6]

In the process of updating the states all at once by the equation represented by the formula, performing the state updating sequentially for each block, and in the process of performing the state updating, the parameter p of the saturation type linear transfer function is changed every time the state is updated. When the minimum eigenvalue of the diagonal block matrix W _kk of the synapse coupling matrix W is min {λ _k }, the parameter q is changed to _−0.
It is characterized in that it is changed in a constant value or in a desired manner within a range of α × min {λ _k } / 2 or more, and the sum of p and q is γ.

Here, the connection between the neuron elements is the excitatory synapse between the neuron elements arranged in the parallax constant direction and between the neuron elements arranged in the depth direction. It is desirable to have a bond.

[0031]

The neural network of the present invention is an interconnected neural network in which the neuron elements are three-dimensionally arranged. The neuron element is divided into m blocks. It is arbitrary how many neuron elements form one block. That is, assuming that the number of neuron elements of the neural network is n, 1 ≦ m ≦ n. If m = 1, one neuron element constitutes one block, and if m = n, the neural network One block is composed of all neuron elements.

Now, the k-th block (however, k = 1, 2, ...,
m) is the output state vector x ^(l) of the neuron element in the l-th block (where l = 1, 2, ..., M ⁾ multiplied by the coupling weight matrix W _kl, and the input bias vector θ.
^(k) flows in. This input bias vector θ ^(k) is
It is what shows the similarity of the feature-value of two parallax images.

Then, the total input vector u ^(k) to the k-th block represented by the equation (1) is obtained by these inflows, and the influence of the input from the outside is controlled on the total input vector u ^(k). Of the vector obtained by adding the value obtained by multiplying the output state vector x ^(k) of the element in the k-th block by the parameter γ that controls the influence of the output value before the state update The saturated linear transfer function represented by the equation (2) is applied to each component, and the resulting vector is the output state vector x ^(k) of the k-th block at the next time.
As described above, the state is updated all at once by the expression represented by the equation (3), and this state update is sequentially performed for each block.

In the process of updating the state as described above, the parameter p of the saturated linear transfer function is changed so as to approach 0 each time the state is updated. Also,
Although q is also a parameter of the saturated linear transfer function, this parameter q is −α × mi when the minimum eigenvalue of the diagonal block matrix W _kk of the synapse coupling matrix W is min {λ _k }.
It may be a constant value in the range of n {λ _k } / 2 or more, or may be changed in a desired manner. Then, the sum of p and q is made γ.

As described above, the feature of the present invention is that block-sequential state update is first performed. That is, the neuron elements in the block synchronously update the state and sequentially update the state from the first block to the m-th block, which allows efficient and high-speed processing.

Further, the present invention is characterized in that the saturation type linear transfer function represented by the equation (2) takes a continuous value between 0 and 1 within the range of p <y <q. Is.
Of course, when the final steady state is reached, the output of the neuron element becomes 0 or 1, but until the steady state is reached, the output of the neuron element takes 0 or 1 or a value in between. . That is, the neural network of the present invention is not a conventional binary model.

Furthermore, the present invention is characterized in that the parameters p, q and γ of the saturated linear transfer function are not fixed values but are changed in the process of updating the state.
Of course, the value of q may be a constant value in the process of updating the state, as described above. This makes it possible to avoid the drawbacks of the conventional binary model.

By interacting with each other, the conventional problems are solved, the processing efficiency is improved, the processing speed is increased, and the shape of the target object can be restored well regardless of the shape of the target object. .

[0039]

Embodiments will be described below with reference to the drawings.
FIG. 1 is a diagram showing a configuration of an embodiment of a neural network according to the present invention, and FIG. 2 is a neuron element section 1 shown in FIG.
FIG. 3 is a diagram showing a configuration example of each block of 1, in which 11 is a neuron element unit, 12 is an operation evaluation unit, 13 is a parameter setting unit, 21 is a total input operation unit, and 22 and 23 are operation units. Note that, here, it is assumed that the number of dots in the vertical direction and the number of dots in the horizontal direction of the parallax image are the same.

The neuron element section 1 includes n neuron elements, and these neuron elements are divided into m blocks. That is, the first block is x
_{, 1} , x _1,2 , ..., X _{1, s1} consisting of s1 neuron elements, and the second block is x _2,1 , x _2,2 ,.
It is composed of s2 neuron elements of _{2, s2} , and similarly,
A block is composed of sm neuron elements x _{m, 1} , x _{m, 2} , ..., X _{m, sm} . That is, s1 + s2 + ...
+ Sm = n.

In the neuron element, as described with reference to FIGS. 10 to 12, image data of two pieces of image information captured from different viewpoints with substantially equal distances are input,
Although the state is updated, the mode of the state update will be described in detail below.

In the neural network shown in FIG. 1, the state update is a block-sequential state update. That is, the n neuron elements are divided into m blocks, the neuron elements in each block update their states synchronously, and the blocks from the 1st block to the mth block are updated.
The state is sequentially updated up to the block.

Then, now, a state vector x = (x ₁ , ..., X _n ) ^T (0≤0) representing the states of n neuron elements.
x _l ≤ 1; l = 1,2, ..., n) with m subvectors x
^(k) (however, if k = 1, 2, ..., M), each subvector updates its state according to the following equations (1) and (3).

[0044]

[Equation 7]

Here, θ ^(k ) in the equation (1) is a subvector of the threshold vector θ = [θ ^{(1) T} , θ ^{(2) T} , ..., θ ^{(m) T} ], and α is It is an arbitrary coefficient. Also, W _kl in the equation (1) is _calculated from the 1st block (where l = 1,2, ..., m) to the kth block.
It is a small matrix showing synaptic connections to blocks, and is related to the synapse matrix W by the following formula. The coefficient γ will be described later.

[0046]

[Equation 8]

The function represented by Sat in the equation (3) is a saturated linear transfer function defined by the following equation (2), and this function Sat acts on each vector component.

[0048]

[Equation 9]

Now, p and q in the equation (2) are parameters of the saturation type linear transfer function Sat, and it is as follows when considering what values these parameters p and q should take. .

Now, it is assumed that the energy function for the symmetric matrix W = (wl _k ) is defined by the following equation (5).

[0051]

[Equation 10]

Here, the coefficient matrix W and the coefficient vector θ of the energy function E are calculated from the two parallax images to be 3 of the target object.
It is determined by comparing with the coefficient of the evaluation function of the optimization problem of restoring the three-dimensional shape. Therefore, in order to favorably restore the three-dimensional shape of the target object, it is given by the equation (5). It can be seen that the minimum value or good local minimum value of the energy function E should be obtained. The method of obtaining the coefficient matrix W and the coefficient vector θ will be described later.

Now, using the k-dimensional vector e ^(k) of e ^(k) = (1, 1, ..., 1), two functions F and G
Are defined as the following equations (6) and (7), respectively.

[0054]

[Equation 11]

A method of defining such a function and obtaining a solution is a so-called deterministic annealing. According to the results of analysis of these functions or various simulations, the parameters p, It has been confirmed that the value of the function G can be decreased by updating the state of the neuron element when q and γ satisfy the following expressions.

[0056]

[Equation 12]

Here, λ _k is a set of eigenvalues of the diagonal block matrix W _kk of the synapse coupling matrix W, and min {λ _k } is the minimum value among the eigenvalues, that is, the minimum eigenvalue.

Next, regarding the parameter p, p
When variously changed, the following was found.

When p is sufficiently large, a point slightly deviated from the minimum point of the function F becomes the only minimum point of the function G,
The minimum point of the energy function E does not appear as the minimum point of the function G.

When p is sufficiently close to 0, all the minimum points of the energy function E appear as the minimum points of the function G, and the minimum points of the function G are limited to the minimum points of the energy function E.

When the value of p is in the middle of the above value, the local minimum point of the energy function E may not be the minimum point of the function G, and the local minimum point of the energy function E is eliminated.

Regarding this, the following is a description of a simple case.

Now, assuming that the energy function E is expressed by a homogeneous linear equation regarding two variables x and y of E (x, y) =-0.8xy + 0.2x + 0.2y, the graph of the energy function E is shown in FIG. 3A. As shown in. From this graph, the energy function E has a minimum point a and a local minimum point b, and using the method of simply descending the energy function E by the steepest descent method, either the minimum point a or the local minimum point b is used. It turns out that it converges on one side.

At this time, the functions F and G are as follows: F (x, y) = 0.5 (x (x-1) + y (y-1)) G (x, y) = E (x, y) ) + F (x, y) and the graphs are as shown in FIGS. 3B and 3C, respectively. It can be seen from the graph of FIG. 3B that the function F has the minimum point c as the only minimum point. Further, from the graph shown in FIG. 3C, it is understood that the local minimum point b of the energy function E is not the minimum point in the function G. That is, by adding the function F to the energy function E, there is a loophole from the local minimum point of the energy function E to the minimum point of the energy function E having a smaller value.

As described above, the minimum point of the function G, which is newly generated by adding the function F to the energy function E, is eliminated in the process of approaching p, so that the state update process is repeated. It can be seen that when p is gradually brought closer to 0 in, it can finally be converged to the minimum point or a good minimum point of the energy function E.

From the above, it was found that it is preferable to repeat the state update while controlling the values of the parameters p, q and γ according to the following equations.

[0067]

[Equation 13]

As a result of various simulations, it was confirmed that q may be a constant value within the range defined by the equation (4-c), or may be appropriately changed within this range.

By repeating the state update as described above, the energy function E finally converges to the minimum point or a good minimum point, and therefore "1" is set when the output of the neuron element reaches the steady state. The three-dimensional shape of the target object can be obtained by the relative positional relationship of the outputting neuron elements.

In the present invention, the state update as described above is repeated, but in order to perform the state update, it is necessary that the coupling weight W between neuron elements and the input bias θ be fixed. Therefore, a method of determining the coupling weight W between the neuron elements and the input bias θ will be described below.

In order to determine the coupling load and the input bias, it is necessary to first set the physical constraint conditions. Here, the following three conditions are defined as constraint conditions.

(1) One point in one image matches with at most one point in the other image (2) Parallax changes smoothly over almost the entire image (3) Has the same characteristics Only the ones match the constraint conditions (1) and (2) above that determine the connection state between the neuron elements.
It is expressed by equation (9).

[0073]

[Equation 14]

Here, D is the maximum parallax. For example, D
If = 25 is set, this means that the parallax shift amount of two given parallax images can be recognized up to a depth of 25 steps.
N is the number of dots in the vertical and horizontal directions of the parallax image. Therefore, if N = 128, the number of dots in the left and right images is 128 × 128 = 16384.

FIG. 4 shows the coupling state between the neuron elements which is determined by the constraint conditions (1) and (2). The description of FIG. 4 is as follows. Now, pay attention to a certain neuron element. This element of interest is □ in Fig. 4.
Assuming that it is in the position indicated by,
The neuron element at the position shown by ◯ has excitatory connection, and the neuron element at the position shown by Δ has inhibitory connection. It has no connection with the neuron elements at other positions.

As is apparent from FIG. 4, in this neural network, not only between neighboring neuron elements located in the same parallax direction but also between neighboring neuron elements located in the depth direction and in the height direction. Since there is also excitatory coupling between neighboring neuron elements located at, it is possible to restore the three-dimensional shape of the target object regardless of the shape of the target object. .

Note that S in the equation of E ₃ indicates an index set indicating a set of neuron elements having excitatory coupling with the target element indicated by □ in FIG.

The constraint condition (3) is a condition for determining the input bias and is expressed by the following equation (10).

[0079]

[Equation 15]

Here, Ψ is a threshold value, and R and L are intensity values of the left and right images, respectively. V is an index set of a set of neuron elements located in a predetermined range in which h of point (i, j, h) is constant, and here, h of point (i, j, h) is constant. It is assumed that the index set represents a set of neuron elements located within a certain 3 × 3 range.

Now, when the output of the neuron element i is represented by X _i , the above constraint condition is represented by an evaluation function of the form (11), where A, B and C are arbitrary constants.

E = AE ₁ + BE ₂ + CE ₃ (11) Further, at this time, if the energy function is represented by the following equation (12), the above equation (11) and the following (1)
The coupling coefficient W _ijh and the input bias θ _ijh can be determined by comparing the coefficient with the energy function of the equation (2). The input bias θ _ijh indicates the degree of similarity between the feature amounts of the two input parallax images.

[0083]

[Equation 16]

Specifically, it is as follows. The following expression (13) is obtained from the expressions (11) and (12).

[0085]

[Equation 17]

[0086] In this equation (13), both sides of the x _IJH ² by replacing x _IJH, it is possible to deform the quadratic form in homogeneous primary format. Because this deformation does not change the position of the minimum point of the energy function.

Next, these two sides are differentiated by x _ijh ,

[0088]

[Equation 18]

By using, the following expression (14) is obtained.

[0090]

[Formula 19]

Here, each term on the right side of the equation (14) is as follows.

[0092]

[Equation 20]

Further, here, 2B and 2C are newly replaced by B and C, respectively, and the coefficients on both sides of the equation (14) are compared to obtain the coupling weight and the input bias as follows.

[0094]

[Equation 21]

Here,

[0096]

[Equation 22]

Are both Kronecker deltas, and

[0098]

[Equation 23]

It is

Next, the operation of the configuration shown in FIGS. 1 and 2 will be described. First, the coupling load W and the input bias θ are determined as described above. Then, the determined coupling weight W is given to each block of the neuron element unit 11 of FIG.

After that, the state update is started. Now, the state update of the l-th block is completed, and then the k-th block is updated.
Explaining the case where the state of the block is updated, first, when updating the state of the l-th block, the parameter setting unit 13 sets the values of the parameters p, q, and γ and gives them to the blocks of the neuron element unit 11. At the same time, the input bias θ determined for each block is made to flow.

As a result, the total input calculation unit 21 of the l-th block obtains the total input vector u ^(l) by the equation (1). The calculated total input vector u ^(l) is input to the arithmetic unit 22. Further, the calculation unit 22 includes the parameter setting unit 1
3, γ is given, and the output state vector x ^(l) (t) at the time of the state update performed last time is fed back from the calculation unit 23.

As a result, the calculation unit 22 calculates γx ^(l) (t) + αu ^(l) (21) using the preset coefficient α and outputs it to the calculation unit 23.

The arithmetic unit 23 uses the parameters p and q given from the parameter setting unit 13 to perform the arithmetic operation of the equation (3) on the value input from the arithmetic unit 22. As a result, a new output state vector x ^(l) (t + 1) is obtained and the state is updated. This output state vector x
^(l) (t + 1) is fed back to the calculation unit 22 and is output to the calculation evaluation unit 12. As described above, the output state vector fed back to the arithmetic unit 22 is used in the next state update.

The operation evaluation section 12 monitors the output states of all the neuron elements based on the output state vector input from each block, and evaluates whether or not the steady state is reached. In this evaluation, the output state vector is time t and time (t
+1) and the value of the predetermined energy function is the time t and the time (t + 1).
This can be done by checking whether or not the output state vector has changed, or whether the output state vector satisfies the constraint conditions. If it is determined that the steady state has been reached, all neuron elements that output 1 Output the coordinates of. As described above, the three-dimensional shape can be restored based on this output.

However, when it is determined that the steady state has not been reached yet, the operation evaluation section 12 instructs the parameter setting section 13 to output the next value of the parameters p, q, γ. The new p, q, and γ parameter values set by the parameter setting unit 13 are given to each block of the neuron element unit 11, and at the same time, the input bias θ flows into each block.

As a result, the total input computing unit 21 of the kth block computes the total input vector u ^(l) by the equation (1), the computing unit 22 computes the equation (21), and the computing unit 23 creates a new one. Output state vector x ^(k) (t + 2) is obtained, and the state is updated.

This output state vector x ^(k) (t + 2) is fed back to the arithmetic unit 22 and also output to the arithmetic evaluation unit 12, and the above-described processing is repeated.

Here, the parameters p, q, and γ are changed every time the state is updated as described above, but the manner of the change is arbitrary. An example thereof is shown in FIG. In addition,
In FIG. 5, Y ₀ indicates the position of y = −α × min {λ _k } / 2.

FIG. 5A shows the change of q and the change of the slope of the linear portion of the saturated linear transfer function Sat when (p + q) is made constant, that is, when γ is made constant and p is made to approach 0. Shows. Further, FIG. 5B shows a change in the slope of the linear portion of the saturated linear transfer function Sat when q is changed and p is changed so as to approach 0.
Further, FIG. 5C shows a change in q when p is brought close to 0 while keeping the slope of the linear portion of the saturated linear transfer function Sat constant.

The value of q is determined by determining the minimum eigenvalue min {λ _k } of the diagonal block matrix W _kk of the synapse coupling matrix W.
Must be defined. This minimum eigenvalue is basically obtained by actually _obtaining all the eigenvalues of the diagonal block matrix W _kk of the synapse coupling matrix W and adopting the minimum value.
When the neuron elements arranged along the i-direction or the j-direction shown in FIGS. 10 and 12 are regarded as one block, the minimum eigenvalue of the coupling matrix in the k-th block is the neuron element forming the block. Let B be the number N _k
It is known that it is calculated by (1-N _k ). In addition,
B is the coefficient B used in the above equation (11).

Therefore, for example, when the six neuron elements surrounded by the frame 9 shown in FIG. 12 and arranged in the i direction are regarded as one block, N _k = 6 in this block. The minimum eigenvalue of the block is -5B
Therefore, the minimum value of q is 5B / 2.

As described above, the method of changing the parameters p, q, and γ can be set arbitrarily. However, the present inventor sets q to a constant value and changes only p by the following equation. In the case of, it was found that the three-dimensional shape can be restored well.

P = 4.0-0.08z (22) Here, z is the number of state updates, and in this case, p = 0 at the 50th state update.

Further, since α is an arbitrary coefficient, α = 1 may be set in practice. Furthermore, for the threshold value Ψ of the above equation (10) and the coefficients A, B, and C of the equation (11), A = B = Ψ
= 1.0, C = 3.5, and it has been confirmed that the initial state of all neuron elements may be given as 0.0, for example.

As described above, when the neural network is operated and the coordinate value of the neuron element that outputs 1 when the constraint condition is satisfied is extracted, it is one of the three-dimensional shapes of the target object. It was confirmed that the solution was given.

Next, the result of a simulation in which the three-dimensional shape of the target object is restored from the random dot stereogram by the above-mentioned operation and comparison with the conventional one will be described.

FIGS. 6 and 7 are views showing a first example, and are based on the object having the continuous curved surface shown in FIG. 6A.
A random dot stereogram shown in Figure 2 was created, and a simulation for restoring a three-dimensional shape from these two parallax images was performed.

At this time, N = 128, D = 25, α = 1, A
= B = Ψ = 1.0 and C = 3.5. For the block, 25 neuron elements arranged along the i direction are set as one block. That is, N _k = 25. Therefore, since B = 1.0, the minimum eigenvalue of each block is −24, and α = 1
Therefore, the minimum value of q is 12, but here it is a constant value of q = 12. From the above, the number m of blocks is m = 12
It is 8 x 128. Further, p was changed by the equation (22). The initial state of all neuron elements was 0.0.

In this case, since D = 25 for N = 128, a block having less than 25 neuron elements can be formed, but the minimum eigenvalue of the block must be larger than that of a block having 25 elements. Since it has been confirmed that this is sufficient, the number of elements in the block may be less than 25.

As a result of simulating the above-described operation of the present invention under the above conditions, the three-dimensional shape shown in FIG. 7A was restored.

On the other hand, when a simulation of three-dimensional shape restoration is performed using the Marr-Poggio model, FIG.
What was shown in was obtained. In this case, N =
128 and the initial state of all neuron elements was 0.0.

Comparing FIG. 7A and FIG. 7B, it was confirmed that according to the present invention, for an object having a continuous curved surface, the three-dimensional shape of the target object can be restored better than before.

FIG. 8 and FIG. 9 are diagrams showing a second example, in which the random dot stereogram shown in FIG. 8B is created based on the object constituted by the plane shown in FIG. 8A, and from these two parallax images. A simulation for restoring a three-dimensional shape was performed. The conditions at this time were the same as those shown in FIGS.

As a result of simulating the above-described operation of the present invention, the three-dimensional shape shown in FIG. 9A was restored.

On the other hand, when the three-dimensional shape restoration simulation is performed using the Marr-Poggio model, FIG. 9B is obtained.
What was shown in was obtained. In this case, N =
128 and the initial state of all neuron elements was 0.0.

Comparing FIG. 7A and FIG. 7B, it was confirmed that according to the present invention, the three-dimensional shape of the target object can be restored better than before even with respect to an object constituted by a plane.

[0128]

As is apparent from the above description, according to the present invention, even if the surface of the target object to be restored is composed of a flat surface or has a curved surface that continuously changes in the depth direction. However, it is possible to satisfactorily restore the three-dimensional shape from the parallax image regardless of the object shape.

Moreover, when restoring the three-dimensional shape,
Since it is not necessary to perform post-processing such as smoothing processing, the structure does not become complicated.

Furthermore, since the status update is performed block-sequentially, the processing can be performed at high speed and efficiently.

[Brief description of drawings]

FIG. 1 is a diagram showing a configuration of an embodiment of a neural network according to the present invention.

FIG. 2 is a diagram showing a configuration example of each block of a neuron element section 11 shown in FIG.

FIG. 3 is a diagram for explaining parameters p and q.

FIG. 4 is a diagram showing a connection relationship between neuron elements according to the present invention.

FIG. 5 is a diagram showing an example of how the parameters p and q change.

FIG. 6 is a diagram showing a first simulation example showing a comparison between three-dimensional shape restoration according to the present invention and conventional three-dimensional shape restoration.

FIG. 7 is a diagram showing a first simulation example showing a comparison between three-dimensional shape restoration according to the present invention and conventional three-dimensional shape restoration.

FIG. 8 is a diagram showing a second simulation example showing a comparison between the three-dimensional shape restoration according to the present invention and the conventional three-dimensional shape restoration.

FIG. 9 is a diagram showing a second simulation example showing a comparison between the three-dimensional shape restoration according to the present invention and the conventional three-dimensional shape restoration.

FIG. 10 is a diagram showing a schematic configuration of an apparatus for reconstructing a three-dimensional shape of an object from two parallax images by using a neural network.

[Fig. 11] Fig. 11 is a diagram for describing input of a parallax image to a neural network.

[Fig. 12] Fig. 12 is a diagram for describing input of a parallax image to a neural network.

FIG. 13 is a diagram for explaining restoration of a three-dimensional shape.

[Explanation of symbols]

DESCRIPTION OF SYMBOLS 1 ... Neural network, 2 ... Image input part, 3 ... Target object, 4, 6 ... Lens, 5, 7 ... Light receiving part, 11 ... Neuron element part, 12 ... Computation evaluation part, 13 ... Parameter setting part, 21 ... Total Input arithmetic unit, 22, 23 ... Arithmetic unit.

Claims (2)

[Claims] 1. A mutual connection type neural network in which neuron elements are three-dimensionally arranged, each neuron element having a predetermined synaptic connection with a neighboring neuron element, and the neuron element is divided into m blocks. The output state vector x ^(l) of the neuron element in the l-th block (where l = 1, 2, ..., M ⁾ is passed through the coupling weight matrix W _kl to the k-th block (where k = 1, 2, ..., m), and the similarity of the feature amounts of the two parallax images is further added to the kth block as the input bias vector θ ^(k) to the kth block represented by the equation (1). The total input vector u of
^(k) is obtained, and A value obtained by multiplying the total input vector u ^(k) by a parameter α that controls the influence of an external input, and an influence of the output state vector x ^(k) of the element in the k-th block before the state update. For each component of the vector obtained by adding the value obtained by multiplying the parameter γ for controlling A vector obtained by operating the saturated linear transfer function represented by is defined as the output state vector x ^(k) of the k-th block at the next time, In the process of updating the states all at once by the formula represented by, performing the state update sequentially for each block, and performing the state update, the parameter p of the saturated linear transfer function is set to 0 every time the state is updated. The parameter q is changed so as to approach the minimum eigenvalue of the diagonal block matrix W _kk of the synapse coupling matrix W.
When min {λ _k }, the neural network is characterized in that it is changed in a constant value or in a desired manner in a range of −α × min {λ _k } / 2 or more, and the sum of p and q is γ. . 2. The neural network according to claim 1, wherein excitatory synaptic connections are present between neuron elements arranged in a parallax constant direction and between neuron elements arranged in the depth direction.