JPH0719834A - Recognizing apparatus for object - Google Patents

Abstract

PURPOSE:To recognize a three-dimensional object efficiently by active sensing. CONSTITUTION:A corresponding point is estimated with a corresponding-point determining part 14 out of an inner model 10. Then, parallel movement and rotation are performed based on a parallel moving amount 16 and rotation transformation amount 20, which are initially set with a coordinate transformation part 15. Sensor information is predicted with a sensor projecting part 26. A recognizing and processing part 25 operates the error between the estimated sensor information and the observed sensor information, corrects the parallel moving amount 16 and the rotation transformation amount 20 by reverse propagation so as to reduce the error and determines the mutual coordinate transformation of the inner model 10 and an object to be observed 34. At the same time, the corresponding point is estimated by propagating the error into the corresponding-point determining part 14 and correcting the relation of the corresponding point. Furthermore, a visual sensor 30 is actively moved with an actuator 32. The above described coordinate and the determination of the corresponding point without the contradiction with respect to the information of a plurality of obtained images are performed.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION The present invention relates to a robot, an industrial device,
The present invention relates to an object recognition device used as an active sensing technology such as audiovisual recognition.

[0002]

2. Description of the Related Art For example, in sensing technology such as computer vision, a two-dimensional image of a three-dimensional object to be observed is acquired by a visual sensor such as a television camera, and the three-dimensional feature quantity Guess the dimensional structure. Next, recognition is performed by collating with the model of the observation target that has knowledge.

[0003]

However, in such conventional object recognition, as is apparent from the fact that objects having different shapes have the same two-dimensional appearance, in general, a single object is recognized. It is difficult to uniquely restore the three-dimensional structure of the observation target from the two-dimensional image. Therefore, stereo vision using multiple visual sensors and active sensing technology that activates the visual sensors are used to acquire multiple images, and the 3
The approach is to reconstruct the dimensional structure and then match it with the model to be observed.

Therefore, there is a problem in that the device configuration and processing become complicated as the number of sensors increases. Therefore, it is desired to develop an efficient three-dimensional structure reconstructing technique and a technique for collating the reconstructed three-dimensional structure of the observed object with the model. The present invention has been made in view of the above conventional problems, and is to reconstruct a three-dimensional structure of an observation target object using a plurality of images acquired by active sensing, coordinate between an internal model and the observation target object. It is an object of the present invention to provide an object recognition device that can simultaneously determine the transformation of a system and the correspondence relationship between the internal model and the feature quantity of the observation target object.

[0005]

FIG. 1 is a diagram illustrating the principle of the present invention. The object recognition device of the present invention basically includes a corresponding point determination unit 14, a coordinate conversion unit 15, a sensor projection unit 26, and a recognition processing unit 25. Corresponding point determination unit 14
Determines the correspondence between the feature amount of the internal model 10 and the feature amount of the observation target object 34. The coordinate transformation unit 15 includes a translation unit 18 and a rotation transformation unit 20, and rotates and translates the internal model 12 based on the translation amount 16 and the rotation transformation amount 20.

The sensor projection unit 26 generates sensor information from the internal model 10. Further, the recognition processing unit 25 uses the correspondence relationship of the corresponding point determination unit 14 so as to minimize the error energy obtained by the error calculation unit 28 of the sensor information of the observation target object 34 and the predicted sensor information predicted from the internal model 10. , The translation amount 16 and the rotation transformation amount 20 with respect to the coordinate conversion unit 15 are determined, and the internal model 10 and the observation target object 3 are determined.
4. Identification is performed.

Here, the recognition processing unit 25 uses the correspondence relationship and coordinate transformation of the corresponding point determination unit 14 so as to minimize the error energy between the three-dimensional structure of the observation target object 34 and the three-dimensional structure predicted from the internal model 10. By determining the parallel movement amount 16 and the viewpoint conversion amount 20 of the unit 15, the internal model 10
And the observation target object 34 is identified. Also, the corresponding point determination unit 1
4 is configured by a two-layered neural network that outputs the feature amount of the observation target object 34 from the output layer unit from the feature amount of the internal model of the observation target object 34 input to the input layer unit, and observes Observed object 3
The correspondence is estimated by modifying the weight of the neural network so as to minimize the error energy between the feature amount of 4 and the output feature amount.

This neural network is added with a constraint that the total value of the weights connected to each input / output unit becomes a certain value. Specifically, the connection weight of each input / output unit is 0 or The constraint that it becomes 1 is added. Further, in the present invention, the visual sensor 30 operable by the actuator 32 is provided as means for recognizing the observation target object 34. Therefore, the recognition processing unit 25 is characterized by operating the actuator 32 of the visual sensor 30 to identify the three-dimensional structure of the observation target object 34 reconstructed for a plurality of observation images and the internal model 10. .

[0009]

In the object recognition apparatus of the present invention, first, the corresponding point determining unit 1
4 estimates corresponding points from the internal model 10. Next, this is translated and rotated based on the parallel movement amount 16 and the rotation transformation amount 20 initialized by the coordinate conversion unit 15, and the sensor projection unit 26 predicts the sensor information. The recognition processing unit 25
The error between the predicted sensor information and the observed sensor information is calculated by the error calculation unit 28, and back propagation is performed so as to reduce this error, whereby the rotation conversion amount 16 and the parallel movement amount 2
0 is corrected, and mutual coordinate transformation between the internal model 10 and the observation target object 34 is determined.

At the same time, the corresponding point is estimated by propagating the error to the corresponding point determining unit 14 and correcting the corresponding point relationship. Further, the visual sensor 30 is actively operated by the actuator 32, and the coordinate transformation and the corresponding points are determined consistently with respect to the acquired plurality of pieces of image information, whereby the internal model 10 and the object to be observed are observed. 34 are identified.

[0011]

【Example】

<Table of contents> 1. Hierarchical sensory information processing model of the present invention 2. Intentional sensing 3. Recognition model (1) Outline of model (2) Sensor information conversion model (3) Permutation neural network 4. Object recognition (1) Reconstruction of three-dimensional structure (2) Correlation with internal model Simulation experiment (1) Reconstruction of three-dimensional structure (2) Correlation with internal model 1. Hierarchical Sensory Information Processing Model FIG. 2 shows an outline of a hierarchical sensory information processing model applied to the object recognition device of the present invention. This hierarchical sensory information processing model includes a sensor 30 that takes in information from the outside world 38 and an outside world 3
8 has a structure in which a plurality of autonomous processing units 36 are connected to each other in a hierarchical manner on an actuator 34 which acts on the actuator 8.

FIG. 3 schematically shows the processing unit 36 of FIG. The processing unit 36 is composed of three modules, a recognition module 42, a movement module 44, and a sensorimotor fusion module 46. The recognition module 42 converts the sensory information 48 from the lower layer into higher sensory information 50 having a higher degree of abstraction and transmits it to the upper layer. Exercise module 4
4 is a motion command 54 from the upper layer to a motion command 5 to the lower layer
Convert to 6 and work on the processing unit in the lower layer.

The feature of the hierarchical sensory information processing model applied to the present invention is that the processing unit 36 accepts a target (intention) 52 from a host and effectively uses it for recognition, and an organic recognition and movement mechanism. Introducing a sensorimotor fusion module 46 that couples to. The sensorimotor fusion module 46 recognizes a change in the sensory information 48 caused by the motion by prediction, and recognizes it as a prediction target 58 by the recognition module 42.
It has a function to notify. Further, the sensory movement fusion module 46, based on the sensory information 50 from the recognition module 42 and the target (intention) 52 from the upper layer, the movement module 44.
It realizes the function of generating a predicted motion target 60 for

In the conventional recognition, the basic information transmission path was only the vertical path as shown by the solid line in FIG. 3, but in the model of the present invention, information is further transmitted between recognition and movement. The internal feedback that integrates the recognition and the movement is realized by introducing the transmission path of. Therefore, each processing unit 36 can operate autonomously based on the intention 52 from the upper layer, and the plurality of processing units 36 can operate cooperatively toward the target. 2. Intentional sensing Regarding the activity in the object recognition device of the present invention, regarding an actuator that operates a sensor that captures an observation target, a sensor with an intention that goes beyond the conventional active sensing frame in which the actuator is simply used as a part of sensing. We are trying to realize information processing, that is, intentional sensing.

Unlike the conventional passive recognition processing, the intentional sensing does not catch the recognition as independent from the movement, but operates the recognition and the movement in parallel and cooperatively.
It is intended to improve the recognition accuracy. Now, assuming that a visual recognition target, that is, a sensing intention is given, a sensing behavior that achieves this intention is realized. For example, when an intention to measure a certain object is given, a search action for the target object or an action for grasping the whole image of the target object is performed.

In this way, by instructing the object or action to be observed as an intention, an appropriate sensing action utilizing knowledge about the object can be taken, and improvement in recognition accuracy and recognition speed can be expected. FIG. 4 shows an embodiment of the actuator and sensor used for the intentional sensing of the present invention. A multi-degree-of-freedom manipulator is used as the actuator 32, and a CCD camera or the like is installed as a visual sensor at the tip of the manipulator. By using the actuator 32 equipped with such a visual sensor 30, it is possible to realize various sensing actions such as a search action, an overall image grasp, and a gaze at an ambiguous point with respect to the observation target object 34, for example, a cup. .

The sensorimotor fusion module 46 used in the processing unit 36 shown in FIG. 3 applied to the present invention predicts the sensor information which will be obtained at the next time, from the sensor information and the motion command at a certain time. It is a sensorimotor fusion model that organically connects recognition and movement by using a hierarchical neural network of a forward model. This model uses the error backpropagation method (backpropagation method)
A function of generating a motion command that brings the sensor information closer to an intended target value and a function of notifying the predicted sensor information to the recognition module 42 are realized.

Further, as the recognition module 42 in FIG. 3, a recognition model having a structure in which the intent 52 can be used for sensing using a hierarchical neural network is used.
By using this model, the intent from the upper layer 52
The sensor information predicted by the sensory-motor fusion model 46, that is, the prediction target 58 can be effectively used for sensing. Thus, the sensory fusion movement module 46 and the recognition module 4
By using a neural network for 2, it is expected to obtain a model by learning a complicated sensor processing process and construct a model having adaptive ability.

In the following description of the embodiments, there are two processes of recognizing a three-dimensional object according to the present invention, determining the correspondence between the object to be observed and the internal model, and projecting the internal model to the sensor information. By modeling with a processing element, a recognition model, that is, an object recognition device that can handle changes in the internal model of the recognition target object and addition and deletion of sensors is realized. 3. Recognition model (1) Outline of model First, the basic idea of the recognition model used for object recognition of the present invention will be described. As shown in FIG. 5, the observation target object 3
4 and the internal model 10 are defined by feature points on different coordinate systems. Where X is the object center coordinate system
The object and the internal model coordinate system are called Xmodel.

Further, the sensor information is expressed on the two-dimensional sensor coordinate system U having the origin Uo as a point on which the origin O of the object center coordinate system is projected. On the basis of the sensor information 62 on the sensor coordinate system U obtained from the observation target object 34, the correspondence relationship (estimation of the correspondence relationship) between the observation target object 34 and all the feature points of the internal model 10 and the object center coordinates. Coordinate conversion to make the system and the internal model coordinate system match, that is, the parallel displacement X
o, the object to be observed 3 is estimated by estimating the rotation conversion amount θ.
Identify and recognize 4

FIG. 6 is a block diagram of an embodiment for realizing the recognition model. First, the replacement neural network 14 functioning as a corresponding point determination unit is provided with the observation target object 34 and the internal model 1.
Estimate the correspondence between 0 feature points. Further, the energy minimization unit 12 uses the conversion model 64 of the sensor information for converting the feature points defined in the internal model coordinate system to the sensor coordinate system as a basic element, and estimates the corresponding points and coordinates using the energy minimization method. Make the conversion decision and recognize the object to be observed.

FIG. 7 shows the details of the sensor information conversion model 64 provided in the energy minimization unit 12 of FIG. The conversion model 64 includes a parallel moving unit 18, a rotation converting unit 22, and a sensor projecting unit 26. Although FIG. 7 shows one system of the conversion model 64, in FIG.
Three systems are shown because it is processing of one corresponding point. When the observation target object predicted as the intent 52 is given from the upper layer, the processing according to the embodiment of the recognition model of FIG. 6 detects the feature points of the internal model 10 of the observation target object 34 intended from one certain viewpoint. Projecting onto a coordinate system to generate sensor prediction information. This processing is the processing of the parallel moving unit 18, the rotation converting unit 22, and the sensor projection unit 26 of FIG. 7, and the predicted sensor information is given to the error calculating unit 28.

Next, an error between the predicted sensor information and the sensor information of the object 34 to be observed actually observed by the visual sensor 30 is calculated by the error calculator 28, and this error is back propagated to minimize the error energy. Then, the correspondence between the observation target object 34 and the internal model 10 is estimated, and the coordinate conversion between the object center coordinate system and the internal model coordinate system, that is, the translation amount 16 and the rotation conversion unit 20 are updated.

Further, the visual sensor 30 by the actuator 32 is coded to continue the processing by the recognition model in accordance with the sensing action of changing the viewpoint position of the observation target object 34, so that the correspondence relationship between the observation target object 34 and the internal model 10 is established. The coordinate conversion between the coordinate systems, that is, the parallel movement amount 16 and the rotation conversion amount 20 are determined. (2) Sensor Information Conversion Model FIG. 8 shows the function of the sensor information conversion model 64 provided in the energy minimization unit 12 of FIG. This conversion model 64 rotates an arbitrary point X in the internal model coordinate system Xmodel by a rotation angle θ.
The sensor information from the internal model 10 of the recognition target object 34 is predicted by projecting X ′ that has been rotated only by the central projection transformation onto the sensor coordinate system.

Now, assuming that the point in the sensor coordinate system corresponding to X '= (x', y ', z') is U = (u, v) and the distance from the sensor coordinate plane to X'is d, the center Projection conversion unit 26
The central projection transformation by

[0026]

[Equation 1]

Can be expressed as However,

[0028]

[Equation 2]

[0029] Here, the translation amount of the coordinate system is X
If the rotation angle is 0 and the rotation angle (rotation conversion amount) is θ, the conversion between coordinate systems is

[0030]

[Equation 3]

It is represented by When the rotation angle θ of the coordinate system is described by Euler angles (α, β, γ),

[0032]

[Equation 4]

It is Where each rotation transformation matrix is

[0034]

[Equation 5]

It is Therefore, the conversion from the internal model 10 to the sensor information is expressed by the following equation.

[0036]

[Equation 6]

FIG. 9 shows an embodiment of the replacement neural network 14 used as the corresponding point determining unit of the present invention. The replacement neural network 14 has an internal model 1 in which the feature points of the observation target object 34 associated with the output unit 74 of the output layer 72 and the input unit 68 of the input layer 66 are associated with each other.
It is a two-layered network that determines the correspondence relationship of 0 feature points.

Therefore, the correspondence relation is set by modifying the network connection weight Wij so as to reduce the error between the coordinate value of the predicted feature point of the observation target object 34 and the coordinate value of the observed feature point. To do. The number of the input units 68 and the output units 74, that is, the number of feature points of the observation object 34 and the internal model 10 is n, the input value of the i-th input unit 68, that is, the feature point coordinates IX of the internal model 10.
(I), the output value of the output unit 74, that is, the observation target object 3
The predicted value of the feature point coordinates of 4 is OX (i), and the observation target object 3
Let 4 (X) be the actually observed feature points.

If the weight of the i-th input unit 68 and the j-th output unit 74 is Wij and a linear neuron is used as the output unit 74, the output value of the j-th output unit 74 is

[0040]

[Equation 7]

It becomes Here, a squared error between the observed feature point X (i) of the observation target object 34 and the predicted feature point OX (i) is defined as energy E, and

[0042]

[Equation 8]

The updating formula of the weight of the network is defined as the following formula.

[0044]

[Equation 9]

Here, εα _in , α _out and β are positive constants. In the first term that minimizes the energy E shown in equation (8) on the right side of equation (9), the learned network is a replacement between the feature points of the internal model 10 and the feature points of the observation target object 34. A weight constraint term is added so that The second and third terms of the equation (9) realize a constraint term having the total value of the weights coupled to the input unit 68 and the output unit 74 as 1 shown in the following equations (10) and (11). The weight limiting units 70 and 76 for

[0046]

[Equation 10]

Furthermore, the fourth term on the right side of the equation (9) is (1
This is a constraint term that sets the weight shown in the expression (2) to 0 or 1. 3. Object recognition The processing of the recognition model used in the present invention described above is reconstructed in the object center coordinate system of the feature points of the observation object, and the correspondence between the reconstructed observation object and the internal model is determined. It divides into processing and recognizes a three-dimensional object. (1) Reconstruction of three-dimensional structure First, a process of reconstructing an object to be observed by the object center coordinate system Xobject will be described. Now, as shown in FIG. 10A, the object center coordinate system Xobj is set so that the direction of the visual sensor 30 is the z axis, and the x axis and the y axis are on the plane perpendicular to the z axis.
Define ect.

In this state, the visual sensor 30 is attached to the manipulator as the actuator 32 as shown in FIG. 4, and as shown in FIG. 10 (b), at a constant distance d with the origin O of the object center coordinate system as the center. Perform a rotational movement. Then, the rotational movement of the visual sensor 30 at a certain characteristic point of the object to be observed, that is, the coordinate in the sensor coordinate system U of the characteristic point at the rotational position indicated by 30-1, 30-2, 30-3 is secured as a plurality of sensor information. To do.

FIG. 11 shows an embodiment of a reconstruction model of three-dimensional information. Here, assuming that the number of feature points is n and the number of rotational movements, that is, the number of observations is m, it is obtained by m observations (m ×
(n) The feature point coordinates in the sensor coordinate system U are (m ×
n) Corresponding conversion models 64 of sensor information. The coordinates of the feature points of the observation target object 340 to be reconstructed in the object center coordinate system Xobject are X (i) = [x (i), y (i), z (i)] and correspond to the same feature point. It is shared by m sensor information processing models. Also, a conversion model 6 for all sensor information
The parallel movement X0 of 4 is 0 in this case. The rotation angle at time t is represented by θ (t), and at time t = 0, θ (0) =
If 0, the rotation angle θ (0), ..., θ (m−1) of the visual sensor 30 with respect to the object center coordinate system Xobject.
Is a known value from the motion command for the manipulator. Distance d to the origin of the object center coordinate system Xobject
Is measured and set by means such as a distance sensor or stereo vision.

Here, i is a feature point, and t is the number of rotations,

[0051]

[Equation 11]

Is the coordinate of the feature point in the two-dimensional sensor coordinate system U predicted from the coordinate X (i) of the feature point of the observation object 34 by the sensor information conversion model 64 at time t,

[0053]

[Equation 12]

Predict as the coordinates of the feature points observed by the visual sensor 30. The energy function E in the energy minimization unit 12 is a squared error between the predicted sensor information and the observed sensor information.

[0055]

[Equation 13]

Then, the three-dimensional structure of the observation object 34 is reconstructed by updating the feature point coordinates X (i) in the object center coordinate system Xobject of the observation object 340 to be reconstructed according to the following equation. It

[0057]

[Equation 14]

Here, ε _X is a positive constant. That is, for example, X (i) = 0 is given as an initial value, and X (i) is set so as to minimize the energy E of the equation (15) according to the equation (16).
By sequentially updating (i), it is possible to determine X = [X (0), ... X (n-1)], which outputs sensor information that approaches E = 0. By thus minimizing the error energy of the sensor information acquired over the entire sensing behavior, it is possible to reconstruct the three-dimensional structure of the observation target object 34 with less contradiction.

Considering the number of sensing actions required for reconstruction, n feature points X (i) = [x (i), y (i), z (i)] where i = 0 , N−1 is 3n, and the equation obtained by one observation is 2n, so at least 2 is required for reconstruction of the three-dimensional structure.
Number of sensing times that satisfies mn> 3n (n ≧ 2)
is necessary. (2) Correlation with Internal Model Next, identification and recognition by associating the observation target object with the internal model using the replacement neural network 14 shown in FIG. 9 will be described. That is, using the replacement neural network 14 shown in FIG. 9, the correspondence between the observation target object 34 and the feature points of the internal model 10 and the object center coordinate system Xobje.
The reconstructed object to be observed is identified and recognized as the internal model by determining the coordinate transformation that matches ct and the internal model coordinate system Xmodel, that is, the translation amount and the rotation transformation amount.

FIG. 12 shows the correspondence with the internal model.
That is, the internal model 1 in the internal model coordinate system Xmodel
0 feature point coordinates of and replaced by a substituting neural network 14, translating the rotational transformation f _r of equation (4) X0
And the rotation θ to convert the object center coordinate system Xobject
The three-dimensional structure of the observation target object 340 in is predicted and given to the error calculator 28. The predicted feature points are X _p (i),
The observed feature points are represented by X (i), and the energy function is expressed by the square error of these.

[0061]

[Equation 15]

It is defined by In order to minimize this energy E, the parallel movement amount X0 and the rotation angle θ are corrected based on the following equations, and the rotation conversion and the parallel movement for matching the coordinates of the feature points of the observation target object 34 and the internal model 10 are determined. is doing.

[0063]

[Equation 16]

At the same time, the replacement neural network 14
Of energy with respect to the output value OX (i) of

[0065]

[Equation 17]

Then, the replacement neural network 14 is obtained.
And the weighting coefficient is corrected to determine the correspondence between the observation target object 34 and the feature points of the internal model 10. 4. Simulation Experiment When the effectiveness of the recognition model for realizing the object recognition device of the present invention described above was evaluated by computer simulation, the following results were obtained. (1) Reconstruction of three-dimensional structure A simulation experiment for reconstructing a three-dimensional structure using the conversion model 64 of sensor information is performed. As previously mentioned,
The Euler angle is used for the rotation conversion, and the center projection conversion is used for the projection from the observation target object to the sensor information. FIG. 13 shows the sensor information generated for the simulation experiment. That is, five feature points in the object center coordinate system P0 = (0.5,0.5,0.5) P1 = (-0.5,0.3, -0.2) P2 = (-0 .4, -0,5,0.0) P3 = (0.4, -0.3,0.0) P4 = (0.2,0.9, -0.2) Distance d = 2, Δα = 30 °, Δβ = 30 °, Δγ = −30 ° around the origin of the object center coordinate system
The sensor information was generated by rotating the visual sensor three times at each step. The conversion model 64 of the sensor information is associated with the feature point coordinates in these 15 sensor coordinate systems, and the three-dimensional structure of the observation target object is reconstructed by energy minimization.

FIG. 14 shows the error on the ordinate and the number of updates by energy minimization on the abscissa, and the error and average error of each coordinate value of X, Y, Z at the reconstructed object center coordinates have the illustrated characteristics. became. As is clear from FIG. 14, it is understood that the error is within 0.01 within 300 updates and the three-dimensional structure is reconstructed with high accuracy. (2) Correlation with internal model As shown in FIG. 12, the rotation conversion of the sensor information conversion model 64 and the replacement neural network 14 are combined,
The correspondence between the reconstructed observation target object 340 and the feature points of the internal model 10, the parallel movement amount X0 and the rotation angle θ of the object center coordinate system Xobject and the model coordinate system Xmodel.
To decide.

Here, in the internal model coordinate system (± 0.
5, ± 0.5, ± 0.5) for the 8 points of the rectangular parallelepiped,
Random numbers were uniformly added in the range of ± 0.25 to generate 10 internal models. Then, for each internal model, X0 =
Move in parallel at (0.1, 0.1, 0.1) and rotate by changing α and β in 5 ° increments in the range of 0 <α <90 ° 0 <β <90 ° γ = α The object was an object to be observed in the object center coordinate system.

Further, the correspondence between feature points and the parallel movement amount X
0 and the rotation angle θ are unknown, the correspondence between the observation target object and the feature points of the internal model, the parallel movement amount X0, and the rotation angle θ
A simulation experiment to determine Moreover, the initial value of the weight of the replacement neural network is determined by the number of stages. FIG. 15 shows the experimental result of the association with the internal model, and FIG. 16 shows the success rate of identification averaged with respect to α. It should be noted that the case where the correct correspondence, the parallel movement amount X0, and the rotation angle θ are determined by updating within 100 times is regarded as successful. As is clear from FIG. 16, when the rotation angle α between the object center coordinate system and the internal model coordinate system is within 40 °, it is confirmed that the correspondence relationship and the parallel movement amount X0 and the rotation angle θ can be almost certainly determined. It was

Of course, the present invention is not restricted by the concrete numerical values shown in the above-mentioned embodiments.

[0071]

As described above, according to the present invention, the three-dimensional structure of an observation target object is reconstructed using a plurality of images acquired by active sensing, and the coordinate system between the internal model and the observation target object is reconstructed. It is possible to realize efficient recognition of a three-dimensional object by simultaneously determining the conversion of the above, and further determining the correspondence relationship between the internal model and the feature amount of the observation target object.

Furthermore, the sensor is actively operated by the actuator to perform consistent coordinate conversion and determination of corresponding points on the observation object using a plurality of acquired images, whereby the internal model and the observation object are determined. It is possible to sufficiently enhance the accuracy of identifying the object.

[Brief description of drawings]

FIG. 1 is an explanatory view of the principle of the present invention.

FIG. 2 is an explanatory diagram of a hierarchical sensory information model realized by the present invention.

FIG. 3 is an explanatory diagram showing the configuration of the hierarchical sensory information model unit of FIG.

FIG. 4 is a structural diagram of an embodiment of a visual sensor and an actuator used in the present invention.

FIG. 5 is an explanatory diagram showing the basic idea of the recognition model in the present invention.

FIG. 6 is a configuration diagram of an embodiment of the device configuration of the present invention.

7 is a configuration diagram of an embodiment showing details of an energy minimization unit in FIG.

FIG. 8 is a diagram showing an embodiment of a sensor information conversion model of the present invention.

FIG. 9 is a configuration diagram of an embodiment of a replacement neural network of the present invention.

FIG. 10 is an explanatory diagram showing reconstruction of a three-dimensional object in the present invention.

FIG. 11 is a configuration diagram of an embodiment showing a reconstruction model of a three-dimensional object in the present invention.

FIG. 12 is an explanatory diagram showing correspondence with an internal model according to the present invention.

FIG. 13 is an explanatory diagram of sensor information used in a simulation of a three-dimensional structure obtained by rotating a visual sensor.

FIG. 14 is a characteristic diagram showing a simulation result of the number of updates and an error of the replacement neural network.

FIG. 15 is a three-dimensional graph diagram showing rotation angles α and β and a success rate by a simulation of association with an internal model.

FIG. 16 is a graph showing the relationship between the rotation angle α and the average success rate in FIG.

[Explanation of symbols]

 10: Internal model 12: Energy minimization unit 14: Corresponding point determination unit (replacement neural network) 15: Coordinate conversion unit 16: Parallel movement amount (X) 18: Parallel movement unit 20: Rotation conversion amount (θ) 22: Rotation Conversion unit 25: Recognition processing unit 26: Sensor projection unit (center projection conversion unit) 28: Error calculation unit 30: Visual sensor 32: Actuator 34: Observed object 36: Processing unit 38: Outside world 40: Object recognition module 42: Recognition Module 44: Motion module 46: Sensorimotor fusion module 48, 50: Sensory information 52: Intention (target) 54, 56: Motor command 58, 60: Prediction target 64: Conversion model 66: Input layer 68: Input unit 70, 75 : Weight limiting unit 72: output layer 74: output unit

 ─────────────────────────────────────────────────── ─── Continuation of the front page (72) Inventor Shigemi Nagata 1015 Kamiodanaka, Nakahara-ku, Kawasaki-shi, Kanagawa Fujitsu Limited

Claims (6)

[Claims] 1. A corresponding point determining unit (14) for determining a correspondence relationship between a feature amount of an internal model (10) and a feature amount of an observation target object (34), a translation amount (16) and a rotation conversion amount (20). ), A coordinate transformation unit (15) that rotates and translates the internal model (10), a sensor projection unit (26) that generates sensor information from the internal model (10), and a sensor of the observation target object (34). Information and internal model (1
0) error calculation unit of predicted sensor information (28)
By determining the correspondence relationship of the corresponding point determination unit (14), the parallel movement amount (16) and the rotation conversion amount (20) with respect to the coordinate conversion unit (15) so as to minimize the error energy obtained in An object recognition device comprising an internal model (10) and a recognition processing unit (25) for identifying an observation target object (34). 2. The object recognition apparatus according to claim 1, wherein the recognition processing section (25) has an error energy of a three-dimensional structure predicted from the three-dimensional structure of the observation target object (34) and the internal model (10). Corresponding point determination unit (1
4) to identify the internal model (10) and the observation target object (34) by determining the correspondence relationship (4), the parallel movement amount (16) and the rotation conversion amount (20) of the coordinate transformation unit (15). Object recognition device characterized by. 3. The object recognition apparatus according to claim 1, wherein the corresponding point determination unit (14) determines the observation target object from the feature amount of the internal model of the observation target object (34) input to the input layer unit. The feature amount of (34) is configured by a two-layered neural network that outputs from the output layer unit, and the error energy between the observed feature amount of the observation target object (34) and the output feature amount is minimized. An object recognizing device which estimates a correspondence by correcting a weight of the neural network. 4. The object recognition apparatus according to claim 3, wherein the neural network is added with a constraint that a total value of weights connected to each input / output unit is a constant value. Recognition device. 5. The object recognizing device according to claim 3, wherein the neural network is added with a constraint that the weight of the coupling of each input / output unit becomes 0 or 1. 6. The object recognizing device according to claim 2, wherein a visual sensor (30) operable by an actuator (32) is provided as means for recognizing an object to be observed (34),
The recognition processing unit (25) identifies the three-dimensional structure of the observation target object (34) reconstructed by operating the actuator (32) of the visual sensor (30) and the internal model (10). Object recognition device characterized by.