JP3281946B2 - 3D object recognition method - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a three-dimensional object recognition method for recognizing a three-dimensional shape.

[0002]

2. Description of the Related Art There is a method for recognizing a three-dimensional object using a two-dimensionally arranged ultrasonic sensor and a neural network (JP-A-5-19052). According to this method, even in an outdoor environment where brightness changes or a shadow occurs, it is possible to recognize a three-dimensional object located in a free direction in front, rear, left, right, up and down from an observation position. In this method, the distance distribution to the object surface and the background is obtained using an ultrasonic imaging method, the object shape is extracted from the background serving as a reference plane based on the distance information, and the depth information of those objects is transmitted by the error back propagation method. Is input to a layered neural network using, and a three-dimensional object is recognized.

FIG. 6 shows one of the configurations of the shape recognition apparatus described above.
It is a block diagram showing an example. In the figure, 10 is an object to be recognized, 11 is a sensor array as a range sensor, 1
1 _1, 11 _2, ~, 11 _n ultrasonic sensors disposed in the sensor array 11, the perceptron type neural network 13, 14 the input layer of the neural network 13, 15 is an intermediate layer of the neural network 13, 1
6 is an output layer of the neural network 13, and 19 is measurement data obtained from the sensor array 11. A is distance information from the sensor array 11 to the object 10, and b is distance information from the sensor array 11 to the background.

In this method, since a neural network is used, there is an advantage that an object can be recognized without using a complicated procedure for extracting feature values of different shapes for various objects. Incidentally, the measurement data 19 is observed differently depending on the relative position of the sensor array 11 and the object 10 and the rotation state of the object 10 about the normal line of the surface of the sensor array 11 as an axis.

Therefore, in order to recognize the object regardless of the position or the rotation state, the object is learned by moving in parallel so that the position of the center of gravity of the object is at the center of the array of the measurement data or by rotating the center of gravity. A rotation contrast method is used to determine whether or not the shape matches a certain shape. Thus, 3
In order to recognize a three-dimensional object, the shape is measured, and then the shape is recognized based on the measured information, whereby the object is recognized. Hereinafter, this recognition will be described in detail.

[Measurement of Shape] The distance to the surface of the object 10 can be directly measured by irradiating the surface of the object 10 with ultrasonic waves and measuring the time required for reflection of the ultrasonic wave from the surface of the object 10. . Therefore, by arranging the ultrasonic sensors two-dimensionally as shown in FIG. 6, distance information to the surface of the object 10 facing each sensor can be obtained. Also, depth information of the object can be obtained from the difference between the distance to the reference plane on which the object 10 is placed and the distance to the surface of the object 10. Then, the object 1
A three-dimensional image of 0 is obtained. These measurements are called a pulse delay time method, and a distance resolution about the wavelength of the ultrasonic wave to be used is obtained in the depth direction of the recognition target, and precise measurement can be performed.

In this method, even if the directivity angle of the ultrasonic sensor used is ± 5 degrees or less, the lateral resolution perpendicular to the above-mentioned normal line is limited to about the sensor diameter. An aperture synthesis method may be used to compensate for this disadvantage and to use the sensor array 11 by ultrasonic waves as shown in FIG. When this aperture synthesis method is used, a plane image corresponding to the focal length can be obtained.
Further, in this case, the lateral resolution depends on the focal length, but when the distance is substantially equal to the aperture length, a resolution of about the wavelength can be obtained. Note that, in addition to the above-described range sensor using ultrasonic waves, a range sensor based on a light beam slit method using a laser light source may be used.

Here, although not a range sensor, a method using a video camera is also practical in terms of improving only the resolution in the horizontal direction. This method also has the advantage that color information can be used. The shape information obtained by the video camera has a very high resolution in the horizontal direction. However, distance information cannot be obtained with a video camera alone. Moreover, since a video camera extracts shape information based on a difference in brightness, it is difficult to obtain accurate data in an environment where brightness changes or a shadow occurs. However, if the method is used together with the method based on the ultrasonic method, it is possible to extract a shape that is difficult with a camera image alone.

The mechanism used in combination is described in Japanese Patent Application No. 4-351457. If a video image of an object can be extracted, it is possible to obtain shape information with much better lateral resolution than an ultrasonic image. By using this information and the above-described distance information having excellent depth direction resolution, a highly accurate three-dimensional object image can be obtained. According to the various methods described above, based on the obtained three-dimensional object image, shape information including size information such as a cross-sectional area, a height, a volume, and an outer shape of the recognition target object can be obtained.

[Shape Recognition] Next, a method of recognizing a recognition target based on the obtained shape information of the recognition target will be described. As a method of recognition, basically,
A layered neural network using the same backpropagation method as described in JP-A-19052 may be used, that is, a perceptron-type neural network. This corresponds to the neural network 13 shown in FIG.
It has the same structure as that of the first embodiment, and is composed of an input layer 14, an intermediate layer 15, and an output layer 16. The intermediate layer 15 may have two or more layers.

In the neural network 13, a signal input to the input layer 14 can be learned so that the output layer 16 can output a constant signal. Once learning is performed, the neural network 13 outputs a learned signal to an input signal substantially similar to the learned input signal. Therefore, it is possible to obtain a unique recognition result corresponding to the shape, even though a complicated procedure for extracting the feature amount from the shape information is unnecessary.

Then, the neural network 13
Then, the recognition is performed in correspondence with the learning data. Input layer 1
As shown in FIG. 6, measurement data 19 indicating a planar distribution of distance information measured by the sensor array 11 is input to each synapse 4. Therefore, the same object cannot be recognized as the same object when the observation position, the orientation, the distance, and the like are different because the data becomes different.

To compensate for this drawback, the position of the stage to be recognized is normalized by translating the measurement data so that the center of gravity of the measurement data matches the center of gravity of the learning data. For example, the object to be recognized is recognized by using the rotation contrast method described above. With these methods, the object can be recognized regardless of the distance, position, and orientation of the object. In addition, at the learning stage, the normalized size is given to the neural network from the obtained shape information.

By this normalization, the difference between all the learning data given to the neural network represents only the difference between the shapes. Then, in the actual recognition stage, in the same manner as described above, the height information of the measurement data of the object to be recognized is normalized, the cross-sectional area is further enlarged and reduced, and input to the neural network after matching with the standard value. . Thus, the shape of the object can be recognized irrespective of the distance to the object to be recognized and its position, and the orientation and size of the object.

[0015]

Conventionally, as described above, objects having the same shape but different sizes are recognized as having the same shape but different shapes. There was a problem that it could not be recognized. In the case of recognizing objects of the same shape and different sizes by the conventional method, it is necessary to learn all the objects having different sizes. As the number of types increases, the learning time increases exponentially.

For example, even if two objects have similar shapes, if they are different in size, learning is required as different objects. For this reason, for example, it is necessary to learn spheres of various sizes just to recognize spheres. In other words, unless a huge amount of learning is performed, it is necessary to recognize spheres of various sizes. Can not. That is, there is a large practical problem that learning becomes difficult even when a high-speed and large-capacity computer is used.

The present invention has been made to solve the above problems, and has as its object to enable practical recognition of various three-dimensional objects having different shapes and sizes.

[0018]

According to a three-dimensional object recognition method of the present invention, an object placed on a reference plane is observed from one side to obtain shape information of the object, and the shape information is obtained in a depth direction and a plane direction. Recognizes the shape of the object using the normalized shape information normalized to the shape of the object. Based on the shape recognition result and the size information such as the cross-sectional area, volume, or diameter of the object based on the shape information, the object is recognized based on two or more components. It is characterized by recognizing.

[0019]

By learning one representative shape, objects that are similar but different in size are recognized as being different.

[0020]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS One embodiment of the present invention will be described below with reference to the drawings. FIG. 1 is a flowchart for explaining one embodiment of the method for recognizing a three-dimensional object according to the present invention. This embodiment will be described with reference to the flowchart of FIG. In the recognition of the three-dimensional object, first, learning in a neural network is performed.

In this learning, first, an object to be learned (recognition target) is measured. In the measurement of this object, the ultrasonic sensor array in which a plurality of ultrasonic sensors are arranged,
As in the conventional case, distance information of the object to be learned from the ultrasonic sensor array is obtained (step S1). Then, noise is removed from the obtained distance information (step S2), and the shape information in the depth direction of the object is extracted therefrom (step S3).

On the other hand, image data of the object to be learned is captured by a video camera or the like, and information on the shape of the object in the horizontal direction is extracted (step S4). Then, the center of gravity of the object is obtained from the extracted shape information (step S).
5). As described above, steps S1 to S5 are measurement. Then, as described below, a size and position standardization process is performed.

First, at the learning stage, the data obtained by normalizing the size and position as learning data from the obtained shape information is given to the shape recognition neural network (step S6). For example, in the obtained shape information, the height information of each part of the object is divided by the maximum height information therein to normalize, and the center of the data sequence in the shape information from which the center of gravity of the object is obtained is obtained. , And provide this to the shape recognition neural network.

Next, the information amount of the learning data is normalized and given to the shape recognition neural network (step S).
7). For example, in the obtained shape information, the cross-sectional area is obtained by binarizing the object portion into 1 and the background portion serving as the reference plane into 0 and calculating the sum. Then, a process of enlarging or reducing the measurement data so as to match a predetermined standard value of the cross-sectional area is performed so as to match the standard value. That is, in this case, the cross-sectional area of the plane image of the object is calculated, and the shape information is scaled and normalized so that this area matches the standard value, and this is provided to the shape recognition neural network.

As a result of this normalization, the difference between all the learning data given to the shape recognition neural network represents only the shape difference. Then, in the actual recognition stage, in the same manner as described above, the height information of the measurement data of the object to be recognized is normalized, the cross-sectional area is further scaled up and adjusted to the standard value, and the shape recognition neural network is used. input. The rotation control method is used for recognition. As described above, steps S6 and S7 are portions for performing normalization.

Through the above processing, the object can be recognized by the shape recognition neural network regardless of the distance to the object to be recognized and its position, and the orientation and size of the object (step S8). In the above-described neural network, it goes without saying that the same applies even if a neural network having a pattern recognition function is used similarly to the perceptron.

The feature of the present invention is that by using the shape information and the size information of the object recognized as described above,
The point is to recognize differences including the size of the object (object recognition). Object size information includes cross-sectional area, volume,
Height, diameter, etc. FIG. 2 shows a representative example of an object having the same shape information. In the figure, (a) is a normalized shape obtained by learning, (b) is an object having the same height as the normalized shape but different in cross-sectional area and volume, (c)
Is an object having the same cross-sectional area as the normalized shape but different in height and volume, and (d) is an object having the same volume as the normalized shape but different in height and cross-sectional area.

The objects shown in FIGS. 2 (b), (c) and (d) are the same as the normalized shape and the same sectional area, height or volume, but are all different objects. Conventionally, the shapes of these objects could be recognized. For example, these objects are not recognized as cylinders, but are recognized as having the same shape as the normalized shape. However, object recognition that these objects are different objects could not be performed.

In order to recognize an object, at least two pieces of information on the size of the object, such as the cross-sectional area and height, or the cross-sectional area and volume, or the volume and height, are added to the normalized shape information. is necessary. That is, in the present invention, in addition to the above-described normalization, by giving at least two pieces of information to the object recognition neural network (step S
9) The feature is that the object recognition neural network recognizes the object (step S10).

Hereinafter, the results of the evaluation test performed based on the above-described method will be described. FIG. 3 is a configuration diagram showing the configuration of the shape recognition device according to one embodiment of the present invention. In the figure, reference numeral 31 denotes a transmission / reception ultrasonic sensor (MA200A1 manufactured by Murata Manufacturing Co., Ltd.) having a frequency of 200 kHz.
A total of 64 ultrasonic sensor arrays, 8 in length and width on a flat plate at mm intervals, 32 is a video camera composed of a CCD image sensor for photographing a recognition target from a hole 31a in the center of the ultrasonic sensor array 31, and 33 is Object to be recognized,
Reference numeral 34 denotes a measurement surface serving as a reference surface on which the recognition target is placed.

Reference numeral 35 denotes a normalizing unit for normalizing shape information obtained from the ultrasonic sensor array 31 and the video camera 32, and 36 denotes an input layer 36a, an intermediate layer 36b, and an output layer 3.
6c, which is a shape recognition neural network that performs shape recognition based on the shape information normalized by the normalization unit 35. The ultrasonic sensor and the video camera 32 are fixed to the sensor array 31 so that the relative positions do not change. The input synapse number of the input layer 36 of the neural network 36 of the shape recognition unit is 2704, and the standard value of the cross-sectional area of the learning model in the learning stage is 800.

Here, as described above, even when an ultrasonic sensor having a high directivity of 200 KHz is used, about 10 mm, which is equal to the outer shape of the sensor, is the limit of the lateral decomposition of the ultrasonic sensor alone. All ultrasonic sensors are used for both transmission and reception. Not only does the oscillating sensor receive reflected waves,
By receiving the signal oscillated by the next sensor, 8
Measurement of 15 × 15 points is possible with × 8 sensors. The image obtained from the video camera 32 depends on the characteristics of the CCD image sensor constituting the image.
For example, the obtained pixels are composed of 512 rows and 485 pixels, and one pixel has brightness information of 8 bits, that is, 256 gradations.

The operation of the shape recognition device in this evaluation test will be described below. First, from the grayscale image data measured by photographing the object 33 with the video camera 32,
A three-dimensional object image is extracted based on a difference in brightness. Next, the cross-sectional area is calculated based on the measured number of pixels of the three-dimensional object image, the extraction width of the image is adjusted so that the cross-sectional area matches the standard value, and the plane image is extracted again to obtain the cross-sectional area. The size (information amount) of the shape information to be obtained is normalized. Further, the size of the image based on the shape information obtained from the object 33 is normalized by the maximum height of the object image obtained by the ultrasonic sensor. Then, the normalized information is input to the shape recognition neural network 36 to perform shape recognition.

Using this shape recognition device, as shown in FIG. 3, samples shown below are placed one by one at arbitrary positions in the measurement area (about 21 cm ² ) of the ultrasonic sensor array 31 in arbitrary directions. We installed and tried shape recognition. Experiment 2
I went each time. As a sample, the cross-sectional area is 20cm
² , 50 cm ² , 70 cm ² , three cylindrical,
A total of 12 objects were used, including a rectangular parallelepiped A having a square cross section, a rectangular parallelepiped B having no square cross section, and a triangular prism. The height of all objects was 4 cm. In this evaluation test, the shape recognition neural network 36 learns four types of objects, ie, a cylinder, a rectangular parallelepiped A, a rectangular parallelepiped B, and a triangular prism, as described above.

FIG. 4 is an example of the output of the shape recognition neural network 36 when the rotation contrast method is used.
(A) is a cross-sectional area of 20 cm ² , (b) is a cross-sectional area of 50 cm
² and (c) are waveform diagrams showing the output of the output layer 36c (FIG. 3) obtained as a result of observing the rectangular parallelepiped B having a cross-sectional area of 70 cm ² . In the figure, reference numeral 41 denotes a rectangular parallelepiped A of the output layer 36c.
Is a synapse output indicating a cylinder of the output layer 36c, 43 is a synapse output indicating the rectangular parallelepiped B of the output layer 36c, and 44 is a synapse output indicating a triangular prism of the output layer 36c.

The vertical axis indicates the synapse output, and the horizontal axis indicates the rotation angle of the input measurement data around the center of gravity. Here, the synapse output 41 indicating the rectangular parallelepiped A becomes the output 1 (maximum) when the recognition target completely matches the learned rectangular parallelepiped A, and the other synapse outputs 42 to 44 become 0. Similarly, when the recognition target completely matches the learned cylinder, the synapse output 42 indicating the cylinder becomes output 1 (maximum), and the other synapse outputs 43 and 44 including the synapse output 41 indicating the rectangular parallelepiped A become 0. Become. In other words, the synapse output 41 indicating the rectangular parallelepiped A is such that a rectangular parallelepiped having a square cross section is recognized at the learning stage so that the output becomes 1 (learned). At this time, naturally, the other synapse outputs 42 to 44 are set to 0.

As shown in FIG. 4, the recognition result has a peak at a rotation angle corresponding to the symmetry axis of each learned object. This indicates that the learning data and the input data match at a specific rotation angle. Each synapse output becomes large when the recognition target has the same shape as when learning. Then, the synapse output 43 is small even if the recognition target is the rectangular parallelepiped B, if the rotation angle of the recognition target is different from that at the time of learning.

The synapse output 43 indicating the rectangular parallelepiped B of the output layer 36c is 1
The highest peak is shown every 80 degrees. High peaks are shown every 180 degrees because the rectangular parallelepiped B has one axis of symmetry.
Because there is only a book. Further, for example, the output layer 36c
Although the synapse output 42 indicating the cylinder of No. shows peaks at 90 degrees and 270 degrees, this is an erroneously recognized portion. As a whole, the synapse output 43 becomes the highest peak every 180 degrees, and at this time, the other synapse outputs 41, 4
Since 2 and 44 are 0, the observation target is recognized as the rectangular parallelepiped B.

Table 1 below shows the results of the above-described object shape recognition. This is a result obtained from the output of four synapses corresponding to each target object shape in the output layer 36c in the shape recognition of the 12 objects described above. That is, the determination rate in Table 1 is obtained by normalizing the maximum value of the synapse outputs 41 to 44 corresponding to each target object shape by the sum of the four synapse outputs 41 to 44. Therefore, a confirmation rate of 100% indicates a state in which there is no output of erroneous recognition.

[0040]

The experimental results obtain common results regardless of the size. For example, when the object 33 is a cylinder, any of the cross-sectional areas of 20, 50 and 70 cm ² is recognized as a cylinder. Then, correct answers were obtained for the cylinder, the rectangular parallelepiped B, and the triangular prism. On the other hand, in the case of the rectangular parallelepiped A, there was a case where it was erroneously recognized as a cylinder.

As described above, it has been shown that by normalizing the measured information, for example, even if the cylinders have different sizes, they can be determined to be cylinders by learning one cylinder. However, this still recognizes cylinders of different sizes as the same. here,
As described above, by using two pieces of information indicating the size, which is a feature of the present invention, it is possible to perform object recognition for distinguishing between them. This object recognition will be described below.

In this object recognition, as described above, the shape recognition neural network 36 shown in FIG.
In addition, an object recognition neural network is used.
This object recognition neural network has four synapses to which the synapse outputs of the output layer 36c of the shape recognition neural network 36 shown in FIG. , And an input layer including six synapses for inputting the height in three stages. Then, as shown in FIG. 5, in the output layer of the object recognition neural network, an output is obtained from 4 × 3 × 3 = 36 synapses.

In this evaluation test for object recognition, the cross-sectional areas are 20, 50, 70c, respectively, as in the above-described evaluation test.
3 types of m ² , and a rectangular parallelepiped A with a height of 4 cm, a column and a rectangular parallelepiped B with a height of 7 cm, heights of 4 cm, 7 cm and 10 c
A total of 18 samples of m triangular prisms were used. As a result of the evaluation test, all 18 samples were successfully recognized. This is considered to be the result of the object recognition neural network comprehensively determining the shape and size, that is, the height and the cross-sectional area.

As the results shown in Table 1 show, erroneous recognition also occurs with only the shape recognition neural network as in the case of a cylinder and a cube. However, in the object recognition neural network, the judgment is made in addition to the cross-sectional area information and the height information in addition to the result of the shape recognition. That is, in the shape recognition, there was an erroneous recognition in the shape recognition of the rectangular parallelepiped A having a cross-sectional area of 20, 50 cm ^{2, but in} the next object recognition test, the height was different between the rectangular parallelepiped A and the cylinder. The object is determined without performing.

By the way, the function in the recognition of the neural network is different from the cascade connection of the discriminating circuits. The discriminating circuit is for sequentially discriminating the obtained measurement data by comparing with a threshold value set for each of, for example, shape judgment, cross-sectional area judgment, and height judgment. In such a discriminating circuit, a comprehensive correct answer cannot be obtained unless a correct answer can be obtained in all judgment portions of the discriminating circuit. On the other hand, the neural network has a function of judging the likelihood from each piece of information and comprehensively estimating the correct answer.

In the present invention, an ultrasonic sensor and a video camera are used together for shape measurement as an embodiment. However, the present invention is not particularly limited to this, and a laser range sensor using a light cutting method or an optical phase method may be used. Needless to say, this is possible.

[0048]

As described above, according to the present invention, after normalizing the shape information obtained by the measurement,
This is input to the neural network for recognition, and in addition, two or more of the information on the size of the object to be recognized are used for recognition by the neural network. Therefore, there is an effect that objects having similar shapes having different sizes can be distinguished and recognized only by learning a representative shape having a certain size.

[Brief description of the drawings]

FIG. 1 is a flowchart for explaining one embodiment of a three-dimensional object recognition method according to the present invention.

FIG. 2 is a perspective view illustrating a representative example of an object having similar shape information.

FIG. 3 is a configuration diagram showing a configuration of a shape recognition device according to an embodiment of the present invention.

FIG. 4 is a waveform diagram showing an example of an output of a shape recognition neural network when observing a rectangular parallelepiped B;

FIG. 5 is a configuration diagram showing an object recognition neural network according to the present invention.

FIG. 6 is a configuration diagram illustrating an example of a configuration of a conventional shape recognition device.

[Explanation of symbols]

31 ... Ultrasonic sensor array, 32 ... Video camera, 33
... Object, 34 ... Measurement plane, 35 ... Normalization unit, 36 ... Shape recognition neural network, 36a ... Input layer, 36b ...
Intermediate layer, 36c ... Output layer.

──────────────────────────────────────────────────続 き Continued on the front page (58) Fields investigated (Int. Cl. ⁷ , DB name) G01B 11/00-11/30 102 G01B 17/00-21/32 G06G 7/60 G06T 1/00 G06T 7 / 00 G06T 7/60

Claims (1)

(57) [Claims] 1. A three-dimensional object recognition method for performing shape recognition of an object and object recognition for recognizing the object itself including the size in addition to the shape of the object using a neural network. Observing the object arranged at one side from one side to obtain shape information of the object, and shape recognition by identifying the shape of the object using normalized shape information obtained by normalizing the shape information in the depth direction and the plane direction. Object recognition by identifying the object based on a result of identifying the shape of the object and two or more components of size information such as a cross-sectional area, a volume, or a diameter of the object based on the shape information. A three-dimensional object recognition method.