JPS6050407A - Recognizing apparatus of object - Google Patents

Abstract

PURPOSE:To make it possible to perform high speed processing, by providing a picture processing device and a distance measuring device, separately processing both devices, synthesizing the results, and recognizing a body in three dimensions. CONSTITUTION:An ITV1, a distance measuring ultrasonic wave element 2, a servomotor, which rotates the ITV1 and the ultrasonic wave element 2, a servomotor controlling device 3, a picture processing device 4 which processes the picture taken by the ITV1, and an ultrasonic wave processing device 5, which controls the timing of the emission of the ultrasonic wave from the ultrasonic wave element 2 and obtains a distance from the received ultrasonic wave signal, are provided. A synthesizing apparatus 6 synthesizes the processed results from the picture processing device 4 and the ultrasonic wave processing device 5 and performs three-dimensional recognition. The high-speed, three-dimensional recognition is carried out by the processing device 6.

Description

[Detailed description of the invention] [Field of application of the invention] The present invention relates to improvements in object recognition devices.

[Background of the invention]

When a robot performs assembly work or when a mobile robot determines a movement route, it is necessary to recognize objects and their three-dimensional positions and postures, and to do so, it is necessary to calculate the distance and direction to the objects. Chiru.

For this reason, conventionally, the method shown in Figure 1 or Figure 2 has been used. .

an image processing device for processing the image captured in 12;
4 is an object; in FIG. 2, 21 is an ITV, 22 is a light-shielding plate with a slit 23 installed at a distance from the ITV 21, and can be rotated around axis 0; 24 is an object;
5 is an image processing device that processes images captured by the ITV 21. Both Figures 1 and 2 are processed based on the principle of triangulation, but in order to find the coordinates of point P on the object 14 using the method shown in Figure 1, the images of ITVI1 and 12 are processed using the image processing bag ft.
It is necessary to process the information in step 13 and match the two sides.

However, this process is complicated, takes a long time, and is somewhat unreliable. Therefore, it is unsuitable as a visual device for robots that require high speed. Furthermore,
Correlation may become extremely difficult depending on lighting conditions.

Next, the method shown in FIG.
The linear slit light passed through the object 24 is applied to the object 24.
The degree of curvature of the line on 4 is captured by the ITV 21, processed by the image processing device 25, and the distance between each point is determined by triangulation.
It recognizes objects. However, the disadvantage of this method is that in bright places, the difference in brightness due to the slit light cannot be clearly seen, and when the object is thick, the slit light does not bend.
It is impossible to distinguish between an object and the background.

Figure 1 requires two expensive ITVs, and Figure 2 requires a light source that emits slit light.
Furthermore, the cost is high, and FIG.

The method shown in FIG. 2 has the disadvantage that distance calculation and image processing are closely related, and therefore cannot be processed independently.

[Purpose of the invention]

Therefore, an object of the present invention is to provide an object recognition device that is economical and capable of high-speed processing.

[Summary of the invention]

The feature of the present invention is that an image processing device and a distance measuring device are provided, and after processing is performed independently by both, the results are subjected to combined processing to recognize the object f in three dimensions.

[Embodiments of the invention]

Hereinafter, the implementation sequence of the present invention will be explained in detail using FIG. 3.

In Figure 3, 1 is an ITV, 2 is an ultrasonic element for distance measurement, and 3
i, 1: TT 1 and a servo motor for rotating the ultrasonic element 2 and its control device; 4 is an image processing device that processes images captured by the IT Vl; 5 is an ultrasonic wave emitted from the ultrasonic element 2. 6 is a composite processing device that performs three-dimensional recognition by synthesizing the results processed by the image processing device 4 and the ultrasonic processing device 5. . Although ultrasonic waves are used for distance measurement here, laser light, infrared light, or other light may also be used.

Next, the operation of this embodiment will be explained in detail using FIG. 4 as an example. First, assume that an object is placed as shown in (a). This is visualized using ITVI, and from the obtained image, an image processing device 4 performs image processing such as noise removal processing to remove brightness noise contained in the image, and boundary line enhancement processing necessary for separating objects. The result is shown in Figure 4(b). An example of the configuration of the image processing device 4 that realizes this is shown in FIG. That is, when the central processing unit slave 51 receives an image processing request from the external interface 52, it gives a command to the clock generator 53. The clock generating section 53 generates a clock signal for transmitting the video signal from the ITVI to the A/Df.
Converter 54 is provided. At this time, the image information converted into digital signals by the A/D converter 54, excluding the horizontal synchronization signal and vertical synchronization signal included in the video signal, is stored in the image memory 55, or processor 5
given to 6. The image processing such as noise removal and border enhancement described above is performed by the image processor 56, and these processes are performed using weighting coefficients for the pixel of interest Xlj, including the surrounding 8 pixels, as shown in FIG. Multiply and take the sum,
This can be done by a so-called product-sum operation. That is, the output pixel y
Ij is framed. This product-sum operation may be processed by liftware, or may be processed by hardware.

In addition, the picture I image grosser 5G may process the picture marks from the A/D converter 54 and store them in the image memory 55, or it may process the pictures from the picture memory 55 and store them in the picture memory again. You can also do that. This switching control is performed by the central processing unit 5.
Do it in 1.

Next, a method for determining the position (distance, direction) of the object shown in FIG. 4(a) will be described. An ultrasonic sensor for distance measurement uses one with a short duration of ultrasonic waves. This may be, for example, a capacitor type made of plastic film.

The principle of measuring the position of an object is as shown in FIG. 7(a), by rotating the ultrasonic sensor 2, for example, in the direction ■→■→■, and processing the reflected waves of the transmitted ultrasonic waves from the object 72. That is, when the reflected ultrasound waves from the directions ■, ■, and ■ are observed, the corners A and B of the object 72 (in this case, a square prism) are observed as shown in FIG. 7(b).

Since the reflected wave from C can be clearly detected, the direction of the object from the direction of the ultrasonic sensor 2 when the reflected wave is at its maximum can be determined by the distance from the direction of the ultrasonic sensor 2 to the time from when the ultrasonic wave is transmitted until the reflected wave returns. I put it on. FIG. 8 shows an example of the configuration of the ultrasonic processing apparatus 5 (FIG. 3) for realizing this. In FIG. 8, when the arithmetic processing unit 81 receives an object position measurement request from the external ink-7 ace 82, it issues a request to start ultrasonic transmission from the ultrasonic sensor 2 and to the A/D conversion unit 83 to take in a signal.

Upon receiving the request, the A/D converter 83 supplies a clock signal to the A/D converter, converts the reflected wave received by the ultrasonic sensor 2 into a digital signal, and stores the digital signal in the memory 84 . This is done for a predetermined period of time. After that, the arithmetic processing unit 81 examines the contents of the memory 84 and calculates the distance to the object and the peak value of the reflected wave. That is, in FIG. 9, the peak values are PI + P2, P
3, and the distance r is calculated as r=vt σ by the time ta from when the reflected wave crosses a predetermined threshold value to α when it crosses zero and the sound speed V.

After completing the above processing, a rotational start is given to the servomorph via the 1t11 control line 800 to rotate the ultra-long range sensor 2 by a predetermined angle, and the reflected waves described above are recorded.

Calculate distance and check peak value. When this is repeated over a predetermined angle range, the civic value P+ becomes a function of the rotation angle as shown in FIG. 10 due to the directivity characteristics of the ultrasonic sensor. The direction of the object is defined as the angle at which the maximum value is reached among the observed peak values. Furthermore, in order to improve accuracy, it is conceivable to interpolate p from a plurality of peak values, for example, by the method of least squares, as shown in FIG. 10, and to determine the angle θ0.8 at which the maximum peak occurs.

Therefore, through these processes, the object device can be viewed as a plan view as shown in FIG. 4(C). This processing can be performed independently of the subsequent image processing once the above-mentioned image has been loaded into the image memory, thereby shortening the overall processing time.

Next, as shown in FIGS. 4(b) and 4(c), the obtained image information and position information are subjected to composite processing by a composition processing device 6. As shown in FIG. 11, the synthesis processing device 6 includes a central processing unit 111, an external interface 112 for interfacing with an image processing device, an external interface 113 for interfacing with an ultrasound processing device, and a memory for storing a three-dimensional model. 114.

First, as pre-processing for the composite processing, the image information (b
), it is necessary to make a correspondence between the location information (C),
From the position information (C), as shown in Figure 12, the distances to the object are rA, rB, rc and the direction is 0 people.

Since θdoor and θC are round, the coordinates of the corners of the object can be found using the following equation. However, the y-axis is perpendicular to the plane of the paper and the direction from the back side to the front side is positive.

(xA, ZA) = (rAcosoA, rAsinθ^
)(Xi, zs 1=(r, cosθ”trllal
llθB) (Xc, Zc)・= (rc cO5
θc+rC81 [1θC) Since the rotation of the ultrasonic sensor is performed in a plane parallel to the X-Z plane, y is a constant value y. It is.

The correspondence between the coordinates obtained by the above equation and the coordinates on the image sensor of the ITV can be determined by proportional calculation using FIG.

That is, rut It! for A in FIG. The coordinates (ξE, η^) on the element are the focal length of the ITV lens F
It is claimed by XA ξA2□ as.

As a result, each point obtained from the position information (C) in Figure 4 can be associated with the image t'tt information (b), and as a composite process, for example, as shown in Figure 14, the points obtained from the origin 0 can be distance 1
It is also possible to erase images above 0 and perform object recognition while leaving only the object 142, or vice versa.

When the procedure described above is converted into a flowchart, it becomes as shown in FIG. 15.

In this embodiment, the ultrasonic sensor is rotated to adjust the position (distance, direction), but in another embodiment, a plurality of ultrasonic sensors 161 to 16 are used as shown in FIG.
There is also a method in which the NFCs are arranged at appropriate angles, and the scanning device 160 sequentially oscillates ultrasonic waves from the ultrasonic sensors 161 to 16n. In this method, position measurement processing can be performed in parallel from the process of capturing one image.

Furthermore, as shown in FIG. 17, ITv171
It is also conceivable to install the ultrasonic sensor 172 at a predetermined angle of 0.

〔Effect of the invention〕

As described above, according to the present invention, object recognition in a complex background, which would be extremely difficult and time consuming to process using only one image, can be performed with a simple configuration and in a short time.

[Brief explanation of drawings]

Figure 1 shows a conventional three-dimensional recognition device using two ITVs.
Fig. 2 shows a conventional three-dimensional recognition device using slit light, Fig. 3 shows an example of the present invention, and Fig. 4 shows a three-dimensional recognition device according to the present invention.
A model for explaining the method of dimensional recognition, Fig. 5 is an example of the configuration of an image processing device, Fig. 6 is an explanatory diagram of image processing by a dual-image processor, and Fig. 7 is an object generated by ultrasonic waves. Place 1f
The principle of TG perception and the reflected waveform from an object, Figure 8 shows an example of the configuration of an ultrasonic processing device, Figure 9 shows the ultrasonic waveform recorded in memory, and Figure 10 shows the direction of ultrasonic emission and the intensity of the reflected wave. Fig. 11 shows an example of the configuration of a multifunction processing device, Fig. 12 shows an explanatory diagram of a method for calculating the three-dimensional coordinates of an object, and Fig. 13 shows how the object position obtained by the ultrasonic sensor is associated with the position on the image. An explanatory diagram of the method, Fig. 14 is an explanatory diagram of one model of processing by a compound processing device, Fig. 15 is an image processing device, an ultrasonic processing device,
Flowchart of 3D g-knowledge in multiprocessing equipment, Part 1
Figure 6 shows another embodiment of the ultrasonic sensor part of the present invention, the first
FIG. 7 shows another embodiment of the present invention. 11.12,21.1...mouth'V, 13,25.4.
...Image processing device, 14, 24. ．． 72...object, 2
2... Light shielding plate, 23... Slit, 2... Ultrasonic sensor, 3... Two-botor motor and controller, 5...
... Ultrasonic processing device, 6... Complex processing device, 51.
...Central processing unit, 52...External interface,
53... Clock generation section, 54... A/D converter,
55... Image memory, 56... Image processor,
81... Arithmetic processing section, 82... External interface, 83... A/D & switching section, 84... Memo ILIII
... Central processing unit, 112° 113 ... External interface, 114 ... Memory, 141, L42, 14
3...Object, 160...Scanning device R1161~16n
...Ultrasonic sensor, 171...Cod [71 Wipe 2 Kasumi foil 3 (2) (C) What 6 Mouth 70 (α) Sweetfish 8 No. 90 Shikishi I J6gu 1O Figure 1/ Figure 1? Figure 13 Continued from page 1 0 Inventor Morio Kanazaki 3-chome, Saiwai-cho, Hitachi City;
Inside the office

Claims (1)

[Claims] 1. A first means for measuring the coordinates of an object, and a second means for imaging the object, wherein the coordinates measured by the first means and the image taken by the second means are provided. An object recognition device characterized in that it is configured to three-dimensionally recognize an image captured by the second means by correlating points on the image. 2. Claim 1 is characterized in that the first means for measuring the coordinates of the object is an ultrasonic sensor, and the ultrasonic sensor is attached to a rotating device and configured to rotate at any angle. Object recognition device. 3. An object recognition device according to claim 1, characterized in that the first means for measuring the coordinates of the object includes a plurality of ultrasonic sensors. 4. An object recognition device according to claim 2, characterized in that second means for capturing an image of the object is attached to the rotating device.