JPH0467116A - Vision recognizing device - Google Patents

Abstract

PURPOSE:To obtain a high-speed vision recognizing device of low power consumption by performing a spatial frequency filtering process on a Fourier transformation plane so as to detect an equal-height surface from an object to be recognized and electrically or optically performing corresponding-point calculation on the detected equal-height surface. CONSTITUTION:After an object to be recognized is displayed on the first spatial photoelectric conversion element 21, spatial, frequency filtering is performed on the displayed picture with an optical filter displayed on the second spatial photoelectric conversion element 25 and the processed image is again refracted through the second lens 27. As a result, only the picture near the focal point of a TV camera 20, namely, an equal-height surface image is obtained. When the equal-height surface image is inputted to a computer 30 and recognition process is executed based on the geometrical shape of the image, a monocular three-dimensional vision is obtained without making any corresponding-point detection. Therefore, a vision recognizing device which is short in processing time and low in power consumption can be obtained.

Description

DETAILED DESCRIPTION OF THE INVENTION Field of the Invention The present invention relates to a visual recognition device that performs three-dimensional shape recognition using an optical information processing function.

Conventional technology 3D shape recognition using optical information processing function The visual recognition device will be explained based on FIG.
A CD element; 4 is a second CCD element disposed at the imaging position of the second camera lens 2; 5 is means for correlation processing of output signals of the first and second CCD elements; 6 is a target object.

Next, the operation of this conventional visual recognition device for robots that performs three-dimensional shape recognition will be explained using Figs. 9 and 10. In Fig. 10, L is the center of the first camera lens. rD\f in the second camera lens. f2 is the focal pressure of the first and second camera lenses i1i, 0 is the radius on the target object 6, θ and Φ are the base line LR and the principal ray OL, respectively.
If the angle formed by the principal ray OR and the length of the base line LR is set as D, the distance to point 0 can be expressed by equation (1).h=[)tanθtanΦ(tanθ+tanΦ)...
・Formula (1) tanθ=fl/di ---
Formula (2) tanΦ=f2/d2 --- Formula (3) di: Distance d2・R between L and beam detection point PL on CCD element 3 and distance between beam detection point P2 on CCD element 4. In the conventional visual recognition device for robots! L
Since D, fl and f2 are known, we can calculate dl and d2.The distance to the target object 6 can also be obtained.Therefore, the CCD
A cross-correlation calculation of the outputs of the element 3 and the CCD element 4 is performed using the correlation processing means 6, and the beam detection points where the correlation between the output signals of both is high are respectively assigned to the CC corresponding to the point O on the target object 6.
By regarding the beam detection point on the D element 3 and the beam detection point on the CCD element 4, that is, corresponding points on the left and right, the distance to the target object 6 can be obtained using equations (1) to (3). , it is possible to perform three-dimensional space recognition using this.a Problems to be Solved by the Invention However, the above-mentioned configuration requires the execution time of cross-correlation processing performed by a computer to determine corresponding points. In view of these problems, the present invention detects contour surfaces by performing spatial frequency filtering processing on the Fourier transform surface for the image of the target object. The purpose of the present invention is to provide a high-speed visual recognition device with low power consumption by electrically or optically performing corresponding point calculations on surfaces. In the invention, the visual recognition device 11 includes a first spatial light modulator that displays an image captured by a TV camera, a light source that illuminates the first spatial light modulator, and a location of the first spatial light modulator. a first lens whose front focal plane is a tilted surface;
a second spatial light modulator disposed on the rear focal plane of the lens; a second lens whose front focal plane is the surface on which the second spatial light modulator is placed; an image sensor disposed on the rear focal plane; It is characterized by displaying.

Or i: A first spatial light modulator that displays an image captured by an LTVTV camera, a light source that illuminates the first spatial light modulator, and a surface on which the first spatial light modulator is placed. a first lens having a front focal plane of , a second lens whose front focal plane is the surface on which the second and third spatial light modulators are placed, and a photodetector disposed at the rear focal plane of the second lens. The transmittance of each component pixel of the second spatial light modulation element is spatially modulated to a value corresponding to the distance from the center pixel, and a spatial filter is displayed; and the configuration image is displayed on the third spatial light modulation element. A first spatial light modulator that displays an image captured by a TV camera; a light source that illuminates the spatial light modulator, a first lens whose front focal plane is the surface on which the first spatial light modulator is placed, a first lens, and a rear of the first lens. a third lens whose front focal plane is a side focal plane; a fourth lens whose front focal plane is a rear focal plane of the third lens;
a second spatial light modulation element and a third spatial light modulation element disposed between the rear focal plane of the lens;
a second lens whose front focal plane is the surface on which the spatial light modulation element is placed; and a photodetector disposed at the rear focal plane of the second lens. A spatial filter is displayed by spatially modulating the transmittance of each constituent picture element of the element to a value according to the distance from the center picture element, and the transmittance of the constituent picture element is transmitted to the third spatial light modulation element. A computer-generated hologram is displayed by spatially modulating the system.The above configuration displays the target object image on the first spatial light modulation element, and the second spatial light modulation element displays the image of the target object on the first spatial light modulation element. Spatial frequency filtering is performed using the optical filter shown in . Even if the processed image is re-diffracted by a second lens, the resultant image is an image only near the focal point of the TV camera, that is, a contour image is obtained.This contour image is used as the input image of the computer and its geometric shape (For example, by performing recognition processing based on ('L aspect ratio figure center of gravity, etc.) by computer, monocular stereoscopic vision can be realized without detecting corresponding points.Also, direct spatial frequency filtering can be performed without re-diffraction. By optically performing pattern matching between the image and the computer-generated hologram of the reference figure displayed on the third spatial light modulation element, monocular stereoscopic viewing with even higher speed and lower power consumption is realized. By using a lens system, the spread of the spectrum can be arbitrarily set. By changing the range of spatial frequency filtering, the resolution in the height direction of the contour image for optically detecting corresponding points can be changed. By performing this spatial frequency filtering as a pre-processing for images taken by multiple TV cameras and using contour plane images as input images for both the left and right eyes, calculation processing for detecting corresponding points by a computer is possible. This system realizes binocular stereoscopic vision at high speed.It also optically detects corresponding points by performing pattern matching using computer-generated holograms on the contour images of both the left and right eyes. This enables binocular stereoscopic viewing with even higher speed (L) and lower power consumption.

Embodiment (First Embodiment) The configuration of a first embodiment of the present invention will be described with reference to FIG. FIG. 1 is a plan view of a first embodiment of the invention. 20 is a TV camera, 21 is a first liquid crystal display that displays an input image captured by the TV camera 20, 22 is a semiconductor laser, 23 is a collimator lens fS that collimates the light from the semiconductor laser 22, and 24 is a first liquid crystal display. 1, a first liquid crystal display 21 is disposed on the front focal plane of the first lens 24, and a second liquid crystal display 25 is disposed on the front focal plane of the second lens. Reference numeral 28 denotes an image sensor disposed on the back focal plane of the second lens 27, 30 uses the signal from the image sensor 28 as an input image, and 31 is a space to be displayed on the second liquid crystal display. This is a memory that stores frequency filter data.

Next number Mi The operation of the first embodiment of the present invention configured as described above will be explained below using FIGS. 1 to 4. FIG. 2 is an explanatory diagram of the spatial frequency filter displayed on the second liquid crystal display 25. FIG.
0ptical Tran of the imaging lens of V right camera 0
FIG. 2 is an explanatory diagram of a sfer function (hereinafter also referred to as OTF). Figure 4 shows the target object and TV right camera 0
It is a figure showing the positional relationship of. The pattern of the target object imaged by the TV right camera 0 is displayed on the first liquid crystal display 2.
This first liquid crystal display 21 is illuminated with coherent light from a semiconductor laser 22 that is collimated by a collimator lens 23.
Since the liquid crystal display 21 is arranged on the front focal plane of the first lens 24, the object imaged by the TV camera 20 can be seen on the rear focal plane of the first lens 24, that is, on the second liquid crystal display 25. A Fourier spectrum of patterns is formed. The spread of this Fourier spectrum, that is, the distance from the optical axis in one day, is expressed by equation (4).rs=f+Xλ×fs...Equation (4) f
l: Focal length of the first lens 24 λ: Wavelength fs of the semiconductor laser 22 Spatial frequency of the target object pattern, ie, plate The spread of the Fourier spectrum is proportional to the spatial frequency included in the target object.

Therefore, for example, if a pattern like the one shown in FIG. 2 is displayed on the second liquid crystal display 25, that is, a pattern that blocks light from the center of the second liquid crystal display and transmits only the peripheral part, bypass filtering in the spatial frequency domain is performed. You can also execute 1'j5. The spatial frequency included in the pattern of the target object imaged by the TV camera 20 changes depending on whether or not the imaging lens of the TV camera is in the focused position. This will be explained using FIG. 3. Figure 3 is an OTF diagram of the imaging lens, where the horizontal axis shows the spatial frequency and the vertical axis shows the imaging contrast.Currently, the imaging lens is an ideal lens with no aberrations under train coherent illumination (natural light illumination). If it is in a focused state, it will exhibit the characteristics shown by the two-dot chain line in the figure. Tokoro-hojo If you deviate from the in-focus point, OT will occur as shown in Figure 3.
F is a degrading wave plate T ■ The high frequency components of the image taken by the camera 20 are gradually lost.

Therefore, for example, the cutoff frequency of the spatial filter shown in FIG. Therefore, when a three-dimensional object with steps is imaged by the TV right camera 0, even if only the image near the in-focus point is output by the image sensor 28, the result is as shown in FIG. K A contour image of the target object 6 can be obtained by moving the TV right camera 0, or by moving only the imaging lens, or by displaying different spatial filters on the second liquid crystal display based on data from the memory 31. Although it is possible to realize 3D object recognition with a single eye by performing recognition processing on a known 2D pattern on this contour image using the computer 30, according to this embodiment, conventional passive binocular stereoscopic viewing is possible. compared to
Since there is no need to detect corresponding points between the left and right eyes, the speed can be increased.

(Second Embodiment) The configuration of the second embodiment of the present invention will be explained using FIG.
Show the same thing. A pair of TV left cameras 20a and 20bi are placed apart from each other by the distance 20a and 20bi, and correspond to left and right eyes for imaging the target object 6.
The operation of the second embodiment of the present invention configured as described above will be explained with reference to FIG. Imaged slight force <
, is displayed on the first liquid crystal display 21, and is subjected to spatial frequency filtering as in the first embodiment of the present invention. As a result, a contour image from the TV left camera 0a is input to the computer 30. Next, by the switching means 32, T
The slight force imaged by the V left camera 0b is displayed on the first liquid crystal display 21, and after similar processing, the contour image by the TV left camera 0b is input to the computer 30. 3. The computer 30 performs a known correlation calculation to detect corresponding points for the left and right binocular contour images, based on equations (1) to (3).
Dimensional shape recognition can be performed.

According to this embodiment, 1'L Compared to conventional passive binocular stereopsis, corresponding point candidates can be optically narrowed down based on their height information as a preprocessing, so corresponding point detection processing can be performed. It is possible to increase speed and reduce the possibility of incorrect response.
Recognition Accuracy Can Be Improved (Third Embodiment) Next, the configuration of the third embodiment of the present invention will be explained based on FIG. 6. However, the numbers in the figures that are the same as in Figures 1 to 5 indicate the same thing. 33 is the third liquid crystal display, which is arranged in close contact with the second liquid crystal display 25. In other words, the second liquid crystal display 25 and the third
A liquid crystal display 33 is disposed at a focal plane position behind the first lens 24. A second memory 34 stores data regarding the computer-generated hologram of the reference pattern. By spatially modulating the transmittance of the picture elements based on this, a computer generated hologram of the reference pattern is displayed on the third liquid crystal display 33. 35 is a photodetector placed on the back focal plane of the second lens 27.

The operation of the third embodiment of the present invention constructed as described above will be explained based on FIG. The pattern of the target object is displayed on the first liquid crystal display 21, and the first liquid crystal display 21 is illuminated with coherent light from the semiconductor laser 22, which is collimated by the collimator lens 23. Since the display 21 is placed on the front focal plane of the first lens 24, it is placed on the rear focal plane of the first lens 24, that is, on the second liquid crystal display 25! Melon TV camera 2
A Fourier spectrum of the pattern of the target object 6 imaged by 0 is formed, and therefore, by means of, for example, moving only the imaging lens and displaying different spatial filters on the second liquid crystal display according to the data from the mold or memory 31. A Fourier spectrum image regarding the contour plane of the target object 6 is obtained. Fourier transform computer holograms using contour planes corresponding to these various contour plane Fourier spectrum images as reference patterns are calculated in advance and stored in the memory 34. By spatially modulating the transmittance of the picture elements based on the data in the memory 34, a Fourier transform computer generated hologram of the reference contour plane is displayed on the third liquid crystal display 33.

Here, the plane on which the contour plane Fourier spectrum image of the target object 6 and the Fourier transform computer hologram image of the reference contour plane are formed! Friend is the rear focal plane of the first lens 24,
Since the frame is located on the front focal plane of the second lens 27, the pattern matching between the contour plane of the target object 6 and the reference contour plane carried out optically.

the result! According to this embodiment, as described above, by first performing spatial frequency filtering, Contour image, i.e. 2
Convert to dimensional pattern. Next, by performing optical pattern matching on the obtained contour images, it is possible to perform 3D shape recognition with a single eye, thus increasing the speed compared to conventional passive binocular stereoscopic viewing. Compared to the first embodiment of the present invention, since k pattern mapping is performed optically, the speed can be increased.

(Fourth Embodiment) The configuration of the fourth embodiment of the present invention will be explained based on FIG. 7. However, although FIG. 7 is a configuration diagram of the fourth embodiment of the present invention, The numbers indicate the same items as in Figures 1 to 6.

36 is a third lens whose front focal plane is the rear focal plane of the first lens 24, and 37 is the third lens 36.
This is a fourth lens whose front focal plane is the rear focal plane of the lens.

In this embodiment, there is a second lens on the rear focal plane of the fourth lens 37 (
LCD display 25 and third LCD display 3
3 is placed.

The operation of the fourth embodiment of the present invention constructed in this way will be explained using FIG. 7.

Similarly to the third embodiment of the present invention, the pattern of the target object imaged by the TV camera 20 is displayed on the first liquid crystal display 21.
is irradiated with coherent light from the semiconductor laser 22 that is collimated by the collimator lens 23. As shown in equation (4), a Fourier spectrum based on the spatial frequency contained in the target object appears on the back focal plane of (mi).

However, the spread of this Fourier spectrum (is determined by the focal length f1 of the first lens 24). According to the present embodiment shown in FIG.
t, fs (focal length of the third lens) and fa (fourth lens)
In other words, the light is multiplied by f4/fs and irradiated onto the second liquid crystal display 25. This expanded or reduced Fourier spectrum is subjected to spatial frequency filtering by a spatial filter displayed on the second liquid crystal display 25, and pattern matching between contour planes is performed as in the third embodiment of the present invention. As described above, according to this embodiment, monocular stereoscopic vision can be realized. Furthermore, according to this embodiment, regardless of the focal length f1 of the first lens 24, the spread of the Fourier spectrum of the target object can be set by arbitrarily setting the ratio of fs and f4. Since the frequency range, that is, the height range of the contour plane of the target object can be adjusted, the height resolution of recognition can be varied.

(Fifth Embodiment) The configuration of the fifth embodiment of the present invention will be explained based on FIG. 8. However, the numbers in the figure are the same as those in FIGS.
Table shows the same thing.

In this embodiment, the TV right camera 0a and 20b are used.The operation of the fifth embodiment of the present invention will be explained based on FIG. 8. First, a slight force imaged by the TV right camera 0a (displayed on the first liquid crystal display 21 and subjected to spatial frequency filtering as in the third embodiment of the present invention) is re-diffracted by the second lens 27. Rope board Pattern matching result for a contour image of the target object 6 Performed by the computer generated hologram displayed on the third liquid crystal display 33 Next, the slight force length imaged by the TV left camera 0b by the switching means 32 The first Similar processing a is displayed on the liquid crystal display 21. Result of pattern matching on the contour image of the target object 6 captured by the TV left camera 0b is executed by the computer generated hologram displayed on the third liquid crystal display 33.

In this way, the TV right camera 0a and 20b, i.e., the plate Therefore, compared to conventional passive binocular stereopsis, this embodiment makes it possible to optically narrow down the corresponding point candidates based on their height information as preprocessing. The corresponding point detection process itself can also be converted into optical information processing - 3D shape recognition can be greatly speeded up, and the possibility of erroneous correspondence can be reduced, so recognition accuracy can be improved. As described above, according to the present invention, a contour surface is detected by performing spatial frequency filtering processing on a target object image on a Fourier transform surface, and corresponding point calculations are performed electrically or optically on this contour surface. , a visual recognition device with reduced processing time and low power consumption can be obtained, which has great industrial effects.

[Brief explanation of the drawing]

FIG. 1 is a diagram illustrating the operation of the first embodiment of the present invention. FIG. 5 is an explanatory diagram of the operation of the first embodiment of the present invention. Figure 6 shows the configuration of the third embodiment of the present invention. Figure 7 shows the configuration of the fourth embodiment of the present invention. Figure 8 shows the configuration of the fifth embodiment of the present invention. FIG. 9 is a diagram illustrating the structure of a conventional visual recognition device. FIG. 10 is an explanatory diagram of the principle of the conventional device. 6...Object 20...TV left camera 21...First liquid crystal display, 22...Semiconductor laser, 24
...First Len X 27...Second Len X 2
8...Image sensor, 30...Calculation screen 33...Third liquid crystal display, 36...Third lens X 37.
... Name of the 4th Ren-like agent Patent attorney Shigetaka Awano and 1 other person 2nd figure Figure diagram

Claims (4)

[Claims] (1) The first screen that displays the image captured by the TV camera.
a spatial light modulator; a light source that illuminates the first spatial light modulator; a first lens whose front focal plane is the surface on which the first spatial light modulator is placed; and a second spatial light modulator disposed between the lens and the rear focal plane of the first lens, and a second spatial light modulator whose front focal plane is the surface on which the second spatial light modulator is placed. a lens, and an image sensor disposed on the rear focal plane of the second lens, and the transmittance of each component pixel of the second spatial light modulation element is set to a value according to the distance from the center pixel. A visual recognition device characterized by spatially modulating and displaying a spatial filter. (2) The first screen that displays the image captured by the TV camera.
a spatial light modulator; a light source that illuminates the first spatial light modulator; a first lens whose front focal plane is the surface on which the first spatial light modulator is placed; A second spatial light modulation element and a third spatial light modulation element are arranged on the rear focal plane of the first lens, and the plane on which the second and third spatial light modulation elements are placed is the front focal plane thereof. and a photodetector disposed on the back focal plane of the second lens, and the transmittance of each component pixel of the second spatial light modulation element is measured from the central pixel. It is characterized by displaying a spatial filter by spatially modulating a value according to the distance, and displaying a computer-generated hologram by spatially modulating the transmittance of its constituent picture elements on the third spatial light modulation element. visual recognition device. (3) The first screen that displays images captured by the TV camera.
a spatial light modulator; a light source that illuminates the first spatial light modulator; a first lens whose front focal plane is the surface on which the first spatial light modulator is placed; a third lens whose front focal plane is the rear focal plane of the first lens; a fourth lens whose front focal plane is the rear focal plane of the third lens; A second spatial light modulator and a third spatial light modulator arranged between the rear focal plane, and a spatial light modulator whose front focal plane is the plane on which the second and third spatial light modulators are placed. 2 lens and a photodetector disposed on the rear focal plane of the second lens, and the transmittance of each component pixel of the second spatial light modulation element is determined by the distance from the center pixel. A spatial filter is displayed by spatially modulating a corresponding value, and a computer-generated hologram is displayed by spatially modulating the transmittance of its constituent picture elements on the third spatial light modulation element. visual recognition device. (4) Multiple Ts with TV cameras placed a certain distance apart
4. The visual recognition device according to claim 1, 2 or 3, wherein the visual recognition device comprises a V-camera and is capable of sequentially switching and displaying images captured by the plurality of TV cameras on the first spatial light modulation element.