JPS62162175A - Stereo visual device - Google Patents

Abstract

PURPOSE:To substantially raise detection accuracy by synthesizing detection edge components with the same azimuth difference in vertical and horizontal directions and obtaining an edge group forming a closed pattern, for instance, as sectional information on an object. CONSTITUTION:A camera 3 is moved in the horizontal direction to input a horizontal stereo visual picture, and the edge component in the vertical direction is extracted, and distance-decomposed on the basis of the azimuth difference in a stereo visual picture. In the same manner the camera 3 is shifted in the vertical direction to input the stereo visual picture in the vertical direction, and its edge component in the horizontal direction is extracted and distance- decomposed on the basis of the azimuth difference. Correspondence is given to edges with the same azimuth difference, and a set of edge components forming a closed pattern, for instance, are extracted as information showing a vertical section on the optical axis of the object. A set of the edge components expressing said section information are overlapped with respect to an original picture. Therefore, whether the detected section information is correct or not is tested.

Description

DETAILED DESCRIPTION OF THE INVENTION The present invention relates to a stereo visual apparatus capable of highly accurately obtaining information on a cross section of an object perpendicular to the optical axis of the camera from a stereo image.

[Technical background of the invention]

Recently, with the development of robot control technology and image processing technology 1 (I), there has been active research and development into autonomous robots with visual abilities.

This type of robot is used for tasks such as maintenance and inspection of nuclear power plants, where it is dangerous for humans to enter. It is required to perform work under 'a environment.

Now, in order for a robot to move autonomously, it must automatically recognize environmental information in its direction of movement, such as whether there are obstacles or corners, and then move the robot while detecting spaces it can pass through. A function to control is required.

However, typical conventional three-dimensional space information input methods include: ■ Calculating the distance between propagation points to the target object using lasers or ultrasonic waves; ■ Calculating the distance to the target object using the principle of triangulation. There are methods that measure Δ1, and methods that recognize objects using constraints on the environment such as changes in the object's shadow and shape due to different viewpoints, and textures. Among them, the stereo method, which is a type of method (2) above, is attracting attention as being suitable for robot vision because of its versatility and ease of input.

In the stereo method described above, an object is viewed using a plurality of cameras having different viewpoint positions, and the distance to the object is determined from the parallax by applying the principle of triangulation.

Specifically, the contour edges of the object are extracted from two images (stereo current images) obtained by inputting the object from positions separated by a predetermined distance in the horizontal direction, and the edges of the object are extracted from the two images. Find the corresponding edge between. Then, the distance to the edge (object) is calculated using the parallax with respect to the corresponding edge in the two images.

[Problems with Background Art] However, such conventional methods have the following problems.

In other words, when a stereo image is obtained by moving the camera in parallel in the horizontal direction, the vertical edge components can be correlated with high precision by using the constraint (Ebibora line). For edge components that are not oriented horizontally, it is difficult to make highly accurate correspondences. In addition, the edge component is caused by the M environment condition in which the object is placed, the shadow of the object on the floor, unwanted reflected light from the floor, quantization noise associated with image processing, etc. Incorrect correspondence is likely to occur, and its detection reliability is poor. Furthermore, there are other problems, such as that it is difficult to verify whether the edges associated in the stereo image are truly contour edges of the object.

For this reason, it is necessary to determine the contour edges of the object from the stereo image. S
In order to accurately detect the object and measure the distance to the object with high precision and reliability, further consideration and ingenuity were required.

[Purpose of the invention]

The present invention has been made in consideration of these circumstances, and its purpose is to provide a stereo visual device that can detect edge components of objects from stereo images with high precision and reliability. 12 It is to serve.

C. Summary of the Invention] The present invention provides dynamic input of an object from positions equidistant from each other in the horizontal direction and the vertical direction within a plane perpendicular to the optical axis of the camera. Two-phase stereo viewing with different viewpoints in the horizontal and vertical directions, in which the object is imaged by moving the same distance in parallel in the horizontal and vertical directions in a plane perpendicular to the optical axis. Obtain a loincloth.

Then, the edge components in the vertical direction are detected from the stereo images with different viewpoints in the horizontal direction, and the edge components in the horizontal direction are detected from the stereo images with different viewpoints in the vertical direction. Each detected edge component in the direction is distance-decomposed according to the parallax in each of the stereo current images.

Then, among these distance-resolved detected edge components,
Detection edge components having equal parallax in the vertical and horizontal directions are combined to obtain information on a cross section perpendicular to the optical axis of the object, and information on this cross section is converted into an original image (image) that is the stereo image. The cross-section information detected regarding the object is verified by cross-width comparison.

〔Effect of the invention〕

Thus, according to the present invention, from two sets of stereoscopic images with different viewpoints in the horizontal and vertical directions, each edge component in the vertical and horizontal directions that can be accurately correlated in conjunction with the constraint conditions is obtained. , each of the detected edge components in the vertical and horizontal directions in each of the stereoscopic images is distance-decomposed according to the zero disparity, and the detected edge components in the vertical and horizontal directions that have the same disparity are synthesized. In this way, for example, a group of edges forming a closed pattern is obtained as cross-sectional information of the object, so that the detection accuracy can be made sufficiently high.

Moreover, since the cross-sectional information of the object detected in this way is compared by superimposing it on the original image and verifying the detected cross-sectional information, stereo visual information about the object can be obtained with high precision and reliability. It becomes possible to obtain. Therefore, it can be effectively applied to the visual acuity of robots, and great practical effects can be achieved.

(Embodiment of the invention) An embodiment of the invention will be described below with reference to the drawings.

FIG. 1 shows the basic processing flow in the embodiment device, and FIG. 2 shows 1a12! of the embodiment device. FIG.

A processing device main body 1, which includes an image memory, a performance device, etc., drives a camera position system tit+ti structure 2 to control the image position of an object by a camera 3.

The camera 3 is moved horizontally and vertically by equal distances in parallel within a plane perpendicular to its optical axis; for example, camera support! Distance 11 in the horizontal direction based on the 3 level mA
111, and to a position C, which is a distance wia apart from the normal force H, and each of the above @A, B, C
It is designed to input each object.

The image inputted as a wL image by camera 3 in this way is
The image is displayed and monitored on the image monitor 4, and is also provided to the processing device main body 1 for image processing.

The processing device main body 1 has the positions fffA, 8., which are set by the above-described parallel movement. Camera 3 is van
The input images are stored in an image memory (not shown), and a pair of images (q) at positions A and B separated by a distance MJ2 in the horizontal direction is used as a horizontal stereoscopic image. A pair of ii obtained at HA and C separated by a distance #tC in the direction is taken as a vertical stereoscopic image.

The following processing is then performed on these two sets of stereoscopic images to obtain information on a cross section of the object perpendicular to the optical axis of the camera 3.

That is, as shown in FIG. 1, the processing device main body 1 first moves the camera 3 in the horizontal direction to input a horizontal stereoscopic image, and in step a), converts the vertical edge components of this stereoscopic image into ↑ (Step b)
. Then, as will be described later, each rafter direction edge component is distance-decomposed based on the parallax in one stereo image (step C).

Thereafter, similarly, the camera 3 is moved in the vertical direction to input a vertical stereo image (step d), and horizontal edge components in this stereo image order are extracted (step e).

Then, each horizontal edge component is distance-resolved based on the parallax in this stereoscopic image (step f).

Through the above procedure, edges with equal parallax are associated with each other between the two sets of stereoscopic images, and, for example, a set of edge components forming a closed pattern is formed into a cross section perpendicular to the optical axis of the object. Then, by superimposing a set of edge components representing this cross-sectional information on the original image, it is verified whether the detected cross-sectional information is correct (Step Q). Step 1))
.

Through this series of processing, it becomes possible to detect only the edge components related to the object from the stereoscopic image with high precision and reliability.

The above processing will be explained in more detail.

For example, an image of a rectangular parallelepiped object is shown in Figure 3 (Ia).
Assume that an image as shown in (one of 112 images forming one stereoscopic image) is obtained. In the case of a horizontal stereoscopic image, only the vertical edge component is extracted as shown in FIG. Extract only the horizontal edge component.

Extraction of edge components in the horizontal and vertical directions is realized, for example, by spatial filtering processing on the original image. Specifically, when extracting edge components using a <3X3)iIi prime type digital filter, when extracting vertical edges, the process shown in Fig. 4 (a
), and when extracting horizontal edges, a mask weighted as shown in FIG. 4(b) may be used.

When edges are detected from stereoscopic images in this way, in general, in addition to the edge components showing the outline of the counterstain, as shown in FIG.
I edge component is detected.

Distance decomposition processing using parallax is performed on the edge component J3 of each stereoscopic image obtained in this way as follows. FIG. 5 shows an example of a complicated procedure for distance decomposition of edge components using this parallax, and FIG. 6 shows an example of the processed image.

That is, when the optical axes of the stereo cameras are parallel and the vertical edge images obtained from the two images constituting the stereoscopic image are shown in FIGS. 6(a) and 6(b),
On the other hand, for example, using the left image Ia@ at the position A as a reference, move the bu (1 image) horizontally at the position B (step p), and move the two images as shown in the same figure (C). Edge superimposition processing is performed between the images (step q).

Theoretically, it is sufficient to perform edge matching processing by relative parallel movement of the two images.

Through this superimposition process, a correlation coefficient between the two edge images is calculated (step r), and the calculated value is subjected to threshold processing to obtain corresponding edges (step S).

This extraction of corresponding edges involves, for example, A ij of each density value of two edge images in a stereoscopic image.

Blj, and the correlation coefficient within the specified window Wmn is 3in.
This can be achieved by calculating as follows.

(O≦Smn≦1) And this correlation coefficient i! Smn is compared with a predetermined threshold value, and only edge components that obtain a correlation coefficient of 3 nm greater than the threshold value are extracted as necessary corresponding components as shown in FIG. 6(d). This process is repeated +-T while sequentially changing the amount of translation. For example, edge components on the equidistant cross-section are obtained sequentially from the near side, and the edge components in the edge image are distance-resolved.

This process is performed simultaneously between the horizontal edge images in the vertical stereoscopic images.

The distance decomposition of the edge component that exposes this parallax is based on the following parallax theory.

That is, as shown in FIG. 7, for example, the position Q1. It is assumed that an image at a forward position P separated by a distance from Q2 is imaged. However, the above distance 11D is
It is assumed that the distance from the lens center position fi01.02 is d, and the distance from this lens center position fiOLO2 to the imaging surface is d. Then, the deviation width of the imaging position P1 from the lens center on the imaging plane of the elephant P seen from the position Q1 is 21, and the deviation width from the lens center of the image P seen from the position Q2 on the 189th plane @position P2q Assume that the shift width is 22.

In this case, the parallax S relative to the object P in the stereo image is: 3 = fil +, Q
2= (Ql Px (d/D)) + (Q2 Px (d, /D) 1= d
Bow/D. In this way, the parallax S and the rough alignment to the counter-dyed object P are the alignment of edge images such as the one shown in the camera (which has a linear relationship with the distance between the stereoscopic images, therefore, the distance between the two). By performing the processing while changing the amount of parallax, it becomes possible to perform distance decomposition of each edge component in the edge image.

After performing distance decomposition processing for each edge component in the horizontal and vertical directions in this manner, edge components having the same parallax are combined to extract a set of edges forming, for example, a closed region.

Specifically, the vertical edge component with the parallax of
The horizontal edge components having the same parallax degree are similarly extracted as shown in FIG. 1 (1). Then, these two edge images are superimposed as shown in FIG. 8(C), and r! is determined by the horizontal and vertical edge components. A set of edges forming the j1 region is detected. This set of edges is obtained as information on a cross section of the object perpendicular to the optical axis regarding the parallax.

As a result, cross-sectional information regarding the object is obtained corresponding to 211 from the camera 3.

However, in this apparatus, the cross-sectional information obtained in this way, that is, the set of edges, is superimposed on the original image @ (one side of the stereoscopic image), and it is verified whether the detection result is correct or not. This verification may be a visual inspection using the image monitor 4, but here, one Ishikawa of the original painting inside the closed area surrounded by the set of edges is determined, and whether or not the value is greater than or equal to a predetermined threshold value is determined. is being determined. That is, when a closed region surrounded by a set of edges is a cross section of an object, the detection result is verified by utilizing the fact that 1 degree inside the closed region is clearly different from 1 degree in the background.

The above series of processes prevents incorrect object detection due to false edges caused by reflections, shadows, noise, etc.
Only highly accurate and reliable information about objects is now required.

As described above, this device detects only edge components with high detection reliability that comply with the constraint conditions from 2Ki stereoscopic images obtained from different viewpoints in the horizontal and vertical directions. After distance decomposing the edge components according to the 7M difference, a set of edge components having the same parallax is obtained, and a set of edges forming the closed #4 area is obtained as cross-sectional information of the object. Therefore, the detection accuracy can be made sufficiently high.

Furthermore, cross-sectional information detected in this way is verified by matching it with the original image, so that only truly correct cross-sectional information is obtained.

Therefore, it is possible to achieve great practical effects, such as improving detection accuracy and improving detection reliability.

Note that the present invention is not limited to the embodiments described above. For example, a plurality of cameras may be used to simultaneously obtain horizontal and vertical stereoscopic views (p).

Furthermore, for individual image processing, conventionally known techniques can be used as appropriate. In short, the present invention can be implemented with various modifications without departing from the gist thereof.

[Brief explanation of drawings]

The figures show one embodiment of the present invention. Fig. 1 is a diagram showing the concept of image processing, Fig. 2 is a schematic configuration diagram of the i-th embodiment, and Fig. 3 is a diagram showing an example of edge component detection. , Fig. 4 is a diagram showing an example of a mask used for edge detection, Fig. 5 is a diagram showing the flow of distance decomposition processing of edge components based on parallax, and Fig. 6 is an example of an occluded image in distance decomposition of edge components. FIG. 7 is a diagram showing the parallax principle, and FIG. 8 is a diagram showing an example of a cross-sectional information detection processed image. 1... Processing device main body, 2... Camera position control structure,
3...Camera, 4...Image monitor. Applicant: Todoroki Director of Industrial Technology i4i ) No. 8! S71

Claims (2)

[Claims] (1) Images of the object are input from positions equidistant in the horizontal and vertical directions in a plane perpendicular to the optical axis of the camera, and two sets of images are captured with different viewpoints in the horizontal and vertical directions. Means for obtaining stereoscopic images, means for detecting vertical edge components from stereoscopic images with different viewpoints in the horizontal direction, and detecting edge components in the horizontal direction from stereoscopic images with different viewpoints in the vertical direction means for distance-decomposing each detected edge component in the vertical and horizontal directions according to the parallax in each stereoscopic image; and means for distance-decomposing each of the detected edge components in the vertical and horizontal directions according to the parallax in each of the stereoscopic images; a means for obtaining information on a cross section perpendicular to the optical axis of the object by combining detected edge components having a cross section of the object; A stereo visual device characterized by comprising means for verifying information. (2) The two sets of stereoscopic images are obtained by moving one camera equidistantly in parallel in the horizontal and vertical directions within a plane perpendicular to its optical axis. The stereo visual device according to item 1.