JPH0933247A - Image object detecting apparatus - Google Patents

Abstract

PROBLEM TO BE SOLVED: To obtain a detecting apparatus, which can perform measurement quickly and can measure even a moving object with a simple constitution without a movable part by obtaining the position of an object based on a plurality of the images obtained by photographing the object to be detected at the different positions. SOLUTION: Image obtaining devices 1a, 1b and 1c are arranged at observing points on the X axis, respectively, in x-y orthogonal coordinates. The scanning lines, which sequentially operate the image pickup planes, are approximately in parallel with the (x) coordinate axis and fixed so that the centers of all image visual fields are included on the same plane including (x) coordinate axis. The image picked up by the image obtaining devices 1a, 1b and 1c are stored in an image memory device 2. An image processor 3 processes the images for extracting the feature points from the images stored in the device 2. A distance operator 4 reads the feature points obtained by the processor 3 and obtains the space coordinates of the feature points of the object to be measured. A three-dimensional information display device 5 correlates the relation between the distance and the images with colors, luminances, patterns and the like and displays the information based on the output of the operator 4.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image object detecting apparatus for deriving position information of an object in a visual field based on a plurality of images taken from different positions.

[0002]

2. Description of the Related Art One of the methods for determining the position of each point on the surface of an object arranged in a three-dimensional space is to use a stereo image. If it is possible to obtain a pair of stereo images by imaging the target object from at least two left and right viewpoints, and to determine which point in the left image corresponds to which point in the right image, each point on the object surface The position of can be easily calculated by the principle of triangulation.

There is a three-dimensional image measuring system as an image object detecting device based on this principle. FIG. 8 shows an operation principle diagram of the three-dimensional image measurement system. A method of detecting an object will be described with reference to FIG. First, images from the left measurement viewpoint C _L and the right measurement viewpoint C _R on the X axis are obtained. Objects 51 and 52 in the field of view appear in these images. Here, attention is paid to the characteristic points 5 and 6 of the objects 51 and 52 which are measurement objects. A feature point is a point that has a feature that can be distinguished from the surroundings in the measurement target,
It is obtained by performing image processing on the acquired image. There are various methods for image processing, but for example, binarization processing and edge extraction are generally used.

If the feature points shown in the image of the left measurement viewpoint C _L and the feature points shown in the right measurement viewpoint C _R can be obtained, the angles of the feature points 53 of the object 51 and The angle of the feature point 54 of the object 52 is known. Therefore,
As shown in FIG. 8, starting from each of the left measurement viewpoint C _L and the right measurement viewpoint C _R, a straight line is drawn toward the feature point 53 of the object 51 and the feature point 54 of the object 52, and
The feature point 53 of the object 51 is obtained by finding these intersections.
And the position of the feature point 54 of the object 52 can be known.

However, when there are a plurality of measurement targets and the correspondence between the feature points of these targets is not clear in a plurality of images, false corresponding points may occur. For example, in the case of FIG. 8, intersections of straight lines from the viewpoints C _L and C _R also occur at points B and C other than the true positions A and D. Thus, if there are only two viewpoints for imaging, the true corresponding point A,
Accurate measurement is difficult because it is difficult to distinguish between D and false corresponding points. Here, the corresponding points are the corresponding feature points when the same observation target is observed from different viewpoints.

Conventionally, as a method of three-dimensional image measurement for solving this problem, there has been a method of obtaining a large number of images while moving one image acquisition device. This method will be described with reference to FIG. This figure is shown in Takashi Omori, Takeshi Morishita, "Object Detection Using Multi-View Stereo Images," Proceedings of the Society of Instrument and Control Engineers Vol. 18, No. 7, page 716.
In the figure, 15 is a movable image acquisition device. The image acquisition device 15 captures images from a plurality of viewpoints while moving on the x-axis. Further, the imaging surface 16 of the image acquisition device 15 is kept parallel to the x-coordinate axis throughout the movement of the image acquisition device 15 from the viewpoint C _L to C _R.

Only the images taken from the viewpoints C _L and C _R
Similar to the above-mentioned three-dimensional image measurement system, false corresponding points B,
C is made, and it is impossible to distinguish between true and false. Here, the image captured at the intermediate viewpoint C _i is used as another viewpoint,
By overlapping the line of sight from the intermediate viewpoint C _i , the corresponding points B,
C can be distinguished as a false corresponding point. This principle can be applied even if there are a large number of feature points and lines of sight.

The above is formulated using xy Cartesian coordinates. For simplicity of description, the position on the image is represented by one dimension, and the following notation is used. The leftmost viewpoint is C _L = C ₀ , the rightmost viewpoint is C _R = C _n , and the middle viewpoint is C _i . All these viewpoints are on the x-axis. Where i = 0, 1, 2, 3, ..., N-1, n. View Vi to C _i
Coordinates. It is C _i (Vi, 0). Let θ _{i be the} angle between the direction of the feature point measured from the viewpoint C _i and the y coordinate axis. θi
Is positive in the counterclockwise direction from the y coordinate axis. Let Di (Xi) be an image including the feature points obtained at the viewpoint C _i . Xi is a coordinate on the image pickup surface 16 of the image pickup device 15. Origin X of coordinate Xi
At i = 0, it is assumed that the image is taken in the direction of the center of the visual field.

The image acquisition device generally used is fixed so that the image pickup surface 16 and the direction of the center of the visual field are perpendicular to each other. Further, since the image pickup surface 16 of the image acquisition device 15 and the x coordinate axis are configured to be parallel, when the focal length of the optical system of the image pickup device 15 is f, the coordinates of the feature points on the image pickup surface 16 are expressed by the formula ( Given in 1). Xi = f × tan (θi) (1)

FIG. 10 shows the feature points 53, 5 shown on the imaging plane 16 at the left-end viewpoint C _L , the intermediate viewpoint C _i as one of a plurality of intermediate viewpoints, and the right-side viewpoint C _R.
4 is represented. Here, the vertical axis is a numerical value representing features such as brightness intensity and edges, and the horizontal axis is feature points 53, 5.
4 is the position on the imaging surface.

The position X _{L of the} feature point represented in the image D _L at the leftmost viewpoint C _{L and} the image D _{R at} the rightmost viewpoint C _R
When the position X _{R of} the feature point represented by ## EQU1 ## and the position X _{I of the} feature point represented by the image D _i at the intermediate viewpoint C _i satisfy the relationship of Expression (2), these feature points are the same. It is a thing. Xi = (XR (VL-Vi) -XL (VR-Vi)) / (VL-VR) (i = 2,3, ・ ・ ・, n-1) ・ ・ ・ (2)

If the feature points can be found by the equation (2), the coordinates XR, X of the feature points on the image pickup surface 16 which are in correspondence with each other can be obtained.
Based on L and the coordinates VR and VL of the viewpoint, the accurate spatial coordinates P (xp, yp) of this feature point can be obtained. xp = (XR ・ VL- XL ・ VR) ／ (XR-XL) yp = f ・ (VL-VR) ／ (XR-XL) ・ ・ ・ (3)

As described above, in the conventional three-dimensional image measuring system, the image acquisition device 15 moves on the x-coordinate axis from the left end viewpoint C _L to the right end viewpoint C _R (or in the opposite direction). _Meanwhile , the image is taken from a plurality of intermediate viewpoints C _i . The correspondence between the feature points can be accurately determined based on these plural images. In this way, the conventional three-dimensional image measuring system can obtain accurate three-dimensional information of the measuring object in the visual field.

[0014]

The conventional three-dimensional image measuring system obtains a plurality of images while one image pickup device moves as described above in order to obtain the correct correspondence relationship based on the equation (2). It has the following problems.

When the object to be measured is moving, the object to be measured moves while the image acquisition device 15 is moving the viewpoint, so that it is impossible to accurately associate the characteristic points. . Further, even when it is mounted on a robot or an automobile, the three-dimensional measurement system body moves, so that measurement is impossible.

Since there is a movable part for moving the image acquisition device 15, stable measurement is difficult. Furthermore, since there are movable parts, the entire three-dimensional image measurement system becomes complicated and large, and stable measurement cannot be expected, for example, in a situation where the volume of the installation site is limited and the environment changes greatly, such as robots and automobiles.

It takes a certain amount of time to move the image acquisition device 15, and the measurement cannot be performed quickly.

A large number of images (n-2 in the above example) at the intermediate position were required to obtain the correspondence between the feature points. Therefore, the processing takes time and the scale of hardware such as a memory increases.

The present invention has been made in order to solve the above-mentioned problems, and has a simple structure, has no moving parts, and can be quickly measured, and can also measure a moving object. An object is to provide a detection device.

[0020]

According to a first aspect of the present invention, there is provided an image object detecting device comprising three or more image acquiring means for picking up an object to be detected from different positions, and the plurality of image acquiring means. And image analysis means for determining the position of the object based on the plurality of images.

An image object detecting apparatus according to a second aspect of the present invention fixes the plurality of image acquisition means on the same fixed axis, and the fixed axis and the scanning line of the image acquisition means are substantially parallel to each other. It is arranged in each.

An image object detecting apparatus according to a third aspect of the present invention comprises a rotating means for rotating the plurality of image acquiring means about an axis intersecting the fixed axis.

According to a fourth aspect of the present invention, there is provided an image object detecting apparatus, wherein each of the plurality of image acquisition means is fixed on the same fixed axis, and at least one of the image acquisition means has its scanning line fixed to the same. It is arranged so as to intersect the axis.

An image object detecting device according to a fifth aspect of the invention is provided with an angle adjusting device for adjusting a crossing angle between the scanning line and the fixed axis.

[0025]

BEST MODE FOR CARRYING OUT THE INVENTION

[Example 1] FIG. 1 is a block diagram of an example of a three-dimensional image measuring system as an image object detecting apparatus according to the present invention. 1
a, 1b, 1c are image acquisition devices, 2 is an image storage device, 3
Is an image processing device, 4 is a distance calculation device, and 5 is a three-dimensional information display device. The image storage device 2 to the distance calculation device 4 constitute image analysis means for obtaining the position of the object based on the output images of the image acquisition devices 1a to 1c.

In the xy Cartesian coordinates shown in FIG. 2, the image acquisition devices 1a, 1b and 1c each have a viewpoint C on the x axis.
It is located at _L , C ₂ , C _R. These viewpoints C _L , C
₂ and _CR , the image acquisition devices 1a, 1b, 1c
The scanning lines for sequentially scanning the image pickup surface are substantially parallel to the x coordinate axis, and the centers of all the image pickup visual fields are fixed so as to be included in the same plane including the x coordinate axis. Here, the scanning line means a strip-shaped output that is substantially parallel to the x-coordinate axis in the acquired image, such as a scanning line of a television or an output of a row of elements that is substantially parallel to the x-coordinate axis of a two-dimensional CCD element. Say. Since the scanning line of the image acquisition device 1 is fixed substantially parallel to the x-axis, the calculation for obtaining the distance can be performed for each scanning line.

The relationship between the image acquisition device 1 and the xy coordinates is shown in FIG. In the figure, 36 is the image acquisition device 1.
, 37 is an axis parallel to the y coordinate axis and passing through the viewpoint C _i ,
Reference numeral 38 is the direction of the visual field center of the image acquisition device 1, and 39 is the direction of the characteristic points. Further, f is the focal length of the image acquisition device 1, θ
i is the angle of the feature point direction 39 measured from the y axis (counterclockwise is positive), and φ is the visual field center direction 38 measured from the y axis.
Is an angle of the feature point direction 39 measured from the visual field center direction 38. The direction of the visual field center is perpendicular to the image plane of the image. In addition, the coordinates of the feature point on the captured image plane are Xi = a.

If the respective visual fields of the image acquisition devices 1a, 1b, 1c capture the same object to be measured, the visual field center direction of the image need not be parallel to the y coordinate axis as shown in FIG. FIG. 3 shows a state in which the scanning line intersects the x axis. At this time, the image acquisition devices 1a, 1b, and 1c can simultaneously image the measurement targets.

Although only three image acquisition devices are shown in FIGS. 1 and 2, any number may be used as long as it is three or more. The images captured by the image acquisition devices 1a, 1b, 1c are stored in the image storage device 2.

The image processing device 3 performs image processing for extracting feature points from the image stored in the image storage device 2. Further, when necessary, processing such as area division for use in the three-dimensional information display means is also performed. There are various methods for area division, and for example, there is a method of making the area divided by edge extraction into one area.

The distance calculation device 4 reads the characteristic points obtained by the image processing device 3 and obtains the spatial coordinates of the characteristic points to be measured. The three-dimensional information display device 5 displays the relationship between the distance and the image in association with each other by color, brightness, pattern, etc., based on the output of the distance calculation device 4.

Now, a method of deriving the distance in the distance calculation device 4 will be described. First, the equations (2) and (3) for deriving the spatial coordinates of the measurement target are expressed using θi. Note that the spatial coordinates (x, y) used in the following description, the viewpoint coordinates Vi, the image Di (Xi), the angle θi, and the coordinates Xi on the captured image plane.
Is already defined in the description of the conventional example,
Description is omitted.

The image DL obtained at the left-hand viewpoint C _L ,
And right viewpoints C image feature points that are similar to each other on the resulting image DR of the _R is present and, on the image obtained in the middle of the viewpoint _{C 2, tan (θi) =} ((VL-Vi ) tan (θR)-(VR-Vi) tan (θL)) / (VL-VR) ・ ・ ・ If there is a similar feature point in the direction of θi that satisfies (4),
These feature points can be determined as corresponding points. Therefore, xp = (VL tan (θR) -VR tan (θL)) / (tan (θR)-tan (θL)) yp = (VL -VR) / (tan (θR)-tan (θL))・ The feature point P (xp, yp) that satisfies (5) can be determined as the coordinates of the measurement target point. From this, three-dimensional information of the measurement target can be obtained.

At this time, the distance from the origin of the feature point P (xp, yp) is r = (xp ² + yp ² ) ^1/2 (6)

It can be represented by Here, the origin is
The point O (0,0) is 0 on both the x-coordinate axis and the y-coordinate axis.

Next, with reference to FIG. 3, a method for obtaining the direction θi of the feature point from the coordinates Xi of the image captured by the image acquisition device at the viewpoint C _i will be described. The angle φ is measured in advance when configuring the three-dimensional image measuring system. The angle ψ can be expressed by the following formula using the focal length f of the optical system of the image acquisition means and the coordinate a of the feature point on the image pickup image plane. ψ = tan ^-1 (a / f) (7) Therefore, θi is as follows. θi = φ + φ (8) When the visual field center direction 38 is parallel to the y-axis as shown in FIG. 2, φ = 0 and θi = φ. In this case, the processing becomes simple.

By the above processing, the direction of the characteristic point can be derived from the coordinate Xi even when the visual field center direction of the image pickup device 1 is not parallel to the y coordinate axis. Then, the three-dimensional information of the measurement target can be obtained by using the equation (5). Here, the visual field center direction 38 is perpendicular to the imaging surface 36,
If the angle formed by the visual field center direction 38 and the imaging surface 36 is measured in advance, then θi can be derived even if it is not vertical.

The flow of the above processing will be described with reference to the flowchart of FIG. 4 in correspondence with each component. [ST1] The image acquisition devices 1a, 1b, and 1c acquire images to be measured. [ST2] The acquired image is stored in the image storage device 2. [ST3] The image processing apparatus 3 extracts feature points by using a method such as edge extraction, and at the same time, performs area division if area information is required.

Steps 4 to 7 are the flow of processing inside the distance calculation device 4. [ST4] θi is derived from the equation (8) using the coordinates Xi on the image. [ST5] A corresponding point satisfying the expression (4) is determined. [ST6] The spatial coordinates (x, y) of the feature point are derived from the equation (5). [ST7] The spatial coordinates are converted into distances by the equation (6).

Steps 8 and 9 are for the three-dimensional information display device 5.
Is the flow of processing inside the. [ST8] The color, the brightness, the pattern, etc. according to the distance and the area are associated with the image. [ST9] The measurement target is displayed.

As described above, according to the image object detecting apparatus of this Example 1, three or more images are simultaneously obtained by three or more image acquiring apparatuses, and the distance is derived from these images. Even when moving or when the image object detection device itself mounted on the robot is moving, it is possible to accurately correspond to the characteristic point of the measurement target and obtain an accurate distance. Moreover, since there are no movable parts, stable measurement is possible.

The above processing may be performed after averaging adjacent scanning lines. Generally, many feature points can be measured when the x-coordinate axis is horizontal. However, for a measurement target whose distance changes in the vertical direction such as a stepped object, the x-coordinate axis may be set in the vertical direction.

In addition, three image acquisition devices 1a, 1b, 1
It is also possible to fix c at equal intervals on the x coordinate axis and fix the visual field center direction of these image acquisition devices so as to be parallel to the y coordinate axis. Output DL (X
The relationship between L), D2 (X2) and DR (XR) is shown in FIG. Here, the vertical axis is a numerical value representing a feature such as brightness intensity or edge, and the horizontal axis is the direction of the line of sight in a plane including the feature points 53 and 54 and the x coordinate axis. In this figure, 34 is the output of the feature point 53, 3
5 is an output of the feature point 54.

With this configuration, the calculation of the equation (8) when deriving θi becomes unnecessary, and the calculation amount of the distance calculation device 4 can be reduced. If the coordinate XL is smaller than XR and there is the coordinate X2 that satisfies the average value of the coordinates XL and XR, it can be determined that the XL and XR are corresponding points. Therefore, the amount of calculation is reduced as compared with the case of using the equation (4).

In an image acquisition device in which the direction of the visual field center is parallel to the y-coordinate axis, when measuring a measurement object that is very close,
The measurement target may not be included in the visual fields of all the image acquisition devices at the same time, and measurement may not be possible. In order to avoid such inconvenience, the three image acquisition devices may be fixed at substantially equal intervals on the x-coordinate axis so that the directions of the visual field centers of the three image acquisition means intersect at substantially one point. With this configuration, it is possible to measure an object to be measured at a short distance.

[Example 2] FIG. 6 is an overall configuration diagram of the three-dimensional image measuring system of Example 2. In the figure, 6a,
6c is an image capturing apparatus 1a, the angle phi _a relative image capturing field center of 1c, and phi _b viewing angle adjustment device for varying in a plane including the x axis. Other components 1 to 5 are example 1
1 is the same as or equivalent to that shown in FIG.

Next, the operation will be described. The image acquisition devices 1a, 1b, 1c are fixed at equal intervals on the x coordinate axis. The image acquisition devices 1a and 1c are provided with viewing angle adjusting devices 6a and 6c so as to change the direction of the center of the imaging visual field. The viewing angle adjustment devices 6a and 6c are configured to detect the image acquisition devices 1a and 1c according to the expected distance to be measured.
The direction of the visual field center of c is changed. Viewing angle adjustment device 1a,
The changes in the angles φ _a and φ _b in the visual field center direction, which are changed by 1 c, are sent to the distance calculation device 4. The distance calculation device 4 derives the spatial coordinates of the feature points by using the method described in the first embodiment. The three-dimensional information display device 5 displays the relationship between the distance and the image by associating them with colors, brightness, patterns and the like.

According to this example 2, the direction of the visual field center of the image pickup device can be oriented in an arbitrary direction according to the distance of the measuring object, so that one three-dimensional object from a distant measuring object to a nearby measuring object can be obtained. Image measurement system enables measurement. Therefore, it is possible to measure distances where there is a large change in the surrounding environment, such as when a distance from a nearby obstacle to a distant passage must be measured, such as when mounted on a robot. Becomes Further, the angle adjusting device attached to the image acquisition means can measure from a short distance to a far distance with one device.

In the second embodiment, the visual field center directions of the two image pick-up devices on both sides are changed, but the visual field angle adjusting devices are attached to all the image pick-up devices to change the visual field center directions. The same effect can be obtained.

Although the case where there are three image pickup devices has been described, the distance can be measured by the same method even when there are three or more image pickup devices.

Since the scanning line of the image acquisition device is fixed substantially parallel to the x-axis, the calculation for obtaining the distance is
This can be done for each scan line. The above processing may be performed after averaging adjacent scan lines that are substantially parallel to the x coordinate axis.

Generally, many feature points can be measured when the x-coordinate axis is horizontal. However, for a measurement object whose distance changes in the vertical direction such as a stepped object, the x-coordinate axis may be set in the vertical direction.

[Example 3] FIG. 7 is an overall configuration diagram of a three-dimensional image measuring system according to Example 3. In FIG.
Are image acquisition devices 1a, 1 mounted on the x-coordinate axis.
A rotation device for rotating b and 1c around the y-coordinate axis as a central axis, and a rotation correction device 8 for correcting an error generated in the output of the distance calculation device 4 due to the rotation of the rotation device 7.
The other components 1 to 5 are the same as or equivalent to those shown in FIG.

Next, the operation will be described. The image acquisition devices 1a, 1b, 1c are fixed on the x coordinate axis, and are rotated by the rotation device 7 about the y coordinate axis. The change in the angle due to the rotation is sent to the rotation correction device 8. Rotation correction device 8
Corrects the error of the rotation angle of the image generated by the rotation device 7 with respect to the image obtained from the distance calculation device 4, and outputs so that the image is always in the vertical direction or in an arbitrarily determined direction. . The three-dimensional information display device 5 displays the relationship between the distance and the image by associating them with colors, brightness, patterns and the like.

According to this Example 3, the direction (x-axis) in which the image acquisition device is mounted can be arbitrarily changed, so that an optimum image can always be obtained. For example, in an environment in which a measurement target such as a pillar having a large distribution of feature points in the horizontal direction and a measurement target such as a staircase having a large distribution of feature points in the vertical direction are mixed, the orientation of the image acquisition device is set to the horizontal direction,
By setting the vertical direction, the spatial coordinates of the feature points can be measured with one three-dimensional image measuring system. In addition, the rotation means attached to the image acquisition means can measure the distances of the feature points distributed horizontally, vertically, or in any direction with one device.

In Example 3, the case where there are three image pickup devices has been described, but the distance can be measured by the same method even when there are three or more image pickup devices.

Since the scanning line of the image acquisition device is fixed substantially parallel to the x-axis, the calculation for obtaining the distance is
This can be done for each scan line. The above processing may be performed after averaging adjacent scan lines that are substantially parallel to the x coordinate axis.

Further, the viewing angle adjusting device described in the above-mentioned Example 2 may be added to this example. With this configuration, it is possible to measure the distance of the measurement object from a distant point to a near point in which the distribution direction of the feature points is arbitrary, with one device.

[Brief description of drawings]

FIG. 1 is a configuration diagram of a three-dimensional image measurement system of Example 1 according to the present invention.

FIG. 2 is an explanatory diagram of an operation principle of Example 1 according to the present invention.

FIG. 3 is an explanatory diagram of a relationship between an image acquisition device of Example 1 according to the present invention and coordinate axes.

FIG. 4 is a flowchart showing a processing flow of example 1 according to the present invention.

FIG. 5 is an explanatory diagram of processing of Example 1 according to the present invention.

FIG. 6 is an overall configuration diagram of a three-dimensional image measuring system of example 2 according to the present invention.

FIG. 7 is an overall configuration diagram of a three-dimensional image measurement system of Example 3 according to the present invention.

FIG. 8 is an operation principle diagram of a conventional three-dimensional image measurement system.

FIG. 9 is an overall configuration diagram of a conventional three-dimensional image measurement system.

FIG. 10 is an explanatory diagram of processing of a conventional three-dimensional image measurement system.

[Explanation of symbols]

1 image acquisition device, 2 image storage device, 3 image processing device, 4 distance calculation device, 5 three-dimensional information display device, 6
Viewing angle adjustment device, 7 rotation device, 8 rotation angle correction device.

Front page continuation (72) Inventor Satoshi Wakabayashi 2-3-3 Marunouchi, Chiyoda-ku, Tokyo Sanryo Electric Co., Ltd.

Claims (5)

[Claims] 1. Three or more image acquisition means for picking up an object to be detected from different positions, and an image analysis means for determining the position of the object based on a plurality of images obtained by the plurality of image acquisition means. An image object detection device comprising: 2. The plurality of image acquisition means are fixed on the same fixed axis, and are arranged so that the fixed axis and the scanning line of the image acquisition means are substantially parallel to each other. The image object detection device according to claim 1. 3. Centering an axis intersecting the fixed axis,
The image object detecting apparatus according to claim 2, further comprising a rotating unit that rotates the plurality of image acquiring units. 4. The plurality of image acquisition means are respectively fixed on the same fixed axis, and at least one of the image acquisition means is arranged so that its scanning line intersects with the fixed axis. The image object detection device according to claim 1, wherein 5. An angle adjusting device for adjusting an intersection angle between the scanning line and the fixed shaft is provided.
The image object detection device described.