JP2580516B2 - Real-time three-dimensional motion measuring apparatus and method - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a real-time three-dimensional motion measuring apparatus and method for measuring the motion state of an unknown object that moves (moves) in a three-dimensional space or an object that is difficult to model. The present invention relates to a real-time three-dimensional motion measurement device suitable for a robot that operates using measurement results and a method thereof.

[0002]

2. Description of the Related Art As a method for measuring three-dimensional motion in real time, for an object whose shape is known, a method is known in which a model is compared with an obtained image to determine a motion state (for example, Dnald B. Generaly: Internet
al Journal ofComputer Vis
ion, 243, (1992)). In this measurement method, the model is rotated and moved while estimating the motion of the object in a computer, and the obtained shape is superimposed on an actual image to estimate the motion of the object. Having an accurate model makes it possible to estimate the motion of the object.

[0003] For the case where the imaging result is not a moving image, there is a conventional measuring method using image recognition by stereoscopic vision. In this case, two cameras are used but fixed. The position of the object is calculated based on the principle of triangulation by taking correspondence between the left and right images obtained from the two cameras.

[0004]

In the method using the above model, an accurate model is absolutely necessary. However, in the real world, the shape of an object is unknown or the shape is complicated and modeling is difficult. In many cases, there is a problem that measurement is difficult without a model.

[0005] In stereo vision of a fixed camera, it is difficult to detect corresponding points, it takes time, and real-time processing is difficult.
Further, since a corresponding point cannot be obtained even when a zoom lens is used for a distant measurement, the measurement result includes a large error.

SUMMARY OF THE INVENTION It is therefore an object of the present invention to provide a real-time three-dimensional motion measuring apparatus and a measuring method capable of tracking an object whose motion state is to be measured without using a model.

[0007]

In order to achieve the above object, the present invention is directed to an image pickup means for picking up an image of a moving object to be measured by two image pickup devices, Movable means for moving the line-of-sight directions of the two imaging devices, detection means for detecting a pixel position of a feature point of an object having an optical feature from among imaging results obtained by the imaging means, A control unit that controls the tracking of the movable unit so that the two imaging devices gaze at a feature point of the object based on the detection result, wherein the detection unit detects a peripheral point in an imaging result of the imaging unit. A pixel having the largest difference in luminous intensity level from the pixel and the position of the pixel are detected, and the position is set as a pixel position of a feature point of the object having the optical characteristic.

According to a second aspect of the present invention, there is provided an arithmetic processing means for calculating a moving speed of the object by using a plurality of gazing positions gazed in time series by the two imaging devices and a moving time between the gazing positions. Further features are provided.

According to a third aspect of the present invention, a moving object to be measured is imaged at a fixed interval by two image pickup devices, and an image of the object having optical characteristics is obtained from the image pickup results of the two image pickup devices. As a feature point, a pixel having a largest difference in luminous intensity level from a peripheral pixel and its position are detected, and the feature point is set as a gazing point in the next imaging by the two imaging devices and the two imaging devices are used as the gazing points. It is characterized by tracking an object.

According to a fourth aspect of the present invention, a moving object to be measured is imaged at a fixed interval by two imaging devices, and the characteristic of the object having an optical characteristic from the imaging results of the two imaging devices. A point is extracted, the feature point is used as a gazing point in the next imaging by the two imaging devices, and the object is tracked by the two imaging devices. The imaging result is an image signal for each pixel of one screen. It is characterized in that the pixel density in the central part of the one screen is made small and the peripheral parts other than the central part of the one screen are made relatively rough.

[0011]

According to the first and third aspects of the present invention, as a feature point of an object, a pixel having a largest difference in luminous intensity level between peripheral pixels in an image pickup result is tracked as a gazing point of two image pickup devices. It is possible to track an object that is difficult to model or an unidentified object.

According to a second aspect of the present invention, the moving speed of the object is calculated using the gaze position, and the calculation result is a measurement result.

According to a fourth aspect of the present invention, since the number of pixels is smaller than that in a screen configuration in which all pixel densities are small, the extraction time of a feature point is shortened. Since the position of the feature point moves to the center of the screen, the tracking accuracy does not deteriorate.

[0014]

Embodiments of the present invention will be described below in detail with reference to the drawings.

FIG. 1 is an explanatory diagram showing the principle of the measuring method according to the present invention.

In FIG. 1, when a video camera corresponding to the left eye and the right eye gazes at a specific point of gaze in an object, the coordinates (x, y, z) of the position P of the point of gaze are the positions of the left eye and the right eye, and the left eye and the right eye. It is determined by the line of sight vector from each right eye to the gazing point based on the principle of triangulation.

Further, the position P 'of the gazing point after a certain unit time
Have a similar relationship.

Therefore, according to the present invention, a point having a characteristic of an object (hereinafter, referred to as a characteristic point) is automatically detected from the imaging result, and the characteristic point is tracked as a gazing point of two video cameras. . For this purpose, it is characterized in that the attitude control of the two video cameras is executed by the (real-time three-dimensional) measuring device, and the motion state of the position and speed of the object is measured by the measuring device under such control. is there.

FIG. 2 shows an example of a circuit configuration of a measuring device mounted on a robot adopting such a measuring method.

In FIG. 2, an analog image signal obtained for each pixel as an imaging result of the right-eye and left-eye video cameras 5B and 5A is converted into a digital form by analog / digital (A / D) converters 3B and 3A. The data is transferred to the microprocessor (MPU) 1 via the input / output interface (I / O) 2. The MPU 1 stores one screen of the image signal for each pixel in the internal memory at every fixed unit time. When the MPU 1 detects a feature point by performing image processing described later, the MPU 1 determines a new gazing point of the video cameras 5A and 5B, and performs attitude control such that the video cameras 5A and 5B gaze at the gazing point. In addition, the speed of the measurement target is calculated using the position of the previous gazing point, the current position of the gazing point, and the moving time of the gazing point (the characteristic point of the object).

The imaging results of the video cameras 5A and 5B are composed of one screen, and the pixels constituting the screen have pixels with a narrow space at the center of the screen like a human retina. Have coarse pixels. If it is difficult to implement such a pixel arrangement with hardware, it is sufficient to average the imaging results of a high pixel density and set the pixel density as described above.

The video cameras 5A and 5B are installed on a platform (not shown, called a pan / tilt movable platform) which can be turned up and down and left and right along the horizontal direction as viewed from the eyes of the robot.
It is moved in the above and below and left and right directions by well-known drive mechanisms 4A and 4B using motors and the like.

As the driving mechanisms 4A and 4B, any mechanism may be used as long as it can move the video cameras 5A and 5B by the moving direction and the moving amount instructed by the MPU 1.

The operation of the circuit of FIG. 2 and the contents of the measurement processing will be described below with reference to the flowchart of FIG.

FIG. 3 functionally shows an execution processing procedure of the MPU 1. This execution procedure is actually stored and executed in the MPU 1 in the form of a program language. In FIG. 3, as initial processing, the video cameras 5A and 5B look at a specific position in front of the robot so as to gaze at a specific position (see FIG. 1)
Is determined, and the posture position of the video camera is initialized (S1). In such a state, when the measurement target enters the field of view of the video cameras 5A and 5B, the MPU 1 next acquires the imaging results from the video cameras 5A and 5B and stores them in the internal memory. Further, the MPU 1 detects, for each video camera, a pixel having a largest change (difference) in the luminous intensity level, for example, the luminance from peripheral pixels (hereinafter, referred to as a maximum change pixel) (S2, S3). More specifically,
Using each pixel of one screen as a pixel of interest, the index value f
Is calculated, and the pixel having the maximum value among the values of f is determined as the maximum change pixel.

[0026]

(Equation 1)

Here, Pr, Pg, and Pb are red of a certain pixel,
Indicates the intensity value of green and blue light.

O indicates the position of the pixel of interest. K represents a peripheral pixel position around the target pixel. Equation 1 is an example in which 5 × 5 pixels around a target pixel are used for detecting a maximum change pixel.

Thus, the video cameras 5A and 5B
When the maximum change pixel is detected from among the imaging results, the gazing point is set to the current robot front position so that the maximum change pixel is set as the gazing point, that is, the maximum changing pixel becomes the center position of the imaging result. Move from. The moving direction and the moving amount for this are calculated by a predetermined arithmetic expression, and the moving direction and the moving amount are instructed from the MPU 1 to the driving mechanisms 4A and 4B (S4). At this time, since there is a possibility that feature points are extracted with coarse pixels, the video image is captured again (second time), and the direction of the camera is finely adjusted in the direction of the feature point moved to the vicinity of the center (S5). . Next, MP
U1 takes in the imaging results (the third time) of the video cameras 5A and 5B again in S6, and moves the camera in the direction in which the correlation between S6 and the image becomes maximum near the center of each of 5A and 5B (S6). The movement amount of the object is calculated from the difference in the camera direction between S6 and S5, and the speed is calculated based on the movement amount. Here, S6 may be continuously performed a plurality of times to increase the accuracy of the speed measurement. MPU 1 again has video camera 5A,
Another point of regard is found from the result of the imaging of the 5B (the fourth time) (S7). The execution processing of the MPU1 returns from S7 to S4.
In the loop processing of S7, the attitude control of the video cameras 5A and 5B, that is, the measurement target is positioned at the center of the imaging result, that is,
Attitude control for tracking an object is performed.

A third imaging of the object is performed, and when the maximum pixel position of the imaging result is detected, the speed of the object is calculated in S6. The details of this processing will be described. The gazing point positions of the video cameras 5A and 5B at time t are (x, y, z),
When the gazing point position at time t + Δt is (x ′, y ′, z ′) (see FIG. 1), the velocity of the object in the three-dimensional space is expressed by the following equation.

[0031]

Vx = (x′−x) / Δt

[0032]

Vy = (y′−y) / Δt

[0033]

## EQU4 ## It is given as Vz = (z'-z) /. DELTA.t.

The position of the gazing point is

[0035]

[Outside 1]

[0036]

[Outside 2]

Then (see FIG. 1), two straight lines represented by the following equations are obtained.

[0038]

(Equation 5)

[0039]

(Equation 6)

Is obtained as the intersection of

The above calculation result is output as a signal as the measurement position and the measurement speed of the object. As described above,
In the present embodiment, since the point having the maximum luminous intensity change of the object is used as the optical feature point, it is possible to easily recognize the presence of the object without performing complicated image processing such as model recognition.

The following example can be carried out in addition to this embodiment.

1) Of course, the video camera may have a zoom function.

2) In this embodiment, the real-time three-dimensional motion device mounted on the robot has been described, but it can be mounted on other devices.

3) Maximum luminous intensity change defined here (Equation 1)
The present invention can be implemented by using a feature extraction method other than the method f).

[0046]

As described above, according to the present invention, it is possible to provide a real-time three-dimensional motion measuring apparatus capable of not only tracking a moving object to be measured without using a model but also performing a measurement process quickly.

[Brief description of the drawings]

FIG. 1 is an explanatory view showing a measuring method of the present invention.

FIG. 2 is a block diagram showing a circuit configuration of an embodiment of the present invention.

FIG. 3 is a flowchart showing a processing procedure executed by an MPU 1 of FIG. 2;

[Explanation of symbols]

 1 MPU 2 I / O 3A, 3B A / D converter 5A, 5B Video camera

Claims (4)

(57) [Claims] 1. An image pickup means for picking up an image of a moving object to be measured by two image pickup devices, a movable means for moving the line-of-sight directions of the two image pickup devices of the image pickup means, and the image pickup means. Detecting means for detecting a pixel position of a feature point of an object having an optical feature from an imaging result; and detecting the feature points of the object by the two imaging devices based on the detection result of the detecting means. Control means for performing tracking control of the movable means, wherein the detection means detects a pixel having the largest difference in luminous intensity level from peripheral pixels in the imaging result of the imaging means and its position, and A real-time three-dimensional motion measuring device, wherein a position is a pixel position of a feature point of an object having the optical feature. 2. The image processing apparatus according to claim 1, further comprising: an arithmetic processing unit configured to calculate a moving speed of the object by using a plurality of gaze positions watched in time series by the two imaging devices and a movement time between the gaze positions. The real-time three-dimensional motion measuring device according to claim 1. 3. An image of a moving object to be measured is taken at fixed intervals by two image pickup devices, and the image pickup results of the two image pickup devices are used as peripheral feature points of the object having optical characteristics. Detecting the pixel having the largest difference in luminous intensity level from the pixel and its position, and causing the two imaging devices to track the object using the feature point as a gazing point in the next imaging by the two imaging devices. A real-time three-dimensional motion measuring method characterized by the following. 4. A moving object to be measured is imaged at regular intervals by two imaging devices, and feature points of the object having optical characteristics are extracted from imaging results of the two imaging devices, The feature points are used as gazing points in the next imaging by the two imaging devices, and the two imaging devices are caused to track the object. The imaging result is configured by an image signal for each pixel of one screen, and image processing is performed. A real-time three-dimensional motion measurement method, characterized in that the pixel density in the central portion of the one screen is made small and the peripheral portions other than the central portion of the one screen are made relatively coarse for the purpose of improving the efficiency.