JP2000304508A - Three-dimensional input device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To realize three-dimensional input which is not dependent on projection angle information of reference light, obtain three-dimensional input data of high precision, independently of precision of projection angle control, and improve reliability of data. SOLUTION: Three-dimensional input equipment 1 projecting reference light U comprises a pair of imaging systems 20A, 20B which image an object at positions isolated from each other, a first matching means 311 which makes pixel positions of a first and a second imaging surfaces correspond with each other on the basis of imaging data when the reference light is projected, a second matching means 312 which makes pixel positions of the first and second imaging surfaces correspond with each other on the basis of imaging data when the reference light is not projected, and a selecting means 313 selecting either one out of the two obtained correspondences. Data D corresponding to a light receiving angle decided by the selected correspondence relation are outputted.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a three-dimensional input device for projecting reference light onto an object, scanning the object, and outputting data for specifying the shape of the object.

[0002]

2. Description of the Related Art A non-contact type three-dimensional input device called a range finder can perform higher-speed measurement than a contact type, and therefore can input data to a CG system or a CAD system, measure a body, and use a robot. It is used for visual recognition.

[0003] As a measurement method suitable for a range finder, a slit light projection method (also called a light cutting method) is known. In this method, an object is optically scanned and a distance image (3
Dimensional image), which is a type of active measurement method for projecting a specific reference beam to photograph an object. The distance image is a set of pixels indicating three-dimensional positions of a plurality of parts on the object. In the slit light projection method, slit light in which a cross section of a projection beam has a linear band shape is used as reference light. At a certain point during the scanning, a part of the object is irradiated, and a bright line that is curved according to the undulation of the irradiated part appears on the imaging surface. Accordingly, a group of data (three-dimensional input data) for specifying the object shape can be obtained by periodically sampling the luminance of each pixel on the imaging surface during scanning.

Conventionally, the incident angle of slit light reflected by an object and incident on the imaging surface is determined based on the position of the bright line in the imaging surface, and the incident angle, the projection angle of the slit light, and the base line length ( The position of the object is calculated from the distance between the projection start point and the light receiving reference point) by triangulation. That is, the position calculation based on the projection direction and the light receiving direction of the reference light has been performed. It should be noted that the position of the object is calculated based on the projection direction and the light receiving direction even when spot light (beam section is point-shaped) is used as the reference light as in the device of JP-A-10-2722.

[0005]

Conventionally, the accuracy of the three-dimensional input data depends on the accuracy of the control of the projection angle of the reference light, so that sufficiently high accuracy of the three-dimensional input data cannot be obtained. There have been problems that expensive components must be used to ensure accuracy, and that it takes time to adjust the mounting posture of the light projecting system. The reason why it is difficult to ensure accuracy is that the light projection system has a movable mechanism for deflecting the reference light, and its operation is easily affected by changes in the use environment such as temperature and humidity.

The present invention realizes a three-dimensional input that does not depend on the projection angle information of the reference light, obtains highly accurate three-dimensional input data regardless of the accuracy of the projection angle control, and improves the reliability of the data. The purpose is. Another object is to improve practicality by diversifying operation modes.

[0007]

According to the present invention, an object illuminated by reference light and an object illuminated by environmental light are sequentially photographed using two points apart from each other as viewpoints. The position of the object is calculated by a triangulation method from the distance between the points and the azimuth of the point of interest on the object viewed from each viewpoint (inclination with respect to a straight line connecting the viewpoints). At this time, in order to specify the direction of the point of interest, in addition to the pixel correspondence search (matching) for two captured images having different viewpoints when projecting the reference light, the captured image when the reference light is not projected A corresponding search is also performed for. Depending on the shape and color scheme of the object, the correspondence search when the reference light is not projected has higher reliability. By properly selecting the results of the two types of correspondence search, the reliability of the data can be improved.

According to a first aspect of the present invention, there is provided an apparatus for projecting a reference beam from a starting point toward an object to be scanned so as to scan an object to be inputted, and an apparatus for photographing the object at a first position. A first photographing system, a second photographing system for photographing the object at a second position away from the first position, and projecting the reference light to project the object into the first and second images. Based on the shooting data when shooting at the same time with the shooting system of
First matching means for associating pixel positions where light from the same point on the object is incident between the imaging surface of the first imaging system and the imaging surface of the second imaging system; and Based on photographing data when the object is photographed by the first and second photographing systems without projecting light, a photographing surface of the first photographing system and a photographing surface of the second photographing system are compared. Second matching means for associating pixel positions where light from the same point on the object is incident between the first and second imaging means, and each pixel on an imaging surface of the first imaging system is obtained by the first matching means. Selecting means for selecting one of the correspondence and the correspondence obtained by the second matching means, and receiving light at each of the first and second positions determined by the correspondence selected by the selection means Depends on the angle It is a three-dimensional input device which outputs the data as the positional information of the object.

In the three-dimensional input device according to the second aspect of the present invention, the selecting means has a higher reliability among the correspondence obtained by the first matching means and the correspondence obtained by the second matching means. Choose one.

[0010] In the three-dimensional input device according to the third aspect of the present invention, the selecting means is configured such that the reliability of the correspondence obtained by the first matching means is smaller than a first reference value, and the reliability is obtained by the second matching means. Only when the reliability of the obtained correspondence is larger than the second reference value, the correspondence obtained by the second matching means is selected.

According to a fourth aspect of the present invention, in the three-dimensional input device, the identification data indicating which of the first matching means and the second matching means the correspondence selected by the selecting means is obtained. Output in association with the data.

According to a fifth aspect of the present invention, there is provided a three-dimensional input device for outputting data corresponding to a light receiving angle at each of the first and second positions determined by the correspondence obtained by the first matching means. A second mode for outputting data corresponding to a light receiving angle at each of the first mode and the first position determined by the correspondence selected by the selection unit;
And a third mode for outputting data corresponding to the light receiving angle at each of the first and second positions determined by the correspondence obtained by the second matching means. Is selected as the operation mode.

[0013]

FIG. 1 is a functional block diagram of a three-dimensional input device 1 according to the present invention. The three-dimensional input device 1 includes a light projecting system 10 that projects slit light U, two imaging systems 20A and 20B having the same configuration, and two light reception signal processing units 30A and 30B having the same configuration. I have.

The light projecting system 10 includes a semiconductor laser 12 as a light source, a lens group 13 for beam shaping, and a galvanomirror 14 as beam deflecting means for changing a projection angle.
And is arranged in the middle of the imaging systems 20A and 20B. The lens group 13 includes a collimator lens and a cylindrical lens. The galvanomirror 14 is provided with a deflection control signal from the light projection control circuit 32 via the D / A converter 33.

The photographing system 20A includes a light receiving lens 21, a beam splitter 22, an image sensor 24A for photographing the object Q irradiated with the slit light U to obtain a distance image, and a color image sensor for photographing the object Q in color. 25
A, and a lens drive mechanism 26 that enables zooming and focusing. The beam splitter 22
The emission wavelength range of the semiconductor laser 12 (for example, center wavelength 685
nm) and visible light. Similarly, the shooting system 20B
Also, it comprises a light receiving lens 21, a beam splitter 22, an image sensor 24B, a color image sensor 25B, and a lens driving mechanism 26. Image sensors 24A, 24B
The color image sensors 25A and 25B are two-dimensional imaging devices (area sensors). As these sensors, a CCD sensor or a CMOS sensor can be used.

The outputs of the image sensors 24A and 24B are converted into light receiving data of a predetermined number of bits by an A / D converter 35, and are sequentially transferred to a memory circuit 37. Memory circuit 3
7, a light receiving angle θA, θ described later according to the value of the light receiving data.
The time centers of gravity TA and TB specifying B are stored. The output of the color image sensor 25 is converted into light receiving data by an A / D converter 36, and is sequentially stored by a color image memory 38. The stored color image is used for three-dimensional input by stereo vision. Addressing of the memory circuit 37 and the color image memory 38 is performed by the memory control circuit 39.

The CPU 31 for controlling the three-dimensional input device 1 gives an instruction to the control target in a timely manner. Further, the CPU 31
The time centroids TA and TB are fetched from the respective memory circuits 37 of the light receiving signal processing units 30A and 30B, and a color image is read from each of the two image memories 38 to perform two types of matching processing to generate distance data. . The distance data is output to a not-shown external device as three-dimensional input data as appropriate. At this time, the light receiving signal processing unit 3
0A and / or 30B color image memory 38
Is also output as reference information. The external device includes a computer, a display, a storage device, and the like. The CPU 31 includes a block 311 for performing matching between temporal centroid images described later,
A block 312 for matching between color images,
And a data processing program for realizing each function of the block 313 for selecting one of the two types of matching results.

The three-dimensional input device 1 is provided with three modes for generating distance data. In the first mode, distance data is obtained based only on the result of matching of the time-center-of-gravity images. In the second mode, distance data is obtained by selecting one of the result of matching of the temporal barycenter image and the result of matching of the color image in accordance with a predetermined condition.
In the third mode, distance data is obtained based only on the result of color image matching. These modes are appropriately selected according to the shape and color arrangement of the object. For example, when the color of an object is uniform, the reliability of color image matching is low, so the first mode is suitable. Conversely, in the case of an object having a low reflectance of the reference light, the third mode is suitable because the reliability of matching of the temporal barycenter image is low. The second mode is suitable for an object in which portions having different reflectivities are mixed.

The CPU 31 detects the designated mode by determining the state of the mode designating section 51. The mode designation unit 51 includes an operation panel, a dip switch provided separately from the operation panel, an interface for remote operation, and the like.

FIG. 2 is a schematic diagram of the projection. The three-dimensional input device 1 projects the slit light U so as to scan the virtual surface VS with a point on the reflection surface of the galvanometer mirror 14 as a starting point C. The virtual plane VS corresponds to a cross section orthogonal to the depth direction of the space (range within the angle of view) that can be photographed by the image sensor 24. Then, the image sensor 24 of the virtual surface VS
A range ag corresponding to each pixel in A and 24B is a sampling section for three-dimensional input. In FIG. 2, a starting point C of light projection and viewpoints (principal points of light reception) A and B are arranged on a straight line, and the position of the starting point C is intermediate between the viewpoints A and B. Here, the viewpoints A and B are arranged along the vertical direction, and the slit length direction of the slit light U is the horizontal direction.

FIG. 3 is an explanatory diagram of the time center of gravity. In the three-dimensional input device 1, the slit width on the imaging surface S2 has a pitch pv
A relatively wide slit light U corresponding to a plurality of pixels g arranged in a row is projected on the object Q. Specifically, the width of the slit light U is set to about 5 pixels. The slit light U is deflected around the starting point C at a constant angular velocity in the vertical direction in the figure. The slit light U reflected by the object Q passes through the principal point A of imaging (the principal point on the rear side of the zoom lens) and enters the imaging surface S2. The object Q (strictly speaking, an object image) is scanned by periodically sampling the amount of light received by each pixel g on the imaging surface S2 during the projection of the slit light U. One frame of photoelectric conversion signal is output from the image sensors 24A and 24B for each sampling cycle (imaging cycle).

Assuming that the surface of the object Q is flat and there is no noise due to the characteristics of the optical system, the amount of light received by each pixel g on the imaging surface S2 is within the range (strictly, the center) of the pixel g. It becomes maximum when the optical axis of the slit light U passes through the object surface ag, and its temporal distribution is close to a normal distribution. In FIG. 3, attention is paid to the j-th pixel g _j in the vertical direction on the imaging surface S2, and the transition of the amount of received light is shown in FIG. Received light amount between the n-th sampling time T _n and the preceding (n-1) th sampling time T _n-1 in the example of FIG. 3 (b) is the largest. The point at which the amount of received light in one pixel g of interest in this manner is maximized, that is, the inflection point near the top of the luminance distribution curve is defined as "time barycenter". However, in the detection of the time barycenter by discriminating the magnitude of the discrete sampling data, there is a maximum shift of サ ン プ リ ン グ of the sampling period between the detection result (time point T _{n in} the example of FIG. 3) and the actual time barycenter. .

Here, the incident angle of the slit light U with respect to each pixel g is uniquely determined from the positional relationship between the principal point A and each pixel g on the imaging surface S2. Therefore, the time barycenter is “the slit light U at a specific angle (this is called the light receiving angle θA).
At the time of incidence on the principal point A ".

FIG. 4 is a diagram for explaining a method of generating a distance image that does not depend on projection angle information, and FIG. 5 is a schematic diagram of matching between time centroid images. The outline of the three-dimensional input of the object Q not depending on the projection angle information is as follows.

In the first mode and the second mode, the slit light U is projected, and the deflection angle of the galvanometer mirror 14 is controlled in synchronization with the photographing of the two image sensors 24 at the frame period. At this time, the two image sensors 24A, 2A
4B are driven at the same timing. That is, the object Q is photographed simultaneously from the viewpoints A and B. Then, each pixel of each of the image sensors 24A and 24B is detected at which point in time the projection of the slit light U deflected by the moment is illuminated.

Based on the output of one image sensor 24A, a time barycenter TA is detected for each pixel g on the photographing surface S2, and a time barycenter image GA, which is a set of detection results, is recorded in a memory. At this time, a memory space for recording each time barycenter TA is specified by a horizontal address i indicating a pixel position in the horizontal direction (main scanning direction) of the imaging surface S2 and a vertical address j indicating a pixel position in the vertical direction. Similarly, based on the output of the other image sensor 24B, the photographing surface S2
, A time barycenter TB is detected for each pixel g, and a time barycenter image GB, which is a set of detection results, is recorded in the memory.

Focusing on the pixel g _iAjA of the iA-th horizontal direction and the jA-th vertical direction in one image sensor 24A, when the slit light U passes through a point P on the line of sight corresponding to the pixel g _iAjA , the output becomes maximum. Becomes At this time, paying attention to the output of the other image sensor 24B, the output of the pixel _giBjB corresponding to the line of sight passing through the point P becomes the maximum. Here, assuming that the epipolar constraint is established in the vertical direction with respect to the temporal center-of-gravity image GA and the temporal center-of-gravity image GB, the pixel g _iAjA and the pixel g
The horizontal position iB of _iBjB is uniquely determined. Also, the pixel g
vertical position jB pixel g _IBjB for vertical position jA of _IAjA, among the pixel columns in the horizontal direction position iB time centroid in the image GB, output time and the time at which the output of the pixel g _IAjA becomes maximum You can tell if you find the largest pixel. Therefore, the time centroids TA _iAjA , TB _iBjB , which are the times at which the output of each pixel of the image sensors 24A, 24B is maximum.
, It is possible to find out the pixel of the temporal centroid image GB corresponding to each pixel of the temporal centroid image GA.

When the point P corresponds to the pixel g _iAjA of the time barycenter image GA, the light receiving angle θA determined by the position of the pixel g _iAjA
The point P exists on a straight line specified by _iAjA and the spatial coordinates of the viewpoint A. Similarly when the point P corresponds to the pixel g _IBjB time centroid image GB, to the presence of a point P on the line specified by the spatial coordinates of the light receiving angle .theta.B _IBjB and viewpoint B determined by the position of the pixel g _IBjB Become. That is, the point of intersection of these two straight lines is point P. Therefore, based on the light receiving angles θA _iBjB , θB _iBjB and the base length L, the distance _{Di AjA} in the depth direction between the base line passing through the viewpoints A and B and the point P can be calculated by applying the principle of triangulation. , The relative position between the viewpoints A and B and the point P can be specified. Then, if the above processing is performed for each pixel g of the temporal center-of-gravity image GA, three-dimensional position information (distance images) of sampling points of the object Q for the number of pixels can be obtained.

Next, a specific configuration of a circuit for detecting the time centroids TA and TB will be described. In the following, a subscript representing a pixel position is used unless it is necessary to distinguish the pixel position.
The description of _iAjA and _iBjB is omitted.

FIG. 6 is a block diagram of a first example of the memory circuit. The example memory circuit 37 includes two memories 371,
376, a comparator 377, and an index generator 378.

The memory 371 receives data from the A / D converter 35.
The optical data D35 is input, and the
Frame number T is input from the
You. . The comparator 377 is provided for each pixel of the image sensor 24.
Received data of the t-th frame that is the latest input data
D35 and received light data previously written to memory 371
D35 is compared with the latest received light data D35.
When the data is larger than the optical data D35, the memories 371 and 376
Write is permitted for. In response to this, each memory 37
1,376 stores the latest input data in an overwrite format
You. When the comparison result is reversed, the data is stored in each of the memories 371 and 376.
And the previous stored contents are retained. Therefore, the scan
At the time of completion, the memory 371 receives the data for each pixel g.
The maximum value of the optical data D35 is stored.
The number of the frame in which the received light data D35 is maximum for each element
T will be stored. Shooting of each frame is a fixed cycle
, The frame number T is the time during the scanning period.
(Elapsed time from the start of scanning). That is, the memory 3
The frame number T stored in 76 is the time barycenter TA described above._iA
jA, TB_iBjBAnd the light receiving angle θA_iAjA, ΘB_iBjBof
This is information for specifying the correspondence.

According to this example, a relatively simple circuit configuration
By time center of gravity TA_iAjA, TB_{iB jB}Can detect
it can. However, the resolution of the detection is
Depends on the shooting cycle. The resolution improvement is as follows.
This is a second example.

FIG. 7 is a block diagram of a second example of the memory circuit.
FIG. 8 is a diagram showing the relationship between the luminance distribution on the photographing surface and the received light data. 7, the elements corresponding to FIG. 6 are denoted by the same reference numerals as in FIG.

The memory circuit 37b of the second example includes a memory 37
1 plus 4 memories 372, 37 of the same size
3, 374, and 375, and each of the memories 372 to 3 is provided with a total of four 1-frame delay memories 379a to 379d.
The configuration is such that 75 data inputs are sequentially delayed one frame at a time with respect to the memory 371. That is,
In the memory circuit 37b, five consecutive frames of light receiving data D35 for each pixel g are simultaneously stored. The comparator 377 has a third memory 3 whose input is delayed by two frames.
Compare the 73 inputs and outputs. The input data value of the memory 373 is the output data value (the previously written data value)
If larger, the memory 371 to 375 and the memory 3
76 writes are allowed.

At the end of each scan, the memory 3
Reference numeral 73 stores the maximum value of the light receiving data D35 for each pixel g. Also, the memories 371, 372, 374,
With 375, the received light data D35 of two frames before, one before, one after, and two after the frame where the received light data D35 is maximum are stored. And
The memory 376 stores the number T of the frame in which the light receiving data D35 is maximum for each pixel g.

Here, as shown in FIG. 8A, it is assumed that the width of the slit light image formed on the photographing surface is 5 pixels and the luminance distribution is a single-peak mountain-like shape. At this time, paying attention to one pixel g, light reception data of a change corresponding to the luminance distribution is obtained as shown in FIG. Therefore, the memory 37
By performing the center of gravity calculation based on the light receiving data D35 for five frames stored in 1 to 375, the time centers of gravity TA and TB can be calculated in finer increments than the frame period (that is, the pixel pitch). In the example of FIG. 8B, the time barycenter TA (TB) is between the t-th and (t + 1) -th sampling times.

According to the second example, the resolution is improved.
There is a problem that desired accuracy cannot be obtained depending on the luminance distribution. That is, in actual shooting, some noise is added to the image due to the characteristics of the optical system and the like. For this reason,
A plurality of peaks are generated in the luminance distribution, or the luminance distribution is flat and the peaks are unclear. If the luminance distribution deviates greatly from the ideal shape, the reliability of the center-of-gravity calculation decreases.

The influence of such noise is caused by the calculation of the center of gravity based on the luminance distribution in a sufficiently long period, not in a period as short as the sum of the frame in which the maximum value of luminance is obtained and several frames before and after the frame. By doing so, it can be reduced. This is realized in the following third example.

FIG. 9 is a block diagram of a third example of the memory circuit.
FIG. 10 is a conceptual diagram of the center of gravity according to FIG. The memory circuit 37c of the third example includes a memory 3710, a steady light data storage unit 3720, a subtraction unit 3730, a first addition unit 3740, and a second
It comprises an adder 3750 and a divider 3760, and calculates the time centroid based on the light receiving data D35 for the number of frames for each pixel g.

The memory 3710 stores light reception data D35 of a predetermined number q of frames obtained by scanning the object Q. The light receiving data values of a frame of T th pixel g (T = 1 to q) is represented as x _T. Stationary light data storage unit 3720
Stores steady light data representing the amount of unnecessary incident light other than the slit light U. The steady light data is calculated based on the received light data D35 when the slit light U is not incident.
The value s may be a predetermined fixed value, or may be obtained in real time using the received light data D35. In the case of a fixed value, when the light receiving data D35 is 8 bits (256 gradations), for example, "5", "6", or "10" is used. Subtraction unit 3730 subtracts the value s of the stationary light data from the value x _T of the light reception data D35 read from the memory 3710. Here, the value of the output data from the subtraction unit 3730 is set to X _T again. First adder 3740
, For the q received light data D35 for each pixel g, performs multiplication of each value X _T and the frame number T corresponding thereto, and outputs the sum of the resulting product. Second adder 3
750 is a value X _T of q light receiving data D35 for each pixel g.
Output the sum of The division unit 3760 includes the first addition unit 37
The output value of 40 is divided by the output value of second adder 3750, and the obtained time centroids TA and TB are output.

Although the above-described three-dimensional input uses the slit light projection method, a three-dimensional input that does not depend on projection angle information can be performed by using a stereo vision method. FIG. 11 is an explanatory diagram of matching between color images.

In the second and third modes, color photographing of the object Q is performed from the viewpoints A and B without projecting the slit light U. A color image CA captured by one color image sensor 25A and another color image sensor 2
Assuming that an epipolar constraint is established in the vertical direction between the color image CB and the color image CB taken in 5B, the color image CA
The search direction of the pixel PB ′ (iB, jB ′) in the color image CB, which is the corresponding point of the pixel PA (iA, jA) in the middle, is unique to the horizontal position iB with respect to the horizontal position iA of the pixel PA. Decided. It is assumed that the image size and each pixel of the image sensors 24A and 24B and the color image sensors 25A and 25B match. A well-known correlation method is used for searching for corresponding points between color images. Here, the contents will be briefly described.

FIG. 12 is a diagram showing a general example of correlation, and FIG. 13 is a diagram showing an example of a reliability value. In the correlation method, pixels are associated by comparing local patterns of images. For the target pixel PA of the color image CA, a window W is set around the target pixel PA, and a pixel column along the search direction of the color image CB (on the pixel column at the horizontal position iB)
Is checked for the similarity with the window W centered on each pixel. For the window W, for example, 3 × 3, 5 ×
An appropriate size of an odd × odd configuration such as 5, 3 × 5 is selected. Of the window W of the color image CB,
The center pixel PB ′ (iB, jB ′) of the window W having the highest similarity and having a sufficiently sharp correlation curve peak shape.
Is the corresponding point of the target pixel PA.

Here, a so-called correlation value is used for evaluating the similarity. The correlation value S is calculated by the following equation.

[0045]

(Equation 1) The correlation value S is in the range of “−1 to +1”, and the closer to “1”, the higher the similarity between the two windows W. The corresponding points are
It is determined by searching for the peak of the curve as shown in FIG. If the correlation value S is constant over a wide range, it means that the image pattern does not change. Even if the correlation value is large, the corresponding point cannot be specified. Also, even if a peak exists, if the correlation value of the peak is not sufficiently large, it is inappropriate to determine the pixel at the peak position as the corresponding point. When a peak exists and the correlation value S of the peak is sufficiently large, the pixel PB ′ (iB, jB ′) is determined as a corresponding point of the target pixel PA (iA, jA). Here, the reliability value _RC is determined as follows. When a peak does not exist, a constant “0” is multiplied by a constant “1” when a peak exists, and the result is used as a reliability value. At this time, a value indicating a negative value is forcibly set to “0”.

A block 313 (see FIG. 1) which is a functional element of the CPU 31 and corresponds to the selecting means of the present invention.
Has a confidence value R _T, which is calculated from the correlation value S _T of the corresponding points block 311 is determined by the matching of the time center between images, the correlation values for corresponding points block 312 is determined by the matching of each other color image S _C and the reliability value R _C calculated from S _C are compared, and the one showing a value closer to “1” is selected as the corresponding point. In this embodiment,
The matching time center between images, is not used a correlation method, the correlation value S _T is multiplied by the constant "1" as the reliability value R _T in here indicates a negative value forced "0 ". Also, when no corresponding point is found, the reliability value _RT is forcibly set to "0". Alternatively, only when the reliability value R _T is smaller than a predetermined reference value Ra and the reliability value R _C is larger than a predetermined reference value Rb, a corresponding point obtained by matching between color images is selected. To be configured.

By selecting such a corresponding point for each pixel of the photographing surface S2 (the pixel arrangement is the same as that of the color image sensor 25A) of the image sensor 24A, the slit light projection method and the stereo vision method are used together. That is, the reliability of the distance data D can be enhanced by taking advantage of both.

FIG. 14 is a flowchart showing the general operation of the three-dimensional input device 1. When the color mode, which is the third mode described above, is designated, color imaging of the object Q is performed by the imaging systems 20A and 20B without performing projection (# 1, # 2), and each pixel of the color image CA is A corresponding pixel of the color image CB is searched (# 3). Then, distance data D is calculated and output based on the result of matching between the color images (# 4). The output here is
It means data transmission to an external device or recording (data storage) in a memory in the device.

If it is determined in step # 1 that the mode is not the color mode, the semiconductor laser 12 and the galvanomirror 1
4 is turned on to start scanning (# 5). The light receiving data D35 obtained by photographing at the viewpoints A and B is recorded (# 6).
After the scanning is completed and the semiconductor laser 12 is turned off, color imaging of the object Q is performed by the imaging systems 20A and 20B without performing projection (# 7, # 8). Then, the mode is determined again (# 9).

When the corresponding point selection mode, which is the second mode, is designated, the temporal center-of-gravity images GA and GB are matched to calculate the corresponding reliability value _RT , and the color images CA and CB are combined. And the corresponding reliability value _RC is calculated (# 10, # 11). Then, the reliability values R _T and R _C are compared, and one of the associations is selected for each pixel as described above (# 12). Thereafter, the distance data D is calculated and output as in the case of the color mode (# 4). However, in the corresponding point selection mode, identification data Ds indicating which matching result is the distance data D is added to the distance data D and output.

If the mode is not the corresponding point selection mode in step # 9, that is, if the first mode is designated, the distance data D is calculated and output based on the result of matching between the temporal centroid images GA and GB ( # 13, # 4).

When the color image is not output as the reference information, the order of the mode discrimination in step # 9 and the color photographing in step # 8 is changed so that the color photographing is not performed in the first mode. You may.

FIG. 15 is a diagram showing an example of setting the positional relationship between light projection and light reception. In the arrangement of the light projecting system 10 and the light receiving system 20, the starting point C of light projection and the main point (viewpoint) of light reception are not necessarily required.
It is not necessary to adopt a configuration as shown in FIG. 15A or 15B in which A and B are aligned. For example, when the three points A, B, and C are arranged in an L-shape when viewed from the object side, the configuration of FIG.
The configuration of FIG. 15D arranged in a character shape may be adopted. In particular, if the starting point C is arranged between the viewpoint A and the viewpoint B as shown in FIG. 15B or 15D, occlusion generated due to the difference between the viewpoints A and B and the starting point C can be reduced. it can. In this case, it is preferable to make the distance d between the projection point C and each of the viewpoints A and B equal.

[0054]

According to the first to fifth aspects of the present invention,
A three-dimensional input that does not depend on the projection angle information of the reference light is realized, so that highly accurate three-dimensional input data can be obtained regardless of the accuracy of the projection angle control, and the reliability of the data can be improved. .

According to the fifth aspect of the present invention, the practicality can be improved by diversifying the operation modes.

[Brief description of the drawings]

FIG. 1 is a functional block diagram of a three-dimensional input device according to the present invention.

FIG. 2 is a schematic diagram of projection.

FIG. 3 is an explanatory diagram of a time center of gravity.

FIG. 4 is a diagram for explaining a method of generating a distance image that does not depend on projection angle information.

FIG. 5 is a schematic diagram of matching between time centroid images.

FIG. 6 is a block diagram of a first example of a memory circuit.

FIG. 7 is a block diagram of a second example of the memory circuit.

FIG. 8 is a diagram illustrating a relationship between a luminance distribution on a photographing surface and received light data.

FIG. 9 is a block diagram of a third example of the memory circuit.

FIG. 10 is a conceptual diagram of a center of gravity according to FIG. 9;

FIG. 11 is an explanatory diagram of matching between color images.

FIG. 12 is a diagram illustrating a general example of correlation.

FIG. 13 is a diagram illustrating an example of a reliability value.

FIG. 14 is a flowchart showing a schematic operation of the three-dimensional input device 1.

FIG. 15 is a diagram illustrating a setting example of a positional relationship between light projection and light reception.

[Explanation of symbols]

 Q Object 1 3D input device C Starting point U Slit light (reference light) 10 Projection system A Viewpoint (first position) B Viewpoint (second position) 20A, 20B Imaging system D35 Light reception data (imaging data) CA, CB color image (photographing data) 311 blocks (first matching means) 312 blocks (second matching means) 313 blocks (selection means) TA, TB time center of gravity θA, θB light receiving angle D distance data (position information) Ra reference value Rb Reference value 51 Mode specification part Ds identification data

──────────────────────────────────────────────────続 き Continued on the front page (51) Int.Cl. ⁷ Identification symbol FI Theme coat ゛ (Reference) G06F 15/62 415 F Term (Reference) 2F065 AA04 AA53 BB05 DD06 FF01 FF02 FF05 FF09 GG06 HH05 JJ03 JJ05 JJ26 LL06 LL08 LL08 LL13 LL62 MM16 QQ01 QQ03 QQ24 QQ25 QQ26 QQ27 QQ29 QQ31 QQ36 QQ38 QQ41 2H059 AA07 AA18 5B057 BA02 BA11 BA21 CA13 CB13 CC01

Claims (5)

[Claims] A projection system for projecting reference light from an origin to the object so as to scan an object to be input; a first imaging system for imaging the object at a first position; A second photographing system for photographing the object at a second position distant from the first position, and the object is simultaneously photographed by the first and second photographing systems by projecting the reference light. Pixel positions where light from the same point on the object is incident between the imaging surface of the first imaging system and the imaging surface of the second imaging system based on the imaging data at that time. First matching means, based on photographing data when the object is photographed by the first and second photographing systems without projecting the reference light,
A second matching unit that associates pixel positions where light from the same point on the object is incident between the imaging surface of the first imaging system and the imaging surface of the second imaging system; and Selecting means for selecting one of the correspondence obtained by the first matching means and the correspondence obtained by the second matching means for each pixel on the imaging surface of the first imaging system; A data corresponding to a light receiving angle at each of the first position and the second position determined by the correspondence selected according to the information as position information of the object. 2. The method according to claim 1, wherein said selecting means selects one of the correspondence obtained by said first matching means and the correspondence obtained by said second matching means having a higher reliability. Dimension input device. 3. The method according to claim 2, wherein the reliability of the correspondence obtained by the first matching means is smaller than a first reference value, and the reliability of the correspondence obtained by the second matching means is a second reliability. The three-dimensional input device according to claim 1, wherein the correspondence obtained by the second matching means is selected only when the value is larger than a reference value. 4. An identification data indicating which of the first matching means and the second matching means has obtained the correspondence selected by the selecting means is output in association with the data. The three-dimensional input device according to claim 3. 5. A first mode for outputting data corresponding to a light receiving angle at each of the first and second positions determined by the correspondence obtained by the first matching means, and a mode selected by the selecting means. A second mode for outputting data corresponding to the light receiving angle at each of the first and second positions determined by the corresponding relationship, and the first and second modes determined by the corresponding relationship obtained by the second matching means. 5. A third mode for outputting data corresponding to the light receiving angle at each of the positions is provided, and any one of these is selected as an operation mode. 6. 3D input device.