JP2000088539A - Method and apparatus for three-dimensional inputting - Google Patents

Abstract

PROBLEM TO BE SOLVED: To realize three-dimensional inputting without recourse to angle-of- projection information on a reference light, and to obtain high-precision three- dimensional input data regardless of the precision of angle-of-projection control. SOLUTION: A reference light is projected toward a virtual surface so as to scan it, and the reference light reflected by a substance Q is received simultaneously at a first position A and a second position B separated from each other, and a point of time when the reference light reflected by the substance passes each sampling section being a fractionized virtual surface is detected, and the position of the substance Q is computed in each sampling section, on the basis of an angle of light reception θA of the reference light at the first position A, an angle of light reception θB of the reference light at the second position B, and the distance L between the first and second positions A, B.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a three-dimensional input method and a three-dimensional input device for projecting reference light onto an object to scan the object and obtaining data for specifying the shape of the object.

[0002]

2. Description of the Related Art A non-contact type three-dimensional input device called a range finder can perform higher-speed measurement than a contact type, and therefore can input data to a CG system or a CAD system, measure a body, and use a robot. It is used for visual recognition.

[0003] As a measurement method suitable for a range finder, a slit light projection method (also called a light cutting method) is known. In this method, an object is optically scanned and a distance image (3
Dimensional image), which is a type of active measurement method for projecting a specific reference beam to photograph an object. The distance image is a set of pixels indicating three-dimensional positions of a plurality of parts on the object. In the slit light projection method, slit light in which a cross section of a projection beam has a linear band shape is used as reference light. At a certain point during the scanning, a part of the object is irradiated, and a bright line that is curved according to the undulation of the irradiated part appears on the imaging surface. Accordingly, a group of data (three-dimensional input data) for specifying the object shape can be obtained by periodically sampling the luminance of each pixel on the imaging surface during scanning.

Conventionally, the incident angle of slit light reflected by an object and incident on the imaging surface is determined based on the position of the bright line in the imaging surface, and the incident angle, the projection angle of the slit light, and the base line length ( The position of the object is calculated from the distance between the projection start point and the light receiving reference point) by triangulation. That is, the position calculation based on the projection direction and the light receiving direction of the reference light has been performed. It should be noted that the position of the object is calculated based on the projection direction and the light receiving direction even when spot light (beam section is point-shaped) is used as the reference light as in the device of JP-A-10-2722.

[0005]

Conventionally, the accuracy of the three-dimensional input data depends on the accuracy of the control of the projection angle of the reference light, so that sufficiently high accuracy of the three-dimensional input data cannot be obtained. There have been problems that expensive components must be used to ensure accuracy, and that it takes time to adjust the mounting posture of the light projecting system. The reason why it is difficult to ensure accuracy is that the light projection system has a movable mechanism for deflecting the reference light, and its operation is easily affected by changes in the use environment such as temperature and humidity.

In a stereo vision ranging apparatus that performs pattern light projection, the orientations of object positions as viewed from a plurality of viewpoints are obtained by matching feature points of a plurality of epipolar-constrained images, and based on those orientations. The object position is calculated by a triangulation method. In this three-dimensional input method, the accuracy of three-dimensional input data does not depend on the accuracy of pattern light projection, but depends on the accuracy of matching. Variations in sensitivity between pixels of the light receiving device also affect matching.

An object of the present invention is to realize three-dimensional input that does not depend on projection angle information of reference light, and to obtain highly accurate three-dimensional input data regardless of the accuracy of projection angle control.

[0008]

According to the present invention, an object partially illuminated by reference light is imaged at two points apart from each other as viewpoints, and the distance between the viewpoints and the object viewed from each viewpoint are taken. The position of the irradiated part is calculated from the azimuth of the upper irradiated part (inclination with respect to the straight line connecting the viewpoints) by a triangulation method.

According to a first aspect of the present invention, a reference light is projected so as to scan the virtual surface toward a virtual surface, and the reference light reflected by an object is simultaneously reflected at first and second positions separated from each other. Receiving and detecting a point in time at which the reference light reflected by the object passes through each sampling section obtained by subdividing the virtual plane, and detecting the reference light at each of the first and second positions at each detected time. A three-dimensional input method for calculating a position of the object for each of the sampling sections based on a light receiving angle and a distance between the first and second positions.

According to a second aspect of the present invention, there is provided an apparatus for projecting a reference light so as to scan the virtual surface toward a virtual surface, and a first and a second system which separate the reference light reflected by an object from each other. An imaging system that receives light simultaneously at a position and converts it into an electric signal, and based on the electric signal corresponding to each position, determines when the reference light reflected by an object passes through each sampling section obtained by subdividing the virtual plane. Signal processing means for detecting,
A three-dimensional input device that outputs data corresponding to the light receiving angle of the reference light at each of the first and second positions at each time point detected by the signal processing unit as position information of a plurality of parts of the object. It is.

In the three-dimensional input device according to the present invention, the starting point of the projection of the reference light is a position between the first position and the second position. 5. The three-dimensional input device according to claim 4, wherein a starting point of the projection of the reference light is a position equidistant from both the first and second positions.

[0012]

FIG. 1 is a functional block diagram of a three-dimensional input device 1 according to a first embodiment. 3D input device 1
Has a light projecting system 10 that projects slit light U, a light receiving system 20 including two imaging systems 20A and 20B having the same configuration, and two light receiving signal processing units 30A and 30B having the same configuration. I have.

The light projecting system 10 includes a semiconductor laser 12 as a light source, a lens group 13 for beam shaping, and a galvanomirror 14 as beam deflecting means for changing a projection angle.
Consists of The lens group 13 includes a collimator lens and a cylindrical lens. The galvanomirror 14 is provided with a deflection control signal from the light projection control circuit 32 via the D / A converter 33.

Each of the imaging systems 20A and 20B includes a light receiving lens 2
1, a beam splitter 22, an image sensor 24 for obtaining a range image representing the shape of the object Q, a color image sensor 25 for obtaining a two-dimensional image for monitoring, and a lens driving mechanism 26 for enabling zooming and focusing Consists of The beam splitter 22 separates light in the emission wavelength range (for example, center wavelength 670 nm) of the semiconductor laser 12 from visible light. The image sensor 24 and the color image sensor 25 are CCD area sensors. However, the CMOS area sensor is replaced with the color image sensor 25.
You may use as. The output of the image sensor 24 is A
The data is converted into light receiving data of a predetermined number of bits by the / D converter 35 and sequentially transferred to the memory circuit 37. Memory circuit 3
7, a light receiving angle θA, θ described later according to the value of the light receiving data.
Data (TA, TB) specifying B is stored. The output of the color image sensor 25 is converted into light receiving data by an A / D converter 36, and is sequentially stored by a color image memory 38. Memory circuit 37 and color image memory 3
The address designation of 8 is performed by the memory control circuit 39.

CPU 31 for controlling the three-dimensional input device 1
Gives an instruction to the control target in a timely manner, and performs an operation of reading data from the memory circuit 37 of each of the light receiving signal processing units 30A and 30B to obtain distance image data. The distance image data is output as appropriate three-dimensional input data to an external device (not shown). At this time, the light receiving signal processing sections 30A, 3
The two-dimensional color image stored in at least one of the color image memories 38 is also output. The external device includes a computer, a display, a storage device, and the like.

FIG. 2 is a schematic diagram of the projection, FIG. 3 is a diagram for explaining the generation procedure of the distance image data, and FIG. 4 is a diagram showing parallax in imaging. The three-dimensional input device 1 projects the slit light U so as to scan the virtual surface VS with a point on the reflection surface of the galvanometer mirror 14 as a starting point C. The virtual plane VS is
Space that can be imaged by image sensor 24 (range within angle of view)
Corresponds to a cross section orthogonal to the depth direction. Then, a range of the virtual plane VS corresponding to each pixel g in the image sensor 24 is a three-dimensional input sampling section. In FIG. 2, a starting point C of light projection and viewpoints (principal points of light reception) A and B are arranged on a straight line. Here, the viewpoints A and B are arranged along the vertical direction, and the slit length direction of the slit light U is the horizontal direction.

The outline of the three-dimensional input of the object Q is as follows. The deflection angle of the galvanomirror 14 is controlled in synchronization with the imaging of the frame period by the two image sensors 24. At this time, the two image sensors 24 are driven at the same timing. That is, the object Q is imaged simultaneously from the viewpoints A and B. Then, each pixel of each image sensor 24 is detected at which point in time the projection of the slit light U, which is deflected every moment, is illuminated.

The image sensor 24 of one image pickup system 20A
Pixel g in the iA-th horizontal direction and the jA-th vertical direction in
_iAjANote that pixel g_iAjAPoint P on the line of sight corresponding to
When the slit light U passes through, the output becomes maximum.
At this time, the image sensor 24 of the other imaging system 20B
Focusing on the output, the pixel g corresponding to the line of sight passing through the point P
_iBjBOutput becomes maximum. Here, imaging is performed by the imaging system 20A.
And a second image captured by the imaging system 20B.
Has an epipolar constraint in the vertical direction
Then the pixel g_iAjAFor the horizontal position iA of
Prime g_iBjBIs determined uniquely in the horizontal direction iB. Also,
Prime g_iAjAG for the vertical position jA of_iBjBVertical
The direction position jB is a pixel at the horizontal position iB in the second image.
Pixel g in the column_iAjAAt the same time as the output of
You can tell if you find the pixel that has the maximum output every moment. But
Therefore, each of the image sensors 24 of the imaging systems 20A and 20B
Time (luminance peak time) TA at which the output of the pixel becomes maximum
_iAjA, TB_iBjBIs known, each pixel in the first image is
A corresponding pixel in the second image can be found.

The point when P corresponds to the pixel g _IAjA of the first image, in the presence of a point P on the line specified by the spatial coordinates of the light receiving angle .theta.A _IAjA and the viewpoint A determined by the position of the pixel g _IAjA Become. Similarly, when the point P corresponds to the pixel g _iBjB of the second image, the light receiving angle θB determined by the position of the pixel g _iBjB
The point P exists on a straight line specified by _iBjB and the spatial coordinates of the viewpoint B. That is, the point of intersection of these two straight lines is point P. Therefore, the light receiving angle θA _iBjB ,
Based on θB _iBjB and the distance L between viewpoints, the principle of triangulation is applied to calculate the depth _DiAjA between the base line passing through the viewpoints A and B and the point P, and the viewpoints A and B can be calculated. The relative position with respect to the point P can be specified. Then, if the above processing is performed for each pixel g of the first image, three-dimensional position information of the sampling points of the object Q for the number of pixels can be obtained.

Next, a specific configuration of a circuit for detecting a luminance peak time will be described. Incidentally, unless the following necessary to distinguish the pixel position in which the index represents the pixel position _IAjA, omitted the description of _{iB jB.}

FIG. 5 is a block diagram of a first example of the memory circuit. The example memory circuit 37 includes two memories 371,
376, a comparator 377, and an index generator 378.

The memory 371 receives the light receiving data D35 from the A / D converter 35, and the memory 376 receives the frame number T from the index generator 378. . The comparator 377 compares, for each pixel of the image sensor 24, the received light data D35 of the t-th frame, which is the latest input data, with the received light data D35 previously written in the memory 371. Memory 371, 376
Write is permitted for. In response to this, each memory 37
1, 376 overwrites the latest input data. If the comparison result is reversed, the previous storage contents are held in the memories 371 and 376. Therefore, when the scanning is completed, the memory 371 stores the light receiving data D for each pixel g.
The maximum value of 35 is stored, and the memory 376 stores the number T of the frame having the maximum light receiving data D35 for each pixel. Since the imaging of each frame is performed at a fixed period, the frame number T indicates a time during the scanning period (elapsed time from the start of scanning). That is, the frame number T stored in the memory 376 corresponds to the above-mentioned times TA _iAjA and TB _iBjB , and is information for specifying the correspondence _between the light receiving angles θA _iAjA and θB _iBjB .

According to this example, the light receiving angles θA _iAjA and θB _{iB jB} can be detected with a relatively simple circuit configuration. However, the resolution for detecting the light receiving angle depends on the pixel pitch of the image sensor 24. The second example in which the resolution is improved is as follows.

FIG. 6 is a block diagram of a second example of the memory circuit.
FIG. 7 is a diagram showing the relationship between the luminance distribution on the imaging surface and the received light data. 6, the elements corresponding to FIG. 5 are denoted by the same reference numerals as in FIG.

The memory circuit 37b of the second example includes a memory 37
1 plus 4 2 memories 37 of the same size
2, 373, 374, and 375, and a total of four 1-frame delay memories 379a to 379d intervene in each memory 3.
The configuration is such that the data input of 72 to 375 is delayed by one frame to the memory 371 in order. That is, in the memory circuit 37b, five consecutive frames of received light data D35 are simultaneously stored for each pixel g. The comparator 377 compares the input and the output of the third memory 373 whose input is delayed by two frames. Memory 37
If the input data value of No. 3 is larger than the output data value (the previously written data value), writing to the memories 372 to 375 and the memory 376 is permitted.

At the end of each scan, the memory 3
Reference numeral 73 stores the maximum value of the light receiving data D35 for each pixel g. Also, the memories 371, 372, 374,
With 375, the received light data D35 of two frames before, one before, one after, and two after the frame where the received light data D35 is maximum are stored. And
The memory 376 stores the number T of the frame in which the light receiving data D35 is maximum for each pixel g.

Here, as shown in FIG. 7A, it is assumed that the width of the slit light image formed on the imaging surface is 5 pixels and the luminance distribution is a single-peak mountain-like shape. At this time, paying attention to one pixel g, light reception data of a change according to the luminance distribution is obtained as shown in FIG. 7B. Therefore, the memory 37
The center of gravity is calculated based on the light receiving data D35 for five frames stored in the memory cells 1 to 375, so that the time T can be calculated in finer increments than the frame period (that is, the pixel pitch).
A and TB can be calculated. In the example of FIG. 7B, the time TA (TB) is between the t-th sampling time and the (t + 1) -th sampling time.

According to the second example, the resolution is improved.
There is a problem that desired accuracy cannot be obtained depending on the luminance distribution. That is, in actual imaging, some noise is added to the image formation due to characteristics of the optical system and the like. For this reason,
A plurality of peaks are generated in the luminance distribution, or the luminance distribution is flat and the peaks are unclear. If the luminance distribution deviates greatly from the ideal shape, the reliability of the center-of-gravity calculation decreases.

The influence of such noise is caused by the calculation of the center of gravity based on the luminance distribution in a sufficiently long period, not a short period in which the frame in which the maximum value of the luminance is obtained and the several frames before and after the frame. By doing so, it can be reduced. This is realized in the following third example.

FIG. 8 is a block diagram of a third example of the memory circuit.
FIG. 9 is a conceptual diagram of the center of gravity according to FIG. The memory circuit 37c of the third example includes a memory 3710, a stationary light data storage unit 3,
720, a subtractor 3730, a first adder 3740, a second adder 3750, and a divider 3760, and calculates a centroid (time centroid) for each pixel g based on the light receiving data D35 for the number of frames.

The memory 3710 stores received light data D35 of a predetermined number k of frames obtained by scanning the object Q. The light receiving data values of a frame of T th pixel g (T = 1 to k) is expressed as x _T. Stationary light data storage unit 3720
Stores steady light data representing the amount of unnecessary incident light other than the slit light U. The steady light data is calculated based on the received light data D35 when the slit light U is not incident.
The value s may be a predetermined fixed value, or may be obtained in real time using the received light data D35. In the case of a fixed value, for example, “5”, “6”, or “10” when the light reception data D35 is 8 bits (256 gradations). Subtraction unit 3730 subtracts the value s of the stationary light data from the value x _T of the light reception data D35 read from the memory 3710. Here, the value of the output data from the subtraction unit 3730 is set to X _T again. First adder 3740
, For the k received light data D35 for each pixel g, performs multiplication of each value X _T and the frame number T corresponding thereto, and outputs the sum of the resulting product. Second adder 3
750 is a value X _T of k light receiving data D35 for each pixel g.
Output the sum of The division unit 3760 includes the first addition unit 37
The output value of 40 is divided by the output value of second adder 3750, and the obtained center of gravity is output as time TA (or TB).

FIG. 10 is a diagram showing an example of setting the positional relationship between light emission and light reception. In the arrangement of the light projecting system 10 and the light receiving system 20, the starting point C of light projection and the main point (viewpoint) of light reception are not necessarily required.
It is not necessary to adopt a configuration as shown in FIG. 10A or 10B in which C is aligned. For example, the configuration shown in FIG. 10C in which three points A, B, and C are arranged in an L shape when viewed from the object side, or the configuration shown in FIG. 10D in which the three points A, B, and C are arranged in a T shape may be adopted. In particular, if the origin C is arranged between the viewpoint A and the viewpoint B as shown in FIG. 10B or 10D, occlusion caused by the difference between the viewpoints A and B and the origin C can be reduced. it can. In this case, the distance d between the starting point C of the light projection and each of the viewpoints A and B
Are preferably equal.

FIG. 11 is a functional block diagram of the three-dimensional input device 2 according to the second embodiment. In FIG. 11 and the following drawings, the functions of the components denoted by the same reference numerals as those in FIG. 1 are the same as those of the above-described three-dimensional input device 1.

In the second embodiment, spot light V having a beam cross section is projected instead of slit light, and imaging is performed using a one-dimensional image sensor (line sensor) 27.

The three-dimensional input device 2 includes a light projecting system 10b, a light receiving system 20b including two imaging systems 20Ab and 20Bb having the same configuration, and two light receiving signal processing units 30A having the same configuration.
b, 30Bb. The light projecting system 10b includes a semiconductor laser 12, a collimator lens 13b, and a galvanomirror 14b.

Each of the imaging systems 20Ab and 20Bb includes a light receiving lens 21, a lens driving mechanism 26, an infrared cut filter F
1. Bandpass filter F2, filter switching mechanism 2
8, and an image sensor 27 that is used for both three-dimensional input and monitor shooting. The image sensor 27 is a three-line CCD sensor having a pixel row corresponding to each color of RGB. In case of monitor shooting, infrared cut filter F1
Is used. In the case of three-dimensional input, a bandpass filter F2 that transmits light in a laser wavelength range is used, and only the output of the R pixel row of the image sensor 27 is used as light reception information.

In the case of using the spot light V, the time TA, T
By detecting B, the position of point P on object Q can be calculated. Since the principle is the same as that of the case using the slit light U, the description is omitted here.

FIG. 12 shows an input of the entire circumference by rotation [FIG.
(A)], a diagram showing an example of an apparatus configuration for realizing omnidirectional input (FIG. 12 (b)). FIGS. 13 and 14 realize omnidirectional input (FIG. 13) and omnidirectional input (FIG. 14) using mirrors. FIG. 3 is a diagram showing an example of a device configuration for performing the above.

The three-dimensional input device 3 shown in FIG. 12A includes an optical system 40 for projecting light and capturing an image, and a turntable 45 for rotating with an object Q placed thereon. The optical system 40 is configured to perform three-dimensional input and color imaging with one image sensor by switching filters. When the turntable 45 scans the object Q from a certain direction to find the center of gravity, the turntable 45 rotates by a predetermined angle. By repeating the scanning by the optical system 40 and the rotation of the turntable 45 N times, it is possible to perform a three-dimensional input over the maximum 360 ° angle range of the outer peripheral surface of the object Q. The optical system 40 is provided with a memory capable of storing data N times. Since the direction component is obtained in the form of n × θ because of the data of the scan of the third time, three-dimensional position data in the space of the object to be measured can be obtained. It should be noted that the object Q may be stationary and the optical system 40 may rotate around the object Q.

In the three-dimensional input device 4 shown in FIG. 12B, an optical system 41 is mounted on a turntable 46. The use of the three-dimensional input device 4 enables omnidirectional three-dimensional input of the inner wall surface of a hollow object.

FIG. 13 shows convex curved mirrors 211 and 21.
2 shows an example of the configuration of an omnidirectional three-dimensional input device 5 that uses an input device 2.
The curved mirrors 211 and 212 are arranged concentrically with their reflecting surfaces facing downward, and the curved mirrors 211 and 212 are arranged concentrically.
Imaging device (for example, video camera) 2 one by one below 2
0Ae and 20Be are arranged such that their optical axes coincide with the mirror axes. Each of the curved mirrors 211 and 212 has, for example, a hyperboloid, and has a shape in which a cross section of a plane including an axis and having an axis is a curve that monotonically increases its inclination.
Thereby, the imaging devices 20Ae, 20Be and the light projecting system 10
Except for the range in which the optical system itself including e is shown, the image around the axis in the range of E1 and E2 in the figure is taken by the imaging device 2.
It is taken into 0Ae and 20Be.

The light projecting system 10e includes curved mirrors 211 and 21.
And a light source 12, a lens 13e, a scanning mirror 15, and a mirror rotating mechanism 18. Light source 1
The light emitted from 2 is adjusted to a beam having an appropriate diameter by the lens 13e, and is reflected and projected by the scanning mirror 15. The scanning mirror 15 can control the angle around an axis perpendicular to one axis of the curved mirror (in the direction M2 in the figure), and deflects the beam within the angle range E. This is referred to as sub-scanning. For example, if a scanning mirror 15 having an angle control mechanism such as a galvano scanner is used, this sub-scanning can be realized.

The mirror rotating mechanism 18 rotates the scanning mirror 15 during sub-scanning alone or together with at least one of the lens 13e and the light source 12 around the curved mirror axis (in the direction M1 in the figure). Thereby, the beam scans all around the curved mirror axis, that is, around the optical axis of the imaging device. This is called main scanning.

The angle of the scanning mirror 15 is changed by an amount corresponding to the resolution in the sub-scanning direction M2 for one cycle of the main scanning (one rotation in the direction M1). If this is repeated in the one-way sub-scanning period in the angle range E, the beam can scan the entire range around the curved mirror axis in the angle range E.

One cycle of the main scanning is performed by the imaging devices 20Ae and 2Ae.
The exposure time is set to be equal to or shorter than 0Be exposure time. The trajectory of reflected light of the beam projected in all directions at the beam projection angle, that is, the deflection angle φ of the scanning mirror 15 in the sub-scanning direction M2 can be imaged. Scanning mirror 1 every time an image is taken
5 is changed by the resolution in the sub-scanning direction M2.
By repeating this operation, the barycenter image A (a set of pixels indicating the time TA) as the above-described first image is created, and the center of gravity image B (a set of pixels indicating the time TB) is created as the second image. .

In these two time centroid images A and B, epipolar constraints are established in a radial direction from the image center position (the position corresponding to the curved mirror axis). With reference to the center-of-gravity image A, a pixel of the center-of-gravity image B corresponding to each pixel g in the center-of-gravity image A is obtained. Positional relationship between curved mirrors 211 and 212 and imaging devices 20Ae and 20Be, curved mirror 21
The line of sight (straight line formula) for each pixel g is obtained from the shape definition formulas 1, 212. By finding the intersection of the line of sight of the corresponding pixels of the centroid images A and B, the position of the object Q in the vertical direction and the depth direction is obtained. Further, the azimuth can be obtained from each pixel g of the center-of-gravity image A and the image center position. Therefore, it is possible to calculate a three-dimensional position in space with respect to each pixel g in the center-of-gravity image A.

FIG. 14 shows a curved mirror 220 having an inverted truncated cone shape.
2 shows an example of the configuration of an omnidirectional three-dimensional input device 6 that uses the same. The curved mirror 220 whose inner surface is a reflection surface is arranged with its axis oriented in the vertical direction. The mirror shape can be expressed by the following equation, with the coordinate axis h taken vertically upward on the mirror axis, the mirror cross-sectional radius with respect to a certain height h as r, and the angle of the mirror surface with respect to the h-axis as θm.

R = h · tan θm + R where R is the value of r for h = 0. An object to be input (a human body in the illustrated example) Q inside the curved mirror 220
Is located. Above the curved mirror 220, the imaging devices 20Af, 20Bf and the light projecting system 10f are arranged so that their optical axes coincide with the mirror axes. Apex angle of inverted truncated cone in curved mirror 220, imaging device 20
By appropriately setting the distance to Af and 20Bf and the angle of view of imaging, the imaging devices 20Af and 20Bf can use the curved mirror 220 to convert the image of the entire circumference of the object Q excluding the vicinity of the vertex into one image. Can be caught inside. The light projecting system 10f can rotate the projection direction of the light beam about the mirror axis in the direction M1. This is called main scanning. At the same time, it is assumed that the projection angle θ of the light beam can be scanned in the direction M2 in the vertical plane. This is referred to as sub-scanning. The scanning in the direction M1 can be realized by rotating a part or all of the light projecting system 10f using a power source such as a motor. The change in the direction M2 can be easily achieved by using a scanning device such as a galvano scanner. In addition, the light projecting system 10f includes the imaging device 20A.
In the field of view of f, 20Bf, the size, shape, and vertical installation position are determined so that the mirror does not exist and does not participate in the input of three-dimensional information. .

The image of the point P on the object Q is
Is assumed to be observed as a point PAi at an image height r _PA from the image center position in the image captured at. Since the three-dimensional input device 6 is axially symmetric, for simplicity, the azimuth will be fixed and the following description will focus on a certain vertical section including the mirror axis.

The point P exists on a straight line x 'expressed by the following equation. r = h tan [atan (r _PA / H) + 2θm] + Rhx [H
tan _{[atan (r PA / H) +} 2θm ] + r _PA] / (H tanθ
m + r _PA ) Here, the principal point position of the lens system of the imaging device 20Af is h = hx = 0, the distance between the principal point and the imaging plane is H, h = hx
Is set to Rhx.

The image of the point P on the object Q is
It is assumed that an image captured at 0Bf is observed as a point PBi at an image height r _PB from the image center position. At this time, the point P exists on a straight line y 'expressed by the following equation. r = h tan [atan (r _PB / H) + 2θm] + Rhy [H
tan _{[atan (r PB / H) +} 2θm ] + r _PB] / (H tanθ
m + r _PB ) Here, the principal point position of the lens system of the imaging device 20Af is h = hx = 0, the distance between the principal point of the lens system of the imaging device 20Bf and the imaging surface is H, and the value of r for h = hy. To Rhy
And

From these facts, the position of the point P (point Pi in the image) (position in the depth direction: r, position in the vertical direction: h) is determined as the intersection of the straight line x 'and the straight line y'. Therefore, the position of the point P is determined by the imaging device 20A.
f, 20Bf, it is possible to calculate from the points PAi, PBi observed. Further, since the azimuth can be determined from the point PAi and the image center position, it is possible to calculate the three-dimensional position in space with respect to each pixel on the imaging surface.

The process of three-dimensional input will be described, although it will be partially repeated. One cycle of main scanning is performed by the imaging devices 20Af and 20Af.
The exposure time is set to be equal to or shorter than the exposure time of Bf. In one main scan, the trajectory of reflected light of the beam projected in all directions at the beam projection angle, that is, the deflection angle θ of the scanning mirror 15 can be imaged. Each time an image is taken, the argument θ is changed by the resolution in the sub-scanning direction M2. By repeating this operation, the center-of-gravity image A as the above-described first image is created using the imaging device 20Af, and at the same time, the center-of-gravity image B as the above-described second image is created using the imaging device 20Bf.

In these two center-of-gravity images A and B, an epipolar constraint is established in a radial direction with the image center position as a center. With reference to the center-of-gravity image A, a pixel in the center-of-gravity image B corresponding to each pixel in the center-of-gravity image A is determined (in a direction in which the epipolar constraint is satisfied). From the positional relationship between the curved mirror 220 and the imaging devices 20Af and 20Bf, and the shape definition formula of the curved mirror 220, the line of sight (straight line formula) for each pixel is obtained. By obtaining the intersection of the line of sight of the corresponding pixels of the centroid images A and B, the positions in the vertical direction and the depth direction are obtained. Further, the azimuth can be obtained from each pixel of the center-of-gravity image A and the image center position. Therefore, the center of gravity image A
It is possible to calculate a three-dimensional position in space for each pixel of.

[0055]

According to the first to fourth aspects of the present invention,
Realize three-dimensional input that does not depend on projection angle information of reference light,
Highly accurate three-dimensional input data can be obtained regardless of the accuracy of the projection angle control.

[Brief description of the drawings]

FIG. 1 is a functional block diagram of a three-dimensional input device according to a first embodiment.

FIG. 2 is a schematic diagram of projection.

FIG. 3 is a diagram for explaining a point of generation of distance image data.

FIG. 4 is a diagram illustrating parallax in imaging.

FIG. 5 is a block diagram of a first example of a memory circuit.

FIG. 6 is a block diagram of a second example of the memory circuit.

FIG. 7 is a diagram illustrating a relationship between a luminance distribution on an imaging surface and received light data.

FIG. 8 is a block diagram of a third example of the memory circuit.

9 is a conceptual diagram of the center of gravity according to FIG.

FIG. 10 is a diagram illustrating a setting example of a positional relationship between light projection and light reception.

FIG. 11 is a functional block diagram of a three-dimensional input device 2 according to a second embodiment.

FIG. 12 is a diagram illustrating an example of a device configuration that realizes omnidirectional input or omnidirectional input by rotation.

FIG. 13 is a diagram showing an example of an apparatus configuration for realizing omnidirectional input by a mirror.

FIG. 14 is a diagram illustrating an example of a device configuration that realizes omnidirectional input by a mirror.

[Explanation of symbols]

 1-6 3D input device A viewpoint (first position) B viewpoint (second position) P point (specific part on object) U, V reference light L distance (distance between viewpoints) VS virtual plane TA, TB time (time when passing through the sampling section) θA, θB Light receiving angle 10 Light projecting system 20 Imaging system 37 Memory circuit (signal processing means)

 ────────────────────────────────────────────────── ─── Continued on the front page (72) Inventor Hiroshi Uchino 2-3-13 Azuchicho, Chuo-ku, Osaka-shi, Osaka F-term (reference) 2F065 AA04 AA17 AA53 FF01 FF02 FF04 FF09 GG06 HH04 HH05 JJ02 JJ03 JJ05 JJ25 JJ26 LL00 LL08 LL13 LL19 LL22 LL26 LL62 MM04 MM16 PP13 QQ00 QQ01 QQ03 QQ17 QQ23 QQ24 QQ25 QQ26 QQ27 QQ29 QQ31

Claims (4)

[Claims] 1. A reference light is projected so as to scan it toward a virtual surface, and the reference light reflected by an object is simultaneously received at first and second positions separated from each other, and reflected by an object. Detecting a point in time at which the reference light passes through each of the sampling sections obtained by subdividing the virtual plane; and detecting a reception angle of the reference light at each of the first and second positions at each of the detected times; And calculating a position of the object for each of the sampling sections based on the distance between the second position and the second position. 2. A light projecting system for projecting a reference light so as to scan the virtual surface toward a virtual surface, and a first and a second light sources which separate the reference light reflected by an object from each other.
An imaging system that receives light at the same time and converts it into an electric signal, and based on the electric signal corresponding to each position, a point in time at which the reference light reflected by the object passes through each sampling section obtained by subdividing the virtual plane. Signal processing means for detecting the position of the reference light at each of the first and second positions at each time point detected by the signal processing means. A three-dimensional input device for outputting as position information. 3. The three-dimensional input device according to claim 2, wherein a starting point of the projection of the reference light is a position between the first position and the second position. 4. A projection starting point of said reference light is located at a position equidistant from both said first and second positions.
3. The three-dimensional input device according to claim 1.