JP4032556B2 - 3D input device - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a three-dimensional input device that projects reference light onto an object, scans the object, and outputs data for specifying the object shape.
[0002]
[Prior art]
A non-contact type 3D input device called a range finder is capable of high-speed measurement compared to the contact type, so it can be used for data input to CG and CAD systems, body measurement, visual recognition of robots, etc. It's being used.
[0003]
A slit light projection method (also called a light cutting method) is known as a measurement method suitable for a range finder. This method is a method for obtaining a distance image (three-dimensional image) by optically scanning an object, and is a kind of active measurement method for photographing an object by projecting specific reference light. The distance image is a set of pixels indicating the three-dimensional positions of a plurality of parts on the object. In the slit light projection method, slit light in which the cross section of the projection beam is a straight belt is used as reference light. At a certain point during scanning, a part of the object is irradiated, and a bright line bent according to the undulation of the irradiated part appears on the imaging surface. Therefore, a group of data (three-dimensional input data) specifying the object shape can be obtained by sampling the luminance of each pixel on the imaging surface periodically during scanning.
[0004]
Conventionally, based on the position of the bright line in the imaging surface, the incident angle of the slit light reflected by the object and incident on the imaging surface is obtained, and the incident angle, the projection angle of the slit light, and the base line length (projection origin). And the distance from the light receiving reference point), the position of the object is calculated by a triangulation method. That is, position calculation based on the projection direction and the light receiving direction of the reference light has been performed. Even when spot light (the beam cross-section is dotted) is used as the reference light as in the apparatus disclosed in Japanese Patent Laid-Open No. 10-2722, the position of the object is calculated based on the projection direction and the light receiving direction.
[0005]
[Problems to be solved by the invention]
Conventionally, the accuracy of the three-dimensional input data depends on the accuracy of the projection angle control of the reference light. For this reason, the three-dimensional input data with sufficiently high accuracy cannot be obtained, or expensive parts are required to ensure the accuracy. There has been a problem that it has to be used and it takes time and effort to adjust the mounting posture of the light projecting system. The reason why it is difficult to ensure the accuracy is that the light projecting system has a movable mechanism for deflecting the reference light, and its operation is easily influenced by changes in the usage environment such as temperature and humidity.
[0006]
Note that in stereo vision ranging devices that perform pattern light projection, the orientation of object positions viewed from multiple viewpoints can be obtained by matching feature points of multiple images that are epipolar-constrained, and triangulation can be performed based on those orientations. The object position is calculated by the method. In this three-dimensional input method, the accuracy of the three-dimensional input data does not depend on the accuracy of pattern light projection, but depends on the accuracy of matching. Sensitivity variation between pixels of the light receiving device also affects matching.
[0007]
An object of the present invention is to realize three-dimensional input that does not depend on the projection angle information of reference light, and to obtain highly accurate three-dimensional input data regardless of the accuracy of projection angle control. Another object is to reduce the cost of the imaging system.
[0008]
[Means for Solving the Problems]
In the present invention, an object partially illuminated with reference light or ambient light is imaged using two points separated from each other as viewpoints, and the distance between the viewpoints and the direction of the irradiated part on the object viewed from each viewpoint Based on (inclination with respect to a straight line connecting the viewpoints), data obtained by calculating the position of the object with high accuracy by the triangulation method or data for calculation is output. One viewpoint and the starting point of projection are set to the same position in the baseline direction. Thus, one-dimensional imaging may be performed for the viewpoint, and the device can be reduced in price and signal processing can be simplified as compared with the case of performing two-dimensional imaging.
[0009]
The apparatus according to the first aspect of the present invention is directed from the starting point toward the object so as to scan the object to be input. slit A light projection system for projecting light; Along the direction perpendicular to the slit length direction of the slit light An imaging system that simultaneously images the object at first and second positions separated from each other on a virtual base line, and the position of the starting point in the direction along the base line is the same as the second position Yes, based on imaging data corresponding to each of the first and second positions, for each of the plurality of parts in the object, the light reflected by the part at each of the first and second positions This is a three-dimensional input device that detects a light reception angle and outputs data corresponding to the detected set of light reception angles as position information of the part.
[0010]
A three-dimensional input device according to a second aspect of the present invention is configured to capture an image at the first position. The pixel array is along the baseline direction, which is the slit length direction and the direction perpendicular thereto Performed by a two-dimensional imaging device, and imaging at the second position Pixel array is along the baseline direction This is done with a one-dimensional imaging device. We would like to receive an assessment that you should patent after examination.
[0011]
DETAILED DESCRIPTION OF THE INVENTION
FIG. 1 is a functional block diagram of a three-dimensional input device 1 according to the present invention.
The three-dimensional input apparatus 1 includes a light projecting system 10 that projects slit light U, a light receiving system 20 including two imaging systems 20A and 20B, and a total of two light receiving units corresponding to the imaging systems 20A and 20B. Signal processing units 30A and 30B.
[0012]
The light projecting system 10 includes a semiconductor laser 12 as a light source, a lens group 13 for beam shaping, and a galvanometer mirror 14 as beam deflecting means for changing the projection angle. The lens group 13 includes a collimator lens and a cylindrical lens. A deflection control signal is given to the galvanometer mirror 14 from the light projection control circuit 32 via the D / A converter 33.
[0013]
The imaging system 20A includes a light receiving lens 21, a beam splitter 22, an image sensor 24A for obtaining a distance image representing the shape of the object Q, a color image sensor 25 for obtaining a two-dimensional image for monitoring, and zooming and focusing. It consists of a lens drive mechanism 26 that enables it. The beam splitter 22 separates light in the emission wavelength range (for example, the center wavelength of 685 nm) of the semiconductor laser 12 and visible light. The image sensor 24A and the color image sensor 25 are two-dimensional imaging devices (area sensors). A CCD sensor or a MOS sensor can be used as these sensors.
[0014]
The imaging system 20B includes an anamorphic lens 27 that forms an object image by compressing the object image in the length direction of the slit light, a bandpass filter F2 that transmits light in the laser wavelength region, and an image sensor (CCD sensor) for obtaining a distance image. 24B and a lens driving mechanism 26. The image sensor 24B is a one-dimensional imaging device (line sensor), and is arranged so that the pixel arrangement direction of the light receiving surface thereof is the baseline direction.
[0015]
The received light signal processing unit 30A performs processing for the output of the imaging system 20A. The photoelectric conversion signal of each pixel output from the image sensor 24A is converted into light reception data having a predetermined number of bits by the A / D converter 35 and sequentially transferred to the memory circuit 37. The memory circuit 37 detects and stores the “time center of gravity TA” that specifies the first light reception angle θA based on the light reception data. The output of the color image sensor 25 is converted into received light data by the A / D converter 36 and is sequentially stored by the color image memory 38. The memory control circuit 39 is responsible for addressing the memory circuit 37 and the color image memory 38.
[0016]
Processing for the output of the imaging system 20B is performed by the received light signal processing unit 30B. The photoelectric conversion signal of each pixel output from the image sensor 24B is converted into light reception data having a predetermined number of bits by the A / D converter 45 and sequentially transferred to the memory circuit 47. The memory circuit 47 detects and stores “spatial center of gravity jB” for specifying the second light receiving angle θB based on the light receiving data. The memory control circuit 49 is responsible for addressing the memory circuit 47.
[0017]
The CPU 31 that controls the three-dimensional input device 1 gives an instruction to the control target in a timely manner, and performs calculation to read out data from the memory circuits 37 and 47 and obtain distance image data. The distance image data is output to an external device (not shown) as three-dimensional input data in a timely manner. At this time, a two-dimensional color image stored in the color image memory 38 of the received light signal processing unit 30A is also output. Examples of the external device include a computer, a display, and a storage device.
[0018]
FIG. 2 is a schematic diagram of projection.
The three-dimensional input device 1 projects the slit light U so as to scan the virtual surface VS with a point on the reflection surface of the galvano mirror 14 as a starting point C. The virtual plane VS corresponds to a cross section orthogonal to the depth direction of a space (range within the angle of view) that can be imaged by the image sensors 24A and 24B. A range ag corresponding to each pixel in the image sensors 24A and 24B in the virtual surface VS becomes a sampling section for three-dimensional input. As well shown in FIG. 2B, the light projection starting point C is at the same position as the viewpoint B in the baseline direction, which is the arrangement direction of the viewpoints (light receiving main points) A and B. Here, the viewpoints A and B are arranged along the vertical direction, and the slit length direction of the slit light U is the horizontal direction. The shape of the irradiation part of the object Q viewed from the viewpoint A is a belt-like shape bent according to the undulation of the object Q. Therefore, at the viewpoint A, it is necessary to image the object Q by the area sensor. On the other hand, since the starting point C and the viewpoint B are at the same position in the vertical direction, the shape of the irradiated part of the object Q viewed from the viewpoint B is always a straight belt regardless of the projection angle of the slit light U. That is, the light receiving angle of the slit light U in imaging is the same at any position in the horizontal direction. Therefore, at the viewpoint A, the object Q may be imaged by the line sensor.
[0019]
FIG. 3 is a schematic diagram of image formation by an anamorphic lens.
A photographed image GQ of an object formed on the imaging surface S2b of the image sensor 24B by the anamorphic lens 27 is an image obtained by compressing a photographed image GQ ′ with a normal lens having the same vertical and horizontal magnification in the horizontal direction. In the case of a normal lens, when the object surface at the position where the image sensor 24B is stared has a low reflectance, the reflected light of the slit light does not return to the three-dimensional input device 1, and the object is not imaged on the imaging surface S2b. That is, measurement becomes impossible. On the other hand, in the present embodiment, since the length of the slit light is compressed by the anamorphic lens 27, even if a part of the object surface on which the slit light is projected has a low reflectance, the other parts If the reflected light at is returned, an object can be imaged on the imaging surface S2b and measurement is possible.
[0020]
FIG. 4 is an explanatory diagram of the time centroid.
The three-dimensional input device 1 projects a relatively wide slit light U onto the object Q, in which the slit width on the imaging surface S2 of the image sensor 24A is a plurality of pixels g arranged at the pitch pv. Specifically, the width of the slit light U is about 5 pixels. The slit light U is deflected at a constant angular velocity in the vertical direction in the figure with the origin C as the center. The slit light U reflected by the object Q is incident on the imaging surface S2 through the principal point A (corresponding to the above-described viewpoint) of the imaging. While the slit light U is projected, the object Q is scanned by periodically sampling the amount of light received by each pixel g on the imaging surface S2. A photoelectric conversion signal for one frame is output from the image sensor 24A every sampling period (imaging period).
[0021]
If the surface of the object Q is flat and there is no noise due to the characteristics of the optical system, the amount of light received by each pixel g on the imaging surface S2 is the object surface ag within the range (strictly speaking, the center) of the pixel g. Is maximized when the optical axis of the slit light U passes through, and its temporal distribution is close to a normal distribution. In FIG. 4, the j-th pixel g in the vertical direction on the imaging surface S2. _j Is noticed, and the transition of the amount of received light is shown in FIG. In the example of FIG. 4B, the nth sampling time T _n And the previous (n-1) th sampling time T _n-1 The amount of light received is maximum. The point of inflection near the top of the luminance distribution curve, that is, the point at which the received light amount at one pixel g focused as described above becomes the maximum, is defined as the “time centroid”. However, in the detection of the time center of gravity by discriminating the size of discrete sampling data, the detection result (in the example of FIG. _n ) And the actual time centroid, a maximum deviation of ½ of the sampling period occurs.
[0022]
Here, the incident angle of the slit light U with respect to each pixel g is uniquely determined from the positional relationship between the principal point A and each pixel g on the imaging surface S2. Therefore, the time centroid can also be referred to as “a time at which the slit light U enters the principal point A at a specific angle (this is referred to as a light receiving angle θA)”.
[0023]
FIG. 5 is an explanatory diagram of the space center of gravity.
The slit light U projected from the starting point C and reflected by the object Q passes through the imaging main point B and enters the imaging surface S2b of the image sensor 24B.
[0024]
The “space centroid” is an incident angle of the slit light U with respect to the principal point B at each sampling time T in the periodic sampling for scanning the object Q. Since the incident angle with respect to the principal point B corresponds to the position of the optical axis on the imaging surface S2b, the center of gravity of the space is detected by determining which pixel g has the maximum amount of light received at each sampling time T. be able to. In FIG. 5, the nth sampling time T _n FIG. 5B shows the spatial distribution of the amount of received light. In the example of FIG. 5B, the jth pixel g _j And the (j−1) th pixel g _j-1 The amount of received light is maximum at a position between and. Accordingly, the pixel position jB corresponding to the inflection point near the top of the luminance distribution curve on the imaging surface S2b can also be referred to as the spatial center of gravity. However, in the detection of the space center of gravity by discriminating the size of discrete sampling data, the detection result (pixel g in the example of FIG. 5). _j ) And the actual center of gravity of the space, a maximum deviation of ½ of the pixel pitch pv occurs.
[0025]
FIG. 6 is a diagram for explaining how to generate distance image data, and FIG. 7 is a schematic diagram of matching between a temporal centroid image and a spatial centroid image.
The outline of the three-dimensional input of the object Q is as follows.
[0026]
The deflection angle of the galvanometer mirror 14 is controlled in synchronization with the imaging of the frame period by the two image sensors 24A and 24B. At this time, the two image sensors 24A and 24B are driven at the same timing. That is, the object Q is imaged simultaneously from the viewpoints A and B.
[0027]
Based on the output of the image sensor 24A, the time centroid TA is detected for each pixel g on the imaging surface S2, and the time centroid image GA, which is a set of detection results, is recorded in the memory. At this time, a memory for recording individual time centroids TA by a horizontal address i indicating a pixel position in the horizontal direction (main scanning direction) of the imaging surface S2 and a vertical address j indicating a pixel position in the vertical direction (sub-scanning direction). Specify the space. Further, based on the output of the image sensor 24B, the space center of gravity jB is detected for each frame, and the space center of gravity image GB, which is a set of detection results, is recorded in the memory. At this time, a memory space in which each space center of gravity jB is recorded is designated by a frame number T indicating the imaging time. When the one-dimensional image sensor 24B is used, only one pixel of memory is used for imaging one frame, so that the memory capacity is small.
[0028]
The horizontal pixel iA-th and vertical jA-th pixel g in one image sensor 24A _iAjA Paying attention to pixel g _iAjA When the slit light U passes through the point P on the line of sight corresponding to, the output is maximized. At this time, paying attention to the output of the other image sensor 24B, the pixel g corresponding to the line of sight passing through the point P _jB Output is maximized.
[0029]
Here, assuming that the captured image of the imaging system 20A is a reference image, and the epipolar constraint is established in the vertical direction between the reference image and the captured image of the imaging system 20B, the pixel g _iAjA Pixel g for vertical position jA _jB The vertical direction position jB of the pixel g in the spatial center of gravity image GB _iAjA Spatial center of gravity TA detected for _iAjA And the frame number T is the same or the nearest pixel is found. That is, it is possible to perform matching (corresponding point search) for grasping the correspondence between pixels of the temporal centroid image GA and the spatial centroid image GB.
[0030]
The point P is the pixel g of the image sensor 24A. _iAjA Corresponds to pixel g _iAjA Light receiving angle θA determined by the position of _iAjA And the point P exists on a straight line specified by the spatial coordinates of the viewpoint A. Further, since the line of sight of each pixel of the image sensor 24B becomes a plane by passing through the anamorphic lens 27, the point P is the pixel g of the image sensor 24B. _jB Corresponds to pixel g _jB Light reception angle θB determined by the position of _jB And the point P exists on the plane specified by the spatial coordinates of the viewpoint B. An intersection of such a straight line and a plane is a point P. Therefore, the light receiving angle θA _iAjA , ΘB _jB And the distance D in the depth direction between the base line passing through the viewpoints A and B and the point P by applying the principle of triangulation based on the distance L between the viewpoints _iAjA Can be calculated, and the relative positions of the viewpoints A and B and the point P can be specified. If the above processing is performed for each pixel g of the image sensor 24A, a distance image GD that is three-dimensional position information of sampling points for the number of pixels of the object Q can be obtained. The number of frames may be greater than or equal to the number of pixels in the vertical direction of the image sensor 24A.
[0031]
Next, a specific configuration of a circuit for detecting the time centroid TA and the space centroid jB will be described. In the following, except for the case where it is necessary to distinguish the pixel position, a subscript representing the pixel position _iAjA , _jB The description of is omitted.
[0032]
FIG. 8 is a block diagram of a first example of a memory circuit related to the time centroid.
The illustrated memory circuit 37 includes two memories 371 and 376, a comparator 377, and an index generator 378.
[0033]
The light reception data D35 is input from the A / D converter 35 to the memory 371, and the frame number T is input from the index generator 378 to the memory 376. . The comparator 377 compares the received light data D35 of the t-th frame which is the latest input data for each pixel of the image sensor 24A with the received light data D35 previously written in the memory 371, and the latest received light data D35 is the previous data. If the received light data is larger than the received light data D35, writing to the memories 371 and 376 is permitted. In response to this, each of the memories 371 and 376 stores the latest input data in an overwrite format. When the comparison result is opposite, the previous stored contents are held in the memories 371 and 376. Therefore, when the scanning is completed, the memory 371 stores the maximum value of the light reception data D35 for each pixel g, and the memory 376 stores the number T of the frame in which the light reception data D35 is maximum for each pixel. become. Since the imaging of each frame is performed at a constant cycle, the frame number T represents the time during the scanning period (elapsed time from the start of scanning). That is, the frame number T stored in the memory 376 corresponds to the above time center TA.
[0034]
According to this example, the time center of gravity TA can be detected with a relatively simple circuit configuration. However, the detection resolution depends on the imaging cycle. The following second example is intended to improve the resolution.
[0035]
FIG. 9 is a block diagram of a second example of the memory circuit related to the time centroid, and FIG. 10 is a diagram showing the relationship between the luminance distribution on the imaging surface and the received light data. 9, elements corresponding to those in FIG. 8 are denoted by the same reference numerals as in FIG.
[0036]
The memory circuit 37b of the second example is provided with four memories 372, 373, 374, and 375 of the same size in addition to the memory 371, and a total of four 1-frame delay memories 379a to 379 are interposed therebetween. The data input of ˜375 is configured to be delayed one frame at a time with respect to the memory 371. That is, in the memory circuit 37b, five consecutive frames of received light data D35 are simultaneously stored for each pixel g. The comparator 377 compares the input and the output of the third memory 373 whose input is delayed by 2 frames. When the input data value of the memory 373 is larger than the output data value (the previously written data value), writing to the memories 371 to 375 and the memory 376 is permitted.
[0037]
At the end of each scan, the memory 373 stores the maximum value of the received light data D35 for each pixel g. The memories 371, 372, 374, and 375 store the light reception data D35 of a total of four frames, two frames before, one before, one after, and two after the frame in which the light reception data D35 is maximum. become. The memory 376 stores the frame number T in which the light reception data D35 is the maximum for each pixel g.
[0038]
Here, as shown in FIG. 10A, the width of the slit light image formed on the imaging surface is 5 pixels, and the luminance distribution is a single peak. At this time, if attention is paid to one pixel g, the received light data in accordance with the luminance distribution is obtained as shown in FIG. Therefore, by performing the centroid calculation based on the light reception data D35 for five frames stored in the memories 371 to 375, the time centroid TA can be detected in finer steps than the frame period (that is, the pixel pitch). . In the example of FIG. 10B, the time centroid TA is between the t-th sampling time and the (t + 1) -th sampling time. For the center of gravity calculation, there are a configuration performed by the CPU 31 and a configuration provided with an arithmetic circuit.
[0039]
According to the second example, the resolution is improved, but there is a problem that a desired accuracy cannot be obtained depending on the luminance distribution. That is, in actual imaging, some noise is added to the imaging due to the characteristics of the optical system. For this reason, a plurality of peaks occur in the luminance distribution, or the luminance distribution is flat and unclear. If the luminance distribution deviates greatly from the ideal shape, the reliability of the centroid calculation decreases.
[0040]
The effect of such noise is that the calculation of the center of gravity is performed based on the luminance distribution over a sufficiently long period rather than the short period of time when the frame where the maximum luminance value is obtained and the several frames before and after it are combined. Can be reduced. This is achieved by the third example below.
[0041]
FIG. 11 is a block diagram of a third example of the memory circuit related to the time center of gravity, and FIG. 12 is a conceptual diagram of the center of gravity related to FIG.
The memory circuit 37c of the third example includes a memory 3710, a stationary light data storage unit 3720, a subtraction unit 3730, a first addition unit 3740, a second addition unit 3750, and a division unit 3760, and the number of frames for each pixel g. The time centroid TA is calculated based on the received light data D35.
[0042]
The memory 3710 stores light reception data D35 of a predetermined number k frames obtained by scanning the object Q. The received light data value of the Tth (T = 1 to k) frame of each pixel g is expressed as x. _T It expresses. The steady light data storage unit 3720 stores steady light data representing unnecessary incident light amounts other than the slit light U. The steady light data is calculated based on the received light data D35 when the slit light U is not incident. The value s may be a predetermined fixed value or may be obtained in real time using the received light data D35. In the case of a fixed value, when the received light data D35 is 8 bits (256 gradations), for example, “5” “6” or “10” is set. The subtracting unit 3730 receives the value x of the received light data D35 read from the memory 3710. _T The value s of the stationary light data is subtracted from Here, the value of the output data from the subtracting unit 3730 is changed to X _T And The first addition unit 3740 calculates each value X for the k light reception data D35 for each pixel g. _T Is multiplied by the corresponding frame number T, and the total value of the obtained products is output. The second adder 3750 calculates the value X of the k received light data D35 for each pixel g. _T Output the sum of. The division unit 3760 divides the output value of the first addition unit 3740 by the output value of the second addition unit 3750, and outputs the obtained time centroid TA.
[0043]
The spatial center of gravity jB can also be obtained by a circuit having the same configuration as above.
FIG. 13 is a block diagram of a first example of a memory circuit related to the space center of gravity.
The example memory circuit 47 includes two memories 471, 476, a comparator 477, and an index generator 478. Each of the memories 471 and 476 has a memory space equal to or more than the number of scan frames. Each memory space is addressed by a frame number T from the index generator 478.
[0044]
In each frame, received light data D45 is input from the A / D converter 45 to the memory 471, and a vertical address j is input to the memory 476 as data. The comparator 477 compares the received light data D45 of the j-th pixel, which is the latest input data, with the received light data D45 previously written in the memory 471, and the latest received light data D45 is greater than the previous received light data D45. Write to the memories 471 and 476. In response to this, the memories 471 and 476 store the latest input data in an overwrite format. When the comparison result is opposite, the previous stored contents are held in the memories 471 and 476. Accordingly, when the scanning is completed, the memory 471 stores the maximum value of the light reception data D45 in each frame, and the memory 476 stores the vertical pixel position j of the pixel in which the light reception data D45 is maximum in each frame. Become. The vertical pixel position j stored in the memory 476 corresponds to the spatial center of gravity jB described above, and is information for specifying the light receiving angle θB.
[0045]
According to this example, the spatial center of gravity jB can be detected with a relatively simple circuit configuration. However, the resolution of detection of the light receiving angle depends on the pixel pitch of the image sensor 24B. The following second example is intended to improve the resolution.
[0046]
FIG. 14 is a block diagram of a second example of the memory circuit related to the space center of gravity. In the figure, elements corresponding to those in FIG. 13 are denoted by the same reference numerals as those in FIG.
The memory circuit 47b of the second example is provided with four memories 472, 473, 474, and 475 of the same size in addition to the memory 471, and a total of four one-line delay memories 479a to 479 are interposed therebetween. The data input of ˜475 is configured to be sequentially delayed by one line transfer period with respect to the memory 471. That is, in the memory circuit 47b, the received light data D45 for five pixels having a continuous arrangement order is simultaneously stored for each pixel g. The comparator 477 compares the input and output of the third memory 473 whose input is delayed by two lines. When the input data value of the memory 473 is larger than the output data value (the previously written data value), writing to the memories 471 to 475 and the memory 476 is permitted.
[0047]
At the end of each scan, the memory 473 stores the maximum value of the received light data D45 in each frame. In addition, the memories 471, 472, 474, and 475 store the light reception data D45 of a total of four pixels two before, one before, one after, and two after the line where the light reception data D45 is maximum. become. The memory 476 stores the vertical pixel position j at which the light reception data D45 is the maximum in each frame. By performing the centroid calculation based on the received light data D45 for five pixels stored in the memories 471 to 475, the spatial centroid jB can be detected in finer increments than the pixel pitch pv. For the center of gravity calculation, there are a configuration performed by the CPU 31 and a configuration provided with an arithmetic circuit.
[0048]
According to the second example, the resolution is improved, but there is a problem that a desired accuracy cannot be obtained depending on the luminance distribution. In other words, if the luminance distribution deviates greatly from the ideal shape, the reliability of the centroid calculation decreases.
[0049]
The influence of such noise can be reduced by performing the centroid calculation based on the luminance distribution in a sufficiently wide vertical range, not several pixels including the pixel where the maximum luminance value is obtained. This is achieved by the third example below.
[0050]
FIG. 15 is a block diagram of a third example of the memory circuit related to the space center of gravity.
The memory circuit 47c of the third example includes a memory 4710, a stationary light data storage unit 4720, a subtraction unit 4730, a first addition unit 4740, a second addition unit 4750, and a division unit 4760, and a vertical direction pixel for each frame. A space center of gravity jB is calculated based on the received light data D45 for several minutes.
[0051]
The memory 4710 stores received light data D45 of the number of pixels × the number of frames obtained by scanning the object Q. The jth received data value of each frame is represented by x _j It expresses. The steady light data storage unit 4720 stores steady light data representing unnecessary incident light amounts other than the slit light U. The steady light data is calculated based on the received light data D45 when the slit light U is not incident. The value s may be a predetermined fixed value or may be obtained in real time using the received light data D45. The subtracting unit 4730 receives the value x of the received light data D45 read from the memory 4710. _j The value s of the stationary light data is subtracted from Here, the value of the output data from the subtracting unit 4730 is changed to X _j And The first addition unit 4740 obtains each value X for the received light data D45 for the number of frames. _j Is multiplied by the corresponding vertical pixel position j, and the total value of the obtained products is output. The second addition unit 4750 receives the value X of the received light data D45 for the number of frames. _j Output the sum of. The division unit 4760 divides the output value of the first addition unit 4740 by the output value of the second addition unit 4750 and outputs the obtained space center of gravity jB.
[0052]
In the above embodiment, the example in which the distance data is calculated by detecting the time centroid TA and the space centroid jB has been described. However, the present invention is not limited to this. As with the viewpoint A, the distance data can also be calculated for the viewpoint B by detecting the time center of gravity TB.
[0053]
FIG. 16 is a diagram for explaining a second example of how to generate distance image data, and FIG. 17 is a schematic diagram of matching of two time centroid images.
Even when the temporal centroids TA and TB are detected for each of the viewpoints A and B, as in the case where the temporal centroid TA and the spatial centroid jB are detected as described above, the image sensors 24A and 24B can capture the frame period. The deflection angle of the galvanometer mirror 14 is controlled in synchronization. At this time, the two image sensors 24 are driven at the same timing, and the object Q is simultaneously imaged from the viewpoints A and B. Then, it is detected at which point of time the projection of the slit light U that is deflected momentarily by the pixels of the image sensors 24A and 24B.
[0054]
The pixel i in the horizontal direction iA and the vertical direction jA in the two-dimensional image sensor 24A. _iAjA Paying attention to pixel g _iAjA When the slit light U passes through the point P on the line of sight corresponding to, the output is maximized. At this time, paying attention to the output of the one-dimensional image sensor 24B, the pixel g corresponding to the line of sight passing through the point P _jB Output is maximized. Here, when the captured image (temporal centroid image) GA of the image sensor 24A is used as a reference image, the epipolar constraint is established in the vertical direction between the reference image GA and the captured image (temporal centroid image) GB of the image sensor 24B. Then, pixel g _iAjA Pixel g for vertical position jA _jB Of the vertical direction position jB of the pixel g in the time-centroid image GB. _iAjA It can be found by finding a pixel having the maximum output at the same time as the output of Therefore, the time (time center of gravity) TA at which the output of each pixel of the image sensors 24A and 24B becomes maximum. _iAjA , TB _jB Can be used to find a pixel in the time centroid image GB corresponding to each pixel in the time centroid image GA.
[0055]
Point P is pixel g of time centroid image GA _iAjA Corresponds to pixel g _iAjA Light receiving angle θA determined by the position of _iAjA And the point P exists on a straight line specified by the spatial coordinates of the viewpoint A. Further, the point P is a pixel g of the time centroid image GB. _jB Corresponds to pixel g _jB Light reception angle θB determined by the position of _jB And the point P exists on the plane specified by the spatial coordinates of the viewpoint B. That is, the intersection of the straight line and the plane is the point P. Therefore, the light receiving angle θA _iAjA , ΘB _jB And the distance D in the depth direction between the base line passing through the viewpoints A and B and the point P by applying the principle of triangulation based on the base line length L _iAjA Can be calculated, and the relative positions of the viewpoints A and B and the point P can be specified. When the above processing is performed for each pixel g of the time-centroid image GA, three-dimensional position information of sampling points for the object Q corresponding to the number of pixels of the image sensor 24A is obtained.
[0056]
In the above embodiment, the image sensor 24B only needs to be able to detect a one-dimensional light receiving position. Therefore, a position detection type detector (PSD) may be used instead of the CCD sensor.
[0057]
【The invention's effect】
According to the first or second aspect of the invention, three-dimensional input that does not depend on the projection angle information of the reference light is realized, and high-precision three-dimensional input data can be obtained regardless of the accuracy of the projection angle control. At the same time, the amount of data related to position calculation can be reduced. According to the invention of claim 2, it is possible to reduce the price of the imaging system.
[Brief description of the drawings]
FIG. 1 is a functional block diagram of a three-dimensional input device according to the present invention.
FIG. 2 is a schematic diagram of projection.
FIG. 3 is a schematic diagram of image formation by an anamorphic lens.
FIG. 4 is an explanatory diagram of a time centroid.
FIG. 5 is an explanatory diagram of a space center of gravity.
FIG. 6 is a diagram for explaining how to generate distance image data.
FIG. 7 is a schematic diagram of matching between a temporal centroid image and a spatial centroid image.
FIG. 8 is a block diagram of a first example of a memory circuit related to a time centroid.
FIG. 9 is a block diagram of a second example of the memory circuit related to the time centroid.
FIG. 10 is a diagram illustrating a relationship between luminance distribution on the imaging surface and received light data.
FIG. 11 is a block diagram of a third example of the memory circuit related to the time centroid.
12 is a conceptual diagram of the center of gravity according to FIG. 11. FIG.
FIG. 13 is a block diagram of a first example of a memory circuit related to a space center of gravity.
FIG. 14 is a block diagram of a second example of the memory circuit related to the space center of gravity.
FIG. 15 is a block diagram of a third example of the memory circuit related to the space center of gravity.
FIG. 16 is a diagram for explaining a second example of how to generate distance image data.
FIG. 17 is a schematic diagram of matching of two temporal centroid images.
[Explanation of symbols]
1 3D input device
Q object
U slit light (reference light)
10 Flood light system
A Viewpoint (first position)
B Viewpoint (second position)
20 Light receiving system (imaging system)
Point P (part on the object)
TA, TB time center of gravity
jB center of gravity
θA, θB Light receiving angle
D Distance data (position information)
D35 Light reception data (imaging data)
D45 Light reception data (imaging data)

Claims (2)

A light projecting system that projects slit light from the starting point toward the object so as to scan the object to be input, and first spaced apart from each other on a virtual base line along a direction orthogonal to the slit length direction of the slit light And an imaging system that images the object at the same time at a second position,
The position of the starting point in the direction along the base line is the same as the second position;
Based on the imaging data corresponding to each of the first and second positions, for each of the plurality of parts of the object, the light receiving angle at each of the first and second positions of the light reflected by the part. And outputting data corresponding to the detected set of light receiving angles as position information of the part. Imaging at the first position is performed by a two-dimensional imaging device along a baseline direction in which the pixel arrangement is in the slit length direction and a direction orthogonal thereto, and imaging at the second position is in the baseline direction The three-dimensional input device according to claim 1, which is performed by a one-dimensional imaging device along the line .

Images