JP2012225807A - Distance image camera and distance image synthesis method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a distance image camera and a distance image synthesis method, capable of acquisition of good distance accuracy substantially across the entirety of a distance image regardless of a distance to one or more object or the surface reflectance thereof.SOLUTION: A distance image camera includes: an emission unit 11; an imaging unit 12 for obtaining a distance image having, for each pixel P, distance information calculated based on a measurement value of a time from emission of light to return of the reflected light; an exposure adjustment unit 13; an arithmetic control unit 15 for calculating weighted averages of distance information of respective pixels corresponding to the same pixel positions in a plurality of distance images captured while changing stepwise the exposure, respectively, and acquiring a synthesized distance image produced by synthesizing the calculated weighted averages to have the distance information of each pixel. Each pixel also has reception light level information indicating reception intensity of the reflected light, and in calculating the weighted average of the distance information of each pixel, a weighting factor calculated so as to correspond to the accuracy of the distance information in accordance with the reception light level information of the pixel is used.

Description

  The present invention relates to a range image camera and a range image synthesis method, and more particularly to a range image camera and a range image synthesis method capable of obtaining good range accuracy over almost the entire range image.

  Conventionally, a device capable of acquiring a distance image having distance information for each pixel has been proposed (see, for example, Patent Document 1).

  The rangefinder device described in Patent Document 1 is a rangefinder device that receives reflected light of light projected on a subject and measures the three-dimensional position information of the subject, and includes a light source unit that projects the light, At least one of the light output of the light source unit and the exposure condition of the camera unit based on distance information of the subject and the camera unit that receives the reflected light of the projection light from the light source unit on the subject And a control unit for controlling the above.

JP 2000-241131 A

  However, since the intensity of the reflected light of the projected light greatly varies depending on the distance to the object, the reflectance of the object surface, and the like, it has been difficult to achieve proper exposure when acquiring a distance image. This is because no matter how the intensity of the projection light, the exposure setting, etc. are determined, the distance accuracy cannot always be maintained well over the entire distance image.

  For example, if the shutter speed is slowed down in order to increase the amount of exposure, the reflected light from an object at a short distance or an object with a high surface reflectance becomes strong, and the received light level signal tends to be saturated. Conversely, if the shutter speed is increased in order to reduce the exposure amount, the reflected light from an object at a long distance or an object with low surface reflectance becomes weak and the S / N ratio of the received light level signal deteriorates. In any case, the distance accuracy will deteriorate.

  Even if the intensity of the projected light is changed instead of changing the shutter speed to increase or decrease the exposure amount, the received light level signal is likely to saturate in an object with a short distance or high surface reflectance, Since the S / N ratio of the light reception level signal may be deteriorated with an object having a low distance or surface reflectance, the distance accuracy is also deteriorated.

  In view of such a problem of the prior art, the object of the present invention is to provide a good distance over almost the entire range image regardless of the distance to one or more objects existing in the imaging target space and the surface reflectance. A distance image camera and a distance image synthesis method capable of obtaining accuracy are provided.

  In order to achieve the above object, the distance image camera of the present invention is calculated from a light emitting unit that irradiates light to an imaging target space and a measured value of time until the reflected light of the light emitted from the light emitting unit returns. An image capturing unit that acquires a distance image having distance information for each pixel, an exposure adjustment unit that changes exposure when the distance image is acquired by the image capturing unit, and an exposure adjustment unit that changes exposure in stages In the plurality of distance images captured by the imaging unit, the weighted average value of the distance information of each pixel corresponding to the same pixel position is calculated, and the weighted average value thus calculated is the distance of each pixel. A calculation control unit that obtains a combined distance image so as to be information, and each of the pixels also has received light level information indicating a received light intensity of the reflected light. In the calculation of the weighted average value of the distance information, wherein the weighting coefficient calculated so as to correspond to the precision of the distance information in response to the received light level information of the pixels is used.

  Here, the weighting coefficient is selected from one or more functions that are relatively high in the intermediate region of the light reception level information and relatively low outside the intermediate region, and have different maximum positions. It may be determined based on the determined function. Further, the stepwise change of the exposure by the exposure adjusting unit may be performed by at least one of an increase / decrease in shutter speed or a change in intensity of light emitted from the light emitting unit.

  According to the distance image camera having such a configuration, good distance accuracy can be obtained over almost the entire distance image regardless of the distance to one or more objects existing in the imaging target space and the surface reflectance. Is possible.

  In the distance image camera of the present invention, the distance image is acquired by the imaging unit a plurality of times with the same exposure, and the distance information for each pixel and the average value of the light reception level information are provided for each pixel. The averaged distance image may be used instead of the distance image used in the calculation control unit.

  According to the range image camera having such a configuration, since the distance information and the light reception level information for each pixel averaged by performing multiple imaging with the same exposure setting are used, measurement with imaging at each exposure setting is performed. The error is reduced, and as a result, the distance accuracy can be further improved.

  Alternatively, in order to achieve the above object, the distance image composition method of the present invention has distance information calculated from a measurement value of time until the reflected light of the irradiated light returns for each pixel, and for each pixel. A step exposure distance image acquisition step of acquiring a distance image having light reception level information indicating the received light intensity of the reflected light a plurality of times while changing the exposure setting stepwise, and the distance information according to the light reception level information. Weighting coefficient calculation step for calculating a weighting coefficient corresponding to the accuracy of the plurality of distance images acquired in the step exposure distance image acquisition step, the weighted average value of the distance information of each pixel corresponding to the same pixel position, Calculation is performed using the weighting coefficient calculated in the weighting coefficient calculation step, and the weighted average value thus calculated is combined as distance information of each pixel. Characterized in that it comprises a distance image synthesizing step of obtaining a composite range image.

  Similar to the distance image camera described above, the weighting coefficient is relatively high in the intermediate area of the light reception level information and relatively low outside the intermediate area, and the positions where the maximum values are different are different from each other. The determination may be based on a function selected from two or more functions. Further, the stepwise change in exposure in the stepwise exposure distance image acquisition step may be performed by at least one of an increase / decrease in shutter speed or a change in intensity of light emitted from the light emitting unit.

  According to the distance image composition method having such a configuration, good distance accuracy is obtained over almost the entire distance image regardless of the distance to one or more objects existing in the imaging target space and the surface reflectance. It becomes possible.

  In the distance image composition method of the present invention, the distance image in the stepwise exposure distance image acquisition step is acquired a plurality of times with the same exposure, and the average value of each time of the distance information and the light reception level information for each pixel is obtained for each pixel. The averaged distance image may be used in place of the distance image used in the distance image synthesis step.

  According to the distance image synthesizing method having such a configuration, the distance information and the light reception level information for each pixel averaged by performing multiple imaging with the same exposure setting are used. The measurement error is reduced, and as a result, the distance accuracy can be further improved.

  According to the distance image camera and the distance image composition method of the present invention, good distance accuracy over almost the entire distance image regardless of the distance to one or more objects existing in the imaging target space and the surface reflectance. Can be obtained.

1 is a block diagram illustrating a schematic configuration of a range image camera 10 according to an embodiment of the present invention. 4 is a flowchart showing an outline of calculation processing by a distance image camera 10. It is a schematic explanatory drawing about the scanning of each pixel in the arithmetic processing shown in FIG. FIG. 3 is a flowchart showing an outline of processing in a “distance image acquisition” subroutine in the arithmetic processing shown in FIG. 2. FIG. 3 is a flowchart showing an outline of processing in a “weighting coefficient calculation” subroutine in the arithmetic processing shown in FIG. 2. It is the schematic which shows the example of the function fz for calculating the weighting coefficient c in the process shown in FIG. It is the schematic which shows another example of the function fz for calculating the weighting coefficient c in the process shown in FIG. It is the schematic which shows another example of the function fz for calculating the weighting coefficient c in the process shown in FIG.

  Embodiments of the present invention will be described below with reference to the drawings.

<Schematic configuration of range image camera 10>
FIG. 1 is a block diagram showing a schematic configuration of a range image camera 10 according to an embodiment of the present invention.

  As shown in FIG. 1, the distance image camera 10 includes an infrared light emitting unit 11 that irradiates an imaging target space with infrared light, and the infrared light emitted from the infrared light emitting unit 11 is reflected and returned. An image sensor 12 capable of acquiring a distance image having distance data calculated on the basis of a measurement value of the time until arrival for each pixel arranged in a grid, and at the time of distance image acquisition by the image sensor 12 An exposure adjustment unit 13 that can change the exposure setting by changing the shutter speed, a frame buffer 14 as a large-capacity memory that temporarily stores a distance image acquired by the image sensor 12, and an infrared light emitting unit. 11 irradiates infrared light (irradiation intensity, irradiation time, timing, etc.), obtains a distance image by the image sensor 12, and shuts off the exposure adjustment unit 13. -An arithmetic control unit 15 (for controlling the speed change and the like, reading one or more sets of distance images acquired by the image sensor 12 and stored in the frame buffer 14 to perform arithmetic processing (details will be described later)) For example, a CPU). Each pixel of the distance image acquired by the image sensor 12 also has received light level data indicating the received light intensity of the reflected light.

  Examples of the infrared light emitting unit 11 include, but are not limited to, a light emitting diode and a semiconductor laser that emit infrared light. By making the irradiation intensity and irradiation time of infrared light variable, the exposure setting of the image sensor 12 may be performed in a wider range in combination with the exposure adjusting unit 13.

  Examples of the image sensor 12 include a so-called TOF (Time of Flight) sensor, but are not limited thereto.

  The exposure adjusting unit 13 is not limited to one that can change the shutter speed. For example, a diaphragm mechanism or a filter that can electrically change the light transmittance may be used instead, or one or more of these may be used in combination. However, it is preferable that the exposure setting can be changed at high speed so that the time required for obtaining the distance image does not become too long.

  FIG. 2 is a flowchart showing an outline of calculation processing by the distance image camera 10. FIG. 3 is a schematic explanatory diagram of scanning of each pixel in this calculation process.

  First, the exposure adjustment unit 13 performs exposure setting / change for imaging by the image sensor 12 of the distance image camera 10 (step S1). As will be described later, this step S1 is repeated a predetermined number of times N in the loop. For example, when N = 10, that is, when it is repeated 10 times in total, the first time is set as an initial setting, and the shutter speed is set so that the exposure is underexposed (−D) by a predetermined amount with respect to the appropriate exposure when the entire screen is simply averaged. May be set on the high speed side. From the second time onward, the shutter speed is set to the low speed side step by step so that the exposure is shifted to the over side by a fixed amount ΔD. This fixed amount ΔD may be determined such that the final exposure setting is overexposed (+ D) with respect to the above-described proper exposure.

  Then, a subroutine “distance image acquisition” for acquiring a distance image having distance data and light reception level data for each pixel by performing image capturing with such exposure setting is called (step S2). Details of the processing in this subroutine will be described later with reference to FIG.

  The distance data and the light reception level data for each pixel of the acquired distance image are temporarily stored in the frame buffer 14 for later calculation or the like (step S3).

  It is checked whether or not the acquisition of distance images by all exposure patterns has been completed (step S4). If it has not been completed, the process returns to step S1, and if it has already been completed, the process proceeds to the next.

  After each process in the loop from step S1 to step S4 is repeated a predetermined number of times, distance data and light reception level data for each pixel of N distance images stored in the frame buffer 14 are sequentially extracted for each time. (Step S5). As will be described later, this step S5 is also repeated a predetermined number of times N in the loop.

  Each pixel of the distance image extracted from the frame buffer 14 in step S5 is scanned while sequentially moving the target pixel position (step S6).

  Then, a subroutine “Calculate weighting coefficient” for calculating the weighting count is called (step S7). Details of the processing in this subroutine will be described later with reference to FIGS.

  Since each process of steps S6 and S7 needs to be performed for all pixel positions while sequentially moving the target pixel position, it is confirmed whether or not scanning has been completed for all the pixels in the distance image (step S8). If not completed yet, the process returns to step S6 and the target pixel position is moved to repeat the same processing. If scanning for all the pixels has already been completed, the process proceeds to the next.

  Since it is necessary to perform each process of steps S6 and S7 for all N distance images, it is confirmed whether or not the process for all frames has been completed (step S9), and if not completed, the process returns to step S5. The same processing is repeated for the next distance image. If the processing for all the frames has already been completed, the process proceeds to the next.

  Finally, using the weighting coefficient calculated by the double loop from step S5 to step S9, the pixel distance data of each frame is weighted and added for each pixel position (step S10).

By executing each of these processes, as shown in FIG. 3, in the distance image 1, the distance image 2, the distance image 3,. A weighted average value of the distance data of the pixels P ₁ , P ₂ , P ₃ ,..., P _N corresponding to the same pixel position is calculated. When the same weighted average value is calculated for all pixel positions, a distance image using the weighted average value thus calculated as distance data of each pixel can be obtained.

  FIG. 4 is a flowchart showing an outline of processing in the “distance image acquisition” subroutine in the arithmetic processing shown in FIG.

  First, a parameter corresponding to the exposure setting change in step S1 of FIG. 2 is acquired (step S21).

  In accordance with the parameters acquired in step S21, the image sensor 12 of the range image camera 10 is imaged with exposure setting to acquire one range image (step S22).

  Distance data and light reception level data for each pixel of the acquired distance image are temporarily stored in the frame buffer 14 (step S3).

  In order to perform the imaging in step S22 a specified number of times, it is confirmed whether the number of acquired distance images has reached the specified number (step S24), and if not, the process returns to step S22 and the same processing is repeated. If acquisition of the prescribed number of distance images has already been completed, the process proceeds to the next.

  With respect to the prescribed number of distance images stored in the frame buffer 14, all distance data and light reception level data for each pixel are extracted (step S5).

  Finally, a simple averaging process is performed on the distance data of the pixels in each frame for each pixel position (step S10).

  In this way, measurement errors can be reduced by performing multiple imaging with the same exposure setting and using the averaged distance data of the acquired distance images (averaged distance image).

  FIG. 5 is a flowchart showing an outline of processing in the “weighting coefficient calculation” subroutine in the arithmetic processing shown in FIG.

  First, the weighting coefficient c is determined based on the value of the distance data z out of the distance data z and the light reception level a of the pixels determined by the frame and the pixel position at the time when this subroutine is called in step S7 in FIG. A function fz for calculation is determined (step S31). Here, z = 0, 1,... N.

  Then, the weighting coefficient c is calculated from the light reception level a of the pixel and the function fz determined in step S31 (step S32).

  FIG. 6 is a schematic diagram illustrating an example of a function fz for calculating the weighting coefficient c. FIG. 7 is a schematic diagram illustrating another example of the function fz. FIG. 8 is a schematic diagram illustrating still another example of the function fz.

  As a very simple example of the function fz, for example, there are two functions f0 and f1 as shown in FIG. 6, and in this example, n = 1. These functions are relatively higher for both f0 and f1 in a region where the light reception level signal is highly likely to be saturated and in a region where the light reception level signal is very low and the S / N ratio is considerably deteriorated. It is low, almost zero at both ends, and has a curved shape with a relatively high value in the middle region between these regions. Further, the peak at which the coefficient c is maximum is on the higher light reception level a side of the function f1 than the function f0.

  When an object or the like is at a relatively short distance, the reflected light may be strengthened. Therefore, the function f1 may be selected so as to emphasize the distance data of the pixel having higher light reception level data. On the other hand, when an object or the like is at a relatively long distance, the reflected light may be weak. Therefore, if the function f0 is selected so that the distance data of the pixel having the lower light reception level data is emphasized. Good.

  As a simpler example of the function fz, for example, as shown in FIG. 7, only one function f0 that is substantially intermediate between the function f0 and the function f1 in FIG. 6 may be considered. As yet another example, as shown in FIG. 8, only a function f0 that is a constant value within a predetermined range excluding the low and high light reception levels can be considered. In this case, it is possible that the received light level signal is saturated, or the distance data with low reliability is excluded as in the case where the S / N ratio is considerably deteriorated, and other relatively reliable signals are excluded. This means simple averaging of highly accurate distance data.

  It should be noted that the present invention can be implemented in various other forms without departing from the spirit or main features thereof. Therefore, the above-mentioned embodiment is only a mere illustration in all points, and should not be interpreted limitedly. The scope of the present invention is indicated by the claims, and is not restricted by the text of the specification. Further, all modifications and changes belonging to the equivalent scope of the claims are within the scope of the present invention.

DESCRIPTION OF SYMBOLS 10 Distance image camera 11 Infrared light emission unit 12 Image sensor 13 Exposure control part 14 Frame buffer 15 Arithmetic control unit

Claims (8)

A light emitting unit for irradiating light to the imaging target space;
An imaging unit for acquiring a distance image for each pixel having distance information calculated from a measurement value of time until the reflected light of the light emitted from the light emitting unit returns;
An exposure adjustment unit that changes the exposure when the distance image is acquired by the imaging unit;
In the plurality of distance images picked up by the image pickup unit while changing the exposure stepwise by the exposure adjustment unit, the weighted average value of the distance information of each pixel corresponding to the same pixel position is calculated, and so on. An arithmetic control unit that obtains a synthesized distance image that is synthesized so that the weighted average value calculated as described above becomes distance information of each pixel,
Each of the pixels also has received light level information indicating the received light intensity of the reflected light,
In the calculation of the weighted average value of the distance information of each pixel in the arithmetic control unit, a weighting coefficient calculated so as to correspond to the accuracy of the distance information according to the light reception level information of the pixel is used. Distance image camera. The range image camera according to claim 1,
The weighting coefficient is a function selected from one or more functions that are relatively high in the intermediate area of the light reception level information and relatively low outside the intermediate area, and have different maximum positions. A range image camera characterized by being determined based on The range image camera according to claim 1 or 2,
The stepwise change of exposure by the exposure adjusting unit is performed by at least one of increase / decrease of shutter speed or change of intensity of light emitted from the light emitting unit. The range image camera according to any one of claims 1 to 3,
The arithmetic control unit obtains an averaged distance image obtained by performing acquisition of the distance image by the imaging unit a plurality of times with the same exposure, and having an average value for each time of the distance information and the light reception level information for each pixel. A range image camera that is used in place of the range image used in the camera. Obtaining a distance image having distance information calculated from the measured value of the time until the reflected light of the irradiated light returns for each pixel, and also having received light level information indicating the received light intensity of the reflected light for each pixel Stepwise exposure distance image acquisition step of performing multiple times while changing the exposure setting stepwise,
A weighting coefficient calculating step of calculating a weighting coefficient corresponding to the accuracy of the distance information according to the light reception level information;
In a plurality of distance images acquired in the stepwise exposure distance image acquisition step, a weighted average value of distance information of each pixel corresponding to the same pixel position is calculated using the weighting coefficient calculated in the weighting coefficient calculation step. And a distance image synthesizing method for obtaining a synthesized distance image obtained by synthesizing the weighted average value thus calculated as distance information of each pixel. In the range image composition method according to claim 5,
The weighting coefficient is a function selected from one or more functions that are relatively high in the intermediate area of the light reception level information and relatively low outside the intermediate area, and have different maximum positions. A distance image synthesis method characterized by determining based on In the range image composition method according to claim 5 or 6,
The stepwise change of exposure in the stepwise exposure distance image acquisition step is performed by at least one of increasing / decreasing a shutter speed or changing the intensity of light emitted from the light emitting unit. In the range image composition method according to any one of claims 5 to 7,
An averaged distance image obtained by performing the distance image acquisition in the step exposure distance image acquisition step a plurality of times with the same exposure, and having the distance information for each pixel and the average value of the light reception level information for each pixel, A distance image composition method, wherein the distance image composition method is used instead of the distance image used in the distance image composition step.

Images