JP2012085093A - Imaging device and acquisition method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To accurately calculate information on a distance from images taken with/without auxiliary light emitted from a stroboscope, the information having been not accurate due to influence of a reflectance ratio of a subject.SOLUTION: An acquisition method acquires a first image of a subject image while irradiating thereof with the auxiliary light, and a second image of the identical subject image without irradiating thereof with the auxiliary light. Moreover, a value relating to external light from which the influence of the reflectance ratio of the subject is removed is calculated from image data constituting the acquired first image and image data constituting the second image. Then, on the basis of the calculated value and properties of light attenuating corresponding to a distance, information on a distance from the imaging device to the subject is acquired.

Description

  The present invention relates to an imaging apparatus and an acquisition method for acquiring information about distance using an imaging apparatus.

  A normal camera has a function of photographing information about the brightness of an illuminated subject as a monochrome or color grayscale image. In addition to this photographing function, it is useful in various processes if information about the distance to the subject is obtained as a two-dimensional distance image. If information about the distance can be acquired, image processing using a bokeh with a shallow depth of field can be performed with a compact camera using a small-diameter lens.

  As a conventional method for obtaining a distance image, there is a method in which two images with parallax are photographed with a two-lens stereo camera, and the depth is obtained by detecting the parallax between corresponding points in the two images.

  Further, there is a technique called TOF (Time of Flight), and there is a method of obtaining a distance by measuring the arrival time of reflected light from a subject for each pixel using a special image sensor and a light source modulated at high frequency (particularly, Patent Document 1 and Patent Document 2).

  Furthermore, there is a technique for photographing a plurality of images by changing illumination with respect to a single camera and estimating a distance using a difference between the images. One of them is a stereo range finder technique, which measures distance using modulated illumination or laser light (see particularly Patent Document 3).

  In addition, there is a technique for obtaining a distance from a change in the amount of light using two images of flash on and off, although the accuracy is very low. Disclosed is a technique for blurring the background by separating the subject area image data and the background area image data using the flash on and off images, and applying a filter process with a blurring effect to the background area image data and combining them. (In particular, see Patent Document 4)

JP 2000-121339 A JP 2006-84429 A Japanese Patent Laid-Open No. 2001-12909 JP 2005-229198 A

  However, the prior art has the following problems.

  All of the technologies such as the binocular, multi-view, and TOF methods require a camera having a special optical system and an image sensor, and are difficult to easily use for general consumer digital cameras. In addition, the stereo range finder method has a problem that special illumination means are required and it cannot be easily used in consumer equipment.

  In addition, a method using a flash on / off image that can be easily applied to a consumer device has a problem that the accuracy is extremely low.

  In this technology, since the difference between the flash-on image and the off-image is a component that increases the amount of light by flash light, a pixel with a large light amount increase is a foreground, and a pixel with a small light amount increase or no light amount increase is a distant view. , Separated into two regions. However, the increase in the amount of light is not necessarily far away, but the increase in the amount of light is small even when the reflectance of the subject is small (dark color), that is, the black subject is determined to be far and the white subject is determined to be near. Will not necessarily represent the perspective correctly.

  Furthermore, since the balance between the illuminance of outside light and flash light varies greatly depending on the shooting environment, it is difficult to correctly separate the foreground and the background only by the magnitude of the difference value.

  In view of the above problems, the present invention calculates a value obtained by removing the influence of the reflectance of a subject from an image with or without auxiliary light emitted by a strobe obtained by using a strobe function mounted on a general digital camera. An object of the present invention is to provide an imaging apparatus and an acquisition method that can accurately acquire information related to a distance to a subject based on a calculated value.

  In order to solve the above-described problems, the present invention is an imaging apparatus that captures a subject and generates information about a distance to the subject, and includes an optical unit that forms a subject image of the subject and an image based on the subject image. An imaging unit that obtains a signal, an auxiliary light emitting unit that irradiates the subject, and the optical unit and the imaging unit are controlled so that the auxiliary light emitting unit irradiates the subject with auxiliary light and captures a first image. In addition, a shooting control unit including at least a shooting mode for controlling the optical unit and the shooting unit so as to take a second image without irradiating auxiliary light, and the first image obtained by shooting in the shooting mode. A value obtained by removing the influence of the reflectance of the subject is calculated using the image data constituting the image and the image data constituting the second image, and the calculated value and the auxiliary light are calculated as the distance to the subject. By using the attenuation characteristic is a characteristic to attenuate Flip comprises a distance acquisition section that acquires information about the distance to the subject.

  With the configuration described above, the photographing apparatus of the present invention can acquire the first image in which the same subject image is irradiated with the auxiliary light and the second image in which the auxiliary light is not irradiated. Furthermore, a value related to external light from which the influence of the reflectance of the subject is removed can be calculated from the acquired image data constituting the first image and image data constituting the second image. Then, based on the calculated value and the characteristic that the light attenuates according to the distance, information regarding the distance from the imaging device to the subject can be acquired. Thus, even when information on the distance to the subject is calculated from the image data of the image irradiated with the auxiliary light and the image not irradiated with the auxiliary light, the distance is more preferably determined without depending on the reflectance of the subject. It is possible to obtain information regarding the above.

  Preferably, the imaging apparatus further includes a distance measuring unit that measures information related to a distance to a main subject image located in a focus area in a subject image formed in the optical unit, and the distance acquisition unit Is calculated based on information on the distance to the main subject image, a value corresponding to the main subject among values obtained by removing the influence of the reflectance of the subject, and a value obtained by removing the influence of the reflectance of the subject. Information relating to the distance to the subject is acquired using a value obtained by removing the influence of external light and the attenuation characteristic.

  With the above-described configuration, it is possible to calculate a value from which the influence of external light is removed by using, as a reference, information related to the distance of the main subject that can acquire information related to the distance more accurately than others. Furthermore, the distance to the subject can be acquired based on the value from which the influence of external light has been removed. As a result, even when the outside light changes due to the influence of a shielding object such as a building, information on the distance can be acquired based on the value obtained by removing the outside light. Can be obtained.

  Preferably, the imaging apparatus further includes a conversion processing unit that performs conversion processing based on local gradation characteristics for image data constituting the second image, and the distance acquisition unit includes the first acquisition unit. The ratio of the difference between the image data constituting the second image and the image data constituting the second image and the image data constituting the second image subjected to the conversion process is removed from the influence of external light Value.

  By configuring as described above, luminance unevenness can be reduced with respect to the second image, and thus it is possible to acquire information on the distance with higher accuracy.

  According to the present invention, accurate distance information can be acquired from two images with and without auxiliary light emitted by a strobe without being affected by the color (reflectance) of the subject.

Schematic of an imaging apparatus including a shooting environment in the first embodiment Illustration of shooting conditions with and without flash Illustration for explaining the captured image A graph showing the relationship between subject distance and strobe light intensity Distance evaluation function flowchart Schematic diagram of an imaging apparatus according to the second embodiment External light estimation unit flowchart Flow chart of distance estimation unit Schematic of the imaging device in the third embodiment Configuration diagram of local gradation processing unit The figure which shows the graph of the conversion characteristic of a dynamic gradation correction part

  Hereinafter, the digital camera according to the present embodiment will be described with reference to the drawings.

[First Embodiment]
In the present embodiment, a digital camera that uses a strobe (synonymous with flash) as auxiliary light will be described below as an example.

<Configuration>
FIG. 1 is a schematic configuration diagram of an imaging apparatus including a shooting environment in the present embodiment.

  The imaging apparatus 100 includes an optical system 101, an image sensor 102, a focus control unit 103, an auxiliary light emitting unit 104, a shooting control unit 105, a memory 106, and a distance acquisition unit 107.

  The optical system 101 is an optical system including a plurality of lenses that collect an optical image of a subject. The optical system 101 is an optical system having a focus adjustment function. The imaging optical system 101 guides the optical image from the main subject 202, the other subject 203, and the other subject 204 that is illuminated by the illumination light 201 from outside light to the image sensor 102.

  @ 102 is an image sensor that converts an optical image collected by the optical system 101 into an electrical signal. For example, the image sensor 102 converts an optical image of a subject shown in the shooting scene 200 into an electrical signal. Here, the shooting scene 200 shows a certain shooting scene. This shooting scene 200 includes irradiation light 201 by external light that is external light, a main subject 202 in the shooting scene 200, a main subject 202 that is a subject in the background, and other subjects 204.

  The focus control unit 103 is a control unit that adjusts a lens included in the optical system 101 in order to focus on a subject. The focus control unit 103 moves the optical system 101 back and forth in response to an instruction from the imaging control unit 105 to form an image of the irradiation light 201 by external light on the image sensor 102.

  @ 104 is a strobe that emits light for assisting photographing with respect to the subject.

  The imaging control unit 105 issues a command for light emission from the auxiliary light emitting unit 104 and focus adjustment in the focus control unit 103. Furthermore, the imaging control unit 105 reads the electrical signal captured by the imaging unit 102. The photographing control unit 105 causes the auxiliary light emitting unit 104 to emit light and photographs the photographing scene 200 and stores it in the memory unit 106. Hereinafter, an image obtained by shooting the shooting scene 200 by causing the auxiliary light emitting unit 104 to emit light is referred to as a first image. Further, the photographing control unit 105 photographs the photographing scene 200 without causing the auxiliary light emitting unit 104 to emit light, and stores the photographing scene 200 in the memory unit 106. Hereinafter, an image obtained by shooting the shooting scene 200 without causing the auxiliary light emitting unit 104 to emit light is referred to as a second image. Note that the first image and the second image are taken continuously in time. That is, the photographing control unit 105 has a photographing mode in which at least the first image and the second image are photographed continuously in time. The shooting interval between the two images is sufficiently short so as not to be affected by the movement of the subject. Note that the order of shooting with and without strobe light emission may be arbitrary. Also, it is not always necessary to match the exposure amount due to the aperture and shutter time of the two shots, and the exposure is made with the optimum exposure amount for both strobe light emission and non-light emission. It can be corrected by.

  The memory unit 106 temporarily stores the electrical signal acquired by the imaging control unit 105.

  @ 107 acquires information related to the distance from the imaging device 100 to the subject based on the electrical signal stored in the memory unit 106. The memory unit 106 has an attenuation characteristic that is a characteristic in which light attenuates according to the distance, and calculates information related to the distance from the electrical signal stored in the memory unit 106 based on the attenuation characteristic.

  The distance acquisition unit 107 performs distance estimation using the first and second images stored in the memory unit 106 and outputs the distance image.

<Operation>
Hereinafter, the operation of the imaging apparatus 100 will be described with reference to the drawings.

  FIG. 2 is an explanatory diagram showing shooting conditions with and without strobe light emission.

  FIG. 3 is a diagram illustrating an image captured by the imaging apparatus 100.

  FIG. 2A shows shooting conditions with strobe light emission.

  FIG. 2B shows shooting conditions without strobe light emission.

  Here, the intensity level of the external light 201 is C, the intensity level of the strobe light is W, the distance to an arbitrary subject is L, and the reflectance of the subject is R.

  Under this condition, the intensity C of the external light 201, the distance L and the reflectance R of the arbitrary subject 204, and the intensity W of the strobe light reaching the arbitrary subject 204 are all unknown.

  The operation of the distance acquisition unit 107 will be described using mathematical expressions with reference to FIGS. 2 (a) and 2 (b).

  Hereinafter, it is assumed that the memory unit 106 stores a first image Img1 photographed with strobe light emission and a second image Img2 photographed with no strobe light emission. This Img1 includes the influence of the light from the strobe and the light from the other subject 203. Further, Img2 includes the influence of the light of the other subject 203.

  Here, when an arbitrary subject 204 is located at the coordinates (x, y) in the image, the pixel value representing the brightness of Img1 is proportional to the product of the amount of illumination light and the reflectance of the subject, so that k is set. It can be expressed by the following formula as a proportionality constant.

(Formula 1) Img1 (x, y) = k * (C + W) * R
Similarly, the same pixel when shooting without flash is expressed by the following equation.

(Formula 2) Img2 (x, y) = k * C * R
The distance acquisition unit 107 calculates a value not affected by the reflectance of the subject from Img1 and Img2. Hereinafter, the calculated value is expressed as H. This H is obtained by canceling the influence of the reflectance from the pixel values of Img1 and Img2 by setting the assumption that the external light C can be regarded as uniform for any subject. The distance acquisition unit 107 calculates H as the ratio of the pixel value (Img1-Img2) based on strobe light and the pixel value (Img2) based only on external light. Since the reflectance R is removed by taking the ratio as described above, it is possible to evaluate the distance regardless of the subject color. In short, the memory unit 106 calculates H by the following equation.

(Formula 5) H (x, y) = (Img1 (x, y) −Img2 (x, y)) / Img2 (x, y) = w (x, y) / C
Hereinafter, the operation of the distance acquisition unit 107 will be described with reference to the drawings.

  FIG. 4 is a flowchart when the distance acquisition unit 107 calculates H.

  First, the distance acquisition unit 107 acquires the pixel value Img1 of the first image captured by emitting light from the auxiliary light emitting unit 104 (S10).

  Next, the distance acquisition unit 107 acquires the pixel value Img2 of the coordinates (x, y) of the second image captured without emitting the auxiliary light emitting unit 104 (S11). Note that the imaging acquisition order in S10 and S11 may be reversed.

  Then, the distance acquisition unit 107 calculates the ratio of the pixel level (Img1-Img2) by only the strobe light to the pixel level (Img2) by only the external light as H using (Equation 5).

  Hereinafter, an operation in which the distance acquisition unit 107 calculates information on the distance based on H will be described.

  FIG. 5 is a diagram illustrating the relationship between the subject distance L and the intensity W of the strobe light. Since the light from the strobe light can be regarded as a point light source, in principle, when the distance increases, the light attenuates approximately in inverse proportion to the square of the subject distance. Based on this attenuation characteristic, the distance is calculated from H. Note that this attenuation characteristic may be a characteristic set based on an actual measurement value. In that case, it becomes possible to deal with factors that may actually occur.

  Specifically, the memory unit 106 can calculate the relative distance of the subject in the captured image from the calculated H and the attenuation characteristic shown in FIG.

  As described above, the distance acquisition unit 107 according to the present embodiment can remove or reduce the influence of the reflectance from the two images of strobe light emission / non-light emission. For this reason, the intensity C of the external light itself is unknown, but if it is regarded as a constant that is unique to each shooting scene, the relative distances (relative context) of many subjects in the scene can be calculated. Thereby, foreground and background can be separated regardless of the color of the subject.

  In addition, the assumption that the outside light 201 is constant is not always true, but evaluation experiments in various typical shooting scenes (landscapes, portraits of people, close-up of flowers, etc.) that are often taken in reality As a result, it has been found that the above assumptions are generally satisfied in the shooting of these scenes and that there is no problem.

  In addition, the evaluation formula which cancels the influence of a reflectance by using the ratio of an image in this way can be made innumerable as another example below. For example, the value calculated from the following equation may be H.

(Equation 6) H ′ (x, y) = Img1 (x, y) / Img2 (x, y) = (C + w) / C
With the above configuration, it is possible to calculate information related to the distance from the imaging apparatus 100 to the subject without including the reflectance R. For this reason, it is possible to suitably acquire information regarding the distance even in a scene where subjects having various reflectances exist.

  In short, the imaging apparatus 100 according to the present embodiment is an imaging apparatus 100 that shoots a subject and generates information about a distance to the subject, and includes an optical system 101 that forms a subject image of the subject and the subject image. An image sensor 102 that obtains an image signal, an auxiliary light emitting unit 104 that irradiates the subject, and the optical system 101 and the auxiliary light emitting unit 104 to irradiate the subject with auxiliary light and take a first image. A shooting control unit 105 that controls the image sensor 102 and includes at least a shooting mode for controlling the optical system 101 and the image sensor 102 so as to take a second image without irradiating auxiliary light, and in the shooting mode Image data constituting the first image and image data constituting the second image obtained by photographing Information about the distance to the subject using the calculated value and an attenuation characteristic that is a characteristic that the auxiliary light attenuates according to the distance to the subject. Is provided.

  With the configuration described above, the imaging apparatus 100 can acquire a first image in which auxiliary light is irradiated on the same subject image and a second image in which auxiliary light is not irradiated. Furthermore, a value related to external light from which the influence of the reflectance of the subject is removed can be calculated from the acquired image data constituting the first image and image data constituting the second image. Then, based on the calculated value and the characteristic that the light attenuates according to the distance, information regarding the distance from the imaging device to the subject can be acquired. Thus, even when information on the distance to the subject is calculated from the image data of the image irradiated with the auxiliary light and the image not irradiated with the auxiliary light, the distance is more preferably determined without depending on the reflectance of the subject. It is possible to obtain information regarding the above.

[Second Embodiment]
Next, Embodiment 2 will be described with reference to the drawings.

<Configuration>
FIG. 6 is a schematic configuration diagram of an imaging apparatus according to the second embodiment. The same elements as those in the first embodiment shown in FIG.

  The imaging device 300 is an imaging device in the present embodiment. The imaging apparatus 300 newly includes a distance acquisition unit 307, an external light estimation unit 308, and a distance estimation unit 309.

  @ 307 calculates information regarding the distance from the imaging apparatus 300 to the subject from the image data stored in the memory unit 106. The distance acquisition unit 307 includes an outside light estimation unit 308 and a distance estimation unit 309.

  The outside light estimation unit 308 estimates the intensity C of outside light using information obtained from @ 103. Detailed operation of estimation will be described later.

  The distance estimation unit 309 calculates information related to the distance from the imaging device 300 to the subject based on the intensity C of the external light calculated by the external light estimation unit 308 and the image data stored in the memory unit 106. The distance estimation unit 309 performs the same operation as the distance acquisition unit 107. The difference lies in the use of the external light intensity C estimated by the external light estimation unit 308 when calculating the distance information from H calculated based on the attenuation characteristics. This makes it possible to calculate the absolute distance, not the relative distance of the subject in the captured image.

  The distance acquisition unit 107 according to the first embodiment calculates the information about the distance using H in (Equation 5). However, since there is an unknown external light intensity C for each shooting scene, the same external light intensity and It was only used for relative comparison of distances within a single shooting scene.

  In the second embodiment, an image having a distance acquisition unit 307 that can estimate the distance C without the influence of the reflectance R that differs for each subject and without the influence of the intensity of the external light by estimating the intensity C of the external light. An apparatus 300 is proposed.

<Operation>
Hereinafter, the operation of the external light estimation unit 308 will be described with reference to the drawings.

  Actually, the intensity C of the outside light is unknown and varies greatly depending on the photographing conditions. For example, C is very large when shooting in the daylight outdoors in the daytime, and C is small under indoor and backlight conditions. Therefore, when the intensity C of the external light is unknown, the relative absolute relationship is known by H in (Equation 5), but the actual absolute distance is unknown.

  The external light estimation unit 308 receives from the memory unit 106 Img1 captured with strobe light emission and Img2 without strobe light emission, and further receives the focus distance information L0 of the main subject 202 obtained from the focus control unit 103 to perform estimation. The external light intensity C is sent to the distance estimation unit 309. The reflectance of the main subject is R0.

  Hereinafter, the operation of the external light estimation unit 308 will be described with reference to the drawings.

  FIG. 7 is a flowchart showing the operation of the external light estimation unit 308.

  First, the external light estimation unit 308 inputs the distance L0 of the main subject 202 obtained in the process of controlling the focus and adjusting the focus from the focus control unit 103 (S20).

  Next, the external light estimation unit 308 obtains the intensity W0 of the strobe light that illuminates the main subject 204 using the attenuation characteristics shown in FIG. 5 measured in advance (S21).

  Further, the external light estimation unit 308 obtains a reference H0 by the following equation using the pixel levels of the image 1 and the image 2 at the position (x0, y0) in the image of the main subject 202 as shown in FIG. S22).

(Formula 7) H0 = (Img1 (x0, y0) −Img2 (x0, y0)) / Img2 (x0, y0)
Then, @ 208 estimates the intensity C of outside light from the reference H0 and the intensity W0 of the reference strobe light (S23).

  As described above, the intensity C of the external light can be estimated based on the ratio of the strobe light to the external light at the pixel position (x0, y0) where the distance L0 can be known by the focus adjustment.

  Next, the operation of the distance estimation unit 309 will be described using the flowchart of FIG.

  Since the estimation of the distance is performed for each pixel, a processing loop for each pixel is configured.

First, the distance estimation unit 309 acquires the intensity C of the external light estimated by the external light estimation unit 308 (S30).
Next, the distance estimation unit 309 performs initialization of pixel positions to be processed x = 0, y = 0 (S31). Note that S30 and S31 are initial setting operations in the distance estimation unit 309. That is, the distance estimation unit 309 executes S30 and S31 before performing the process in units of pixels.

  Next, the distance estimation unit 309 acquires H from the pixel level of the pixel (x, y) in the images 1 and 2 (S32).

  Then, the @ distance estimating unit 309 obtains the strobe light intensity W of the pixel (x, y) from H and the intensity C of the external light (S33).

  Further, the distance estimation unit 309 estimates the distance L from the strobe light intensity W of the pixel (x, y).

  Then, the distance estimation unit 309 confirms whether there is an unprocessed pixel (S35). If there is an unprocessed pixel, the process proceeds to S36. If there is no unprocessed pixel, the operation is terminated as it is.

  The distance estimation unit 309 changes the processing target to an unprocessed pixel after finishing the processing from S32 to S34 for a specific pixel (S35). And the process of S32 to S34 is performed with respect to an unprocessed pixel.

  By repeating the processes from S32 to S34 for the number of pixels, the distances for all the pixels can be obtained and output as a distance image.

  In the present embodiment, the distance estimation is performed by estimating the intensity of external light using the pixel levels of the two captured images of the strobe light emission / non-light emission in the main subject whose actual distance is known from the focus information. The configuration has been described as an example.

  However, the point of the present embodiment is not to estimate the intensity of the external light itself, but to estimate the distance by using two captured images of strobe light emission / non-light emission on the basis of a main subject whose only actual distance is known. The idea is to calibrate the results. In fact, the distance estimated using H using the two pixel levels in the two photographed images does not have to explicitly estimate the intensity of the external light in the main subject whose distance is known. This means that the range is adjusted and calibrated to match the actual distance. Although the estimation of the intensity of outside light is one of the means for realizing this, as long as it is based on this idea, the same result can be obtained by any method.

  In this embodiment, the influence of the reflectance (color) that differs for each subject can be canceled, and the difference in the intensity of external light that differs for each shooting scene can also be canceled, so the absolute value of the distance can be known. It is also possible to compare the distance between different shooting scenes.

  The distance to the subject can be treated as a continuous value, and the application range is extremely wide.

  For example, even in applications where the background is blurred and an image with a shallow depth of field is generated, the amount of blur can be controlled continuously for each distance rather than simply separating the foreground and background. Near high-quality blur generation is possible. In any shooting scene, a stable bokeh can be obtained depending on the distance.

  The imaging apparatus 300 according to the second exemplary embodiment includes a focus control unit 103 that measures information about a distance to a main subject image located in a focus area in a subject image formed in the optical system 101 in addition to the imaging device 100. The distance acquisition unit 307 includes information on the distance to the main subject image, a value corresponding to the main subject among values obtained by removing the influence of the reflectance of the subject, and the influence of the reflectance of the subject. Information on the distance to the subject is acquired using the value obtained by removing the influence of the external light calculated based on the value obtained by removing the light and the attenuation characteristic.

  With the above-described configuration, it is possible to calculate a value from which the influence of external light is removed by using, as a reference, information related to the distance of the main subject that can acquire information related to the distance more accurately than others. Furthermore, the distance to the subject can be acquired based on the value from which the influence of external light has been removed. As a result, even when the outside light changes due to the influence of a shielding object such as a building, information on the distance can be acquired based on the value obtained by removing the outside light. Can be obtained.

[Third Embodiment]
Next, the third embodiment will be described.

<Configuration>
FIG. 9 is a schematic configuration diagram of an imaging apparatus according to the third embodiment. Components common to the second embodiment shown in FIG. 6 are assigned the same element numbers and will not be described.

  The imaging device 400 is the imaging device according to the third embodiment, and the external light correction unit 401 is an external light correction unit that corrects the influence of external light from the captured image.

  In this embodiment, the reflectance is canceled by setting the assumption that the first and second embodiments can assume that the intensity C of the external light is uniform for any subject in the shooting scene. Was. This assumption has been valid in many scenes.

  However, there are scenes where this assumption does not hold in real scenes.

  For example, the intensity of external light is smaller than expected in shaded areas such as shade. For this reason, the area is brightened by the strobe light regardless of whether it is dark. Therefore, when the distance of this area is estimated by the method of the above-described embodiment, it is estimated that the distance is closer than the actual distance. Conversely, the slopes such as the roof facing the sun are estimated to be farther away than the actual distance because the outside light is stronger than expected.

  In the third embodiment, the non-uniformity of the outside light is reduced, thereby reducing the distance error due to the way the outside light hits.

  FIG. 10 shows a configuration of a local gradation conversion unit 500 which is a main functional block constituting the external light correction unit.

  The ambient brightness detection unit 501 is an ambient brightness detection unit that detects average brightness around a relatively wide range.

  @ 502 is a dynamic gradation conversion unit in which gradation conversion characteristics change according to the average brightness of the surroundings.

<Operation>
The operation of the external light correction unit 401 will be described.

  The external light correction unit 401 corrects the strobe light only image (Img1-Img2) obtained by subtracting Img2 obtained by photographing without firing the strobe from Img1 obtained by photographing with the strobe. Without correction, correction of external light is performed using only the local gradation processing unit 500 shown in FIG. 10 only for Img2 that is illuminated only with external light.

  The ambient lightness detection unit 501 constituting the local gradation processing unit 500 outputs an average US of a relatively wide range of brightness around the pixel IS to be processed in the image.

  Further, as shown in FIG. 11, the dynamic gradation conversion unit 502 has different gradation conversion characteristics (input IS, output OS) for each average value US of ambient brightness output by the ambient brightness detection unit 501.

  As for this conversion characteristic, a curve (bold line in the figure) connecting the intersections of IS = US (black circles in the figure) has an inverted S-characteristic. This means that under the condition of IS = US, which means a gradation characteristic of a relatively large area, the dark part is brightly converted over a wide range and the bright part is darkened over a wide range. Further, in the local region around the pixel to be processed (US = a range that can be regarded as constant), since the conversion characteristic is a straight line passing through the origin, the local contrast is maintained.

  Therefore, the local gradation processing unit 500 having this characteristic operates to reduce a general change in brightness and darkness that leads to a change in lighting without changing the local contrast that leads to a change in reflectance of the subject. It works to remove and reduce uneven illumination due to external light.

  It should be noted that the local gradation conversion unit 500 can execute any operation that can reduce an overall change in brightness and darkness that leads to a change in lighting without changing the local contrast that leads to a change in the reflectance of the subject. Such a method may be used.

  In addition, known Retinex technology and local histogram processing can also be used because they tend to reduce illumination unevenness.

  According to the present embodiment, it is possible to perform more accurate distance estimation and obtain a good distance image even in a shooting scene where the assumption that the external light hitting method assumed in the above-described embodiment is uniform is lost.

  With the above configuration, the imaging apparatus 400 performs an external light correction unit 401 that performs conversion processing based on local gradation characteristics on the image data constituting the second image, in addition to the imaging apparatus 100 and the imaging apparatus 300. The distance acquisition unit 307 includes a difference value between image data constituting the first image and image data constituting the second image, and an image constituting the second image subjected to the conversion process. The ratio to the data is set to a value from which the influence of external light is removed.

  By configuring as described above, luminance unevenness can be reduced with respect to the second image, and thus it is possible to acquire information on the distance with higher accuracy.

[Other Embodiments]
In the imaging device described in the above embodiment, each block may be individually made into one chip by a semiconductor device such as an LSI, or may be made into one chip so as to include a part or the whole. Here, although LSI is used, it may be called IC, system LSI, super LSI, or ultra LSI depending on the degree of integration.

  Further, the method of circuit integration is not limited to LSI, and implementation with a dedicated circuit or a general-purpose processor is also possible. An FPGA (Field Programmable Gate Array) that can be programmed after manufacturing the LSI or a reconfigurable processor that can reconfigure the connection and setting of circuit cells inside the LSI may be used.

  Further, if integrated circuit technology comes out to replace LSI's as a result of the advancement of semiconductor technology or a derivative other technology, it is naturally also possible to carry out function block integration using this technology. Biotechnology can be applied as a possibility.

  Each processing of the above embodiment may be realized by hardware, or may be realized by software (including a case where the processing is realized together with an OS (Operating System), middleware, or a predetermined library). Further, it may be realized by mixed processing of software and hardware. Needless to say, when the stereoscopic imaging apparatus according to the above-described embodiment is realized by hardware, it is necessary to perform timing adjustment for performing each process. In the above embodiment, for convenience of explanation, details of timing adjustment of various signals generated in actual hardware design are omitted.

  Moreover, the execution order of the processing method in the said embodiment is not necessarily restricted to description of the said embodiment, The execution order can be changed in the range which does not deviate from the summary of invention.

  The specific configuration of the present invention is not limited to the above-described embodiment, and various changes and modifications can be made without departing from the scope of the invention. For example, although a strobe is used as auxiliary light, any illumination such as a flash or an LED light source may be used.

  The imaging apparatus and acquisition method according to the present invention can be applied to a photographing apparatus such as a digital camera.

DESCRIPTION OF SYMBOLS 100, 300, 400 Image pick-up device 101 Optical system 102 Image sensor 103 Focus control part 104 Auxiliary light emission part 105 Imaging | photography control part 106 Memory part 107,307 Distance acquisition part 109,309 Distance estimation part 200 Shooting scene 201 Irradiation light by external light 202 Main subjects 203 and 204 Other subjects 308 Ambient light estimation unit 401 Ambient light correction unit 500 Local gradation conversion unit 501 Ambient lightness detection unit 502 Dynamic gradation correction unit

Claims (4)

An imaging device that shoots a subject and generates information about a distance to the subject,
An optical unit for forming a subject image of the subject;
A photographing unit for obtaining an image signal based on the subject image;
An auxiliary light emitting unit for illuminating the subject;
The optical unit and the imaging unit are controlled to irradiate the subject with auxiliary light by the auxiliary light emitting unit to capture the first image, and the optical to capture the second image without irradiating the auxiliary light. A photographing control unit comprising at least a photographing mode for controlling the photographing unit and the photographing unit;
Using the image data constituting the first image and the image data constituting the second image obtained by photographing in the photographing mode, a value obtained by removing the influence of the reflectance of the subject is calculated, and the calculated value And a distance acquisition unit that acquires information on the distance to the subject using attenuation characteristics that are characteristics in which light attenuates according to distance.
Imaging device The imaging apparatus further includes a distance measuring unit that measures information related to a distance to a main subject image located in a focus area in a subject image formed in the optical unit,
The distance acquisition unit is based on information on a distance to the main subject image, a value corresponding to the main subject among values obtained by removing the influence of the reflectance of the subject, and a value obtained by removing the influence of the reflectance of the subject. The image pickup apparatus according to claim 1, wherein information relating to a distance to the subject is acquired using a value obtained by removing an influence of external light calculated in the above and the attenuation characteristic. The imaging apparatus further includes a conversion processing unit that performs conversion processing based on local gradation characteristics with respect to image data constituting the second image,
The distance acquisition unit includes a difference value between image data constituting the first image and image data constituting the second image, and image data constituting the converted second image. The imaging apparatus according to claim 1, wherein the ratio is a value from which an influence of external light is removed. An acquisition method for acquiring information related to a distance to a subject in an imaging apparatus that captures the subject,
An optical unit for forming a subject image of the subject;
A photographing unit for obtaining an image signal based on the subject image;
An auxiliary light emitting unit for illuminating the subject;
A control unit for controlling the own device,
The controller is
The optical unit and the imaging unit are controlled to irradiate the subject with auxiliary light by the auxiliary light emitting unit to capture the first image, and the optical to capture the second image without irradiating the auxiliary light. Control the imaging unit and the imaging unit,
Using the image data constituting the first image and the image data constituting the second image obtained by photographing in the photographing mode to calculate a value obtained by removing the influence of the reflectance of the subject,
Obtaining information on the distance to the subject using the calculated value and an attenuation characteristic that is a characteristic that light attenuates according to the distance;
Acquisition method.