JP2023113158A - Imaging apparatus and white balance control method - Google Patents

Abstract

To provide an imaging apparatus capable of performing light source estimation with high accuracy by detecting an infrared ray made incident from a photographing field angle through a lens optical system.SOLUTION: The imaging apparatus including an attachable/detachable photographic lens having the lens optical system includes: an imaging element which detects light made incident through the lens optical system, photoelectrically converts the detected light, and outputs a signal; an infrared ray reflection filter which is arranged on the object side of the imaging element; an infrared ray sensor which is arranged between the lens optical system and the infrared ray reflection filter and outside an optical path of the light passing through the lens optical system; and a control part. The infrared ray of the light made incident through the lens optical system is reflected by the infrared ray reflection filter, the infrared ray reflected by the infrared ray reflection filter is detected by the infrared ray sensor, and the control part obtains the intensity of the signal of the infrared ray detected by the infrared ray sensor and the intensity of the signal of visible light transmitted through the infrared ray reflection filter and detected by the imaging element.SELECTED DRAWING: Figure 2

Description

The present invention relates to an imaging device and a white balance control method, and more particularly to a white balance control method including subject light source estimation using infrared detection performed in the imaging device.

Conventionally, in a digital camera, which is a type of imaging device, a technology has been disclosed in which an infrared sensor is effectively used to determine the type of light source in the imaging environment and appropriately control the white balance of an image to be captured. there is

Patent Literature 1 discloses an imaging device, a white balance control method, and a white balance control program that enable white balance control by effectively using an existing infrared sensor in the imaging device.

According to the invention disclosed in Patent Document 1, an existing phase difference sensor required for autofocus control in an imaging device and an existing infrared sensor required for remote control are effectively used. By doing so, it is possible to control the white balance of an image to be captured. Also, by effectively using these existing infrared sensors, it is possible to determine the type of light source in the imaging environment and appropriately control the white balance of the captured image.

Patent No. 4033140

However, in the invention disclosed in Patent Document 1, the angle of view of the infrared rays acquired by the infrared sensor does not necessarily match the angle of view of the captured image. There is a risk of estimating a different angle of view from the light source in the environment. In addition, when the user unintentionally blocks the remote control sensor having the infrared sensor while operating the imaging device, or the interchangeable lens of a camera with interchangeable lenses blocks the remote control sensor having the infrared sensor, It may not be possible to estimate the light source for the shooting angle of view in the shooting environment.

SUMMARY OF THE INVENTION It is an object of the present invention to provide an imaging apparatus capable of estimating a light source with high accuracy by detecting infrared rays incident from a shooting angle of view through a lens optical system. and

In order to solve the above-described problems, an imaging apparatus according to a first aspect of the present invention is an imaging apparatus equipped with a detachable photographing lens having a lens optical system, in which light incident through the lens optical system is detected. , an image sensor that photoelectrically converts the detected light and outputs a signal, an infrared reflective filter arranged on the object side of the image sensor, and between the lens optical system and the infrared reflective filter, light passing through the lens optical system Equipped with an infrared sensor placed outside the optical path of the lens, a control unit, and an image processing unit that converts the signal output by the image sensor into image data and outputs it. The infrared sensor detects infrared rays reflected by the filter and reflected by the infrared reflection filter, the control unit obtains the intensity of the infrared signal detected by the infrared sensor, and the visible light passes through the infrared reflection filter and is detected by the imaging device. is obtained by the control unit through the image processing unit.

In the imaging device according to the second aspect of the present invention, the control unit includes a light source estimation unit, the light source estimation unit compares the intensity of the infrared signal and the intensity of the visible light signal acquired by the control unit, It is characterized by calculating the index of the light source in the shooting environment based on the result of the comparison.

In the imaging apparatus according to a third aspect of the present invention, the control unit includes a flicker detection unit, and the flicker detection unit detects when the index of the light source calculated by the light source estimation unit is greater than a threshold determined from the intensity of the visible light signal. The flicker detection is performed on the image data output by the image processing unit by changing the reading frame rate of the imaging device.

In the image pickup apparatus according to the fourth aspect of the present invention, the control section includes a white point distribution setting section and a white balance setting section, and the white point distribution setting section performs the following operations on the image data output by the image processing section. The white point distribution is calculated based on the index of the light source calculated by the light source estimation unit, and the white balance setting unit calculates and changes the white balance setting value based on the white point distribution.

In a white balance control method according to a fifth aspect of the present invention, an infrared reflective filter reflects infrared rays of light incident through a lens optical system, and the infrared reflective filter is between the lens optical system and the infrared reflective filter. In a white balance control method for an imaging device in which an infrared sensor arranged outside the optical path of light passing through an infrared sensor detects infrared rays, and an imaging element detects visible light transmitted through an infrared reflection filter, the step of detecting infrared rays by the infrared sensor; , the imaging device detecting visible light;
a step of comparing the intensity of the infrared signal and the intensity of the visible light signal; calculating a light source index of the shooting environment according to the comparison result; The method includes the steps of performing detection, calculating a white point distribution based on the index of the light source, and changing a white balance setting value based on the white point distribution.

According to the present invention, it is possible to detect infrared rays incident through the lens optical system, that is, infrared rays incident from the shooting angle of view. An imaging device capable of estimation can be provided. In addition, it is possible to omit flicker detection in an imaging device in an environment other than an artificial light environment, and it is possible to prevent deterioration of the user's operational feeling due to control when flicker is detected in the imaging device. Furthermore, it is possible to improve the accuracy of white balance control by estimating the light source with high accuracy.

1 is a block diagram showing the main configuration of an imaging device according to an embodiment of the present invention; FIG. 4 is a diagram for explaining the arrangement of an infrared reflective filter 42 and an infrared sensor 43 and the behavior of the infrared reflective filter 42 reflecting infrared rays out of the light flux passing through the lens optical system 32. FIG. 4 is a flowchart for explaining the flow of processing from light source estimation to white balance setting in the camera body 20; It is a figure for demonstrating the white point distribution under a natural light environment, and the white point distribution under an artificial light environment.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. It should be noted that the present invention is not limited by this embodiment. Also, each drawing has been simplified for ease of explanation of the present invention.

FIG. 1 is a block diagram showing the main configuration of an imaging device 10 of this embodiment. The imaging apparatus 10 is composed of a camera body 20 and a taking lens 30 which is an interchangeable lens that can be attached to and detached from the camera body 20 . The camera body 20 has a camera-side mount portion 21, and the photographing lens 30 has a lens-side mount portion 31, which are detachably fixed by coupling the two mount portions.

The camera body 20 includes an imaging element 41 , an infrared reflection filter 42 , an infrared sensor 43 , a control section 50 , an image processing section 60 and an image display section 70 .

The photographing lens 31 includes a lens optical system 32 having a plurality of lens groups (not shown) including a focus lens group. In FIG. 1, the lens optical system 32 is illustrated as one lens for the sake of simplicity.

The imaging device 41 receives the light from the subject condensed by the lens optical system 32, photoelectrically converts the light, and outputs an analog image signal of the subject image. In this embodiment, a CMOS image sensor is used as the imaging device 41 . The light receiving surface of the imaging device 41 is composed of a large number of pixels.

The infrared reflective filter 42 is arranged in front of the imaging device 41, and reflects infrared light components and transmits visible light components of the light guided to the imaging device 41 through the lens optical system 32. FIG.

The infrared sensor 43 is located between the lens optical system 32 and the infrared reflecting filter 42 and outside the optical path of light passing through the lens optical system 32, detects infrared rays, and outputs an infrared signal. The infrared sensor 43 is arranged, for example, on the outer edge of the back surface of the surface of the camera-side mount section 21 that contacts the lens-side mount section 31 . Also, a plurality of infrared sensors may be arranged. By arranging a plurality of infrared sensors, it is possible to improve the detection accuracy of infrared rays.

The image processing unit 60 performs various kinds of signal processing on the analog image signal output from the imaging element 41 and outputs it as digital image data. For example, amplification processing for amplifying the read color component signal, shading correction for ensuring linearity between the amount of light received by the pixel and the output value, color reproduction processing, gamma correction processing, JPEG format and TIFF format Performs development processing, etc. on image data. It also performs white balance setting, which will be described later.

The control unit 50 has a CPU, a ROM storing programs and data, and a RAM used as a work area, and comprehensively controls the imaging apparatus 10 as a whole. For example, readout control of the imaging device 41 is performed. The control unit 50 outputs a signal that determines the driving timing of the image sensor 41, thereby controlling horizontal driving and vertical driving for each pixel, and RGB color component signals are read out from each pixel. Also, in the process of detecting flicker, which will be described later, the readout timing of the image sensor 41 and the exposure time are changed.

The image display section 70 displays the digital image data processed by the image processing section 60 as through image data. In addition, images and the like read from a recording medium (not shown) are displayed.

Next, a method for detecting infrared rays reflected by the infrared reflection filter 42 by the infrared sensor 43 will be described. FIG. 2 is a diagram for explaining the arrangement of the infrared sensor 43 and the infrared reflective filter 42 and how the infrared reflective filter 42 reflects the infrared rays of the light passing through the lens optical system 32 .

Of the light Φ1 incident through the lens optical system 32, the infrared reflecting filter 42 reflects the infrared component Φ2. An infrared sensor 43 arranged between the lens optical system 32 and the infrared reflective filter 42 detects the infrared component Φ2 reflected by the infrared reflective filter 42 . The visible light component Φ3 that has passed through the infrared reflecting filter 42 enters the imaging device 41 . By arranging the infrared sensor 43 and the infrared reflection filter 42 in this way, it is possible to detect infrared rays incident from the actual photographing angle of view regardless of the type of interchangeable lens.

FIG. 3 is a flowchart for explaining the flow of processing from light source estimation to white balance setting in the camera body 20 . The control unit 50 has an infrared intensity calculation unit 51, a visible light intensity calculation unit 52, a light source estimation unit 53, a flicker detection unit 54, a white point distribution setting unit 55, and a white balance setting unit 56 inside. ing. Hereinafter, the processing of each unit of the control unit 50 will be described in detail using flowcharts.

This flowchart starts when the camera body 20 performs an operation corresponding to starting the camera or switching from the setting screen of the camera to the shooting mode.

In S01, the infrared intensity calculator 51 acquires the infrared signal detected by the infrared sensor 43, and calculates the intensity Bv_IR of the infrared ray based on the intensity of the infrared signal. When a plurality of infrared sensors are used, the intensity of infrared signals from the plurality of infrared sensors may be detected, and the intensity of infrared rays Bv_IR may be calculated from the maximum value.

In S02, the visible light intensity calculation unit 52 acquires the digital image data output by the image processing unit 60, and calculates the visible light intensity Bv_RGB. The intensity Bv_RGB of the visible light is the intensity Bv_RGB(x, y) of the R value (red), G value (green), and B value (blue) in the image signal of the pixel block obtained by arbitrarily dividing the light receiving surface of the image sensor 41. Calculated by accumulating and averaging.

In S03, the light source estimation unit 53 calculates the artificial illumination reliability α from the infrared intensity Bv_IR calculated by the infrared intensity calculation unit 51 in S01 and the visible light intensity Bv_RGB calculated by the visible light intensity calculation unit 52 in S02. calculate.

The artificial illumination reliability α is an index obtained by comparing the values of the infrared intensity Bv_IR and the visible light intensity Bv_RGB. When the infrared intensity Bv_IR is sufficiently small and the visible light intensity Bv_RGB is sufficiently large, the artificial light source reliability α is set to 1, and when the infrared intensity Bv_IR is sufficiently large, the artificial lighting reliability α is set to 0. α takes a value between 0 and 1, and when α is equal to or greater than a preset threshold value T (0<T≦1), it is estimated that the environment is under artificial light. The threshold T takes a preset value corresponding to the intensity Bv_RGB of visible light.

The artificial illumination reliability α may be calculated for each intensity Bv_RGB(x, y) in pixel blocks obtained by arbitrarily dividing the light receiving surface of the image sensor 41 . By calculating the artificial illumination reliability α for each pixel block, it is possible to estimate the light source for the area corresponding to each pixel block on the image sensor 41 within the shooting angle of view.

In S04, the light source estimation unit 53 compares the artificial illumination reliability α calculated in S03 with a preset threshold value T to estimate the light source. If the artificial lighting reliability α is equal to or greater than the preset threshold value T, it is assumed that the environment is under artificial lighting, and the process proceeds to S05. If the artificial lighting reliability α is smaller than the threshold value T, it is assumed that the environment is under natural light, and the process proceeds to S06.

In S05, the flicker detection unit 54 changes the readout frame rate of the image sensor 41 and performs flicker detection. A specific method of flicker detection in the present invention can employ a known technique. Using the data of the difference in the image data between the two frames at an interval of N + 0.5 times, it is determined whether the phases of the flicker fringes occurring in each of the images between the two frames are opposite phases, If the phases are opposite, there is a method of determining that the light of the flicker cycle to be detected is the cause of flicker.

When the flicker detection unit 54 detects flicker from the digital image data sent from the image processing unit 60, it changes the reading cycle and exposure time of the image sensor 41 in accordance with the flicker cycle. This avoids the influence of flickering on the through image data and the recorded image.

In S04, when it is estimated that it is under the natural light environment, flicker detection in S05 is not performed. In flicker detection, the timing control and reading speed change processing as described above are performed. Therefore, if the user checks the through image data on the image display unit 70 during flicker detection, the appearance will be poor due to the time lag due to the processing and the reduction in the update speed of the through image. It is not preferable because it deteriorates Therefore, when it is assumed that the shooting angle of view is in a natural light environment, it is assumed that flicker is not occurring in an artificial light environment, and by not performing flicker detection, it is possible to provide good-looking through-the-lens image data. becomes.

In S06, the white point distribution setting unit 55 sets the white point distribution in the shooting environment using the artificial lighting reliability α calculated in S03.

The white point distribution is defined by the color ratio R/G and color A distribution diagram obtained by calculating a ratio B/G and plotting the value of R/G on the horizontal axis and the value of B/G on the vertical axis, which is presumed to have been calculated based on a white image signal. It refers to the distribution range of color component information (R/G, B/G). In this embodiment, the white point distribution in the shooting environment is written as W(R/G, B/G).

FIG. 4 is a diagram for explaining the white point distribution under the natural light environment and the white point distribution under the artificial light environment. W _N in FIG. 4 is a white point distribution map under a natural light environment, and W _A is a white point distribution map under an artificial light environment. In each distribution diagram, the horizontal axis indicates the value of the color ratio R/G, and the vertical axis indicates the value of the color ratio B/G. Here, the black body locus is indicated by BR, and the white point area of each light source under the natural light environment and the artificial light environment is determined in advance according to the black body locus BR. These data are pre-stored in the ROM in the control unit 50, and differ depending on the sensitivity characteristics of the image sensor 41, the processing contents of the image processing unit 60, etc., and are set for each image capturing apparatus.

In FIG. 4, N1, N2, N3, and N4 are shown as white point areas of each light source in W _N , and A1 and A2 are shown as white point areas of each light source in W _A . In the white point distribution W _N (R/G, B/G) under the natural light environment and the white point distribution W _A (R/G, B/G) under the artificial light environment, each white point area of each light source is white. It is presumed to have been calculated based on the image signal. For example, the white point distribution W _N1 (R/G, B/G) under the N1 light source in the white point distribution diagram under the natural light environment is output as digital image data by the image processing unit 60 under the N1 light source under the natural light environment. white digital image data from the color ratio R/G and the color ratio B/G of the R value (red), G value (green), and B value (blue) from each pixel of the image sensor 41. It is an area where

The white point distribution setting unit 55 sets the artificial light source reliability α calculated in S03, the white point distribution W _N (R/G, B/G) under the natural light environment, and the white point distribution W _A (R/ G, B/G) is used to calculate the white point distribution W (R/G, B/G) in the shooting environment by the following equation.

According to the above formula, the white point distribution W (R/G, B/G) is divided into the white point distribution W _N (R/G, B/G) under the natural light environment and the white point distribution W _A (R /G, B/G) and weighted by the artificial light source reliability α calculated in S03. That is, since the light source is estimated by comparing the infrared light and the visible light in advance, it becomes easy to distinguish between the white area N1 under the natural light environment and the white area A1 under the artificial light environment, for example.

In S07, the white balance setting unit 56 sets the white balance gain based on the white point distribution W (R/G, B/G) calculated by the white point distribution setting unit 55 in S06.

The white balance gains (RGain, GGain, BGain) are the R value (red), G value (green) and B value (blue) (R , G, B) is a value for correcting the color when reflected by a white object to an achromatic color (that is, R=G=B).

The white balance setting unit 56 sets the R value (red), the G value (green), and the B value of each pixel of the image sensor 41 converted as digital image data by the image processing unit 60 as values for setting the white balance gain. The horizontal direction of (blue) is x-th, and the vertical direction is y-th. Based on the white point distribution W (R/G, B/G) in the shooting environment calculated by the white point distribution setting unit 55 in S06, a white image is obtained. Color component information calculated from the R value (red), G value (green), and B value (blue) of pixels (x, y) of the image sensor 41 included in the distribution range estimated to be output based on the signal (R _x,y /G _x,y , B _x,y /G _x,y ) is used for calculation according to the following formula.

Further, the white balance gain is set by the following formula based on Expression 2.

The image processing unit 60 converts the R value (red), G value (green), and B value (blue) (R, G, B) of each pixel of the image sensor 41 into digital image data, and converts them into digital image data. The white balance of the digital image signal is adjusted by multiplying the number 3 set by 56 .

As described above, according to the imaging apparatus and the light source estimation method according to the present invention, the infrared rays incident through the lens optical system are detected, and the light source estimation at the shooting angle of view is performed with high accuracy. can be improved.

10 imaging device 20 camera main body 21 camera side mount section 30 photographing lens 31 lens side mount section 32 lens optical system 41 imaging element 42 infrared reflection filter 43 infrared sensor 50 control section 60 image processing section 70 image display section

Claims (5)

In an imaging device equipped with a detachable photographing lens having a lens optical system,
An image sensor that detects light incident through the lens optical system, photoelectrically converts the detected light and outputs a signal, an infrared reflective filter arranged on the object side of the image sensor, and the lens optical system and the infrared reflective filter. and an infrared sensor arranged outside the optical path of light passing through the lens optical system, a control unit, and an image processing unit that converts the signal output by the image sensor into image data and outputs it,
The infrared reflective filter reflects the infrared rays out of the light incident through the lens optical system, the infrared sensor detects the infrared rays reflected by the infrared reflective filter, and the controller obtains the strength of the infrared signal detected by the infrared sensor. 1. An imaging apparatus, wherein a controller obtains, through an image processor, the intensity of a visible light signal detected by an imaging device through an infrared reflective filter. The control unit includes a light source estimation unit,
2. The light source estimation unit according to claim 1, wherein the light source estimation unit compares the intensity of the infrared signal and the intensity of the visible light signal acquired by the control unit, and calculates the index of the light source in the shooting environment based on the comparison result. imaging device. The control unit has a flicker detection unit,
The flicker detection unit changes the reading frame rate of the image pickup device when the index of the light source calculated by the light source estimation unit is greater than a threshold determined from the intensity of the visible light signal, and performs flicker detection on the image data output by the image processing unit. 3. The imaging device according to claim 2, wherein: The control unit includes a white point distribution setting unit and a white balance setting unit,
The white point distribution setting unit calculates the white point distribution based on the index of the light source calculated by the light source estimation unit for the image data output by the image processing unit, and the white balance setting unit calculates the white point distribution. 4. The imaging apparatus according to claim 2, wherein the white balance setting value is calculated and changed accordingly. The infrared reflective filter reflects the infrared ray out of the light incident through the lens optical system. In a white balance control method for an imaging device in which an imaging device detects visible light transmitted through an infrared reflective filter,
detecting infrared rays by an infrared sensor; detecting visible light by an imaging device; comparing the intensity of the infrared signal and the intensity of the visible light signal; performing flicker detection when the index of the light source is greater than a threshold determined from the intensity of visible light; calculating a white point distribution based on the index of the light source; and calculating a white point distribution based on the white point distribution A white balance control method, comprising: changing the white balance setting value to

Images