JP6272006B2 - Imaging apparatus, image processing method, and program - Google Patents

Description

  The present invention relates to an image pickup apparatus that picks up a slide material projected on a screen as a video in a scene such as a presentation performed in a conference room, an image processing method at the time of image pickup, and a computer to execute the image processing method. Related to the program.

  In an imaging apparatus such as a digital still camera or a digital video camcorder, exposure control is performed by calculating a photometric value from a luminance level in an input image signal. There are various methods for performing photometry using an input image signal. One of them is spot photometry, in which the brightness value of a part of an image is measured and exposure control is performed so as to obtain an appropriate brightness.

  For example, when shooting a scene such as a presentation in a conference room, the main subject is a material projected on a screen, and it is necessary to control the brightness so that the projection material can be easily seen. For this reason, it is preferable to use spot photometry that detects a high-luminance region (image or video projected on the screen) from the image and controls the brightness to be easy to see.

  At that time, for example, if the sun is reflected in the shooting screen as a high-luminance subject other than the image or video projected on the screen, spot metering cannot be performed correctly due to the influence of this high-luminance subject. , You may not be able to shoot at a brightness that is easy to see. Therefore, as a technique for solving such a problem, there has been proposed an imaging apparatus that performs photometry while excluding an area where a high-brightness object is reflected in a shooting screen (see, for example, Patent Documents 1 and 2).

JP 2002-277922 A JP 2001-45363 A

  However, the techniques described in Patent Documents 1 and 2 are based on the photometric value detected from the input image signal and determine whether the area is a high-luminance area or not. Judge by value. Therefore, there arises a problem that when the exposure state changes, it becomes impossible to accurately determine the high luminance region.

  An object of the present invention is to provide a technique capable of detecting a high luminance area other than the main subject with high accuracy and performing appropriate exposure control on the main subject.

An imaging apparatus according to the present invention divides an input image into a plurality of regions, and a photometric unit that acquires a photometric value for each of the plurality of divided regions, and for each of the plurality of regions based on a photometric value acquired by the photometric unit EV value calculating means for calculating an EV value at the same time, determination means for determining whether or not the EV value calculated for each of the plurality of areas is smaller than a predetermined threshold, and among the plurality of areas, When the input image is captured based on the EV value calculated by the EV value calculating unit and the photometric value measured by the photometric unit for the region determined to be an EV value smaller than the threshold by the determining unit. exposure control, white balance control, and an evaluation value calculation means for calculating an evaluation value used for at least one control of the focus control and gradation control, the evaluation value calculating means calculates, prior Detecting a maximum luminance region having the highest luminance value among the regions with threshold smaller EV value, the luminance value of the maximum luminance region detected and said evaluation value and to Rukoto used for the gradation control.

  According to the present invention, even if a high-luminance subject other than the main subject exists in the input image, it is possible to perform photometry correctly based on an absolute reference that does not depend on the exposure parameter of the main subject region. Thus, for example, when shooting a scene such as a presentation in a conference room by spot metering, the main subject such as an image or video projected on the screen is reflected in the input image with a high-luminance subject other than the main subject. Even if this happens, you can shoot at a brightness that is easy to see.

1 is a block diagram illustrating a schematic configuration of a digital video camcorder that is an example of an imaging apparatus according to an embodiment of the present invention. (A) is a figure which shows the example of the input image at the time of imaging | photography with the digital video camcorder of FIG. 1, (b) excluded the high-intensity area | regions other than the main photographic subject projected on the screen from the input image of (a). It is a figure which shows a state typically. 2 is a flowchart of processing for calculating a maximum value of an EV value in an input image, which is executed by an EV value calculation unit included in the digital video camcorder when shooting with the digital video camcorder of FIG. 1.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. Here, a digital video camcorder is taken up as an imaging apparatus according to the present invention, but the present invention is not limited to this.

<First Embodiment>
FIG. 1 is a block diagram showing a schematic configuration of a digital video camcorder which is an example of an imaging apparatus according to an embodiment of the present invention. The schematic configuration of the digital video camcorder shown in FIG. 1 is common not only to the first embodiment but also to the schematic configuration of each digital video camcorder according to the second to fourth embodiments described later.

  The digital video camcorder of FIG. 1 includes a system control unit 100, an optical system 101, an image sensor 102, an A / D conversion unit 103, a color channel (CH) gain control unit 104, a signal processing unit 105, and a gamma circuit 106. The arrows shown in FIG. 1 indicate the signal flow.

  The system control unit 100 controls the entire system of the digital video camcorder. Details of the system control unit 100 will be described later.

  The optical system 101 includes a diaphragm, an ND filter, a lens, and the like, and adjusts input light (subject light flux) that is reflected light from the subject. The input light adjusted by the optical system 101 forms an optical image on the light receiving surface of the image sensor 102. The image sensor 102 is, for example, a CMOS sensor, a CCD sensor, or the like, and performs an optical conversion on an optical image formed on the light receiving surface according to a shutter speed determined by the system control unit 100, whereby an analog electric signal ( Analog image signal).

  The A / D conversion unit 103 converts the analog electrical signal acquired from the image sensor 102 into a digital image signal that is a digital electrical signal in accordance with the gain control value determined by the system control unit 100. At this time, the A / D conversion unit 103 performs amplification processing of the digital image signal according to the gain value as necessary. For example, in the case of a dark subject, a control signal is output from the system control unit 100 to the A / D conversion unit 103 to amplify the digital image signal and increase the output signal level.

  The inter-color channel gain control unit 104 multiplies each color channel by a gain according to the color balance determined by the system control unit 100 with respect to the digital image signal acquired from the A / D conversion unit 103, thereby adjusting the black level. Correction and white balance (WB) adjustment are performed. The signal processing unit 105 performs processing to convert the digital image signal acquired from the color channel gain control unit 104 into image data such as a final storage image signal or display image signal. Further, the signal processing unit 105 outputs a luminance signal, a color signal, and the like of an image during or after image processing to the system control unit 100 in accordance with an instruction from the system control unit 100. The gamma circuit 106 converts the luminance characteristic of the image based on the gamma characteristic.

  The system control unit 100 includes a photometry unit 111, an EV value calculation unit 112, a memory 113, an evaluation value calculation unit 114, an exposure control unit 115, a focus control unit 116, a WB control unit 117, a gradation control unit 118, and a gamma control unit 119. Have

  The photometric unit 111 divides the input image into a plurality of regions, and acquires an average luminance value for each region and an average signal value for each color of RGB from the signal processing unit 105 as a photometric value. In the present embodiment (first to fourth embodiments), the average luminance value and the RGB average signal value as the color signal are used as the luminance signal as the photometric value as appropriate, but either the luminance signal or the color signal is used. Only one may be used. In addition to the average value, a maximum value, minimum value, median value, integral value, or the like may be used as the photometric value, and a luminance gradient amount or edge strength for each region may be used.

  The EV value calculation unit 112 acquires the reference sensitivity of the digital video camcorder main body held in the memory 113, acquires exposure parameters at the time of shooting from the exposure control unit 115, and normalizes the exposure amount to perform the reference EV. Calculate the value. In this normalization, a difference between the reference position of the exposure element and the current control position is obtained, and the obtained difference is increased or decreased with respect to the obtained reference sensitivity. That is, for the exposure element, when the exposure amount from the reference exposure state is large, the differential exposure amount is subtracted from the reference sensitivity, and when the exposure amount from the reference exposure state is small, the differential exposure amount is added to the reference sensitivity. . The exposure elements are iris, exposure time (electronic shutter), sensitivity (gain), ND filter, and the like.

  Thereby, the intensity of the reflected light (input light that forms an image on the image sensor 102 through the optical system 101) from the subject can be calculated as an EV value without depending on the exposure state. The EV value calculation unit 112 calculates the resolution of the photometric value in the calculated reference EV value, and calculates each of the divided photometric values of each of the regions, the difference from the predetermined reference photometric value, and the calculated resolution. The EV value of the area is calculated.

  The WB control unit 117 is described in detail in a second embodiment described later, the focus control unit 116 is described in detail in third embodiment, and the gradation control unit 118 and the gamma control unit 119 are described in detail in later-described fourth embodiment. Explained.

  The first embodiment is characterized by exposure control during shooting. That is, in the first embodiment, the evaluation value calculation unit 114 uses the EV value calculated by the EV value calculation unit 112 and the photometry value by the photometry unit 111 to obtain an evaluation value used for exposure control in the exposure control unit 115. calculate. For example, in a scene such as a presentation in a conference room, generally, an image / video area projected on a screen can be a high luminance area. Therefore, the evaluation value calculation unit 114 detects the maximum luminance region having the highest luminance value in each region, increases the weight of the luminance value of the detected maximum luminance region, and calculates the weighted average of the luminance values of each region. Calculated as an evaluation value. The exposure control unit 115 compares the evaluation value calculated by the evaluation value calculation unit 114 with a brightness target value that is easy to see as a captured image (captured video), and distributes the exposure change amount to the exposure element according to the comparison result. Thus, the optical system 101, the image sensor 102, and the A / D conversion unit 103 are controlled.

  FIG. 2A is a diagram showing an example of an input image at the time of shooting by the digital video camcorder according to the present embodiment. Here, it is assumed that a scene in which a document image, a document video, or the like is projected on the screen is captured as an input image.

  As shown in FIG. 2A, for example, when a high-luminance area such as the sun 202 exists in the screen 200 in addition to the image / video area 201 projected on the screen, in spot metering, other than the image / video area 201 The weight of the luminance value of the area becomes large, and the exposure cannot be controlled properly.

  Here, there is a limitation (standard) on the output of the projector that projects the presentation material, and an output range of 3000 lumens to 10,000 lumens is generally used. In addition, since the reflectance of the projected screen has a certain limit and the distance from the projector to the screen in use is also certain, the luminance level of the reflected light from the screen is generally 9 EV to 12 EV. Fits in range. Therefore, if there is a region that greatly exceeds the range of the EV value, for example, if there is a region having an EV value of 15 EV or more, it is considered that the region is not the image / video region 201 projected on the screen. Can do.

Therefore, by setting a threshold Th _EV for determining whether the EV value which can not take an image and video area 201 projected on the screen, image and video of the EV is projected the threshold Th _EV or more areas on the screen It is determined that the area is not 201. The threshold value Th _EV can be set to 13 _EV , for example.

  FIG. 3 is a flowchart of processing for calculating the maximum value of the EV value in the input image, which is executed by the EV value calculation unit 112 when shooting with the digital video camcorder. First, in step S301, the EV value calculation unit 112 selects a first (i = 1) region for obtaining an EV value. It is assumed that the input image is divided into n areas (n is an integer of 2 or more).

The EV value calculation unit 112 acquires the photometric value of the i-th region (region i) from the photometry unit 111 in step S302, and calculates the EV value of the region i in step S303. Then, in order to exclude the high brightness area other than the image / video area 201 projected on the screen from the calculation of the evaluation value, in step S304, the EV value calculation unit 112 determines that the EV value calculated in step S303 is the threshold value Th _EV. Or less. When the EV value of the region i is smaller than the threshold value Th _EV (YES in S304), the EV value calculation unit 112 advances the process to Step S305, and when the EV value of the region i is equal to or greater than the threshold value Th _EV (NO in S304), The process proceeds to step S307.

  In step S305, the EV value calculation unit 112 determines whether or not the EV value of the region i is larger than the maximum EV value obtained so far. It should be noted that the EV value of the region i that first proceeds to step S305 is set to the initial value of the maximum EV value. The EV value calculation unit 112 advances the process to step S306 when the EV value of the area i is the maximum value (YES in S305), and performs the process when the EV value of the area i is not the maximum value (NO in S305). Advances to step S307.

  In step S306, the EV value calculation unit 112 updates the maximum EV value. The EV value calculation unit 112 adds 1 to the value of i in step S307, and then determines whether or not the value of i is larger than the value of n in step S308. That is, in step S308, it is determined whether or not the processing in steps S302 to S306 has been performed for all n regions. If the EV value calculation unit 112 has not checked the EV values for all the regions (NO in S308), the process returns to step S302, and if the EV values have been checked for all the regions (YES in S308), this process is performed. Terminate.

  FIG. 2B is a diagram schematically illustrating a state in which a high luminance area other than the image / video area 201 which is the main subject projected on the screen is excluded from the input image illustrated in FIG. As shown in FIG. 2B, it can be seen from the determination in step S304 that the region in which the upper left sun 202 having an EV value of 15 EV is reflected is excluded.

When the region having the maximum EV value is identified by the processing according to the flowchart of FIG. 3, the evaluation value calculation unit 114 increases the weight of the photometric value of the region and calculates the weighted average of the luminance values of each region. Thus, an evaluation value for exposure control is calculated. Note that, instead of adopting a configuration in which an area exceeding the threshold Th _EV (= 13 _EV ) is not considered when calculating the evaluation value, the weight of the area may be set to zero (0).

  By performing exposure control as described above, it is possible to control the brightness of the image / video area 201 projected on the screen to an appropriate exposure, and it is possible to shoot with brightness that is easy to see.

Second Embodiment
The second embodiment is characterized by white balance control (WB control) during shooting. Specifically, in the digital video camcorder shown in FIG. 1, the evaluation value calculation unit 114 uses the EV value calculated by the EV value calculation unit 112 and the photometry value measured by the photometry unit 111 to use the WB in the WB control unit 117. An evaluation value used for control is calculated.

  As already described, in a scene such as a presentation in a conference room, generally, an image / video area projected on a screen can be a high luminance area. Therefore, the evaluation value calculation unit 114 detects the maximum luminance area having the highest luminance value in each area of the input image divided into a plurality of areas, and calculates the RGB ratio of the detected maximum luminance area.

However, at this time, as in the first embodiment, when there is a region where the EV value greatly exceeds the range of 9 EV to 12 EV (for example, the EV value is 15 EV or more), the image is projected on the screen.・ It is considered not to be a video area. Therefore, the evaluation value calculation unit 114 determines whether the EV value calculated for each area by the EV value calculation unit 112 is an EV value that cannot be taken as an image / video area projected on the screen, using the threshold Th _EV (for example, 13 _EV ). ). Then, the maximum luminance region is detected from the region where the EV value is smaller than the threshold value Th _EV .

  The evaluation value calculation unit 114 compares the RGB ratio calculated for the maximum luminance area with the target value of the RGB ratio that is easy to see as the captured image (captured video), and uses the difference from the target value as the evaluation value as the WB control unit 117. To communicate.

In the case of the input image of FIG. 2 (a), as shown in FIG. 2 (b), the evaluation value calculation unit 114 has a maximum luminance region in which the EV value takes the maximum value in a region where the EV value is smaller than the threshold Th _EV. Is detected. Then, the evaluation value calculation unit 114 calculates the RGB ratio of the detected maximum luminance region, and transmits the calculated ratio to the WB control unit 117. The WB control unit 117 calculates a gain change amount for each color channel according to the difference amount from the target value transmitted from the evaluation value calculation unit 114, and transmits the calculated change amount to the inter-color channel gain control unit 104. Thus, WB control is performed.

  By performing the WB control as described above, it is possible to appropriately control the WB of the image / video area 201 projected on the screen, and thus it is possible to take an image with an easy-to-see WB.

<Third Embodiment>
The third embodiment is characterized by focus control during shooting. Specifically, in the digital video camcorder shown in FIG. 1, the evaluation value calculation unit 114 uses the EV value calculated by the EV value calculation unit 112 and the photometric value measured by the photometry unit 111 to focus on the focus control unit 116. Edge strength is calculated as an evaluation value used for control.

  As described above, in a scene such as a presentation in a conference room, generally, an image / video area projected on a screen can be a maximum luminance area. Therefore, the evaluation value calculation unit 114 detects the maximum luminance area having the highest luminance value in each area of the input image divided into a plurality of areas, calculates the edge intensity of the detected maximum luminance area, and calculates it. The edge strength thus transmitted is transmitted to the focus control unit 116.

However, at this time, as in the first embodiment, when there is a region where the EV value greatly exceeds the range of 9 EV to 12 EV (for example, the EV value is 15 EV or more), the image is projected on the screen.・ It is considered not to be a video area. Therefore, the evaluation value calculation unit 114 determines whether the EV value calculated for each area by the EV value calculation unit 112 is an EV value that cannot be taken as an image / video area projected on the screen, using the threshold Th _EV (for example, 13 _EV ). ). Then, the maximum luminance region is detected from the region where the EV value is smaller than the threshold value Th _EV .

In the case of the input image of FIG. 2A, as shown in FIG. 2B, the evaluation value calculation unit 114 has a maximum brightness at which the EV value is maximum in a region where the EV value is smaller than the threshold Th _EV. Detect areas. Then, the evaluation value calculation unit 114 calculates the edge strength of the detected maximum luminance region, and transmits the calculated edge strength to the focus control unit 116. The focus control unit 116 searches for an in-focus position where the intensity of the edge intensity (luminance gradient) transmitted from the evaluation value calculation unit 114 reaches a peak while moving the focus lens included in the optical system 101 in the optical axis direction. Thus, focus control is performed.

  By performing the focus control as described above, it is possible to perform shooting while maintaining high focusing accuracy for the image / video area 201 projected on the screen.

<Fourth embodiment>
The fourth embodiment is characterized by gradation control during shooting. That is, in the digital video camcorder shown in FIG. 1, the evaluation value calculation unit 114 uses the EV value calculated by the EV value calculation unit 112 and the photometry value measured by the photometry unit 111 to perform gradation control in the gradation control unit 118. The evaluation value used for is calculated.

  As described above, in a scene such as a presentation in a conference room, generally, an image / video area projected on a screen can be a maximum luminance area. Therefore, the evaluation value calculation unit 114 detects the maximum luminance region having the highest luminance value in each region for the input image divided into a plurality of regions, and the luminance value of the detected maximum luminance region is a gradation control unit. 118.

However, at this time, as in the first embodiment, when there is a region where the EV value greatly exceeds the range of 9 EV to 12 EV (for example, the EV value is 15 EV or more), the image is projected on the screen.・ It is considered not to be a video area. Therefore, the evaluation value calculation unit 114 determines whether the EV value calculated for each area by the EV value calculation unit 112 is an EV value that cannot be taken as an image / video area projected on the screen, using the threshold Th _EV (for example, 13 _EV ). ). Then, the maximum luminance region is detected from the region where the EV value is smaller than the threshold value Th _EV .

In the case of the input image of FIG. 2A, as shown in FIG. 2B, the evaluation value calculation unit 114 has a maximum brightness at which the EV value is maximum in a region where the EV value is smaller than the threshold Th _EV. The area is detected, and the brightness value of the detected maximum brightness area is transmitted to the gradation control unit 118. The gradation control unit 118 determines the gradient of the gamma characteristic around the acquired luminance value and generates a gamma characteristic smoothly connected to the peripheral part by interpolation in order to give the acquired luminance value a gradation. The gradation control unit 118 transmits the generated gamma characteristic to the gamma control unit 119. The gamma control unit 119 performs gradation control by reading out the register setting value corresponding to the acquired gamma characteristic from the memory 113 and setting it in the register of the gamma circuit 106.

  In addition, since the luminance value from the evaluation value calculation unit 114 is high, there may be a case where sufficient gradation cannot be ensured only by the gamma characteristic. In this case, the gradation control unit 118 narrows the exposure through the exposure control unit 115 and further increases the dynamic range by controlling the gamma characteristics through the gamma control unit 119 according to the exposure change amount. Similarly, since the input luminance value is small, there may be a case where sufficient gradation cannot be ensured only by the gamma characteristic. In this case, the gradation control unit 118 reduces the dynamic range by increasing the exposure through the exposure control unit 115 and further controlling the gamma characteristics through the gamma control unit 119 according to the exposure change amount.

  By performing the gradation control as described above, it is possible to shoot with an easy-to-view gradation that matches the image / video area 201 projected on the screen.

<Other embodiments>
Although the present invention has been described in detail based on preferred embodiments thereof, the present invention is not limited to these specific embodiments, and various forms within the scope of the present invention are also included in the present invention. included. Furthermore, each embodiment mentioned above shows only one embodiment of this invention, and it is also possible to combine each embodiment suitably.

  The present invention can also be realized by executing the following processing. That is, software (program) that realizes the functions of the above-described embodiments is supplied to a system or apparatus via a network or various storage media, and a computer (or CPU, MPU, etc.) of the system or apparatus reads the program code. It is a process to be executed. In this case, the program and the storage medium storing the program constitute the present invention.

  For example, when the present invention is implemented as image processing software that can be used by a computer, data stored as meta information in an image file is used as an exposure parameter in EV value calculation. In addition, regarding the reference sensitivity information of the imaging apparatus main body, if the reference sensitivity information includes only information that is common among the models of the imaging apparatus, it is stored in advance as data information of the image processing software, or includes individual difference information. As with the exposure parameter, it may be held as meta information attached to the image file.

DESCRIPTION OF SYMBOLS 100 System control part 101 Optical system 102 Image pick-up element 104 Color channel gain control part 105 Signal processing part 106 Gamma circuit 111 Photometry part 112 EV value calculation part 114 Evaluation value calculation part 115 Exposure control part 116 Focus control part 117 WB control part 118 Gradation control unit 119 Gamma control unit

Claims (7)

A photometric means for dividing the input image into a plurality of areas and obtaining a photometric value for each of the divided areas;
EV value calculating means for calculating an EV value for each of the plurality of regions based on the photometric value acquired by the photometric means;
Determination means for determining whether an EV value calculated for each of the plurality of areas is smaller than a predetermined threshold;
Based on the EV value calculated by the EV value calculating means and the photometric value measured by the photometric means for the area determined by the determining means to be an EV value smaller than the threshold value among the plurality of areas, Evaluation value calculation means for calculating an evaluation value used for at least one of exposure control, white balance control, focus control, and gradation control when photographing the input image ,
The evaluation value calculating means, the detected maximum luminance region having the highest luminance value among the regions with threshold smaller EV value shall be the evaluation value using the luminance value of the maximum luminance region detected in the gradation control An imaging apparatus characterized by that.   The photometric unit is configured to measure an average value, a maximum value, a minimum value, or a median value of at least one of the luminance signal and the color signal for each of the plurality of divided areas, or an edge intensity for each of the plurality of areas. The imaging device according to claim 1, wherein the imaging device is acquired as:   The EV value calculating means calculates a reference EV value by calculating a difference of the current control position with respect to a reference position of the exposure element, and increasing / decreasing the difference of the control position with respect to the reference sensitivity of the imaging device. The imaging apparatus according to claim 1, wherein an EV value for each of the plurality of regions is calculated based on a reference EV value and a photometric value calculated by the photometric unit. A gradation control means for determining a gradient of the gamma characteristic around the luminance value acquired from the evaluation value calculating means, and generating a gamma characteristic smoothly connected to the peripheral part;
The imaging apparatus according to any one of claims 1 to 3, characterized in that it comprises converting means for converting the luminance characteristics of the input image based on the gamma characteristic in which the gradation control means is generated. Exposure control means for controlling exposure based on the brightness value of the maximum brightness region;
The imaging apparatus according to claim 4, wherein the gradation control unit generates the gamma characteristic in accordance with an exposure change amount by the exposure control unit. A division step of dividing the input image into a plurality of regions;
A photometric step of obtaining a photometric value including at least a luminance value for each of the plurality of regions divided by the dividing step;
An EV value calculating step for calculating an EV value for each of the plurality of regions based on the photometric value acquired in the photometric step;
A determination step of determining whether or not the EV value calculated for each of the plurality of regions in the EV value calculation step is smaller than a predetermined threshold;
A detection step of detecting a maximum luminance region having the highest luminance value among the regions determined to have an EV value smaller than the threshold value by the determination step;
At least one of exposure control, white balance control, focus control, and gradation control when photographing the input image based on the photometric value measured by the photometric step for the maximum brightness area detected in the detecting step. One of have a evaluation value calculation step of calculating an evaluation value used for control,
In the evaluation value calculation step, a maximum luminance region having the highest luminance value is detected from among regions having an EV value smaller than the threshold value, and the luminance value of the detected maximum luminance region is set as an evaluation value used for the gradation control. An image processing method characterized by the above. A program for causing a computer to execute the image processing method according to claim 6 .