JP2013113911A - Exposure control device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a new, improved exposure control device able to exert exposure control more appropriately than conventional ones.SOLUTION: According to a certain standpoint of the present invention, in order to solve the problem, the exposure control device is provided, which comprises: a picked-up image capturing part that captures a picked-up image; a block dividing part that divides the picked-up image into blocks; a feature quantity calculating part that calculates a feature quantity including a luminance for each block; a complexity calculating part that calculates the complexity of each block on the basis of the feature quantity of each block; a corrected luminance calculation part that calculates the corrected luminance of each block by weighting the luminance of each block on the basis of the complexity of each block; and an exposure control part that controls exposure for imaging, based on the corrected luminance average value.

Description

  The present invention relates to an exposure control device.

  An imaging device that calculates the average luminance after weighting the luminance of the central area of the captured image and controls exposure so that the average luminance approaches the target luminance, an imaging device that performs so-called center-weighted photometry is known. . The reason for controlling the exposure in this manner is that it is considered that there are many cases where the subject to be imaged is drawn in the central region of the captured image.

JP 2006-106617 A Japanese Unexamined Patent Publication No. 2009-164677 JP 2007-158995 A

  However, with this technique, when the subject is drawn out of the center area, appropriate exposure cannot be obtained for the subject. Even if the central area of the captured image is weighted, the average luminance is strongly influenced by the luminance of the background area other than the central area. This is because the background area has a larger area than the central area. For this reason, when the background area has a uniformly high luminance with respect to the central area (for example, when shooting a person with a white wall in the background), the average luminance affects the luminance of the background area. I will receive it. In this case, since the conventional imaging device performs control to lower the average luminance, a captured image with a generally insufficient exposure has been generated.

  On the other hand, when the background area has a uniformly low luminance with respect to the central area (for example, when shooting a person with a black wall in the back), the average luminance is affected by the luminance of the background area. End up. In this case, since the conventional imaging device performs control to increase the average luminance, a captured image that is excessively exposed as a whole has been generated. In this captured image, for example, the dark part may be unnaturally bright (floating), and the human face may be saturated.

  Patent Documents 1 to 3 disclose techniques related to exposure control. The technique disclosed in Patent Document 1 extracts a high luminance group having a luminance of a predetermined value or more from a captured image, and controls exposure based on the luminance and area of the high luminance group. However, with this technique, the white wall described above belongs to the high-intensity group, so when shooting a person with the white wall on the back, a captured image that is also generally underexposed is generated. It was. Therefore, this technique cannot solve the above-described problem.

  The technique disclosed in Patent Literature 2 extracts a face image from a captured image by performing a so-called face detection process on the captured image, and controls exposure based on the brightness of the face image and its surroundings. However, this technique requires processing with high calculation cost such as face detection. Furthermore, Patent Document 2 did not disclose anything about how to control the balance of the brightness of the face image and its surroundings. Therefore, with this technique, as in the above-described example, when a person with a white wall on the back is photographed, the brightness balance between the person's face image and the background image cannot be adjusted appropriately. Therefore, even with this technique, the above-described problem cannot be solved.

  The technique disclosed in Patent Document 3 divides a captured image into a plurality of blocks, and calculates a luminance difference between adjacent blocks. And this technique detects the block of backlight from the block from which a luminance difference becomes more than predetermined value. Therefore, since this technique is effective only during backlighting, the above-described problem cannot be solved.

  Accordingly, the present invention has been made in view of the above problems, and an object of the present invention is to provide a new and improved exposure control apparatus capable of performing exposure control more appropriate than conventional ones. There is to do.

  In order to solve the above-described problem, according to an aspect of the present invention, a captured image acquisition unit that acquires a captured image, a block division unit that divides the captured image into a plurality of blocks, and a feature amount including luminance for each block A weight calculation unit for calculating the complexity of each block based on the feature amount of each block, and a weight for the luminance of each block based on the complexity of each block. There is provided an exposure control device comprising: a corrected luminance calculating unit that calculates a corrected luminance of each block; and an exposure control unit that controls exposure during imaging based on the corrected luminance.

  According to this aspect, the exposure control device calculates the complexity of each block based on the feature amount of each block. Then, the exposure control device calculates the corrected luminance of each block by weighting the luminance of each block based on the complexity of each block. Then, the exposure control device controls the exposure at the time of imaging based on the corrected luminance. As a result, the exposure control apparatus can reduce the correction luminance of a region having a uniform luminance, for example, a background region, and can increase the correction luminance of a region in which the luminance changes rapidly, for example, a person region. Therefore, the exposure control device can control the exposure using a wider dynamic range, in particular, the corrected luminance of the person area.

  Here, the block dividing unit calculates the pixel integrated value by integrating the pixel values for each block, and the feature amount calculating unit calculates the feature amount of each block based on the pixel integrated value of each block. May be.

  According to this aspect, since the exposure control device calculates the feature amount of each block based on the pixel integrated value, the complexity of each block can be calculated more accurately.

  Further, the complexity calculation unit may calculate the complexity of each block based on a difference in feature amount between adjacent blocks.

  According to this aspect, since the exposure control device calculates the complexity of each block based on the difference in the feature amount between adjacent blocks, the complexity of each block can be calculated more accurately.

  In addition, the feature amount calculation unit calculates a plurality of types of feature amounts for each block, and the complexity calculation unit calculates the complexity of each block for each type of feature amount, and calculates the complexity for each type of feature amount. By mixing each block, the mixed complexity of each block may be calculated, and the corrected luminance calculation unit may weight the luminance of each block based on the mixed complexity of each block.

  According to this aspect, the exposure control device calculates a plurality of types of feature amounts for each block. Then, the exposure control device calculates the complexity of each block for each type of feature quantity, and calculates the mixed complexity of each block by mixing these complexity for each block. Then, the exposure control device weights the luminance of each block based on the mixed complexity of each block. Thereby, the exposure control apparatus can calculate the corrected luminance more accurately.

  In addition, the complexity calculation unit may complement the complexity of a face area in which a human face is drawn in each block.

  According to this aspect, the exposure control device can more accurately control the exposure of the face area.

  The corrected luminance calculation unit may calculate an average luminance that is an average value of the corrected luminances, and the exposure control unit may control the exposure at the time of imaging so that the average luminance approaches a predetermined target luminance.

  According to this aspect, the exposure control device reduces the influence of the uniform luminance area, for example, the background area, on the average luminance, and reduces the influence of the area where the luminance changes rapidly, for example, the person area, on the average luminance. be able to. Therefore, the exposure control device can control the exposure using a wider dynamic range, in particular, the corrected luminance of the person area.

  According to another aspect of the present invention, a captured image acquisition step for acquiring a captured image, a block division step for dividing the captured image into a plurality of blocks, and a feature amount calculation for calculating a feature amount including luminance for each block A step of calculating the complexity of each block based on the feature amount of each block; and the corrected luminance of each block by weighting the luminance of each block based on the complexity of each block There is provided an exposure control method comprising: a corrected luminance calculating step for calculating an image, and an exposure control step for controlling an exposure at the time of imaging based on the corrected luminance.

  The exposure control method according to this aspect calculates the complexity of each block based on the feature amount of each block. Then, the exposure control method calculates the corrected luminance of each block by weighting the luminance of each block based on the complexity of each block. The exposure control method controls the exposure at the time of imaging based on the corrected luminance. As a result, the exposure control method can reduce the correction luminance of a region having a uniform luminance, for example, a background region, and can increase the correction luminance of a region where the luminance change is sharp, for example, a person region. Therefore, the exposure control method can control the exposure using a wider dynamic range, particularly the corrected luminance of the person area.

  As described above, according to the present invention, the correction brightness of the background area is reduced while the correction brightness of the person area is increased. Therefore, the exposure control apparatus uses a wider dynamic range, particularly the correction brightness of the person area. The exposure can be controlled. Therefore, the exposure control device can perform exposure control more appropriate than before.

1 is a block diagram illustrating a configuration of an imaging apparatus according to a first embodiment of the present invention. It is explanatory drawing which shows an example of the complexity map produced | generated by the imaging device. It is explanatory drawing which shows an example of the complexity map produced | generated by the imaging device. It is explanatory drawing which shows an example of the complexity map produced | generated by the imaging device. It is explanatory drawing which shows the production | generation process of a complexity map. (A) It is explanatory drawing which shows an example of the hue integrated value map which shows the hue integrated value of each block. (B) It is explanatory drawing which shows an example of the hue complexity map which shows the hue complexity of each block. It is explanatory drawing which shows an example of the normalized hue complexity map which is a normalized hue complexity map. It is explanatory drawing which shows an example of a normalization saturation complexity map. It is explanatory drawing which shows an example of a normalization brightness complexity map. It is explanatory drawing which shows an example of the mixing complexity map which shows the mixing complexity of each block. It is a flowchart which shows the procedure of the process by an imaging device. It is a graph which shows the correspondence of the size of a skin color area | region, and the effective area | region considered as a person's face. It is explanatory drawing which shows an example of an effective area map. It is explanatory drawing which shows an example of a captured image. (A), (b) It is explanatory drawing for demonstrating the content of the face area determination by 2nd Embodiment. (A) It is explanatory drawing which shows an example of the mixing complexity map before correction | amendment. (B) It is explanatory drawing which shows an example of the mixed complexity map after correction | amendment. 10 is a flowchart illustrating a processing procedure performed by the imaging apparatus according to the second embodiment. It is explanatory drawing which shows an example of a captured image typically. It is a histogram which shows the luminance distribution after performing exposure correction | amendment with respect to the captured image of FIG. It is explanatory drawing which shows an example of a captured image typically. FIG. 21 is a histogram showing a luminance distribution after performing exposure correction on the captured image of FIG. 20. FIG. It is explanatory drawing which shows an example of a captured image typically. It is a histogram which shows the luminance distribution after performing exposure correction | amendment with respect to the captured image of FIG. It is explanatory drawing which shows an example of a captured image typically. It is a histogram which shows the luminance distribution after performing exposure correction | amendment with respect to the captured image of FIG.

  Exemplary embodiments of the present invention will be described below in detail with reference to the accompanying drawings. In addition, in this specification and drawing, about the component which has the substantially same function structure, duplication description is abbreviate | omitted by attaching | subjecting the same code | symbol.

<1. Problems with center-weighted metering>
The inventor has studied the problems of a conventional imaging device, specifically, an imaging device that performs center-weighted photometry, and as a result, has arrived at the imaging device according to the present embodiment. First, the problems of the conventional imaging device will be described in detail.

  The conventional imaging device calculates the average luminance after weighting the luminance of the central area of the captured image, and controls the exposure so that the average luminance approaches the target luminance. This target luminance is set to about medium luminance (gray color luminance), for example. In center-weighted metering, the center area of the captured image is weighted, but the average brightness is strongly influenced by the brightness of the background area other than the center area.

  When a person with a gray wall on the back is photographed using a conventional imaging device, for example, a captured image 300 shown in FIG. 18 is obtained. In the captured image 300, a human face is drawn in the area A1, a human torso is drawn in the area B1, and a background is drawn in the area C1. The brightness of the area C1 is uniform and close to the brightness of the areas A1 and B1. The conventional imaging device calculates the average brightness of the captured image 300 and controls the exposure so that the average brightness approaches the target brightness. The captured image after the exposure control shows, for example, the luminance distribution shown in FIG. In FIG. 19, the horizontal axis indicates luminance, and the vertical axis indicates frequency, that is, the number of pixels having each luminance. ThL indicates the target luminance. The histogram LA1 indicates the luminance distribution of the region A1, the histogram LB1 indicates the luminance distribution of the region B1, and the histogram LC1 indicates the luminance distribution of the region C1. In the case of this example, since the brightness of the area C1, which is the background area, is close to the target brightness, appropriate exposure is performed.

  On the other hand, when a person with a black wall is photographed using a conventional imaging device, a captured image 310 shown in FIG. 20 is obtained, for example. In the captured image 310, a person's face is drawn in the area A2, a person's torso is drawn in the area B2, and a background is drawn in the area C2. The brightness of the area C2 is uniform and lower than the brightness of the areas A2 and B2. In this example, the brightness of the area C2 as the background area is lower than the target brightness, and the brightness of the area C2 strongly affects the average brightness. That is, the average luminance is smaller than the target luminance due to the influence of the luminance of the region C2. For this reason, the conventional imaging apparatus performs control to increase the average luminance in order to bring the average luminance close to the target luminance. Therefore, the conventional imaging device cannot reflect the brightness of the areas A2 and B2 in the exposure control. In other words, the conventional imaging apparatus cannot effectively use the dynamic range of the captured image.

  The captured image after the exposure control shows, for example, the luminance distribution shown in FIG. In FIG. 21, the horizontal axis indicates luminance, and the vertical axis indicates frequency, that is, the number of pixels having each luminance. ThL indicates the target luminance. The histogram LA2 indicates the luminance distribution of the region A2, the histogram LB2 indicates the luminance distribution of the region B2, and the histogram LC2 indicates the luminance distribution of the region C2. As shown in FIG. 21, the brightness of the areas A2 and B2 is increased by being dragged by the brightness of the area C2, so that the brightness of the area A2 is saturated (that is, whiteout occurs). In addition, the luminance of the region C2 becomes unnaturally high.

  On the other hand, when a person with a white wall on the back is photographed using a conventional imaging device, for example, a captured image 320 shown in FIG. 22 is obtained. In the captured image 320, a person's face is drawn in the area A3, a person's torso is drawn in the area B3, and a background is drawn in the area C3. The brightness of the area C3 is uniform and higher than the brightness of the areas A3 and B3. In the case of this example, the luminance of the background region C3 is higher than the target luminance, and the luminance of the region C3 strongly affects the average luminance. That is, the average luminance is higher than the target luminance due to the influence of the luminance of the region C3. For this reason, the conventional imaging apparatus performs control to lower the average luminance in order to bring the average luminance closer to the target luminance. Therefore, also in this case, the conventional imaging device cannot reflect the brightness of the areas A3 and B3 in the exposure control. In other words, the conventional imaging apparatus cannot effectively use the dynamic range of the captured image.

  The captured image after the exposure control shows, for example, the luminance distribution shown in FIG. In FIG. 23, the horizontal axis indicates luminance, and the vertical axis indicates frequency, that is, the number of pixels having each luminance. ThL indicates the target luminance. The histogram LA3 indicates the luminance distribution of the region A3, the histogram LB3 indicates the luminance distribution of the region B3, and the histogram LC3 indicates the luminance distribution of the region C3. As shown in FIG. 23, the luminances of the regions A3 and B3 are reduced by the luminance of the region C3, so that the luminances of the regions A3 and B3, that is, the exposure is insufficient.

  Next, consider a more general scene. When a person standing on the ground with snow is photographed, for example, a captured image 330 shown in FIG. 24 is obtained. In this captured image 330, the ground with snow and the clear sky are drawn in a relatively wide area as a background. The brightness of these areas, that is, the background area is substantially uniform and higher than the brightness of the person area. In this example, the brightness of the background area is higher than the target brightness, and the brightness of the background area strongly affects the average brightness. That is, the average luminance is higher than the target luminance due to the influence of the luminance of the background area. For this reason, the conventional imaging apparatus performs control to lower the average luminance in order to bring the average luminance closer to the target luminance. Therefore, also in this case, the conventional imaging device cannot reflect the brightness of the person area in the exposure control. In other words, the conventional imaging apparatus cannot effectively use the dynamic range of the captured image.

  The captured image after the exposure control shows, for example, the luminance distribution shown in FIG. In FIG. 25, the horizontal axis indicates luminance, and the vertical axis indicates frequency, that is, the number of pixels having each luminance. The histogram L10 shows the luminance distribution of the captured image 330. In particular, the peak portion L10a indicates the luminance distribution of the background region. As shown in FIG. 25, since the brightness other than the background area, for example, the person area brightness is decreased by being dragged by the brightness of the background area, for example, the brightness of the person area, that is, the exposure is insufficient.

  As described above, the conventional imaging apparatus simply controls the average luminance to be close to the target luminance without considering the type of subject drawn in each area of the captured image. The average luminance is strongly influenced by the background area in the captured image. For this reason, in a conventional imaging device, an actual subject with high luminance (such as when a human directly looks at the subject) is drawn as a background area, such as a clear sky or snowy ground. In addition, control for lowering the average brightness was performed. That is, the conventional imaging apparatus controls the exposure of the luminance of the background area in the direction opposite to the original exposure direction (the direction in which the exposure is increased). Further, in the conventional image pickup apparatus, the exposure is greatly changed as the background area is wider. Therefore, the larger the background area, the greater the difference between the brightness of the captured image and the actual brightness of the subject.

  For this reason, the conventional imaging apparatus has a problem that the captured image is underexposed or overexposed even though the dynamic range is not used up. An object of the present embodiment is to provide an imaging device (exposure control device) that can perform exposure more appropriately than in the past.

<2. Overview of Processing According to First Embodiment>
Next, an overview of processing according to the first embodiment will be described. The imaging apparatus 10 (see FIG. 1) according to the first embodiment divides a captured image into a plurality of blocks (for example, 16 × 16 blocks), and calculates a feature amount such as luminance for each block. The imaging device 10 calculates a difference between feature amounts of adjacent blocks, and calculates the complexity of each block based on the absolute value of the difference. And the imaging device 10 produces | generates the complexity map which shows the complexity of each block. For example, the imaging device 10 generates the complexity map 100 illustrated in FIG. 2 for the captured image 300 illustrated in FIG. In the complexity map 100, the complexity is set for each block 101. The degree of complexity is indicated by the hatching density of each block 101. The greater the complexity, the greater the hatch density (closer to black). For example, since the block 101a is arranged at the boundary between the region B1 and the region C1, the complexity increases. On the other hand, since the block 101b is arranged in the region C1, the complexity is reduced.

  Similarly, the imaging apparatus 10 generates a complexity map 110 illustrated in FIG. 3 for the captured image 310 illustrated in FIG. In the complexity map 110, the complexity is set for each block 111. For example, since the block 111a is arranged at the boundary between the region B2 and the region C2, the complexity increases. Note that the imaging apparatus 10 also generates a map similar to the complexity map 110 for the captured image 320 illustrated in FIG. This is because the captured image 310 and the captured image 320 have different feature amounts in the regions C2 and C3, which are background regions, but the absolute value of the difference between the feature amounts is the same value in each block.

  More specifically, the imaging apparatus 10 generates a complexity map 130 illustrated in FIG. 4 for the captured image 330 illustrated in FIG. In the complexity map 130, the complexity is set for each block 131. In the complexity map 130 shown in FIG. 4, a numerical value indicating the complexity is described in each block. Each block 131 is hatched with a density corresponding to the complexity. In this example, the complexity of the background area (area where the sky and the ground are drawn) having a large luminance and a large area is small.

  Then, the imaging device 10 calculates the corrected luminance by weighting the luminance of each block based on the complexity. The imaging device 10 increases the weight as the complexity increases. The imaging apparatus 10 sets the average value of the corrected luminances as the average luminance, and controls the exposure so that the average luminance approaches the target luminance.

  Therefore, the correction brightness of the area where the brightness is uniform, for example, the background area is small, and the correction brightness of the area where the brightness is drastically changed, for example, the person area, is large. The influence of the person area on the average luminance is increased. In other words, when the vertical axis of the histogram shown in FIG. 25 or the like is (frequency) × (corrected luminance), the histogram becomes more flat. That is, the value of (frequency) × (corrected luminance) for each luminance in the histogram is more evenly distributed around the target luminance. For this reason, the imaging device 10 can calculate the average luminance using a wider dynamic range and can control the exposure. Since the luminance of the background area of the captured image 300 is close to the target luminance, the complexity indicated by the complexity map 100 is smaller than the complexity indicated by the other complexity maps 110 and 130. Accordingly, the corrected luminance of each block 101 is approximately the same as the luminance before correction, and thus the exposure amount is approximately the same as in the conventional case. Therefore, the side effects according to this embodiment do not occur.

<3. Configuration of Imaging Device>
Next, the configuration of the imaging device (exposure control device) 10 according to the first embodiment will be described based on the block diagram shown in FIG. The imaging device 10 includes a lens unit 11, an imaging device 12, an analog processing unit 13, a signal processing unit 14, a block division unit 15, a feature amount calculation unit 16, a complexity calculation unit 17, and a corrected luminance calculation. A unit 18, an exposure amount calculation unit 19, an exposure control unit 20, and a driver 21 are provided.

  Note that the photographing apparatus 10 has hardware such as a CPU, ROM, RAM, and flash memory, and the CPU reads and executes the program stored in the ROM, thereby realizing the processing by the above functional blocks. Is done. That is, in the ROM, the imaging device 10 includes the above functional blocks, particularly the analog processing unit 13, the signal processing unit 14, the block dividing unit 15, the feature amount calculating unit 16, the complexity calculating unit 17, and the correction. A program for realizing the luminance calculation unit 18, the exposure amount calculation unit 19, the exposure control unit 20, and the driver 21 is stored.

  As other embodiments, an exposure control apparatus (for example, an exposure control unit 20 including a block dividing unit 15, a feature amount calculating unit 16, a complexity calculating unit 17, a corrected luminance calculating unit 18, an exposure amount calculating unit 19, and an exposure control unit 20). A computer such as a personal computer or a server) can be considered. In this embodiment, the exposure control device acquires a captured image from the imaging device.

  The lens unit 11 is detachable from the photographing apparatus 10 and includes a plurality of lenses, a diaphragm mechanism, a shutter, and the like. The lens unit 11 captures visible light from the outside of the photographing apparatus 10 and causes the captured light to enter the image sensor 12 via a lens or the like.

  The image sensor 12 includes a plurality of unit elements arranged in a matrix and an image sensor control unit (not shown). The unit element corresponds to a unit pixel of the captured image. Each unit element outputs R information, G information, and B information (hereinafter collectively referred to as “RGB information”) relating to the reception intensity of each of red light, green light, and blue light to the image sensor control unit. The imaging element control unit generates a captured image having RGB information for each pixel based on the RGB information given from each unit element. The imaging element control unit outputs the generated captured image to the analog processing unit 13.

  The analog processing unit 13 performs analog / digital conversion on the captured image given from the image sensor 12 and outputs the converted captured image, that is, RAW data of the captured image, to the signal processing unit 14 and the block dividing unit 15. The signal processing unit 14 performs various correction processes (such as optical black removal, white balance adjustment, color reproduction correction, and gamma correction) on the captured image, and the captured image after the correction process is stored in the flash memory of the image sensor 10 or the like. To remember.

  The block dividing unit 15 divides the captured image into a plurality of blocks (for example, 16 × 16 blocks), and assigns an identification number to each block. Then, the block dividing unit integrates R information, G information, and B information for each block, so that an R integrated value, a G integrated value, and a B integrated value (hereinafter, collectively referred to as “RGB integrated value”). Is calculated. Further, the block dividing unit calculates the luminance of each block based on these integrated values and the following equation (1).

  In equation (1), Y is the luminance of the block, and R, G, and B are the R integrated value, G integrated value, and B integrated value, respectively. The block dividing unit 15 generates an integrated value map in which the RGB integrated value and the block identification number are associated with each other, and outputs the integrated value map to the feature amount calculating unit 16. Further, the block dividing unit 15 generates a luminance map in which the luminance and the block identification number are associated, and outputs the luminance map to the feature amount calculating unit 16 and the corrected luminance calculating unit 18.

  The feature amount calculation unit 16 calculates the hue, saturation, and log conversion luminance as the feature amount of each block based on the integrated value map and the luminance map and the following formulas (2) to (5).

  In formulas (2) and (3), H is a hue and takes a value of 0 to 360. The hue of H = 0 and the hue of H = 360 are the same. That is, the hue value circulates. R, G, and B are an R integrated value, a G integrated value, and a B integrated value, respectively. MAX is the maximum value among the RGB integrated values, and MIN is the minimum value among the RGB integrated values.

  In equation (4), S is the saturation, and MAX and MIN are the same as in equations (2) and (3). Saturation takes a value from 0 to 1.

  In Expression (5), L is the log conversion luminance, and Y is the luminance. In the first embodiment, the luminance is log-converted so that the luminance difference calculated by the complexity calculating unit 17 described later approximates the luminance difference perceived by humans, regardless of the luminance value. Of course, the complexity calculation unit 17 may calculate the luminance difference using the luminance obtained by the equation (1) as it is. In this case, the difference between the luminance difference calculated in the high luminance region and the luminance difference perceived by humans is larger than the difference in the low luminance region. The feature amount calculation unit 16 includes a hue map in which the hue of the block and the block identification number are associated, a saturation map in which the saturation of the block and the block identification number are associated, and the log conversion luminance of the block and the block The log conversion luminance map associated with the identification number is generated and output to the complexity calculation unit 17.

  The complexity calculation unit 17 calculates the hue complexity, the saturation complexity, and the luminance complexity of each block based on the hue map, the saturation map, and the log conversion luminance map. Specifically, the complexity calculating unit 17 calculates the difference between the feature amounts of each block and four blocks adjacent to the block in the up, down, left, and right directions, and calculates the complexity of each feature amount based on the difference. calculate. For example, the complexity calculation unit 17 calculates the complexity of the block 31a illustrated in FIG. 5 based on the following equations (6) to (9).

  In equations (6) and (7), CH is the hue complexity, and A1 to E1 are the hues of the blocks 31a to 31e shown in FIG.

  In Expression (8), CS is the saturation complexity, and A2 to E2 are the saturations of the blocks 31a to 31e shown in FIG.

  In Equation (9), CL is the luminance complexity, and A3 to E3 are the log conversion luminances of the blocks 31a to 31e shown in FIG.

  The complexity calculation unit 17 calculates the hue complexity, the saturation complexity, and the luminance complexity of each block by performing the processing described above for each block. Then, the complexity calculation unit 17 generates a hue complexity map indicating the hue complexity of each block, a saturation complexity map indicating the saturation complexity of each block, and a luminance complexity map indicating the luminance complexity of each block. Generate.

  For example, when the hue map 40 illustrated in FIG. 6A is generated, the complexity calculation unit 17 generates the hue complexity map 50 illustrated in FIG. Among the blocks of the hue map 40, the blocks around the frame 42 have a substantially uniform hue (200 to 220), whereas the blocks in the frame 42, particularly the block 41, are blocks of the blocks around the frame 42. It has a hue (120) of about 3/5. Therefore, the hue complexity of the block surrounded by the frame 52 in the hue complexity map 50, particularly the block 51, is larger than the hue complexity around the frame 52. The complexity calculating unit 17 may calculate the complexity based on the difference from the feature amounts of the eight blocks adjacent in the up / down / left / right and diagonal directions.

  The complexity calculator 17 normalizes the complexity of each block based on the following equations (10) to (12). Thereby, the complexity calculator 17 generates a normalized hue complexity map, a normalized saturation complexity map, and a normalized luminance complexity map.

  In Expression (10), IH (n) is the normalized hue complexity of the block with the identification number n, CH (n) is the hue complexity of the block with the identification number n, and IMAX is the maximum saturation complexity after the normalization. Value, which is set in advance to a desired value. PH is a normalization coefficient of hue complexity, and is set to a desired value in advance. max (CH) is the maximum value among the hue complexity levels indicated by all the blocks of the hue complexity map, that is, the maximum value of the hue complexity level. max (PH, max (CH)) is a larger value of PH and max (CH). Therefore, the complexity calculation unit 17 normalizes the hue complexity of each block with the maximum value of the hue complexity, but when the maximum value of the hue complexity is extremely small, normalization is performed with a fixed value PH. Will be.

  In equation (11), IS (n) is the normalized saturation complexity of the block with identification number n, CS (n) is the saturation complexity of the block with identification number n, and IMAX is the saturation complexity after normalization. Is set to a desired value in advance. PS is a normalization coefficient of saturation complexity, and is set to a desired value in advance. max (CS) is the maximum value of the saturation complexity indicated by all the blocks of the saturation complexity map, that is, the maximum value of the hue complexity. max (PS, max (CS)) is the larger value of PS and max (CS).

  In Expression (12), IL (n) is the normalized luminance complexity of the block with the identification number n, CL (n) is the luminance complexity of the block with the identification number n, and IMAX is the maximum luminance complexity after the normalization. And is set in advance to a desired value. PL is a normalization coefficient for luminance complexity, and is set to a desired value in advance. max (CL) is the maximum value of the luminance complexity indicated by all the blocks of the luminance complexity map, that is, the maximum value of the luminance complexity. max (PL, max (CL)) is a larger value of PL, max (CL).

  In Expression (10), assuming that IMAX = 8 and PH = 90, the hue complexity map shown in FIG. 6B is normalized to the normalized hue complexity map 60 shown in FIG. Each block 61 of the normalized hue complexity map 60 has a normalized hue complexity of 0-8.

  Furthermore, the complexity calculation unit 17 calculates the mixed complexity by mixing the normalized hue complexity, the normalized saturation complexity, and the normalized luminance complexity for each block. Specifically, the complexity calculation unit 17 calculates the mixing complexity based on the following equation (13). Then, the complexity calculation unit 17 generates a mixed complexity map in which the mixed complexity of the block is associated with the block identification number.

  In equation (13), I (n) is the mixed complexity of the block with identification number n. WH, WS, and WL are mixing weight coefficients for hue, saturation, and luminance, and are set in advance to desired values. By adjusting these coefficients, the proportion of each complexity contributing to the mixing complexity can be adjusted. max (IH (n) × WH, IS (n) × WS, IL (n) × WL) is the largest of IH (n) × WH, IS (n) × WS, IL (n) × WL Indicates the value. For example, when the complexity calculation unit 17 generates the normalized hue complexity map 60 illustrated in FIG. 7, the normalized saturation complexity map 70 illustrated in FIG. 8, and the normalized luminance complexity map 80 illustrated in FIG. 9. Generates a mixed complexity map 90 shown in FIG. 10 by mixing them. The complexity calculation unit 17 outputs the mixed complexity map to the corrected luminance calculation unit 18.

  The corrected luminance calculation unit 18 calculates the corrected luminance of each block by multiplying the luminance of each block by the mixing complexity. That is, the corrected luminance calculation unit 18 weights the luminance of each block based on the mixing complexity. Furthermore, the corrected luminance calculation unit 18 calculates the corrected average luminance of the captured image based on the equation (14).

  In equation (14), CY is the corrected average luminance of the captured image, and Y (n) × I (n) is the corrected luminance of the block with the identification number n. Therefore, the corrected luminance calculation unit 18 calculates the corrected average luminance by performing weighted averaging of the luminance of each block. A block having a mixed complexity of 0 has a large influence on the weighted average change. Therefore, the corrected luminance calculating unit 18 may add 1 as an offset to the mixed complexity of each block constituting the mixed complexity map so that there is no block having a mixed complexity of 0.

  Further, the corrected luminance calculation unit 18 calculates the reference average luminance by performing conventional photometric processing (for example, center-weighted photometry, multi-segment photometry, etc.) on the above-described luminance map. Then, the corrected luminance calculation unit 18 calculates the mixed average luminance by mixing the corrected average luminance and the reference average luminance based on the following equation (15).

  In Equation (15), FY is the mixed average luminance, NY is the reference average luminance, and CY is the corrected average luminance. WN and WC are mixing weight coefficients for the reference average luminance and the corrected average luminance, and are set to desired values in advance. By adjusting these coefficients, it is possible to adjust the ratio at which each average luminance contributes to the mixing complexity. The corrected luminance calculation unit 18 clips the corrected average luminance to a predetermined value when the difference between the reference average luminance and the corrected average luminance is large in order to prevent the luminance correction due to the mixing complexity from being excessively effective. May be. The corrected luminance calculation unit 18 generates mixed average luminance information related to the mixed average luminance and outputs it to the exposure amount calculation unit 19.

  Based on the mixed average luminance information, the exposure amount calculation unit 19 determines the exposure amount of the image sensor 12 so that the mixed average luminance approaches a preset target luminance (that is, the difference between them approaches 0). calculate. The exposure amount calculation unit 19 generates corrected exposure amount information related to the calculated exposure amount, that is, the corrected exposure amount, and outputs it to the exposure control unit 20.

  The exposure control unit 20 controls the driver 21 based on the corrected exposure amount information. The driver 21 adjusts the aperture of the lens unit 11, the drive timing of the image sensor 12, the gain of the analog processing unit 13, and the like under the control of the exposure control unit 20. Thereby, the exposure amount of the image sensor 12 matches the corrected exposure amount.

<4. Processing procedure by imaging device>
Next, a processing procedure by the imaging apparatus 10 will be described with reference to a flowchart shown in FIG. In step S11, the block dividing unit 15 divides the captured image given from the analog processing unit 13 into a plurality of blocks (for example, 16 × 16 blocks), and assigns an identification number to each block. Next, the block dividing unit 15 calculates an RGB integrated value by integrating R information, G information, and B information for each block. The block dividing unit 15 generates an integrated value map in which the RGB integrated value and the block identification number are associated with each other, and outputs the integrated value map to the feature amount calculating unit 16. Further, the block dividing unit calculates the luminance of each block based on these integrated values and the above-described equation (1). Next, the block dividing unit 15 generates a luminance map in which the luminance and the block identification number are associated, and outputs the luminance map to the feature amount calculating unit 16 and the corrected luminance calculating unit 18.

  Next, the feature amount calculation unit 16 calculates the hue, saturation, and log conversion luminance as the feature amount of each block based on the integrated value map and the luminance map and the above-described equations (2) to (5). .

  Next, the feature amount calculation unit 16 includes a hue map in which the hue of the block and the block identification number are associated, a saturation map in which the saturation of the block and the block identification number are associated, and the log conversion luminance of the block And a log conversion luminance map in which the block identification numbers are associated with each other and output to the complexity calculation unit 17.

  In step S13, the complexity calculation unit 17 calculates the hue complexity, the saturation complexity, and the luminance complexity of each block based on the hue map, the saturation map, and the log conversion luminance map. Specifically, the complexity calculating unit 17 calculates the difference between the feature amounts of each block and four blocks adjacent to the block in the up, down, left, and right directions, and calculates the complexity of each feature amount based on the difference. calculate. For example, the complexity calculating unit 17 calculates the complexity of the block 31a illustrated in FIG. 5 based on the above-described equations (6) to (9).

  The complexity calculation unit 17 calculates the hue complexity, the saturation complexity, and the luminance complexity of each block by performing the same processing for each block. Next, the complexity calculator 17 generates a hue complexity map indicating the hue complexity of each block, a saturation complexity map indicating the saturation complexity of each block, and a brightness complexity map indicating the brightness complexity of each block. Generate.

  In step S14, the complexity calculation unit 17 normalizes the complexity of each block based on the above-described equations (10) to (12). Thereby, the complexity calculator 17 generates a normalized hue complexity map, a normalized saturation complexity map, and a normalized luminance complexity map.

  In step S15, the complexity calculation unit 17 calculates the mixed complexity by mixing the normalized hue complexity, the normalized saturation complexity, and the normalized luminance complexity for each block. Specifically, the complexity calculation unit 17 calculates the mixing complexity based on the above-described equation (13). Next, the complexity calculation unit 17 generates a mixed complexity map in which the mixed complexity of the block is associated with the block identification number. The complexity calculation unit 17 outputs the mixed complexity map to the corrected luminance calculation unit 18.

  In step S16, the corrected luminance calculation unit 18 calculates the corrected luminance of each block by multiplying the luminance of each block by the mixing complexity. Furthermore, the corrected luminance calculation unit 18 calculates the corrected average luminance of the captured image based on the above-described equation (14).

  In step S <b> 17, the corrected luminance calculation unit 18 calculates the reference average luminance by performing conventional photometric processing (for example, center-weighted photometry, multi-segment photometry, etc.) on the above-described luminance map. Then, the corrected luminance calculation unit 18 calculates the mixed average luminance by mixing the corrected average luminance and the reference average luminance based on the above-described equation (15). The corrected luminance calculation unit 18 generates mixed average luminance information related to the mixed average luminance and outputs it to the exposure amount calculation unit 19.

  In step S <b> 18, the exposure amount calculation unit 19 makes the imaging element 12 based on the mixed average luminance information so that the mixed average luminance approaches a preset target luminance (that is, the difference between them approaches 0). The amount of exposure is calculated. The exposure amount calculation unit 19 generates corrected exposure amount information related to the calculated exposure amount, that is, the corrected exposure amount, and outputs it to the exposure control unit 20.

  Next, the exposure control unit 20 controls the driver 21 based on the corrected exposure amount information. The driver 21 adjusts the aperture of the lens unit 11, the shutter speed of the image sensor 12, the gain of the analog processing unit 13, and the like under the control of the exposure control unit 20. That is, the exposure control unit 20 controls exposure at the time of imaging. Thereby, the exposure amount of the image sensor 12 matches the corrected exposure amount.

  As described above, according to the first embodiment, the imaging apparatus 10 calculates the complexity of each block based on the feature amount of each block, specifically, the hue, saturation, and log conversion luminance. And the imaging device 10 calculates the correction brightness | luminance of each block by weighting the brightness | luminance of each block based on the complexity of each block. Then, the imaging device 10 controls the exposure at the time of imaging based on the average value of the corrected luminance, that is, the corrected average luminance. As a result, the imaging apparatus 10 can reduce the correction luminance of a region having a uniform luminance, for example, a background region, and can increase the correction luminance of a region where the luminance changes rapidly, for example, a boundary region between a person and the background.

  As a result, the influence of the background area on the average brightness is reduced, while the influence of the person area on the average brightness is increased. Therefore, the imaging apparatus 10 has a wider dynamic range (that is, the corrected brightness of the person area and the background area). Can be used to control the exposure. Therefore, the imaging apparatus 10 can perform exposure control more appropriate than before.

  Furthermore, since the imaging apparatus 10 calculates the feature amount of each block based on the pixel integrated value, specifically, the RGB integrated value, the complexity of each block can be calculated more accurately.

  Furthermore, since the imaging device 10 calculates the complexity of each block based on the difference between the feature amounts of adjacent blocks, the complexity of each block can be calculated more accurately.

  Furthermore, the imaging apparatus 10 calculates a plurality of types of feature amounts, that is, hue, saturation, and log conversion luminance for each block. Then, the imaging apparatus 10 calculates the hue complexity, saturation complexity, and luminance complexity of each block, and calculates the mixed complexity of each block by mixing these complexity for each block. And the imaging device 10 weights the brightness | luminance of each block based on the mixing complexity of each block. Thereby, the imaging device 10 can calculate the corrected luminance more accurately.

<5. Second Embodiment>
Next, a second embodiment will be described. The gist of the second embodiment is as follows. That is, when a region where a human face is drawn in the captured image, that is, the face region is large, the complexity in the face region may be small. Therefore, in the second embodiment, a process for complementing the complexity in the face area is performed.

  Specifically, the imaging device 10 performs the following processing in addition to the above-described processing. That is, the complexity calculation unit 17 extracts a skin color region in which the hue and saturation indicate skin color from the mixed complexity map based on the hue map and saturation map. Furthermore, the complexity calculation unit 17 determines an effective area based on the effective area determination graph L1 illustrated in FIG. 12 and the effective area determination map 230 illustrated in FIG. The complexity calculating unit 17 determines that the skin color area in the effective area is a face area, and complements the complexity of the face area. Specifically, the complexity calculation unit 17 increases the complexity of the face area. A desired value is set in advance for the increase amount. Thereby, the complexity calculation unit 17 generates a face region complementation complexity map. Subsequent processing is the same as in the first embodiment.

  Here, the effective area determination graph L1 shows the correspondence between the size of the skin color area and the effective determination possible number that can be determined as the effective area. For example, when the size of the skin color area is equal to or smaller than Th1, the valid determination possible number is 2, and when the size of the skin color area is equal to or larger than Th2, the valid possible determination number is 8. The effective area determination map 230 is a map to which determination numbers 1 to 8 are assigned for each block. That is, the complexity calculation unit 17 specifies the valid determination possible number based on the size of the skin color region and the effective region determination graph L1. Then, the complexity calculation unit 17 determines a block having a determination number equal to or greater than the effective determination possible number among the blocks of the effective region determination map 230 as the effective region.

  For example, when the valid determination possible number is 2, the complexity calculation unit 17 determines the region 232 in the effective region determination map 230 as the effective region. On the other hand, when the valid determination possible number is 8, the complexity calculation unit 17 determines the region 233 in the valid region determination map 230 as a valid region. Thus, when the skin color area is large, the complexity calculation unit 17 determines the center of the image as an effective area in order to prevent the skin color area other than the face from being erroneously determined as the face area. The complexity calculating unit 17 acquires the effective determination possible number by rounding the acquired value when the effective determination possible number for the skin color region is not an integer.

  For example, when the captured image 200 shown in FIG. 14 is generated, the mannequin and the desk at the center of the image have a skin color. In this case, the complexity calculation unit 13 generates a mixed complexity map 210 shown in FIG. In FIG. 15A, the mixing complexity of each block is omitted. Then, the complexity calculation unit 13 extracts the skin color region 211 from the mixed complexity map 210. The size of the skin color area 211 is assumed to be Th2 or more. Then, the complexity calculation unit 13 acquires 8 as an effective determination possible number corresponding to the size of the skin color region 211 based on the effective region determination graph L1. The complexity calculating unit 13 determines an effective area based on the effective determination possible number and the effective area determination map 230. Thereby, the complexity calculation unit 13 excludes the skin color region 211b surrounded by the frame 220 from the skin color region 211, and determines only the skin color region 211a as a face region. Then, the complexity calculation unit 17 complements only the face area in the mixed complexity map 200. For example, the complexity calculation unit 17 generates a face region complementation complexity map 240 shown in FIG. 16B by complementing the mixed complexity map 200 shown in FIG.

<6. Processing According to Second Embodiment>
Next, a processing procedure according to the second embodiment will be described with reference to a flowchart shown in FIG. In the second embodiment, the imaging apparatus 10 performs the processes of step S21 and step S22 illustrated in FIG. 17 between step S15 and step S16 described above.

  That is, in step S21, the complexity calculation unit 17 extracts a skin color region whose hue and saturation indicate skin color from the mixed complexity map based on the hue map and the saturation map.

  In step S <b> 22, the complexity calculation unit 17 specifies an effective determination possible number based on the size of the skin color area and the effective area determination graph L <b> 1. Then, the complexity calculation unit 17 determines a block having a determination number equal to or greater than the effective determination possible number among the blocks of the effective region determination map 230 as the effective region. Next, the complexity calculation unit 17 determines that the skin color area in the effective area is a face area, and complements the complexity of the face area. Thereby, the complexity calculation unit 17 generates a face region complementation complexity map. The complexity calculation unit 17 outputs the face area complementation complexity map to the corrected luminance calculation unit 18. Thereafter, the imaging device 10 performs the processing from step S16 onward based on the face region complementation complexity map. According to the second embodiment, the imaging device 10 can more accurately control the exposure of the face area.

  The preferred embodiments of the present invention have been described in detail above with reference to the accompanying drawings, but the present invention is not limited to such examples. It is obvious that a person having ordinary knowledge in the technical field to which the present invention pertains can come up with various changes or modifications within the scope of the technical idea described in the claims. Of course, it is understood that these also belong to the technical scope of the present invention.

DESCRIPTION OF SYMBOLS 10 Imaging device 11 Lens unit 12 Image sensor 13 Analog processing part 14 Signal processing part 15 Block division part 16 Feature amount calculation part 17 Complexity calculation part 18 Correction brightness calculation part 19 Exposure amount calculation part 20 Exposure control part 21 Driver

Claims (7)

A captured image acquisition unit for acquiring a captured image;
A block division unit for dividing the captured image into a plurality of blocks;
A feature amount calculation unit that calculates a feature amount including luminance for each block;
A complexity calculation unit for calculating the complexity of each block based on the feature amount of each block;
Based on the complexity of each block, by weighting the luminance of each block, a corrected luminance calculating unit that calculates the corrected luminance of each block;
An exposure control apparatus comprising: an exposure control unit that controls exposure at the time of imaging based on the corrected luminance. The block dividing unit calculates a pixel integrated value by integrating pixel values for each block,
The exposure control apparatus according to claim 1, wherein the feature amount calculation unit calculates a feature amount of each block based on a pixel integrated value of each block.   The exposure control apparatus according to claim 1, wherein the complexity calculation unit calculates the complexity of each block based on a difference in feature amount between adjacent blocks. The feature amount calculation unit calculates a plurality of types of feature amounts for each block,
The complexity calculation unit calculates the complexity of each block for each type of feature quantity, and calculates the mixed complexity of each block by mixing the complexity for each type of feature quantity for each block. ,
The exposure control apparatus according to claim 1, wherein the corrected luminance calculation unit weights the luminance of each block based on the mixing complexity of each block.   5. The information processing apparatus according to claim 1, wherein the complexity calculation unit supplements the complexity of a face area in which a human face is drawn among the blocks. . The corrected luminance calculation unit calculates an average luminance that is an average value of the corrected luminance,
The exposure control apparatus according to claim 1, wherein the exposure control unit controls exposure at the time of imaging so that the average brightness approaches a predetermined target brightness. A captured image acquisition step of acquiring a captured image;
A block dividing step of dividing the captured image into a plurality of blocks;
A feature amount calculating step for calculating a feature amount including luminance for each block;
A complexity calculating step for calculating the complexity of each block based on the feature amount of each block;
A corrected luminance calculating step for calculating the corrected luminance of each block by weighting the luminance of each block based on the complexity of each block;
An exposure control step of controlling exposure at the time of imaging based on the corrected luminance.