JP2013176119A - Imaging apparatus and image processing method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To perform a gradation correction suitable for a taken image in accordance with luminance distribution in the taken image.SOLUTION: An imaging apparatus comprises: first determination means which determines first correction data for performing, on image data obtained by imaging, gradation correction to change the luminance range of an input luminance corresponding to that of an output luminance; second determination means which determines second correction data for performing, on the image data obtained by imaging, gradation correction corresponding to the height of a luminance frequency; and control means which restricts execution of gradation correction using one of the first correction data and the second correction data, on image data to undergo gradation correction using the other.

Description

  The present invention relates to an image gradation correction technique.

  Conventionally, in order to obtain an image with preferable brightness and contrast, a method of performing gradation correction by analyzing a luminance histogram and subject information of an acquired image is known.

  In particular, in a so-called backlight scene where the brightness of the main subject is significantly darker than the brightness of the background, tone correction is effective because the main subject portion of the captured image becomes dark. There are the following methods for so-called “flying” in which the brightness of the background becomes extremely bright. First, at the stage of exposing the image sensor, an image is captured with an exposure that is darker than the appropriate exposure so as to suppress jumping. Then, by using a correction table (tone curve) for converting the luminance value of the obtained image data into a predetermined output luminance value, the luminance value of the darkly photographed area is set to an appropriate level, and the brightness of the area already brightly photographed is set. The gradation correction is performed so that the luminance value is kept as it is. This is known as an effective method for pseudo-expanding the dynamic range of the image sensor. (For example, refer to Patent Document 1.)

  In addition, a method has been proposed in which gradation correction is performed using a low-frequency component signal of a luminance component of an image, thereby obtaining an effect like so-called dodging. With this method, it is possible to perform gradation correction that brightens a subject that has been shot dark while maintaining the contrast of the image. (For example, see Patent Document 2.)

  On the other hand, even if there is a white part in the screen, such as a white wall or snow scene, the automatic exposure control determines the exposure so that it reaches the specified brightness level. May end up. For such scenes, if there are few or no high-luminance components, the output luminance area assigned to the high-luminance input luminance is narrowed, and gradation correction is performed so that gradation assignment to unnecessary areas is eliminated. Thus, it is also known to obtain an effect of taking a white object white.

JP 2007-124604 A JP 2008-072604 A

  However, as in Patent Document 1, when tone correction is performed so that a dark portion of an image is brightly corrected using a tone curve and correction is originally performed to suppress correction of a bright portion, contrast in a luminance band intermediate between the main subject and the background. May be lost, resulting in a crisp image.

  Further, as described in Patent Document 2, when tone correction is performed using a low-frequency component signal of a luminance component of an image, an image may be unnatural because there is too much contrast depending on the scene.

  The present invention has been made in view of the above problems, and an object of the present invention is to perform tone correction suitable for an image without causing deterioration in image quality according to the luminance distribution of the captured image.

In order to achieve the above object, an image pickup apparatus according to the present invention performs tone correction for changing a luminance range of input luminance corresponding to a luminance range of output luminance with respect to image data obtained by imaging. First determination means for determining first correction data, and second correction data for performing gradation correction corresponding to the height of the luminance frequency for the image data obtained by imaging. Tone correction using the other of the first and second correction data for the image data subjected to the tone correction using the second determining means and one of the first and second correction data And control means for restricting the implementation.

  According to the present invention, gradation correction suitable for an image can be performed according to the luminance distribution of the captured image.

It is a block diagram which shows the structure of the imaging device in embodiment of this invention. It is a flowchart which shows the flow of the gradation correction method in embodiment of this invention. It is a flowchart which shows the temporary determination process of D + amount in embodiment of this invention. It is a flowchart which shows the determination process of the aperture_diaphragm | restriction, a shutter, a gain, and D + amount in embodiment of this invention. It is a flowchart which shows the determination process of D-amount in embodiment of this invention. It is a flowchart which shows the determination process of the dark part correction amount in embodiment of this invention. It is a flowchart which shows the procedure of the image processing in embodiment of this invention. It is a figure for demonstrating the outline | summary of D-correction in embodiment of this invention. It is a figure for demonstrating the outline | summary of the dark part correction | amendment in embodiment of this invention. It is a figure which shows the mode of the resolution conversion for dark part correction | amendment in embodiment of this invention. It is a figure which shows an example of the tone curve used for dark part correction | amendment in embodiment of this invention. It is a figure which shows an example of the tone curve used by D + correction | amendment in embodiment of this invention. It is a timing diagram which shows the timing of the process before and after SW1ON concerning D + amount calculation in embodiment of this invention. It is a timing diagram after SW2 ON of the gradation correction processing in the embodiment of the present invention.

  The best mode for carrying out the present invention will be described below in detail with reference to the accompanying drawings. However, the dimensions, materials, shapes, relative arrangements, and the like of the components exemplified in this embodiment should be changed as appropriate according to the configuration of the apparatus to which the present invention is applied and various conditions. However, the present invention is not limited to these examples.

  DESCRIPTION OF EMBODIMENTS Hereinafter, embodiments for carrying out the present invention will be described in detail with reference to the accompanying drawings.

Configuration of Imaging Device FIG. 1 is a block diagram showing the configuration of the imaging device 100 in the embodiment of the present invention.

  In FIG. 1, reference numeral 60 denotes a system control unit that controls the operation of the entire imaging apparatus 100.

  Reference numeral 10 denotes a photographing lens that collects external light. Although the lens is shown as one in the figure, it is usually composed of a plurality of lenses. The photographing lens 10 includes, for example, a focus lens for adjusting the focus and a zoom lens for adjusting the angle of view by moving the lens driving circuit 42 back and forth along the optical axis. Further, based on the camera shake amount measured by the camera shake detection circuit 44, the camera shake correction circuit 40 moves the lens to change the optical axis in the direction of canceling camera shake, thereby performing optical camera shake correction. Including an image stabilization lens to perform. Moreover, it is possible to improve the portability by adopting a system in which the lens barrel portion including the photographing lens 10 is extended and retracted, and the main body volume is reduced when the camera is not used.

  Note that the shake amount detection circuit 44 includes a gyro sensor, and in FIG. 1, camera shake correction is realized by driving a lens. However, camera shake correction is similarly performed by driving an image sensor 16 described later. Is also possible.

  Reference numeral 12 denotes a mechanical shutter (hereinafter simply referred to as “shutter”). The system control unit 60 can control the shutter 12 by transmitting mechanical shutter control information to the shutter drive circuit 28. The exposure time during still image shooting is determined by the opening / closing time of the shutter 12, and the system control unit 60 determines this opening / closing time and issues an instruction to the shutter drive circuit 28.

  Reference numeral 14 denotes an aperture for adjusting the amount of light passing through the photographing lens 10. Examples of the diaphragm 14 include an iris diaphragm composed of a plurality of wings and a round diaphragm in which holes are punched out in advance with various diameters. The system control unit 60 can control the diaphragm 14 by transmitting the diaphragm control information to the diaphragm driving circuit 26. The system control unit 60 uses the diaphragm 14 and the diaphragm driving circuit 26 to control the diaphragm so as to reduce the amount of light when the subject brightness is high, and to open the diaphragm and capture more light when the subject brightness is low. Can be controlled.

  Reference numeral 16 denotes an image sensor such as a CCD or CMOS sensor, which converts an optical image of a subject incident through the taking lens 10, the shutter 12, and the aperture 14 into an electrical signal (image signal) and outputs it. The system control unit 60 can control the image sensor 16 by transmitting an image sensor control signal to a TG (Timing Generator) 24.

  The TG 24 drives the image sensor 16 based on the control information received from the system control unit 60. The image pickup device 16 periodically performs charge accumulation in each pixel and readout operation of the accumulated charge, and this operation is performed with reference to a drive signal from the TG 24. It is possible to read out charges from only a specific line or a specific area among the charges accumulated in the image sensor 16. This can be realized by changing the reading method by the read control pulse output from the TG 24. The system control unit 60 determines an optimum reading method according to the situation and instructs the TG 24. For example, since high resolution is required during still image shooting, all data of the image sensor 16 is read out, and when a high frame rate such as 30 fps or 60 fps is required during electronic viewfinder or movie shooting, only specific lines are thinned out. It is used properly such as reading. Also, the exposure time of the image sensor 16 can be controlled via the TG 24. This is made possible by outputting a drive signal from the TG 24 to the image sensor 16 so as to clear the charge charged by the image sensor 16 at an arbitrary timing.

  Reference numeral 18 denotes a CDS (Correlated Double Sampler) circuit that removes noise components of image data from the electrical signal (image signal) output from the image sensor 16 by a correlated double sampling method. Reference numeral 20 denotes a PGA (Programmable Gain Amplifier) circuit that attenuates / amplifies the level of the image data, and the amount of amplification is controlled by a system control unit 60 described later. Usually, in order to obtain image data of an appropriate level, the amount of light incident on the image sensor 16 is appropriately set by the aperture 14 and the exposure time is appropriately set by the shutter 12 so that the exposure of the image sensor 16 is appropriate. In addition, by attenuating / amplifying the image data signal by the PGA circuit 20, it is possible to play a role of changing the exposure of the image data in a pseudo manner. This is a function of providing the user with a concept of sensitivity as one of the exposure conditions at the time of photographing along with the aperture and shutter speed.

  The image data processed by the PGA circuit 20 is converted from an analog signal to a digital signal by an A / D (Analog / Digital) circuit 22. Depending on the device, the bit width of the digital signal may be 10 bits, 12 bits, 14 bits, or the like. The subsequent image processing circuit 50 can cope with a plurality of types of bit widths. In FIG. 1, the CDS circuit 18, the PGA circuit 20, and the A / D circuit 22 are shown as different configurations, but it is also possible to adopt a circuit in which the functions of these circuits are mounted in one IC package. .

  The digitized image data output from the A / D circuit 22 is input to the image processing circuit 50. The image processing circuit 50 includes a plurality of blocks and realizes various functions.

  For example, the image sensor 16 generally extracts light of a specific color component for each pixel through a color filter, and photoelectrically converts the extracted light. The image signal output from the A / D circuit 22 has a data format corresponding to the pixel of the image sensor 16 and the color filter arrangement, and automatic exposure control (AE: AutoExposureControl) that performs exposure control by evaluating only the luminance component. Not suitable for use with. Therefore, the image processing circuit 50 has a function of removing color information from the image signal and extracting only luminance information.

  Furthermore, the image processing circuit 50 has a function of extracting only the frequency component of the image signal read from the image sensor 16, and can be used during automatic focusing control (AF: AutoFocus). A function is provided in which the frequency component of which area of the image signal read from the image sensor 16 is extracted or the area can be divided and set. At the time of this AF processing, it is possible to drive the image sensor 16 to be suitable for focus adjustment. In the case of television AF using the image sensor 16, the focus lens is driven to a new focus position each time an image signal is read from the image sensor 16, so that the higher the frame rate of the image sensor 16, the more quickly the focus lens is focused. Can be moved to a position. For this reason, the image sensor 16 can be driven so as to have a fast frame rate only during AF. Conversely, by reducing the frame rate and reading more pixel data from the image sensor 16, the amount of data that can be analyzed by the image processing circuit 50 increases, enabling more accurate focus adjustment control. There is also. Such use of the image sensor 16 can be determined adaptively according to the shooting mode of the camera and the brightness of the subject.

  Further, the image processing circuit 50 has a function of adjusting the level of the image data digitized by the A / D circuit 22 and operating the color effect of the image, and also plays a role of adjusting the image quality of the photographed image. . Regarding the level of image data, there are a function for increasing and decreasing the level with a uniform amplification factor for the entire image and a tone curve (gamma) function for converting the signal level according to the magnitude of the original signal level. In addition, various level adjustments such as a function of increasing / decreasing the level with an amplification factor corresponding to a frequency component for each region in the screen can be performed.

  The image data digitized by the A / D circuit 22 can be input to the image processing circuit 50 and simultaneously stored in the temporary storage memory 30. The image data once stored in the temporary storage memory 30 can be read again, the image data can be referred to from the system control unit 60, and the read image data can be input to the image processing circuit 50. Furthermore, the image data processed by the image processing circuit 50 can be written back to the temporary storage memory 30 or arbitrary data can be written from the system control unit 60.

  Image data appropriately processed by the image processing circuit 50 can be input to the image recognition circuit 38. The image recognition circuit 38 can recognize the character information of a person's face, its facial expression, and characters in addition to recognizing the brightness, focus, and color of the input image. In addition, a plurality of images can be input to the image recognition circuit 38. For example, it is possible to determine whether the images are the same by inputting two images and comparing the characteristics of the two images. In addition to the image recognition process in the image recognition circuit 38, the system control unit 60 can also perform the image recognition process. The system control unit 60 can execute a pre-encoded program on the CPU, but can read the image data stored in the temporary storage memory 30 and analyze the image data to recognize the situation of the scene. it can.

  An operation unit 70 includes a power ON / OFF switch 102, a shutter switch 104, a mode change switch 110, a parameter selection switch group 151, and the like. The shutter switch 104 instructs the preparation and start of shooting by a two-step pressing operation when pressed lightly (SW1) and when pressed deeply (SW2). In the case of an imaging device that performs automatic exposure control and automatic focus adjustment control, pressing the shutter switch 104 shallowly (SW1 ON) performs automatic exposure control and automatic focus adjustment control as preparation for shooting, and pressing it deeply (SW2 ON) still image shooting And an operation for instructing image recognition can be realized. The mode switch 110 can switch camera operation modes such as a still image shooting mode, a moving image shooting mode, and a playback mode. Here, although expressed as a member capable of switching several modes, it is possible to provide many still image modes such as a landscape shooting mode and a person shooting mode optimized for a specific scene to be shot. In addition, the user can select the shooting conditions such as the distance measurement area and the photometry mode, the selection of shooting conditions at the time of shooting, the page feed when playing back the shot image, and the general operation settings of the camera by the parameter selection switches 151a to 151e Can do. Furthermore, On / Off of the above-described electronic viewfinder can be selected. Further, the image display device 108 can display an image and can be configured as an input device as a touch panel.

  Reference numeral 108 denotes an image display device constituted by an LCD or the like. When outputting an image to the image display device 108, the image data subjected to the image processing by the image processing circuit 50 is developed on the VRAM 34 and converted into analog data by the digital / analog (D / A) circuit 36. Then display. In addition, the electronic viewfinder function can be realized by sequentially updating and displaying the continuous images read from the image sensor 16 on the image display device 108. When the image data is expanded on the VRAM 34, the VRAM 34 is adapted to correspond to various display forms such that one image data is maximized on the image display device 108, or a plurality of images are displayed on a multi-screen. Can be deployed on top.

  Further, the image display device 108 can display not only an image but also arbitrary information alone or together with the image. Display of camera status, character information such as shutter speed, aperture value, sensitivity information selected by the user or determined by the camera, histogram such as brightness distribution measured by the image processing circuit 50, face recognition result, scene recognition result Etc. can also be displayed. The display position and display color of information can be arbitrarily selected. By displaying these various types of information, a user interface can be provided. The image display device 108 can also display image data stored in the image storage medium 82. When the image data is compressed, the image data is decompressed by the compression / decompression circuit 32 and the data is expanded in the VRAM 34. The expanded data is converted into analog data by the D / A circuit 36 and output.

  It is also possible to adopt a configuration in which a conventional optical finder is also provided. In that case, it is possible to reduce power consumption by turning off the electronic finder function and using the optical finder.

  The image storage medium 82 is non-volatile, and can mainly store captured image data via the image storage medium interface 80. Regarding the storage of image data, for example, a folder hierarchy can be provided, or folders and file names can be named in ascending order in the order of shooting. Shooting information such as aperture value, shutter speed, ISO sensitivity, shooting time, and the like can be added to each image data, and these can be stored together with the image data. Furthermore, the stored image data can be read out, and the image can be copied, moved, and deleted.

  Reference numeral 86 denotes an external device such as a printer, a PC, or a display device, which is connected via an external device interface 84.

  Next, the gradation correction method in the present embodiment, which is executed in the imaging apparatus 100 having the above configuration, will be described. In this embodiment, a case where the following three types of gradation correction methods are used will be described.

(1) First Tone Correction Method In this embodiment, as a first tone correction method, in a scene where a low-brightness subject and a high-brightness subject are mixed, A gradation correction method that suppresses saturation). Hereinafter, this first gradation correction method is referred to as D + correction.

  More specifically, normally, the aperture value, shutter speed, gain amount, etc. are determined as shooting conditions before shooting. At this time, the appropriate exposure is determined according to the brightness information of the screen and the presence or absence of a human face. Try to shoot. However, the dynamic range of the image sensor is not so wide, no matter how the aperture, shutter, and gain are set, in a backlit scene such as when shooting a person standing against the sun, the person is too dark. Or the background is too bright. It is D + correction that reduces the phenomenon that the background is too bright.

  In D + correction, first, shooting is performed by setting an exposure value that is underexposed so as not to cause whiteout in a high-luminance portion, and a gamma that is set to raise the luminance of the low-luminance portion of the obtained image data. Correction is performed using a curve (tone curve).

(2) Second Gradation Correction Method In the present embodiment, as the second gradation correction method, a gradation in which unnecessary gradation assignment is not performed according to the state of luminance data in the high-luminance area of the captured image. The correction method is used. Hereinafter, this second gradation correction method is referred to as D-correction. FIG. 8A is a diagram illustrating an example of a histogram of a scene in which D-correction is effective. When there is a uniform luminance distribution from dark to bright in the screen, the histogram shown in FIG. In FIG. 8A, it can be determined that the scene has no brightness distribution on the high brightness side and the brightness difference in the screen is not so severe.

  When the luminance distribution covers a wide range, a gamma curve such as a normal gamma curve 302 shown in FIG. 8B is set so that the output luminance is assigned to any input luminance. However, when there is no distribution in the high luminance region as in the histogram of FIG. 8A, even if the output luminance is assigned to the input luminance, it is not effective gradation expression. Accordingly, the D-gamma curve shown in FIG. 8B is used, in which the luminance value 312 that is the maximum luminance value in which the histogram distribution exists is set as the upper limit of the input luminance, and the luminance value 312 is assigned to the maximum luminance value that the image data can take. A gamma curve such as 320 is set. The D-gamma curve 320 effectively assigns output luminance to input luminance up to the input luminance 324 and does not assign luminance exceeding the input luminance 324. FIG. 8B shows a case where the luminance distribution area and the input luminance allocation range of the D-gamma curve 320 completely match. However, when selecting one of a plurality of D-gamma curves prepared in advance, the input luminance distribution and the input luminance allocation range of the D-gamma curve may not completely match.

(3) Third Gradation Correction Method In the present embodiment, as the third gradation correction method, low frequency and low frequency are selected according to the situation such as the low frequency area of the captured image, the luminance information, and the presence or absence of a human face. A gradation correction method for adjusting the luminance level of a luminance region is adopted. Hereinafter, this third gradation correction method is referred to as dark portion correction.

  FIG. 9 shows a scene in which dark part correction is effective and an image diagram of the result of dark part correction of the scene. FIG. 9 shows a scene where a person stands with the sun behind. Before the dark part correction, the human face, the human body, the ground, etc. appear dark due to the backlight scene. If tone correction is performed with a uniform tone curve so as to brighten the darkened dark part, the black eyes of the person and the pattern of the worn horizontal stripe shirt may be corrected more brightly than necessary. In that case, the image is a low quality image with a reduced contrast. Therefore, a low frequency luminance component is extracted, and dark portion correction is performed so that gradation correction is performed when the low luminance region has a certain area.

  This is shown in FIG. In order to extract the low-frequency component of the image, in FIG. 10, three different resolution images are generated from the original image. In FIG. 10A, the resolution of the original image is almost maintained, but the resolution is lowered as in FIGS. 10B and 10C. In FIG. 10C, it is difficult to determine a person. The degree of the resolution reduction may be set variously depending on the dark part correction effect to be obtained. A dark portion correction gamma as shown in FIG. 11 is applied to the plurality of images. This gamma is a conversion table in which a high output luminance is obtained in a region where the input luminance is low. After tone correction of each image, the three images are combined so as to be averaged. Then, an area having a certain luminance frequency such as an eye or clothes pattern is not so darkly corrected, and an effect of applying dark area correction to an area having a low luminance frequency such as pants or the ground can be obtained.

Flow of Overall Correction Processing Next, an overall flow for obtaining an image with a more appropriate gradation using the above-described three gradation correction methods will be described with reference to the flowcharts of FIGS. .

  After the image pickup apparatus 100 is started, various devices are initialized and started to perform preparations for shooting such as EVF image display, and the switch waits for SW1 to be turned on by pressing the shutter switch 104 (S201). When SW1ON is detected, a D + amount tentative determination process used for D + correction is executed (S203). Here, the D + amount tentative determination process will be described with reference to FIG.

  First, in the image processing circuit 50, image data obtained by photographing with the image sensor 16 is equally divided into, for example, four blocks in length and width, and a histogram is generated for each block to obtain a luminance distribution (S301). The histogram generated here may be for the entire screen, but it is possible to grasp a more detailed distribution situation by generating each block. Then, the system control unit 60 analyzes the luminance distribution on the high-luminance side of the histogram of each block, and determines how much whiteout occurs when shooting with this exposure (S303). Then, how much exposure should be suppressed in order to suppress the occurrence of overexposure is calculated (S305), which is referred to as the D + amount. Since the determination is made based on the whiteout condition of the entire image, the D + amount obtained here is hereinafter referred to as “total D + amount DPAll”. In the present embodiment, as the total D + amount DPAll, there is no correction (DPAll = 0), weak correction (DPAll = Low), medium correction (DPAll = Mild), and strong correction (DPAll = DPAll) according to the overexposure state of the entire image. High).

  Further, the image recognition circuit 38 detects a preset subject (hereinafter referred to as “human face”) such as a human face from the same image data, and the presence or absence of the face, the coordinates and size of the face area. The face information including the length is acquired (S315). If it is determined that there is no face in the image (NO in S317), the total D + amount DPAll obtained in step S305 is adopted as the temporary D + amount DP1 (S337). On the other hand, if it is determined that there is a face in the image (YES in S317), a histogram near the main face region is generated to obtain a luminance distribution (S319). Then, the system control unit 60 analyzes the high brightness area of the brightness distribution near the face area, grasps the whiteout situation limited to the face area (S321), and determines the D + amount according to this whiteout situation ( S323). Since the determination is made based on the whiteout situation limited to the face area, the D + amount obtained here is hereinafter referred to as “face D + amount DPFace”. In this embodiment, as face D + amount DPFace, as in step S305, no correction (DPFace = 0), weak correction (DPFace = Low), medium correction (DPFace = Mild), and strong correction (DPFace = High). There are four stages. Note that the D + amount is represented by the number of stages, and in this embodiment, Low = 1/3, Mild = 2/3, and Hi = 1.

  Then, the total D + amount DPAll and the face D + amount DPFace are compared (S335). As a result of comparison, if the total D + amount DPAll is larger, the total D + amount DPAll is adopted as (S337), and if not, the face D + amount DPFace is adopted (S339) as the provisional D + amount DP1.

  When the provisional determination of the D + amount is completed, the process returns to the process of FIG. 2 and proceeds to step S205, where the final aperture value, shutter speed, gain, and D + amount are determined based on the provisional D + amount DP1 obtained in FIG. Final decision processing is performed (S205). Here, the final determination process will be described with reference to FIG.

  First, subject brightness Bv1 is acquired from image data obtained by photographing with the image sensor 16 (S401). The subject luminance is calculated based on the luminance integrated values obtained by dividing the screen into blocks of about 24 × 16 vertical and horizontal dimensions. In the case of a photometric method that performs photometry of an arbitrary narrow area in the screen, such as spot photometry, the luminance value can be calculated from the integrated luminance value of one block at that location without dividing the block. An exposure value Ev1 is obtained from the obtained subject brightness Bv1 and photographing sensitivity Sv, and based on a known program diagram table expressing a combination of the exposure value, the aperture value, the shutter speed, and the gain, the aperture value Av1, the shutter speed Tv1, and the gain Gain 1 is obtained (S403). Note that the program diagram can be configured to have a diagram different for each photographing mode.

  Next, the gain and shutter speed necessary for suppressing exposure are adjusted by the provisional D + amount DP1 described with reference to FIG. First, the gain Gain1 obtained from the program diagram is compared with the provisional D + amount DP1 (S405). When the gain Gain1 is large, a value obtained by subtracting the temporary D + amount DP1 from the gain Gain1 is set as a new gain value Gain2 (S407). In this case, since the D + amount can be underexposed only by adjusting the gain, Tv1 obtained from the program diagram is directly adopted as Tv2 as the shutter speed (S409). On the other hand, when the gain Gain1 is equal to or less than the provisional D + amount DP1, it is impossible to underexpose the provisional D + amount DP1 by the gain alone. Therefore, first, the minimum value MinGain that can be set with the gain is set (S411). Subsequently, by increasing the shutter speed, underexposure for the provisional D + amount DP1 is realized. Here, with respect to Tv1 obtained from the program diagram, the shutter speed is increased by the number of stages that cannot be adjusted by the gain, and is set to Tv2 (S413).

  As described above, after adjusting the gain and the shutter speed so that the exposure is underexposed by the provisional D + amount DP1, it is checked whether or not this exposure can be realized (S415). The exposure limit in the shooting mode set by the imaging apparatus 100 is determined from the mechanical control limit, and the control limit can be set for each mode. Therefore, it is checked whether the exposure value including the aperture value, shutter speed, and gain exceeds the exposure limit MaxEv (exposure when the shutter speed cannot be further increased or the gain cannot be decreased). If it is within the limit, the provisional D + amount DP1 is determined as the final D + amount DP2 (S417). On the other hand, if the limit is exceeded, a value obtained by correcting the provisional D + amount DP1 by the excess is determined as the final D + amount DP2 (S419). This is because, since the subject has high brightness, when the set exposure value is near the exposure limit, it is effective to perform D + correction, but D + correction cannot be performed. When there is no need to perform D + correction, the provisional D + amount DP1 = 0, which is always smaller than Gain1. Therefore, Av1, Tv1, and Gain1 obtained in step S403 are determined as exposure values (Av1, Tv2, and Gain2) as they are and the D + amount DP2 = 0 is obtained through the flow of steps S405, S407, and S409.

  FIG. 13 shows the vertical synchronization signal VD of the image sensor 16 before and after SW1 ON, exposure to the image sensor 16, reading of accumulated charges from the image sensor 16, face detection, histogram generation, photometric block luminance detection, D + amount calculation, exposure It is a timing diagram which shows the timing of value calculation. Exposure and readout of accumulated charge are performed every vertical period. FIG. 13 shows that face detection is performed once every two vertical periods, but this is not a limitation, and the detection period varies depending on the performance of image recognition processing by the image recognition circuit 38. Regarding the histogram generation, since it is desired to calculate the D + amount using the result of face detection and histogram generation based on image data obtained by the same exposure, the histogram generation period is set in accordance with the face detection period. I have control.

  In this relationship, when SW1 is turned on, the D + amount calculation is performed using the face detection result (5) detected immediately before and the histogram (5) generated from the image data (5). Do. In the D + amount provisional determination process 511, the process described with reference to FIG. 3 is executed. After the D + amount provisional determination process 511 is completed, final determination of the aperture value, shutter speed, gain, and D + amount is performed (513). Similarly, the photometric result used in the automatic exposure control is calculated based on the screen block division luminance detected from the image data (5). In the exposure value calculation 513, the process described with reference to FIG. 4 is executed.

  As described above, when the aperture value, shutter speed, gain, and D + amount for D + correction are determined, the process returns to FIG. 2 and waits for SW2 to be turned on (S207). When SW2ON is detected, the image sensor 16 is exposed using the exposure value obtained in step S205, and the charge is read after the exposure time has elapsed and converted to image data (S209). Then, it is determined whether or not to perform D + correction on the obtained image data from the D + amount DP2 determined in step S205 (S211). Specifically, it is determined whether D + amount DP2> 0. When D + correction is not performed (that is, when DP2 is 0), D-amount determination processing is performed (S213). Here, the D-amount determination process will be described with reference to FIG.

  First, a histogram of the entire screen is generated in the image processing circuit 50 from image data obtained by photographing with the image sensor 16 (S501). Then, the system control unit 60 analyzes the situation of the high luminance area in the luminance distribution information obtained from this histogram, and grasps how much data exists in the high luminance area. Then, the extent to which there is no need to assign the gamma output luminance is analyzed, and the D-amount DM1 is determined according to the degree of the region that does not require the gamma output luminance assignment (S505). In the present embodiment, the D-amount DM1 has no correction (DM1 = 0), weak correction (DM1 = Low), medium correction (DM1 = Mild), depending on the extent of the area where gamma output luminance allocation is unnecessary. There are four levels of strong correction (DM1 = High). The D-amount is represented by the number of stages, and in this embodiment, Low = 1/3 stage, Mild = 2/3 stage, and Hi = 1 stage.

  Subsequently, it is checked again whether or not the D + amount DP2 obtained in step S205 is larger than 0, that is, whether or not D + correction is performed (S515). If D + correction is not performed, DM1 determined in step S505 is determined as the D-amount. If D + correction is performed, DM1 is set to 0 so that D-correction is not performed. (S517). As described above, in the present embodiment, when D + correction is performed, D-correction is not performed. This is because it is difficult to assume a scene where D + correction and D- correction need to be performed simultaneously. The D + correction operates on a scene where there is a risk of whiteout due to high luminance. However, as described with reference to FIG. 4, when the exposure value is determined so as to be underexposed in order to prevent whiteout, It is assumed that the histogram has a luminance distribution up to the limit. There is no need to perform D-correction on image data having such a histogram.

  When the D-amount is determined as described above, the process returns to the process of FIG. In step S215, it is determined whether or not D-correction is performed (S215). As described above, when D + correction is performed, D- correction is not performed. Therefore, if “YES” is determined in the step S211, NO is determined here. In addition, it is determined that if the D-value DM1 = 0 set in step S213 is not implemented, it is determined that the D-value DM1 = 0 is not implemented. When the D-correction is not performed, a dark portion correction amount determination process is performed (S217). On the other hand, when D-correction is performed, dark part correction is not performed. This is because both D-correction and dark area correction have the effect of increasing the luminance level, and if it is determined that both corrections are to be performed here, there is a possibility that the correction will be applied more than necessary, resulting in an image that is too bright. Because there is. In addition, since a scene to which D-correction is applied is a scene having no luminance distribution in a high luminance area, it is unlikely that the scene has a strong luminance difference such as a backlight scene, and is a scene that does not require dark part correction. Is assumed. Here, the dark part correction amount determination process performed in step S217 will be described with reference to FIG.

  First, image data obtained by photographing with the image sensor 16 is divided into a plurality of blocks in the image processing circuit 50, and an integral value is obtained for each block, thereby obtaining block luminance (S601). Brightness correction is simply performed on the obtained block brightness using the D + amount DP2 obtained in step S205 (S602). This is because when the D + correction is performed, the image data as read from the image sensor 16 is underexposed for the amount of D +, and thus simple brightness correction is performed to return to the original brightness level. Subsequently, face information including the presence / absence of a face, face coordinates, and face size is acquired from the image data whose luminance has been corrected by the image recognition circuit 38 (S605). Then, a luminance value (face luminance) near the face is calculated from the obtained face information and block luminance (S607), and a histogram of the entire screen is generated (S609). The system control unit 60 analyzes the extent of the luminance distribution from the histogram of the entire screen to the low luminance region (S611), and this requires dark portion correction when there is no luminance distribution in the low luminance region. Because there is no, it is used for the judgment. Next, ISO sensitivity (gain) information is acquired as an exposure condition (S613). The dark portion correction is performed in order to obtain an effect of amplifying the level of a pixel having a low luminance level. However, since the luminance is low, image quality deterioration due to S / N deterioration is easily exposed. If the image data is originally obtained with high ISO sensitivity (high gain), the S / N may have already deteriorated, so the ISO sensitivity is used to make a determination to suppress the dark portion correction amount according to the gain value. Get information.

  Then, based on the obtained various information, the maximum dark part correction amount is determined (S615). When a human face is detected, even if it is a backlight scene, if too large dark area correction is applied, the S / N of the human face deteriorates, resulting in a poor impression image. Therefore, as described above, in order to know the original S / N of the image data, the maximum dark part correction amount is determined in consideration of ISO sensitivity information.

  Next, based on the histogram analysis result and the maximum dark area correction amount, it is determined how much dark area correction should be performed for the entire screen (S617). The correction amount determined here is set as the overall dark portion correction amount DarkGainAll. Subsequently, the presence / absence of a face is determined (S619), and if there is no face, the previously determined overall dark portion correction amount DarkGainAll is adopted as the dark portion correction amount DarkGain1 (S625). On the other hand, if there is a face, the face dark portion correction amount DarkGainFace is determined based on the face luminance and the maximum dark portion correction amount obtained in step S607 (S621). Then, the overall dark portion correction amount DarkGainAll and the face dark portion correction amount DarkGainFace are compared (S623), and the larger correction amount is adopted as the dark portion correction amount DarkGain1 (S625 or S627).

  When the dark portion correction amount DarkGain1 is determined as described above, the processing returns to FIG. By performing the processing up to step S217, the correction amount at the time of execution, non-execution, and execution of each correction is determined. Therefore, image processing is performed based on such information (S219), and the corrected image data is stored in the memory card. Save to etc. Here, with reference to FIG. 7, the details of the image processing that can be divided in step S219 will be described.

  First, referring to the D + amount DP2, it is determined whether DP2 is greater than 0, that is, whether D + correction is to be performed (S703). If D + correction is to be performed, a tone curve is determined from the D + amount DP2 (S707).

  During normal shooting, shooting is performed with the aperture, shutter, and gain obtained by the automatic focus adjustment processing, but with D + correction, as described above, exposure is underexposed for the calculated D + amount for this exposure condition. To suppress overexposure. However, if underexposure is performed to suppress overexposure, the brightness of the entire screen will be underexposed. Here, if the tone curve is set so that the luminance value of the luminance area that should not be under-exposure among the image data obtained by under-exposure shooting is corrected so as to return it to the original amount, the high-luminance area Can be suppressed. Also, the brightness level of the brightness area that is not overexposed does not become too dark, and a good result should be obtained.

  FIG. 12 is a diagram for explaining an example of a tone curve used for D + correction, and shows a D + gamma curve 300 for performing D + correction for one stage with respect to the normal gamma curve 302 when D + correction is not performed. When the input luminance 304 that has been normally shot with appropriate exposure without performing D + correction is converted with the normal gamma curve 302, an output luminance 308 is obtained. On the other hand, as described with reference to FIG. 4, when the exposure is underexposed for D + amount (one stage), the input luminance 304 becomes the input luminance 306 having a luminance level of half. Therefore, in order to convert the input luminance 306 having the half luminance level into the output luminance 308 that is not underexposed, that is, the same as the case of photographing with appropriate exposure, the D + gamma curve 300 is applied for conversion. . Here, when the exposure is underexposed, overexposure should be suppressed in the high luminance region. Therefore, as shown in the D + gamma curve 300, by increasing the luminance in the low luminance portion and laying the curve as the luminance increases, it is possible to obtain an output luminance in which overexposure is suppressed. Note that FIG. 12 shows a gamma curve when the D + amount is one stage, but the present invention is not limited to this. For example, the D + amount is set to 1/3 stage, 2/3 stage. It is also possible to make the number of steps finer. In that case, it is necessary to set a gamma curve corresponding to each D + amount. Further, as described above, a plurality of types of tone curves may be prepared in advance according to the D + amount and selected from them, or the tone curve may be calculated from the D + amount.

  When D + correction is not performed (NO in S703), it is determined whether DM1 is larger than 0, that is, whether D-correction is performed (S705) with reference to the D-amount DM1. When performing D-correction, a tone curve is determined from the D-quantity DM1 as described above with reference to FIG. 8 (S709). Similar to the D + correction tone curve, this tone curve may be selected from a plurality of D− tone curves, or may be obtained by calculating a tone curve from the D− amount.

  When neither D + correction nor D- correction is performed, it is determined that it is not necessary to perform specific gradation correction, and a default tone curve (for example, the normal gamma curve 302) is selected (S711). Then, tone curve processing is performed on the image data with any of the tone curves selected in this way (S713).

  Thereafter, the D-amount DM1 is referred to in order to determine whether or not to perform dark part correction (S715). When DM1> 0 is not satisfied (that is, when D-correction is not performed), dark portion correction is permitted, and a dark portion correction tone curve is determined from the dark portion correction amount DarkGain1 obtained in step S217 (S717). Then, dark portion correction processing is performed using this tone curve (S719). In the dark portion correction process, the low resolution image shown in FIG. 10 and the dark portion correction tone curve shown in FIG. 11 are used to perform processing to brightly correct the luminance of the low luminance frequency and low luminance region.

  In this way, by using the dark portion correction amount DarkGain1, the low-resolution image and the dark portion correction tone curve processing described above with reference to FIGS. 10 and 11 are performed, thereby correcting the luminance level of the dark portion without impairing the contrast. be able to.

  On the other hand, when D-correction is performed, execution of dark part correction is limited. In the present embodiment, the implementation is not permitted and the processing is terminated as it is.

  FIG. 14 is a timing chart after the SW2 ON of the gradation correction processing shown in FIG. After the SW2 is turned on, the charge signal (554) read by exposing the image sensor 16 is stored in the temporary storage memory 30 as image data (556), and at the same time, luminance detection (558) and face detection (560). ), Luminance low frequency detection is performed (562). Based on the detection result thus obtained, the dark part correction amount and the D-amount are calculated (564), and the image processing circuit 50 performs gradation correction (566).

  As described above, according to the present embodiment, in an imaging apparatus capable of executing a plurality of gradation correction methods suitable for different luminance distributions of an image, a plurality of gradation correction methods according to the luminance distribution of the captured image. Determine whether or not to implement. An image having good brightness and contrast can be obtained by setting and correcting an appropriate tone curve from the D + amount or D− amount of the gradation correction method to be executed and the dark portion correction amount. In particular, both the D-correction and dark part correction, both of which have the effect of increasing the luminance level, are restricted from being performed. As a result, it is possible to avoid excessive correction and an image that is too bright, so that the image quality does not deteriorate.

<Modification>
In the above embodiment, when D-correction is performed, dark portion correction is not permitted to be performed, but the amount to be performed may be limited. In that case, after determining that DM1> 0 in step S715 in FIG. 7, the same processing as in steps S717 and S719 is performed, but a gamma having a smaller change than the dark part correction gamma shown in FIG. Correction based on That is, the gamma is set to have a smaller gain difference between the luminance regions than when D-correction is not performed.

  10: photographic lens, 12: mechanical shutter, 14: aperture, 16: image sensor, 18: CDS, 20: PGA circuit, 22: A / D circuit, 24: TG, 26: aperture drive circuit, 28: shutter drive circuit , 34: VRAM, 36: D / A circuit, 38: image recognition circuit, 50: image processing circuit, 60: system control unit, 70: operation unit, 108: image display device

Claims (6)

  First determination means for determining first correction data for performing gradation correction for changing the luminance range of the input luminance corresponding to the luminance range of the output luminance for the image data obtained by imaging; Second determination means for determining second correction data for performing gradation correction according to the height of the luminance frequency for the image data obtained by imaging;
  Control means for restricting execution of gradation correction using the other of the first and second correction data for image data subjected to gradation correction using one of the first and second correction data An imaging apparatus comprising:   The imaging apparatus according to claim 1, wherein the first determination unit determines the first correction data based on a luminance distribution of the image data obtained by the imaging.   3. The imaging apparatus according to claim 1, wherein the first determination unit determines the first correction data based on a luminance of a high luminance region of the image data obtained by imaging. .   The said 2nd determination means determines the said 2nd correction data based on the luminance distribution of the image data obtained by the said imaging, The one of Claim 1 thru | or 3 characterized by the above-mentioned. Imaging device.   The control means performs gradation correction using the other of the first and second correction data on image data on which gradation correction using one of the first and second correction data is performed. The imaging device according to claim 1, wherein the imaging device is not.   The first determining means determines first correction data for performing gradation correction for changing the luminance range of the input luminance corresponding to the luminance range of the output luminance for the image data obtained by imaging. A first determination step;
  A second determination step in which a second determination unit determines second correction data for performing gradation correction in accordance with the height of the luminance frequency for the image data obtained by imaging;
  The control means performs gradation correction using the other of the first and second correction data on the image data on which the gradation correction using one of the first and second correction data is performed. Control process to limit,
  An image processing method comprising: