JP2010119051A - Imaging device and imaging method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an imaging device capable of performing image processing for correcting white balance without increasing a processing load. <P>SOLUTION: The imaging device includes a first division means of dividing an image signal obtained by an imaging element into a plurality of first blocks, a second division means of further dividing the first blocks into M second blocks, a position changing means of changing a second block as an object of operation by frames, an arithmetic means of performing arithmetic processing on the second block changed by the position changing means before recording and on the first blocks during the recording, and a light source estimating means of estimating a light source of the image signal using an arithmetic result of the closest M frames with respect to the second blocks and an arithmetic result with respect to the first blocks. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to an imaging apparatus and an imaging method.

  In an electronic imaging device that electronically captures a subject, such as a digital still camera, image signal processing (white balance correction processing) that corrects white balance so that a white subject is photographed white regardless of the light source of the illumination Is generally incorporated. In the white balance correction process, depending on the illumination light source, a color difference occurs between the subject image observed with the naked eye and the captured image. In this case, the correction process is performed so that a white subject is reproduced in white. For example, there is a difference in color temperature between natural light such as sunlight and artificial light such as a fluorescent lamp, and as a result, the white reproducibility of a captured image is greatly affected. Therefore, by applying the white balance correction process, it is possible to prevent the image reproduced by the imaging apparatus from feeling uncomfortable.

  For white balance correction, the image signal obtained from the image sensor is divided into a plurality of blocks, and an integrated value for each color such as R (red), G (green), and B (blue) is obtained for each divided block. It is determined whether to control as an achromatic color or to process as a chromatic color. In general, white balance control is performed so that an integrated value in a block determined to be an achromatic color is R = G = B.

  However, if mixed colors occur due to the presence of a plurality of colors in the divided blocks, it becomes impossible to correctly determine chromatic colors / achromatic colors. Therefore, a technique is disclosed in which the occurrence of color mixing is suppressed as much as possible by finely dividing a block so that white balance can be judged normally (see, for example, Patent Documents 1 to 4).

JP 2007-336107 A JP 2007-228516 A JP 2006-211144 A JP 2005-12763 A

  However, if the block to be subjected to image signal processing is divided finely in order to suppress the occurrence of color mixing, there is a problem that the processing time for white balance control becomes longer and the processing cannot be completed within one frame. . In particular, during live view in which images obtained from an image sensor are sequentially displayed on a monitor provided in the image pickup device, if the calculation time of the white balance gain is extended, the scene actually viewed with the eye and the live view on the monitor are displayed. A time difference will occur between the displayed scenes. Even during still image shooting, if the processing time of white balance control becomes longer, there is a problem that the time until the end of shooting becomes longer, especially when shooting continuously in a short time, etc. An operation delay to the photographing process occurs. Furthermore, when the mobile phone or other mobile terminal is provided with an image sensor and has a camera function, it is needless to say that the processing time can be shortened from the viewpoint of reducing power consumption.

  Therefore, the present invention has been made in view of the above problems, and an object of the present invention is to finely divide a block to be subjected to image signal processing at a stage before recording in order to suppress the occurrence of color mixing. However, at the time of recording, a new and improved imaging apparatus and imaging capable of performing image signal processing for correcting white balance without increasing the processing load by using the evaluation results for the finely divided blocks at the stage before recording. It is to provide a method.

  In order to solve the above-described problem, according to one aspect of the present invention, a first dividing unit that divides an image signal obtained from an image sensor into a plurality of first blocks, and the first block is further divided into M second blocks. A second dividing means for dividing the block into blocks, a position changing means for changing the second block to be calculated for each frame, and a calculation process for the second block changed by the position changing means before recording, Calculation means for performing calculation processing on the first block at the time of recording, and light source estimation means for estimating the light source of the image signal using the calculation result of the calculation means for the nearest M frame for the second block and the calculation result for the first block Is provided.

  According to this configuration, the first dividing unit divides the image signal obtained from the image sensor into a plurality of first blocks, and the second dividing unit further divides the first block into M second blocks. Then, the position changing unit changes the second block to be calculated for each frame, and the light source estimating unit uses the calculation result of the calculation unit for the most recent M frame for the second block and the calculation result for the first block. Estimate the light source. As a result, the block to be subjected to image signal processing is finely divided as the second block at the stage before recording, and at the time of recording, the calculation result for the finely divided second block at the stage before recording and the first at the time of recording are recorded. By using the calculation result for the block, image signal processing can be performed without increasing the processing load.

  A storage means for storing at least M frames of light source estimation results of the light source estimation means, and a white balance correction using the light source estimation results for the most recent M frames and the light source estimation results for the first block stored in the storage means And gain calculating means for calculating the gain to be performed.

  The gain calculation means may determine a predetermined weighting factor using the light source estimation results for the most recent M frames for the second block, and calculate the gain by multiplying the calculation result for the first block by the predetermined weighting factor. .

  The predetermined weight coefficient may be a larger coefficient as the number of small blocks determined to be achromatic as a result of light source estimation in the light source estimation means is larger.

  In order to solve the above-described problem, according to another aspect of the present invention, a first dividing step of dividing an image signal obtained from an image sensor into a plurality of first blocks, and further M first blocks. A second dividing step for dividing the second block into a second block, a position changing step for changing the second block to be calculated for each frame, and a calculation process for the second block changed in the position changing step before recording. The light source of the image signal is estimated using a calculation step for performing calculation processing on the first block at the time of recording, and a calculation result of the most recent M frame for the second block and a calculation result for the first block in the calculation step. A light source estimation step.

  As described above, according to the present invention, at the stage before recording, the block to be subjected to image signal processing is finely divided to suppress the occurrence of color mixing, and at the time of recording, it is finely divided at the stage before recording. By using the evaluation result for the block, it is possible to provide a new and improved imaging apparatus and imaging method capable of image signal processing for correcting white balance without increasing the processing load. This reduces the processing time for white balance correction during still image shooting, especially during high-speed continuous shooting, without being affected by the time difference between the screen display before shooting and still image shooting. Can do.

  Exemplary embodiments of the present invention will be described below in detail with reference to the accompanying drawings. In addition, in this specification and drawing, about the component which has the substantially same function structure, duplication description is abbreviate | omitted by attaching | subjecting the same code | symbol.

  First, before describing a preferred embodiment of the present invention, problems of the prior art will be described with reference to the drawings.

  FIG. 10 is an explanatory diagram for explaining block division in the conventional white balance correction processing. As shown in FIG. 10, the conventional white balance correction process divides an image signal taken out from an image sensor such as a CCD or CMOS into a plurality of blocks, and R, G, The integrated values (ΣR, ΣG, ΣB) of the pixels of each color B are calculated. When the integrated values (ΣR, ΣG, ΣB) of the R, G, and B pixels are calculated, it is determined for each block whether the block is controlled as an achromatic color or processed as a chromatic color. A general method is to calculate an appropriate white balance gain (Kr, Kg, Kb).

  Here, when a plurality of colors are mixed in the divided blocks, color mixing occurs and the white balance correction process cannot be correctly determined. Further, when the area of the block to be divided is increased, the frequency of color mixing increases.

  FIG. 11 is an explanatory diagram showing an example of an image obtained from an image signal taken out from an image sensor such as a CCD or CMOS, which is a target of white balance correction processing, and FIG. 12 shows the image shown in FIG. FIG. 13 is a diagram illustrating a case where the obtained image signal is divided into 4 horizontal blocks × 3 vertical blocks. FIG. 13 illustrates a case where the image signal from which the image illustrated in FIG. 11 is obtained is divided into 12 horizontal blocks × 9 vertical blocks. It is explanatory drawing which shows. In normal white balance correction processing, the red petal portion and the green leaf portion in the image are not regarded as light sources and are determined to be chromatic colors. However, if red and green are mixed in the block during the integration processing in each block, color mixing occurs and it is erroneously determined that the subject is yellow (mixed red and green). If this yellow subject has the same color balance as a light source such as tungsten light, it is determined as the light source color (achromatic color), and the white balance is erroneously determined. Therefore, in order to reduce the occurrence frequency of color mixing, there is a method of increasing the number of blocks and reducing the area of the blocks.

  FIG. 14 is an explanatory diagram showing the arrangement of blocks that are erroneously determined to be yellow subjects due to color mixing when the image signal is divided into 4 horizontal blocks × 3 vertical blocks, and FIG. It is explanatory drawing which shows arrangement | positioning of the block mistakenly determined to be a yellow photographic subject by color mixture, when it divides | segments into 12 blocks x vertical 9 blocks.

  In the case shown in FIGS. 14 and 15, 83% of the blocks are erroneously determined to be yellow subjects when divided into 4 × 3, whereas 28% when divided into 12 × 9. It is misjudged as a yellow subject. Thus, the frequency of color mixing can be suppressed by increasing the number of blocks and reducing the area per block.

  However, if the number of blocks is increased, the number of data that must be processed at one time increases, and the processing time until the white balance gain is calculated increases, making it impossible to complete the processing within one frame. There was a problem that. In particular, during live view in which images obtained from an image sensor are sequentially displayed on a monitor provided in the image pickup device, if the calculation time of the white balance gain is extended, the scene actually viewed with the eye and the live view on the monitor are displayed. There will be a time difference from the displayed scene.

  Even during still image shooting, if the processing time of white balance control becomes longer, there is a problem that the time until the end of shooting becomes longer, especially when shooting continuously in a short time, etc. An operation delay to the photographing process occurs. Furthermore, when the mobile phone or other mobile terminal is provided with an image sensor and has a camera function, it is needless to say that the processing time can be shortened from the viewpoint of reducing power consumption.

  Therefore, in a preferred embodiment of the present invention described below, an imaging apparatus and an image processing method characterized by suppressing an increase in processing time while reducing the block area in order to suppress the occurrence frequency of color mixing. Will be described.

  First, the configuration of an imaging apparatus according to an embodiment of the present invention will be described. FIG. 1 is an explanatory diagram illustrating a configuration of an imaging apparatus 100 according to an embodiment of the present invention. Hereinafter, the configuration of the imaging apparatus 100 according to the embodiment of the present invention will be described with reference to FIG.

  As shown in FIG. 1, an imaging apparatus 100 according to an embodiment of the present invention includes a zoom lens 102, a diaphragm 104, a focus lens 106, a CCD (Charge Coupled Devices) element 108, and an amplifier-integrated CDS. (Correlated Double Sampling) circuit 110, A / D converter 112, image input controller 114, image signal processing unit 116, compression processing unit 120, LCD (Liquid Crystal Display) driver 122, LCD 124, timing Generator 126, motor drivers 142a, 142b, 142c, CPU (Central Processing Unit) 128, operation unit 132, memory 134, VRAM (Video Rand) And m Access Memory) 136, a media controller 138, and a recording medium 140.

  The zoom lens 102, the diaphragm 104, the focus lens 106, and the CCD element 108 constitute an exposure unit. In the present embodiment, the exposure unit is configured using the CCD element 108, but the present invention is not limited to this example, and a CMOS (Complementary Metal Oxide Semiconductor) element may be used instead of the CCD element. Since the CMOS element can convert the image light of the subject into an electric signal at a higher speed than the CCD element, it is possible to shorten the time from when the subject is photographed until the image is recorded.

  The zoom lens 102 is a lens whose focal length continuously changes by moving back and forth in the optical axis direction, and shoots while changing the size of the subject. The diaphragm 104 adjusts the amount of light entering the CCD element 108 when taking an image. The focus lens 106 adjusts the focus of the subject by moving back and forth in the optical axis direction.

  Motor drivers 142a, 142b, and 142c control motors that operate the zoom lens 102, the diaphragm 104, and the focus lens 106. By operating the zoom lens 102, the diaphragm 104, and the focus lens 106 via the motor drivers 142a, 142b, and 142c, the size of the subject, the amount of light, and the focus are adjusted.

  The CCD element 108 is an element for converting light incident from the zoom lens 102, the diaphragm 104, and the focus lens 106 into an electric signal. In this embodiment, the incident light is controlled by the electronic shutter to adjust the time for extracting the electric signal. However, the time for extracting the electric signal may be adjusted by controlling the incident light using the mechanical shutter.

  The CDS circuit 110 is a circuit in which a CDS circuit that is a kind of sampling circuit that removes noise from the electrical signal output from the CCD element 108 and an amplifier that amplifies the electrical signal after removing the noise are integrated. . In the present embodiment, the imaging apparatus 100 is configured using a circuit in which a CDS circuit and an amplifier are integrated. However, the CDS circuit and the amplifier may be configured as separate circuits.

  The A / D converter 112 converts the electrical signal generated by the CCD element 108 into a digital signal, and generates raw image data.

  The image signal processing unit 116 is a circuit that performs various signal processing on the raw image data generated by the A / D converter 112.

  The compression processing unit 120 performs a compression process for compressing the data subjected to the signal processing by the image signal processing unit 116 into image data of an appropriate format. The image compression format may be a reversible format or an irreversible format. As an example of a suitable format, you may convert into a JPEG (Joint Photographic Experts Group) format and a JPEG2000 format.

  The LCD 124 performs live view display before performing a shooting operation, various setting screens of the imaging apparatus 100, display of captured images, and the like. Display of image data and various information of the imaging apparatus 100 on the LCD 124 is performed via the LCD driver 122.

  The timing generator 126 inputs a timing signal to the CCD element 108. The shutter speed is determined by the timing signal from the timing generator 126. That is, the drive of the CCD element 108 is controlled by the timing signal from the timing generator 126, and the image signal from the subject is incident within the time that the CCD element 108 is driven, thereby generating an electrical signal that is the basis of the image data. The

  The CPU 128 issues a signal-related command to the CCD element 108, the CDS circuit 110, and the like, and issues an operation-related command for the operation of the operation unit 132. In the present embodiment, only one CPU is included. However, in the present invention, the signal system instruction and the operation system instruction may be executed by separate CPUs or DSPs (Digital Signal Processors).

  The operation unit 132 includes a function as a shooting mode selection unit, and members for operating the imaging apparatus 100 and various settings at the time of shooting are arranged. The members arranged in the operation unit 132 include a power button, a cross key and a selection button for selecting a shooting mode and a shooting drive mode and setting an effect parameter, a shutter button for starting a shooting operation, and the like.

  The memory 134 temporarily stores captured images and images subjected to signal processing by the image signal processing unit 116. The memory 134 has a storage capacity sufficient to store a plurality of images. Reading and writing of images to and from the memory 134 is controlled by the image input controller 114.

  The VRAM 136 holds contents displayed on the LCD 124, and the resolution and the maximum number of colors of the LCD 124 depend on the capacity of the VRAM 136.

  The recording medium 140 records a photographed image. Input / output to / from the recording medium 140 is controlled by the media controller 138. As the recording medium 140, a memory card that is a card-type storage device that records data in a flash memory can be used.

  The configuration of the imaging apparatus 100 according to the embodiment of the present invention has been described above using FIG. Next, the configuration of the image signal processing unit 116 included in the imaging device 100 according to the embodiment of the present invention will be described.

  FIG. 2 is an explanatory diagram illustrating the configuration of the image signal processing unit 116 included in the imaging apparatus 100 according to the embodiment of the present invention. Hereinafter, the configuration of the image signal processing unit 116 included in the imaging apparatus 100 according to the embodiment of the present invention will be described with reference to FIG.

  As shown in FIG. 2, the image signal processing unit 116 according to the embodiment of the present invention includes an image front process processing unit 152, an image evaluation unit 154, a white balance gain calculation unit 156, and a white balance correction unit 158. A demosaic processing unit 160, a color correction processing unit 162, and a gamma correction processing unit 164.

  The image front process processing unit 152 performs image front process processing such as defective pixel correction and black level correction on the image signal photoelectrically converted by the CCD element 108 and converted into a digital signal by the A / D converter. Is. The image signal subjected to the image front process processing by the image front process processing unit 152 is sent to the image evaluation unit 154 and the white balance correction unit 158.

  The image evaluation unit 154 calculates a white balance gain for correcting the white balance by the white balance gain calculation unit 156 with respect to the image signal subjected to the image front process processing by the image front process processing unit 152. The image evaluation process is executed. The image evaluation unit 154 divides the image signal that has been subjected to the image front process process into a plurality of blocks, obtains an integrated value (ΣR, ΣG, ΣB) of RGB pixels for each of the divided blocks, The light source is estimated based on the integrated value of. The configuration of the image evaluation unit 154 will be described in detail later. The result of the image evaluation process in the image evaluation unit 154 is sent to the white balance gain calculation unit 156.

  The white balance gain calculation unit 156 calculates the white balance gain (Kr, Kg, Kb) based on the result of the image evaluation process in the image evaluation unit 154. The white balance gain calculated by the white balance gain calculation unit 156 is sent to the white balance correction unit 158.

  The white balance correction unit 158 multiplies the image signal subjected to the image front process processing by the image front process processing unit 152 by the white balance gain calculated by the white balance gain calculation unit 156. The white balance correction unit 158 can execute white balance correction processing on the image signal by multiplying the image signal with white balance gain. The image signal subjected to the white balance correction process is sent to the demosaic processing unit 160.

  The demosaic processing unit 160 performs color complementation processing (demosaic processing) on the image signal that has been subjected to white balance correction processing. A color image can be generated by performing a demosaic process on the image signal subjected to the white balance correction process by the demosaic processing unit 160, but the details of the demosaic process are not directly related to the present invention. Therefore, detailed description is omitted. The image signal that has been subjected to the demosaic processing is sent to the color correction processing unit 162.

  The color correction processing unit 162 performs color correction processing on the image signal that has been subjected to the complementary processing (demosaic processing) by the demosaic processing unit 160. An appropriate color image can be generated by performing color correction processing in the color correction processing unit 162, but details of the color correction processing are not directly related to the present invention, and thus detailed description thereof is omitted. The image signal subjected to the color correction processing is sent to the gamma correction processing unit 164.

  The gamma correction processing unit 164 performs gamma correction processing on the image signal that has been subjected to color correction processing by the color correction processing unit 162. Since the details of the gamma correction processing are not directly related to the present invention, a detailed description thereof will be omitted. The image signal subjected to the gamma correction processing is sent to the compression processing unit 120 to be subjected to compression processing, recorded on the recording medium 140, or displayed on the LCD 124.

  The configuration of the image signal processing unit 116 according to the embodiment of the present invention has been described above. Next, the configuration of the image evaluation unit 154 included in the image signal processing unit 116 according to the embodiment of the present invention will be described.

  FIG. 3 is an explanatory diagram illustrating the configuration of the image evaluation unit 154 included in the image signal processing unit 116 according to the embodiment of the present invention. Hereinafter, the configuration of the image evaluation unit 154 included in the image signal processing unit 116 according to the embodiment of the present invention will be described with reference to FIG.

  As shown in FIG. 3, the image evaluation unit 154 according to an embodiment of the present invention includes a first block dividing unit 172, a second block dividing unit 174, a block position changing unit 176, and an integration processing unit 178. The storage unit 180 and the light source estimation unit 182 are included.

  The first block division unit 172 divides the image signal sent to the image evaluation unit 154 into blocks (large blocks) of a predetermined size. For example, as described above, the first block dividing unit 172 may divide the image signal into 4 horizontal blocks × 3 vertical blocks, or may divide the image signal into 12 horizontal blocks × 9 vertical blocks. Of course, it goes without saying that the number of divisions is not limited to these examples. Further, the shape of the large block at the time of division may be a square, a rectangle, or another shape.

  The second block dividing unit 174 divides each large block divided by the first block dividing unit 172 into blocks (small blocks) having a predetermined size. For example, the first block dividing unit 172 divides the image signal into large blocks of horizontal 4 blocks × vertical 3 blocks, and each large block is divided into horizontal 3 blocks × vertical 3 blocks by the second block dividing unit 174. Divide into small blocks. Of course, it goes without saying that the number of divisions is not limited to this example. In addition, the shape of the small block at the time of division may be a square, a rectangle, or other shapes. In the following description, it is assumed that one large block is divided into nine small blocks of 3 horizontal blocks × 3 vertical blocks.

  FIG. 4 is an explanatory diagram showing a case where the second block dividing unit 174 divides one large block into small blocks of 3 horizontal blocks × 3 vertical blocks. In FIG. 4, for convenience of description, alphabets A to I are attached to the small blocks.

  The block position changing unit 176 changes a small block to be processed by an integration processing unit 178 described later for each frame at a stage before the imaging process (a stage before recording in the imaging apparatus 100), and performs an imaging process stage (imaging process). At the stage of recording in the apparatus 100), a large block to be processed by the integration processing unit 178 is designated. For example, when one large block is divided into nine small blocks of 3 horizontal blocks × 3 vertical blocks as shown in FIG. 4 by the second block dividing unit 174, the block position changing unit 176 One small block is cyclically selected from the small blocks A to I for each frame.

  FIG. 5 is an explanatory diagram for explaining the block position changing process in the block position changing unit 176 at the stage before the imaging process. FIG. 5 shows an example in which the small block to be processed is changed for each frame from the small blocks A to I divided by the second block dividing unit 174. FIG. 5 shows a case where the small blocks are changed in the order of A → E → G → C → H → F → B → D → I → A →. Of course, it goes without saying that the order is not limited to that shown in FIG. In this embodiment, the block position changing unit 176 cyclically changes the small frame. However, in the present invention, the small block to be processed by the integration processing unit 178 may be changed randomly.

  When the photographer operates the operation unit 132 of the image capturing apparatus 100 to execute the image capturing process, the block position changing unit 176 specifies a large block to be processed by the integration processing unit 178. FIG. 6 is an explanatory diagram for explaining the block position changing process in the block position changing unit 176 before and after the imaging process. In the stage before the imaging process, the block position changing unit 176 changes the small block to be processed for each frame from the small blocks A to I divided by the second block dividing unit 174, and the imaging process is performed. When started, the block position changing unit 176 designates the large block J to be processed.

  The integration processing unit 178 calculates the white balance gain for the R, G, and B color pixels included in the small block or large block specified by the block position changing unit 176 in the small block. The integrated values ΣR, ΣG, ΣB of the RGB pixels and the integrated values ΣR2, ΣG2, ΣB2 of the RGB pixels in the large block are obtained. By determining the integrated values ΣR, ΣG, ΣB, ΣR2, ΣG2, and ΣB2 of the RGB pixels, the coloring in the small block or the large block can be determined. The integrated values ΣR, ΣG, ΣB, ΣR2, ΣG2, and ΣB2 of the RGB pixels obtained by the integration processing unit 178 and the color determination result in the small block or large block may be temporarily stored in the storage unit 180. good. Then, the integrated values ΣR, ΣG, ΣB, ΣR2, ΣG2, and ΣB2 of the RGB pixels obtained in the integration processing unit 178 are sent to the light source estimation unit 182 and used for the light source estimation process in the light source estimation unit 182.

  The light source estimation unit 182 executes light source estimation processing for estimating a light source using the RGB pixel integrated values ΣR, ΣG, ΣB, ΣR2, ΣG2, and ΣB2 obtained by the integration processing unit 178. The result of the light source estimation process in the light source estimation unit 182 may be temporarily stored in the storage unit 180. The result of the light source estimation processing in the light source estimation unit 182 is sent to the white balance gain calculation unit 156 and used for calculation of the white balance gain.

  FIG. 7 is an explanatory diagram illustrating an example of a relationship between a block to be evaluated and a frame when the light source estimation unit 182 estimates a light source. The example shown in FIG. 7 shows the light source estimation results of the latest nine frames of small blocks (A to I) acquired in the previous stage of the imaging process, and the large frame (J) acquired in the stage of the imaging process. ) The case where the light source estimation process is executed using the light source estimation result is shown.

  In addition, although this embodiment demonstrated the structure which included the light source estimation part 182 in the image evaluation part 154, it cannot be overemphasized that this invention is not limited to this example. For example, the function of the light source estimation unit 182 may be included in the white balance gain calculation unit 156.

  The configuration of the image evaluation unit 154 included in the image signal processing unit 116 according to the embodiment of the present invention has been described above. Next, an imaging method according to an embodiment of the present invention will be described.

  FIG. 8 is a flowchart illustrating an imaging method according to an embodiment of the present invention. Hereinafter, an imaging method according to an embodiment of the present invention will be described with reference to FIG.

  First, the block position changing unit 176 selects a small block to be evaluated for white balance (step S102). As described above, the small block to be evaluated for white balance selected by the block position changing unit 176 is changed for each frame.

  When the block position changing unit 176 selects a small block for white balance evaluation in step S102, the integration processing unit 178 calculates the integrated values ΣR, ΣG, and ΣB of the RGB pixels in the small block (step S104). . When the integration values ΣR, ΣG, ΣB of the RGB pixels are calculated by the integration processing unit 178 in step S104, the light source estimation unit 182 then uses the integration values ΣR, ΣG, ΣB of the RGB pixels to color the small block. Is determined (step S106).

  In step S106, when the light source estimation unit 182 determines the coloring of the small block using the integrated values ΣR, ΣG, and ΣB of the RGB pixels, the small block is subsequently used using the coloring determination result of the small block. Are estimated by the light source estimation unit 182 to determine whether the small block is a chromatic color or an achromatic color (step S108). Information regarding whether the color is a chromatic color or an achromatic color may be a binary value or a value of three or more levels. Then, when the light source of the small block is estimated by the light source estimation unit 182 in step S108, the estimation result is stored in the storage unit 180 (step S110).

  When the light source estimation result is stored in the storage unit 180 in step S <b> 110, a white balance gain is calculated using the estimation result stored in the storage unit 180. First, the white balance gain calculation unit 156 reads the estimation result for the most recent frame stored in the storage unit 180 (step S112). In this embodiment, since one large block is divided into nine small blocks, in step S112, the white balance gain calculation unit 156 reads the estimation results for the latest nine frames from the storage unit 180.

  When the estimation result in the most recent frame stored in the storage unit 180 is read in step S112, the white balance gain calculation unit 156 performs weighted averaging on the read estimation result (step S114). Here, an example of a weighted average will be described. Assuming that the evaluation frame is f, the number of blocks handled as achromatic colors in each frame is n, and the RGB integrated values of the blocks handled as achromatic colors are r, g, b, the color balances Cr, Cg, Cb for 9 frames are as follows: It can obtain | require by Numerical formula 1 of these.

  When the color balances Cr, Cg, and Cb for nine frames are obtained as in Equation 1, the white balance gain calculation unit 156 then uses the obtained color balances Cr, Cg, and Cb to obtain the white balance gain for the image signal. Is calculated (step S116). The white balance gains Kr, Kg, Kb can be obtained by the following formula 2.

  The above formula 2 is a calculation formula when the value of Cg is the largest among the color balances Cr, Cg, and Cb. When the values of other colors are large, the white balance gain of that color is 1. Thus, the above formula is changed as appropriate.

  When the white balance gain is calculated in step S116, it is subsequently determined whether or not a photographing operation by the photographer has been executed (step S118). The photographing operation by the photographer may be a press of a shutter button included in the operation unit 132. Further, the CPU 128 may determine whether or not the photographing operation by the photographer has been executed.

  As a result of the determination in step S118, when it is determined that the photographing operation is not being performed by the photographer, the process returns to step S102, and the block position changing unit 176 selects a small block to be evaluated for white balance. To do. On the other hand, if it is determined as a result of the determination in step S118 that the shooting operation has been performed by the photographer, the block position changing unit 176 selects a large block that is to be evaluated for white balance (step S120).

  In step S120, when the block position changing unit 176 selects a large block to be evaluated for white balance, the integration processing unit 178 subsequently calculates the integrated values ΣR2, ΣG2, and ΣB2 of the RGB pixels in the large block. (Step S122). When the integration values ΣR2, ΣG2, and ΣB2 of the RGB pixels are calculated by the integration processing unit 178, the light source estimation unit 182 then uses the integration values ΣR2, ΣG2, and ΣB2 of the RGB pixels in the large block to color the large block. Is determined (step S124).

  If the light source estimation unit 182 determines the coloring of the large block in step S124, the light source estimation result of the small block stored in the storage unit 180 in step S110 is read for the latest nine frames (step S126). The white balance gain calculation unit 156 calculates a white balance gain using the light source estimation results for the last nine frames read out in step S126.

  In order to calculate the white balance gain by the white balance gain calculation unit 156, first, the weight for the calculation processing result of the large block is determined (step S128). The weight for the calculation processing result of the large block may be determined depending on the number of small blocks determined to be achromatic. FIG. 9 is an explanatory diagram illustrating an example of the relationship between the number of small blocks determined to be achromatic and the weighting factor for the calculation processing result of the large block. In the graph shown in FIG. 9, the horizontal axis represents the number of small blocks determined to be achromatic, and the vertical axis represents the weighting factor for the calculation processing result of the large block. In the example shown in FIG. 9, when the number of small blocks determined to be achromatic increases, the weighting factor is increased in a quadratic function. Of course, it goes without saying that the relationship between the number of small blocks determined to be achromatic and the weighting coefficient is not limited to the example shown in FIG.

  Referring to the graph shown in FIG. 9, when the maximum number of small blocks determined to be achromatic is nine, the maximum weighting factor is 1.0. In addition, when there are eight small blocks determined to be achromatic, the weighting factor is set to 0.75. When there are seven small blocks determined to be achromatic, the weighting factor is set to 0.75. 0.5. In this way, when applying a weighting factor that represents the reliability of whether a large block is achromatic when shooting a still image, in a large block where color mixture is likely to occur, the calculation for the large block is performed. The degree of influence of processing results can be reduced to reduce misjudgments.

  When the weight for the calculation processing result of the large block is determined in step S128, the light source in the large block selected in step S120 is estimated using the determined weight (step S130). Then, using the light source information estimated in step S130, the white balance gain calculation unit 156 calculates a white balance gain (step S132). The imaging apparatus 100 corrects the white balance by applying the calculated white balance gain by the white balance correction unit 158.

  The imaging method according to the embodiment of the present invention has been described above using FIG. In this way, signal processing is performed on small blocks with different positions across multiple frames at the time before shooting, and at the time of shooting, the signal processing results for the small blocks in the immediately preceding frame and the large size at the time of shooting operation execution. By using the result of signal processing on the block, it is possible to perform smaller block division without increasing the processing load per frame, and to suppress color mixing.

  As described above, according to one embodiment of the present invention, in the stage before recording, a block to be subjected to image signal processing is finely divided as small blocks in order to suppress the occurrence of color mixing. Image signal processing that corrects white balance without increasing the processing load by using the evaluation result for a large block of small blocks and the evaluation result for a finely divided small block in the most recent frame before recording. Is possible. Further, the gain for correcting the white balance is calculated based on the integrated value of the pixels at the time of shooting, and the data calculated in the past frame is used only for determining the weighting coefficient, and therefore occurs when the block is finely divided. No adverse effects due to delays in calculation of evaluation data occur. And, especially during high-speed continuous shooting, where multiple still images are shot continuously in a short period of time, white balance correction processing is performed during still image shooting without being affected by the time difference between the screen display before shooting and still image shooting. Since the burden can be reduced, the processing time can be shortened.

  Note that the above-described operation of the imaging apparatus 100 may be performed by storing a computer program in the imaging apparatus 100, and reading out the computer program and sequentially executing the computer program.

  As mentioned above, although preferred embodiment of this invention was described referring an accompanying drawing, it cannot be overemphasized that this invention is not limited to the example which concerns. It will be apparent to those skilled in the art that various changes and modifications can be made within the scope of the claims, and these are naturally within the technical scope of the present invention. Understood.

  The present invention can be applied to an imaging apparatus and an image processing method, and in particular, can be applied to an electronic imaging apparatus that electronically captures a subject and an image processing method using the electronic imaging apparatus.

It is explanatory drawing shown about the structure of the imaging device 100 concerning one Embodiment of this invention. FIG. 10 is an explanatory diagram illustrating a configuration of an image signal processing unit 116. FIG. 10 is an explanatory diagram illustrating a configuration of an image evaluation unit 154. It is explanatory drawing shown about the case where one large block is divided | segmented into the small block of 3 horizontal blocks x 3 vertical blocks. It is explanatory drawing explaining the change process of the block position in the block position change part 176 in the step before an imaging process. It is explanatory drawing explaining the change process of the block position in the block position change part 176 before and behind an imaging process. It is explanatory drawing explaining an example of the relationship between the block and frame of evaluation object at the time of estimating a light source in the light source estimation part 182. FIG. It is a flowchart explaining the imaging method concerning one Embodiment of this invention. It is explanatory drawing which shows an example of the relationship between the number of the small blocks judged to be an achromatic color, and the weight with respect to the light source estimation result of a small block. It is explanatory drawing explaining the block division | segmentation in the conventional white balance correction process. It is explanatory drawing which shows an example of the image obtained from the image signal taken out from image pick-up elements, such as CCD and CMOS used as the object of a white balance correction process. It is explanatory drawing which shows the case where the image signal from which the image shown in FIG. 11 is obtained is divided into 4 horizontal blocks × 3 vertical blocks. It is explanatory drawing which shows the case where the image signal from which the image shown in FIG. 11 is obtained is divided into horizontal 12 blocks × vertical 9 blocks. It is explanatory drawing which shows an example of generation | occurrence | production of the color mixture at the time of dividing | segmenting an image signal into 4 horizontal blocks x 3 vertical blocks. It is explanatory drawing which shows an example of generation | occurrence | production of the color mixture at the time of dividing | segmenting an image signal into horizontal 12 blocks x vertical 9 blocks.

Explanation of symbols

DESCRIPTION OF SYMBOLS 100 Imaging device 102 Zoom lens 104 Aperture 106 Focus lens 108 CCD element 110 CDS circuit 112 A / D converter 114 Image input controller 116 Image signal processing part 120 Compression processing part 122 LCD driver 124 LCD
126 Timing Generator 128 CPU
132 Operation unit 134 Memory 138 Media controller 140 Recording medium 142a, 142b, 142c Motor driver 152 Image front process processing unit 154 Image evaluation unit 156 White balance gain calculation unit 158 White balance correction unit 160 Demosaic processing unit 162 Color correction processing unit 164 Gamma Correction processing unit 172 First block dividing unit 174 Second block dividing unit 176 Block position changing unit 178 Integration processing unit 180 Storage unit 182 Light source estimation unit

Claims (5)

First dividing means for dividing an image signal obtained from the image sensor into a plurality of first blocks;
Second dividing means for further dividing the first block into M second blocks;
Position changing means for changing the second block to be calculated for each frame;
Calculation means for performing calculation processing on the second block changed by the position change means before recording, and calculation processing for the first block during recording;
Light source estimation means for estimating the light source of the image signal using the calculation result of the most recent M frame for the second block and the calculation result for the first block of the calculation means;
An imaging apparatus comprising: Storage means for storing light source estimation results of the light source estimation means for at least M frames;
Gain calculating means for calculating a gain for correcting white balance using a light source estimation result for the latest M frames for the second block and a light source estimation result for the first block stored in the storage means;
The imaging apparatus according to claim 1, further comprising:   The gain calculating means determines a predetermined weighting factor using the light source estimation results for the most recent M frames for the second block, and calculates the gain by multiplying the calculation result for the first block by the predetermined weighting factor. The imaging device according to claim 2.   The imaging apparatus according to claim 3, wherein the predetermined weight coefficient is a coefficient that increases as the number of small blocks determined to be achromatic as a result of light source estimation by the light source estimation unit increases. A first dividing step of dividing an image signal obtained from the image sensor into a plurality of first blocks;
A second dividing step of further dividing the first block into M second blocks;
A position changing step for changing the second block to be calculated for each frame;
An arithmetic step for performing arithmetic processing on the second block changed in the position changing step before recording, and for performing arithmetic processing on the first block at the time of recording;
A light source estimation step of estimating a light source of the image signal using a calculation result of the latest M frame for the second block and a calculation result for the first block in the calculation step;
An imaging method comprising:

Images