JP4074883B2 - Image processing device - Google Patents

Description

  The present invention relates to an image processing apparatus, and more particularly to an image processing apparatus that performs face recognition and changes an evaluation value of automatic white balance.

  In recent years, with the development of personal computers, the use of digital cameras as image input devices has become widespread. In particular, digital cameras are mostly used by photographers who do not have special operation techniques, and in order to eliminate shooting failures, the shutter speed, exposure, and focus are automatically set according to the subject. There are many things that are. In general, the subject is often a person, and there are digital cameras specially equipped with a person-specific setting mode.

Therefore, Patent Document 1 discloses a new function for image expression such as a portrait that blurs the background of the subject and a correction of the subject color in order to make the subject appear beautiful, regardless of conditions such as illuminance. A video camera that can be realized is disclosed. According to this, when the output signal of the image sensor is processed by the signal processing circuit and converted into a video signal, the extraction means for extracting a specific subject portion from the output video signal of the signal processing circuit, and the signal processing circuit This is realized by providing control means for individually controlling the signal processing parameters of the specific subject part and other parts according to the extraction output of the extraction means.
Further, Patent Document 2 discloses a technique that enables effective automatic focus control, automatic exposure control, automatic white balance control, and electronic zoom control even when the subject is not located at the center of the screen. Has been. According to this, it is an imaging device that converts an optical image obtained via the imaging optical system into an electrical image signal by a CCD and stores it, a read-only memory that stores a specific image pattern, and the stored image A partial image of the signal; a correlation value calculation unit that calculates a correlation value of the specific image pattern; a data read control circuit that detects a maximum value of the correlation value calculated by the correlation value calculation unit; and a detected maximum value This is realized by having an AF, AE, and AWB control circuit for performing predetermined control based on the partial image corresponding to the above.

As described above, Patent Document 1 and Patent Document 2 both recognize a specific subject and perform AWB and AE from information on the subject. In the present invention, when a specific subject (face) is detected, AWB is performed by excluding information of the portion.
JP-A-5-110936 JP 2001-119622 A

In other words, the conventional image processing apparatus having the function of automatically controlling the white balance extracts the achromatic part of the object in the screen and controls the color adjustment so that the part is white (achromatic color). Do. The achromatic part becomes a different shade due to the change of the color of the light source irradiated to the object. In view of this, the range of colors that change when a natural light source is applied to the achromatic portion is defined as the achromatic color detection range. However, when a person was photographed, the color of the person's face and the achromatic part when the light of the low color temperature light source was irradiated was very close, so it was misrecognized, making the face white. There is a problem.
In addition, a method has been proposed in which face detection is performed and white balance is controlled so that the color of the face becomes natural when a face is detected. There is also a problem that cannot be done.

  In view of such a problem, the present invention performs face detection, and does not use color information of the face part for white balance calculation, so that it is possible to perform good image processing so that the face part does not perform an uncomfortable white balance such as pale. An object of the present invention is to provide an image processing apparatus that realizes the above.

In order to solve this problem, the present invention provides an automatic white balance evaluation value that divides a processing target screen into a plurality of areas and acquires an automatic white balance evaluation value that is color information in each divided area. An acquisition unit; an automatic white balance calculation unit that performs white balance calculation based on the automatic white balance evaluation value; a white balance control unit that automatically performs white balance control based on a result of the automatic white balance calculation unit; A face area detecting unit that detects an area where a face exists from all the divided areas of the target screen, and when the area where the face exists is detected by the face area detecting unit The automatic white balance evaluation value acquisition means obtains the automatic white balance evaluation value as a previous value. Obtained from the area where the face is not detected as an area that exists among the divided plurality of areas, characterized in that it does not get from the detected area as an area where the face is present.

The acquisition area of the automatic white balance (AWB) evaluation value is usually the entire screen. The conventional image processing apparatus having the function of automatically controlling the white balance extracts the achromatic part of the object in the screen and controls the color adjustment so that the part is white (achromatic color). Do. Therefore, if there is a person's face in that area, the color of the person's face and the achromatic part when the light of the low color temperature light source is irradiated are very close, so the recognition error occurs and the face is There was a problem that turned white. Therefore, when a face is recognized by the face detection unit in the face recognition operation mode, the area for acquiring the automatic white balance evaluation value is moved from the face part so that the calculation of the white balance of the face part is not performed.
According to this invention, the image processing apparatus further includes face detection means for detecting the presence of a human face in the image to be processed, and when the face detection mode is set and the face detection means detects the face. By changing the area from which the automatic white balance evaluation value is acquired to an area other than the face, good image processing of the face portion can be realized.

According to a second aspect of the present invention, an automatic white balance evaluation value acquisition unit that divides a processing target screen into a plurality of areas and acquires automatic white balance evaluation values that are color information in the divided areas; An automatic white balance calculating means that originally calculates white balance, a white balance control means that automatically performs white balance control based on a result of the automatic white balance calculating means, and among the plurality of divided areas of the processing target screen Face area detecting means for detecting an area where a face is present from all areas, and when the area where the face is present is detected by the face area detecting means, the automatic white balance calculating means includes the division Of the plurality of areas that were not detected as the area where the face exists. The obtained the automatic white balance evaluation value used in the automatic white balance calculation from not using the automatic white balance evaluation value obtained from the detected area as an area where the face is present in the automatic white balance calculation It is characterized by that.

When performing face detection, face detection is performed prior to the imaging operation, and if detected, AWB calculation and control are performed using an AWB evaluation value created from an area other than the area including the face. Is called. By excluding this area from the AWB calculation target by face detection, it is possible to prevent the AWB control with a sense of incongruity from being performed.
According to this invention, the image processing apparatus further includes face detection means for detecting the presence of a human face in the image to be processed, and when the face detection mode is set and the face detection means detects the face. By not using the automatic white balance evaluation value acquired from the area where the face is detected for the automatic white balance calculation, it is possible to prevent the AWB control having a sense of incongruity from being performed.

Claim 3, when the area where there is the face by the face area detection means is not detected, and characterized by setting the area for acquiring the automatic white balance evaluation value with your Keru all area in the process target screen To do.
Some imaging apparatuses have a function for selecting a person shooting mode as the shooting mode and other cases. In a mode other than the portrait shooting mode, the evaluation value acquisition area is designated as a normal area because no face is shot. Similarly, in the case of the human photographing mode, the evaluation value acquisition area is designated as the normal area when the face is not detected by the operation of the face detection process. According to such technical means, when the face detection mode is not set, or when the face is not detected by the face detection means, the area for acquiring the automatic white balance evaluation value is set as a normal area, which is unnecessary. Processing can be omitted.

According to a fourth aspect of the present invention, there is further provided luminance detection means for detecting the brightness of each part of the screen, and the brightness of an area that is not detected as an area where the face exists by the face area detection means among the plurality of divided areas. When the brightness is extremely dark or bright, the automatic white balance evaluation value of the area detected as the area where the face exists is not excluded from the target of the automatic white balance calculation.
In the mode for executing face detection, face detection is performed prior to the imaging operation, and if it can be detected, the brightness difference between the detected face area and the other areas is calculated by the luminance detecting means. In the face detection mode, the person who is the main subject is set to the proper exposure state. Therefore, when the other part is extremely bright or dark, the AWB evaluation value area is not moved or excluded. When the part other than the face is extremely bright, it is backlit, etc. In that case, the part other than the main subject may be saturated, and the color information at that time is accurately detected. There is no guarantee. In addition, when the part other than the face is extremely dark, it means that only the person is illuminated with a spotlight.In this case, the signal of the part other than the main subject is small, so it is affected by noise. It is easy and accurate color information may not be extracted.
Therefore, if the AWB evaluation value acquisition area is moved or excluded based on the face recognition result, there is a possibility that good AWB control cannot be performed.
According to this technical means, when the brightness of the area other than the face part detected by the face detection means is extremely dark or bright, the process of excluding the color information of the face part from the target of the automatic white balance calculation By not performing AWB, malfunction of AWB can be prevented.

According to the present invention, claim 1 further comprises face detection means for detecting the presence of a human face in the image to be processed, and the face detection mode is set to the face detection mode and the face is detected by the face detection means. If detected, by changing the area from which the automatic white balance evaluation value is acquired to an area other than the face, good image processing of the face portion can be realized.
Further, the present invention further comprises face detection means for detecting the presence of a human face in the image to be processed, and when the face detection mode is set and the face detection means detects the face. By not using the automatic white balance evaluation value acquired from the area where the face is detected for the automatic white balance calculation, it is possible to prevent the AWB control having a sense of incongruity from being performed.
According to a third aspect of the present invention, when the face detection mode is not set, or when the face is not detected by the face detection unit, the area for obtaining the automatic white balance evaluation value is set as a normal area, which is unnecessary. Processing can be omitted.
According to a fourth aspect of the present invention, when the brightness of an area other than the face part detected by the face detection unit is extremely dark or bright, the process of excluding the color information of the face part from the target of the automatic white balance calculation By not performing it, it is possible to prevent malfunction of AWB.

Hereinafter, the present invention will be described in detail with reference to embodiments shown in the drawings. However, the components, types, combinations, shapes, relative arrangements, and the like described in this embodiment are merely illustrative examples and not intended to limit the scope of the present invention only unless otherwise specified. .
FIG. 1 is a block diagram of a digital camera according to an embodiment of the present invention. In this configuration, a lens 1 that collects an optical image of a subject, a diaphragm unit 2 that focuses a light beam collected by the lens 1, and a plurality of lenses (not shown) are moved and focused, or the diaphragm unit 2 is A motor driver 11 to be driven, a CCD (Charge Coupled Device) 3 that photoelectrically converts an optical image that has passed through the lens 1 and the diaphragm 2, and a CDS (Correlated Double Sampling) 4 that reduces noise contained in the CCD 3. An A / D converter 5 that converts an analog signal from the CCD 3 into a digital signal, a timing generator 13 that generates the timing of the CCD 3, the CDS 4, and the A / D converter 5, and image processing according to image processing parameters. Imaging by a digital signal processing circuit 7 to be performed, a frame memory 6 for storing a recorded image of an imaging pixel and an image processed image, and a liquid crystal display (LCD) A display unit 8 for displaying an image, an image compression / expansion circuit 9 for compressing or expanding image data processed by the digital signal processing circuit 7 into source image data, and image data compressed by the image compression / expansion circuit 9 are stored. A memory card 10, a CPU 14 for executing predetermined control based on a control program, an EEPROM (Electrically Erasable and Programmable Read Only Memory) 16 for storing parameters, a release button for an operator to operate the camera body, and the like The camera operation unit 17 is provided.

  Next, an outline of the operation of the digital still camera 100 having this configuration will be described with reference to FIG. When the operator looks into the subject from a finder (not shown) and presses the release button of the camera operation unit 17, the CPU 14 detects the signal and drives the lens 1 and the aperture 2 to the motor driver 11 to display the subject image. Focus on CCD3. A series of these operations are automatically performed by the CPU 14 based on sensor information (not shown). The image focused on the CCD 3 is sequentially taken out by the clock generated from the timing generator 13, and the noise contained in the data is reduced by the CDS 4. Here, the reset noise is dominant as the noise included in the output signal of the CCD 3. In order to reduce this noise, the reset noise is canceled by subtracting both from the video signal included in the signal period, the reset noise, and the reset noise included only in the field-through period. The analog signal is converted into a 10-bit digital signal by the A / D converter 5, input to the digital signal processing circuit 7, and temporarily stored in the frame memory 6. Then, the data temporarily stored in the frame memory 6 is processed according to an instruction from the CPU 14 according to the parameters stored in the EEPROM 16, and the processed image is stored in the frame memory 6 again. This processing includes white balance processing. Further, the data written in the frame memory 6 is sent to the control unit of the display unit 8, and the content is displayed on the LCD. The frame memory 6 is an image memory that can store image data of at least one screen of image pickup pixels. For example, a video random access memory (VRAM), a static random access memory (SRAM), a dynamic random access memory (DRAM). ) Or a commonly used memory such as SDRAM (Synchronous DRAM).

Here, when it is desired to record the image on the memory card 10 at the operator's intention, when an instruction is given from the camera operation unit 17, the CPU 14 sends the image to the digital signal processing circuit 7 accordingly. Execute the control to transfer to. That is, the image-processed image is read from the frame memory 6 and sent to the image compression / expansion circuit 9. Here, the image is compressed by the JPEG (Joint Photographic Experts Group) method and stored in the memory card 10. As this encoding algorithm, ADCT (adaptive discrete cosine) is used, and a hierarchical encoding in which an image with a low resolution is first encoded and gradually becomes higher is also adopted. Thus, the memory card 10 is for compressing the data stored in the frame memory 6 and storing the compressed data. In addition to this, for example, it may be configured to record in an internal memory of about 8 MB or a SmartMedia CompactFlash (registered trademark).
Conversely, when displaying the contents of the memory card 10 on the display unit 8 or when transferring image data by connecting to another PC via an external terminal, the desired memory card is inserted into a connector (not shown) of the camera body. Then, when instructed from the camera operation unit 17, the CPU 14 instructs the digital signal processing circuit 7 to read the compressed image data of the memory card, inputs it to the image compression / decompression circuit 9, and decompresses the data according to the compression algorithm. Then, the image is restored and the image is displayed on the display unit 8.

  FIG. 2 is an operation flowchart according to the first embodiment of the present invention. FIG. 3 is a diagram in which an automatic white balance (hereinafter referred to as AWB) evaluation value acquisition area is moved when a face is detected. The operation will be described with reference to FIGS. First, it is determined whether or not the imaging apparatus is in the face recognition operation mode (S1), and when it is not in the face recognition operation mode (NO route), the AWB evaluation value acquisition area in FIG. 3 is designated as the normal area 20 (S5). In the face recognition operation mode in step S1 (YES route), face recognition processing is performed (S2), and it is determined whether or not the face is recognized (S3). When the face is not recognized (NO route), the AWB evaluation value acquisition area in FIG. 3 is designated as the normal area 20 (S5). When the face is recognized in step S3 (YES route), the AWB evaluation value acquisition area in FIG. 3 is moved to the area 21 other than the face 22 (S4). Then, an AWB evaluation value acquisition area is set (S6), and the AWB evaluation value is acquired (S7). Then, the AWB value is calculated (S8), and the final AWB gain is set (S9).

As described above, the automatic white balance (AWB) evaluation value acquisition area 20 is usually the entire screen. The conventional image processing apparatus having the function of automatically controlling the white balance extracts the achromatic part of the object in the screen and controls the color adjustment so that the part is white (achromatic color). Do. Therefore, if there is a person's face 22 in that area, the color of the person's face and the achromatic portion when the light of the low color temperature light source is irradiated are very close, so that the face is erroneously recognized, There was a problem that whitened. Therefore, when a face is recognized by the face detection unit in the face recognition operation mode, the AWB evaluation value acquisition area 21 is moved from the face part so that the white balance of the face part is not calculated. Thereby, favorable image processing of the face part can be realized. Some imaging apparatuses have a function for selecting a person shooting mode as the shooting mode and other cases. In a mode other than the portrait shooting mode, the evaluation value acquisition area is designated as a normal area because no face is shot.
Further, in the case of the person photographing mode, when the face is not detected by the face detection processing operation, the evaluation value acquisition area is designated as the normal area, so that unnecessary processing can be omitted.

  FIG. 4 is an operation flowchart according to the second embodiment of the present invention. FIG. 5 is a diagram in which the AWB evaluation value is acquired by dividing the screen. The operation will be described with reference to FIGS. 4 and 5 together. First, it is determined whether or not the imaging apparatus is in the face recognition operation mode (S11). If the image recognition apparatus is not in the face recognition operation mode (NO route), the AWB evaluation value acquisition area in FIG. 5 is designated as the normal area 23 (S15). In the face recognition operation mode in step S11 (YES route), face recognition processing is performed (S12), and it is determined whether or not the face is recognized (S13). When the face is not recognized (NO route), the AWB evaluation value acquisition area in FIG. 5 is designated as the normal area 23 (S15). If the face is recognized in step S13 (YES route), the area 24 including the face 25 is excluded from the AWB evaluation value acquisition area 23 in FIG. 5 (S14). And the evaluation value from an AWB evaluation value designation | designated area is acquired (S16). Then, the AWB value is calculated (S17), and the final AWB gain is set (S18).

As described above, the mode in the case of performing face detection performs face detection prior to the imaging operation, and if it can be detected, the AWB evaluation value created from an area other than the area including the face is used to perform AWB. Calculation and control are performed. By excluding this area from the AWB calculation target by face detection, it is possible to prevent the AWB control with a sense of incongruity from being performed.
Further, in the mode in which face detection is performed, face detection is performed prior to the imaging operation, and if it can be detected, the brightness difference between the detected face area and the other areas is calculated by the luminance detection means. . In the face detection mode, the person who is the main subject is set to the proper exposure state. Therefore, when the other part is extremely bright or dark, the AWB evaluation value area is not moved or excluded. When the part other than the face is extremely bright, it is backlit, etc. In that case, the part other than the main subject may be saturated, and the color information at that time is accurately detected. There is no guarantee. In addition, when the part other than the face is extremely dark, it means that only the person is illuminated with a spotlight.In this case, the signal of the part other than the main subject is small, so it is affected by noise. It is easy and accurate color information may not be extracted. Therefore, if the AWB evaluation value acquisition area is moved or excluded based on the face recognition result, there is a possibility that good AWB control cannot be performed. Thereby, malfunction of AWB can be prevented.

  The above image processing function can be applied to all devices that handle images. In particular, it is optimal to apply this image processing technique to a digital camera. In that case, it is implement | achieved by further providing the color image imaging means which converts into a color image signal and outputs the optical signal obtained via the optical system. Accordingly, it is possible to provide a digital camera that realizes good image processing without performing a white balance with an uncomfortable feeling such as a pale face.

1 is a block diagram of a digital still camera according to an embodiment of the present invention. It is an operation | movement flowchart which concerns on the 1st Embodiment of this invention. It is the figure which moved the automatic white balance evaluation value acquisition area when the face of this invention was detected. It is an operation | movement flowchart which concerns on the 2nd Embodiment of this invention. It is a figure which divides | segments the screen of this invention and acquires the AWB evaluation value.

Explanation of symbols

  1 Lens, 2 Aperture, 3 CCD, 4 CDS, 5 A / D Converter, 6 Frame Memory, 7 Digital Signal Processing Circuit, 8 Display, 9 Image Compression / Expansion Circuit, 14 CPU

Claims (4)

The processing target screen is divided into a plurality of areas, and an automatic white balance evaluation value acquisition unit that acquires automatic white balance evaluation values that are color information in each of the divided areas, and white balance based on the automatic white balance evaluation values Automatic white balance calculation means for performing calculation, white balance control means for automatically performing white balance control based on a result of the automatic white balance calculation means, and a face from all the divided areas of the processing target screen A face area detecting means for detecting an area where the face is present, and when the face area detecting means detects the area where the face is present, the automatic white balance evaluation value acquiring means includes the automatic white balance evaluation means. The value is an area where the face is present among the divided areas. Obtained from not detected area as an image processing apparatus characterized by not acquired from the detected area as an area where the face is present. The processing target screen is divided into a plurality of areas, and an automatic white balance evaluation value acquisition unit that acquires automatic white balance evaluation values that are color information in each of the divided areas, and white balance based on the automatic white balance evaluation values Automatic white balance calculation means for performing calculation, white balance control means for automatically performing white balance control based on a result of the automatic white balance calculation means, and a face from all the divided areas of the processing target screen A face area detecting means for detecting an area where the face exists, and when the face area detecting means detects the area where the face is present, the automatic white balance calculating means is configured to detect the plurality of divided areas. Out of the areas that are not detected as the area where the face exists Characterized in that the automatic white balance evaluation value used in the automatic white balance operation, do not use the automatic white balance evaluation value obtained from the detected area as an area where the face is present in the automatic white balance calculation was An image processing apparatus.   The area for acquiring the automatic white balance evaluation value is set to all areas in the processing target screen when the area where the face exists is not detected by the face area detecting unit. The image processing apparatus described. It further comprises a luminance detecting means for detecting the brightness of each part of the screen, and the brightness of the area that is not detected as the area where the face is present by the face area detecting means among the plurality of divided areas is extremely dark 4. The process of excluding an automatic white balance evaluation value of an area detected as an area where the face is present from the target of the automatic white balance calculation when the face is bright. The image processing apparatus according to one item.

Images