JP2003189325A - Image processing apparatus and color image photographic apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image processing apparatus capable of realizing excellent image processing by detecting a face and not employing color information of the face for a white balance arithmetic operation so as to prevent white balance having a sense of incongruity such as a pale face from being controlled. <P>SOLUTION: The method employing the image processing apparatus includes: a step S1 of discriminating whether or not an imaging apparatus is set to a face recognition operating mode; a step S5 of designating an AWB (Automatic White Balance) value acquisition area to an ordinary area 20 when the mode is not set to the face recognition operating mode; a step S2 of performing face recognition processing when the mode is set to the face recognition operating mode; a step S3 of discriminating whether or not a face is recognized; a step S5 of designating the AWB evaluation value acquisition area to the ordinary area 20 when no face is recognized; a step S4 of shifting the AWB evaluation value acquisition area to other area 21 than the face 22 when the step S3 recognizes the face (a YES branch in the step S3); a step S6 of setting the AWB evaluation value acquisition area; a step S7 of acquiring the AWB evaluation value; a step S8 of calculating an AWB value; and a step S9 of setting a final AWB gain. <P>COPYRIGHT: (C)2003,JPO

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image processing apparatus and a color image pickup apparatus, and more particularly to an image processing apparatus for recognizing a face and changing an evaluation value of automatic white balance.

[0002]

2. Description of the Related Art In recent years, along with the development of personal computers, the use of digital cameras as image input devices has become widespread. In particular, digital cameras are mostly used by photographers who do not have special operation technology, and shutter speed, exposure, and
In many cases, the focus is set automatically according to the subject. Further, in general, the subject is often a person, and there is a digital camera having a special setting mode peculiar to the person. Then, Japanese Patent Laid-Open No. 5-11093
No. 6 publication can realize new functions such as a portrait for blurring the background of a subject and a video expression such as correcting the color of the subject to make the subject look beautiful, regardless of conditions such as illuminance. The video camera is disclosed. According to this, when the output signal of the image pickup device is processed by the signal processing circuit and converted into the video signal, the extraction means for extracting a specific subject part from the output video signal of the signal processing circuit, and the signal processing circuit This is realized by providing control means for individually controlling the signal processing parameters of the specific subject portion and other portions in accordance with the extraction output of the extraction means. In addition, Japanese Patent Laid-Open No. 2001-119622
The official gazette discloses a technique that enables automatic focus control, automatic exposure control, automatic white balance control, and electronic zoom control to be effectively performed even if the subject is not located at the center of the screen. According to this, an imaging device which converts an optical image obtained through an imaging optical system into an electric image signal by a CCD and stores the electric image signal, and a read-only memory for storing a specific image pattern, and the stored image A partial image of the signal, a correlation value calculation unit that calculates the correlation value of the specific image pattern, a data read control circuit that detects the maximum value of the correlation value calculated by the correlation value calculation unit, the detected maximum value Based on the partial image corresponding to
This is realized by having an AF, AE, and AWB control circuit that performs predetermined control. As described above,
In both Japanese Patent No. 10936 and Japanese Patent Laid-Open No. 2001-119622, a specific subject is recognized and AWB or AE is performed based on the information of the specific subject. AW excluding information
B is performed.

[0003]

That is, in the conventional image processing apparatus having the function of automatically controlling the white balance, the object on the screen extracts an achromatic portion and the portion is white (achromatic). The color adjustment is controlled so that The achromatic portion has a different shade due to the change in the color of the light source radiated to the object. Therefore, the range of colors that change when the achromatic portion is irradiated with the natural light source is defined as the achromatic color detection range. However, when shooting a person, the color of the person's face and the achromatic portion when the light of the low color temperature light source is irradiated are very close to each other, so it is erroneously recognized,
There is a problem of making the face white. It also performs face detection,
Although a method of controlling the white balance so that the hue of the face becomes natural when a face is detected has been proposed, there is also a problem that the color of the face of a person has individual differences and cannot be controlled well. The present invention, in view of such problems,
(EN) An image processing apparatus that realizes good image processing by performing face detection and not using color information of a face portion for white balance calculation, thereby preventing white balance that is uncomfortable such as a pale portion of the face portion. The purpose is to

[0004]

In order to solve the above problems, the present invention provides an automatic white balance evaluation value, which is color information in at least one arbitrary area of a screen to be processed. Image having white balance evaluation value acquisition means, automatic white balance calculation means for calculating white balance based on the color information, and white balance control means for automatically performing white balance control according to the result of the automatic white balance calculation means In the processing device, further comprising a face detection means for detecting the presence of a person's face in the image to be processed, when the face detection mode is set and the face is detected by the face detection means, The area for acquiring the automatic white balance evaluation value is changed to an area other than the face. The acquisition area of the automatic white balance (AWB) evaluation value is usually the entire screen. Then, the conventional image processing apparatus having a function of automatically controlling the white balance controls the color adjustment so that the object on the screen extracts an achromatic portion and makes the portion white (achromatic color). To do. Therefore, if there is a person's face in that area, the color of the person's face and the achromatic portion when the light of the low color temperature light source is emitted are very close to each other, resulting in erroneous recognition. There was a problem that it turned white. Therefore, when a face is detected by the face detection means in the face recognition operation mode, the acquisition area of the automatic white balance evaluation value is moved from the face portion so that the white balance of the face portion is not calculated. According to this invention, it further comprises a face detection unit that detects the presence of a human face in the image to be processed,
When the face detection mode is set and the face is detected by the face detection unit, good image processing of the face portion is performed by changing the area for acquiring the automatic white balance evaluation value to an area other than the face. Can be realized. According to a second aspect of the present invention, an automatic white balance evaluation value acquisition unit that acquires an automatic white balance evaluation value that is color information in at least one arbitrary area of the screen to be processed, and a white balance calculation based on the color information. In an image processing apparatus having an automatic white balance calculation means and a white balance control means for automatically performing white balance control based on the result of the automatic white balance calculation means, it is detected that a human face is present in an image to be processed. When the face detection mode is set and the face is detected by the face detection unit, the automatic white balance evaluation value acquired from the area in which the face is detected is set to the automatic white. It is not used for balance calculation. When the face detection is executed, the face detection is performed prior to the imaging operation, and when the face detection is possible, the AWB calculation and control are performed using the AWB evaluation value created from the area other than the area including the face. Be seen. By excluding this area from the AWB calculation target by the face detection, it is possible to prevent the AWB control having a strange feeling from being performed. According to this invention, the face detection means for detecting the presence of a person's face in the image to be processed is further provided, and when the face detection mode is set and the face is detected by the face detection means, By not using the automatic white balance evaluation value acquired from the area in which the face is detected in the automatic white balance calculation, it is possible to prevent the AWB control that causes a sense of discomfort.

According to a third aspect of the present invention, when the face detection mode is not set, or when the face is not detected by the face detecting means, the area for obtaining the automatic white balance evaluation value may be set as a normal area. It is an effective means of the invention. Some image pickup apparatuses have a function of selecting a shooting mode of a person or a mode other than that. Since the face is not photographed in the evaluation value acquisition area in the modes other than the person photographing mode, it is designated as the normal area. Also, in the case of the person photographing mode, when the face is not detected by the operation of the face detection processing, the evaluation value acquisition area is similarly designated as the normal area. According to such a technical means, when the face detection mode is not set, or when the face is not detected by the face detection means, the area for acquiring the automatic white balance evaluation value is set to the normal area, which is unnecessary. Processing can be omitted. According to a fourth aspect of the present invention, the face portion further includes brightness detecting means for detecting the brightness of each portion of the screen, and when the area other than the face portion detected by the face detecting means is extremely dark or bright, the face portion is detected. It is also an effective means of the present invention not to perform the process of excluding the color information of from the target of the automatic white balance calculation. In the mode in which face detection is executed, face detection is performed prior to the image pickup operation, and if detected, the difference in brightness between the detected face area and other areas is calculated by the brightness detection means. In the mode in which face detection is performed, the person who is the main subject is set to the proper exposure state, and therefore, when other portions are extremely bright or dark, the AWB evaluation value area is not moved or excluded. In cases such as when the area other than the face is extremely bright or when it is backlit, the areas other than the main subject may be saturated, and the color information at that time is accurately detected. There is no guarantee. Also, when the area other than the face is extremely dark, it means that only the person is illuminated by the spotlight.In this case, the signal of the area other than the main subject has a small value, so it is affected by noise. Easy,
There is a possibility that accurate color information cannot be extracted. Therefore, if the process of moving or excluding the AWB evaluation value acquisition area based on the face recognition result is performed, there is a possibility that good AWB control cannot be performed. According to such a technical means, when the brightness of the area other than the face portion detected by the face detecting means is extremely dark or bright, the color information of the face portion is excluded from the target of the automatic white balance calculation. By not performing the above, it is possible to prevent the malfunction of the AWB.

According to a fifth aspect, at least one of the screens to be processed is
An automatic white balance evaluation value acquisition unit that acquires an automatic white balance evaluation value that is color information in the above arbitrary area, an automatic white balance calculation unit that calculates a white balance based on the color information, and the automatic white balance Via an image processing device and an optical system having a white balance control means for automatically performing white balance control according to the result of the calculation means and a face detection means for detecting the presence of a human face in the image to be processed. A color image pickup means for converting the obtained optical signal into a color image signal and outputting the color image signal, and the color image signal obtained by the color image pickup means is processed by the image processing device. The above image processing function can be applied to all devices that handle images. In particular, it is optimal to apply this image processing technology to a digital camera. In that case, it is realized by further including a color image capturing means for converting an optical signal obtained through the optical system into a color image signal and outputting the color image signal. According to the invention, by processing the color image signal obtained by the color image capturing means by the image processing device, good image processing is performed without performing unnatural white balance such as a pale face. It is possible to provide a digital camera that realizes.

[0007]

BEST MODE FOR CARRYING OUT THE INVENTION The present invention will be described in detail below with reference to the embodiments shown in the drawings. However, the constituent elements, types, combinations, shapes, relative arrangements, and the like described in this embodiment are merely explanatory examples, not the gist of limiting the scope of the present invention thereto, unless specifically stated. .. FIG. 1 is a block diagram of a digital camera according to an embodiment of the present invention. In this configuration, a lens 1 for condensing an optical image of a subject, a diaphragm 2 for converging the light flux condensed by the lens 1, and a plurality of lenses (not shown) are moved to focus or the diaphragm 2 is moved. Motor driver 11 to drive
And a CCD (Charge Coupled Device) that photoelectrically converts an optical image that has passed through the lens 1 and the diaphragm 2.
3 and the CDS (Corre
image processing, an A / D converter 5 for converting an analog signal from the CCD 3 into a digital signal, a timing generator 13 for generating the timing of the CCD 3, the CDS 4, and the A / D converter 5, and an image processing. A digital signal processing circuit 7 for performing image processing according to parameters, a frame memory 6 for recording image-captured pixels and storing the image-processed image, a display unit 8 for displaying the captured image by a liquid crystal (LCD: Liquid Crystal Display), An image compression / expansion circuit 9 which compresses the image data processed by the digital signal processing circuit 7 or expands the original image data, a memory card 10 which stores the image data compressed by the image compression / expansion circuit 9, and a control program CPU 14 for executing predetermined control by means of an EEPROM and EEPROM for storing parameters
(Electrically Erasable and Programmable Read Only
(Memory) 16 and a camera operation unit 17 including a release button for an operator to operate the camera body.

Next, an outline of the operation of the digital still camera 100 of this configuration will be described with reference to FIG.
When the operator looks into the subject from a viewfinder (not shown) and presses the release button of the camera operating unit 17, the CPU 14 detects the signal and drives the lens 1 and the diaphragm 2 to the motor driver 11 to display the image of the subject. Focus on CCD3. These series of operations are automatically performed by the CPU 14 based on information from a sensor (not shown). CCD3
The image focused on is sequentially extracted by the clock generated from the timing generator 13, and the noise included in the data is reduced by the CDS 4. Here, the reset noise is dominant in the noise included in the output signal of the CCD 3. In order to reduce this noise, the reset noise is canceled by subtracting the video signal and the reset noise included in the signal period and the reset noise included only in the field through period. Then, the analog signal is converted into a 10-bit digital signal by the A / D converter 5 and input to the digital signal processing circuit 7,
It is temporarily stored in the frame memory 6. And EEPR
The parameter stored in the OM 16 processes the data temporarily stored in the frame memory 6 according to an instruction from the CPU 14, and the processed image is stored in the frame memory 6 again. White balance processing is included in this processing. Further, the data written in the frame memory 6 is sent to the control unit of the display unit 8 and the contents are displayed on the LCD. The frame memory 6 has at least one of the image pickup pixels.
An image memory capable of accumulating image data larger than the screen, such as a VRAM (Video Random Access Mem).
ory), SRAM (Static Random Access Memory), DR
AM (Dynamic Random Access Memory) or SD
A commonly used memory such as a RAM (Synchronous DRAM) is used. Here, in the case where the operator wants to record the image in the memory card 10, if an instruction is given from the camera operating unit 17, the CPU 14 follows the instruction to the digital signal processing circuit 7 and displays the image in the memory card 10. Control to transfer to. That is, the image-processed image is read from the frame memory 6 and sent to the image compression / expansion circuit 9. Here, JPEG (Joint Photogra
The image is compressed by the phic Experts Group) method and stored in the memory card 10. The encoding algorithm is ADC
Using T (adaptive discrete cosine), a low-resolution image is first encoded, and a hierarchical encoding in which the resolution is gradually increased is also adopted. In this way, the memory card 10
This is for compressing the data stored in the frame memory 6 and storing the compressed data. Besides this,
For example, the data may be recorded in an internal memory of about 8 MB or a smart media compact flash (registered trademark). On the contrary, when the contents of the memory card 10 are displayed on the display unit 8 or when another P
When connecting to C and transferring image data, a desired memory card is inserted into a connector (not shown) of the camera body, and when an instruction is given from the camera operation unit 17, the CPU 14 instructs the digital signal processing circuit 7 to output the memory card. The compressed image data of is read and input to the image compression / expansion circuit 9, the data is expanded and restored according to the compression algorithm, and the image is displayed on the display unit 8.

FIG. 2 is an operation flowchart according to the first embodiment of the present invention. FIG. 3 is a diagram in which an automatic white balance (hereinafter, referred to as AWB) evaluation value acquisition area is moved when a face is detected. The operation will be described with reference to FIGS. 2 and 3 together. First, it is determined whether the image pickup apparatus is in the face recognition operation mode (S1), and if it is not in the face recognition operation mode (NO route), the AWB evaluation value acquisition area in FIG. 3 is designated as the normal area 20 (S5). If the face recognition operation mode is selected in step S1 (YES route), face recognition processing is performed (S2), and it is determined whether the face is recognized (S3). When it is not recognized as a face (NO route), the AWB evaluation value acquisition area in FIG. 3 is designated as the normal area 20 (S5). When the face is recognized in step S3 (YE
S route), the AWB evaluation value acquisition area of FIG.
It is moved to the area 21 other than (S4). And AW
The B evaluation value acquisition area is set (S6), and the AWB evaluation value is acquired (S7). Then, the AWB value is calculated (S
8) The final AWB gain is set (S9). As described above, the automatic white balance (AWB) evaluation value acquisition area 20 is usually the entire screen. Then, the conventional image processing apparatus having a function of automatically controlling the white balance controls the color adjustment so that the object on the screen extracts an achromatic portion and makes the portion white (achromatic color). To do. Therefore, if there is a person's face 22 in that area,
There is a problem that the color of a person's face and the achromatic portion are very close to each other when they are irradiated with light from a low color temperature light source, so that they are erroneously recognized and the face becomes white. Therefore, if a face is recognized by the face detection means in the face recognition operation mode, A
The WB evaluation value acquisition area 21 is moved from the face portion so that the white balance of the face portion is not calculated. As a result, good image processing of the face portion can be realized. Further, some image pickup apparatuses have a function of selecting a shooting mode of a person or a mode other than that. Since the face is not photographed in the evaluation value acquisition area in the modes other than the person photographing mode, it is designated as the normal area. Also, in the case of the person photographing mode, similarly, when the face is not detected by the operation of the face detection processing, the evaluation value acquisition area is designated as the normal area, so that unnecessary processing can be omitted.

FIG. 4 is an operation flowchart according to the second embodiment of the present invention. Fig. 5 shows the screen divided into AWB
It is the figure which has acquired the evaluation value. The operation will be described with reference to FIGS. 4 and 5 together. First, it is determined whether the image pickup apparatus is in the face recognition operation mode (S11), and if it is not in the face recognition operation mode (NO route), the AWB evaluation value acquisition area in FIG. 5 is designated as the normal area 23 (S15). In the face recognition operation mode in step S11 (YES route), face recognition processing is performed (S12), and it is determined whether or not a face is recognized (S13). If it is not recognized as a face (NO
The route of AWB evaluation value acquisition area of FIG. 5 is designated as the normal area 23 (S15). If the face is recognized in step S13 (YES route), the area 24 not including the face 25 is excluded from the AWB evaluation value acquisition area 23 in FIG. 5 (S14). Then, the evaluation value from the AWB evaluation value designation area is acquired (S16). Then, the AWB value is calculated (S17) and the final AWB gain is set (S17).
18).

As described above, in the mode for executing face detection, face detection is performed prior to the image pickup operation, and if detected, an AWB evaluation value created from an area other than the area containing the face is used. Then, AWB calculation and control are performed. By excluding this area from the AWB calculation target by the face detection, it is possible to prevent the AWB control having a strange feeling from being performed. Further, in the mode in which face detection is performed, face detection is performed prior to the image pickup operation, and if detected, the difference in brightness between the detected face area and other areas is calculated by the brightness detection means. . In the mode in which face detection is performed, the person who is the main subject is set to the proper exposure state, and therefore, when other portions are extremely bright or dark, the AWB evaluation value area is not moved or excluded. In cases such as when the area other than the face is extremely bright or when it is backlit, the areas other than the main subject may be saturated, and the color information at that time is accurately detected. There is no guarantee. Also, when the area other than the face is extremely dark, it means that only the person is illuminated by the spotlight.In this case, the signal of the area other than the main subject has a small value, so it is affected by noise. It is easy and may not be able to accurately extract color information. Therefore, if the process of moving or excluding the AWB evaluation value acquisition area based on the face recognition result is performed, a good AWB is obtained.
It may not be controlled. As a result, malfunction of the AWB can be prevented. Further, the above image processing functions can be applied to all devices that handle images. In particular, it is optimal to apply this image processing technology to a digital camera. In that case, it is realized by further including a color image capturing means for converting an optical signal obtained through the optical system into a color image signal and outputting the color image signal. As a result, it is possible to provide a digital camera that achieves excellent image processing by preventing white balance, which causes an uncomfortable feeling such as a pale part of the face.

[0012]

As described above, according to the present invention, claim 1 further comprises a face detecting means for detecting the presence of a human face in the image to be processed, and the face detecting mode is set. When the face is detected by the face detecting means, good image processing of the face portion can be realized by changing the area for acquiring the automatic white balance evaluation value to an area other than the face. . Further, claim 2 further comprises a face detection means for detecting the presence of a person's face in the image to be processed, and when the face detection mode is set and the face is detected by the face detection means. By not using the automatic white balance evaluation value acquired from the area in which the face is detected in the automatic white balance calculation, it is possible to prevent the AWB control that causes a sense of discomfort. Further, according to claim 3, when the face detection mode is not set, or when the face is not detected by the face detection means, the area for acquiring the automatic white balance evaluation value is set to a normal area, which is unnecessary. Processing can be omitted.
According to a fourth aspect of the present invention, when the brightness of the area other than the face part detected by the face detecting means is extremely dark or bright, the color information of the face part is excluded from the target of the automatic white balance calculation. By not doing
It is possible to prevent a malfunction of the WB. Claim 5
Is a digital device that realizes good image processing by processing the color image signal obtained by the color image capturing means by the image processing device so as not to perform uncomfortable white balance such as a pale face. A camera can be provided.

[Brief description of drawings]

FIG. 1 is a block diagram of a digital camera of an embodiment of the present invention.

FIG. 2 is an operation flowchart according to the first embodiment of the present invention.

FIG. 3 is a diagram showing movement of an automatic white balance evaluation value acquisition area when a face of the present invention is detected.

FIG. 4 is an operation flowchart according to the second embodiment of the present invention.

FIG. 5 is a diagram in which an AWB evaluation value is acquired by dividing the screen of the present invention.

[Explanation of symbols]

1 lens, 2 diaphragm part, 3 CCD, 4 CDS, 5
A / D converter, 6 frame memory, 7 digital signal processing circuit, 8 display section, 9 image compression / expansion circuit, 14
CPU

Continued front page    F term (reference) 5C065 AA01 BB02 CC01 CC09 DD02                       EE12 GG30 GG32 GG44                 5C066 AA01 BA01 CA08 EA15 GB01                       HA03 KA11 KD06 KE09 KM02                       KM13 LA02

Claims (5)

[Claims] 1. An automatic white balance evaluation value acquisition means for acquiring an automatic white balance evaluation value, which is color information in at least one arbitrary area of a screen to be processed, and white balance calculation based on the color information. In an image processing apparatus having an automatic white balance calculation means and a white balance control means for automatically performing white balance control based on a result of the automatic white balance calculation means, it is detected that a human face is present in an image to be processed. When the face detection mode is set and the face is detected by the face detection unit, the area for acquiring the automatic white balance evaluation value is changed to an area other than the face. A characteristic image processing device. 2. An automatic white balance evaluation value acquisition means for acquiring an automatic white balance evaluation value, which is color information in at least one arbitrary area of a screen to be processed, and white balance calculation based on the color information. In an image processing apparatus having an automatic white balance calculation means and a white balance control means for automatically performing white balance control based on a result of the automatic white balance calculation means, it is detected that a human face is present in an image to be processed. When the face detection mode is set and the face is detected by the face detection unit, the automatic white balance evaluation value acquired from the area in which the face is detected is set to the automatic white. An image processing device, which is not used for balance calculation. 3. When the face detection mode is not set,
Alternatively, when the face is not detected by the face detecting means, the area for acquiring the automatic white balance evaluation value is set to a normal area, and the image processing apparatus according to claim 1 or 2. 4. A brightness detecting means for detecting the brightness of each part of the screen is further provided, and when the brightness of the area other than the face part detected by the face detecting means is extremely dark or bright, the face part is detected. 4. The process of excluding the color information of the above from the target of the automatic white balance calculation is not performed.
The image processing device according to claim 1. 5. An automatic white balance evaluation value acquisition means for acquiring an automatic white balance evaluation value, which is color information in at least one arbitrary area of a screen to be processed, and white balance calculation based on the color information. An automatic white balance calculation means, a white balance control means for automatically performing white balance control based on the result of the automatic white balance calculation means, and a face detection means for detecting the presence of a human face in an image to be processed. An image processing device having: and a color image capturing means for converting an optical signal obtained through an optical system into a color image signal and outputting the color image signal, wherein the color image signal obtained by the color image capturing means is A color image pickup device characterized by being processed by an image processing device.