JP2012175500A - Image processing method, control program, and image processing device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To perform correction processing more efficiently and obtain a good image quality, in an image processing method and device detecting a region corresponding to a human face from an image data and performing image correction processing.SOLUTION: A method includes: a face information acquisition step of acquiring face information added to image data as information for specifying a face region corresponding to a human face in an image represented by the image data; determination step of determining a need for face color correction based on a color of the face region specified by the face information; and correction step of performing the face color correction to the face region in the image depending on a result of the determination step. In the determination step, in the case that the color of the face region belongs to a first skin color region R1 that is preliminarily set as a region corresponding to a skin color in a color space, and also belongs to a second skin color region R2 that is included in the first skin color region and smaller than the first skin color region, it is determined that the correction step is unnecessary. On the other hand, in the case that the color of the face region belongs to the first skin color region in the color space and does not belong to the second skin color region, it is determined that the correction step is needed.

Description

  The present invention relates to an image processing method and an image processing apparatus, and more particularly to a technique for performing image correction processing by detecting an area corresponding to a human face from an image.

  In an output device such as a digital camera that shoots a subject and generates image data, or a printer that displays or prints the image data, an area corresponding to a human face included in the image is detected. Those having a function of performing appropriate correction processing have been put into practical use. For example, in the technique described in Patent Document 1, information indicating what processing has been performed by performing image processing such as face detection and red-eye correction as necessary at the time of shooting on the imaging device side such as a digital camera. Is recorded in a tag of an image Exif (Exchangeable Image File Format) file. On the other hand, the printing apparatus that receives the image file and prints the image reads the tag information attached to the image file to determine whether the correction process is necessary in order to avoid duplication of the correction process. Only when the determination is made, the correction processing for the image is executed.

Japanese Patent Laying-Open No. 2008-282278 (for example, FIG. 12)

  In an imaging device such as a digital camera, a function for performing face recognition from a captured image in order to automatically adjust shooting conditions such as focus and exposure is already widely used, and a face in an image as a result of face recognition. It is not technically difficult to add information for specifying a region (herein referred to as “face information”) to an image file. On the other hand, in order to construct the system described in Patent Document 1 described above, tag information that clearly indicates what kind of image processing the image recorded in the image file has been subjected to is attached. I must. However, there are various contents of the image processing, and further examination is required on how to manage them with tag information.

  As described above, the above-described conventional technique cannot be applied to, for example, an image file to which only face information as a result of face recognition is attached, and necessary correction cannot be performed or correction is performed in duplicate. However, there are still problems such as inefficiency in processing and inability to obtain desired image quality.

  Some embodiments according to the present invention provide an image processing method and apparatus for performing image correction processing by detecting a region corresponding to a human face from image data and performing the correction processing more efficiently. The present invention provides a technique capable of obtaining good image quality.

  One aspect of the present invention is a face information acquisition step of acquiring face information added to image data as information for specifying a face area corresponding to a human face in an image represented by the image data; A determination step of determining whether or not face color correction is necessary based on the color of the face area specified by the information; and a correction step of performing face color correction on the face area in the image according to a result of the determination step; In the determination step, the color of the face region belongs to a first skin color region set in advance as a region corresponding to a skin color in a color space, and is included in the first skin color region and the first skin color When it belongs to a second skin color area that is narrower than the area, it is determined that the correction step is unnecessary, while the color of the face area belongs to the first skin color area and belongs to the second skin color area in a color space. If there is no such correction work An image processing method characterized by determining require.

  Another aspect of the present invention is a control program that causes a computer to execute the above steps.

  In these inventions, the necessity of face color correction is determined based on the color of the face area specified by the face information added to the image data. The judgment criteria are as follows. First, a first skin color area that is an area corresponding to a skin color in the color space and a second skin color area that is included in the first skin color area and narrower than the first skin color area are set. Then, face color correction is not performed when the face area color belongs to the second skin color area, and face color correction is performed when the face area color belongs to the first skin color area and does not belong to the second skin color area. . The meaning is as follows.

  The first skin color region is a “region corresponding to the skin color” in the color space, and the meaning thereof is that the color within the region is in a category that can be called “skin color”. On the other hand, the second skin color region is a limited partial region in the first skin color region. In this sense, the color belonging to the first skin color region is defined in a broad sense, and the color belonging to the second skin color region is defined in a narrow sense. It can be considered skin color.

  When the face area specified from the face information added to the image data is in the “skin color” category, there is a high probability that the area corresponds to a human face, but some correction has already been made to the area. It may not be clear whether it has been made or not. The present invention performs the necessary correction processing even in such a case, and avoids performing unnecessary correction processing. It is divided into.

  That is, according to the present invention, face color correction is performed in principle for face areas belonging to the first skin color area whose color can be said to be a broad skin color, while among those belonging to the second skin color area which can be said to be a narrow skin color, Face color correction is not performed regardless of the above principle. In this way, rather than simply determining whether or not correction is necessary based on whether or not the face area is a skin color, a judgment criterion that can distinguish between “skin color that needs correction” and “skin color that does not need correction” is used. And determining whether correction is necessary by applying the determination criterion to the color of the face area. Then, by appropriately setting the first skin color region and the second skin color region, the present invention performs the necessary correction processing without omission and avoids unnecessary correction processing, thereby further improving the correction processing. It can be performed efficiently and good image quality can be obtained.

  Here, for example, an area having a hue within a predetermined skin color range in the color space is set as the first skin color area, and an area having a lightness within the predetermined appropriate range is set as the first skin color area. Two skin color regions may be used. A typical case in which face color correction is necessary is a case where the face color of a subject appears dark due to backlight or underexposure. In such a case, correction is necessary if there is a face area that is flesh-colored but has insufficient brightness, while correction is not required if the face area is flesh-colored and has sufficient brightness. It is possible to appropriately cope with such a case by setting a region in which the hue is in a predetermined range as a first skin color region and a region in which lightness is considered appropriate as a second skin color region.

  Further, for example, when the color of the face area is outside the first skin color area in the determination step, an area that matches a predetermined matching condition corresponding to a human face is searched in the image, and the area is recognized as the face area. The face recognition processing step to be performed may be executed. As a case where the face area specified by the face information is not flesh-colored, a case where the subject or the camera moves after the face recognition at the time of imaging can be considered. In such a case, since the credibility of the face information added to the image data is low, it is preferable to newly perform face recognition processing as described above. Further, for example, the same applies when face information is not added to the image data. In such a case, it is preferable to newly perform face recognition processing. In this case, face color correction may be executed when at least one face area is recognized in the face recognition processing step.

  When there are a plurality of face areas specified by the face information in the image, for example, the determination step may be executed based on an average value of the colors of the plurality of face areas. Since the situation where face color correction is required commonly occurs in the image, whether or not to correct the entire image for a plurality of face areas is determined rather than determining whether or not correction is necessary for individual face areas. A better result can be obtained by the determination, and the time required for the processing can also be shortened.

  According to another aspect of the present invention, there is provided face information acquisition means for acquiring face information added to the image data as information for specifying a face area corresponding to a human face in the image represented by the image data, A determination unit that determines whether or not face color correction is necessary based on the color of the face area specified by the face information, and a face color correction is performed on the face area in the image according to the determination result of the determination unit Correction means, wherein the determination means belongs to a first skin color area that is preset as an area corresponding to the skin color in a color space, and is included in the first skin color area When it belongs to the second skin color area narrower than the first skin color area, the correction means does not execute face color correction, while the color of the face area belongs to the first skin color area in the color space, and the second When it does not belong to the skin color area An image processing apparatus, characterized in that to execute the complexion correction on the correction means to.

  In the invention configured as described above, similar to the above-described invention of the image processing method, it is possible to determine whether correction is necessary according to the color of the face area specified from the face information, and to perform necessary face color correction. Therefore, it is possible to perform necessary correction processing without omission, avoid unnecessary processing, and perform correction processing more efficiently and obtain good image quality.

1 is a diagram illustrating a printing system using an embodiment of an image processing apparatus according to the present invention. The figure which shows an example of the image file produced with a digital camera. The elements on larger scale of FIG. The figure which shows an example of the face recognized by the face recognition process. 3 is a flowchart showing image processing and printing operation executed by the printing apparatus. The figure which shows the principle of the necessity judgment of the face color correction based on the color of a face area. The figure which shows the example of the color expression by HSB color space typically.

  FIG. 1 is a diagram showing a printing system using an embodiment of an image processing apparatus according to the present invention. In this printing system, image data acquired by photographing with the digital camera 200 is transferred to the printing apparatus 100 by a memory card M, a USB (Universal Serial Bus) cable, a wireless LAN (Local Area Network), or the like. It is something to print. That is, here, it is assumed that the user captures an image with the digital camera 200, generates image data, reads the image data as it is with the printing apparatus 100, and prints it, so-called direct printing, but the present invention is applicable. The printing system is not limited to this. That is, the present invention can also be applied to a printing system that captures image data generated by the digital camera 200 into a personal computer or a mobile phone, and transmits the image data from the personal computer to the printing apparatus 100 for printing. Further, the present invention is not limited to a system including such a printing apparatus 100, and the present invention can be applied to all image processing apparatuses that perform various processes on image data.

  In the digital camera 200, as shown in the figure, a CPU (Central Processing Unit) 201, a ROM (Read Only Memory) 202, a RAM (Random Access Memory) 203, a CCD (Charge Coupled Device) 204, a graphic processor (Graphic Processor; GP) 205 and interface (I / F) 206 are connected to each other via a bus 207, and information can be exchanged between them. The CPU 201 controls the digital camera 200 while executing various arithmetic processes in accordance with programs stored in the ROM 202. At this time, temporarily required data is stored in the RAM 203. The CCD 204 converts the optical image from the subject collected by the optical system 208 into an electrical signal and outputs the electrical signal. The optical system 208 is composed of a plurality of lenses and actuators, and forms an optical image of the subject on the light receiving surface of the CCD 204 with the plurality of lenses while adjusting the focus and the like with the actuators. Further, the GP 205 executes display image processing based on a display command supplied from the CPU 201, and supplies the obtained display image data to a liquid crystal display (LCD) 209 for display.

  The I / F 206 provides an input / output function of the digital camera 200, and appropriately converts the data representation format when information is exchanged between the operation button 210, the gyro sensor 211, and the card I / F circuit 212. Device. The operation button 210 connected to the I / F 206 includes buttons such as a power source, a mode switch, and a shutter, and input means that can set various functions. With these, the user can arbitrarily control and operate the digital camera 200. Is possible. The gyro sensor 211 generates and outputs a signal indicating an angle of the camera body (an angle with respect to a horizontal plane) when the subject is photographed by the digital camera 200. The digital camera 200 generates various information (for example, information on exposure, subject, etc.) at the time of shooting, including the angle of the camera body described above.

  In this embodiment, the digital camera 200 has a structure in which shooting information is described in Exif (Exchangeable Image File Format) information and an image file added to the image data can be generated. The structure of this Exif image file is basically the usual JPEG (Joint Photographic Experts Group) image format itself, in which data such as thumbnail images and shooting-related data are embedded in a form complying with the JPEG conventions. Is.

  The card I / F circuit 212 is an interface for reading / writing information from / to the memory card M inserted in the card slot 213. Further, the I / F 206 also has a connection function with an external device such as a USB or wireless LAN (not shown), and can exchange image files with the printing apparatus 100 by wire or wirelessly. . The image file (image data + Exif information) created by the digital camera 200 and given to the printing apparatus 100 will be described in detail later.

  The printing apparatus 100 is an apparatus that prints an image captured by the digital camera 200, and is configured as follows. In the printing apparatus 100, a CPU 101, a ROM 102, a RAM 103, an EEPROM (Electrically Erasable and Programmable ROM) 104, a GP 105, and an I / F 106 are connected to each other via a bus 107, and information can be exchanged between them. . The CPU 101 executes various arithmetic processes according to programs stored in the ROM 102 and the EEPROM 104 and controls each unit of the printing apparatus 100. Further, programs and data to be executed by the CPU 101 are temporarily stored in the RAM 103, while data and the like that are retained even after the printing apparatus is turned off are stored in the EEPROM 104. Further, the CPU 101 gives a display command to the GP 105 as necessary, and the GP 105 executes display image processing according to the display command, and supplies the processing result to the LCD 108 for display.

  The I / F 106 is a device that appropriately converts the data representation format when information is exchanged among the operation buttons 109, the card I / F circuit 110, and the printer engine controller 111. In the printing apparatus 100, the operation button 109 is configured to be pressed when performing menu selection or the like of the printing apparatus 100. The card I / F circuit 110 is connected to the card slot 112 and reads an image file generated by the digital camera 200 from the memory card M inserted into the card slot 112. The I / F 106 also has a function of connecting to an external device such as a USB or wireless LAN (not shown), and can exchange image files with the digital camera 200 by wired communication or wireless communication. It has become.

  When the printing apparatus 100 receives the image data via the memory card M or through data communication, the printing apparatus 100 performs various processes by the CPU 101 and controls the printer engine 113 by the printer engine controller 111, thereby corresponding to the image data. Print the image to be printed. Hereinafter, the configuration of the image file will be described with reference to FIGS. 2 to 4, and the image processing operation and the printing operation in the present embodiment will be described in detail.

  FIG. 2 is a diagram showing an example of an image file created by the digital camera configured as described above. 3 is a partially enlarged view of FIG. 2, and FIG. 4 is a diagram showing an example of a face recognized by face recognition processing. In this embodiment, the digital camera 200 stores image data captured by the CCD 204 in the RAM 203. The digital camera 200 performs face recognition processing when a person is included in the subject (note that many face recognition methods have been proposed in the past, and any of these methods may be employed). . In the present embodiment, in the face recognition result, coordinate information as shown in FIG. 4 is expressed as face position information, that is, face information. That is, as shown in the figure, the image data is composed of pixels of a predetermined image width (Width) and image height (Height), X represents the horizontal axis, Y represents the vertical axis, and the upper left is The origin is (0, 0). When a face is detected from the image data, the face area FR is represented by an area surrounded by the coordinates of the four points of upper left (LT), lower left (LB), upper right (RT), and lower right (RB). The coordinates indicating the region FR, that is, the face coordinates (face position) are stored in the RAM 203 together with the number of recognized faces (that is, the number of faces) Nc. In the present specification, as will be described later, the face coordinates obtained by the face recognition by the digital camera 200 are obtained because the face recognition is performed by the printing apparatus 100 in the same manner as the face recognition by the digital camera 200. Are referred to as “camera face coordinates” and are indicated by upper left (LTc), lower left (LBc), upper right (RBc), and lower right (RBc). In addition, since a plurality of faces may be detected by the face recognition process, in FIG. 3, the nth camera face coordinates are represented by upper left (LTnc), lower left (LBnc), upper right (RTnc), and lower right (RBnc). ).

  In this embodiment, the image data, the number of faces, the face coordinates, and the like are stored in the RAM 203 as described above, but the digital still camera image file format standard Exif Ver. 2.2.1 is used as the recording method. ing. The structure of this Exif image file is basically the usual JPEG (Joint Photographic Experts Group) image format itself, in which data such as thumbnail images and shooting-related data are embedded in a form complying with the JPEG conventions. Is.

  The image file used in the present embodiment has an SOI (Start of image) 301 first, as shown in the left part of FIG. Thereafter, APP1 (application marker segment) 302, DQT (Define Quantization Table) 303, and DHT (Define Huffman Table) 304 are arranged in this order. After that, SOF (Start of Frame) 305, SOS (Start of Stream) marker 306, and compressed data (Compress Data) 307 are arranged in this order. Finally, there is an EOI (End of Image) 308. Of these, APP1 has a structure shown in the center of FIG. 2 as a data area for use in an application program. The APP1 structure has an APP1 Marker area 302a at the top. Next, there is a Length area 302b.

  In the first 6-byte area 302c of the data following the Length area 302b, the ASCII character “Exif” is followed as an identifier followed by 2 bytes of 0x00. From there, data is stored in the TIFF (Tagged Image File Format) format. The first 8 bytes of the TIFF format are a TIFF header area 302d.

  Further, in the 0th IFD (IFD of main image) area 302e next to the TIFF header area 302d, as shown on the right side of the figure, image related information such as image width and image height (or simply referred to as image information). ) Is stored. Then, after the 0th IFD, there is a 0th IFD Value area 302f. Further, an Exif IFD area 302g is provided next, and shooting-related information (or simply referred to as shooting information) such as an exposure time, an F number, and a shooting scene type is stored. In addition, the Exif IFD value area 302h is in the Exif IFD area 302g. At present, there is no standard that specifies whether or not face recognition processing has been executed, and the number of faces detected by the face recognition process and Nc face coordinates in the Exif tag. The face detection implementation status indicating whether or not face recognition processing has been executed on the camera 200 side, the number of faces Nc, and the like are written in the Exif IFD area 302g, and Nc face coordinates are written in the Exif IFD Value area 302h. The description will be continued on the assumption that information pointing to the insertion position is written in the Exif IFD area 302g. Of course, the information may be written in an area depending on the manufacturer. As described above, the face information, which is information on the face position (face coordinates), is described in the Exif information and attached to the image data.

  Next, an image file having the above-described data structure (FIG. 2) is stored in the memory card M, and the image file is read from the memory card M, and based on various information included in the image file by the printing apparatus 100. Two embodiments of the operation of executing predetermined image processing and printing will be described in order.

  FIG. 5 is a flowchart showing image processing and printing operations executed by the printing apparatus of FIG. When the user inserts the memory card M into the card slot 112 of the printing apparatus 100 and operates the operation button 109 to give a print command, the CPU 101 controls each part of the apparatus according to the program stored in the ROM 102 to display the following image. Perform processing and printing operations.

  First, the CPU 101 acquires an image file to be printed from the memory card M, executes a Huffman decompression process, and obtains a quantized DCT (Discrete Cosine Transform) coefficient (step S10). More specifically, the CPU 101 acquires an entropy coding table from the image file acquired from the memory card M, and includes Y (luminance) component, Cr (color difference component), and Cb (color difference) included in the compressed data. The DC coefficient and the AC coefficient of each block of the component) are decoded. At this time, decoding is performed in units of MCU (Minimum Coded Unit) which is a minimum coding unit.

  Further, the CPU 101 inversely quantizes the quantized DCT coefficient obtained in step S10 (step S11). Specifically, the CPU 101 obtains a DCT coefficient by acquiring a quantization table from an image file and multiplying the quantized DCT coefficient obtained in step S10 (by inverse quantization).

  Next, the CPU 101 caches information (rotate information) necessary for rotating the image, for example, in the RAM 103 (step S12). Specifically, when rotating an image compressed by the JPEG method, each of the DC component (direct current component) and AC component (alternating current component) of the MCU must be expanded once. Here, with respect to the DC component, the difference between adjacent DC component values is Huffman-encoded, so the correlation with the adjacent MCU becomes a problem. In addition, the AC component has a problem that the data length is not constant in each MCU due to the Huffman coding process, and it becomes unclear which data in the bit stream of JPEG data is the AC component value of the MCU to be obtained. . Therefore, in step S12, the rotation process is enabled by obtaining and caching the DC component value and the AC component address of each MCU.

  Then, the CPU 101 obtains the original pixel value by performing an inverse DCT operation on the DCT coefficient obtained in step S11 (step S13). Further, the CPU 101 converts the image in the YCC space obtained by the process in step S13 into an image in the RGB (Red Green Blue) space and an image in the HSB (Hue Saturation Brightness) space (step S14).

  Then, the CPU 101 stores and holds each image of YCC, RGB, and HSB obtained in the processing of step S13 and step S14 in the RAM 103. At this time, in order to reduce the data amount, the pixels may be thinned out at a predetermined rate to reduce the image, and then stored in the RAM 103 (step S15). Here, it is assumed that the image is normalized to an image of (320 × 240) pixels corresponding to a QVGA (Quarter Video Graphics Array) size regardless of the original image size.

  In addition, the CPU 101 calculates a histogram for each of the YCC, RGB, and HSB image components stored in the RAM 103 in step S15 (step S16). Specifically, for RGB images, histograms are calculated for R, G, and B images. As a result, the distribution of each component constituting the image is obtained.

  As described above, in the present embodiment, the processing in steps S10 to S16 is performed in units of MCUs, and the CPU 101 returns to step S10 to perform processing until it is confirmed that the processing for all the MCUs has been completed. Run repeatedly. On the other hand, when the processing for all the MCUs is completed, the process proceeds to the next step S17, and processing corresponding to the main part of the face recognition processing is executed.

  In step S17, information related to face recognition processing on the camera side is read from the Exif tag of the image file acquired from the memory card M (step S17). Specifically, the face detection status is read out and it is determined whether or not the face recognition process has been performed on the camera side (step S18). (Step S30). If face recognition processing has already been performed on the camera 200 side (YES in step S18), it is further determined whether or not the face detection number Nc on the camera side ascertained from the Exif tag is a value greater than zero ( Step S19). When the face detection number Nc is 0 (NO in step S19), it means that face recognition processing has been performed on the camera side, but no face area has been detected. Therefore, also in this case, the face recognition process (step S30) is performed again on the printing apparatus 100 side.

  On the other hand, if YES in step S19, that is, if at least one face area (camera face area) is detected in the face recognition process on the camera side, the face coordinates representing these face areas are normalized to the QVGA size, The coordinates in the image reduced to 320 × 240 dots are converted (step S20). Then, the average value of the color of each face area in the QVGA size image specified by the coordinates, that is, the average face color is calculated based on the image data corresponding to the area (step S21).

  In this embodiment, the necessity determination of the face color correction process is performed based on the average face color thus obtained. This face color correction process is a process for correcting the face color to a more natural face color when, for example, the face color of a person included in the image appears unnaturally dark due to backlight or insufficient exposure during imaging. is there. Such correction processing may be performed by an imaging device such as a digital camera or an image processing device. In such a case, if correction is performed repeatedly on the printing device side, an unnatural result is obtained. End up. Therefore, when correction has been performed on the camera side, it is necessary not to perform correction on the printing apparatus 100 side.

  This becomes a problem when it is determined that the face recognition processing has been performed with the Exif tag information acquired from the image file. Therefore, in this embodiment, when face recognition processing is performed on the camera side and at least one face area is detected as a result, whether or not face color correction is necessary is determined according to the color of the face area. To do. Specifically:

  FIG. 6 is a diagram showing the principle of determining whether face color correction is necessary based on the color of the face area. As shown in FIG. 6A, the face area FR whose four corners are specified by the camera face coordinates can include, for example, eyes, hair, and shadows in addition to the skin, so that the whole has a uniform skin color. Not that. Therefore, the face area FR is subdivided into a large number of blocks, for example, pixels, and the color of each block is calculated based on the image data, and the ratio of blocks that can be said to belong to “skin color” out of all the blocks is predetermined. If it is equal to or greater than the threshold, it can be said that the face area is “skin color”. As for the above average face color, when there is only one face area, a color obtained by averaging the colors of the blocks in the face area, or a block belonging to the skin color among the blocks in the face area The average color can be used. In addition, when there are a plurality of face areas, the color obtained by averaging the colors of all the blocks included in each face area, or only the color of the block belonging to the skin color among the blocks in each face area is extracted and averaged. Can be color.

  Next, the necessity of face color correction is determined based on the average face color thus obtained. As shown in FIG. 6B, it is determined whether facial color correction is necessary or not based on the combination of the average facial color shade and its brightness. More specifically, when the average face color is in a predetermined “skin color” range in the hue and the brightness is in the predetermined “appropriate brightness” range, the face color correction is performed. Judge that it is unnecessary. As described above, the face color correction process is a process for correcting, for example, an unnatural face color due to backlight or underexposure during imaging to a more natural face color. Therefore, if the face area is flesh-colored and already has an appropriate brightness, no further correction is necessary. As a result, it is avoided that correction is made on the printing apparatus side in duplicate with correction on the camera side.

  Further, when the average face color of the face area belongs to the area R2 that is hatched with dots, the hue of which is in the “skin color” range but the brightness is outside the “appropriate brightness” range, face color correction is required. It is judged. Furthermore, when the average face color belongs to the plain region R3 whose hue is not “skin color” in the first place, there is a high possibility that the camera face coordinates do not correctly represent the face position. As a case where this situation occurs, the camera or subject moved between the adjustment and shooting because face recognition processing with the camera is mainly used to adjust shooting conditions such as focus and exposure adjustment. Cases are considered. Therefore, in such a case, the face recognition process itself is executed again on the printing apparatus 100 side.

  Here, there are various known representation methods for expressing colors quantitatively, and color determination in this embodiment can also be performed under any color representation, but here it is associated with human color vision. A case where the determination is performed in the HSB color space in which the skin color can be easily specified will be described.

  FIG. 7 is a diagram schematically showing an example of color expression in the HSB color space. As shown in FIG. 7A, in the HSB color space represented by three parameters of H (Hue angle), S (Saturation), and B (Brightness), the H value is 0. ° corresponds to red and 60 ° corresponds to yellow. The hue corresponding to the skin color is intermediate between red and yellow. For example, as shown in FIG. 7B, two H values H1 and H2 (> H1) larger than 0 ° and smaller than 60 ° are determined in advance. The hue within the angle range RH with the values H1 and H2 as the lower limit and the upper limit, respectively, can be defined as “skin color”. In the example of FIG. 7B, the color belonging to the space area Z1 hatched with dots in the HSB color space is regarded as “skin color”.

  Further, the “appropriate brightness range” is designated by the brightness B value indicating the brightness of the color. That is, a lower limit value B1 and an upper limit value B2 (> B1) of brightness that allow brightness to be considered appropriate as a face color are set, and a space indicated by diagonal lines within which the brightness B value is within this range in the space area Z1 corresponding to the skin color. The color belonging to the area Z2 can be a skin color having appropriate brightness. When the average face color belongs to the spatial area Z2 in the HSB color space, the face color correction is unnecessary. When the average face color belongs to the spatial area Z1 but does not belong to the spatial area Z2, the face color correction is unnecessary. It is determined that face recognition processing is necessary.

  Further, as shown by a hatched area Z3 in FIG. 7C, “skin color” and “brightness” may be defined by a combination of the H value, the S value, and the B value. That is, when the combination of the H value, S value, and B value of the average face color belongs to the region Z3 in the color space that satisfies the predetermined condition, it is determined that the average face color is an “appropriate skin color” that does not require correction. Also good. The range regarded as skin color and the range regarded as appropriate in brightness can be appropriately determined. The same can be done when other color expression methods are used.

  Referring to the flowchart of FIG. 5, it is determined in step S22 whether or not the average face color is a flesh color. To be judged. Thereby, the necessity of face color correction is determined. If the average face color is within the skin color and appropriate brightness range (both YES in steps S22 and S23), it is determined that face color correction is not necessary, and the enhancement process in the subsequent process is not performed (step S24). If the average face color is skin color but not within the appropriate brightness range (YES in step S22 and NO in step S23), it is determined that face color correction is necessary, and the process proceeds to step S42. If the average face color is not a skin color (NO in step S22), the process proceeds to the next face detection step (step S30).

  In step S30, a face area satisfying a predetermined condition is detected from the image using the image data reduced and held in the QVGA size. Various techniques for detecting a region that can be regarded as a face from an image are known, and any technique can be applied to the face detection here, so a detailed description is omitted here. When a face area is detected in the image, the four vertex coordinates of the face area are registered in a face registration list provided in the RAM 103, for example.

  When face detection is performed by the printing apparatus 100 in this way, in the next step S40, the CPU 101 determines whether face information is registered in the face registration list. If face information is registered (“YES” in step S40), the CPU 101 calculates all face positions based on the face information registered in the face registration list, and further calculates the average face color of the face. After the acquisition (step S41), enhancement parameters are calculated so that the face color is optimal (step S42). Although details of the enhancement parameter calculation are omitted, feature amounts of each image quality parameter such as contrast, brightness, saturation, and sharpness are extracted based on the RGB histogram stored in the RAM 103, and the extracted feature amount is a predetermined image quality parameter. Set the enhancement parameters to approach the feature amount.

  Also, if NO is determined in step S23, that is, if the average face color of the face area detected on the camera side is a skin color but not an appropriate brightness, step S42 is also executed. This is because the average face color of the face area has already been calculated in step S21, and it is not necessary to execute step S41.

  By executing step S42 in this way, necessary enhancement parameters are calculated for the face area detected by the digital camera 200 or the face area detected by the printing apparatus 100. This step S42 is skipped when it is determined that the face area detected on the camera side does not require correction on the printing apparatus 100 side.

  On the other hand, when face information is not registered in the face registration list (“NO” in step S40), that is, when a human face is not detected in the image, the CPU 101 optimizes the entire image. An enhancement parameter is calculated (step S43).

  Subsequently, the CPU 101 resets the file pointer indicating the position to be decompressed in the image file to be printed (step S44), and restores the processing position to the top of the image file. The CPU 101 prints an image based on the image data by repeating the following steps S45 to S52.

  The CPU 101 performs Huffman decompression processing on the image data for one MCU line cached in the RAM 103 to obtain quantized DCT coefficients (step S45). Here, one MCU line means an MCU group of one column in the column direction constituting the image when rotating the image, and an MCU group of one column in the row direction constituting the image when not rotating. Say. Then, the CPU 101 inversely quantizes the quantized DCT coefficient obtained in the process of step S45 (step S46), and further performs an inverse DCT operation on the DCT coefficient obtained in step S45. Obtain (step S47).

  The CPU 101 converts the YCC space image thus obtained into an RGB space image (step S48). Then, the CPU 101 applies the enhancement parameter calculated in step S42 or S43 to each pixel constituting the RGB space image, thereby correcting the printed image to an optimum hue (step S49). Proceed to step S50. If it is determined that correction is not necessary, the enhancement parameter is not set and the correction is naturally not performed.

  The CPU 101 performs layout processing such as resizing and rotation on the image data corrected as necessary in this way (step S50), and supplies it to a band buffer (not shown) of the printer engine controller 111. Receiving this, the printer engine controller 111 controls each part of the printer engine 113 to print an image corresponding to the image data (step S51). When the printing process is completed, the CPU 101 updates the cache state of the RAM 103 (step S52). Then, when the above steps S45 to S52 are completed for all the MCU lines, a series of processing is ended.

  As described above, according to the present embodiment, when face recognition processing is performed by the digital camera 200, information indicating that fact is recorded as tag information of the Exif image file. Then, the printing apparatus 100 determines whether or not face detection has been performed on the camera side from the tag information attached to the image file. If face detection has been performed, the face area detected on the camera side is determined. The necessity of face color correction is determined based on the color. Specifically, face color correction is not performed when the color of the face area is a skin color having an appropriate brightness, and face color correction is executed when the color is the skin color but the brightness is not within an appropriate range. By doing so, it is possible to avoid performing double correction on the image on which the face color correction has been performed on the camera side, and unnatural or unnecessary processing of the image resulting from the double correction. It is possible to eliminate the waste that is performed and to perform correction processing efficiently and with good image quality.

  Further, when face detection is not performed on the camera side, or the face area is not skin color, it is necessary when face detection is performed on the printing apparatus 100 side and the face area is detected. The correct face color correction process. In this way, even if face detection is not performed on the camera side or the face area is not correctly specified, appropriate face color correction can be performed on the image.

  As described above, in the printing system of the above-described embodiment, the printing apparatus 100 functions as the “image processing apparatus” of the present invention, and the CPU 101 executes a predetermined control program, so Means ”and“ correction means ”. In the image processing operation of this embodiment (FIG. 5), step S17 corresponds to the “face information acquisition process” of the present invention, and steps S18 to S23 correspond to the “judgment process” of the present invention. Steps S42, S43, and S49 correspond to the “correction step” of the present invention, and step S30 corresponds to the “face recognition processing step” of the present invention.

  In the present embodiment, the region R1 in FIG. 6B and the space region Z1 in FIG. 7B correspond to the “first skin color region” of the present invention, while the region R2 in FIG. 6B. The space region Z2 in FIG. 7B and the space region Z3 in FIG. 7C correspond to the “second skin color region” of the present invention.

  The present invention is not limited to the above-described embodiment, and various modifications other than those described above can be made without departing from the spirit of the present invention. For example, in the above-described embodiment, an image file in which direction coordinates or the like are added to image data is recorded on the memory card M, supplied to the printing apparatus 100 via the memory card M, and printed. Even when the image file is supplied to the printing apparatus 100 by the above, the same effect as the above-described embodiment can be obtained by applying the present invention.

  In the above embodiment, the coordinate position in the HSB color space is used to quantitatively represent the color of the face area and determine whether it is a skin color. However, another color expression method, for example, on the xy chromaticity diagram, is used. The color may be expressed by the coordinate position at or the coordinate position in the Lab color space, and the skin color may be defined in those planes or spaces to perform color determination.

  Further, in the printing apparatus 100 according to the above-described embodiment, necessity of face color correction or face detection on the printing apparatus side based on information indicating the face detection implementation state on the camera side among the Exif tag information included in the image file. Judging. However, when implementing the present invention, it is not essential that the image file includes information indicating the face detection implementation status on the camera side. Even if such information is not included, for example, an effective camera face for the image file Whether or not face color correction or face detection is necessary may be determined based on whether or not there is information. The image file is not limited to the one created by the camera, and may be an image file obtained by editing an image captured by the camera with a personal computer or the like.

  In the above-described embodiment, the image processing apparatus and method according to the present invention are applied to the printing apparatus 100. However, in the printing system including a plurality of electronic apparatuses, one electronic apparatus other than the printing apparatus ( For example, the present invention can also be applied to a multifunction machine, a facsimile machine, and the like.

  In addition, a control program for executing the image processing method according to the above embodiment is stored in a storage medium such as a CD-ROM, an optical disk, a magneto-optical disk, and a nonvolatile memory card, and the program is read as a code from the storage medium. You may perform in. That is, the storage medium storing the program and the control program itself are also included in one embodiment of the present invention. For example, the present invention can be applied to application software for editing an image captured by a digital camera on a personal computer or a workstation.

  DESCRIPTION OF SYMBOLS 100 ... Printing apparatus (image processing apparatus) 101 ... CPU (judgment means, correction means), 102 ... ROM, 103 ... RAM, 104 ... EEPROM, 106 ... Interface (face information acquisition means), 200 ... Digital camera, M ... Memory card

Claims (8)

A face information acquisition step of acquiring face information added to the image data as information for specifying a face region corresponding to a human face in the image represented by the image data;
A determination step of determining whether or not face color correction is necessary based on the color of the face area specified by the face information;
A correction step of performing face color correction on the face area in the image according to the result of the determination step,
In the determination step,
A color of the face area belongs to a first skin color area set in advance as an area corresponding to a skin color in a color space, and is included in the first skin color area and is narrower than the first skin color area In the case of belonging to, while determining that the correction step is unnecessary,
An image processing method comprising: determining that the correction step is necessary when a color of the face region belongs to the first skin color region and does not belong to the second skin color region in a color space.   An area in the color space whose hue is within a predetermined skin color range is defined as the first skin color area, and an area of the first skin color area in which the lightness is within a predetermined appropriate range is defined as the second skin color. The image processing method according to claim 1, wherein the region is an area.   When the color of the face area is outside the first skin color area in the determination step, an area that matches a predetermined matching condition corresponding to a human face is searched in the image, and the area is recognized as the face area. The image processing method according to claim 1, wherein a face recognition processing step is performed.   When the face information is not added to the image data, a region that matches a predetermined matching condition corresponding to a person's face is searched in the image, and a face recognition processing step that recognizes the region as a face region is executed. The image processing method according to claim 1.   The image processing method according to claim 3 or 4, wherein the correction step is executed when at least one face region is recognized in the face recognition processing step.   6. The image processing according to claim 1, wherein when there are a plurality of face regions specified by the face information in the image, the determination step is executed based on an average value of colors of the plurality of face regions. Method. A face information acquisition step of acquiring face information added to the image data as information for specifying a face region corresponding to a human face in the image represented by the image data;
A determination step of determining whether or not face color correction is necessary based on the color of the face area specified by the face information;
According to the result of the determination step, the computer executes a correction step for performing face color correction on the face area in the image, and
In the determination step,
A color of the face area belongs to a first skin color area set in advance as an area corresponding to a skin color in a color space, and is included in the first skin color area and is narrower than the first skin color area While the computer determines that the correction step is unnecessary,
A control program that causes the computer to determine that the correction step is necessary when the color of the face region belongs to the first skin color region and does not belong to the second skin color region in a color space. Face information acquisition means for acquiring face information added to the image data as information for specifying a face area corresponding to a human face in the image represented by the image data;
Determining means for determining whether or not face color correction is necessary based on the color of the face area specified by the face information;
Correction means for performing face color correction on the face area in the image according to the determination result of the determination means,
The determination means includes
A color of the face area belongs to a first skin color area set in advance as an area corresponding to a skin color in a color space, and is included in the first skin color area and is narrower than the first skin color area While the correction means does not execute the face color correction,
An image processing apparatus, wherein when the color of the face area belongs to the first skin color area and does not belong to the second skin color area in a color space, the correction unit performs face color correction.

Images