JP2007087123A - Image correction method, device and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To appropriately and efficiently correct the brightness and color of a face in a natural manner by surely extracting the head hair in a general digital picture even for head hair of a color other than black, and correcting the colors of the face and the head hair. <P>SOLUTION: An area designation part 31 designates an area on a personal image In designated by an operator as a designation area Ru containing the face and head hair to be corrected of a single person, a head hair color kind information acquisition part 33 acquires head hair color kind information Vh showing the kind of color of the head hair based on color information of the designation area Ru or the like. A head hair area extraction part 34 extracts the head hair area by a predetermined algorithm according to the kind shown by the information Vh. Correction means 36, 40 and 50 correct the color of a correction target area consisting of a skin area containing the face extracted based on color information of the designation area Ru by a skin area extraction part 32 and the head hair area extracted by the head hair area extraction part 34. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to an image correction method and apparatus for correcting brightness and color of a partial area such as a human face and skin in a digital image, and a program for the same.

  Some digital photographic images obtained by taking a photograph using a digital camera or reading a photograph output on a paper medium with a scanner have a mode in which the brightness and color are appropriate depending on the photographing conditions and the characteristics of the subject. Some of them are not expressed in a desired form, and there are cases where it is desired to correct the brightness and color, but it may not be appropriate to apply a certain correction process to the entire image.

  For example, in a photographic image where a person is photographed with underexposure or backlighting, the person's face becomes dark.Therefore, the person's face may be corrected to an appropriate brightness. If such gradation conversion processing is applied to the entire image, the background may be overexposed. Also, in a group photo taken in a studio, one person is extremely tanned in a photo image in which the person in the back row corner is not illuminated and only that person appears dark, or in a picture in which two people are taken In the case of a photographic image, etc., the face of one person becomes dark.Therefore, you may want to adjust the brightness of that person's face to match the brightness of the face of another person. If the entire image is subjected to gradation conversion processing that makes it brighter, the face of another person may become too bright beyond the proper range.

  In order to avoid such a problem, there are cases where partial correction is desired, such as only the face and skin of a person, not the entire image, and various methods have been proposed so far for partial correction of the image. .

  For example, Patent Document 1 describes an ID photo system that performs image processing only on a person. Here, an image is automatically separated into a person area and a background area other than the person, A method has been proposed in which the skin color is automatically adjusted, the background is replaced with a separately prepared background, and the person after the automatic adjustment is synthesized with the other background.

  Patent Document 2 describes an automatic face dodging technique, and here, a method of extracting a face area, applying a preferable process to the skin area for the face area, and synthesizing the face area other than the face area. Proposed.

  Further, Patent Document 3 describes a face extraction method. In this case, if a color based on a skin color part pixel is extracted as a face in a specified region, there is a case where a photo image is erroneously extracted. A method has been proposed in which a face template composed of a wire frame or the like is displayed next to the face, the face in the image is aligned with the face template, and the face is extracted based on the result.

  Although it is not a technique related to face area extraction or face correction, Patent Document 4 proposes a technique for improving the efficiency of automatic red-eye correction by simply specifying a red-eye area in advance by an operator. A method for extracting a specific area in a specific area after specifying the specific area has been disclosed.

  However, each of the methods disclosed in Patent Documents 1 to 4 has problems as described below.

  Since the ID photo system of Patent Document 1 is intended for ID photos with a simple background, it is relatively easy to automatically separate a person and a background (it can also extract hair and clothes), but a general photo has a background. Due to the complexity of the system, this system cannot simply be applied to general photography. In practice, even if the skin color is automatically adjusted, proper correction may not be performed, and means for manually correcting the skin color is necessary.

  The automatic face dodging method of Patent Document 2 simply synthesizes the corrected face and the background, and depending on the extraction accuracy of the face area, distortion or the like appears in the face outline, There is a case. In addition, since the dodging is performed considering only the skin area of the face, the combination of the face and the hair color may become unnatural. Furthermore, since a general image often has a complicated background, the probability of failure in automatic face extraction is high.

  In the face extraction method of Patent Document 3, the work of aligning face templates is complicated and inefficient.

  The area specifying method of Patent Document 4 only discloses a technique for extracting a specific area from a manually specified area, and does not propose an efficient technique for face and skin correction.

  In addition to the above method, a method of manually performing partial image processing using commercially available image processing software such as Photoshop (registered trademark) is also possible, but the operation is complicated and inefficient.

  Therefore, the following image correction method has been proposed by the present applicant. That is, the user designates a partial area including the face on the digital image, and extracts the face area and the hair area from the designated area based on the color information of the image of the designated area or its vicinity, etc. A region composed of a region and a hair region is defined as a correction target region, and the color of the correction target region is corrected.

According to this image correction method, the face area and the hair area are extracted from the area designated by the user or the periphery thereof, and the area including the extracted face area and the hair area is set as the correction target area. By a simple operation from the user to specify the area, the position of the face can be roughly specified, the face can be extracted accurately, and the correction target area is not only the face but also the area consisting of the face and hair. The color imbalance between the face and hair after correction can be eliminated, the face can be reliably extracted even in general natural paintings, and the color of the face can be corrected appropriately and efficiently in a manner that does not become unnatural. It becomes possible to do.
JP 2001-186323 A JP-A-10-126184 JP 2001-209802 A JP 2001-148780 A

  However, the above image correction method proposed by the present applicant is based on the premise that the type of hair color to be corrected is black hair, and the accuracy of hair region extraction is poor in cases other than black hair. There is a problem of becoming.

  For example, chestnut hair, brown hair, blonde hair, white hair often seen in elderly people, blonde hair or silver hair often seen in Westerners, etc. Since the lightness and hue of the hair are different, it cannot be handled by the black hair extraction algorithm alone.

  In view of the above circumstances, the present invention extracts a face area and a hair area from an area including a face designated on a digital image or its vicinity, and corrects the color of a correction target area composed of the face area and the hair area. In the processing, an image correction method and apparatus capable of accurately extracting a hair region not only for black hair but also for other types of hair, accurately extracting a correction target region and performing appropriate color correction, and therefore The purpose is to provide a program.

  The image correction method of the present invention is based on an area designating step of designating an area including a single person's face in a human image with a person as a subject according to an input from an operator, and color information of the designated area. A skin region extracting step for extracting a skin region including the face of the single person, a hair color type information acquiring step for acquiring hair color type information representing a type of hair color of the single person, and the hair color According to a predetermined algorithm corresponding to the type of hair color represented by the type information, a hair region extraction step for extracting the hair region of the single person, and a region composed of the skin region and the hair region as a correction target region A correction target region setting step to be set; and a correction step of correcting at least one of hue, saturation and lightness in the correction target region. .

  In the image correction method of the present invention, the hair color type information acquisition step may be a step of acquiring information representing the color type of the hair input from the operator as the hair color type information.

  In the image correction method of the present invention, the hair color type information acquisition step specifies a partial region on the hair of the single person according to an input from an operator, and is based on color information of the partial region. A step of determining a hair color type of the single person and generating information indicating the determined type as the hair color type information.

  In the image correction method of the present invention, the designated area is an area having a outline that is substantially circumscribed on the face and hair of the single person, and the hair color type information acquisition step includes an outline of the designated area. Based on the position information, the upper contour of the hair region of the single person is extracted by approximating an elliptical arc inscribed in the contour of the designated region, and the skin region and the designated region are extracted from the designated region. Out of the region excluding the contour and the region surrounded by the upper contour of the hair region, a region within the rectangular region that is 1/2 to 1/4 above the specified region is extracted as a temporary hair region, and the temporary hair region It may be a step of determining the hair color type of the single person based on the color information of the region, and generating information representing the determined type as the hair color type information.

  In the image correction method of the present invention, the hair region extraction step obtains a histogram of lightness values of pixels constituting the image of the designated region when the type represented by the hair color type information is black hair, and the lightness value is It may be a step of extracting an image area composed of pixels corresponding to an area on the histogram smaller than a predetermined threshold as the hair area.

  In the image correction method of the present invention, the hair region extraction step obtains a histogram of lightness values of pixels constituting the image of the designated region when the type represented by the hair color type information is gray hair, It may be a step of extracting an image area composed of pixels corresponding to an area on the histogram having a value larger than a predetermined threshold value as the hair area.

  Further, in the image correction method of the present invention, the hair region extraction step includes the step of extracting pixels constituting the image of the designated region when the type represented by the hair color type information is a predetermined type of either blond or brown hair. A hue value histogram is obtained, and an image region comprising pixels corresponding to the largest area among the peaks on the histogram whose hue value is within a predetermined range including the average hue value of the predetermined type of hair It may be a step of extracting as a hair region.

  The image correction apparatus according to the present invention is based on area designation means for designating an area including a single person's face in a human image of a person as a subject in response to an input from an operator, and color information of the designated area. Skin area extracting means for extracting a skin area including the face of the single person, hair color type information acquiring means for acquiring hair color type information representing a type of hair color of the single person, and the hair color According to a predetermined algorithm corresponding to the type of hair color represented by the type information, a hair region extraction means for extracting the hair region of the single person, and a region composed of the skin region and the hair region as a correction target region It is characterized by comprising correction target area setting means for setting and correction means for correcting at least one of hue, saturation and lightness in the correction target area.

  In the image correction apparatus of the present invention, the hair color type information acquisition unit may acquire information representing the type of hair color input from the operator as the hair color type information.

  In the image correction apparatus of the present invention, the hair color type information acquisition unit specifies a partial area on the hair of the single person according to an input from an operator, and is based on color information of the partial area. In this case, the hair color type of the single person may be determined, and information representing the determined type may be generated as the hair color type information.

  Further, in the image correction apparatus of the present invention, the designated area is an area having a outline that is substantially circumscribed on the face and hair of the single person, and the hair color type information acquisition unit is configured to obtain an outline of the designated area. Based on the position information, the upper contour of the hair region of the single person is extracted by approximating an elliptical arc inscribed in the contour of the designated region, and the skin region and the designated region are extracted from the designated region. Out of the region excluding the contour and the region surrounded by the upper contour of the hair region, a region within the rectangular region that is 1/2 to 1/4 above the specified region is extracted as a temporary hair region, and the temporary hair region The type of hair color of the single person may be determined based on the color information of the area, and information representing the determined type may be generated as the hair color type information.

  In the image correction apparatus of the present invention, the hair region extracting means obtains a histogram of lightness values of pixels constituting the image of the designated region when the type represented by the hair color type information is black hair, and the lightness value is An image area composed of pixels corresponding to the area on the histogram smaller than a predetermined threshold may be extracted as the hair area.

  In the image correction apparatus of the present invention, the hair region extracting means obtains a histogram of lightness values of pixels constituting the image of the designated region when the type represented by the hair color type information is gray hair, An image region composed of pixels corresponding to regions on the histogram whose value is larger than a predetermined threshold may be extracted as the hair region.

  In the image correction apparatus of the present invention, the hair region extracting means may be configured to detect pixels of the designated region when the type represented by the hair color type information is a predetermined type of either blond or brown hair. A hue value histogram is obtained, and an image region comprising pixels corresponding to the largest area among the peaks on the histogram whose hue value is within a predetermined range including the average hue value of the predetermined type of hair It may be extracted as a hair region.

  The program according to the present invention is based on area designation means for designating an area including a single person's face in a human image of a person as a subject in response to an input from an operator, and color information of the designated area. A skin region extracting means for extracting a skin region including the face of the single person, a hair color type information acquiring means for acquiring hair color type information representing a hair color type of the single person, and the hair According to a predetermined algorithm corresponding to the type of hair color represented by the color type information, a hair region extracting means for extracting the hair region of the single person, and a region composed of the skin region and the hair region is a correction target region And a correction target area setting unit that is set as a correction unit and a correction unit that corrects at least one of hue, saturation, and brightness in the correction target area.

  In the program according to the present invention, the hair color type information acquisition unit may acquire information indicating the color type of the hair input from the operator as the hair color type information.

  Further, in the program of the present invention, the hair color type information acquisition unit specifies a partial region on the hair of the single person according to an input from an operator, and based on the color information of the partial region A hair color type of a single person may be determined, and information representing the determined type may be generated as the hair color type information.

  Further, in the program of the present invention, the designated area is an area having a outline that is substantially circumscribed on the face and hair of the single person, and the hair color type information acquisition means includes contour position information on the designated area. Based on the above, the upper contour of the hair region of the single person is extracted by approximating an elliptical arc inscribed in the contour of the designated region, and from the designated region, the skin region, the contour of the designated region, and Out of the region excluding the region surrounded by the upper outline of the hair region, the region in the rectangular region that is 1/2 to 1/4 of the designated region is extracted as a temporary hair region, and the temporary hair region The hair color type of the single person may be determined based on the color information, and information representing the determined type may be generated as the hair color type information.

  In the program of the present invention, the hair region extraction means obtains a histogram of lightness values of pixels constituting the image of the designated region when the type represented by the hair color type information is black hair, and the lightness value is a predetermined value. An image area composed of pixels corresponding to an area on the histogram smaller than a threshold value may be extracted as the hair area.

  In the program of the present invention, the hair region extracting means obtains a histogram of lightness values of pixels constituting the image of the designated region when the type represented by the hair color type information is white hair, and the lightness value is An image area composed of pixels corresponding to an area on the histogram that is larger than a predetermined threshold may be extracted as the hair area.

  Further, in the program of the present invention, the hair region extracting means, when the type represented by the hair color type information is a predetermined type of either blond or brown hair, the hue value of the pixels constituting the image of the designated region And an image region composed of pixels corresponding to the largest area among the peaks on the histogram whose hue value is within a predetermined range including an average hue value of the predetermined type of hair. It may be extracted as

  According to the image correction method and apparatus and the program therefor according to the present invention, the type of hair color to be extracted is obtained in advance by a predetermined means, and the color information in the region designated by the operator or the color in the vicinity thereof is obtained. Based on the specified region or the vicinity thereof, the person's face region is extracted, and the hair region is extracted according to a predetermined algorithm corresponding to the previously determined hair color type, and the correction is made up of the face region and the hair region. Since the color of the target area is corrected, it is possible to accurately extract the hair area not only for black hair but also for other types of hair. However, it is possible to reliably extract the correction target area including the face area and the hair area, and to appropriately and efficiently correct the face color in a manner that does not become unnatural.

  Embodiments of the present invention will be described below with reference to the drawings.

  FIG. 1 is a block diagram showing the configuration of the first embodiment of the image correction apparatus of the present invention. This image correction device is a device for obtaining the same effect on a digital image as that of so-called dodging in analog photographic technology, and corrects only the face darkly expressed on the image and the surrounding skin partially brightly. Is.

  As shown in FIG. 1, the image correction apparatus 1 includes an image data storage unit 10 that stores a plurality of image data, and image data of a person to be corrected from image data stored in the image data storage unit 10. An image selection unit 20 for selecting a region, and a region designation for designating a partial region including the face of a single person on the person image In represented by the image data of the person selected by the image selection unit 20 in response to an input from the operator Unit 31, a skin region extraction unit 32 that extracts a skin region Rs including the face of the single person based on the color information of the designated region Ru designated by the region designation unit 31, and the color of the hair of the single person The hair color type information acquisition unit 33 that acquires the hair color type information Vh representing the type, and extracts the hair region Rh of the single person according to a predetermined algorithm according to the type of hair color represented by the hair color type information Vh. Hair area to do The extraction unit 34, a correction target region setting unit 35 that sets a region composed of the extracted skin region Rs and the hair region Rh as the correction target region Rm, and determines the weight of correction performed on each pixel in the correction target region Rm. A mask creating unit 36 that creates a mask image Mask, an image correcting unit 40 that obtains an overall corrected image Mod by correcting at least one of hue, saturation, and brightness in the entire image of the person image In, a person image In, and a mask image Each of the image data stored in the image composition unit 50 and the image data storage unit 10 represents the corrected image In ′ in which only the correction target region Rm is partially corrected by combining the Mask and the entire correction image Mod. In addition to images, a person image In, a mask image Mask, a corrected image In ′, etc., are laid out in a predetermined layout in response to an input from the operator or automatically. A display control unit 60 that performs control to display on the screen of the image display unit 70 described later, an image display unit 70 that displays various images on the screen under the control of the display control unit 60, image selection by the image selection unit 20, The configuration includes an input operation unit 80 for inputting information indicating various requests such as selection of an image to be displayed on the image display unit 70 to each unit.

  Note that the mask creation unit 33, the image correction unit 40, and the image composition unit 50 in this embodiment function as correction means in the image correction apparatus of the present invention.

  The image data storage unit 10 stores a plurality of image data, and stores a magnetic recording device such as a hard disk for storing image data acquired by photographing with a digital camera, image data acquired by reading with a scanner, and the like. It is composed of

  The image selection unit 20 selects a correction target from among the image data stored in the image data storage unit 10 based on information specifying image data to be corrected input by the operator through the input operation unit 80. The image data of the person image to be selected is selected. For example, on the screen of the image display unit 70, a list of images represented by the image data stored in the image data storage unit 10 is displayed as thumbnail images under the control of the display control unit 60. A desired image to be corrected is selected by a GUI (graphical user interface) while viewing the screen.

  The area designating unit 31 designates a partial area including the face of a single person on the person image In represented by the image data of the person selected by the image selecting unit 20 in response to an operator input. For example, when the operator designates two predetermined points with a pointer on the person image In displayed on the screen of the image display unit 70 using a mouse, a keyboard, or the like that constitutes the input operation unit 80, the region designation unit 31 designates a rectangular area whose diagonal is a straight line connecting the two points.

  The skin region extraction unit 32 extracts the skin region Rs including the face of the single person based on the color information of the region Ru (hereinafter referred to as the “designated region”) designated by the region designation unit 31. Details of the method for extracting the skin region Rs will be described later.

  The hair color type information acquisition unit 33 acquires hair color type information Vh representing the type of hair color of the single person. In the first embodiment, the hair color type information acquisition unit 33 acquires information representing the type of hair color input by the operator as the hair color type information Vh. That is, the operator looks at the person image In displayed on the screen of the image display unit 70, determines the hair color type of the single person, and selects one of a plurality of options prepared in advance. It is assumed that information indicating the color type of the hair is input according to the format to be performed.

  Depending on the type, amount, and distribution of the melanin pigment in the hair, the type of hair color may be “blackish brown” (black hair), “yellowish red” (blonde), or “maroon” located between them ( (Brown hair, chestnut hair) and the “white hair system” from which pigments are missing. Therefore, in the present embodiment, considering these types, the above options are black hair, white hair, blonde / brown hair, and others (color colors other than blonde / brown hair).

  The hair region extracting unit 34 extracts the hair region Rh of the single person according to a predetermined algorithm corresponding to the type of hair represented by the hair color type information Vh. The hair region Rh has a feature that it is often present in almost the same color except in special cases. Therefore, the hair region Rh is generally extracted using color information in the extraction target region. However, if the color of the hair is different, the characteristics of the information of the color, that is, the characteristics of lightness, hue, and saturation are different, so it is not easy to extract various colors of hair with a predetermined algorithm. Absent. Therefore, by classifying the hair into rough types according to color and extracting the hair region Rh according to a predetermined algorithm prepared for each type, it is possible to extract the hair region Rh with higher accuracy.

  The correction target region setting unit 35 sets a region composed of the skin region Rs extracted by the skin region extraction unit 32 and the hair region Rh extracted by the hair region extraction unit 34 as the correction target region Rm.

  The mask creating unit 36 creates a mask image Mask that determines the weight (correction degree) of correction performed on each pixel in the correction target region Rm. The mask creating unit 36 corrects the entire image of the person image In, which will be described later. A mask image Mask is generated so that when the correction image Mod is multiplied, an image in which the correction is applied only to the correction target region Rm is obtained.

  The mask image Mask has the same image size (resolution) as the person image In, and each pixel thereof is 0 to 1 (0 when the correction degree is 0%) according to the correction degree applied to the pixel. And a pixel value corresponding to a weight of 1) when the correction degree is 100%. The mask image Mask is set so that the degree of correction applied to each position of the correction target region Rm decreases from the inside to the outside of the correction target region Rm in the vicinity of the boundary. The degree of correction is set so as to change more gently in the vicinity of the boundary of the correction target region Rm as the brightness difference between the inside and outside of the correction target region Rm is smaller.

  The image correction unit 40 corrects the correction target region Rm based on the difference between the predetermined target brightness after correction of the correction target region Rm and the average value or intermediate value of the lightness in the correction target region Rm. A correction amount (a degree of correction for each predetermined component on the image) that is considered to be appropriate is calculated, and this is used as an initial value to correct the entire image of the person image In. Thereafter, the corrected image is displayed on the screen of the image display unit 70 under the control of the display control unit 60. The image correction unit can be further finely adjusted while the operator looks at the screen. 40 has a function of changing the correction amount based on information for finely adjusting the correction amount input by the operator. Then, the image is corrected again with the adjusted correction amount, and the corrected image is displayed on the screen of the image display unit 70. Note that the components on the image that can be corrected are not particularly limited, but here, in order to improve the operability related to the correction, as components of at least one of hue, saturation, and lightness, several components are previously stored. The operator can select one of the components and increase or decrease the component. For example, the components on the image that can be corrected are defined as four types of intensity of the three colors of cyan C, magenta M, and aero Y and brightness D, and the screen of the image display unit 70 is controlled by the display control unit 60. In addition to displaying these component items, next to each component item are a counter that displays the correction level of the component step by step, and an increase / decrease button that can increase or decrease the correction level one step at a time, The operator adjusts the correction level of each of these components using the GUI.

The image composition unit 50 composes the entire correction image Mod and the original person image In with a predetermined weight represented by the mask image Mask for each corresponding pixel, whereby only the correction target region Rm is partially corrected. A corrected image In ′ is obtained. That is, these images are synthesized according to the following equation (1) to obtain a corrected image In ′.

Here, x and y represent the x coordinate and the y coordinate in each image, and the possible values of Mask (x, y) are 1 ≧ Mask (x, y) ≧ 0.

  The display control unit 60 controls an image displayed on the screen of the image display unit 70 and its layout based on information indicating various requests input by the operator through the input operation unit 80.

  The image display unit 70 is configured by a CRT monitor, a liquid crystal panel, or the like that displays various images on the screen under the control of the display control unit 60.

  The input operation unit 80 is configured by a keyboard, a mouse, and the like connected by an interface for realizing a GUI.

  Next, the flow of processing in the image correction apparatus 1 of this embodiment will be described. FIG. 2 is a flowchart showing the flow of processing in the image correction apparatus 1.

  First, the screen of the image display unit 60 displays a list of images represented by the respective image data stored by the image data storage unit 10 under the control of the display control unit 60 in response to a request from an operator or automatically. Is displayed as a thumbnail image. Here, when the operator inputs information specifying a person image from the images through the GUI, the image selection unit 20 selects the person image as a person image In to be corrected based on the information. (Step S1). FIG. 3A shows an example of the person image In selected in this way. A person image In shown in FIG. 3A is a photographic image in which two men and women are photographed, and represents a state in which a man is considerably tanned compared to a woman.

  When the person image In to be corrected is selected, as shown in FIG. 4, the original image In or the person image (correction) when the correction effect is reflected is displayed on the screen of the image display unit 60. Area AR1 in which any one of In ′ images is displayed large, area AR2 in which the mask image Mask and the image of the correction target region Rm in the person image In are displayed in a superimposed position, and the correction target on the person image In. AR3 that displays a candidate for the hair color type of a single person as a selection button, the level of each of the above-described C, M, Y, and D components and the increase / decrease of each of these components with respect to the correction target region Rm of the person image In A window Win composed of an area AR4 for displaying buttons is displayed. At this stage, the person image In is displayed in the area AR1, and nothing is displayed in the area AR2.

  Here, when the operator inputs information specifying a rectangular area including the face and hair of a single person having a face or skin to be corrected on the displayed person image In, the area designating unit 31 Based on the information, the rectangular area is designated as a designated area Ru (step S2).

  Further, when the operator presses a button of a type that is considered to be the closest to the hair color type of the single person from among the hair type selection buttons displayed in the area AR3, the hair color type information acquisition unit 33 generates and acquires information representing the selected type as hair color type information Vh (step S3).

  Next, the skin region extraction unit 32 extracts the skin region Rs including the face of the single person from the designated region Ru based on the color information of the designated region Ru (step S4). On the other hand, the hair region extracting unit 34 extracts the hair region Rh of the single person from the designated region Ru according to a predetermined algorithm corresponding to the type of hair color represented by the hair color type information Vh (step S5). ). Then, the correction target region setting unit 35 sets a region surrounding the extracted skin region Rs and the hair region Rh as the correction target region Rm (step S6). The details of the process of extracting the skin region Rs and the process of extracting the hair region Rh (hereinafter, the process combining both processes may be referred to as a process of extracting the correction target region Rm) will be described later.

  The mask creating unit 36 creates a mask image Mask such that the degree of correction with respect to the extracted correction target region Rm satisfies the above-described condition (step S7). Details of the process for creating the mask image Mask will be described later. FIG. 3 (3) is a diagram showing an example of the mask image Mask created in this way. The mask image Mask shown in FIG. 3 (3) is a mask image when a region surrounding the skin and hair including the male face as the subject is extracted as the correction target region Rm, and the boundary of the region is blurred. ing. Here, in the area AR2 in the window Win displayed on the screen of the image display unit 70, the mask image Mask and the image of the correction target region Rm in the person image In are displayed in a superimposed manner. By viewing this image, the operator can confirm whether or not the extracted correction target region Rm is a desired region.

  First, the image correction unit 40 detects a skin color area from the extracted correction target area Rm, and calculates a correction amount such that the brightness of the skin color area becomes an appropriate brightness. That is, the saturation of each pixel in the correction target region Rm is obtained, and other regions are extracted from the correction target region Rm excluding pixels whose saturation is below a predetermined threshold. As a result, the skin color area excluding the hair, eyebrows, black eyes, and shadows in the correction target area Rm is extracted. Then, the brightness of each pixel in this skin color region is obtained, and the average value Dave or intermediate value Dmid of the brightness is calculated. Then, the lightness correction amount Dmod is obtained by subtracting the above-mentioned Dave or Dmid from the lightness that is preferable as the lightness of the skin of the person in the photographic image, that is, the target lightness value after correction (Dmod = Dideal− (Dave or Dmid)). Then, based on the value of the brightness correction amount Dmod, an initial value of the correction amount of each component on the image is set. For example, a correction amount to be corrected for each component of C, M, and Y (may be R, G, and B) is defined as a correction amount level, and a signal value for each color component of each pixel is set to this correction amount level. Conversion is performed by a corresponding conversion curve (for example, a one-dimensional lookup table).

  The image correction unit 40 corrects the entire image of the person image In using the initial value of the correction amount, and the corrected whole image is displayed on the screen of the image display unit 70 under the control of the display control unit 60. Is done. While viewing the screen, the operator can further correct the correction amount by using the GUI to display each component of C, M, Y, and D displayed in the area AR3 in the window Win displayed on the screen of the image display unit 70. Press the correction amount increase / decrease buttons to finely adjust the correction amount of each component.

  The image correction unit 40 corrects the entire image of the person image In again according to the adjusted correction amount setting to obtain an overall correction image Mod (step S8), and the corrected overall image is displayed on the screen. Is done. FIG. 3B is a diagram illustrating an example of the overall correction image Mod obtained in this way. The overall correction image Mod shown in FIG. 3B is an image of the person image In so that the skin area including the male face, which is the subject, is as bright as the skin of the woman in the person image In before correction. It is an image in a state where the brightness of the entire image is corrected to a higher one.

  The image synthesis unit 50 synthesizes these images in accordance with the above equation (1), performs a desired correction only on the correction target region Rm, and the effect of the correction is reduced near the boundary of the correction target region Rm. A person image such as that shown is obtained (step S9). The display control unit 60 performs control to display this image in an updated form in the area AR1 on the screen of the image display unit 70.

  Then, Step S8 and Step S9 are repeated as necessary, and finally a corrected image In ′ obtained by applying a desired correction to the person image In is obtained (Step S10). FIG. 3 (4) is a diagram showing an example of the corrected image In ′ obtained in this way. The corrected image In ′ shown in FIG. 3 (4) is a diagram in which the male skin area is brightly corrected so that the male skin area and the female skin area have the same brightness.

  Next, the process for extracting the correction target region Rm and the process for creating the mask image Mask will be described in detail. Here, first, the flow of these processes will be schematically described, and then more specific processes will be described.

  FIG. 5 is a flowchart showing a basic processing flow in extraction of the correction target region Rm and creation of the mask image Mask. This basic processing algorithm can be broadly divided into four parts: skin seed extraction, skin seed growth, hair detection, and mask creation. Note that “skin seed” refers to a group of pixels extracted with high reliability as belonging to the skin region.

  In the “skin seed extraction” section, a highly reliable skin pixel (hereinafter referred to as a highly reliable skin pixel) using a histogram of hue, saturation, and lightness in a rectangular designated region Ru designated by the operator. Is extracted (step S11). It is determined whether or not a sufficient number of pixels are obtained as skin seeds (step S12). If not enough, in the extraction of the highly reliable skin pixel, separation by threshold determination based on hue, saturation, and lightness is performed. The condition is relaxed, and highly reliable skin pixels are extracted again within the designated region Ru (step S13). After extraction of the highly reliable skin pixels, the gap between the highly reliable skin pixels is filled using the closing process of the morphology filter to obtain several discontinuous skin regions in the designated region Ru (step S14). Since some of the obtained plurality of skin regions may belong to the background, the skin region having the largest area among them is selected as the skin seed from which the region is expanded.

  In the “skin seed growth” section, the contour of the above-mentioned skin seed region is extracted, pixels on this contour are sampled at a predetermined interval, and each sampled pixel is grown into a skin seed. The seed pixel is the original seed pixel. Then, for each seed pixel, the hue, saturation, and brightness of a predetermined area adjacent to the seed pixel are examined, and an area that satisfies the predetermined condition is added as a skin area to grow the skin area. When the skin region is grown for all seed pixels, a skin region having a larger area can be obtained. Further, a skin area having a larger area can be obtained by performing the same processes as those described above on the grown skin area, such as contour extraction, seed pixel setting, and skin area growth. Such a process is repeated until there is no neighboring pixel to be grown or the number of repetitions of this process reaches a predetermined number (steps S15 and S16).

  In the “detection of hair” section, according to a predetermined algorithm corresponding to the type of color of the hair to be extracted, a histogram of pixel brightness or hue in a region not belonging to the skin region of the human image In is obtained. The pixel group corresponding to the region that satisfies the predetermined condition is extracted as the hair region Rh (step S17). A region surrounding the skin region Rs including the face extracted in this way and the hair region Rh is extracted as a correction target region Rm.

  The mask creation unit creates a mask image Mask having the same image size as that of the person image In and cuts out an area other than the extracted correction target area. At this time, the brightness difference (contrast) inside and outside the correction target area Rm is created. ) To blur the boundary of the correction target region Rm (step S18).

  From here, detailed processing in each of the above-described units will be described.

(Extraction of skin seeds)
In this part, a pixel group that is most likely to be skin in the rectangular local region Ru designated by the operator is extracted as a skin seed from which the skin region is grown. It is clear from the applicant's experience that the skin region appears in a narrow predetermined range in the hue component distribution diagram. FIG. 6 is a diagram showing the distribution of hue, saturation, and lightness in the skin area when the gradation is 0 to 255. According to the distribution diagram of FIG. 6, the hue of the skin region can be seen in a relatively narrow range of 235 to 255 and 0 to 30, but the saturation and lightness are wider, and the hue is seen in the entire range of 0 to 255. It is done. From these distribution maps of hue, saturation and lightness, it can be said that in the extraction of the skin region, the distribution of hue can be made more reliable than the distribution of the other two components.

  First, as shown in FIG. 7, a rectangular trust region Rt with high reliability is selected within a designated region Ru designated by the operator. The trust region Rt is a point in which the center is shifted by H / 10 pixels downward from the center of the designated region Ru, where H is the number of pixels in the designated region Ru and W is the number of pixels in the width. Is a rectangular region in which 100 pixels or 21 × H / 40 pixels, whichever is smaller, and the horizontal width is 100 pixels or 2 × W / 3 pixels, whichever is smaller. The center of the designated area designated by the operator may be the skin area, but may be the eyes or eyebrows. In the latter case, the vicinity of the center cannot be adopted as the skin region. The designated area Ru may include a background portion similar to the skin, but since the center of the designated area Ru belongs to the face in most cases, the small area close to the center is almost entirely skin. It is often an area. For this reason, the trust region Rt defined as described above is selected, and image information in this region is acquired.

  After the selection of the trust region Rt, a pixel whose hue is in the range of 250 to 255 and 0 to 20 within the trust region Rt is selected as the pixel most likely to be skin. Then, a hue histogram of these pixels is obtained. As shown in FIG. 8, an area [Hcl, Hch] in which the number of pixels is maximum with a predetermined width (for example, 20) is extracted from the hue histogram. Similarly, in the saturation and lightness histograms, areas [Scl, Sch], [Icl, Ich] in which the number of pixels is maximum within a predetermined width (for example, 30 for saturation and 40 for lightness). ] Is extracted. Then, a skin separation threshold is set using these areas. First, pixels having hues in the 250 to 255 and 0 to 20 areas (referred to as [Hl, Hh]) in the designated area Ru are extracted in the same manner as when pixels are extracted in the trust area Rt. To do. A saturation and lightness histogram is obtained for pixels in the area of hue [Hl, Hh] in the designated region Ru. In the saturation histogram, the bin outside the area of [(Scl + Sch) / 2-30, (Scl + Sch) / 2 + 30] is set to 0 (zero), and in the brightness histogram, [(Icl + Ich) / 2-40, (Icl + Ich) ) / 2 + 40] set the bin outside the area to 0 (zero). Thus, in the histogram with both ends cut off, areas [Sl, Sh], [Il, Ih] where the pixel value is maximum within a predetermined width (for example, 30 for saturation and 40 for lightness) are shown. Extract. FIG. 9 is a diagram showing an example of area extraction in the saturation histogram.

  Finally, in the designated area Ru, pixels whose hue is in the area [Hl, Hh], saturation is in the area [Sl, Sh], and lightness is in the area [Il, Ih] are extracted as the most skinlike pixels. To do. Since these pixels are obtained by separation based on threshold determination, there are many small gaps between these pixels, forming several discontinuous regions. Therefore, these gaps are filled by using a closing process of the morphology filter, and the region having the maximum area is selected as the skin seed Rskin1.

  If the skin seed extracted by the above algorithm does not have a sufficient number of pixels (for example, 1000 pixels or H × W / 8), the conditions for skin seed extraction are relaxed. That is, pixels whose hue is in the predetermined range in the trust region Rt are extracted, and these pixels are set as the most likely pixels as they are. Then, the gap between these pixels is filled using the closing process of the morphology filter, and the region having the maximum area is selected as the skin seed Rskin1.

(Skin seed growth)
Once the skin seed region is obtained, the skin seed region is grown by an adaptive region growth technique. First, the contour CR1 of the skin seed region Rskin1 is extracted, and points [P1, P2,..., Pn] are sampled at predetermined intervals on the contour (for example, a total of 32 points are taken). Then, as shown in FIG. 10A, the region growing method is simultaneously applied to each point Pi. The skin seed region is a continuous region, and when the skin seed grows, the inside of the seed is equivalent to the seed point on the contour. There are many pixels on the outline, and color information is similar between two pixels that are not so far apart. Therefore, in order to reduce the number of times of seed growth, the seed points are set at predetermined intervals in this way. In order to grow the seed region, it is first necessary to set the growth conditions. Here, the following conditions are set.

As shown in FIG. 10 (2), a circle with a radius of 30 pixels is drawn around the point Pi with respect to the seed point Pi, and the inner region Rin that has already been extracted as a skin region within this circle has not yet been extracted. Two regions of the outer region Rout are obtained. In the inner region Rin, the histograms Histin, Histsin and Histin for the hue, saturation and lightness are obtained. Similarly, histograms Histhout, Histsout and Histiout are obtained in the outer region Rout. First, the first hue, saturation, and brightness areas [Hpi0l, Hpi0h], [Spi0l, Spi0h] and [Ipi0l, Ipi0h] are selected using Histhin, Histsin, and Histiin. As shown in FIG. 11, each of these areas has an area where bins exceed 10 pixels in the hue histogram, and the remaining 80% pixels when the area excludes 10% above and below the saturation in the saturation histogram. In the included area and brightness histogram, the area is an area including the remaining 80% of pixels excluding the upper and lower 10% of the brightness. In addition, in the regions Rin and Rout, maximum ranges are obtained for each of hue, saturation, and lightness. The maximum range is an area where the pixel value is maximum at a predetermined interval (for example, 10 for hue, 15 for saturation, and 20 for lightness). Here, these maximum ranges are [Hinpeakl, Hinpeakh], [Sinpeakl, Sinpeakh] and [Iinpeakl, Iinpeakh] in Rin, and [Houtpeakl, Houtpeakh], [Soutpeakl, Soutpeakh] and [Ioutpeakl, Ioutpeakh] in Rout. ]. Then, the area in Rin and the area in Rout are compared, and if the two areas are close enough, the width of the area is expanded. Details of the expansion of the area are as follows.

  That is, when the difference between the intermediate value of the area [Hinpeakl, Hinpeakh] and the intermediate value of the area [Houtpeakl, Houtpeakh] is less than 15, the area [Hpi0l, Hpi0h] is changed to the area [min (Houtpeakl, Hpi0l), max (Houtpeakh). , Hpi0h)]. When the difference between the intermediate value of the area [Sinpeakl, Sinpeakh] and the intermediate value of the area [Soutpeakl, Soutpeakh] is smaller than 30, the area [Spi0l, Spi0h] is changed to the area [min (Soutpeakl, Spi0l), max (Soutpeakh). , Spi0h)]. When the difference between the intermediate value of the area [Iinpeakl, Iinpeakh] and the intermediate value of the area [Ioutpeakl, Ioutpeakh] is smaller than 15, the area [Ipi0l, Ipi0h] is changed to the area [min (Ioutpeakl, Ipi0l), max (Ioutpeakh). , Ipi0h)]. Here, min (α, β) is a function that takes the smaller of α and β, and max (α, β) is a function that takes the larger of α and β.

  Then, the area is further expanded so that it can be applied even when the face has light skin and dark skin. As a rule of thumb, it is clear that for light and dark skin, lightness and saturation have a very wide range of values, but the hue remains almost unchanged. Therefore, the lightness and saturation areas are further expanded while the hue area is fixed.

  In this way, it is possible to obtain a reference area for the growth of the area, that is, an area for each of hue, saturation, and brightness to be included as a skin area. In the region growing process, if there are neighboring pixels in these areas until there are no neighboring pixels to be grown, those pixels are included in the skin region. When this region growth is performed on various points and the growth is completed for all the seed points, a larger skin region Rskin2 is obtained. Further, the same region growth method as described above is applied to the skin region Rskin2. Such a process is repeated about 5 times, for example.

  However, it is difficult to separate the skin region Rs from the background similar to the skin region, and if the background has a hue close to the skin color, the skin color background may be extracted as the skin region Rs. In such a case, as shown in FIG. 13, it is assumed that a region having a contour that is substantially circumscribed on the face and hair of a single person in the person image In is designated as the designated region Ru by the operator. Then, an elliptical arc inscribed in the contour of the designated region Ru is drawn, and the lower contour of the face region is extracted by approximating the elliptical arc.

  In this process, in order to determine the respective areas of hue, saturation, and lightness to be extracted as the skin region Rs, information on the colors inside and outside the region already extracted as the skin region Rs is used. I will explain why. Since the growth of the region is an operation of selecting similar pixels, the applicant of the experience thinks that the information used for the growth of the region is initially sufficient only for the information inside the skin region Rs. It was. However, in practice, only a part of the face and skin could be extracted. The reason is that the skin tends to belong to a wide range in saturation and lightness, and even if the region has the same hue, the seeds in the skin region Rs include information on the neighboring external pixels. This is because there is not. Therefore, in this processing, information on neighboring external pixels is considered.

(Detection of hair)
When the skin region Rs is extracted, the remaining region belongs to the hair region Rh and the background region. Therefore, in order to extract the hair region Rh, it is only necessary to extract the hair region Rh by separating the hair region Rh and the background region in the region excluding the skin region from the designated region Ru. Here, the hair region Rh is extracted by using the following algorithm in accordance with the hair color type represented by the hair color type information Vh already acquired.

  First, when the type represented by the hair color type information Vh is black hair, a histogram of brightness values of pixels constituting the image of the designated area Ru is obtained, and pixels corresponding to areas on this histogram whose brightness values are smaller than a predetermined threshold value. Is extracted as a hair region Rh. When the hair is black hair, the hue and saturation of the hair region Rh tend to vary greatly. Therefore, these components of hue and saturation are difficult to use in extracting the hair region. However, the lightness distribution almost always falls within a narrow area and the lightness is low. Therefore, in this process, a binarization separation method is used to distinguish the background region and the hair region Rh. FIG. 12 shows the method of Ootu (Mr. Otsu), which is this binarization separation method, and the hair region Rh can be detected with high accuracy. In this method, a threshold value Ti is selected in a histogram of brightness values according to a predetermined rule (for example, a pixel value at which the rate of change of the cumulative histogram of brightness values is the lowest is set as the threshold value Ti). Let the small pixel be a pixel like hair. By this method, a large number of hair-like regions are detected. Of these regions, the region having the largest area is selected as the hair region Rh. Two or more local maximum points may appear in the brightness value histogram. In such a case, the hair region Rh cannot be detected well by the Ootu method. The basic idea is to exclude the local maxima with low reliability and leave only two reliable maxima, and apply Ootu's method. What is the existence of many maxima on the histogram? There is a problem of how to find out, and how to find a local maximum point with low reliability. To solve this problem, consider applying the following method. As already mentioned, it is clear that the brightness values of most pixels in the hair region Rh are small. Therefore, after setting a bin larger than a certain value (for example, 128) to 0 (zero), the Ootu method is applied. With this method, it is possible to avoid the influence of a bin having a brightness value larger than a predetermined value (here, 128), and the above problem can be solved. In fact, even in experiments, the hair region Rh was successfully detected by this method.

  Next, when the type represented by the hair color type information Vh is gray hair, a histogram of brightness values of pixels constituting the image of the designated area Ru is obtained, and the brightness value corresponds to an area on the histogram that is greater than a predetermined threshold value Ti. An image region composed of pixels to be extracted is extracted as a hair region. When there are three or more maximum values in the histogram of brightness values, a bin whose brightness value is smaller than a predetermined value (for example, 128) is set to 0 (zero) in order to leave a brightest local maximum point. Devise such as doing.

  In addition, when the type represented by the hair color type information Vh is blonde hair or brown hair, a histogram of hue values of pixels constituting the image of the designated area Ru is obtained, and the hue value is an average hue value of blonde hair or brown hair. The area of the mountain on this histogram that is within the specified range (within the range Hlim0 <H <Hlim1 where the hue value is H, the lower limit value of the hue value corresponding to blonde and brown hair is Hlim0, and the upper limit value is Hlim1) An image region composed of pixels corresponding to the largest one is extracted as a hair region Rh.

  When the type represented by the hair color type information Vh is a type of color other than black hair, white hair, blond hair, or brown hair, the outline of the designated area Ru is abbreviated to the face and hair of a single person to be corrected. Assuming that it is circumscribed, first, within the designated region Ru, a position having a high probability of being in the hair region Rh, for example, 1 / of the vertical width of the designated region Ru upward from the center of the designated region Ru. Hue information in a partial area of a predetermined size at a location moved by a distance of 3 is obtained. Next, a histogram of hue values of pixels constituting the image of the designated area Ru is obtained, and the hue value is within a predetermined range including the average hue value of the partial area (the hue value is H, the average hue value is H0, An image region composed of pixels corresponding to the largest area among the peaks on the histogram, where Hw is Hw and H0−Hw <H <H0 + Hw), is extracted as a hair region.

  The predetermined threshold value and the predetermined range are obtained from an empirical rule for a value or range in which the target type of hair is most effectively extracted.

  In this way, the hair region Rh can be extracted with high accuracy by switching the algorithm according to the color type of the hair. However, when the background is similar to the color of the hair, it is difficult to accurately separate the background region and the hair region Rh, and the boundary between the background region and the hair region Rh may be inaccurate. In such a case, as shown in FIG. 13, it is assumed that a region having a contour that is substantially circumscribed on the face and hair of a single person in the person image In is designated as the designated region Ru by the operator. Then, an elliptical arc inscribed in the contour of the designated region Ru is drawn, and the upper contour of the hair region Rh is extracted by approximating the elliptical arc. Whether the color of the background region and the hair region Rh is similar is determined by, for example, the hue of a pixel or a pixel group at a predetermined position having a high probability of being in the hair region Rh with reference to the contour position of the designated region Ru. What is necessary is to compare with the hue of a pixel or a pixel group at a predetermined position where the probability of being in the background region is high, and determine whether or not the difference is within a certain threshold.

  When the skin region Rs and the hair region Rh are extracted in this way, the region surrounding them is set as the correction target region Rm.

(Create mask)
When the correction target region Rm is extracted, the correction target area Rm is corrected when the entire corrected image Mod in which the entire image of the human image In is corrected and the original human image In are added together with a predetermined weight for each corresponding pixel. A mask image Mask representing the weight is generated so that an image in which only the region Rm is corrected is obtained.

  The mask image Mask has the same image size (resolution) as the person image In, and each pixel has 0 to 1 (when the degree of correction is 0%) depending on the degree of correction applied to the pixel. And a pixel value corresponding to a weight of 1 when 0 or 100%. In this embodiment, the pixel values corresponding to the weights 0 to 1 are assumed to be 0 to 255.

  Here, the process of blurring the mask is performed so that the corrected image In ′ partially corrected, which is finally obtained by combining the person image In, the whole correction image Mod, and the mask image Mask, is not unnatural. Apply. That is, the pixel value of the mask image Mask is set so that the degree of correction decreases from the inside to the outside of the correction target region Rm in the vicinity of the boundary of the correction target region Rm in the mask image Mask. Here, in order to blur the mask, Gaussian blur processing is applied to the mask image Mask. On the head side of the mask, the width to be blurred is defined based on the relationship with the selected rectangular designated area Ru (for example, the horizontal width is W / 4 and the vertical width is H / 4). ). On the other skin side, first, the boundary of the skin in the mask is extracted, and the width to blur is defined based on the relationship between the horizontal width and the vertical width of the position of the boundary (the horizontal width is W / 4, and the vertical width is H / 4).

  In order to further suppress the adverse effect when the background is dark, the value in the mask is slowly changed in the outline of the hair region Rh. That is, the value in the mask is changed from 0 to 255 instead of the binary value of 0 or 255. When an elliptical arc inscribed in the contour of the designated region Ru is selected as an approximation of the upper contour Lhu of the hair region, as shown in FIG. 14, the mask is 0 to 255 from the outside to the inside. Change to take the value of.

  It should be noted that at the stage of studying the mask, only changing the color information of the selected area was considered, but it was not considered to affect other areas, but the color correlation (inside and outside of the mask boundary ( Here, the color correlation represents a correlation between color components and lightness in an image to be compared. For example, lightness difference, red (R) component difference, green (G) component difference, and blue (B) If the sum of the component differences, the difference in hue, the difference in saturation, and the sum of the brightness differences are large), the same blurring condition may affect the areas where improvement by correction is not desired. It tended to be an unnatural description. Therefore, adaptive blurring processing, that is, processing for changing the degree of blurring according to the color correlation inside and outside the mask boundary, is performed. That is, the degree of correction is set so as to change more gently in the vicinity of the boundary of the correction target region Rm as the color correlation inside and outside the correction target region Rm decreases (for example, the radius of the Gaussian function is increased).

  As described above, the reason why the correction degree in the vicinity of the boundary of the correction target region Rm is set to change more gradually as the color correlation is smaller is based on the following facts. In general, when the brightness difference between the inside and outside of the correction target region Rm is large, gradation is easily noticeable between the corrected correction target region Rm and the outside thereof (the expression is corrected), so the correction target region In the vicinity of the boundary, it is preferable that the correction degree changes steeply, and when the brightness difference between the inside and outside of the correction target region Rm is small, the extraction accuracy of the boundary of the correction target region Rm may be poor. It is preferable that the degree of correction is gradually changed in the vicinity of the boundary of the correction target region Rm because bad extraction is likely to be noticeable (the boundary is shifted or uneven). However, even if the lightness difference is small, if the hue is completely different, etc., if the change in the correction degree is moderated according to the above rules, the color gradation may be noticeable. In such a case, it is preferable to consider whether the hues are close by using the sum of the differences between the components of the respective colors. That is, when the hues inside and outside the correction target area are close, it is preferable that the correction degree gradually changes in the vicinity of the boundary of the correction target area, and when the hue is far away, It is preferable that the correction degree changes steeply in the vicinity of.

In the blurring process with the same blurring condition, a Gaussian kernel is used to smooth the boundary of the mask. Here, the mask is Mask (i, j), and the one-dimensional Gaussian kernel is G = [g−n, g− (n−1),..., Gn−1, gn, with n pixels as the radius. ]. Mask blurring can be applied in both the horizontal and vertical directions. One-dimensional blurring can be expressed as:

On the other hand, in the adaptive blur processing, blur weights are introduced, and the formula of the blur processing can be rewritten as follows, for example.

Here, th is a parameter for adjustment. The greater the th, the lightness difference: abs (I (i, j) -I (ik) j) or abs (I (i, j) -I ( i, j−k)) weakens the effect of blurring. The value of th can be set to 80, for example.

  According to the image correction apparatus according to the first embodiment as described above, the type of hair color to be extracted is obtained in advance by a predetermined means, and the color information in the region designated by the operator or in the vicinity thereof is obtained. And extracting a human face area from the designated area or the vicinity thereof, and extracting a hair area according to a predetermined algorithm corresponding to the type of hair color obtained in advance, and the skin area Rs including the face and the hair Since the color of the correction target region Rm composed of the region Rh is corrected, the hair region Rh can be accurately extracted not only for black hair but also for other types of hair. Even if the type is other than black hair, it is possible to reliably extract the correction target region Rm composed of the skin region Rs and the hair region Rh and correct the face color appropriately and efficiently without causing unnaturalness. It becomes ability.

  Next, a second embodiment of the image correction apparatus of the present invention will be described. The configuration of the image correction apparatus according to the second embodiment and the function of each unit are the same as those of the first embodiment except for the function of the hair color type information acquisition unit 33.

  In the first embodiment, the hair type information acquisition unit 33 acquires the hair color type information Vh according to the input from the operator. That is, the operator looks at the person image In displayed on the screen, determines the hair color type of a single person to be corrected, and designates the type. The hair color type information acquisition unit 33 acquires information representing the specified type as the hair color type information Vh.

  On the other hand, in the second embodiment, the hair color type information acquisition unit 33 specifies a partial region Rb on the hair of a single person to be corrected in accordance with an input from the operator, and the partial region Rb The type of hair color of the single person is determined based on the color information, and information representing the determined type is generated as the hair color type information Vh. That is, as shown in FIG. 15, the operator designates only the region where the hair of the single person exists, and the color type of the hair is based on the color information of the region designated by the operator. The image correction apparatus side makes the determination.

  The determination of the type of hair color can be performed, for example, by the following process.

  FIG. 16 is a flowchart showing the flow of processing for determining the type of hair color. First, regarding the partial region Rb on the hair of a single person to be corrected in the person image In, the average value I0, the average value H0 of the hue value, and the average of the saturation values of the pixels constituting the image of this region Each value S0 is calculated (step S21). Then, it is determined whether or not the average value S0 of the saturation values is smaller than a predetermined threshold value S_lim_bkwht (step S22). That is, it is determined whether the hair of the single person belongs to a highly saturated, blonde / brown hair system, a colored hair system, a low saturation system, a black hair system, or a white hair system. Here, if it is determined that the average value S0 of the saturation values is smaller than the threshold value S_lim_bkwht, it is determined that the hair of the single person has a low saturation, black-haired or gray-haired system, and further, brightness It is determined whether or not the average value I0 is smaller than a predetermined threshold value I_lim_wht (step S23). On the other hand, if it is determined in step S22 that the average value S0 of the saturation values is equal to or greater than the threshold value S_lim_bkwht, the hair of the single person has high saturation, blonde / brown hair or colored hair. Further, it is determined that the system is a system, and it is further determined whether or not the average value H0 of the hue values is within a predetermined range from Hlim0 to Hlim1 corresponding to a hue of blonde hair or brown hair (step S24). If it is determined in step S23 that the average value I0 of the lightness value is smaller than the threshold value I_lim_wht, the hair is determined to be a black hair system with low lightness (step S25), and the average value I0 of the lightness value is the threshold value I_lim_wht. If it is determined that the above is true, the hair is determined to be a white hair system with high brightness (step S26). If it is determined in step S24 that the average hue value H0 is within this predetermined range, the hair is determined to be blond / brown (step S27), and the average hue value H0 is determined. Is determined to be outside this predetermined range, the hair is determined to be another colored hair system (step S28).

  After the hair color type information acquisition unit 33 determines the type of the hair color and acquires the hair color type information Vh representing the type of the hair color, the hair region extraction is performed as in the first embodiment. The hair region Rh is extracted by the unit 34 in accordance with a predetermined algorithm corresponding to the type of color of the hair, and the correction target region Rm surrounding the skin region Rs extracted by the skin region extracting unit 32 and the hair region Rh. The color and brightness are corrected.

  According to such an image correction apparatus according to the second embodiment, the following effects can be recognized in addition to the effects described in the first embodiment. That is, according to this image correction apparatus, the color type of the hair is specified based on the color information of the partial region Rb on the hair of a single person to be corrected on the person image In specified by the operator. Since the determination is made, only by a simple operation by the operator of designating the partial region Rb, the hair color information can be surely obtained and the hair color type can be accurately determined, The hair region Rh can be extracted with high accuracy, and as a result, the correction target region Rm can be extracted with high accuracy, and the natural correction corresponding to various hairs with different colors even though the burden on the operator is light. Is possible.

  Next, a third embodiment of the image correction apparatus of the present invention will be described. The configuration of the image correction apparatus according to the third embodiment and the function of each unit are the same as those of the first embodiment except for the function of the hair color type information acquisition unit 33.

  In the first embodiment, the hair type information acquisition unit 33 acquires the hair color type information Vh according to the input from the operator. That is, the operator looks at the person image In displayed on the screen, determines the hair color type of a single person to be corrected, and designates the type. The hair color type information acquisition unit 33 acquires information representing the specified type as the hair color type information Vh.

  On the other hand, in the third embodiment, the hair color type information is assumed on the assumption that the designated region Ru designated by the region designating unit 31 is a region whose outline is a rectangle that substantially circumscribes the face and hair of a single person. As shown in FIG. 17, the acquisition unit 33 approximates the upper contour Lhu of the hair region of the single person to an elliptical arc inscribed in the contour of the designated region Ru, based on the contour position information of the designated region Ru. Of the region excluding the skin region Rs extracted by the skin region extraction unit 31 from the designated region Ru and the region surrounded by the contour of the designated region Ru and the upper contour Lhu of the hair region. The region in the rectangular region that is the upper third of the region Ru is extracted as the temporary hair region Rht, and the type of hair color of the single person is determined based on the color information of the temporary hair region Rht. Information indicating the type of hair color And generates a broadcast Vh. That is, the operator only designates an area that substantially circumscribes the face and hair of a single person to be corrected on the person image In, and the type of color of the hair is the color information of the designated area Ru. Based on the contour position information, all of the image correction apparatus side automatically determines.

  The determination of the type of hair color can basically use substantially the same determination process as the determination of the type of hair color in the second embodiment. In this case, the partial region Rb on the hair may be replaced with the temporary hair region Rht.

  In addition to extracting the temporary hair region Rht by the above-described procedure, the type of hair color of the single person is simply determined based on the most colors in the upper third region of the designated region Ru. You may make it determine.

  According to the image correction apparatus according to the third embodiment, in addition to the effects described in the first embodiment, the following effects are also recognized. That is, according to this image correction apparatus, a rectangular region that is substantially circumscribed by the face and hair of a single person to be corrected on the human image In, designated by the operator, is set as the designated region Ru. Based on the color information and the contour position information, an area that is considered to correspond to the hair area is extracted as the temporary hair area Rht, and the type of the hair color is determined based on the color information of the temporary hair area Rht. Therefore, only by a basic operation by the operator of designating the designated region Ru, the hair color information is accurately obtained to accurately determine the type of the hair color, and the hair region Rh. As a result, the correction target area Rm can be extracted with high accuracy, and natural correction corresponding to various hairs with different colors is possible despite almost no burden on the operator. It becomes.

  Note that the image correction apparatus according to the first to third embodiments has the following effects. That is, a so-called semi-automatic (semi-automatic) correction target region Rm is automatically extracted from the region specified by the operator, including the human face region, so that the degree of correction is determined. Since the correction is performed gently in the vicinity of the boundary of Rm, the correction target area Rm including the face area of the person can be accurately and efficiently extracted even for a human image whose background is complicated and difficult to detect automatically. In addition, it is possible to perform correction while blurring the boundary between the correction target region Rm and other regions, and to extract a face reliably even in a general natural picture, so that the brightness and color of the face are not unnatural. It becomes possible to correct appropriately and efficiently.

  Also, by including not only the skin region Rs including the face but also the hair region Rh in the correction target region Rm, an unnatural brightness difference or hue difference between the hair description and the face description appears in the corrected image. It becomes difficult and more natural correction is possible.

  In addition, the correction unit configured by the image correction unit and the image synthesis unit is configured to first correct the target after correction of the correction target region for a preset type of component related to at least one of hue, saturation, and brightness. The amount of correction is determined based on the difference between the average brightness and the average value or the intermediate value of the actual brightness, and then the correction is performed by fine adjustment according to the input from the operator. The operator can obtain a desired correction result with a simple operation, and can efficiently correct a large number of images.

  In these embodiments, only one correction target region Rm is extracted from one person image In and correction is performed on the correction target region. Rather, a plurality of correction target areas Rm1, Rm2,... Are extracted from one person image In, and a plurality of mask images Mask1, Mask2,. It is also possible to create correction images Mod1, Mod2,... And perform different corrections on each correction target area.

  In these embodiments, the correction target region Rm is extracted as a region surrounding the skin region Rs including the face and the hair region Rh. However, the hair region Rh is not particularly limited to this. Only the skin region Rs including the face not including the face may be extracted as the correction target region Rm.

  The image correction method and apparatus according to the embodiment of the present invention have been described above, but a program for causing a computer to execute each process in the image correction apparatus is also an embodiment of the present invention. A computer-readable recording medium that records such a program is also one embodiment of the present invention.

The block diagram which shows the structure of the image correction apparatus 1 The flowchart which shows the flow of the process performed in the image correction apparatus 1. The figure which shows an example of the image before and after correction | amendment, and the image used for correction | amendment The figure which shows an example of the layout of the image displayed on the screen of the image display part 70 A flowchart showing a flow of processing in extraction of a correction target region Rm and creation of a mask image Diagram showing the distribution of hue, saturation and lightness in the skin area A diagram showing the relationship between the designated region Ru and the trust region Rt The figure for demonstrating the predetermined area on the hue histogram used for extraction of a skin area | region The figure for demonstrating the predetermined various area on the saturation histogram used for extraction of a skin area | region Illustration for explaining the growth process of skin seeds The figure for demonstrating the predetermined area on the histogram used for the growth process of a skin seed The figure which shows the relationship between the hair region and the background region and the brightness histogram The figure for demonstrating the extraction method of the lower outline of a face area | region, and the upper outline of a hair area | region The figure which shows the mask where the vicinity of the boundary of the hair part is blurred The figure which shows a mode that the partial area | region on the hair of the single person who is correction object is specified Flowchart of processing for determining the type of hair color of a single person to be corrected The figure which shows a mode that a temporary hair area | region is extracted

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Image correction apparatus 10 Image data storage part 20 Image selection part 31 Area designation part 32 Skin area extraction part 33 Hair color classification information acquisition part 34 Hair area extraction part 35 Correction target area setting part 36 Mask creation part 40 Image correction part 50 Image Composition unit 60 Display control unit 70 Image display unit 80 Input operation unit

Claims (21)

An area designating step for designating an area including a face of a single person in a human image with a person as a subject in response to an input from an operator;
A skin region extraction step for extracting a skin region including the face of the single person based on color information of the designated region;
A hair color type information acquisition step of acquiring hair color type information representing the type of hair color of the single person;
A hair region extracting step of extracting the hair region of the single person according to a predetermined algorithm according to the type of hair color represented by the hair color type information;
A correction target region setting step for setting a region consisting of the skin region and the hair region as a correction target region;
And a correction step of correcting at least one of hue, saturation, and brightness in the correction target region.   The image correction method according to claim 1, wherein the hair color type information acquisition step is a step of acquiring, as the hair color type information, information representing a type of hair color input from the operator. . The hair color type information acquisition step includes
Specify the partial area on the hair of the single person according to the input from the operator,
Determining the type of hair color of the single person based on the color information of the partial area;
The image correction method according to claim 1, wherein information representing the determined type is generated as the hair color type information. The designated area is an area having a contour that is substantially circumscribed by the face and hair of the single person,
The hair color type information acquisition step includes
Based on the contour position information of the designated area, the upper outline of the hair area of the single person is extracted by approximating an elliptical arc inscribed in the outline of the designated area,
Of the area excluding the skin area and the area surrounded by the outline of the designated area and the upper outline of the hair area from the designated area, a rectangular area that is 1/2 to 1/4 above the designated area The area inside is extracted as a temporary hair area,
Determining the type of hair color of the single person based on the color information of the provisional hair region;
The image correction method according to claim 1, wherein information representing the determined type is generated as the hair color type information.   In the hair region extraction step, when the type represented by the hair color type information is black hair, a histogram of brightness values of pixels constituting the image of the specified region is obtained, and the brightness value is smaller than a predetermined threshold value on the histogram. 5. The image correction method according to claim 1, wherein an image region including pixels corresponding to the region is extracted as the hair region. 6.   In the hair region extraction step, when the type represented by the hair color type information is gray hair, a histogram of lightness values of pixels constituting the image of the specified region is obtained, and the lightness value is larger than a predetermined threshold value on the histogram. 5. The image correction method according to claim 1, wherein an image region including pixels corresponding to the region is extracted as the hair region. 6.   The hair region extracting step obtains a histogram of hue values of pixels constituting the image of the designated region when the type represented by the hair color type information is a predetermined type of blond or brown hair, and the hue value is The step of extracting, as the hair region, an image region composed of pixels corresponding to the largest area among the peaks on the histogram that are within a predetermined range including an average hue value of the predetermined type of hair. 5. An image correction method according to claim 1, wherein An area designating unit for designating an area including a face of a single person in a human image having a person as a subject in response to an input from an operator;
A skin area extracting means for extracting a skin area including the face of the single person based on color information of the designated area;
Hair color type information acquisition means for acquiring hair color type information representing the type of hair color of the single person;
Hair region extracting means for extracting the hair region of the single person according to a predetermined algorithm corresponding to the type of hair color represented by the hair color type information;
Correction target region setting means for setting a region consisting of the skin region and the hair region as a correction target region;
An image correction apparatus comprising: correction means for correcting at least one of hue, saturation, and brightness in the correction target region.   9. The image correction apparatus according to claim 8, wherein the hair color type information acquisition unit acquires information indicating the color type of the hair input from the operator as the hair color type information. . The hair color type information acquisition means
Specify the partial area on the hair of the single person according to the input from the operator,
Determining the type of hair color of the single person based on the color information of the partial area;
9. The image correction apparatus according to claim 8, wherein information representing the determined type is generated as the hair color type information. The designated area is an area having a contour that is substantially circumscribed by the face and hair of the single person,
The hair color type information acquisition means
Based on the contour position information of the designated area, the upper outline of the hair area of the single person is extracted by approximating an elliptical arc inscribed in the outline of the designated area,
Of the area excluding the skin area and the area surrounded by the outline of the designated area and the upper outline of the hair area from the designated area, a rectangular area that is 1/2 to 1/4 above the designated area The area inside is extracted as a temporary hair area,
Determining the type of hair color of the single person based on the color information of the provisional hair region;
9. The image correction apparatus according to claim 8, wherein information representing the determined type is generated as the hair color type information.   The hair region extraction means obtains a histogram of lightness values of pixels constituting the image of the designated region when the type represented by the hair color type information is black hair, and the lightness value is smaller than a predetermined threshold value on the histogram The image correction apparatus according to claim 8, wherein an image region including pixels corresponding to the region is extracted as the hair region.   The hair region extraction means obtains a histogram of lightness values of pixels constituting the image of the designated region when the type represented by the hair color type information is white hair, and the lightness value is larger than a predetermined threshold value on the histogram The image correction apparatus according to claim 8, wherein an image region including pixels corresponding to the region is extracted as the hair region.   The hair region extracting means obtains a histogram of hue values of pixels constituting the image of the designated region when the type represented by the hair color type information is a predetermined type of blond or brown hair, and the hue value is An image region composed of pixels corresponding to the largest area among the peaks on the histogram within a predetermined range including an average hue value of the predetermined type of hair is extracted as the hair region. The image correction apparatus according to claim 8, wherein the image correction apparatus is an image correction apparatus. Computer
An area designating unit for designating an area including a face of a single person in a human image having a person as a subject in response to an input from an operator;
A skin area extracting means for extracting a skin area including the face of the single person based on color information of the designated area;
Hair color type information acquisition means for acquiring hair color type information representing the type of hair color of the single person;
Hair region extracting means for extracting the hair region of the single person according to a predetermined algorithm corresponding to the type of hair color represented by the hair color type information;
Correction target region setting means for setting a region consisting of the skin region and the hair region as a correction target region;
A program for functioning as correction means for correcting at least one of hue, saturation, and brightness in the correction target area.   The program according to claim 15, wherein the hair color type information acquisition unit acquires information representing a color type of the hair input from the operator as the hair color type information. The hair color type information acquisition means
Specify the partial area on the hair of the single person according to the input from the operator,
Determining the type of hair color of the single person based on the color information of the partial area;
16. The program according to claim 15, wherein information representing the determined type is generated as the hair color type information. The designated area is an area having a contour that is substantially circumscribed by the face and hair of the single person,
The hair color type information acquisition means
Based on the contour position information of the designated area, the upper outline of the hair area of the single person is extracted by approximating an elliptical arc inscribed in the outline of the designated area,
Of the area excluding the skin area and the area surrounded by the outline of the designated area and the upper outline of the hair area from the designated area, a rectangular area that is 1/2 to 1/4 above the designated area The area inside is extracted as a temporary hair area,
Determining the type of hair color of the single person based on the color information of the provisional hair region;
16. The program according to claim 15, wherein information representing the determined type is generated as the hair color type information.   The hair region extraction means obtains a histogram of lightness values of pixels constituting the image of the designated region when the type represented by the hair color type information is black hair, and the lightness value is smaller than a predetermined threshold value on the histogram The program according to any one of claims 15 to 18, wherein an image area including pixels corresponding to the area is extracted as the hair area.   The hair region extraction means obtains a histogram of lightness values of pixels constituting the image of the designated region when the type represented by the hair color type information is white hair, and the lightness value is larger than a predetermined threshold value on the histogram The program according to any one of claims 15 to 18, wherein an image area including pixels corresponding to the area is extracted as the hair area.   The hair region extracting means obtains a histogram of hue values of pixels constituting the image of the designated region when the type represented by the hair color type information is a predetermined type of blond or brown hair, and the hue value is An image region composed of pixels corresponding to the largest area among the peaks on the histogram within a predetermined range including an average hue value of the predetermined type of hair is extracted as the hair region. The program according to any one of claims 15 to 18, wherein said program is characterized in that: