JP4946729B2 - Image processing device - Google Patents

Description

  The present invention relates to an image processing technique for setting a region including a face image in a target image.

  A technique for detecting an area representing a face image in a target image (hereinafter also referred to as “face area”) by a predetermined method such as pattern matching using a template is known (for example, Patent Document 1).

JP 2006-279460 A

  The conventional face area detection technique does not necessarily detect the face area with high accuracy. For example, only a part of the face image may be detected as the face area. In particular, detection accuracy tends to decrease for faces in directions other than the front.

  The present invention has been made in order to solve the above-described problems, and an object of the present invention is to provide a technique capable of setting a highly accurate face area.

  In order to solve at least a part of the above problems, the present invention can be realized as the following forms or application examples.

Application Example 1 An image processing apparatus,
A first face area setting unit for setting a first face area that is an area including a face image in the target image;
A skin color value setting unit that sets a range of skin color values based on pixel values of pixels in the first face area;
A skin color pixel detection unit that detects a skin color pixel that is a pixel having a pixel value included in the range of the skin color value in an area including pixels around the first face area;
Based on the detection result of the skin color pixel, a second face region that sets a second face region that is a region that includes a face image in the target image and that includes at least a part of the detected skin color pixel. An image processing apparatus comprising: a setting unit;

  In this image processing apparatus, a skin color pixel is detected in an area including pixels around the first face area, and an area including at least a part of the detected skin color pixel is set as the second face area. Therefore, the second face area, which is an area with higher accuracy than the first face area, can be set as the area including the face image. Further, in this image processing apparatus, since the skin color value range is set based on the pixel values of the pixels in the first face area, the second face area is set regardless of the skin color difference for each person. be able to.

Application Example 2 The image processing apparatus according to Application Example 1, further comprising:
An image processing apparatus comprising: a determination unit that determines whether or not to execute predetermined image processing on the target image based on the first face area and the second face area.

  It is considered that the relationship between the first face area and the second face area changes according to the orientation of the face image. Therefore, in this image processing apparatus, it is possible to determine whether or not to execute predetermined image processing on the target image according to the orientation of the face image.

[Application Example 3] The image processing apparatus according to Application Example 2,
The determination unit is based on at least one of a positional relationship between the first face region and the second face region and a relationship regarding the size of the first face region and the second face region. An image processing apparatus that performs the determination.

  In this image processing apparatus, a face is determined based on at least one of a positional relationship between the first face region and the second face region and a relationship regarding the size of the first face region and the second face region. It is possible to determine whether or not to perform predetermined image processing on the target image according to the orientation of the face image.

Application Example 4 The image processing apparatus according to Application Example 2 or Application Example 3,
The image processing apparatus, wherein the predetermined image processing is deformation processing of an image in an area including the first face area.

  In this image processing apparatus, it is determined whether or not processing can be performed according to the orientation of the face image with respect to the deformation processing of the image in the area including the face area that may cause an unnatural result depending on the orientation of the face image. It is possible to avoid the execution of deformation processing that causes unnatural results.

Application Example 5 The image processing apparatus according to any one of Application Examples 2 to 4, further comprising:
An image processing apparatus comprising: a process execution unit that executes the predetermined image processing when the determination unit determines that execution is possible.

  In this image processing apparatus, when the determination unit determines that execution is possible, predetermined image processing can be executed.

Application Example 6 The image processing apparatus according to Application Example 1, further including:
An image processing apparatus comprising: a process execution unit that executes predetermined image processing on an image of the second face area.

  In this image processing apparatus, it is possible to execute predetermined image processing on the image of the second face area, which is an area with higher accuracy than the first face area, as the area including the face image.

[Application Example 7] The image processing apparatus according to any one of Application Example 1 to Application Example 6,
The image processing apparatus, wherein the first face area setting unit sets the first face area based on at least a position of an image of a facial organ.

  In this image processing apparatus, based on the first face area set based on the position of the image of the facial organ, the second area, which is a higher-accuracy area than the first face area, includes the face image. A face area can be set.

  Note that the present invention can be realized in various modes. For example, an image processing method and apparatus, an image deformation method and apparatus, an image generation method and apparatus, a printing method and apparatus, and functions of these methods or apparatuses. Can be realized in the form of a computer program for realizing the above, a recording medium storing the computer program, a data signal including the computer program and embodied in a carrier wave, and the like.

Next, embodiments of the present invention will be described in the following order based on examples.
A. Example:
A-1. Configuration of image processing device:
A-2. Face shape correction printing process:
A-3. Transformation process:
B. Variations:

A. Example:
A-1. Configuration of image processing device:
FIG. 1 is an explanatory diagram schematically showing the configuration of a printer 100 as an image processing apparatus according to an embodiment of the present invention. The printer 100 of this embodiment is a color inkjet printer that supports so-called direct printing, in which an image is printed based on image data acquired from a memory card MC or the like. The printer 100 includes a CPU 110 that controls each unit of the printer 100, an internal memory 120 configured by, for example, a ROM and a RAM, an operation unit 140 configured by buttons and a touch panel, a display unit 150 configured by a liquid crystal display, A printer engine 160 and a card interface (card I / F) 170 are provided. The printer 100 may further include an interface for performing data communication with other devices (for example, a digital still camera or a personal computer). Each component of the printer 100 is connected to each other via a bus.

  The printer engine 160 is a printing mechanism that performs printing based on print data. The card interface 170 is an interface for exchanging data with the memory card MC inserted into the card slot 172. In this embodiment, an image file including image data as RGB data is stored in the memory card MC. This image file is a file generated in accordance with the Exif (Exchangeable Image File Format) standard by an imaging device such as a digital still camera. For example, in addition to the image data generated by imaging, the aperture / shutter speed at the time of imaging Includes additional data such as lens focal length. The printer 100 acquires an image file stored in the memory card MC via the card interface 170.

  The internal memory 120 stores a face shape correction unit 200, a display processing unit 310, and a print processing unit 320. The face shape correction unit 200 is a computer program for executing a face shape correction process described later under a predetermined operating system. The display processing unit 310 is a display driver that controls the display unit 150 to display a processing menu, a message, and the like on the display unit 150. The print processing unit 320 is a computer program for generating print data from image data, controlling the printer engine 160, and printing an image based on the print data. The CPU 110 implements the functions of these units by reading and executing these programs from the internal memory 120.

  The face shape correction unit 200 includes, as program modules, a deformation mode setting unit 210, a first face region setting unit 220, a second face region setting unit 230, a determination unit 238, a deformation region setting unit 240, A deformation area dividing unit 250 and a divided area deformation unit 260 are included. The second face area setting unit 230 includes a skin color value setting unit 232 and a skin color pixel detection unit 234. The functions of these units will be described in detail in the description of the face shape correction printing process described later. As will be described later, the deformation region setting unit 240, the deformation region dividing unit 250, and the divided region deforming unit 260 deform the image. Therefore, the deformation area setting unit 240, the deformation area dividing unit 250, and the divided area deformation unit 260 can be collectively referred to as a “deformation processing execution unit”.

  The internal memory 120 also stores a dividing point arrangement pattern table 410 and a dividing point movement table 420. These contents will also be described in detail in the description of the face shape correction printing process described later.

A-2. Face shape correction printing process:
The printer 100 prints an image based on the image file stored in the memory card MC. When the memory card MC is inserted into the card slot 172, the display processing unit 310 displays a user interface including a list display of images stored in the memory card MC on the display unit 150. FIG. 2 is an explanatory diagram illustrating an example of a user interface including a list display of images. In the user interface shown in FIG. 2, eight thumbnail images TN1 to TN8 and five buttons BN1 to BN5 are displayed. In this embodiment, the list display of images is realized by using thumbnail images included in the image file stored in the memory card MC.

  The printer 100 according to the present embodiment normally prints a selected image as usual when one (or a plurality) of images is selected and a normal print button BN3 is selected by the user in the user interface shown in FIG. Execute print processing. On the other hand, in the user interface, when one (or a plurality) of images is selected by the user and the face shape correction print button BN4 is selected, the printer 100 selects the face image in the selected image for the selected image. The face shape correction printing process for correcting the shape of the image and printing the corrected image is executed. In the example of FIG. 2, since the thumbnail image TN1 and the face shape correction print button BN4 are selected, the printer 100 performs a face shape correction print process on the image corresponding to the thumbnail image TN1.

  FIG. 3 is a flowchart illustrating a flow of face shape correction printing processing by the printer 100 according to the present exemplary embodiment. In step S100, the face shape correction unit 200 (FIG. 1) executes face shape correction processing. The face shape correction process according to the present embodiment is a process for correcting at least a part of a face in the image (for example, a face outline shape or an eye shape). Note that a part of the face such as eyes, eyebrows, nose and mouth is generally called an organ.

  FIG. 4 is a flowchart showing the flow of the face shape correction process (step S100 in FIG. 3) in the present embodiment. In step S110, the face shape correction unit 200 (FIG. 1) sets a target image TI that is a target of face shape correction processing. The face shape correction unit 200 sets an image corresponding to the thumbnail image TN1 selected by the user in the user interface shown in FIG. 2 as the target image TI. The set image file of the target image TI is acquired by the printer 100 from the memory card MC via the card interface 170 and stored in a predetermined area of the internal memory 120.

  In step S120 (FIG. 4), the deformation mode setting unit 210 (FIG. 1) sets an image deformation mode (type and degree of deformation) for face shape correction. The deformation mode setting unit 210 instructs the display processing unit 310 to display a user interface for setting the type and degree of image deformation on the display unit 150, and the image deformation type designated by the user through the user interface. And the degree are set as the image deformation type and degree used for processing.

  FIG. 5 is an explanatory diagram showing an example of a user interface for setting the type and degree of image deformation. As shown in FIG. 5, this user interface includes an interface for setting an image deformation type. In this embodiment, for example, a deformation type “type A” that sharpens the shape of the face, a deformation type “type B” that increases the shape of the eyes, and the like are set in advance as options. The user specifies the type of image deformation via this interface. The deformation mode setting unit 210 sets the image deformation type designated by the user as the image deformation type used for actual processing.

  The user interface shown in FIG. 5 includes an interface for setting the degree (degree) of image deformation. As shown in FIG. 5, in this embodiment, it is assumed that three levels of options of strong (S), medium (M), and weak (W) are set in advance as the degree of image deformation. The user designates the degree of image deformation via this interface. The deformation mode setting unit 210 sets the designated degree of image deformation as the degree of image deformation used for actual processing.

  In the following description, it is assumed that the deformation type “type A” for sharpening the face shape is set as the image deformation type, and “medium (M)” is selected as the image deformation degree (deformation amount). I do.

  In step S130 (FIG. 4), the first face area setting unit 220 (FIG. 1) sets the first face area FA1 in the target image TI. Here, the first face area FA1 is an image area in the target image TI and includes at least a partial image of the face. The first face area setting unit 220 analyzes the target image TI, detects an area assumed to include a face image, and sets the area as the first face area FA1. The detection of an area assumed to include a face image by the first face area setting unit 220 is performed using a known detection method such as a pattern matching method using a template (see Japanese Patent Application Laid-Open No. 2006-279460). The In general, such a pattern matching method using a template is considered to include a face image from the target image TI based on the position of an image of a facial organ (eyes, eyebrows, nose, mouth, etc.). The detected area is detected.

  FIG. 6 is an explanatory diagram illustrating an example of a setting result of the first face area FA1. In the example of FIG. 6, the target image TI includes images of the faces of two people, a person P1 and a person P2. Therefore, in step S130, the first face area FA1 is set for each of the person P1 and the person P2. As shown in FIG. 6, the first face area FA1 is a rectangular area including images of both eyes, nose and mouth. Note that the first face area setting unit 220 specifies the set first face area FA1 by, for example, the coordinates of the four vertices of the first face area FA1.

  In step S130, if an area that is assumed to include a face image is not detected, the fact is notified to the user through the display unit 150. In this case, normal printing without face shape correction may be performed, or re-detection processing using another detection method may be performed.

  In step S140 (FIG. 4), the second face area setting unit 230 (FIG. 1) sets the second face area FA2 in the target image TI. Similar to the first face area FA1, the second face area FA2 is an image area in the target image TI and includes at least a partial image of the face. The second face area FA2 is set for each first face area FA1 set in step S130 of FIG. That is, in the present embodiment, since two first face areas FA1 are set (see FIG. 6), two second face areas FA2 corresponding to the two first face areas FA1 are set.

  FIG. 7 is a flowchart showing the flow of the second face area setting process (step S140 in FIG. 4) in the present embodiment. In step S142, the skin color value setting unit 232 (FIG. 1) sets a range of skin color values. The range of the skin color value is a range of pixel values of pixels that are assumed to represent skin in the target image TI. In this embodiment, since the image data representing the target image TI is RGB data, the skin color value range is set as a combination range of R value, G value, and B value. The setting of the skin color value range is executed for each first face area FA1 set in step S130 of FIG.

  FIG. 8 is an explanatory diagram showing a method for setting a range of skin color values. It is considered that the pixel value having high frequency in the first face area FA1 set by the first face area setting unit 220 in step S130 (FIG. 4) is the skin color of the person. Therefore, as shown in FIG. 8, the skin color value setting unit 232 sets the R value, G value, and B value of each pixel included in the first face area FA1 set by the first face area setting unit 220 in step S130. A histogram is created, and a pixel value range in which the frequency is equal to or higher than a predetermined threshold (Tfr, Tfg, Tfb in FIG. 8) is set as a skin color value range (Sr, Sg, Sb in FIG. 8). In this way, a range of skin color values is set for each person image included in the target image TI.

  In step S144 (FIG. 7), the second face area setting unit 230 (FIG. 1) sets the search area SA. FIG. 9 is an explanatory diagram illustrating an example of a setting result of the search area SA. In the present embodiment, the search area SA is set as an area including the first face area FA1 and the area around the first face area FA1. Specifically, an area obtained by enlarging the first face area FA1 in the vertical direction and the horizontal direction at a magnification of, for example, 2 is set as the search area SA. The search area SA is preferably set as an area that includes an image of almost the entire head, and the magnification is set in advance so that the search area SA is set as such.

  In step S146 (FIG. 7), the flesh color pixel detection unit 234 (FIG. 1) detects flesh color pixels from the search area SA. The skin color pixel is a pixel having a pixel value (combination of R value, G value, and B value) included in the range of the skin color value set in step S142. The skin color pixel detection unit 234 sequentially examines the pixel values of the pixels included in the search area SA, and detects pixels whose pixel values are included in the skin color value range as skin color pixels.

  In step S148 (FIG. 7), the second face area setting unit 230 (FIG. 1) sets the second face area FA2. FIG. 10 is an explanatory diagram illustrating an example of a setting result of the second face area FA2. The second face area setting unit 230 sets a rectangular area that includes all the skin color pixels detected in step S146 as the second face area FA2. Note that the second face area FA2 is set for each person image in the same manner as the first face area FA1.

  As described above, when the second face area FA2 is set, the second face area FA2 is a rectangular area that substantially includes an image of the skin color portion of the person's head. Therefore, in this embodiment, it is possible to set the second face area FA2 with higher accuracy than the first face area FA1 as the area including the entire face image. In the present embodiment, a skin color value range is set based on the pixel values of the pixels in the first face area FA1, and pixels having pixel values included in the skin color value range are detected as skin color pixels. Therefore, it is possible to set the second face area FA2 with high accuracy regardless of the difference in the skin color of the person.

  Here, as shown in FIG. 10, for the face image of a substantially front-facing person (P <b> 1 in FIG. 10), the second face area FA <b> 2 is approximately equal to the first face area FA <b> 1 vertically and horizontally. It becomes an enlarged area. On the other hand, for the face image of a person (P2 in FIG. 10) whose orientation deviation from the front direction (hereinafter also referred to as “swing”) is large to some extent, the second face area FA2 moves up and down the first face area FA1. It is not an area that is enlarged substantially equally to the left and right, but an area that is greatly enlarged in the direction in which the first face area FA1 is biased. For example, in the example shown in FIG. 10, since the face image of the person P2 has a large rightward swing, the second face area FA2 includes the first face profile image of the person P2. This is an area greatly enlarged in the left direction toward the face area FA1.

  In step S150 (FIG. 4), the determination unit 238 (FIG. 1) determines whether or not the deformation process for correcting the face shape (step S160 in FIG. 4) can be performed. FIG. 11 is an explanatory diagram illustrating a method for determining whether or not to execute the deformation process. As will be described later, the deformation process according to the present embodiment is premised on an image of a person's face almost facing the front, and when applied to an image of a person's face with a certain degree of swing, The image may become unnatural. Therefore, the determination unit 238 determines whether or not the swing is large for each person's face image, and determines that the deformation process cannot be executed for the face image with a large swing.

  Specifically, the determination unit 238 determines that the swing is small when the distance Dc between the center C1 of the first face area FA1 and the center C2 of the second face area FA2 is equal to or less than a predetermined threshold Th. If the distance Dc is greater than the predetermined threshold Th, it is determined that the swing is large. This is because, for an image of a face with a large swing, the second face area FA2 is set as an area that is greatly enlarged in the direction in which the first face area FA1 is biased. In the example of FIG. 11, the face image of the person P1 is determined to be deformable because the swing is small, and the face image of the person P2 is determined to be unexecutable because the swing is large. In this way, it is possible to avoid the execution of deformation processing that causes unnatural results.

  In step S160 (FIG. 4), deformation processing is executed. The deformation process in step S160 is executed only for the face image that is determined to be deformable in step S150. A specific method of the deformation process will be described in detail later in “A-3. Deformation process”.

  When the face shape correction process (FIG. 4) ends, the process returns to the flowchart of FIG. 3, and the face shape correction unit 200 (FIG. 1) displays the target image TI after the face shape correction on the display unit 150. (Step S200). FIG. 12 is an explanatory diagram illustrating an example of a state of the display unit 150 on which the target image TI after the face shape correction is displayed. The display unit 150 displaying the target image TI after the face shape correction allows the user to check the correction result. When the user selects the “Return” button without being satisfied with the correction result, for example, a screen for selecting the deformation type and the deformation degree shown in FIG. 5 is displayed on the display unit 150, and the user again sets the deformation type and the deformation degree. Is executed.

  When the user is satisfied with the correction result and selects the “print” button, the print processing unit 320 (FIG. 1) controls the printer engine 160 to print the target image TI after the face shape correction processing (FIG. 3). Step S300). The print processing unit 320 generates print data by performing processing such as resolution conversion and halftone processing on the image data of the target image TI after the face shape correction processing. The generated print data is supplied from the print processing unit 320 to the printer engine 160, and the printer engine 160 executes printing of the target image TI. Thereby, the printing of the target image TI after the face shape correction is completed.

A-3. Transformation process:
FIG. 13 is a flowchart showing the flow of the deformation process (step S160 in FIG. 4). In step S162, the deformation area setting unit 240 sets the deformation area TA. The deformation area TA is an area on the target image TI and is an area to be subjected to image deformation processing for face shape correction. FIG. 14 is an explanatory diagram showing an example of a method for setting the deformation area TA. FIG. 14 shows a first face area FA1 corresponding to the face image of the person P1 as a target for executing the deformation process (see FIG. 6). A reference line RL shown in FIG. 14 is a line that defines the height direction (vertical direction) of the first face area FA1 and the center in the width direction (horizontal direction) of the first face area FA1. That is, the reference line RL is a straight line that passes through the center of gravity of the rectangular first face area FA1 and is parallel to the boundary line along the height direction (vertical direction) of the first face area FA1.

  As shown in FIG. 14, in the present embodiment, the deformation area TA extends in the direction (height direction) parallel to the reference line RL and the direction (width direction) perpendicular to the reference line RL with respect to the first face area FA1. It is set as a (or shortened) area. Specifically, assuming that the height direction of the first face area FA1 is Hf and the width direction is Wf, the first face area FA1 is k1 · Hf upward and k2 downward. A region that is extended by Hf and that is extended by k3 and Wf to the left and right is set as the deformation region TA. Note that k1, k2, and k3 are predetermined coefficients.

  When the deformation area TA is set in this way, the reference line RL that is a straight line parallel to the height direction outline of the first face area FA1 is also parallel to the height direction outline of the deformation area TA. It becomes a straight line. The reference line RL is a straight line that divides the width of the deformation area TA in half.

  As shown in FIG. 14, the deformation area TA is set as an area that generally includes an image from the jaw to the forehead in the height direction and includes images of the left and right cheeks in the width direction. That is, in the present embodiment, the above-described coefficients k1, k2, and k3 are set in advance based on the relationship with the size of the first face area FA1 so that the deformation area TA is an area that includes an image in such a range. Is set.

  In step S164, the deformation area dividing unit 250 (FIG. 1) divides the deformation area TA into a plurality of small areas. FIG. 15 is an explanatory diagram showing an example of a method of dividing the deformation area TA into small areas. The deformation area dividing unit 250 arranges a plurality of division points D in the deformation area TA, and divides the deformation area TA into a plurality of small areas using a straight line connecting the division points D.

  The arrangement mode of the dividing points D (the number and position of the dividing points D) is defined in association with the deformation type set in step S120 of FIG. 4 by the dividing point arrangement pattern table 410 (FIG. 1). The deformation area dividing unit 250 refers to the dividing point arrangement pattern table 410 and arranges the dividing points D in a manner associated with the deformation type set in step S120. In the present embodiment, as described above, the deformation “type A” (see FIG. 5) for sharpening the face is set as the deformation type, and therefore the division point D is used in a manner associated with this deformation type. Is placed.

  As shown in FIG. 15, the dividing points D are arranged at the intersections of the horizontal dividing line Lh and the vertical dividing line Lv and the intersections of the horizontal dividing line Lh and the vertical dividing line Lv and the outer frame of the deformation area TA. . Here, the horizontal dividing line Lh and the vertical dividing line Lv are reference lines for arranging the dividing points D in the deformation area TA. As shown in FIG. 15, in the arrangement of the dividing points D associated with the deformation type for sharpening the face, two horizontal dividing lines Lh perpendicular to the reference line RL and 4 parallel to the reference line RL are used. A vertical division line Lv is set. The two horizontal dividing lines Lh are called Lh1 and Lh2 in order from the bottom of the deformation area TA. The four vertical dividing lines Lv are referred to as Lv1, Lv2, Lv3, and Lv4 in order from the left of the deformation area TA.

  The horizontal dividing line Lh1 is arranged below the chin image in the deformation area TA, and the horizontal dividing line Lh2 is arranged near the eye image. The vertical dividing lines Lv1 and Lv4 are arranged outside the cheek line image, and the vertical dividing lines Lv2 and Lv3 are arranged outside the eye corner image. The horizontal dividing line Lh and the vertical dividing line Lv are arranged in the deformation area TA set in advance so that the positional relationship between the horizontal dividing line Lh and the vertical dividing line Lv and the image becomes the above-described positional relationship as a result. It is executed according to the correspondence with the size.

  According to the arrangement of the horizontal dividing line Lh and the vertical dividing line Lv described above, the intersection of the horizontal dividing line Lh and the vertical dividing line Lv, and the intersection of the horizontal dividing line Lh and the vertical dividing line Lv and the outer frame of the deformation area TA In addition, the dividing point D is arranged. As shown in FIG. 15, division points D located on the horizontal division line Lhi (i = 1 or 2) are referred to as D0i, D1i, D2i, D3i, D4i, and D5i in order from the left. For example, the dividing points D located on the horizontal dividing line Lh1 are called D01, D11, D21, D31, D41, D51. Similarly, the dividing points D located on the vertical dividing line Lvj (j = 1, 2, 3, 4) are called Dj0, Dj1, Dj2, Dj3 in order from the bottom. For example, the division points D located on the vertical division line Lv1 are called D10, D11, D12, and D13.

  As shown in FIG. 15, the arrangement of the dividing points D in the present embodiment is symmetrical with respect to the reference line RL.

  The deformation area dividing unit 250 divides the deformation area TA into a plurality of small areas by a straight line connecting the arranged dividing points D (that is, the horizontal dividing line Lh and the vertical dividing line Lv). In this embodiment, as shown in FIG. 15, the deformation area TA is divided into 15 rectangular small areas.

  In the present embodiment, since the arrangement of the dividing points D is determined by the numbers and positions of the horizontal dividing lines Lh and the vertical dividing lines Lv, the dividing point arrangement pattern table 410 has the numbers and positions of the horizontal dividing lines Lh and the vertical dividing lines Lv. In other words, it is possible to define that

  In step S166 (FIG. 13), the divided region deformation unit 260 (FIG. 1) performs an image deformation process on the deformation region TA of the target image TI. The deformation process by the divided area deforming unit 260 is performed by moving the position of the dividing point D arranged in the deformed area TA in step S164 and deforming the small area.

  The movement mode (movement direction and movement distance) of the position of each division point D for the deformation process is determined by the division type and the degree of deformation set in step S120 of FIG. 4 by the division point movement table 420 (FIG. 1). It is determined in advance in association with these combinations. The divided region deformation unit 260 refers to the divided point movement table 420 and moves the position of the divided point D with the moving direction and moving distance associated with the combination of the deformation type and the degree of deformation set in step S120. To do.

  In this embodiment, as described above, the deformation “type A” (see FIG. 5) for sharpening the face is set as the deformation type, and the degree of “medium” is set as the deformation degree. The position of the dividing point D is moved with the movement direction and the movement distance associated with the combination of these deformation types and deformation degrees.

  FIG. 16 is an explanatory diagram showing an example of the contents of the dividing point movement table 420. FIG. 17 is an explanatory diagram showing an example of movement of the position of the dividing point D according to the dividing point movement table 420. In FIG. 16, the movement type of the position of the dividing point D defined by the dividing point movement table 420 is associated with a combination of a deformation type for sharpening the face and a degree of “medium” deformation. The movement mode is shown. As shown in FIG. 16, in the dividing point movement table 420, for each dividing point D, the amount of movement along the direction (H direction) perpendicular to the reference line RL and the direction parallel to the reference line RL (V direction). It is shown. In this embodiment, the unit of movement amount shown in the dividing point movement table 420 is the pixel pitch PP of the target image TI. For the H direction, the amount of movement to the right side is represented as a positive value, the amount of movement to the left side is represented as a negative value, and for the V direction, the amount of movement upward is positive. It is expressed as a value, and the downward movement amount is expressed as a negative value. For example, the dividing point D11 is moved to the right along the H direction by a distance that is seven times the pixel pitch PP, and is moved upward along the V direction by a distance that is 14 times the pixel pitch PP. For example, the division point D22 is not moved because the movement amount is zero in both the H direction and the V direction.

  In the present embodiment, the division points D (for example, the division points D10 shown in FIG. 15) located on the outer frame of the deformation area TA so that the boundary between the inner and outer images of the deformation area TA does not become unnatural. The position is not moved. Accordingly, in the dividing point movement table 420 shown in FIG. 16, the movement mode for the dividing point D located on the outer frame of the deformation area TA is not defined.

  In FIG. 17, the division point D before the movement is indicated by a white circle, and the division point D after the movement or the division point D without the movement of the position is indicated by a black circle. Further, the divided point D after the movement is referred to as a divided point D ′. For example, the position of the dividing point D11 is moved in the upper right direction in FIG. 17 and becomes the dividing point D′ 11.

  In the present embodiment, all combinations of two division points D (for example, combinations of division points D11 and D41) that are in a symmetric positional relationship with respect to the reference line RL are the same after the movement of the division point D. The movement mode is determined so as to maintain a symmetrical positional relationship with respect to the line RL.

  For each small area constituting the deformation area TA, the divided area deforming unit 260 is an image of a small area newly defined by moving the position of the dividing point D. The image is deformed so that For example, in FIG. 17, an image of a small region (small region indicated by hatching) having vertices at the division points D11, D21, D22, and D12 is divided into the division points D′ 11, D′ 21, D22, and D′ 12. Is transformed into an image of a small area with the vertex at.

  FIG. 18 is an explanatory diagram showing the concept of the image deformation processing method by the divided region deformation unit 260. In FIG. 18, the division point D is indicated by a black circle. In FIG. 18, in order to simplify the description, for the four small regions, the state before the position movement of the dividing point D is shown on the left side, and the state after the position movement of the dividing point D is shown on the right side. In the example of FIG. 18, the central division point Da is moved to the position of the division point Da ′, and the positions of the other division points D are not moved. Thereby, for example, an image of a rectangular small area (hereinafter also referred to as “pre-deformation noticeable small area BSA”) having the vertices at the division points Da, Db, Dc, Dd before the movement of the division point D is obtained from the division point Da ′. , Db, Dc, and Dd are transformed into an image of a rectangular small area (hereinafter also referred to as “the noticed small area ASA after deformation”).

  In this embodiment, a rectangular small region is divided into four triangular regions using the center of gravity CG of the small region, and image deformation processing is performed in units of triangular regions. In the example of FIG. 18, the pre-deformation attention small area BSA is divided into four triangular areas having the centroid CG of the pre-deformation attention small area BSA as one vertex. Similarly, the post-deformation attention small area ASA is divided into four triangular areas having the centroid CG ′ of the post-deformation attention small area ASA as one vertex. Then, image deformation processing is performed for each corresponding triangular area in each state before and after the movement of the dividing point Da. For example, an image of a triangular area having vertices at the division points Da and Dd and the center of gravity CG in the attention small area BSA before deformation has a vertex at the division points Da ′ and Dd and the center of gravity CG ′ in the attention small area ASA after deformation. It is transformed into an image of a triangular area.

  FIG. 19 is an explanatory diagram showing the concept of an image deformation processing method in a triangular area. In the example of FIG. 19, the image of the triangular area stu with the points s, t, u as vertices is transformed into the image of the triangular area s′t′u ′ with the points s ′, t ′, u ′ as vertices. . For the deformation of the image, the position of a certain pixel in the image of the triangular area s't'u 'after the deformation corresponds to the position in the image of the triangular area stu before the deformation, and the calculated position This is performed by using the pixel value in the image before deformation in step S4 as the pixel value of the image after deformation.

  For example, in FIG. 19, the position of the pixel of interest p ′ in the image of the triangular area s′t′u ′ after the deformation corresponds to the position p in the image of the triangular area stu before the deformation. The position p is calculated as follows. First, coefficients m1 and m2 for expressing the position of the target pixel p ′ by the sum of the vector s′t ′ and the vector s′u ′ as shown in the following equation (1) are calculated.

  Next, by using the calculated coefficients m1 and m2, the position p is obtained by calculating the sum of the vector st and the vector su in the triangular area stu before deformation by the following equation (2).

  When the position p in the triangular area stu before deformation coincides with the pixel center position of the image before deformation, the pixel value of the pixel is set as the pixel value of the image after deformation. On the other hand, when the position p in the triangular area stu before deformation is shifted from the pixel center position of the image before deformation, an interpolation operation such as bicubic using the pixel values of the pixels around the position p. Thus, the pixel value at the position p is calculated, and the calculated pixel value is set as the pixel value of the image after deformation.

  Image deformation from the image of the triangular area stu to the image of the triangular area s't'u 'by calculating the pixel value for each pixel in the image of the triangular area s't'u' after the deformation as described above Processing can be performed. The divided area deformation unit 260 performs a deformation process by defining a triangular area as described above for each small area constituting the deformation area TA illustrated in FIG. 15 and performs an image deformation process in the deformation area TA.

  Here, the face shape correction mode of the present embodiment will be described in more detail. FIG. 20 is an explanatory diagram showing a form of face shape correction in the present embodiment. In this embodiment, as described above, the deformation “type A” (see FIG. 5) for sharpening the face is set as the deformation type, and the degree of “medium” is set as the deformation degree. In FIG. 20, the image of the deformation | transformation aspect of each small area | region which comprises deformation | transformation area | region TA is shown with the arrow.

  As shown in FIG. 20, in the face shape correction of this embodiment, the dividing points D (D11, D21, D31, D41) arranged on the horizontal dividing line Lh1 with respect to the direction parallel to the reference line RL (V direction). Is moved upward, while the positions of the dividing points D (D12, D22, D32, D42) arranged on the horizontal dividing line Lh2 are not moved (see FIG. 16). Therefore, the image located between the horizontal dividing line Lh1 and the horizontal dividing line Lh2 is reduced in the V direction. As described above, since the horizontal dividing line Lh1 is arranged below the chin image and the horizontal dividing line Lh2 is arranged in the vicinity immediately below the eye image, in the face shape correction of this embodiment, the face image is corrected. The image of the part from the inner jaw to the lower part of the eye is reduced in the V direction. As a result, the jaw line in the image moves upward.

  On the other hand, in the direction (H direction) perpendicular to the reference line RL, the position of the dividing point D (D11, D12) arranged on the vertical dividing line Lv1 is moved rightward and arranged on the vertical dividing line Lv4. The position of the division point D (D41, D42) is moved leftward (see FIG. 16). Further, of the two division points D arranged on the vertical division line Lv2, the position of the division point D (D21) arranged on the horizontal division line Lh1 is moved rightward and arranged on the vertical division line Lv3. Of the two divided points D, the position of the divided point D (D31) arranged on the horizontal dividing line Lh1 is moved to the left (see FIG. 16). Therefore, the image located on the left side of the vertical dividing line Lv1 is enlarged on the right side in the H direction, and the image located on the right side of the vertical dividing line Lv4 is enlarged on the left side. An image located between the vertical dividing line Lv1 and the vertical dividing line Lv2 is reduced or moved to the right in the H direction, and an image located between the vertical dividing line Lv3 and the vertical dividing line Lv4 is moved in the H direction. Is reduced or moved to the left. Further, the image located between the vertical division line Lv2 and the vertical division line Lv3 is reduced in the H direction with the position of the horizontal division line Lh1 as the center.

  As described above, the vertical division lines Lv1 and Lv4 are arranged outside the cheek line image, and the vertical division lines Lv2 and Lv3 are arranged outside the corner image. Therefore, in the face shape correction according to the present embodiment, the image of the portion outside the both corners of the face image is reduced in the H direction as a whole. In particular, the reduction rate is high near the jaw. As a result, the shape of the face in the image becomes thinner in the width direction as a whole.

  When the deformation modes in the H direction and the V direction described above are combined, the face shape in the target image TI is sharpened by the face shape correction of this embodiment. Note that a sharp face shape can be expressed as a so-called “small face”.

  Note that the small regions (hatched regions) having the vertexes at the dividing points D22, D32, D33, and D23 shown in FIG. This area includes the image. As shown in FIG. 16, since the dividing points D22 and D32 are not moved in the H direction or the V direction, the small area including the images of both eyes is not deformed. As described above, in this embodiment, the small area including the images of both eyes is not deformed, and the image after the face shape correction is made more natural and preferable.

B. Variations:
The present invention is not limited to the above-described examples and embodiments, and can be implemented in various modes without departing from the gist thereof. For example, the following modifications are possible.

B1. Modification 1:
The setting method of the first face area FA1, the search area SA, the second face area FA2, and the deformation area TA in the above embodiment is merely an example, and these areas can be set by other methods. For example, the first face area FA1 does not necessarily need to be set through area detection that is supposed to include a face image by image analysis of the target image TI, and the first face area FA1 is based on the designation of the position by the user. FA1 may be set.

  Further, the search area SA may be an area including pixels around the first face area FA1, and need not include the first face area FA1 itself. Even in this case, the skin color pixels may be detected in the search area SA, and a rectangular area including all the detected skin color pixels may be set as the second face area FA2.

  Further, the second face area FA2 does not necessarily have to be a rectangular area that includes all the detected skin color pixels, and may be a rectangular area that includes a part of the detected skin color pixels. The deformation area TA may be set based on the second face area FA2 instead of the first face area FA1.

  Further, the first face area FA1, the search area SA, the second face area FA2, and the deformation area TA do not have to be rectangular areas, and may be polygonal or circular areas other than rectangles.

B2. Modification 2:
In the above embodiment, whether or not the deformation process can be performed is determined based on the positional relationship between the first face area FA1 and the second face area FA2, but the first face area FA1 and the second face area FA2 Whether or not the deformation process can be executed may be determined based on the relationship with the size of the face area FA2. That is, as shown in FIG. 11, since an image of a face with a large swing is considered to have many skin color pixels at positions not included in the first face area FA1, In comparison, the ratio of the size of the second face area FA2 to the size of the first face area FA1 is considered to be large. Therefore, when the ratio of the size of the second face area FA2 to the size of the first face area FA1 is larger than a predetermined threshold, it may be determined that the deformation process cannot be performed. Alternatively, based on a condition that combines both the positional relationship between the first face area FA1 and the second face area FA2 and the relationship between the sizes of the first face area FA1 and the second face area FA2, It may be determined whether or not the deformation process can be executed.

  In the above embodiment, whether or not the deformation process can be executed is determined based on the relationship between the first face area FA1 and the second face area FA2. However, the first face area FA1 and the second face area are determined. It is also possible to determine whether other image processing can be executed based on the relationship with the area FA2. Examples of other image processing include skin color correction and brightness correction.

B3. Modification 3:
In the above embodiment, the second face area FA2 is set to determine whether or not the deformation process can be performed. However, the second face area FA2 may be set for other purposes. For example, the second face area FA2 may be set to determine a target area to be subjected to predetermined image processing. The second face area FA2 is an area that includes the image of the entire face and is more accurate than the first face area FA1. Therefore, for example, by performing skin color correction on the second face area FA2, more preferable skin color correction can be realized as compared with the case of performing skin color correction on the first face area FA1. In this way, setting the second face area FA2 can be said to redefine the face area.

B4. Modification 4:
The method of the deformation process in the above embodiment is merely an example, and these processes can be executed by other methods.

B5. Modification 5:
Although the face shape correction printing process (FIG. 3) by the printer 100 as the image processing apparatus has been described in the above embodiment, the face shape correction printing process includes, for example, face shape correction and display of corrected images (steps S100 and S200). It may be executed by a personal computer, and only the printing process (step S300) may be executed by the printer. The printer 100 is not limited to an ink jet printer, and may be another type of printer, such as a laser printer or a sublimation printer. The deformed image data can be used not only for printing but also for any purpose. For example, display by a display device (for example, a projector) may be employed.

B6. Modification 6:
In the above embodiment, a part of the configuration realized by hardware may be replaced with software, and conversely, a part of the configuration realized by software may be replaced by hardware. For example, the entire functions of the modified region dividing unit 250 and the divided region modifying unit 260 of FIG. 1 may be realized by a hardware circuit having a logic circuit.

  In addition, when part or all of the functions of the present invention are realized by software, the software (computer program) can be provided in a form stored in a computer-readable recording medium. In the present invention, the “computer-readable recording medium” is not limited to a portable recording medium such as a flexible disk or a CD-ROM, but an internal storage device in a computer such as various RAMs and ROMs, a hard disk, and the like. An external storage device fixed to the computer is also included.

1 is an explanatory diagram schematically illustrating a configuration of a printer 100 as an image processing apparatus according to an embodiment of the present invention. It is explanatory drawing which shows an example of the user interface containing the list display of an image. 4 is a flowchart illustrating a flow of face shape correction printing processing by the printer 100 according to the present exemplary embodiment. It is a flowchart which shows the flow of the face shape correction process in a present Example. It is explanatory drawing which shows an example of the user interface for setting the type and degree of image deformation. It is explanatory drawing which shows an example of the setting result of 1st face area FA1. It is a flowchart which shows the flow of the 2nd face area setting process in a present Example. It is explanatory drawing which shows the setting method of the range of a skin color value. It is explanatory drawing which shows an example of the setting result of search area SA. It is explanatory drawing which shows an example of the setting result of 2nd face area FA2. It is explanatory drawing which shows the determination method of the feasibility of execution of a deformation | transformation process. It is explanatory drawing which shows an example of the state of the display part 150 on which the target image TI after face shape correction | amendment was displayed. It is a flowchart which shows the flow of a deformation | transformation process. It is explanatory drawing which shows an example of the setting method of deformation | transformation area | region TA. It is explanatory drawing which shows an example of the division method to the small area | region of deformation | transformation area | region TA. It is explanatory drawing which shows an example of the content of the dividing point movement table. It is explanatory drawing which shows an example of the movement of the position of the dividing point D according to the dividing point movement table. 6 is an explanatory diagram showing a concept of an image deformation processing method performed by a divided region deformation unit 260. FIG. It is explanatory drawing which shows the concept of the deformation | transformation processing method of the image in a triangular area | region. It is explanatory drawing which shows the aspect of the face shape correction | amendment in a present Example.

Explanation of symbols

100 ... Printer 110 ... CPU
DESCRIPTION OF SYMBOLS 120 ... Internal memory 140 ... Operation part 150 ... Display part 160 ... Printer engine 170 ... Card interface 172 ... Card slot 200 ... Face shape correction | amendment part 210 ... Deformation mode setting part 220 ... 1st face area setting part 230 ... 2nd Face area setting section 232 ... Skin color value setting section 234 ... Skin color pixel detection section 238 ... Determination section 240 ... Deformation area setting section 250 ... Deformation area division section 260 ... Divided area deformation section 310 ... Display processing section 320 ... Print processing section 410 ... Division point arrangement pattern table 420... Division point movement table

Claims (4)

An image processing apparatus,
A first face area setting unit for setting a first face area that is an area including a face image in the target image;
A skin color value setting unit that sets a range of skin color values based on pixel values of pixels in the first face area;
A skin color pixel detection unit that detects a skin color pixel that is a pixel having a pixel value included in the range of the skin color value in an area including pixels around the first face area;
Based on the detection result of the skin color pixel, a second face region that sets a second face region that is a region that includes a face image in the target image and that includes at least a part of the detected skin color pixel. An image processing apparatus comprising: a setting unit; The image processing apparatus according to claim 1, further comprising:
A determination unit that determines whether or not to execute predetermined image processing on an area set based on the first face area based on the first face area and the second face area ;
The determination unit is based on at least one of a positional relationship between the first face region and the second face region and a relationship regarding the size of the first face region and the second face region. An image processing apparatus that performs the determination . The image processing apparatus according to claim 2,
The image processing apparatus, wherein the predetermined image processing is deformation processing of an image in an area including the first face area. The image processing apparatus according to claim 2 or 3, further comprising:
An image processing apparatus comprising: a process execution unit that executes the predetermined image processing when the determination unit determines that execution is possible.

Images