JP2010056726A - Image processor, image processing method and image processing program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To solve the problem that an operation processing amount required for obtaining a desired image deformed result is large and a user is required to perform a troublesome operation as well. <P>SOLUTION: The image processor includes: a body shape estimation part (S500) for estimating a body shape area corresponding to the body shape of a figure including a face image in an object image including the face image; a correction value acquisition part (S600) for acquiring a correction value for roughly matching aspect ratio of the body shape area with an ideal ratio; and an image deformation part for deforming the area of the object image including the body shape area at least in a specified direction corresponding to the correction value. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to an image processing apparatus, an image processing method, and an image processing program.

An image processing technique for deforming an image for a digital image is known (see Patent Document 1). In Patent Document 1, a partial area (an area representing a cheek image) on a face image is set as a correction area, the correction area is divided into a plurality of small areas according to a predetermined pattern, and set for each small area. Image processing for deforming the shape of a face by enlarging or reducing the image at a specified magnification is disclosed.
JP 2004-318204 A

  However, in image processing for deforming an image by setting a correction area as described above, processing that requires a large amount of calculation processing such as setting a correction area and enlarging or reducing a small area is performed. For this reason, the amount of calculation processing for performing image processing may be excessive. This problem is not limited to the case of deforming a person's face, and is generally common to the process of deforming an image. In addition, in order to allow the user to make various settings such as the necessity of execution of image deformation and the strength (degree) of deformation before executing such image deformation, the user's effort is increased.

  The present invention has been made in view of the above problems, and reduces the amount of calculation processing required for image deformation and automatically obtains an image subjected to ideal deformation processing without imposing a burden on the user. An object is to provide an image processing apparatus, an image processing method, and an image processing program.

  In order to achieve the above object, an image processing apparatus of the present invention includes a body shape estimation unit that estimates a body shape region corresponding to a body shape of a person including a face image in a target image including a face image, and a vertical and horizontal direction of the body shape region. A correction value acquisition unit that acquires a correction value for making the ratio substantially coincide with the ideal ratio; and an image deformation unit that deforms at least the region of the target image including the body shape region in a specific direction according to the correction value. It is set as the structure provided with. According to the present invention, the figure of a person in a target image can be automatically transformed into an ideal figure with a small amount of processing.

  The body type estimation unit may extract the width of the feature portion of the person from the target image and estimate the body region based on the width of the extracted feature portion. Specifically, the body type estimation unit extracts the width of a person's cheek or shoulder or waist as the characteristic portion. According to the said structure, the area | region corresponding to the body shape of the person in a target image can be estimated with a sufficient precision.

  The image processing apparatus includes an age-gender estimation unit that estimates an age group and / or sex of the person, and the correction value acquisition unit changes the acquired correction value according to the estimated age group and / or gender. You may do that. According to the said structure, a person's body shape can be deform | transformed with the correction value appropriately changed according to the estimated age group and sex. The change of the correction value referred to here includes setting the correction amount indicated by the correction value to zero. In this case, the image deforming unit is not substantially deformed.

  The correction value acquisition unit acquires a correction value for making the aspect ratio of the body shape area substantially coincide with the aspect ratio of the body shape of the person perceived when the person is directly visually recognized by both eyes of the observer. It is good. According to this configuration, the body shape of the person in the target image can be transformed into a body shape that gives an impression equivalent to that obtained when the subject (the person) is directly viewed with human eyes.

  When the orientation, inclination, and position of the face image in the target image are within the allowable ranges set for the orientation, inclination, and position, respectively, estimation of the body region, acquisition of correction values, and deformation of the region And may be executed. According to this configuration, since the orientation, inclination, and position of the face image are not within the respective allowable ranges, the effect of image deformation is difficult to be obtained properly (or the image is broken by performing image deformation on the contrary). ), It is possible to prevent unnecessary image deformation.

  So far, the technical idea of the present invention has been described as an image processing apparatus. However, the invention of the image processing method including each process corresponding to each means included in the above-described image processing apparatus and the above-described image processing apparatus are included. The invention of an image processing program that causes a computer to execute each function corresponding to each means can also be grasped. Further, the image processing apparatus, the image processing method, and the image processing program may be specifically realized by hardware such as a personal computer or a server, or a digital still camera or scanner as an image input apparatus, or It can be realized by various products such as a printer (printing device) as an image output device, a projector, and a photo viewer.

1. FIG. 1 is an explanatory diagram schematically showing a configuration of a printer 100 as an example of an image processing apparatus according to the present invention. The printer 100 is a color ink jet printer compatible with so-called direct printing, in which an image is printed based on image data acquired from a recording medium such as a memory card MC. The printer 100 includes an internal memory 120, a CPU 110, an operation unit 140, a display unit 150, a printer engine 160, a card interface (card I / F) 170, and a card slot 172.

  The internal memory 120 includes a ROM and a RAM, and includes an image correction processing unit 200, a display processing unit 310, and a print processing unit 320 as functional blocks. The image correction processing unit 200 is a computer program for executing an image correction process described later under a predetermined operating system. The image correction processing unit 200 includes, as program modules, a face area detection unit 210, an exclusion determination unit 220, a body type estimation unit 230, a correction value acquisition unit 240, a one-way deformation unit 250, and an age and gender estimation unit 260. have. The unidirectional deformation unit 250 executes a unidirectional deformation process as a kind of image correction process by using a buffer area that is a temporary storage area provided in the RAM and the corresponding pixel number table 410. The unidirectional deformation unit 250 corresponds to the image deformation unit in the claims. The printer 100 can transform the body shape of a person image into an ideal body shape (ideal aspect ratio) by executing a unidirectional deformation process on an image including a person image. The functions of these units will be described later.

  The display processing unit 310 is a display driver that controls the display unit 150 to display processing menus, messages, images, and the like on the display unit 150. The print processing unit 320 generates print data defining the ink amount for each pixel by performing predetermined color conversion processing and halftone processing on the image data, and controls the printer engine 160 to control the print data based on the print data. The computer program for executing image printing. For example, image data representing an image after the image correction processing is executed by the image correction processing unit 200 corresponds to the image data to be processed by the color conversion processing and the halftone processing by the print processing unit 320. The CPU 110 implements the functions of these units by reading out the above-described programs from the internal memory 120 and executing them.

  The operation unit 140 includes buttons and a touch panel, and accepts an input such as an instruction from the user. The display unit 150 is configured by a liquid crystal display, for example. The printer engine 160 is a printing mechanism that performs printing based on print data transmitted from the print processing unit 320. The card I / F 170 is an interface for exchanging data with the memory card MC inserted into the card slot 172. In addition to the card I / F 170, the printer 100 may include an interface for performing data communication with other devices (for example, a digital still camera or a personal computer). The above components are connected to each other via a bus.

2. Image Correction Processing FIG. 2 is a flowchart showing image correction processing executed by the printer 100. In step S (hereinafter, step notation is omitted) 100, the image correction processing unit 200 sets (acquires) a target image to be processed. For example, when the memory card MC is inserted into the card slot 172, the display processing unit 310 causes the display unit 150 to display a user interface (UI) including display of an image stored in the memory card MC. Then, the image correction processing unit 200 sets a target image according to the input from the UI.

  FIG. 3 shows an example of a UI displayed on the display unit 150 in the process of S100. For example, the UI includes an image display field IA, two image switching buttons B1 and B2, a print button B3, and a cancel button B4. The user can set the target image by operating the image switching buttons B1 and B2 while selecting the target image while viewing the UI and pressing the print button B3. After completion of the flowchart, the printer 100 prints an image on which the deformation process has been executed (or an image on which the deformation process has not been executed if the deformation process has not been executed in the flowchart). That is, in the present embodiment, the process from S200 onward is automatically executed only by the user pressing the print button B3, and then a print result is obtained. In the example of FIG. 3, the image TI in which the person P1 is captured is selected as the target image. When the user presses the print button B3 in this state, the image correction processing unit 200 sets the image TI as the target image. To do. The UI may be configured to display a list of a plurality of images in the memory card MC.

  In S200, the face area detection unit 210 reads image data representing the target image from the memory card MC to the internal memory 120, and detects the face area in the target image. The face area is an area including at least a part of the face image in the target image, and is an area assumed to include at least a predetermined facial organ (eyes, nose, mouth). The face area detection unit 210 analyzes image data representing the target image and detects a rectangular area that is assumed to include the face organ as a face area. The face area is detected using a known detection method such as a pattern matching method using a template (see Japanese Patent Application Laid-Open No. 2006-279460). The face area detection unit 210 can employ any technique for the pattern matching as long as it can detect the face area. For example, the face area detection unit 210 inputs various pieces of image information (for example, luminance information, edge amount, contrast, etc.) in units of rectangular areas (detection target areas) set in the target image, and the detection target area is the face. A face area may be detected by using a pre-learned neural network that outputs information indicating whether or not the area corresponds, or a face area for each detection target area using a support vector machine It may be determined whether or not.

  FIG. 4 is an explanatory diagram showing an example of a face area detection result in S200. In the example of FIG. 4, since the image of the person P1 is included in the target image TI, the face area Fd corresponding to the face image of the person P1 is detected. The face area Fd1 is a rectangular area that includes all the images of both eyes, nose, and mouth. The face area detection unit 210 identifies the face area Fd based on the coordinates of the four vertices of the area. However, when the detection result of the face area Fd in the target image (information for specifying the position of the face area in the image) already exists as the attached information in the image data representing the target image stored in the memory card MC. There is also. For example, the face area Fd is already detected from the target image by the function of the digital still camera used for shooting the target image, and information indicating the detection result is recorded on the memory card MC together with the image data. . In such a case, the face area detection unit 210 does not need to detect the face area Fd from the target image, and specifies the face area Fd in the target image based on the attached information.

  In S300, the age and gender estimation unit 260 estimates the age group and gender of the face (person) for the image in the face area detected or specified in S200. The age group mentioned here is a layer classified as, for example, an infant, child, adolescent, middle-aged, middle-aged, elderly or the like. The age and sex estimation unit 260 performs an age group and sex estimation process based on an image using a known method. Note that S300 is executed only when the detection or identification of the face area is successful in S200.

  In S400, the exclusion determination unit 220 determines whether or not the target image set in S100 satisfies a condition (exclusion condition) that should be excluded from the target of the unidirectional deformation process. Then, if the target image does not correspond to the exclusion condition, the process proceeds to S500 and the subsequent steps. On the other hand, if the target image meets the exclusion condition, the process ends without performing the processing of S500 and subsequent steps. In the present embodiment, the exclusion determination unit 220 determines that a target image whose face area is not detected in S200 and does not have attached information indicating a detection result of the face area (not a scene in which a person is photographed) is an exclusion condition. It is judged that it corresponds to. That is, the unidirectional deformation process is not performed on the target image that does not include the person image. Further, even when a face area exists in the target image, the exclusion determination unit 220 has the face orientation, inclination, and position indicated by the face area within the allowable ranges set for the face orientation, inclination, and position, respectively. If any one of the face orientation, tilt, and position exceeds the allowable range, it is determined that the exclusion condition is met.

  The face orientation is the direction of face swing based on the front-facing state, and includes rightward, leftward, upward, and downward. The orientation of the face can be determined, for example, based on the positional relationship of each facial organ such as the right eye, left eye, and mouth in the face area. For example, the exclusion determination unit 220 detects a right eye region, a left eye region, and a mouth region from a face region by a known detection method such as pattern matching using a template, and connects from the center of gravity of the right eye region to the center of gravity of the left eye region. A line segment (reference width distance) and a line segment (reference height distance) connecting the line segment to the center of gravity of the mouth area are calculated. If the ratio of the reference height distance to the reference width distance (reference height distance / reference width distance) is below a predetermined threshold Th1, it is determined that the face indicated by the face area is upward or downward, When the ratio of the reference height distance to the reference width distance exceeds a predetermined threshold Th2 (Th2> Th1), it is determined that the face is facing right or left. Such upward, downward, rightward and leftward faces fall under the exclusion condition. In other words, when the ratio falls within the range of the threshold values Th1 and Th2, the exclusion determination unit 220 determines that the face direction indicated by the face region is within the front-facing range (within the allowable range).

  The face inclination is an inclination in the height direction of the face with respect to the vertical direction or the horizontal direction of the target image. In the present embodiment, the long side direction of the target image is the horizontal direction of the target image, and the short side direction of the target image is the vertical direction of the target image. The exclusion determination unit 220 identifies the height direction of the face. For example, the direction in which the line segment orthogonal to the line segment indicating the reference width distance faces is specified as the face height direction (the vertical direction of the face). The angle formed by the face height direction with respect to the vertical direction of the target image is within a predetermined angle range (for example, within 20 °), or the face height direction is formed with respect to the horizontal direction of the target image. When the angle is within the predetermined angle range, the face inclination is determined to be within the allowable range. That is, the exclusion determination unit 220 determines that the exclusion condition is met when the height direction of the face is tilted with respect to both the vertical direction and the horizontal direction of the target image from the predetermined angle.

  As a result of inspecting the angle between the height direction of the face and the vertical and horizontal directions of the target image, the image correction processing unit 200 is between the vertical direction and the horizontal direction of the target image. The direction with the larger angle is specified as the deformation direction for the unidirectional deformation process. That is, the horizontal direction is the deformation direction when the height direction of the face is substantially oriented in the vertical direction, and the vertical direction is the deformation direction when the height direction of the face is substantially oriented in the substantially horizontal direction. In the following, when simply referred to as “deformation direction”, it means a deformation direction for one-way deformation processing. In the present embodiment, as shown in FIG. 4, the height direction of the face of the person P1 is substantially parallel to the vertical direction of the target image, so the horizontal direction of the target image is the deformation direction.

  As described above, when the face orientation and inclination exceed the allowable range, it is difficult to obtain a good image deformation result even if the one-way deformation process is applied to the face image. Furthermore, the exclusion determination unit 220 identifies the height direction of the person's body in the target image, and even if the body height direction and the face height direction do not substantially match, it is determined that the exclusion condition is met. Good. The exclusion determination unit 220 extracts a person including the face area from the target image by, for example, pattern matching or contour extraction, and detects the vertical direction of the extracted person's torso as the body height direction. Then, when an angle greater than a certain angle is generated between the height direction of the face and the height direction of the body, it is determined that the exclusion condition is met. For example, if the person in the target image has a posture in which the face is not tilted but the body is facing sideways or diagonally, a good image deformation result can be obtained even if the one-way deformation process is applied to the person image. Since it is difficult, the flowchart is ended.

  Further, when the deformation direction is the horizontal direction, the exclusion determination unit 220 sets a predetermined area at both ends of the target image in the horizontal direction as the exclusion area, and when the face area is included in the exclusion area, the face area Is determined to exceed the allowable range (corresponding to the exclusion condition). On the other hand, when the deformation direction is the vertical direction, the exclusion determination unit 220 sets a predetermined area at both ends of the target image in the vertical direction as the exclusion area, and when the face area is included in the exclusion area, the face area Is determined to exceed the allowable range (corresponding to the exclusion condition).

  FIG. 5 shows exclusion areas A1 and A2 set in the target area TI when the deformation direction is the horizontal direction, and exclusion areas A3 and A4 set in the target area when the deformation direction is the vertical direction. Is illustrated. In the unidirectional deformation processing, as will be described later, basically, a region at the substantially center in the deformation direction of the target image is often compressed in the deformation direction, and both end regions in the deformation direction are often expanded in the deformation direction. Therefore, when an important face area exists at the end of the target image, it is not possible to apply appropriate compression to the person image including the face area, and conversely, the person is enlarged (thickened). There is also a risk of becoming. Therefore, even when the position of the face in the target image exceeds the allowable range (out of the range), the flowchart ends without performing the unidirectional deformation process.

  Furthermore, the exclusion determination unit 220 may determine whether or not to exclude the target image from the processes in S500 and subsequent steps based on the age group and sex estimated for the face in the target image in S300. For example, the exclusion determination unit 220 determines that the target image satisfies the exclusion condition when the estimated gender is male or the estimated age group is a child or less. For men, children, and infants, it is determined that it is less necessary to apply the deformation process of the present embodiment to make the body shape thinner (thinning), and the flowchart ends. However, the degree of deformation (correction value) at the time of the one-way deformation process is changed according to the sex and the age group, as described later, instead of completely removing the one-way deformation process from being applied depending on the sex or age group. It is good.

  In S500, the body shape estimation unit 230 estimates and sets a body region corresponding to the body shape of the person including the face region in the target image determined not to satisfy the exclusion condition in S400. There are various body shape region estimation methods. As an example, in this embodiment, the width of a feature portion of a face or body is extracted, and a rectangular region having an aspect ratio corresponding to the extracted width is used as a body image as a target image. Set in. For example, the body shape estimation unit 230 extracts a face outline in an image area including the face area, and the length of the outline in the height direction of the face is orthogonal to the height direction of the face at the cheek position of the outline. The length (cheek width) in (the width direction of the face or the left and right direction of the face) is obtained.

  FIG. 6 illustrates the length Lh in the height direction of the face FL extracted from the image area including the face area Fd and the length Lw in the width direction of the face at the cheek position of the outline FL. . The contour FL is extracted by, for example, extracting a skin color pixel in an image region including the face region Fd and specifying a boundary of a region formed by the extracted skin color pixel, or creating a contour of the face around the face region Fd. It can be extracted by detecting the corresponding edge. The cheek position in the height direction of the face for specifying the length Lw is determined by the positional relationship between the left / right eye region and the mouth region detected as described above, and the eyes in the predetermined human face. And can be estimated based on the positional relationship between the cheek and mouth. The body shape estimation unit 230 changes the aspect ratio of the standard rectangle that is the basis of the body region according to the ratio of the length Lh to the length Lw.

  The standard rectangle is a rectangle that is stored in advance in the internal memory 120 as corresponding to the aspect ratio of a standard human figure (for example, a medium-medium profile). The body shape estimation unit 230 increases the length of the standard rectangle as the ratio of the length Lw to the length Lh (Lw / Lh) is larger (larger than a predetermined reference value), that is, the face is thicker. The standard rectangle is deformed by increasing the horizontal length with respect to, and the aspect ratio of the deformed standard rectangle is defined as the aspect ratio of the body region. The body shape estimation unit 230 also increases the horizontal length with respect to the vertical length of the standard rectangle as the ratio (Lw / Lh) is smaller (smaller than the predetermined reference ratio), that is, the face is thinner. The standard rectangle is deformed by reducing the height, and the aspect ratio of the standard rectangle after the deformation is set as the aspect ratio of the body region.

  FIG. 7 illustrates a body area BA estimated in the target image TI. The body shape estimation unit 230 sets the rectangle having the aspect ratio determined according to the width of the cheek of the face as described above in the target image TI based on the positional relationship and the size relationship with the predetermined face region Fd. By setting, the estimation of the body area BA is completed. The positional relationship between the face area Fd and the body area BA is, for example, a position on the center line of the body area BA parallel to the longitudinal direction of the body area BA (long side direction of the body area), and the center line The center of gravity of the face area Fd is placed at a predetermined ratio from the upper end of the face. The relationship between the size of the face area Fd and the body shape area BA is, for example, a rule that the length of the body shape area BA in the vertical direction is a predetermined multiple of the length of the face area Fd in the height direction of the face. .

  The body shape estimation unit 230 may determine the aspect ratio of the rectangle of the body shape region in accordance with the width of the waist and shoulders as the character characteristic portion in addition to the cheeks. In this case, the body type estimation unit 230 determines the shoulder width (shoulder width in the direction parallel to the face width direction) or waist width (waist width in the direction parallel to the face width direction) of the person including the face region from the target image. ), The larger the shoulder width or waist width (larger than the predetermined reference width), the longer the standard rectangle is, the longer the horizontal length is, and the standard rectangle is deformed. The aspect ratio of the rectangle is the aspect ratio of the body area. Also, as the shoulder width or waist width is smaller (smaller than the predetermined reference width), the standard rectangle is deformed by reducing the horizontal length with respect to the vertical length of the standard rectangle, and the standard rectangle after being deformed The ratio is the aspect ratio of the body area. Note that the width of the body feature such as the shoulder and the waist may be specified based on a predetermined positional relationship in the extracted body extracted from the target region by pattern matching or the like. You may specify using the estimation direction of the feature-value described in 252025 gazette and JP, 2008-33654, A.

In S600, the correction value acquisition unit 240 acquires a correction value used for the one-way deformation process in S700, which is a correction value for making the aspect ratio of the body region estimated in S500 substantially match the ideal ratio. The ideal ratio means an aspect ratio of an ideal figure in the present embodiment, and is a numerical value that is set in advance according to market research or user preference and stored in the internal memory 120. Here, the ideal ratio (vertical: horizontal) is H1: W1, the vertical length (number of pixels) and the horizontal length (number of pixels) of the body shape area estimated in S500 are Hp, Wp, respectively (see FIG. 7). When Wi is the ideal lateral length of the corrected body shape region, Wi is expressed by the following equation (1).
Wi = W1 · Hp / H1 (1)

The correction value C is expressed by the following equation (2).
C = {(Wi−Wp) / Wp} · k (2)
k is a predetermined coefficient for adjusting the correction value C, and is basically 0 <k <1. The coefficient k is a numerical value for preventing the correction amount (the amount of deformation of the image) represented by the correction value C from being too large and causing the image after the deformation to fail (the image changes too much to give the user a sense of incongruity). is there. However, the coefficient k may not exist substantially, and k = 1 may be sufficient. The correction value C is a correction value that acts on the approximately horizontal direction of the body shape in the unidirectional deformation process. By correcting (deforming) the body region by the correction value C, the aspect ratio of the body region is approximately the ideal ratio. Can be matched. When the correction value C is a positive value (plus), it means a positive deformation (enlargement), and when the correction value C is a negative value (minus), it means a negative deformation (compression). To do. Note that the ideal ratio in the present embodiment assumes an aspect ratio of a slim figure that is slightly narrower than the aspect ratio of a standard human figure, and therefore the correction value C may be a negative value. Many.

  The correction value acquisition unit 240 may change the coefficient k according to the age group and sex estimated in S300. That is, the age group and gender information estimated in S300 may be used when acquiring the correction value in S600, instead of being used in the exclusion determination in S400. For example, when the estimated gender is male or the estimated age group is a child or less, the correction value acquisition unit 240 is the estimated gender is female and the estimated age group is greater than or equal to the adolescent. The coefficient k that is smaller (closer to 0) than the coefficient k used in the case of As a result, when the person in the target image is a man or an age group below a child, the amount of deformation of the image is suppressed. Further, the correction value acquisition unit 240 may not calculate the correction value C in S600 when the difference between the Wi and Wp is small to some extent (when Wi−Wp falls within the range of 0 ± predetermined value). . That is, when the estimated aspect ratio of the body region is close to the ideal ratio, there is no need to perform image deformation. Therefore, from the viewpoint of reducing the amount of calculation processing, if the difference between Wi and Wp is small to some extent, the correction amount C is not calculated, and if the correction amount C is not calculated, the process ends without proceeding to S700. Let

  Depending on how the person is photographed, only the upper body may be photographed, and the whole body of the person is not necessarily captured in the target image. That is, the body shape region estimated by setting the rectangle having the aspect ratio determined according to the width of the cheek of the face as described above to the target image TI based on the positional relationship and the size relationship with the face region Fd. In some cases, the lower end portion of the image protrudes from the target image. In this case, of the body area that has been estimated once, the amount of the body area that falls within the target image is treated as the body area. However, it is not possible to simply compare the aspect ratio of the body region that fits in the target image with the ideal ratio. Therefore, when a part of the body shape area once estimated as described above protrudes from the target image, the body shape estimation unit 230 calculates how many of the body shape areas are included in the target image. Then, the correction value acquisition unit 240 changes the ideal ratio in accordance with the ratio of the body area that is included in the calculated target image, and the amount that is within the changed ideal ratio and the target image. The correction value C may be calculated using the aspect ratio of the body region.

  In S700, the unidirectional deformation unit 250 performs a unidirectional deformation process on the target image. In the unidirectional deformation process, the unidirectional deformation unit 250 deforms the region of the target image including at least the body shape region in the deformation direction according to the correction value C acquired in S600. In this case, the unidirectional deformation unit 250 divides the target image into a first region to which the correction value C is applied and a second region other than that by a dividing line orthogonal to the deformation direction.

  FIG. 8A illustrates a state in which the target image TI is divided into a first area AC and a second area AE, and FIG. 8B illustrates an image (transformed image) after the unidirectional deformation process is performed on the target image TI. TI '). The unidirectional deformation unit 250 determines the width of the first region AC in the deformation direction based on the width of the body region BA in the deformation direction (for example, at least a width larger than the width of the body region BA in the deformation direction) A region having the determined width and including the body region BA (basically included in the center) by a dividing line orthogonal to the deformation direction, The identified area is set as the first area AC. Accordingly, the first area AC includes the entire body area BA. In addition, the unidirectional deformation unit 250 sets a region other than the first region AC (regions on both sides of the first region AC) in the target region TI as the second region AE.

  FIG. 9 is a flowchart illustrating the unidirectional deformation process in S700, and FIG. 10 schematically illustrates the unidirectional deformation process when the deformation direction is the horizontal direction. FIG. 10A shows an arrangement of pixels in the target image before the one-way deformation process is performed, FIG. 10B shows an example of the corresponding pixel number table 410, and FIG. 10C shows an image on which the one-way deformation process has been performed. The pixel arrangement of (deformed image) is shown. In S710, the unidirectional deformation unit 250 determines whether the deformation direction is the horizontal direction or the vertical direction of the target image. When the deformation direction is the horizontal direction, the process proceeds to S720, and when the deformation direction is the vertical direction, the process proceeds to S750. In the present embodiment, since the horizontal direction of the target image is the deformation direction, the process proceeds to S720. In S720, the unidirectional deformation unit 250 sets a first region and a second region in the target image. Since the setting method of the first region and the second region has already been described, the description thereof is omitted here.

  In S730, the unidirectional deformation unit 250 generates the corresponding pixel number table 410. The corresponding pixel number table 410 is a table representing the number of pixels of the deformed image corresponding to each pixel in the deformation direction of the target image. The unidirectional deformation unit 250 determines the number of pixels of the deformed image (corresponding number of pixels) based on the correction values for the first region and the second region. Then, by storing the determined corresponding pixel number in the corresponding pixel number table 410, the corresponding pixel number table 410 is generated. In generating the corresponding pixel number table 410, the unidirectional deformation unit 250 converts the correction value C into a magnification (deformation magnification) to be applied to the first region. For example, when the correction value C is a numerical value “−0.4”, this means that “0.4” times the original size is subtracted from the original size. To the numerical value (magnification). Conversely, when the correction value C is a numerical value of “0.4”, this means that “0.4” times the original size is added to the original size. To the numerical value (magnification). Here, it is assumed that the correction value C is a negative value, and the unidirectional deformation unit 250 obtains a value of “0.6” as the magnification applied to the first region.

  In addition, the unidirectional deformation unit 250 also sets a magnification applied to the second region. For example, the printer 100 has a table that prescribes a correspondence relationship between the magnification applied to the first area and the magnification applied to the second area in the internal memory 120 or the like, and the one-way deformation unit 250 stores the table The magnification to be applied to the second area can be obtained by referring to the magnification applied to the first area acquired as described above. Alternatively, the unidirectional deformation unit 250 substantially matches the size of the target image and the size of the deformed image according to the ratio of the size of the first and second regions in the target image and the magnification of the first region. The magnification of the second region may be obtained. In the present embodiment, it is assumed that the unidirectional deformation unit 250 obtains a value of “1.2” as the magnification applied to the second region. For example, the unidirectional deformation unit 250 performs binarization by halftone processing on the decimal part of the magnification obtained as described above to determine an array pattern of 0 and 1, and sets the array pattern to a value of 0 or 1 By adding the integer part of the magnification, the number of corresponding pixels for each pixel of the target image can be determined. In generating the corresponding pixel number table 410, the unidirectional deformation unit 250 can use a known method such as dithering or error diffusion as the halftone process. Alternatively, an array pattern stored in advance for each fractional part of the magnification may be used.

  In FIG. 10A, for the sake of simple explanation, there are areas D, E, and F each having 5 pixels in the horizontal direction of the image, the areas D and F are the second areas, and the area E is the first. The description will be made assuming a region. Here, the magnification of the region D is 1.2 times, the magnification of the region E is 0.6 times, and the magnification of the region F is 1.2 times. Therefore, in the corresponding pixel number table (FIG. 10B), Of the five pixels Px1 to Px5, four pixels Px1, Px2, Px3, and Px5 have the corresponding pixel number set to 1, and the remaining one pixel Px4 has the corresponding pixel number set to two. In addition, among the five pixels Px6 to Px10 in the region E, the corresponding number of pixels is set to 1 for the three pixels Px6, Px8, and Px10, and the corresponding number of pixels is set to 0 for the remaining two pixels Px7 and Px9. Has been. Of the five pixels Px11 to Px15 in the region F, the corresponding number of pixels is set to 1 for four pixels Px11, Px12, Px13, and Px15, and the corresponding number of pixels is set to 2 for the remaining one pixel Px14. Has been.

  In S740, the unidirectional deformation unit 250 refers to the corresponding pixel number table 410 and rearranges pixels for one line of the target image using the buffer area in the internal memory 120. A line is a processing unit in the unidirectional deformation process, and is an image area whose length is the total number of pixels in the horizontal direction of the target image and whose width is one pixel. In the example of FIG. 10, as shown in FIG. 10C, pixels Px1, Px2, and Px3 having the corresponding number of pixels are sequentially arranged one by one as the pixels constituting the region D, and then the number of corresponding pixels is 2. Two pixels Px4 are arranged, and then one pixel Px5 having a corresponding number of pixels of one is arranged. Next, the pixels Px6, Px8, and Px10 having the corresponding number of pixels 1 are sequentially arranged one by one as the pixels constituting the region E, and then the pixels Px11, Px12 and Px13 are arranged one by one, two pixels Px14 with the corresponding number of pixels 2 are arranged, and one pixel Px15 with the number of corresponding pixels 1 is arranged. By performing such rearrangement of pixels for one line for all lines, the second area of the target image is enlarged 1.2 times, and the first area is compressed 0.6 times.

  In S750, the unidirectional deformation processing unit 250 determines whether or not pixel rearrangement has been completed for all lines of the target image. When the pixel rearrangement has been completed for all lines, the unidirectional deformation process is terminated, and as a result, the flowchart of FIG. 2 is terminated. On the other hand, when the pixel rearrangement is not completed, the process returns to S740, and S740 and S750 are repeatedly executed until the pixel rearrangement is completed for all lines. In S760, the unidirectional deformation unit 250 divides the target image into a first area and a second area by a dividing line orthogonal to the vertical direction of the target image. In S770, the first area and the second area are the same as S730. The corresponding pixel number table 410 is generated using the respective magnifications. When the deformation direction is the vertical direction, a corresponding pixel number table 410 defining the number of corresponding pixels for each pixel aligned in the vertical direction of the target image (each line aligned in the vertical direction) is generated. Since the method for determining the number of corresponding pixels is the same as that in S730, description thereof is omitted.

  In S780, the one-way deformation unit 250 refers to the corresponding pixel number table and arranges the line of the target image in the storage area of the deformed image provided in the buffer area. When the deformation direction is the horizontal direction, rearrangement is performed according to the number of corresponding pixels (0, 1, 2 or the like) corresponding to the pixel in units of pixels, but when the deformation direction is the vertical direction, Arrangement is performed in line units according to the number of corresponding pixels corresponding to the line. Specifically, one line of the target image stored in the buffer area is added to the storage area of the deformed image in the buffer area as a line corresponding to the number of corresponding pixels corresponding to the one line. In S790, the unidirectional deformation unit 250 determines whether or not the process of S780 has been completed for all the lines of the target image. If the process has been completed, the unidirectional deformation unit 250 ends the unidirectional deformation process. The flowchart of FIG. 2 ends. On the other hand, if not completed, the process returns to S780, and S780 and S790 are repeatedly executed until S780 is performed for all the lines of the target image.

  By the unidirectional deformation process in S700, the first region including the body region is deformed (compressed) in the deformation direction according to the correction value C, and the second region is also deformed in the deformation direction (when the first region is compressed). Is enlarged). That is, as illustrated in FIG. 8B, the first area AC is deformed (compressed) into the first area AC ′, the second area AE is deformed (enlarged) into the second area AE ′, and the deformed image TI. The figure of the middle person P1 has an aspect ratio that substantially matches the ideal ratio (in many cases, the figure after deformation is generally slimmer than the figure before transformation). As described above, according to the present embodiment, only by the user specifying the target image, the printer 100 estimates a rectangle (body shape region) corresponding to the body shape of the person including the face image in the target image in the target image. Calculating a correction value for making the aspect ratio of the body shape region substantially coincide with the ideal ratio, and when deforming the target region in the deformation direction, the first region including the body shape region is deformed according to the correction value. Apply. Therefore, the user can obtain an image in which the person image is automatically corrected to an ideal body shape. Further, since the image deformation process to be executed is only compression and expansion along one direction (deformation direction) of the target image, the amount of calculation processing is very small and the burden on the printer 100 is small.

3. Other Embodiments In the above description, it is assumed that the ideal ratio used in S600 is a ratio that represents a body shape that suits market research and user preferences, that is, a body shape that the user's desire has intervened to some extent (for example, a slim body shape). It was. However, the ideal ratio is not limited to the above. A person may get an impression from an image of the same subject that is different from the impression of looking directly at the subject. Especially when the subject is viewed as an image, the impression is more real It may appear “thick” rather than the impression. Therefore, in the other embodiment (hereinafter referred to as the embodiment), the “ideal ratio” is assumed to be the aspect ratio of the subject perceived when a human directly visually recognizes the subject with both eyes. In this embodiment, a description will be given of a case where, in S600, the correction value C ′ for making the aspect ratio of the body shape area substantially coincide with the aspect ratio of the body shape perceived when a person directly views the subject (person) is described. . The contents other than S600 are basically the same as described above.

  FIG. 11 is a flowchart showing the process executed by the correction value acquisition unit 240 in S600 in the embodiment. In S610, the correction value acquisition unit 240 acquires the shooting distance Sd and the face width Wf of the subject (person P1). The shooting distance Sd means the distance from the digital still camera (specifically, the principal point of the lens of the digital still camera) to the person P1 when the target image is shot by the digital still camera. The shooting distance Sd is calculated or acquired at the time of shooting with a digital still camera, and is recorded on a memory card MC in the digital still camera as a kind of attached information together with image data representing the target image. Therefore, on the printer 100 side, in a situation where such a memory card MC is inserted into the card slot 172, the shooting distance Sd is acquired from the attached information recorded together with the image data of the target image on the memory card MC. be able to. The face width Wf is a value (for example, 200 mm) indicating a standard human face width (actual face size), and is stored in advance in the internal memory 120 or the like. The value of the face width Wf may be different depending on each age group such as infants, children, and adolescents or more, and the correction value acquisition unit 240 corresponds to the age group estimated in S300. The face width Wf may be acquired.

  In S620, the correction value acquisition unit 240 acquires the binocular distance DE. The binocular distance DE is a standard human distance between both eyes, for example, a numerical value such as 60 mm. Both eyes distance DE is also stored in the internal memory 120 in advance. Similarly to the face width Wf, the binocular distance DE may store a different value depending on each age group, and the correction value acquisition unit 240 calculates the binocular distance DE corresponding to the age group estimated in S300. It may be acquired. In S630, the correction value acquisition unit 240 performs the geometric calculation based on the information (Sd, Wf) acquired in S610, and the size of the person in the target image captured through the monocular (referred to as the monocular perception size SM). ) Is calculated. Monocular means a lens of a digital still camera. In S640, the correction value acquisition unit 240 performs the geometric calculation based on the information (Sd, Wf, DE) acquired in S610 and S620, and the size of the person in the target image perceived by the left and right human eyes. (Binocular perception size SB) is calculated.

  FIG. 12 illustrates the positional relationship among the lens 502 as a single eye, the human left and right eyes (right eye RE, left eye LE), the subject OB (person's face), and the virtual imaging plane S1 from above. Represented in two dimensions. The lens 502 (the principal point of the lens 502) and the center point C of the subject OB are located on the Y axis, which is the central axis in FIG. The distance between the lens 502 and the center point C is Sd. In this embodiment, the subject OB is assumed to be a sphere for easy calculation. The radius r of the subject OB is Wf / 2. The right eye RE and the left eye LE are positioned on both sides of the lens 502 with their midpoints coincident with the principal point of the lens 502 and separated from each other by DE. The arrangement of the right eye RE, the lens 502, and the left eye LE is parallel to the X axis. The imaging plane S1 is a plane parallel to the X axis, and is on the opposite side of the subject OB across the plane (observation plane S2) in which the right eye RE, the lens 502, and the left eye LE are arranged. The distance d1 between the observation surface S2 and the imaging surface S1 is not particularly limited. The correction value acquisition unit 240 may set an arbitrary value (distance) as a fixed value as the distance d1 when calculating the monocular perception size SM and the binocular perception size SB.

  In such a positional relationship, in S630, the correction value acquisition unit 240 draws two tangent lines of the subject OB that intersects at the position of the lens 502. A range between the two contact points AS and BS between the two tangents and the subject OB is a lateral width of the subject OB captured through the lens 502. Therefore, the correction value acquisition unit 240 calculates the distance in the X-axis direction range between the two intersections of the two tangents and the imaging plane S1 as the monocular perception size SM. In S640, the correction value acquisition unit 240 first intersects the two tangents of the subject OB intersecting at the position of the right eye RE and the position of the left eye LE as a preceding process for obtaining the binocular perception size SB. The two tangents of the subject OB to be drawn are respectively drawn. The range between the two contact points AR and BR between the two tangents intersecting at the position of the right eye RE and the subject OB is the lateral width of the subject OB perceived by the right eye RE. Therefore, the correction value acquisition unit 240 calculates the distance in the X-axis direction range between the two intersections between the two tangents that intersect at the position of the right eye RE and the imaging plane S1, as the right eye perception range SR. . Similarly, the range between the two contact points AL and BL between the two tangents intersecting at the position of the left eye LE and the subject OB is the lateral width of the subject OB perceived by the left eye LE. Therefore, the correction value acquisition unit 240 calculates the distance in the X-axis direction range between the two intersections between the two tangents that intersect at the position of the left eye LE and the imaging plane S1, as the left eye perception range SL. .

  As described above, there is a shift in the left-right direction in the range of the subject perceived by the right eye RE and the left eye LE. Here, when a human sees an object with both eyes, a common range when the image of the range perceived by the right eye RE and the image of the range perceived by the left eye LE are superimposed is set as the size of the object. Presumed to recognize. Therefore, in this embodiment, the length of the common range when the calculated left-eye perception range SL and right-eye perception range SR are overlapped is set as the binocular perception size SB. In the superimposition process, the position of the gazing point PG on the subject OB is used. The gazing point PG is a point that is estimated to be most noticeable when a human views the subject OB. In this embodiment, the point of interest PG is a point on the subject OB that can be seen from both the right eye RE and the left eye LE. Thus, the distance from the right eye RE is equal to the distance from the left eye LE. The correction value acquisition unit 240 sets the intersection position between the image line S1 and the straight line passing through the gazing point PG and the right eye RE as a point PGr on the right eye perception range SR corresponding to the gazing point PG, An intersection position between the straight line passing through the left eye LE and the imaging plane S1 is set as a point PGl on the left eye perception range SL corresponding to the gazing point PG. Then, the left eye perception range SL and the right eye perception range SR are overlapped so that the point PGr and the point PGl coincide with each other.

  FIG. 13 shows a state in which the left-eye perception range SL and the right-eye perception range SR are overlapped so that the point PGr and the point PGl coincide with each other, and at this time, a range X common to both the ranges SL and SR is shown. The correction value acquisition unit 240 recognizes the axial distance as the binocular perception size SB. The binocular perception size SB obtained in this way is shorter than the monocular perception size SM obtained in S630. Accordingly, it can be said that when a person observes the image of the subject OB imaged through the lens 502, he / she has an impression that the lateral width is wider (thick) than when the subject OB is directly observed. In S650, the correction value acquisition unit 240 calculates the ratio (SB / SM) of the binocular perception size SB to the monocular perception size SM, and sets the ratio (SB / SM) as the correction value C ′ (C ′ = SB). / SM). Thus, in this embodiment, the binocular perception size SB is the length of the common range obtained by superimposing the right eye perception range SR and the left eye perception range SL on the basis of the gazing point PG of the subject OB. Therefore, the ratio of the binocular perception size SB to the monocular perception size SM is such that the subject OB (person) perceived by a person when observing an image of the subject OB (target image including a person) captured by a monocular lens at a subject distance Sd. It can be said that the ratio of the horizontal direction of the subject OB (person) perceived when a person directly observes the subject OB (person) at the subject distance Sd with respect to the horizontal width of) is accurately expressed.

  After the correction value C ′ is calculated in S600, the unidirectional deformation unit 250 performs a unidirectional deformation process (S700) on the target image. At this time, the unidirectional deformation unit 250 performs a deformation process in which the correction value C ′ is applied to the first region including the body region. That is, since the correction value C ′ is the magnification itself, the deformation process to which the correction value C ′ is applied is performed on the first area, so that the first area is compressed at the magnification indicated by the correction value C ′ in the deformation direction. The The unidirectional deformation unit 250 enlarges the second region at a predetermined magnification. As described above, according to the embodiment, in S600, the ratio (SB / SM) of the binocular perception size SB to the monocular perception size SM is obtained as the correction value C ′, and the first region in the target image is determined by the correction value C ′. Compress in the deformation direction. Therefore, the width of the person image included in the first region is also compressed, and as a result, the aspect ratio of the person image is substantially the same as the aspect ratio of the figure perceived when the user directly views the person. Become. In this embodiment, the purpose is to give an impression similar to the impression that the user sees when the subject is actually seen when the user views the image, so the age of the person in the estimated target image The unidirectional deformation process is not performed depending on the layer and sex, and the correction value C ′ calculated as the ratio of the binocular perception size SB to the monocular perception size SM is further determined according to the age group and sex. Do not make any changes.

FIG. 2 is an explanatory diagram schematically illustrating a configuration of a printer as an image processing apparatus. It is the flowchart which showed an example of the image correction process. It is explanatory drawing which showed an example of UI. It is explanatory drawing which illustrated a mode that the face area | region was detected from the target image. It is explanatory drawing which illustrated the exclusion area | region in a target image. It is explanatory drawing which illustrated a mode that the width | variety of the outline of a face was detected. It is explanatory drawing which illustrated the body type area | region estimated in the target image. It is the figure which illustrated a mode that a candidate picture was changed by one way deformation processing. It is the flowchart which showed the detail of the unidirectional deformation process. It is explanatory drawing which showed the one-way deformation | transformation process typically. It is the flowchart which showed the detail of the correction value acquisition process. It is explanatory drawing which shows the mode of calculation of monocular perception size and binocular perception size. It is explanatory drawing which shows a monocular perception size and a binocular perception size.

Explanation of symbols

DESCRIPTION OF SYMBOLS 100 ... Printer, 110 ... CPU, 120 ... Internal memory, 140 ... Operation part, 150 ... Display part, 160 ... Printer engine, 170 ... Card interface, 172 ... Card slot, 200 ... Image correction process part, 210 ... Face area detection , 220 ... Exclusion determination part, 230 ... Body shape estimation part, 240 ... Correction value acquisition part, 250 ... Unidirectional deformation part, 260 ... Age gender estimation part, 310 ... Display processing part, 320 ... Print processing part, 410 ... Correspondence Pixel number table

Claims (8)

A body type estimation unit for estimating a body region corresponding to a body shape of a person including the face image in the target image including the face image;
A correction value acquisition unit for acquiring a correction value for making the aspect ratio of the body region substantially coincide with the ideal ratio;
An image processing apparatus comprising: an image deformation unit configured to deform at least a region of the target image including the body shape region in a specific direction according to the correction value.   The image processing according to claim 1, wherein the body shape estimation unit extracts a width of the feature portion of the person from the target image, and estimates the body shape region based on the width of the extracted feature portion. apparatus.   The image processing apparatus according to claim 2, wherein the body type estimation unit extracts a width of a person's cheek or shoulder or waist as the characteristic part.   An age gender estimating unit for estimating the age group and / or gender of the person is provided, and the correction value acquiring unit changes the acquired correction value according to the estimated age group and / or gender. The image processing apparatus according to any one of claims 1 to 3.   The correction value acquisition unit acquires a correction value for making the aspect ratio of the body shape area substantially coincide with the aspect ratio of the body shape of the person perceived when the person is directly visually recognized by both eyes of the observer. The image processing apparatus according to claim 1, wherein the image processing apparatus is an image processing apparatus.   When the orientation, inclination, and position of the face image in the target image are within the allowable ranges set for the orientation, inclination, and position, respectively, estimation of the body region, acquisition of correction values, and deformation of the region The image processing apparatus according to claim 1, wherein: A body shape estimation step for estimating a body shape region corresponding to the body shape of the person including the face image in the target image including the face image;
A correction value acquisition step for acquiring a correction value for making the aspect ratio of the body region substantially coincide with the ideal ratio;
An image processing method comprising: an image deformation step of deforming at least a region of the target image including the body shape region in a specific direction according to the correction value. A body shape estimation function for estimating a body shape region corresponding to a body shape of a person including the face image in a target image including a face image;
A correction value acquisition function for acquiring a correction value for making the aspect ratio of the body region substantially coincide with the ideal ratio;
An image processing program for causing a computer to execute an image deformation function for deforming a region of the target image including at least the body shape region in a specific direction according to the correction value.

Images