JP2007034724A - Image processor, image processing method and image processing program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To easily generate a deformation image by reflecting the feature of an object image. <P>SOLUTION: A predetermined face image is previously stored, as a reference image, in an image processor, which uses a predetermined corresponding point retrieval method as an input image is input to obtain a corresponding point between the input image and the reference image, and automatically calculates a deformation vector for each corresponding point of the reference image from each corresponding point of the input image. Then, an intermediate point for each of the automatically calculated deformation vectors is set as a deformation destination, and the input image is deformed toward the such deformation destination to generate a deformation image. Furthermore, by normalizing the pixel value of each pixel in the deformation image by the mean value and variance value in an adjacent local area, or the like, a deformation image like an illustration is generated. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to an image processing apparatus, an image processing method, and an image processing program that generate a deformed image by deforming a target image that is a processing target.

  2. Description of the Related Art Conventionally, there has been known a technique for generating a deformed image by deforming a target image that is a processing target, such as generating a portrait or illustration from a face image captured by a digital camera or the like. For example, Patent Literature 1 (Japanese Patent Laid-Open No. 2002-175538), Patent Literature 2 (Japanese Patent Laid-Open No. 2004-120108), Patent Literature 3 (Japanese Patent Laid-Open No. 10-75414), Patent Literature 4 (Japanese Patent Laid-Open No. 10-5673). No. gazette) has a technology for detecting the position of each part such as eyes, nose and mouth from a face image, and generating a portrait image by manually pasting the image of each part prepared in advance to the detected position. It is disclosed.

JP 2002-175538 A JP 2004-120108 A JP 10-75414 A Japanese Patent Laid-Open No. 10-5673

  However, as described below, the above-described conventional techniques have a problem that the operation of generating a deformed image is troublesome and a problem that the characteristics of the original image cannot always be reflected in the deformed image.

  That is, in the above-described conventional technique, it is necessary to manually paste the image of each part, and thus there is a problem that the operation of generating a deformed image is troublesome. Also, with this conventional technique, there is a problem in that the characteristics of the original image cannot always be reflected in the deformed image because the part image corresponding to the original image is not always pasted.

  As a technique for generating a deformed image by deforming a target image, a corresponding point (deformation vector) is manually obtained between two face images (for example, a face along the corresponding point). A method for generating a caricature image by morphing an image) is also known, but in this face image morphing process, it is necessary to manually obtain corresponding points (deformation vectors) between the face images. There is a problem that the generation work is troublesome.

  Accordingly, the present invention has been made to solve the above-described problems of the related art, and an image processing apparatus, an image processing method, and an image processing apparatus capable of easily generating a deformed image reflecting the characteristics of the target image. An object is to provide an image processing program.

  In order to solve the above-described problems and achieve the object, the invention according to claim 1 is an image processing apparatus that generates a deformed image by deforming a target image to be processed, and uses a predetermined corresponding point search method. A corresponding vector between the target image and a predetermined reference image, and a deformation vector calculating means for calculating a deformation vector for the reference image from the target image; and a deformation vector calculated by the deformation vector calculating means. And a deformed image generating means for generating the deformed image by deforming the target image based on the image.

  In the invention according to claim 2, in the above invention, the deformation vector calculation unit calculates a deformation vector for the average image using an average image in which the average of the processing target is expressed as the reference image. The deformed image generating means generates the deformed image by deforming the target image in the plus direction of the deformation vector calculated with respect to the average image.

  In the invention according to claim 3, in the above invention, the deformation vector calculation means calculates a deformation vector for the average image using an average image in which the average of the processing target is expressed as the reference image. The deformed image generating means generates the deformed image by deforming the target image in the negative direction of the deformation vector calculated for the average image.

  According to a fourth aspect of the present invention, in the above-described invention, in the modified image generated by the modified image generating means, the pixel value of each pixel is normalized with an average value and a variance value in a nearby local region. The image processing apparatus further includes an average dispersion normalized image generating unit that generates an average dispersion normalized image of the deformed image.

  In the invention according to claim 5, in the above invention, in the mean variance normalized image generated by the mean variance normalized image generating means, the value of a pixel having a plus component is converted to zero to change the deformation. Further, a negative component image generating means for generating a negative component image of the image is further provided.

  According to a sixth aspect of the present invention, in the above invention, the minus component image generating means converts a plus component pixel to zero and sets a value of a pixel having a minus component less than a predetermined value to the predetermined value. A negative component image of the deformed image is generated by converting into a value.

  According to a seventh aspect of the present invention, in the above invention, a filter is applied to the average dispersion normalized image generated by the average dispersion normalized image generation means or the negative component image generated by the negative component image generation means. The image processing apparatus further includes a filter processing means for performing processing.

  According to an eighth aspect of the present invention, there is provided an image processing method for generating a deformed image by deforming a target image to be processed, the target image and a predetermined reference image using a predetermined corresponding point search technique. A corresponding vector between the target image and a deformation vector calculating step of calculating a deformation vector for the reference image from the target image, and deforming the target image based on the deformation vector calculated by the deformation vector calculating step. A modified image generation step of generating a modified image.

  The invention according to claim 9 is an image processing program for causing a computer to execute an image processing method for generating a deformed image by deforming a target image that is a processing target, using a predetermined corresponding point search method. A corresponding vector is obtained between the target image and a predetermined reference image, a deformation vector calculation procedure for calculating a deformation vector for the reference image from the target image, and the deformation vector calculated by the deformation vector calculation procedure. A modified image generation procedure for generating the modified image by deforming the target image is executed by a computer.

  According to the first, eighth, or ninth invention, a corresponding point is obtained between the target image and the predetermined reference image using a predetermined corresponding point search method, and a deformation vector for the reference image is calculated from the target image. Since the deformed image is generated by deforming the target image based on the deformed vector, the deformed vector (corresponding point) is automatically calculated between the target image and the reference image, and the target image is converted into the deformed vector. As a result of being deformed based on this, it is possible to easily generate a deformed image reflecting the characteristics of the target image.

  According to the second aspect of the present invention, the average image expressing the average of the processing target is used as the reference image, the deformation vector for the average image is calculated, and the target image is deformed in the plus direction of the deformation vector. Therefore, for example, by calculating a deformation vector using the average face image as a reference image, and generating a portrait image by deforming the face image in the plus direction of the deformation vector, It is also possible to generate a caricature image deformed on the taken face.

  According to the invention of claim 3, a deformation vector for the average image is calculated using the average image representing the average of the processing target as a reference image, and the target image is deformed in the negative direction of the deformation vector. Therefore, for example, by calculating a deformation vector using the average face image as a reference image, and generating a portrait image by deforming the face image in the minus direction of the deformation vector, It is also possible to generate a portrait image in which personal features are emphasized.

  Further, according to the invention of claim 4, since the average dispersion normalized image of the modified image is generated by normalizing the pixel value of each pixel in the modified image with the average value and the variance value in the local region in the vicinity, Compared to the binarized gradation conversion process, the gradation conversion process is performed in such a way that the features do not fade, and as a result, an average dispersion normalized image (deformed image) reflecting the characteristics of the target image is generated. Is also possible.

  According to the fifth aspect of the present invention, since the minus component image of the deformed image is generated by converting the value of the pixel having the plus component in the average dispersion normalized image to zero, the gradation conversion for simple edge enhancement is performed. Compared to the processing, a deformed image in which not only the edge portion but also the shadow portion is emphasized is generated. As a result, it is possible to generate a realistic deformed image.

  According to the invention of claim 6, the negative component image of the deformed image is converted by converting the pixel of the positive component to zero and converting the value of the pixel having the negative component less than the predetermined value into the predetermined value. Therefore, it is also possible to generate an illustration-like deformed image in which the number of gradations is reduced while reducing the amount of information.

  In addition, according to the invention of claim 7, since the filtering process is performed on the average dispersion normalized image and the negative component image, for example, by passing the median filter or the integration filter, it is possible to finely control the edge spread. As a result of removing the noise, a more illustration-like deformed image can be generated.

  Exemplary embodiments of an image processing apparatus, an image processing method, and an image processing program according to the present invention will be described below in detail with reference to the accompanying drawings.

  In the following first embodiment, an example in which the present invention is applied to a portrait generation process for generating a portrait image from a face image will be described. Specifically, after describing the outline and features of the image processing apparatus according to the first embodiment, the configuration of the image processing apparatus, the flow of image processing (caricature generation processing), the flow of deformation vector calculation processing, and the effects of the first embodiment Will be described in order.

[Outline and Features of Image Processing Device]
First, the outline and features of the image processing apparatus according to the first embodiment will be described with reference to FIG. FIG. 1 is a diagram for explaining the outline and features of the first embodiment.

  The image processing apparatus according to the first embodiment generally accepts a face image captured by a digital camera or the like as an input image, and deforms the face image to generate a portrait image. The image processing apparatus is mainly characterized in that a portrait image reflecting the person's characteristics can be easily generated, as will be briefly described below.

  That is, as shown in FIG. 1, in the image processing apparatus according to the first embodiment, a predetermined face image is stored in advance as a reference image, and when an input image is input, a predetermined corresponding point search method is used. Then, a deformation vector for the reference image is calculated from the input image. That is, a corresponding point is obtained between the input image and the reference image using a predetermined corresponding point search method, and a deformation vector for each corresponding point of the reference image is automatically calculated from each corresponding point of the input image.

  Thereafter, the image processing apparatus morphs the input image based on the automatically calculated deformation vector. That is, for example, an intermediate point of a deformation vector is set as a deformation destination, and a deformed image is generated by deforming the input image toward the deformation destination. Further, in this image processing device, a gradation conversion process is performed on the deformed image to generate a portrait image. That is, an illustration-like deformed image is generated by, for example, normalizing the pixel value of each pixel in the deformed image with the average value and the variance value in the nearby local region.

  Therefore, according to the image processing apparatus according to the first embodiment, the deformation vector (corresponding point) is automatically calculated between the input image and the reference image, and the input image is deformed based on the deformation vector. Like the main feature described above, it is possible to easily generate a portrait image that reflects the user's feature. In the embodiment, the “face image (input image)” corresponds to the “target image” described in the claims, and the “caricature image” also includes “deformed image” and “ It corresponds to “average variance normalized image” and “minus component image”.

[Configuration of image processing apparatus]
Subsequently, the configuration of the image processing apparatus according to the first embodiment will be described with reference to FIGS. FIG. 2 is a block diagram illustrating the configuration of the image processing apparatus according to the first embodiment, FIG. 3 is a diagram illustrating an example of an input image and a reference image, and FIG. 4 is a diagram illustrating an example of a deformation vector. FIG. 5 is a diagram illustrating an example of the deformation destination of the input image, FIG. 6 is a diagram illustrating an example of the deformed image, and FIG. 7 is a diagram illustrating an example of the gradation conversion image. 8 is a diagram illustrating an example of a portrait, FIG. 9 is a diagram illustrating a conversion formula used for generating an average dispersion normalized image, and FIG. 10 is a diagram for describing generation of a negative component image. .

  As illustrated in FIG. 2, the image processing apparatus 10 according to the first embodiment includes an image input unit 11, an image output unit 12, an image input / output processing unit 13, a storage unit 14, a morphing unit 15, and a gradation. And a conversion unit 16.

  Among these, the image input unit 11 is a unit that inputs a target image (here, a face image) that is a processing target to the image processing apparatus 10, for example, from an imaging unit that captures and inputs a face image, or from a network. An interface unit that receives and inputs an image and an interface unit that reads and inputs an image from an external secondary storage device correspond to this. The face image input by the image input unit 11 is stored in the storage unit 14 as an input image.

  The image output unit 12 is a means for outputting various face images stored in the storage unit 14 of the image processing apparatus 10 (for example, a portrait image generated by a morphing unit 15 or a gradation conversion unit 16 described later). There are, for example, a monitor (or display) that displays and outputs a face image, an interface unit that transmits and outputs an image to a network, and an interface unit that outputs an image to an external secondary storage device. It corresponds to.

  The image output processing unit 13 is a processing unit that outputs an image to the image output unit 12. Specifically, according to the processing status of the morphing unit 15 and the gradation conversion unit 16, the image currently processed, the final processed image, and the like are read from the storage unit 14 and output to the image output unit 12, Further, the selection of an image to be output is received from the user, and the selected image is read from the storage unit 14 and output to the image output unit 12.

  The storage unit 14 is a storage unit (storage unit) that stores data and programs necessary for various processes performed by the morphing unit 15 and the gradation conversion unit 16 to be described later. In particular, the storage unit 14 is a modification that is closely related to the present invention. Reference image used for vector calculation (see FIG. 3), input image input from image input unit 11 (see FIG. 3), modified image generated by morphing unit 15, average variance generated by gradation converting unit 16 Normalized images, minus component images, filtered images, etc. are stored. The reference image is not limited to the case where the reference image is stored in advance in the storage unit 14 prior to input of the input image, and may be input together with the input image.

  The morphing unit 15 is a processing unit that generates a deformed image by deforming an input image. Particularly, as closely related to the present invention, as shown in FIG. 2, a deformed vector calculating unit 15a and a deformed destination setting Unit 15b and a modified image generation unit 15c. The deformation vector calculation unit 15a corresponds to “deformation vector calculation means” recited in the claims, and the deformation destination setting unit 15b and the deformation image generation unit 15c also correspond to “deformation image generation means”.

  Among the morphing units 15, the deformation vector calculation unit 15a calculates a deformation vector for a predetermined reference image (a reference image stored in the storage unit 14) from an input image using a predetermined corresponding point search method. It is. Specifically, when an input image is input from the image input unit 11 (or when an input image to be processed is selected from among various images stored in the storage unit 14), the deformation vector The calculation unit 15a searches for corresponding points between the input image and the reference image using a predetermined corresponding point search method, and automatically calculates a deformation vector for each corresponding point of the reference image from each corresponding point of the input image. (See FIGS. 3 and 4).

  The deformation destination setting unit 15b is a processing unit that sets the deformation destination of the input image based on the deformation vector calculated by the deformation vector calculation unit 15a. Specifically, for example, as shown in FIG. 5, the intermediate point of the deformation vector calculated by the deformation vector calculation unit 15a is set as the deformation destination of the input image.

  The deformed image generation unit 15c is a processing unit that generates a deformed image by deforming the input image. Specifically, a deformed image is generated by deforming (that is, morphing) the input image toward the deformation destination (for example, the midpoint of the deformation vector) set by the deformation destination setting unit 15b (see FIG. 6). ).

  The gradation converting unit 16 is a processing unit that converts the gradation of the deformed image generated by the morphing 15, and particularly as closely related to the present invention, as shown in FIG. A generation unit 16a, a negative component image generation unit 16b, and a filter unit 16c are provided. The average dispersion normalized image generation unit 16a corresponds to the “average dispersion normalized image generation unit” described in the claims, and the minus component image generation unit 16b similarly corresponds to the “minus component image generation unit”. The filter unit 16c also corresponds to “filter processing means”.

  Among the gradation conversion units 16, the average dispersion normalized image generation unit 16 a is a processing unit that generates an average dispersion normalized image of the modified image from the modified image generated by the morphing unit 15. Specifically, in the modified image generated by the morphing unit 15 according to the conversion formula shown in FIG. 9, the average value of the modified image is obtained by normalizing the pixel value of each pixel with the average value and the variance value in the local region in the vicinity. A distributed normalized image is generated.

  The minus component image generation unit 16b is a processing unit that generates a minus component image from the average dispersion normalized image generated by the average dispersion normalized image generation unit 16a. Specifically, according to the conversion rule shown in FIG. 10, in the average dispersion normalized image generated by the average dispersion normalized image generation unit 16a, the value of the pixel having a plus component is converted to zero, and a predetermined value ( For example, a negative component image of a deformed image is generated by converting the value of a pixel having a negative component less than “−kσ”) into the predetermined value. Note that the value of a pixel having a negative component equal to or greater than a predetermined value (for example, “−kσ”) is not converted as shown in FIG.

  The filter unit 16c is a processing unit that performs a filtering process on the negative component image generated by the negative component image generation unit 16b. Specifically, by applying a filter represented by a median filter or an integration filter to the negative component image, a filtered image from which fine noise is removed while suppressing the spread of the edge is generated.

  In this way, the gradation of the modified image as shown in FIG. 6 is converted by each unit of the gradation conversion unit 16 described above, so that the gradation converted image as shown in FIG. A portrait image) is generated. Note that the example of the portrait shown in FIG. 8 is also generated by the processing by the morphing unit 15 and the gradation converting unit 16 described above.

  By the way, although it is the image processing apparatus 10 which concerns on Example 1 mentioned above, information processing apparatuses, such as a known personal computer, a workstation, a mobile telephone, a PHS terminal, a mobile communication terminal, or PDA, a digital camera, a digital television It can also be realized by mounting each function of each of the above-described information appliances.

[Flow of image processing (caricature generation processing)]
Next, the flow of image processing (caricature generation processing) according to the first embodiment will be described with reference to FIG. FIG. 11 is a flowchart illustrating the flow of image processing according to the first embodiment.

  As shown in the figure, in the image processing apparatus 10 according to the first embodiment, when a face image serving as an input image is input from the image input unit 11 (step S101), a predetermined corresponding point search method is used. A deformation vector for a predetermined reference image is calculated from the input image (step S102). Such deformation vector calculation processing will be described later with reference to FIG.

  When the deformation vector is calculated, the image processing apparatus 10 sets a deformation destination of the input image based on the deformation vector (step S103). Specifically, an intermediate point of the deformation vector is set as a deformation destination of the input image. Following this, the image processing apparatus 10 generates a deformed image by deforming the input image (step S104). Specifically, a deformed image is generated by morphing the input image toward the deformation destination (for example, the intermediate point of the deformation vector) set in step S104.

  Thereafter, the image processing apparatus 10 generates an average dispersion normalized image of the deformed image (step S105). Specifically, in the modified image generated in step S104 described above, the average dispersion normalized image of the modified image is generated by normalizing the pixel value of each pixel with the average value and the variance value in the local region in the vicinity. . Then, the image processing apparatus 10 generates a minus component image from the average dispersion normalized image (step S106). Specifically, in the average dispersion normalized image generated in step S105 described above, the value of the pixel having a positive component is converted to zero and has a negative component less than a predetermined value (for example, “−kσ”). By converting the pixel value to the predetermined value, a negative component image of the deformed image is generated.

  Following this, the image processing apparatus 10 performs a filtering process on the negative component image (step S107). Specifically, by applying a filter typified by a median filter or an integration filter to the negative component image generated in step S106, a filter that removes fine noise while controlling the spread of edges. Generate finished images. Through the series of processes described above, a portrait image as shown in FIG. 8 is generated.

[Deformation vector calculation process flow]
Next, the flow of the deformation vector calculation process (the process executed in step S102 shown in FIG. 11) will be described with reference to FIG. FIG. 12 is a flowchart showing the flow of the deformation vector calculation process. Note that the processing described below is an example of a corresponding point search method used by the image processing apparatus 10 according to the first embodiment, and the present invention is similarly applied even when a different corresponding point search method is used. Can do.

  As shown in the figure, in the image processing apparatus 10, when a face image as an input image is input from the image input unit 11 and a reference image is read from the storage unit 14, the reference image is divided into reference partial images. (Step S201). For example, a reference image of 32 × 32 pixels is divided into 25 reference partial images composed of 7 × 7 pixels. Further, the image processing apparatus 10 divides the input image into input partial images (step S202). For example, the input image 22 having 32 × 32 pixels is divided into 25 input partial images having 21 × 21 pixels.

  Then, the image processing apparatus 10 creates a similarity image between the reference partial image and the input partial image (step S203). For example, the similarity image having the pixel portion as the similarity between the input partial image and the reference partial image is created for each input partial image to create a plurality of similarity images. Subsequently, the image processing apparatus 10 detects a position where the pixel value of each similarity image is maximum (position of the maximum value) (step S204).

  Thereafter, the image processing apparatus 10 performs cumulative addition processing in four directions (that is, cumulative addition processing in the j direction that is the horizontal direction of the image, cumulative addition processing in the −j direction that is the reverse direction of the horizontal direction, and vertical direction of the image) The i-direction cumulative addition process and the -i direction cumulative addition process opposite to the vertical direction) are performed independently (step S205), and the j, -j, i, -i direction cumulative addition is performed. Each similarity image is updated by adding the maximum value of 9 pixels within 8 neighborhoods of each pixel of the similarity image to each pixel of the similarity image created in step S203 (step S206). The position of the maximum value of the updated similarity image is detected (step S207).

  Then, the image processing apparatus 10 calculates the position fluctuation of the maximum value of each similarity image (step S208), and checks whether or not the position fluctuations of all the maximum values are within a predetermined value (step S209). If it is not within the value (No at Step S209), the process proceeds to Step S205 and the same processing is repeated. If it is within the predetermined value (Yes at Step S209), this position is determined as a corresponding point (Step S210). Note that the processing in step S209 described above is processing for checking whether or not the change from the position of the maximum value of each previous similarity image to the position of the maximum value of each similarity image is within a predetermined value. If it is within the predetermined value, in step S210, the position of the maximum value of each similarity image after update processing is determined as the corresponding point of the input image with respect to the reference image.

  After that, the image processing apparatus 10 connects the corresponding points of the images obtained in step S210 described above, and calculates a deformation vector for the corresponding points of the reference image from the corresponding points of the input image (step S211). . In the above description, for convenience of explanation, an example in which j-direction cumulative addition processing, -j-direction cumulative addition processing, i-direction cumulative addition processing, and -i-direction cumulative addition processing are sequentially performed has been described. Therefore, parallel processing may be performed.

[Effect of Example 1]
As described above, according to the first embodiment, the morphing unit 15 calculates a deformation vector for a predetermined reference image that is also a face image from an input image that is a face image by using a predetermined corresponding point search method. Since a deformed image is generated by deforming the input image based on the deformed vector, a deformed vector (corresponding point) is automatically calculated between the input image and the reference image, and the input image is converted into a deformed vector. As a result of being deformed based on this, it is possible to easily generate a deformed image reflecting the characteristics of the input image. In addition, there is a problem that various problems occur as leakage of personal information if the face image itself captured by a digital camera or the like is distributed. However, according to the present invention, the characteristics of the person are reflected. Since a portrait can be easily generated, the use of such a portrait can provide an advantage that an individual can be sufficiently identified and recognized while solving such a problem.

  Further, according to the first embodiment, the tone conversion unit 16 normalizes the pixel value of each pixel in the modified image generated by the morphing unit 15 with the average value and the variance value in the local region in the vicinity, thereby transforming the modified image. Compared to simple binarized gradation conversion processing, the gradation conversion processing is performed in a form that does not fade the characteristics, resulting in an average that reflects the characteristics of the input image. It is also possible to generate a dispersion normalized image (deformed image). That is, it is possible to generate a portrait that reflects the person's characteristics as compared with a simple binarized gradation conversion process.

  Further, according to the first embodiment, the tone conversion unit 16 generates a negative component image of the deformed image by converting the value of the pixel having the positive component in the average dispersion normalized image to zero, so that the simple edge Compared to the enhanced gradation conversion process, a deformed image in which not only the edge portion but also the shadow portion is emphasized is generated, so that a realistic deformed image can be generated. That is, it is possible to generate a realistic portrait in comparison with a simple edge-enhanced gradation conversion process.

  Further, according to the first embodiment, when the gradation conversion unit 16 generates a negative component image, the pixel of the positive component is converted to zero and a negative component less than a predetermined value (for example, “−kσ”) is converted. Since the negative component image of the deformed image is generated by converting the pixel value to the predetermined value, it is also possible to generate an illustration-like deformed image with a reduced number of gradations while reducing the amount of information. . That is, it is possible to generate an illustration-like portrait after reducing the number of gradations while reducing the amount of information.

  Further, according to the first embodiment, since the gradation conversion unit 16 performs filtering on the negative component image, for example, through a median filter or an integration filter, fine noise is controlled while controlling the spread of the edge. As a result of the removal, a more illustration-like deformed image can be generated. That is, it is possible to generate a more illustration-like caricature.

  The image processing apparatus according to the first embodiment has been described so far, but the present invention may be implemented in various different forms other than the above-described embodiments. Therefore, various different embodiments will be described below as a second embodiment.

(1) Reference image and change destination For example, the reference image (see FIG. 3) shown in the above embodiment is an example, and the present invention is not limited to this. The deformation vector may be calculated using the described “average image”) as a reference image.

  In the above-described embodiment, the case where the intermediate point of the deformation vector is set as the change destination has been described. However, the present invention is not limited to this, and any numerical value (plus or minus) can be used for the deformation vector. Alternatively, the position obtained by multiplying by 1 or more and -1 or less may be set as the change destination.

  That is, for example, as shown in FIG. 13, while calculating the deformation vector using the above average face as a reference image, the input image is deformed in the plus direction (average face direction) of the deformation vector. Thus, a deformed image may be generated. In this way, if a face image is generated by deforming the face image in the plus direction of the deformation vector calculated using the average face as a reference image, a caricature image deformed to a well-balanced face is generated. Is also possible.

  Furthermore, for example, as shown in FIG. 14, while calculating the deformation vector using the above average face as a reference image, the input image is displayed in the minus direction of the deformation vector (the direction opposite to the average face). You may make it produce | generate a deformation | transformation image by making it deform | transform. In this way, if a face image is generated in the negative direction of the deformation vector calculated using the average face as a reference image to generate a portrait image, a portrait image in which the individual characteristics of the person are emphasized may be generated. Is possible.

  Further, when the change destination is set based on the deformation vector, the change destination may be set by multiplying different numerical values for each deformation vector calculated for each corresponding point. In other words, for example, the input image is deformed for each corresponding point, such as setting the minus direction of the deformation vector as the change destination for the eyes and setting the plus direction of the deformation vector as the change destination for the mouth. Also good. In this way, it is also possible to generate a portrait image in which the personal characteristics of the person are emphasized only by predetermined parts in the face.

(2) Omission of processing steps In the above embodiment, an image processing apparatus that performs morphing by the morphing unit 15 and gradation conversion by the gradation converting unit 16 has been described. However, the present invention is not limited to this. Alternatively, an image processing apparatus that performs only morphing by the morphing unit 15 on the input image may be configured, or an image processing apparatus that performs only tone conversion by the gradation converting unit 16 on the input image may be configured. May be. That is, it is possible to easily generate a deformed image reflecting the characteristics of the input image only by morphing by the morphing unit 15 or only by gradation conversion by the gradation converting unit 16.

  In the above embodiment, the case where the average dispersion normalized image generation, the negative component image generation, and the filter processing are performed as the gradation conversion by the gradation conversion unit 16 has been described. However, the present invention is not limited to this. Instead, only the average dispersion normalized image generation may be performed, or only the average dispersion normalized image generation and the minus component image generation may be performed. Furthermore, the average dispersion normalized image generation and the filter may be performed. Only processing may be performed. That is, in any case, it is possible to easily generate a deformed image that reflects the features of the input image, such as a portrait that reflects the user's characteristics, a realistic portrait, or an illustration-like portrait.

  In the above embodiment, when the negative component image is generated by the gradation converting unit 16, the positive component pixel is converted to zero, and the value of the pixel having a negative component less than a predetermined value (for example, “−kσ”). However, the present invention is not limited to this, and only the process of converting the positive component pixels to zero is performed. (In other words, none of the negative components may be converted). That is, even in this case, it is possible to generate a realistic portrait compared to a simple edge-enhanced gradation conversion process.

(3) Other Modified Images In the above embodiment, the case where a portrait is generated using a face image as an input image has been described. However, the present invention is not limited to this, and the present invention is not limited to this. The invention can be applied as well. That is, by applying the present invention to the coloring picture base image generation, it is possible to easily generate a coloring picture base reflecting the characteristics of the original image.

  Furthermore, by applying the present invention to a landscape image or the like, it is possible to easily generate an illustration-like landscape image reflecting the characteristics of the original landscape. It is also possible to provide a network service that accepts an image captured by a digital camera or the like via a network and sends back a portrait or coloring background.

(4) System configuration etc. In addition, among the processes described in the present embodiment, all or part of the processes described as being automatically performed can be manually performed or manually performed. All or a part of the processing described as a thing can also be automatically performed by a known method. In addition, the processing procedure, control procedure, specific name, and information including various data and parameters shown in the above-described document and drawings can be arbitrarily changed unless otherwise specified.

  The constituent elements of the illustrated image processing apparatus 10 are functionally conceptual, and need not be physically configured as illustrated. That is, the specific form of distribution / integration of the image processing apparatus 10 is not limited to that shown in the figure, and all or a part of the image processing apparatus 10 may be functionally or physically functioned in an arbitrary unit according to various loads or usage conditions. It can be configured to be distributed and integrated. Further, all or a part of each processing function performed in the image processing apparatus 10 is realized by a CPU and a program that is analyzed and executed by the CPU, or is realized as hardware by wired logic. obtain.

  The image processing method described in the present embodiment can be realized by executing a program prepared in advance on a computer such as a personal computer or a workstation. This program can be distributed via a network such as the Internet. The program can also be executed by being recorded on a computer-readable recording medium such as a hard disk, a flexible disk (FD), a CD-ROM, an MO, and a DVD, and being read from the recording medium by the computer.

  As described above, the image processing apparatus, the image processing method, and the image processing program according to the present invention are useful when generating a deformed image by deforming a target image that is a processing target. It is suitable for easily generating a reflected deformed image.

FIG. 3 is a diagram for explaining an overview and features of the first embodiment. 1 is a block diagram illustrating a configuration of an image processing apparatus according to a first embodiment. It is a figure which shows the example of an input image and a reference image. It is a figure which shows the example of a deformation | transformation vector. It is a figure which shows the example of the deformation | transformation destination of an input image. It is a figure which shows the example of a deformation | transformation image. It is a figure which shows the example of a gradation conversion image. It is a figure which shows the example of a portrait. It is a figure which shows the conversion formula used for the production | generation of an average dispersion | distribution normalization image. It is a figure for demonstrating the production | generation of a minus component image. 3 is a flowchart illustrating a flow of image processing according to the first embodiment. It is a flowchart which shows the flow of a deformation | transformation vector calculation process. It is a figure which shows the example of the deformation | transformation destination of an input image. It is a figure which shows the example of the deformation | transformation destination of an input image.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 Image processing apparatus 11 Image input part 12 Image output part 13 Image output process part 14 Storage part 15 Morphing part 15a Deformation vector calculation part 15b Deformation destination setting part 15c Deformation image generation part 16 Gradation conversion part 16a Average dispersion normalization image generation Unit 16b minus component image generation unit 16c filter processing unit

Claims (9)

An image processing device that generates a deformed image by deforming a target image that is a processing target,
Deformation vector calculating means for calculating a deformation vector for the reference image from the target image by obtaining a corresponding point between the target image and the predetermined reference image using a predetermined corresponding point search method;
Modified image generating means for generating the modified image by deforming the target image based on the modified vector calculated by the modified vector calculating means;
An image processing apparatus comprising: The deformation vector calculating means calculates a deformation vector for the average image using an average image representing an average of the processing target as the reference image,
The image processing apparatus according to claim 1, wherein the deformed image generation unit generates the deformed image by deforming the target image in a plus direction of a deformation vector calculated with respect to the average image. . The deformation vector calculating means calculates a deformation vector for the average image using an average image representing an average of the processing target as the reference image,
The image processing apparatus according to claim 1, wherein the deformed image generation unit generates the deformed image by deforming the target image in a negative direction of a deformation vector calculated with respect to the average image. .   In the deformed image generated by the deformed image generating means, an average variance normalization that generates an average variance normalized image of the deformed image by normalizing the pixel value of each pixel with an average value and a variance value in a local region in the vicinity 4. The image processing apparatus according to claim 1, further comprising converted image generation means.   In the mean variance normalized image generated by the mean variance normalized image generating means, a minus component image generating means for generating a minus component image of the deformed image by converting a value of a pixel having a plus component to zero. The image processing apparatus according to claim 4, wherein the image processing apparatus is provided.   The minus component image generation means generates a minus component image of the modified image by converting a pixel having a minus component less than a predetermined value into a predetermined value while converting a pixel having a positive component to zero. The image processing apparatus according to claim 5, wherein:   The image processing apparatus further comprises filter processing means for performing filter processing on the average dispersion normalized image generated by the average dispersion normalized image generation means or the negative component image generated by the negative component image generation means. The image processing apparatus according to claim 4, 5 or 6. An image processing method for generating a deformed image by deforming a target image to be processed,
A deformation vector calculating step of determining a corresponding point between the target image and a predetermined reference image using a predetermined corresponding point search method, and calculating a deformation vector for the reference image from the target image;
A modified image generating step of generating the modified image by deforming the target image based on the modified vector calculated by the modified vector calculating step;
An image processing method comprising: An image processing program for causing a computer to execute an image processing method for generating a deformed image by deforming a target image to be processed,
A deformation vector calculation procedure for calculating a deformation vector for a reference image from the target image by obtaining a corresponding point between the target image and a predetermined reference image using a predetermined corresponding point search method;
A modified image generation procedure for generating the modified image by deforming the target image based on the modified vector calculated by the modified vector calculation procedure;
An image processing program for causing a computer to execute.

Images