JP7110899B2 - Image processing device, image processing method, and image processing program - Google Patents

Description

The present invention relates to an image processing device, an image processing method, and an image processing program.

2. Description of the Related Art Conventionally, predetermined correction processing is performed on an image of a subject obtained by imaging. There are various types of predetermined correction processing. For example, when the user's face is photographed as a subject, the user's face is corrected to be smaller by performing distortion correction on the user's face portion in the image. can be done. A technique related to such distortion correction is disclosed in Patent Document 1, for example.

JP 2012-142772 A

However, in order to use a general technique such as the technique disclosed in Patent Document 1, first, complicated image analysis processing is performed on the image of the subject, and the position of the organ to be corrected and the position of the organ to be corrected are determined. It was necessary to identify the entire contour of the organ to be treated with high accuracy. For example, when correcting the user's eyes, it is necessary to specify the position of the user's eyes and the entire outline of the eyes with high accuracy.
It is desired to realize correction processing by a simpler method without requiring such highly accurate identification of the entire contour by image analysis processing.

SUMMARY OF THE INVENTION The present invention has been made in view of such circumstances, and an object of the present invention is to realize correction processing by a simpler method.

In order to achieve the above object, an image processing apparatus according to the present invention includes acquisition means for acquiring an image showing a predetermined subject, and a degree of correction at each position within a shape corresponding to a correction target area of the predetermined subject. storage means for storing a template including a degree of correction that varies at each position; specifying means for specifying coordinates in the image that match the template; coordinates specified by the specifying means; correction means for correcting the image based on the degree of correction obtained, and the specifying means includes a depth of a first region of the subject in the image and a depth of the subject in the image. The degree of inclination of the subject is specified based on the difference from the depth of the second area, and the degree of correction included in the template is corrected based on the specified degree of inclination of the subject, and the correction means corrects the The correction is performed on the image based on the degree of correction corrected by the specifying means and the coordinates specified by the specifying means .
Further, an image processing method according to the present invention is an image processing method executed by an image processing apparatus, comprising: an obtaining step of obtaining an image including a predetermined subject; a storing step of storing a template containing the degree of correction at each position in the image, the degree of correction varying at each position; an identifying step of identifying coordinates in the image that match the template; and a correction step of correcting the image based on the coordinates and the degree of correction included in the template, wherein the specifying step includes the depth and the depth of the first area of the subject in the image. and specifying the degree of inclination of the subject based on the difference in depth of the second region of the subject in the image, and correcting the degree of correction included in the template based on the specified degree of inclination of the subject. and the correcting step corrects the image based on the degree of correction corrected in the specifying step and the coordinates specified in the specifying step.
Further, the program according to the present invention provides a computer with an acquisition function for acquiring an image of a predetermined subject, and a degree of correction at each position within a shape corresponding to a correction target area of the predetermined subject, a storage function that stores a template containing a position-varying degree of correction; a specific function that identifies coordinates in the image that match the template; the coordinates that the specific function identifies; and the degree of correction included in the template. and a correction function for correcting the image based on the above, wherein the specific function is the depth of the first area of the subject in the image and the depth of the second area of the subject in the image. The degree of inclination of the subject is specified based on the difference from the depth, the degree of correction included in the template is corrected based on the specified degree of inclination of the subject, and the correction function is corrected by the specific function. and the coordinates specified by the specified function.

According to the present invention, correction processing can be realized by a simpler method.

1 is a block diagram showing the hardware configuration of an imaging device according to an embodiment of an image processing device of the present invention; FIG. FIG. 2 is a schematic diagram for explaining an outline of thin face correction processing by the imaging device of FIG. 1; FIG. 2 is a functional block diagram showing a functional configuration for executing thin face correction processing among the functional configurations of the imaging apparatus of FIG. 1; FIG. 4 is a schematic diagram showing the relationship between examples of template shapes and alpha values, and facial regions to which the templates are applied; FIG. 4 is a schematic diagram for explaining matching of a template to an original image; FIG. 4 is a schematic diagram for explaining generation of an alpha map using a plurality of templates; FIG. 4 is a schematic diagram showing an example of an original image before correction; FIG. 8 is a schematic diagram showing an example of an image obtained by correcting the original image shown in FIG. 7 by a conventional technique; FIG. 8 is a schematic diagram showing an example of an image obtained by correcting the original image shown in FIG. 7 by thin face correction processing performed by the imaging device of FIG. 1 ; 4 is a flowchart for explaining the flow of thin face correction processing executed by the imaging device 1 of FIG. 1 having the functional configuration of FIG. 3; 2 is a schematic diagram for explaining a case where trimming processing is performed in the thin face correction processing by the imaging device of FIG. 1; FIG. 4 is a schematic diagram for explaining correction of an alpha value by the imaging device 1; FIG. FIG. 10 is a schematic diagram showing an example of an image subjected to distortion correction after correcting the alpha value in the thin face correction process; FIG. 10 is a schematic diagram showing another example of an image that has undergone distortion correction after correcting the alpha value in the thin face correction process. FIG. 10 is a schematic diagram showing another example of an image that has undergone distortion correction after correcting the alpha value in the thin face correction process. 10 is a flowchart for explaining the flow of thin face correction processing according to Modification 4, which is executed by the imaging device 1 of FIG. 1 having the functional configuration of FIG. 3. FIG.

BEST MODE FOR CARRYING OUT THE INVENTION Hereinafter, embodiments of the present invention will be described with reference to the drawings.

FIG. 1 is a block diagram showing the hardware configuration of an imaging device 1 according to an embodiment of the image processing device of the present invention.
The imaging device 1 is configured as, for example, a digital camera.

As shown in FIG. 1, the imaging device 1 includes a CPU (Central Processing Unit) 11, a ROM (Read Only Memory) 12, a RAM (Random Access Memory) 13, a bus 14, an input/output interface 15, and an imaging device. A unit 16 , an input unit 17 , an output unit 18 , a storage unit 19 , a communication unit 20 , a drive 21 and an illumination unit 22 are provided.

The CPU 11 executes various processes according to programs recorded in the ROM 12 or programs loaded from the storage unit 19 to the RAM 13 .

The RAM 13 also stores data necessary for the CPU 11 to execute various processes.

The CPU 11 , ROM 12 and RAM 13 are interconnected via a bus 14 . An input/output interface 15 is also connected to this bus 14 . The input/output interface 15 is connected with an imaging unit 16 , an input unit 17 , an output unit 18 , a storage unit 19 , a communication unit 20 , a drive 21 and an illumination unit 22 .

The imaging unit 16 includes an optical lens unit and an image sensor (not shown).

The optical lens section is composed of a lens for condensing light, such as a focus lens and a zoom lens, in order to photograph an object.
A focus lens is a lens that forms a subject image on the light receiving surface of the image sensor. A zoom lens is a lens that can freely change the focal length within a certain range.
The optical lens unit is also provided with peripheral circuits for adjusting setting parameters such as focus, exposure, white balance, etc., as required.

The image sensor is composed of a photoelectric conversion element, an AFE (Analog Front End), and the like.
The photoelectric conversion element is composed of, for example, a CMOS (Complementary Metal Oxide Semiconductor) type photoelectric conversion element or the like. A subject image is incident on the photoelectric conversion element from the optical lens section. Therefore, the photoelectric conversion element photoelectrically converts (pictures) a subject image, accumulates an image signal for a certain period of time, and sequentially supplies the accumulated image signal to the AFE as an analog signal.
The AFE executes various signal processing such as A/D (Analog/Digital) conversion processing on this analog image signal. A digital signal is generated by various signal processing and output as an output signal of the imaging unit 16 .
Such an output signal of the imaging unit 16 is appropriately supplied to the CPU 11, an image processing unit (not shown), or the like as captured image data.

The input unit 17 is composed of various buttons and the like, and inputs various kinds of information according to user's instruction operation.
The output unit 18 includes a display, a speaker, and the like, and outputs images and sounds.
The storage unit 19 is composed of a hard disk, a DRAM (Dynamic Random Access Memory), or the like, and stores data of various images.
The communication unit 20 controls communication with another device (not shown) via a network including the Internet.

A removable medium 31 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory is mounted in the drive 21 as appropriate. A program read from the removable medium 31 by the drive 21 is installed in the storage unit 19 as necessary. The removable medium 31 can also store various data such as image data stored in the storage unit 19 in the same manner as the storage unit 19 .

The illumination unit 22 includes, for example, eight LEDs (Light Emitting Diodes) as light-emitting members installed in a circle around the imaging unit 16 and, in the imaging device 1, separated from the eight LEDs. It consists of one LED installed at a position These LEDs selectively emit light according to a user's operation during live view shooting, image recording, or the like. By photographing the face while changing the incoming direction of light rays for illumination in this way, it is possible to obtain a face image in which shadows are artificially controlled.

The imaging apparatus 1 configured as described above has a function of correcting the subject naturally without discomfort by performing distortion correction using a template on an image of the subject. As a result, for example, when the user's face is used as the subject, the user's face can be made into a natural thin face without discomfort.
Hereinafter, a series of processes for performing distortion correction using this template by the imaging device 1 will be referred to as "slim face correction process". Next, with reference to FIG. 2, an outline of the thin face correction process will be described.

[Overview of thin face correction processing]
FIG. 2 is a schematic diagram for explaining an outline of the thin face correction processing by the imaging device 1. As shown in FIG.
FIG. 2 shows an example in which local distortion correction is performed on a region including the bridge of the nose, which is a predetermined organ included in the face, and left and right face line regions, which are predetermined parts.

As shown in FIG. 2A, the imaging device 1 acquires an image before correction (hereinafter referred to as an “original image”) as a target for thin face correction processing. The imaging device 1 also specifies a region to be subjected to distortion correction (hereinafter referred to as a “correction target region”) in this original image. For this reason, the imaging apparatus 1 stores a template representing the likeness of a predetermined organ or the likeness of a predetermined part of the face according to each of the predetermined organs and parts. Further, the imaging device 1 adapts this template to the position of the corresponding predetermined organ or predetermined part of the original image. Here, this template includes a value (hereinafter referred to as "alpha value") indicating the degree of correction at each position within the shape corresponding to the correction target area of the face included in the original image.

In addition, as shown in FIG. 2B, the imaging device 1 performs the A map (hereinafter referred to as an "alpha map") showing the degree of correction of each position in the original image is generated by arranging each template based on the matching result.

Further, as shown in FIG. 2C, the imaging device 1 performs distortion correction on the original image. In this case, the imaging device 1 performs distortion correction at a degree based on the degree of correction at each coordinate included in the alpha map. Specifically, in distortion correction, correction is realized by moving each coordinate of the original image in a predetermined direction. For example, when the user's face is thin, correction is realized by moving each coordinate in the correction target area toward the center of the face. The imaging apparatus 1 performs distortion correction by determining the amount of movement of each coordinate according to the degree of correction at each coordinate included in the alpha map.

According to such an imaging apparatus 1, distortion correction can be performed only on the correction target area corresponding to the template without performing unnecessary correction on areas other than the correction target area. Also, instead of distortion correction based on a uniform degree of correction, it is possible to perform appropriate distortion correction for each organ or predetermined part according to the alpha value included in the template.

In addition, the imaging apparatus 1 uses a template having a shape that expresses the likeness of a predetermined organ or the likeness of a predetermined part, which is the region to be corrected. set to be suppressed. Therefore, even if the contour of the predetermined organ or the contour of the predetermined part to be corrected does not completely match the contour of the template, the effect on the periphery of the contour is small, and the subject as a whole can be corrected. Correction can be performed naturally without discomfort. Therefore, according to the imaging device 1, it is possible to eliminate the need to detect the contour of an organ or a predetermined part with high accuracy. Moreover, even if the contour of an organ or a predetermined part is not detected with high accuracy, distortion correction can be performed without degrading the quality of the image after correction.
In this respect, conventionally, in order to perform appropriate distortion correction, the positions of the user's organs and predetermined parts to be corrected by the user and the entire contours of these organs and predetermined parts must be specified with high accuracy. I had to. On the other hand, as described above, the imaging apparatus 1 eliminates the need to detect the contours of organs and predetermined parts with high accuracy, and performs distortion correction without degrading the quality of the corrected image. can be done.
That is, according to the present embodiment, correction processing can be realized by a simpler method.

[Function block]
FIG. 3 is a functional block diagram showing a functional configuration for executing thin face correction processing among the functional configurations of the imaging apparatus 1 of FIG.
As described above, the thin face correction process refers to a series of processes for performing distortion correction using a template.

When executing thin face correction processing, as shown in FIG. 55 and , function.
An image storage section 61 and a template storage section 62 are set in one area of the storage section 19 .

The image storage unit 61 stores the data of the captured image output from the image capturing unit 16, which is obtained by subjecting the image captured by the image capturing unit 16 to development processing.
The template storage unit 62 stores template data of shapes corresponding to various parts of the face. Details of the template will be described with reference to FIG.

The image acquisition unit 51 acquires data of a captured image obtained by subjecting an image captured by the imaging unit 16 to development processing, or data of an image to be processed from the image storage unit 61 .

A face detection unit 52 detects a face from the original image. Further, the face detection unit 52 detects feature points of predetermined organs and predetermined parts corresponding to the correction target area. Details of the processing by the face detection unit 52 will be described later with reference to FIG.

The matching position specifying unit 53 specifies coordinates in the original image that match the template. Details of the processing by the matching position specifying unit 53 will be described later with reference to FIG.

The alpha map creation unit 54 creates an alpha map based on the coordinates that match the template. The details of the processing by the alpha map generation unit 54 will be described later with reference to FIG.

A distortion correction processing unit 55 performs distortion correction based on the alpha value included in the alpha map. The details of the processing by the distortion correction processing section 55 will be described later after the description of the alpha map generation section 54 with reference to FIG.

[Specific example of template]
Next, details of the template stored in the template storage unit 62 will be described with reference to FIG. FIG. 4 is a schematic diagram showing the relationship between an example of the template shape and alpha value, and the part of the face to which the template is applied.
Here, the template is a template that imitates the shape of a correction target area corresponding to a predetermined organ or a predetermined part of the face in a general face, and expresses the likeness of a specific part of the face. Also, as described above, the alpha value indicating the degree of correction at each position within the shape corresponding to the correction target area is included.

In FIG. 4, a "nose template T1" which is a template corresponding to the nose, a "right face line template T2" which is a template corresponding to the right side of the face line, and a "face template T2" which is a template corresponding to the left side of the face line. This shows an example in which a template T3 for the left side of the line is also applied. In addition, in FIG. 4 and FIG. 5 which will be described later, the template is shown as a closed area indicated by broken lines. In these figures, the alpha values included in the template are shown in black when the value is "0", in white when the value is "1", and when the value is "1". It is indicated by hatching in the case of "intermediate value between 0 and 1".

As shown in FIG. 4, the nasal template T1 is used to reduce the entire nose by making the entire nose a correction target region. Further, the nose template T1 has an alpha value set to "1" so that the area in the central part of the nose is particularly reduced, and an alpha value of "0" so that the area outside the outline of the nose is not reduced. An "intermediate value between 0 and 1" is set so that the portion around the contour within the contour of the nose is less reduced than the central portion. By performing distortion correction based on such an alpha value that is set so that the degree of correction changes stepwise as it approaches the center of the nose, the effect of shrinking the nose more naturally and making the face slimmer is achieved. can get.

Further, as shown in FIG. 4, the right face line template T2 and the left face line template T3 set the left and right face lines as correction target regions, respectively, so that the face lines of the left and right cheeks (particularly, the gill portion) are corrected. used to reduce the The right face line template T2 and the left face line template T3 are symmetrical with respect to the center line of the face so that the left and right cheeks shrink evenly.

In the right face line template T2 and the left face line template T3, the alpha value is set to "1" so that the central region of the face line is particularly reduced, and the region outside the contour of the face line is The alpha value is set to "0" so as not to be reduced, and an "intermediate value between 0 and 1" is set so that the peripheral portion of the contour of the face line is less reduced than the central portion. By performing distortion correction based on the alpha value set so that the degree of correction changes stepwise as the center of the face line is approached, the left and right face lines are reduced more naturally and the face becomes slimmer. You can get the effect of

In order to reduce the amount of data, only one template having a symmetrical shape can be stored and used after being symmetrically transformed. For example, a template corresponding to the left face line can be stored and used as a template corresponding to the right face line by symmetrically transforming it by horizontally inverting it when using it.

[Template adaptation]
Next, the matching of the template to the original image performed by the face detection unit 52 and matching position specifying unit 53 will be described. FIG. 5 is a schematic diagram for explaining matching of a template to an original image. In FIG. 5, a case where the nose template T1 is matched to the nose in the original image acquired by the image acquisition unit 51 will be described as an example.

When matching a template, as shown in FIG. 5A, the face detection unit 52 detects feature points in the original image. Specifically, the face detection unit 52 detects a base point and contour points as feature points. The base point is, for example, a point corresponding to the center of the nose, such as the base point P11 in the figure. The contour points are, for example, contour points P21 and P22 in the figure corresponding to the upper and lower ends of the nose, and contour points P31 and P32 in the figure to the left and right of the nose. It is a point corresponding to both ends.
The matching process described below with reference to FIG. 5(B) is performed based on the coordinates of each feature point detected in this way in the original image. It should be noted that existing face detection technology and feature point detection technology can be used for face detection and feature point detection by the face detection unit 52 .

Next, as shown in FIG. 5B, the matching position specifying unit 53 aligns the templates based on the base points. Specifically, the matching position identifying unit 53 aligns the template by matching the coordinates of the center of the nasal template T1 and the coordinates of the center of the base point P11.

Next, as shown in FIG. 5C, the matching position specifying unit 53 rotates the template based on the angle. Here, as shown in the figure, for example, an angle A1, which is an angle between a straight line connecting contour points P31 and P32 and a straight line parallel to the X-axis, can be used as the angle. Since this angle A1 is an angle that indicates the inclination of the nose, the matching position specifying unit 53 rotates the nose template T1 about the base point P11 by the same angle as the angle A1, thereby adjusting the angle of the template. can be adapted.

Finally, as shown in FIG. 5(D), the matching position specifying unit 53 enlarges or reduces the template based on the positions of the contour points. That is, adapt the size of the template. Specifically, the matching position specifying unit 53 maintains the shape of the nasal template T1 so that both ends of the nasal template T1 are the same size as the two ends of the nose indicated by the contour points P31 and P32. The nasal template T1 is enlarged or reduced as it is. In this case, the matching position specifying unit 53 also maintains the alpha value included in the nasal template T1 without changing it.

As described above with reference to FIGS. 5A to 5D, the template can be adapted to the original image. This matching can be achieved by detecting only the base point and a plurality of contour points, as described above. That is, it is not necessary to specify the entire contour of the organ or part to be corrected with high accuracy. That is, according to the present embodiment, correction processing can be realized by a simpler method.

Also, when there are a plurality of templates, matching coordinates for the plurality of templates are independently specified for each of the plurality of templates. Therefore, even if the face in the original image is almost sideways or tilted, and the angles and sizes of the face lines are not symmetrical, the template is designed to match the left and right face lines. You can adjust the angle and size. That is, according to the present embodiment, a common template prepared in advance can be adapted to each of various original images having different face orientations and sizes.

In the above description, the matching is performed using the base point P11 and the contour points P31 and P32. However, it is also possible to perform matching using other contour points. This point will be described later as Modified Example 1.

[Create Alpha Map]
Next, alpha map generation performed by the alpha map generation unit 54 will be described. FIG. 6 is a schematic diagram for explaining generation of an alpha map using a plurality of templates.
As shown in FIG. 6A, the alpha map creation unit 54 acquires coordinates that match each of the plurality of templates specified by the matching position specifying unit 53 .

As shown in FIG. 6B, the alpha map generator 54 arranges each template on matching coordinates on a black background based on coordinates matching each of the plurality of templates. Note that the black background and the original image are managed using the same coordinate system.

As shown in FIG. 6C, the alpha map is created by arranging all the templates on a black background by the alpha map creating unit 54 .

[Distortion correction based on alpha map]
Next, the distortion correction performed by the distortion correction processing unit 55 based on the alpha values included in the alpha map will be described. As described above with reference to FIG. 2, in this embodiment, distortion correction is performed on the original image based on the alpha values included in the alpha map.

Specifically, the distortion correction processing unit 55 first calculates coordinates after distortion correction for all coordinates of the original image when performing distortion correction without using an alpha map. For example, for the entire face area, coordinates after distortion correction such that the face is reduced are calculated for all coordinates of the original image. An existing distortion correction technique can be used to calculate the coordinates after the distortion correction.

As a result, for all the coordinates of the original image, the coordinates after the distortion correction when the distortion correction is performed without using the alpha map (hereinafter referred to as "coordinates before applying the alpha map") are calculated.

Next, the distortion correction processing unit 55 corrects the degree of correction at each coordinate by applying an alpha map. Specifically, the distortion correction processing unit 55 corrects all the coordinates of the original image based on the following <alpha map application formula>, the coordinates after the distortion correction by applying the alpha map (hereinafter referred to as “after applying the alpha map”). (referred to as "coordinates").

<Alpha map application formula>
(x _out ,y _out )=(x _in ,y _in )×(1−α)+(x _dist ,y _dist )×α
however,
(x _in , y _in ): Coordinates of original image (x _out , y _out ): Coordinates after application of alpha map (x _dist , y _dist ): Corrected coordinates before application of alpha map. Also, the value of α is assumed to be (0≦α≦1).

Then, the distortion correction processing unit 55 performs correction based on the calculated post-alpha map coordinates. That is, the coordinates of the original image are moved to the coordinates after application of the alpha map. Further, the distortion correction processing unit 55 performs complementary processing, etc., which are required along with the movement, using existing technology.

This implements distortion correction using an alpha map. By applying the alpha map in this way, the white coordinates in the alpha map (that is, the coordinates with the alpha value of "1") are subjected to distortion correction by applying the corrected coordinates before the application of the alpha map. Also, for intermediate color coordinates in the alpha map (that is, coordinates with an alpha value of “0<α<1”), distortion correction is performed by applying corrected coordinates before alpha map application according to the alpha value. Furthermore, the corrected coordinates before applying the alpha map are not applied to the coordinates of black in the alpha map (that is, the coordinates with the alpha value of "0"), and the coordinates of the original image remain as they are.
That is, local distortion correction is performed corresponding to the degree of correction in the plurality of templates described above. The effect of such distortion correction will be described with reference to FIGS. 7, 8 and 9. FIG.

FIG. 7 shows an example of an original image before distortion correction. Also, FIG. 8 shows an example of an image obtained by correcting the original image shown in FIG. 7 by a conventional technique (that is, distortion correction without using an alpha map). Further, FIG. 9 shows an example of an image obtained by correcting the original image shown in FIG. 7 by distortion correction using the alpha map in the present embodiment described above.

Comparing the original image in FIG. 7 with the image corrected by the conventional technique in FIG. are equally corrected. In this case, since the entire face is reduced, there is a tendency for the head to appear relatively elongated. In addition, it is difficult to narrow the chin because the entire chin shrinks evenly. Therefore, in the conventional technology, the user may feel uncomfortable and unnatural in the corrected image.

On the other hand, in the present embodiment, local distortion correction is performed corresponding to the degree of correction in the plurality of templates described above. Therefore, when the original image in FIG. 7 and the image corrected by the present embodiment in FIG. 9 are compared, since only the correction target area corresponding to the template, not the entire face, is reduced, the head looks relatively elongated. It can be prevented that it is put away. In addition, since only the correction target regions corresponding to the left and right face lines are reduced instead of the entire jaw, the jaw can be made thinner. Therefore, according to the present embodiment, it is possible to make the user's face look natural and slim without causing discomfort.

[motion]
FIG. 10 is a flowchart for explaining the flow of thin face correction processing executed by the imaging apparatus 1 of FIG. 1 having the functional configuration of FIG.
The thin face correction process is started when the user operates the input unit 17 to start the thin face correction process.
The operation for starting the thin face correction process can be a photographing instruction operation, and the image captured by the imaging unit 16 in response to this photographing instruction operation is subjected to development processing, and then the thin face correction process is performed. It can be carried out. It is also possible to select the data of the captured image stored in the image storage unit 61 and perform the operation of starting the thin face correction process for the selected captured image data.

In step S<b>11 , the image acquisition unit 51 acquires image data obtained by developing an image captured by the imaging unit 16 or an image to be processed from the image storage unit 61 .

In step S12, the face detection unit 52 performs face detection on the image to be processed, and determines whether or not a face has been detected.
If the face is not detected, NO is determined in step S12, and the thin face correction process ends.
If a face is detected, YES is determined in step S12, and the process proceeds to step S13.

In step S13, the face detection unit 52 detects predetermined organs and predetermined parts of the face corresponding to the template.

In step S14, the matching position specifying unit 53 specifies coordinates in the original image that match the template.

In step S15, the alpha map creation unit 54 creates an alpha map based on the coordinates matching the template.

In step S16, the distortion correction processing unit 55 performs distortion correction using an alpha map.
After that, the thin face correction process ends.
By the operation described above, an image is generated in which the user's face is naturally slim, as described above with reference to FIG. 9 .

The imaging apparatus 1 configured as described above has an image acquisition section 51 , a template storage section 62 , a matching position specifying section 53 , and a distortion correction processing section 55 .
The image acquisition unit 51 acquires an image of a predetermined subject.
The template storage unit 62 stores a template containing the degree of correction at each position within the shape corresponding to the region to be corrected of the predetermined subject, and the degree of correction that changes at each position.
The matching position specifying unit 53 specifies coordinates matching the template in the image.
The distortion correction processing section 55 corrects the image based on the coordinates specified by the matching position specifying section 53 and the degree of correction included in the template.
As a result, since correction can be achieved by matching the template, it is possible to eliminate the need for highly accurate detection of the contours of organs and predetermined parts. Therefore, correction processing can be realized by a simpler method. Furthermore, even if the contour of an organ or a predetermined part is not detected with high accuracy, the correction can be performed without deteriorating the quality of the corrected image.
In addition, distortion correction can be performed only on the correction target area corresponding to the template without performing unnecessary correction on areas other than the correction target area. Moreover, instead of performing distortion correction based on a uniform degree of correction, it is possible to perform appropriate distortion correction for each organ or predetermined part according to the degree of correction included in the template.

The matching position specifying unit 53
Detecting coordinates corresponding to the first feature point of the predetermined subject and coordinates corresponding to the second feature point of the predetermined subject by analyzing the image,
Coordinates that match the base point in the template are specified based on the detected coordinates of the first feature point, and coordinates that match the contour in the template are specified based on the detected coordinates of the second feature point.
This makes it possible to specify the coordinates that match the template by detecting only the feature points.

The matching position specifying unit 53
A base point of the template is arranged at the coordinates of the first feature point, and the template is rotated around the base point of the arranged template, thereby specifying the coordinates that match the template.
As a result, a common template can be adapted to various original images with different face orientations and sizes, simply by rotating, without preparing multiple templates for each angle. .

The matching position specifying unit 53
Based on the coordinates of the detected second feature points and the contour of the template, enlargement or reduction is performed while maintaining the shape of the contour of the template.
As a result, without preparing a plurality of templates with different sizes, it is possible to adapt a common template simply by enlarging or reducing various original images with different face orientations and sizes. can.

The matching position specifying unit 53
The degree of correction included in the template is not changed.
As a result, correction can be performed with a degree of correction corresponding to the template regardless of the size of a predetermined subject included in the original image.

The matching position specifying unit 53 specifies the state of the subject in the image, and corrects the degree of correction included in the template based on the specified state of the subject.
The distortion correction processing section 55 corrects the image based on the degree of correction corrected by the matching position specifying section 53 and the coordinates specified by the matching position specifying section 53 .
As a result, distortion correction can be performed based on the corrected degree of correction more appropriately based on the state of the subject. For example, when the face rotates left and right about the vertical axis of the face (i.e., when yawing), and when it rotates up and down about the horizontal direction axis of the face (i.e., when pitching). Distortion correction can be performed based on the correspondingly modified degree of correction. Therefore, for example, distortion correction can be performed in consideration of the balance of the entire face.

Based on the difference between the depth of the first area of the subject in the image and the depth of the second area of the subject in the image, the matching position specifying unit 53 specifies the degree of inclination of the subject as the state of the subject.
This makes it possible to specify the degree of inclination of the subject based on a clear index of depth information. Therefore, it is possible to correct the degree of correction according to the degree of inclination of the subject. For example, if it is more inclined, it can be corrected by increasing (or decreasing) the degree of correction.

The distortion correction processing unit 55
As the correction of the image, distortion correction is performed by moving the coordinates of the image at least by a degree of correction corresponding to the degree of correction included in the template.
As a result, it is possible to perform natural distortion correction with a degree of correction that corresponds to the template.

The template storage unit 62 stores a plurality of templates corresponding to a plurality of different correction target regions,
The matching position specifying unit 53 independently specifies matching coordinates for each of the plurality of templates.
As a result, a common template prepared in advance can be adapted to various original images having different face orientations and sizes.

the predetermined subject is a face,
The correction target area is a facial organ or a facial part.
As a result, the subject's face can be made into a natural thin face without discomfort.

The degree of correction of the template is set such that the portion around the contour within the contour of the template is less corrected than the central portion of the template.
As a result, even if the contour of the predetermined organ or the contour of the predetermined part to be corrected does not completely match the contour of the template, the effect on the periphery of the contour is small, and the subject as a whole can be corrected. can be corrected naturally without discomfort.

It should be noted that the present invention is not limited to the above-described embodiments, and includes modifications, improvements, and the like within the scope of achieving the object of the present invention. For example, the above-described embodiment can be modified to provide modified examples as exemplified below. In addition, modifications such as those exemplified below can be combined.

[Modification 1]
In the example described above with reference to FIG. 5, the matching position specifying unit 53 performed matching using the base point P11 and the contour points P31 and P32. may be used for matching.
For example, contour point P21 and contour point P22 may be used instead of contour point P31 and contour point P32 for matching. In this case, for example, as shown in FIG. 5C, when the template is rotated, an angle (hereinafter referred to as called "angle A2") may be used instead of angle A1. Further, for example, as shown in FIG. 5D, when enlarging or reducing the template, both ends of the nose template T1 are made to have the same size as both ends of the nose indicated by contour points P21 and P22. In addition, the nasal template T1 may be enlarged or reduced while maintaining the shape of the nasal template T1.

Alternatively, for example, along with the contour points P31 and P32, the contour points P21 and P22 may also be used for matching. For example, as shown in FIG. 5C, when rotating the template, the template may be rotated based on an average angle of the angle indicated by the angle A1 and the angle indicated by the angle A2. Further, for example, as shown in FIG. 5D, when enlarging or reducing the template, the difference between both ends of the nose template T1 and both ends of the nose indicated by contour points P21 and P22, and The shape of the nose template T1 is maintained so that the sum of the two differences between both ends of the template T1 and the difference between the two ends of the nose indicated by the contour points P31 and P32 is minimized. T1 may be enlarged or reduced.

Alternatively, for example, in the original image, if the possibility that the user's nose is tilted is low and the size of the nose can be roughly assumed, the template is rotated as shown in FIG. 5(C). The process and the process of enlarging or reducing the template as shown in FIG. 5(D) may be omitted. In this case, only the base point P11 needs to be detected, eliminating the need to detect each contour point.

[Modification 2]
In the above-described embodiments, an alpha map corresponding to the entire original image is created, and distortion correction is performed on the entire original image based on the alpha values included in the alpha map. Without being limited to this, the original image may be partially cut out by trimming, and distortion correction may be performed on the cut out image. This modification will be described with reference to FIG.

First, as shown in FIG. 11A, the face detection unit 52 cuts out, by trimming, a region including a predetermined organ and a predetermined part corresponding to the template from the original image based on the detection result of the feature points.

Next, as shown in FIG. 11B, a cut-out original image corresponding to the cut-out original image is obtained by template matching processing by the matching position specifying unit 53 and alpha map creation processing by the alpha map creation unit 54. Create an alpha map for

Next, as shown in FIG. 11C, the distortion correction processing unit 55 performs distortion correction on the clipped original image using the clipped original image alpha map.
Finally, as shown in FIG. 11D, the face detection unit 52 pastes the cut out image after correction onto the original image.

Correction similar to that of the embodiment described above can also be performed by the processing described above. For example, this modification may be applied when a process of cutting out a region including a predetermined organ and a predetermined part corresponding to a template from an original image can be easily performed by trimming.
By cutting out by trimming in this way, in the matching process of the template by the matching position specifying unit 53, the number of coordinates that are candidates for matching the template can be reduced. Therefore, it is possible to reduce the amount of calculation in the matching process. In addition, by cutting out by trimming in this way, in the distortion correction processing by the distortion correction processing unit 55, the number of coordinates for which post-alpha map application coordinates are calculated can be reduced by the <alpha map application formula>. Therefore, it is possible to reduce the amount of calculation in the matching process. Therefore, it is possible to reduce the amount of calculation for the entire thin face correction process.

[Modification 3]
In the above-described embodiments, distortion correction using an alpha map is performed on the user's face. Without being limited to this, distortion correction using an alpha map may be performed on a portion other than the user's face. Distortion correction using an alpha map may also be performed for living creatures other than the user, or for non-living creatures. That is, distortion correction using an alpha map may be performed on any correction target as long as a template corresponding to the correction target can be prepared.

Further, in the above-described embodiments, distortion correction using an alpha map reduces a predetermined organ or predetermined site within an image. Not limited to this, distortion correction using an alpha map may be used to enlarge a predetermined organ or predetermined site within the image. For example, a template corresponding to both eyes of the user may be prepared, and the eyes of the user may be enlarged by distortion correction using an alpha map.

[Modification 4]
In the above-described embodiment, in the thin face correction process, after the original image is matched with a template, distortion correction is performed based on the alpha value, which is a value indicating the degree of correction at each position, included in the template. rice field. Alternatively, the alpha value included in the template may be modified based on a predetermined condition such as the user's state, and the distortion correction may be performed based on the modified alpha value. For example, the alpha value included in the template may be corrected based on whether or not the user's face is tilted and the degree of tilt in the original image, and distortion correction may be performed based on the corrected alpha value. A case in which the alpha value is corrected in this manner will be described below as Modified Example 4. FIG.

<Correction of alpha value>
12A and 12B are schematic diagrams for explaining correction of the alpha value in Modification 4 by the imaging device 1. FIG. FIG. 12 shows an example in which local distortion correction is performed on each of left and right face line regions, which are predetermined parts included in the face.

FIG. 12 shows an original image 41 as an example of the original image. In this modification, as described above, the alpha value included in the template is modified based on the inclination of the user's face. In this variation, the inclination of the user's face is identified based on depth information (also referred to as depth information) indicating the depth of a predetermined region of the user's face. The depth information is information indicating the distance from the imaging device 1 to the subject (here, the user's face) at the time of imaging. Depth information can be acquired, for example, by installing a depth sensor capable of detecting the distance to the subject using infrared rays in the imaging device 1 . In addition, depth information can be obtained by implementing the imaging device 1 with a stereo camera that can detect the distance to the subject based on the principle of triangulation.

In addition, in this modification, when the face rotates left and right about the vertical axis of the face (that is, yawing), and when it rotates up and down about the horizontal axis of the face (that is, pitching) case) is specified.

First, the case of rotating left and right about the vertical axis of the face (that is, the case of yawing) will be described as an example.
In this modified example, depth information is acquired along with face detection by image analysis. Then, based on this depth information, the degree of tilt in the yawing 43 (hereinafter referred to as "degree of tilt in the yaw direction") that rotates left and right about the vertical axis direction 42 of the face shown in FIG. 12 is specified. Note that the vertical axis direction 42 and yaw 43, and the horizontal axis direction 44 and pitching 45, which will be described later, are not information that is actually displayed together with the original image 41, but information that is illustrated for convenience of explanation.

The degree of inclination can be specified based on the difference in depth information between two predetermined regions of the face. The predetermined regions are not particularly limited as long as they are regions for which the degree of inclination in the yaw direction can be calculated. Below, as an example, the degree of tilt in the yaw direction is calculated based on the depth information (depth value) under the left cheek and the depth information (depth value) under the right cheek using the following <yaw direction tilt degree calculation formula > is calculated using

<Yaw direction inclination degree calculation formula>
yaw = d_l - d_r
however,
d_l: depth value under the left cheek d_r: depth value under the right cheek yaw: degree of inclination in the yaw direction.

Next, based on the degree of inclination in the yaw direction, the alpha value of the template corresponding to each of the left and right face line regions is corrected. This correction is performed for the purpose of balancing the user's face when the subsequent distortion correction is performed. For example, based on the degree of tilt in the yaw direction, if it can be identified that the user is facing left (that is, to the right from the perspective of the observer facing the image), the degree of distortion correction of the user's left face line (that is, , alpha value) and increase the degree of distortion correction of the user's right face line. As a result, the user's right face line, which is on the near side in the image, undergoes distortion correction more than the left face line, so that the user's right face line becomes thinner. Therefore, the left and right faces of the user are balanced.

Therefore, in this modified example, the alpha value is corrected so that distortion correction is performed in this manner. For this reason, the correspondence relationship between the degree of inclination in the yaw direction and the correction rate is set in advance for each template. This setting may be stored, for example, in the form of a table showing correspondence relationships, or may be stored as a predetermined relational expression. Then, when the degree of inclination in the yaw direction is calculated, the corresponding correction rate can be specified by referring to this table or performing calculation using the relational expression.

FIG. 12 is a graph showing an example of the relationship between the degree of inclination in the yaw direction and the correction rate. For example, the correction factor for the right side of the face line shown in FIG. 12 is set to be lower as the yaw direction tilt degree is lower (that is, as the user faces rightward). On the other hand, the correction factor for the left side of the face line shown in FIG. 12 is set to decrease as the yaw direction tilt degree increases (that is, as the user faces leftward). After specifying the correction rate based on the set correspondence relationship in this manner, the corrected alpha value can be calculated using the following <yaw direction correction degree correction formula>.

<Yaw direction correction degree correction formula>
Alpha value after correction=α×correction rate/100

By correcting the distortion of the original image based on the corrected alpha value in this way, it is possible to achieve correction that balances the left and right faces of the user, as described above. For example, in the original image 41 of FIG. 12, the user is facing left. In this case, the alpha value correction described above corrects the “right face line (before correction) T2a” to the “right face line (after correction) T2b” so that the degree of correction is increased. Further, by correcting the alpha value described above, the "left face line (before correction) T3a" is corrected to a lower degree of correction, such as "left face line (after correction) T3b". As a result, the user's right face line, which is on the near side in the image, undergoes distortion correction more than the left face line, so that the user's right face line becomes thinner. Therefore, the left and right faces of the user are balanced.

An example of performing distortion correction after correcting the alpha value based on the degree of correction in the yaw direction in this way will be described with reference to FIG. FIG. 13 is a schematic diagram showing an example of an image subjected to distortion correction after correcting the alpha value in the thin face correction process. As shown in FIG. 13B, when distortion correction is performed without correcting the alpha value, although the face is oriented to the right, the face is distorted in the same way as when the face is facing forward. The left and right face lines are evenly corrected. Therefore, an image in which the left face line is emphasized is obtained. On the other hand, as shown in FIG. 13A, when distortion correction is performed after correcting the alpha value based on the degree of correction in the yaw direction as described above, the right face line of the face is corrected. is suppressed, and the left face line of the face is corrected more, so the user's right face line becomes thinner. Therefore, the left and right faces of the user are balanced.

In this way, in this modified example, for example, without performing complicated image analysis such as detecting the contour of the user's face line, only by calculating the degree of inclination from the depth information and correcting the template, Corrections can be made to balance the user's face. That is, also in this modified example, the correction process can be realized by a simpler method than the conventional method.

Next, in the case of vertical rotation of the face around the horizontal axis, which is performed in the same manner as the correction of the alpha value in the case of horizontal rotation around the vertical axis of the face (that is, the case of yawing) described above. (that is, when pitching) will be described as an example.
In this case, the depth information is acquired along with detection of a face or the like by image analysis. Then, based on this depth information, the degree of tilt in pitching 45 that rotates up and down about the left-right axis direction 44 of the face shown in FIG. The degree of inclination can be specified, similarly to the degree of inclination in the yaw direction described above, based on the difference in depth information between two predetermined regions of the face. This predetermined area is not particularly limited as long as it is between areas for which the degree of tilt in the pitch direction can be calculated. In the following, as an example, the pitch direction tilt degree is calculated using the following <pitch direction tilt degree calculation formula> based on the depth information (depth value) between the eyebrows and the depth information (depth value) of the upper lip. .

<Formula for calculating degree of tilt in pitch direction>
pitch=u_c-d_c
however,
u_c: depth value between eyebrows d_c: depth value of upper lip pitch: degree of tilt in the pitch direction.

Next, based on the degree of inclination in the pitch direction, the alpha value of the template corresponding to each predetermined region of the face is corrected. For example, it modifies the alpha value of the template corresponding to the user's chin and the alpha value of the template corresponding to the user's forehead. This correction, like the degree of tilt in the yaw direction described above, is performed for the purpose of balancing the user's face when the latter distortion correction is performed.

For example, if it can be identified that the user is facing upward based on the degree of tilt in the pitch direction, the degree of distortion correction (that is, the alpha value) of the template corresponding to the user's chin is increased. As a result, when the chin protrudes forward, distortion correction is performed to make the chin thinner, so the user's face is balanced. On the other hand, if it can be identified that the user is facing downward, the degree of distortion correction of the template corresponding to the user's chin is lowered. As a result, when the chin is pulled, distortion correction is performed so that the chin remains, and in this case as well, the user's face is well balanced.

Therefore, in this modified example, the alpha value is corrected so that distortion correction is performed in this manner. For this reason, similarly to the degree of tilt in the yaw direction described above, the correspondence relationship between the degree of tilt in the pitch direction and the correction rate is set in advance for each template. For example, the correction factor of the chin template is set to be lower as the degree of tilt in the pitch direction is lower (that is, as the user faces downward). On the other hand, the correction factor of the chin template is set to be higher as the degree of tilt in the pitch direction is higher (that is, as the user faces upward). After specifying the correction rate based on the set correspondence relationship in this way, the corrected alpha value can be calculated using the following <pitch direction correction degree correction formula>.

<Pitch direction correction degree correction formula>
Alpha value after correction=α×correction rate/100

In this way, even when the alpha value is corrected based on the pitch direction correction degree, as in the above-described case where the alpha value is corrected based on the yaw direction tilt degree, correction is performed to balance the user's face. can be realized.

An example of performing distortion correction after correcting the alpha value based on the degree of pitch direction correction in this way will be described with reference to FIGS. 14 and 15. FIG. 14 and 15 are schematic diagrams showing another example of an image subjected to distortion correction after correcting the alpha value in the thin face correction process.

As shown in FIG. 14B, when distortion correction is performed without correcting the alpha value, the face is facing downward and the area of the chin is relatively small over the entire face. , the chin is corrected as if the face were facing forward. Therefore, an image without a chin is obtained. On the other hand, as shown in FIG. 14A, when distortion correction is performed after correcting the alpha value based on the degree of pitch direction correction as described above, correction of the chin is suppressed. Balances the face and chin.

Further, as shown in FIG. 15B, when the distortion correction is performed without correcting the alpha value, even if the face is facing upward and the area of the chin is relatively large over the entire face, Regardless, the chin is corrected as if the face were facing forward. Therefore, an image in which the chin is conspicuous is obtained. On the other hand, as shown in FIG. 15A, when distortion correction is performed after correcting the alpha value based on the pitch direction correction degree as described above, the jaw is further corrected. Balances the face and chin.

The correction of the alpha value in this modified example has been described above. In this modified example, for one template, either the alpha value correction based on the pitch direction correction degree or the alpha value correction based on the yaw direction tilt degree may be performed, or both may be performed. you can go Alternatively, the alpha value may be corrected based on the degree of correction in the direction that differs from template to template.

<Operation of Modification 4>
FIG. 16 is a flowchart for explaining the flow of thin face correction processing according to this modification, which is executed by the imaging apparatus 1 of FIG. 1 having the functional configuration of FIG. Note that the functional blocks in this modification are basically the same as the functional blocks described above with reference to FIG. However, it further has functions for performing the following operations.

In step S<b>11 , the image acquisition unit 51 acquires image data obtained by developing an image captured by the imaging unit 16 or an image to be processed from the image storage unit 61 . At this time, in this modified example, the image acquisition unit 51 also acquires depth information corresponding to the original image together with the original image.

In step S12, the face detection unit 52 performs face detection on the image to be processed, and determines whether or not a face has been detected.
If the face is not detected, NO is determined in step S12, and the thin face correction process ends.
If a face is detected, YES is determined in step S12, and the process proceeds to step S13.

In step S13, the face detection unit 52 detects predetermined organs and predetermined parts of the face corresponding to the template. At this time, in this modified example, the face detection unit 52 detects depth information of predetermined organs and predetermined parts (for example, the above-described left cheek bottom, right cheek bottom, glabella, and upper lip). is also detected.

In step S14, the matching position specifying unit 53 specifies coordinates in the original image that match the template.

In step S21, the matching position specifying unit 53 specifies the degree of inclination based on the depth information of the predetermined organ or the predetermined part detected by the face detecting unit 52, as in <alpha value correction> described above. .

In step S22, the matching position specifying unit 53 corrects the alpha value of each template based on the degree of inclination specified in step S21, as in <correction of alpha value> described above.

In step S15, the alpha map creation unit 54 creates an alpha map based on the alpha values after correction in step S22 and the coordinates matching the template.

In step S16, the distortion correction processing unit 55 performs distortion correction using an alpha map.
After that, the thin face correction process ends.

By the operation described above, a well-balanced image of the user's face as described above with reference to FIGS. 13, 14, and 15 is generated.
In this way, in this modified example, for example, without performing complicated image analysis such as detecting the contour of the user's face line, only by calculating the degree of inclination from the depth information and correcting the template, Corrections can be made to balance the user's face. That is, also in this modified example, the correction process can be realized by a simpler method than the conventional method.

[Other Modifications]
Further, in the above-described embodiment, the imaging apparatus 1 to which the present invention is applied is described as an example of a digital camera, but it is not particularly limited to this.
For example, the present invention can be applied to general electronic devices having an image processing function. Specifically, for example, the present invention is applicable to notebook personal computers, printers, television receivers, video cameras, portable navigation devices, mobile phones, smart phones, portable game machines, and the like.

The series of processes described above can be executed by hardware or by software.
In other words, the functional configuration of FIG. 3 is merely an example and is not particularly limited. That is, it suffices if the imaging apparatus 1 has a function capable of executing the series of processes described above as a whole, and what kind of functional blocks are used to realize this function is not particularly limited to the example in FIG.
Also, one functional block may be composed of hardware alone, software alone, or a combination thereof.
The functional configuration in this embodiment is realized by a processor that executes arithmetic processing, and processors that can be used in this embodiment are composed of various single processing units such as single processors, multiprocessors, and multicore processors. In addition, it includes a combination of these various processing devices and a processing circuit such as ASIC (Application Specific Integrated Circuit) or FPGA (Field-Programmable Gate Array).

When a series of processes is to be executed by software, a program constituting the software is installed in a computer or the like from a network or a recording medium.
The computer may be a computer built into dedicated hardware. The computer may also be a computer capable of executing various functions by installing various programs, such as a general-purpose personal computer.

A recording medium containing such a program is not only constituted by the removable medium 31 of FIG. It consists of a recording medium, etc. provided to The removable medium 31 is composed of, for example, a magnetic disk (including a floppy disk), an optical disk, or a magneto-optical disk. Optical discs are composed of, for example, CD-ROMs (Compact Disk-Read Only Memory), DVDs (Digital Versatile Disks), Blu-ray (registered trademark) Discs, and the like. The magneto-optical disk is composed of an MD (Mini-Disk) or the like. Also, the recording medium provided to the user in a state of being pre-installed in the apparatus main body is composed of, for example, the ROM 12 in FIG. 1 in which the program is recorded, the hard disk included in the storage unit 19 in FIG.

In this specification, the steps of writing a program recorded on a recording medium are not only processes that are performed chronologically in that order, but also processes that are not necessarily chronologically processed, and that are performed in parallel or individually. It also includes the processing to be executed.

Although several embodiments of the present invention have been described above, these embodiments are merely examples and do not limit the technical scope of the present invention. The present invention can take various other embodiments, and various modifications such as omissions and substitutions can be made without departing from the gist of the present invention. These embodiments and their modifications are included in the scope and gist of the invention described in this specification and the like, and are included in the scope of the invention described in the claims and equivalents thereof.

The invention described in the scope of claims at the time of filing of the present application will be additionally described below.
[Appendix 1]
Acquisition means for acquiring an image showing a predetermined subject;
storage means for storing a template containing a degree of correction at each position within the shape corresponding to the region to be corrected of the predetermined subject, the degree of correction changing at each position;
identifying means for identifying coordinates that match the template within the image;
correction means for correcting the image based on the coordinates specified by the specifying means and the degree of correction included in the template;
An image processing device comprising:
[Appendix 2]
The specifying means is
detecting the coordinates corresponding to the first feature point of the predetermined subject and the coordinates corresponding to the second feature point of the predetermined subject by analyzing the image;
Identifying coordinates that match the base point in the template based on the detected coordinates of the first feature point, and identifying coordinates that match the contour in the template based on the detected coordinates of the second feature point;
The image processing apparatus according to Supplementary Note 1, characterized by:
[Appendix 3]
The specifying means is
Arranging the base point of the template at the coordinates of the first feature point, and rotating the template around the base point of the arranged template to specify the coordinates that match the template;
The image processing apparatus according to appendix 2, characterized by:
[Appendix 4]
The specifying means is
performing enlargement or reduction while maintaining the contour shape of the template based on the coordinates of the detected second feature point and the contour of the template;
The image processing apparatus according to appendix 2 or 3, characterized by:
[Appendix 5]
The specifying means is
Do not change the degree of correction included in the template,
The image processing apparatus according to appendix 4, characterized by:
[Appendix 6]
the identifying means identifies the state of the subject in the image, and modifies the degree of correction included in the template based on the identified state of the subject;
The correcting means corrects the image based on the degree of correction corrected by the specifying means and the coordinates specified by the specifying means.
5. The image processing apparatus according to any one of Appendices 1 to 4, characterized by:
[Appendix 7]
The specifying means specifies the degree of inclination of the subject as the state of the subject based on the difference between the depth of the first area of the subject in the image and the depth of the second area of the subject in the image. do,
The image processing apparatus according to appendix 6, characterized by:
[Appendix 8]
The correcting means is
As the correction of the image, distortion correction is performed by moving the coordinates of the image at least to a degree of correction corresponding to the degree of correction included in the template.
8. The image processing apparatus according to any one of Appendices 1 to 7, characterized by:
[Appendix 9]
The storage means stores a plurality of templates corresponding to a plurality of different correction target regions,
wherein the identifying means independently identifies matching coordinates for each of the plurality of templates;
The image processing apparatus according to any one of Appendices 1 to 8, characterized by:
[Appendix 10]
the predetermined subject is a face,
The correction target area is a facial organ or a facial part,
The image processing apparatus according to any one of Appendices 1 to 9, characterized by:
[Appendix 11]
11. The degree of correction of the template is set such that the degree of correction of the contour peripheral portion within the contour of the template is less than that of the central portion of the template. 2. The image processing device according to any one of the above.
[Appendix 12]
an acquisition step of acquiring an image showing a predetermined subject;
a storing step of storing a template containing a degree of correction at each position within a shape corresponding to the region to be corrected of the predetermined subject, the degree of correction varying at each position;
an identifying step of identifying coordinates in the image that match the template;
a correction step of correcting the image based on the coordinates identified in the identification step and the degree of correction included in the template;
An image processing method characterized by comprising:
[Appendix 13]
to the computer,
an acquisition function for acquiring an image of a predetermined subject;
a storage function for storing a template containing a degree of correction at each position within a shape corresponding to the region to be corrected of the predetermined subject, the degree of correction varying at each position;
a specific function that identifies coordinates in the image that match the template;
a correction function that corrects the image based on the coordinates specified by the specific function and the degree of correction included in the template;
An image processing program characterized by realizing

Reference Signs List 1 imaging device 11 CPU 12 ROM 13 RAM 14 bus 15 input/output interface 16 imaging unit 17 Input unit 18 Output unit 19 Storage unit 20 Communication unit 21 Drive 22 Illumination unit 31 Removable medium 51 Image acquisition Part, 52... Face detection part, 53... Matching position specifying part, 54... Alpha map creation part, 55... Distortion correction processing part, 61... Image storage part, 62... Template storage unit

Claims (10)

Acquisition means for acquiring an image showing a predetermined subject;
storage means for storing a template containing a degree of correction at each position within a shape corresponding to the region to be corrected of the predetermined subject, the degree of correction varying at each position;
identifying means for identifying coordinates that match the template within the image;
correction means for correcting the image based on the coordinates specified by the specifying means and the degree of correction included in the template;
has
The identifying means identifies the degree of tilt of the subject based on a difference between the depth of the first area of the subject in the image and the depth of the second area of the subject in the image, and identifies the degree of inclination of the subject. correcting the degree of correction included in the template based on the degree of inclination of the subject;
The correcting means corrects the image based on the degree of correction corrected by the specifying means and the coordinates specified by the specifying means.
An image processing apparatus characterized by: The specifying means is
detecting the coordinates corresponding to the first feature point of the predetermined subject and the coordinates corresponding to the second feature point of the predetermined subject by analyzing the image;
Identifying coordinates that match the base point in the template based on the detected coordinates of the first feature point, and identifying coordinates that match the contour in the template based on the detected coordinates of the second feature point;
2. The image processing apparatus according to claim 1, wherein: The specifying means is
Arranging the base point of the template at the coordinates of the first feature point, and rotating the template around the base point of the arranged template to specify the coordinates that match the template;
3. The image processing apparatus according to claim 2, wherein: The specifying means is
performing enlargement or reduction while maintaining the contour shape of the template based on the coordinates of the detected second feature point and the contour of the template;
4. The image processing apparatus according to claim 2, wherein: The correcting means is
As the correction of the image, distortion correction is performed by moving the coordinates of the image at least to a degree of correction corresponding to the degree of correction included in the template.
5. The image processing apparatus according to any one of claims 1 to 4 , characterized by: The storage means stores a plurality of templates corresponding to a plurality of different correction target regions,
wherein the identifying means independently identifies matching coordinates for each of the plurality of templates;
6. The image processing apparatus according to any one of claims 1 to 5 , characterized by: the predetermined subject is a face,
The correction target area is a facial organ or a facial part,
7. The image processing apparatus according to any one of claims 1 to 6 , characterized by: The degree of correction of the template is set such that the portion around the contour within the contour of the template is less corrected than the central portion of the template.
8. The image processing apparatus according to any one of claims 1 to 7 , characterized by: An image processing method executed by an image processing device,
an acquisition step of acquiring an image showing a predetermined subject;
a storing step of storing a template containing a degree of correction at each position within the shape corresponding to the region to be corrected of the predetermined subject, the degree of correction varying at each position;
an identifying step of identifying coordinates in the image that match the template;
a correction step of correcting the image based on the coordinates identified in the identification step and the degree of correction included in the template;
has
The identifying step identifies the degree of inclination of the subject based on a difference between the depth of the first area of the subject in the image and the depth of the second area of the subject in the image, and identifies the degree of inclination of the subject. correcting the degree of correction included in the template based on the degree of inclination of the subject;
The correcting step corrects the image based on the degree of correction corrected in the identifying step and the coordinates identified in the identifying step.
An image processing method characterized by: to the computer,
an acquisition function for acquiring an image of a predetermined subject;
a storage function for storing a template containing a degree of correction at each position within a shape corresponding to the region to be corrected of the predetermined subject, the degree of correction varying at each position;
a specific function that identifies coordinates in the image that match the template;
a correction function that corrects the image based on the coordinates specified by the specific function and the degree of correction included in the template;
to realize
The specifying function specifies the degree of inclination of the subject based on a difference between the depth of the first region of the subject in the image and the depth of the second region of the subject in the image, and specifies the degree of inclination of the subject. correcting the degree of correction included in the template based on the degree of inclination of the subject;
The correction function corrects the image based on the degree of correction corrected by the specific function and the coordinates specified by the specific function.
An image processing program characterized by:

Images