JP2019220130A - Image processor, image-processing method, and image-processing program - Google Patents

Abstract

To provide an image processor, an image-processing method, and an image-processing program capable of achieving correction processing by a simpler method.SOLUTION: An imaging apparatus includes: an image acquisition part 51; a template storage part 62; a conforming position location part 53; and a distortion correction processing part 55. The image acquisition part 51 acquires an image for showing a predetermined object. The template storage part 62 stores a template including the degree of correction that is the degree of correction in each position within a shape corresponding to a correction target area of the predetermined object and changes in each position. The conforming position location part 53 locates coordinates conforming to a template within an image. The distortion correction processing part 55 performs correction to an image on the basis of the coordinates located by the conforming position location part 53 and the degree of correction included in the template.SELECTED DRAWING: Figure 3

Description

  The present invention relates to an image processing device, an image processing method, and an image processing program.

  Conventionally, a predetermined correction process has been performed on an image of a subject obtained by imaging. There are various types of predetermined correction processing. For example, when a user's face is photographed as a subject, the user's face is corrected to be small by performing distortion correction on the user's face portion in the image. Can be. A technique relating to such distortion correction is disclosed in, for example, Patent Document 1.

JP 2012-142772 A

However, in order to use a general technique such as the technique disclosed in Patent Literature 1, complicated image analysis processing is first performed on an image of a subject, and the position of an organ to be corrected and the It was necessary to specify the entire contour of the organ to be performed with high accuracy. For example, when correcting the user's eyes, it is necessary to specify the position of the user's eyes and the entire outline of the eyes with high accuracy.
It is desired that the correction process be realized by a simpler method without requiring such an accurate analysis of the entire contour by the image analysis process.

  The present invention has been made in view of such a situation, and an object of the present invention is to realize a correction process by a simpler method.

In order to achieve the above object, an image processing device according to one embodiment of the present invention includes:
Acquiring means for acquiring an image of a predetermined subject,
Storage means for storing a template including a correction degree of each position in the shape corresponding to the correction target region of the predetermined subject, the correction degree changing at each position;
Specifying means for specifying coordinates that match the template in the image,
Correction means for correcting the image based on the coordinates specified by the specifying means and the correction degree included in the template;
It is characterized by having.

  According to the present invention, the correction processing can be realized by a simpler method.

FIG. 1 is a block diagram illustrating a hardware configuration of an imaging device according to an embodiment of an image processing device of the present invention. FIG. 2 is a schematic diagram for describing an outline of a fine face correction process performed by the imaging device of FIG. 1. FIG. 2 is a functional block diagram illustrating a functional configuration for executing a fine face correction process among the functional configurations of the imaging device in FIG. 1. FIG. 9 is a schematic diagram illustrating a relationship between an example of a template shape and an alpha value and a face part to which the template is applied. FIG. 9 is a schematic diagram for explaining adaptation of a template to an original image. FIG. 9 is a schematic diagram for explaining generation of an alpha map using a plurality of templates. FIG. 9 is a schematic diagram illustrating an example of an original image before performing correction. FIG. 8 is a schematic diagram illustrating an example of an image obtained by correcting the original image illustrated in FIG. 7 by a conventional technique. FIG. 8 is a schematic diagram illustrating an example of an image obtained by correcting the original image illustrated in FIG. 7 by a fine face correction process performed by the imaging device in FIG. 1. 4 is a flowchart illustrating a flow of a fine face correction process performed by the imaging device 1 of FIG. 1 having the functional configuration of FIG. 3. FIG. 2 is a schematic diagram for explaining a case where a trimming process is performed in the fine face correction process performed by the imaging apparatus of FIG. 1. FIG. 3 is a schematic diagram for explaining correction of an alpha value by the imaging device. FIG. 9 is a schematic diagram illustrating an example of an image in which distortion has been corrected after an alpha value has been corrected in a fine face correction process. FIG. 10 is a schematic diagram illustrating another example of an image in which distortion has been corrected after the alpha value has been corrected in the fine face correction process. FIG. 10 is a schematic diagram illustrating another example of an image in which distortion has been corrected after the alpha value has been corrected in the fine face correction process. 11 is a flowchart illustrating a flow of a fine face correction process according to Modification 4 executed by the imaging device 1 of FIG. 1 having the functional configuration of FIG.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

FIG. 1 is a block diagram illustrating a hardware configuration of an imaging device 1 according to an embodiment of the image processing device of the present invention.
The imaging device 1 is configured as, for example, a digital camera.

  As shown in FIG. 1, the imaging device 1 includes a CPU (Central Processing Unit) 11, a ROM (Read Only Memory) 12, a RAM (Random Access Memory) 13, a bus 14, an input / output interface 15, It includes a unit 16, an input unit 17, an output unit 18, a storage unit 19, a communication unit 20, a drive 21, and a lighting unit 22.

  The CPU 11 executes various processes according to a program recorded in the ROM 12 or a program loaded from the storage unit 19 to the RAM 13.

  The RAM 13 also appropriately stores data and the like necessary for the CPU 11 to execute various processes.

  The CPU 11, the ROM 12, and the RAM 13 are mutually connected via a bus 14. The bus 14 is also connected to an input / output interface 15. The imaging unit 16, the input unit 17, the output unit 18, the storage unit 19, the communication unit 20, the drive 21, and the illumination unit 22 are connected to the input / output interface 15.

  Although not shown, the imaging unit 16 includes an optical lens unit and an image sensor.

The optical lens unit includes a lens that condenses light, such as a focus lens and a zoom lens, for photographing a subject.
The focus lens is a lens that forms a subject image on the light receiving surface of the image sensor. A zoom lens is a lens that changes the focal length freely within a certain range.
The optical lens unit is also provided with a peripheral circuit for adjusting setting parameters such as focus, exposure, and white balance, as necessary.

The image sensor includes a photoelectric conversion element, an AFE (Analog Front End), and the like.
The photoelectric conversion element includes, for example, a CMOS (Complementary Metal Oxide Semiconductor) type photoelectric conversion element. A subject image is incident on the photoelectric conversion element from the optical lens unit. Therefore, the photoelectric conversion element photoelectrically converts (images) a subject image, accumulates image signals for a certain period of time, and sequentially supplies the accumulated image signals to the AFE as analog signals.
The AFE executes various signal processing such as A / D (Analog / Digital) conversion processing on the analog image signal. A digital signal is generated by various signal processing, and is output as an output signal of the imaging unit 16.
Such an output signal of the imaging unit 16 is appropriately supplied as data of a captured image to the CPU 11, an image processing unit (not shown), and the like.

The input unit 17 is configured by various buttons and the like, and inputs various information in accordance with a user's instruction operation.
The output unit 18 includes a display, a speaker, and the like, and outputs images and sounds.
The storage unit 19 is configured by a hard disk, a DRAM (Dynamic Random Access Memory), or the like, and stores data of various images.
The communication unit 20 controls communication performed with another device (not shown) via a network including the Internet.

  In the drive 21, a removable medium 31 made of a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is appropriately mounted. The program read from the removable medium 31 by the drive 21 is installed in the storage unit 19 as needed. In addition, the removable medium 31 can store various data such as image data stored in the storage unit 19 in the same manner as the storage unit 19.

  The illumination unit 22 is, for example, separated from eight LEDs (Light Emitting Diodes) as light-emitting members installed in a circle around the imaging unit 16 and separated from the eight LEDs in the imaging device 1. And one LED installed at a different position. These LEDs selectively emit light during live view shooting, image recording, and the like in accordance with a user operation. By photographing the face while changing the arrival direction of the light beam for illumination in this way, a face image in which shadows are artificially controlled can be obtained.

The imaging apparatus 1 configured as described above has a function of performing distortion correction using a template on an image obtained by capturing a subject, thereby naturally correcting the subject without a sense of incongruity. Thus, for example, when the user's face is set as the subject, the user's face can be made a natural fine face without a sense of discomfort.
Hereinafter, a series of processes performed by the imaging device 1 to perform the distortion correction using the template will be referred to as “small face correction process”. Next, an outline of the fine face correction process will be described with reference to FIG.

[Overview of fine face correction processing]
FIG. 2 is a schematic diagram for explaining the outline of the fine face correction process performed by the imaging device 1.
FIG. 2 illustrates an example in which local distortion correction is performed on a region including a nose muscle as a predetermined organ included in the face and left and right face line regions as predetermined portions.

  As illustrated in FIG. 2A, the imaging device 1 acquires an image before correction (hereinafter, referred to as an “original image”) as a target of the fine face correction process. In addition, the imaging device 1 specifies an area to be subjected to distortion correction (hereinafter, referred to as a “correction target area”) in the original image. For this purpose, the imaging apparatus 1 stores a template representing the likeness of a predetermined organ or a predetermined part of the face in accordance with each of the predetermined organ or part. Further, the imaging device 1 adapts this template to the position of a predetermined organ or a predetermined site corresponding to the original image. Here, the template includes a value (hereinafter, referred to as an “alpha value”) indicating the degree of correction at each position in the shape corresponding to the face correction target area included in the original image.

  Further, as shown in FIG. 2B, the imaging apparatus 1 sets the alpha value at all coordinates in the original image to “0” for the background (hereinafter, referred to as “black background”). By arranging each template based on the matching result, a map indicating the degree of correction of each position in the original image (hereinafter, an "alpha map" is generated).

  Further, as shown in FIG. 2C, the imaging device 1 performs distortion correction on the original image. In this case, the imaging device 1 performs the distortion correction at a degree based on the degree of correction at each coordinate included in the alpha map. Specifically, in the distortion correction, the correction is realized by moving each coordinate of the original image in a predetermined direction. For example, when the user's face is a fine face, the correction is realized by moving each coordinate in the correction target area toward the center of the face. The imaging apparatus 1 performs distortion correction by determining the amount of movement of each coordinate according to the degree based on the degree of correction at each coordinate included in the alpha map.

  According to such an imaging device 1, distortion correction can be performed only on the correction target region corresponding to the template without performing unnecessary correction on the region other than the correction target region. In addition, it is possible to perform appropriate distortion correction for each organ or each predetermined part according to the alpha value included in the template, instead of performing distortion correction using a uniform correction degree.

In addition, the imaging apparatus 1 uses a template having a shape that represents the likeness of a predetermined organ or a predetermined part as a correction target area, and the degree of correction is greater at the periphery of the outline within the outline of the template than at the center. It is set to be suppressed. Therefore, even if the outline of a predetermined organ or a predetermined part serving as a correction target area does not completely match the outline of the template, the influence on the periphery of the outline is small, and the object as a whole is Correction can be made naturally without discomfort. Therefore, according to the imaging apparatus 1, it is not necessary to detect the contour of an organ or a predetermined part with high accuracy. Further, even when the contour of an organ or a predetermined part is not detected with high accuracy, distortion correction can be performed without substantially lowering the quality of the corrected image.
In this regard, in the related art, in order to perform appropriate distortion correction, the position of the user's organ or a predetermined part to be corrected by the user and the entire contour of the organ or the predetermined part are accurately identified. I needed to. On the other hand, as described above, the imaging apparatus 1 does not need to accurately detect the contour of an organ or a predetermined part, and performs distortion correction without substantially lowering the quality of the corrected image. Can be.
That is, according to the present embodiment, the correction process can be realized by a simpler method.

[Function block]
FIG. 3 is a functional block diagram showing a functional configuration for executing a fine face correction process among the functional configurations of the imaging apparatus 1 of FIG.
As described above, the fine face correction processing refers to a series of processing for performing distortion correction using a template.

When performing the fine face correction processing, as shown in FIG. 3, in the CPU 11, the image acquisition unit 51, the face detection unit 52, the matching position specifying unit 53, the alpha map creation unit 54, the distortion correction processing The unit 55 functions.
In one area of the storage unit 19, an image storage unit 61 and a template storage unit 62 are set.

The image storage unit 61 stores data of a captured image output from the imaging unit 16 and obtained by subjecting the image captured by the imaging unit 16 to a development process.
The template storage unit 62 stores template data having shapes corresponding to various parts of the face. Details of the template will be described with reference to FIG.

  The image acquisition unit 51 acquires data of a captured image obtained by performing a development process on an image captured by the imaging unit 16 or data of an image to be processed from the image storage unit 61.

  The face detection unit 52 detects a face from the original image. In addition, the face detection unit 52 detects a feature point of a predetermined organ and a predetermined site corresponding to the correction target area. Details of the processing by the face detection unit 52 will be described later with reference to FIG.

  The matching position specifying unit 53 specifies coordinates in the original image that match the template. Details of the processing by the matching position specifying unit 53 will be described later with reference to FIG.

  The alpha map creation unit 54 creates an alpha map based on coordinates that match the template. Details of the processing by the alpha map creation unit 54 will be described later with reference to FIG.

  The distortion correction processing unit 55 performs distortion correction based on the alpha value included in the alpha map. Details of the processing by the distortion correction processing unit 55 will be described later after the description of the alpha map generation unit 54 with reference to FIG.

[Specific example of template]
Next, details of the template stored in the template storage unit 62 will be described with reference to FIG. FIG. 4 is a schematic diagram illustrating a relationship between an example of the shape and alpha value of the template and a part of the face to which the template is applied.
Here, the template is a template that simulates the shape of a correction target region corresponding to a predetermined organ or a predetermined portion of the face in a general face, and represents a specific portion of the face. In addition, as described above, the alpha value indicating the degree of correction of each position in the shape corresponding to the correction target area is included.

  In FIG. 4, “nose template T1” corresponding to the nose, “face line right template T2” corresponding to the right of the face line, and “face template corresponding to the left of the face line” An example in which the "line left side template T3" is also applied. In FIG. 4 and FIG. 5 to be described later, the template is shown as a closed area indicated by a broken line. Also, in these figures, the expression of the alpha value included in the template is shown in black when the value is “0”, in white when the value is “1”, and The case of “an intermediate value from 0 to 1” is indicated by hatching.

  As shown in FIG. 4, the nose template T1 is used to reduce the entire nose by setting the entire nose as a correction target area. The nose template T1 has an alpha value set to “1” so that the area in the center of the nose is particularly reduced, and has an alpha value set to “0” so that the area outside the contour of the nose is not reduced. The "intermediate value between 0 and 1" is set so that the portion around the contour within the contour of the nose is more suppressed than the central portion. By performing distortion correction based on the alpha value set so that the degree of correction gradually changes as it approaches the center of the nose, the effect of reducing the nose more naturally and making it a fine face can be achieved. can get.

  Further, as shown in FIG. 4, the right face line template T2 and the left face line template T3 each use the left and right face lines as the correction target areas, thereby forming the face lines of the left and right cheeks (particularly, the gill part). Is used to reduce The right face line template T2 and the left face line template T3 are symmetrical with respect to the center line of the face so that the left and right cheeks are uniformly reduced.

  In addition, the face line right side template T2 and the face line left side template T3 have an alpha value set to “1” so that the area of the center part of the face line is particularly reduced, and the area outside the outline of the face line is The alpha value is set to “0” so as not to be reduced, and the “intermediate value between 0 and 1” is set so that the portion around the contour within the contour of the face line is more reduced than the center portion. By performing distortion correction based on such an alpha value that is set so that the degree of correction gradually changes toward the center of the face line, the left and right face lines are reduced more naturally, Is obtained.

  In order to reduce the amount of data, only one template having a symmetric shape is stored, and the template can be symmetrically converted before use. For example, a template corresponding to the left face line is stored, and is symmetrically transformed by performing left-right inversion when used, and can be used as a template corresponding to the right face line.

[Conformance of template]
Next, the adaptation of the template to the original image performed by the face detection unit 52 and the adaptation position identification unit 53 will be described. FIG. 5 is a schematic diagram for explaining the adaptation of the template to the original image. FIG. 5 illustrates an example in which the nose template T1 is adapted to the nose in the original image acquired by the image acquisition unit 51.

When matching the template, as shown in FIG. 5A, the face detection unit 52 detects a feature point on the original image. Specifically, the face detection unit 52 detects a point serving as a base point and a contour point as feature points. The base point is, for example, a point corresponding to the center of the nose, such as a base point P11 in the drawing. In addition, for example, points corresponding to the contour points P21 and P22 in the figure correspond to the upper and lower ends of the nose, and points corresponding to the left and right sides of the nose such as the contour points P31 and P32 in the figure. It is a point corresponding to both ends.
The processing related to the adaptation described with reference to FIG. 5B and thereafter is performed based on the coordinates of the thus detected feature points in the original image. Note that the face detection and the detection of each feature point by the face detection unit 52 can use an existing face detection technology and a feature point detection technology.

  Next, as shown in FIG. 5B, the conforming position specifying unit 53 performs position adjustment of the template based on the base point. Specifically, the matching position specifying unit 53 aligns the coordinates of the center of the nose template T1 with the coordinates of the center of the base point P11 to perform the template alignment.

  Next, as shown in FIG. 5C, the matching position specifying unit 53 rotates the template based on the angle. Here, as the angle, as shown in the drawing, for example, an angle A1 which is an angle formed between a straight line connecting the contour points P31 and P32 and a straight line parallel to the X axis can be used. Since the angle A1 is an angle indicating the inclination of the nose, the matching position specifying unit 53 rotates the nose template T1 by the same angle as the angle A1 around the base point P11 as a rotation center, thereby changing the angle of the template. Can be adapted.

  Finally, as shown in FIG. 5D, the matching position specifying unit 53 enlarges or reduces the template based on the position of the contour point. That is, the size of the template is adapted. Specifically, the matching position specifying unit 53 maintains the shape of the nose template T1 such that both ends of the nose template T1 have the same size as both ends of the nose indicated by the contour points P31 and P32. The nose template T1 is enlarged or reduced as it is. In this case, the conforming position specifying unit 53 maintains the alpha value included in the nose template T1 without changing.

  As described above, the template can be adapted to the original image as described with reference to FIGS. 5A to 5D. This adaptation can be realized by detecting only the base point and a plurality of contour points, as described above. That is, it is not necessary to specify the entire contour of the organ or part to be corrected with high accuracy. That is, according to the present embodiment, the correction process can be realized by a simpler method.

  Further, when there are a plurality of templates, coordinates suitable for the plurality of templates are specified independently for each of the plurality of templates. Therefore, even if the face in the original image is nearly horizontal or inclined, and the angle or size of the face line is not symmetric, the template is adjusted to fit the left and right face lines. The angle and size can be adjusted. That is, according to the present embodiment, a common template prepared in advance can be adapted to each of various original images having different face directions and sizes.

  In the above description, the adaptation is performed using the base point P11, the contour point P31, and the contour point P32. However, the present invention is not limited thereto, and the adaptation may be performed using another contour point. This point will be described later as a first modification.

[Create Alpha Map]
Next, creation of an alpha map performed by the alpha map creation unit 54 will be described. FIG. 6 is a schematic diagram for explaining generation of an alpha map using a plurality of templates.
As shown in FIG. 6A, the alpha map creating unit 54 acquires coordinates that are specified by the matching position specifying unit 53 and that are suitable for each of the plurality of templates.

  As shown in FIG. 6B, the alpha map creating unit 54 arranges each template on the matching coordinates on the black background based on the coordinates matching each of the plurality of templates. Note that the black background and the original image are managed using the same coordinate system.

  As shown in FIG. 6C, the alpha map is created by placing all templates on a black background by the alpha map creating unit 54.

[Distortion correction based on alpha map]
Next, the distortion correction performed by the distortion correction processing unit 55 based on the alpha value included in the alpha map will be described. As described above with reference to FIG. 2, in the present embodiment, distortion correction is performed on the original image based on the alpha value included in the alpha map.

  Specifically, the distortion correction processing unit 55 first calculates the coordinates after the distortion correction when performing the distortion correction without using the alpha map for all the coordinates of the original image. For example, the coordinates after the distortion correction such that the face is reduced are calculated for all the coordinates of the original image for the entire face area. It should be noted that an existing distortion correction technique can be used to calculate the coordinates after the distortion correction.

  As a result, the coordinates after the distortion correction (hereinafter, referred to as “the coordinates before applying the alpha map”) in the case where the distortion correction is performed without using the alpha map are calculated for all the coordinates of the original image.

  Next, the distortion correction processing unit 55 corrects the degree of correction at each coordinate by applying an alpha map. Specifically, based on the following <alpha map application formula>, the distortion correction processing unit 55 calculates, for all the coordinates of the original image, the coordinates after the distortion correction using the alpha map (hereinafter, “the coordinates after the alpha map is applied”). (Referred to as "coordinates").

<Alpha map application formula>
(X _out , y _out ) = (x _in , y _in ) × (1−α) + (x _dist , y _dist ) × α
However,
(X _in , y _in ): coordinates of the original image (x _out , y _out ): coordinates after alpha map application (x _dist , y _dist ): correction coordinates before alpha map application. Further, the value of α takes a value of (0 ≦ α ≦ 1).

  Then, the distortion correction processing unit 55 performs correction based on the calculated coordinates after application of the alpha map. That is, the coordinates of the original image are moved to the coordinates after the application of the alpha map. Further, the distortion correction processing unit 55 performs, for example, a supplementary process necessary for the movement using the existing technology.

Thus, distortion correction using the alpha map is realized. By applying the alpha map in this manner, for white coordinates in the alpha map (that is, coordinates having an alpha value of “1”), distortion correction is performed using the correction coordinates before the alpha map is applied. Also, for the coordinates of the intermediate color in the alpha map (that is, the coordinates where the alpha value is “0 <α <1”), distortion correction is performed in which the correction coordinates before application of the alpha map are applied according to the alpha value. Further, for black coordinates in the alpha map (that is, coordinates having an alpha value of “0”), the correction coordinates before the application of the alpha map are not applied, and the coordinates of the original image remain unchanged.
That is, local distortion correction corresponding to the correction degree in the plurality of templates described above is performed. The effect of such distortion correction will be described with reference to FIGS. 7, 8, and 9. FIG.

  FIG. 7 shows an example of an original image before performing distortion correction. FIG. 8 shows an example of an image obtained by correcting the original image shown in FIG. 7 by a conventional technique (that is, distortion correction without using an alpha map). Further, FIG. 9 shows an example of an image obtained by correcting the original image shown in FIG. 7 by the distortion correction using the alpha map in the present embodiment described above.

  When the original image of FIG. 7 is compared with the image corrected by the conventional technology of FIG. 8, the conventional technology does not use the alpha map and performs the distortion correction on the correction target area as the entire face. Are evenly corrected. In this case, since the entire face is reduced, the head tends to be relatively elongated. Also, it is difficult to make the chin thin because the entire chin is uniformly reduced. Therefore, in the related art, the user may feel uncomfortable with the corrected image and feel unnatural.

  On the other hand, in the present embodiment, local distortion correction is performed in accordance with the degrees of correction in the plurality of templates described above. Therefore, when the original image in FIG. 7 is compared with the image corrected according to the present embodiment in FIG. 9, only the correction target area corresponding to the template is reduced instead of the entire face, so that the head is relatively extended. Can be prevented. Also, not the entire jaw, but only the correction target area corresponding to the left and right face lines is reduced, so that the jaw can be made thinner. Therefore, according to the present embodiment, the user's face can be made a natural fine face without discomfort.

[motion]
FIG. 10 is a flowchart illustrating the flow of the fine face correction process performed by the imaging apparatus 1 of FIG. 1 having the functional configuration of FIG.
The fine face correction processing is started by an operation of the user to start the fine face correction processing on the input unit 17.
The operation of starting the fine face correction process can be a shooting instruction operation. The developing process is performed on the image captured by the imaging unit 16 in response to the shooting instruction operation, and the fine face correction process is subsequently performed. It can be carried out. Alternatively, the data of the captured image stored in the image storage unit 61 may be selected, and an operation of starting the fine face correction process may be performed on the data of the selected captured image.

  In step S <b> 11, the image acquiring unit 51 acquires data of a captured image in which the image captured by the imaging unit 16 has been subjected to development processing, or an image to be processed from the image storage unit 61.

In step S12, the face detection unit 52 performs face detection on the image to be processed, and determines whether a face has been detected.
If no face is detected, NO is determined in step S12, and the fine face correction process ends.
If a face is detected, YES is determined in step S12, and the process proceeds to step S13.

  In step S13, the face detection unit 52 detects a predetermined organ and a predetermined part of the face corresponding to the template.

  In step S14, the matching position specifying unit 53 specifies coordinates in the original image that match the template.

  In step S15, the alpha map creation unit 54 creates an alpha map based on coordinates that match the template.

In step S16, the distortion correction processing unit 55 performs distortion correction using an alpha map.
Then, the fine face correction processing ends.
By the operation described above, an image in which the user's face is a natural fine face without discomfort as described above with reference to FIG. 9 is generated.

The imaging device 1 configured as described above includes an image acquisition unit 51, a template storage unit 62, a matching position specifying unit 53, and a distortion correction processing unit 55.
The image acquisition unit 51 acquires an image in which a predetermined subject is captured.
The template storage unit 62 stores a template including a correction degree at each position in a shape corresponding to a correction target area of a predetermined subject, the correction degree changing at each position.
The matching position specifying unit 53 specifies coordinates that match the template in the image.
The distortion correction processing unit 55 corrects the image based on the coordinates specified by the matching position specifying unit 53 and the correction degree included in the template.
Accordingly, since the correction can be realized by adapting the template, it is possible to eliminate the necessity of detecting the contour of the organ or the predetermined part with high accuracy. Therefore, the correction process can be realized by a simpler method. Further, even when the contour of an organ or a predetermined part is not detected with high accuracy, the correction can be performed without substantially lowering the quality of the corrected image.
Further, the distortion correction can be performed only on the correction target region corresponding to the template without performing unnecessary correction on the region other than the correction target region. In addition, it is possible to perform appropriate distortion correction for each organ or each predetermined part in accordance with the correction degree included in the template, instead of performing the distortion correction using a uniform correction degree.

The conforming position specifying unit 53
By analyzing the image, coordinates corresponding to a first feature point of a predetermined subject and coordinates corresponding to a second feature point of the predetermined subject are detected,
Based on the detected coordinates of the first feature point, coordinates that match the base point in the template are specified, and based on the coordinates of the detected second feature point, coordinates that match the contour in the template are specified.
Thus, by detecting only the feature points, it is possible to specify coordinates that match the template.

The conforming position specifying unit 53
The base point of the template is arranged at the coordinates of the first feature point, and the template is rotated by using the base point of the arranged template as a rotation center to specify the coordinates that match the template.
This makes it possible to adapt a common template by simply rotating each of various original images having different face directions and sizes, without preparing a plurality of templates corresponding to each angle. .

The conforming position specifying unit 53
Based on the detected coordinates of the second feature point and the outline of the template, enlargement or reduction is performed while maintaining the outline shape of the template.
This makes it possible to adapt a common template only by enlarging or reducing various original images having different face directions and sizes without preparing a plurality of templates having different sizes. it can.

The conforming position specifying unit 53
The correction degree included in the template is not changed.
Thus, the correction can be performed at a correction degree corresponding to the template, regardless of the size of the predetermined subject included in the original image.

The matching position specifying unit 53 specifies the state of the subject in the image, and corrects the degree of correction included in the template based on the specified state of the subject.
The distortion correction processing unit 55 corrects the image based on the correction degree corrected by the conforming position specifying unit 53 and the coordinates specified by the conforming position specifying unit 53.
Thereby, distortion correction can be performed based on the correction degree more appropriately corrected based on the state of the subject. For example, when the face rotates left and right about the vertical axis of the face (that is, when yawing), and when the face rotates vertically about the horizontal axis of the face (that is, when pitching), Distortion correction can be performed based on the correction degree correspondingly corrected. Therefore, for example, distortion correction can be performed in consideration of the balance of the entire face.

The matching position specifying unit 53 specifies the degree of inclination of the subject as the state of the subject based on the difference between the depth of the first region of the subject in the image and the depth of the second region of the subject in the image.
Thereby, the degree of inclination of the subject can be specified based on a clear index called depth information. Therefore, the degree of correction can be corrected to an extent corresponding to the degree of inclination of the subject. For example, when the angle is more inclined, the correction can be made such that the degree of correction is increased (or weakened).

The distortion correction processing unit 55
As the correction for the image, the distortion correction is performed by moving the coordinates of the image with at least the correction degree corresponding to the correction degree included in the template.
Thus, it is possible to perform the natural distortion correction without any discomfort at the correction degree corresponding to the template.

The template storage unit 62 stores a plurality of templates corresponding to a plurality of different correction target areas, respectively.
The matching position specifying unit 53 specifies the coordinates that match the plurality of templates independently for each of the plurality of templates.
This makes it possible to adapt a common template prepared in advance to various original images having different face directions and sizes.

The predetermined subject is a face,
The correction target area is a facial organ or a face part.
As a result, the face of the subject can be made a natural fine face without discomfort.

The degree of correction of the template is set such that a part around the outline within the outline of the template has a lower degree of correction than a central part of the template.
Thus, even if the outline of a predetermined organ or a predetermined part serving as a correction target area and the outline of the template do not completely match, the influence on the periphery of the outline is small, and the object as a whole is Can be corrected naturally without discomfort.

  It should be noted that the present invention is not limited to the above-described embodiment, but includes modifications and improvements as long as the object of the present invention can be achieved. For example, the above-described embodiment may be modified to provide a modification as exemplified below. Further, modifications as exemplified below can be combined.

[Modification 1]
In the example described above with reference to FIG. 5, the matching position specifying unit 53 performs the matching using the base point P11, the contour point P31, and the contour point P32. However, the present invention is not limited to this. The adaptation may be performed by using this.
For example, the matching may be performed using the contour points P21 and P22 instead of the contour points P31 and P32. In this case, for example, as shown in FIG. 5C, when rotating the template, an angle (hereinafter, referred to as an angle) formed between a straight line connecting the contour points P21 and P22 and a straight line parallel to the Y-axis. "Angle A2") may be used instead of angle A1. Further, for example, as shown in FIG. 5D, when the template is enlarged or reduced, both ends of the nose template T1 are the same size as both ends of the nose indicated by the contour points P21 and P22. Alternatively, the nose template T1 may be enlarged or reduced while maintaining the shape of the nose template T1.

  Alternatively, for example, the matching may be performed using the contour points P21 and P22 together with the contour points P31 and P32. For example, as shown in FIG. 5C, when rotating the template, the template may be rotated based on the average of the angle indicated by the angle A1 and the angle indicated by the angle A2. For example, as shown in FIG. 5D, when the template is enlarged or reduced, the difference between both ends of the nose template T1 and both ends of the nose indicated by the contour points P21 and P22, and the nose Of the nose template T1 while maintaining the shape of the nose template T1 such that the sum of two differences between both ends of the nose template T1 and the difference between the ends of the nose indicated by the contour points P31 and P32 is minimized. T1 may be enlarged or reduced.

  Alternatively, for example, in the case where the possibility that the user's nose is tilted in the original image is low and the size of the nose can be roughly assumed, the template is rotated as shown in FIG. The processing and the processing for enlarging or reducing the template as shown in FIG. 5D may be omitted. In this case, only the base point P11 needs to be detected, and it is not necessary to detect each contour point.

[Modification 2]
In the above-described embodiment, the alpha map corresponding to the entire original image is created, and the distortion correction is performed on the entire original image based on the alpha value included in the alpha map. The present invention is not limited to this, and the original image may be partially cut out by performing trimming, and distortion correction may be performed on the cut out image. This modification will be described with reference to FIG.

  First, as shown in FIG. 11A, the face detecting unit 52 cuts out an area including a predetermined organ and a predetermined part corresponding to the template from the original image by trimming based on the detection result of the feature points.

  Next, as shown in FIG. 11B, the cut-out original image corresponding to the cut-out original image by the template matching process by the matching position specifying unit 53 and the alpha map creating process by the alpha map creating unit 54. Create an alpha map for

Next, as shown in FIG. 11C, the distortion correction processing unit 55 performs distortion correction on the cut out original image using the cut out original image alpha map.
Finally, as shown in FIG. 11D, the face detection unit 52 pastes the corrected cut-out image to the original image.

With the processing described above, the same correction as in the above-described embodiment can be performed. For example, the present modified example may be applied to a case where a process of trimming a region including a predetermined organ and a predetermined site corresponding to a template from an original image by trimming can be easily performed.
By performing the cutout by trimming in this manner, the number of candidate coordinates for matching the template can be reduced in the template matching process performed by the matching position specifying unit 53. Therefore, the amount of calculation in the adaptation process can be reduced. In addition, by performing the cutout by trimming in this manner, in the distortion correction processing by the distortion correction processing unit 55, the number of coordinates for which the coordinates after the alpha map is calculated can be reduced by the <alpha map application formula>. Therefore, the amount of calculation in the adaptation process can be reduced. Therefore, it is possible to reduce the amount of calculation as the whole fine face correction process.

[Modification 3]
In the above-described embodiment, the distortion correction using the alpha map is performed on the user's face. The present invention is not limited to this, and distortion correction using an alpha map may be performed on a portion other than the user's face. Also, distortion correction using an alpha map may be performed for living things other than the user or non-living things. That is, as long as a template corresponding to the correction target can be prepared, distortion correction using an alpha map may be performed on any correction target.

  Further, in the above-described embodiment, a predetermined organ or a predetermined part is reduced in an image by distortion correction using an alpha map. The present invention is not limited to this, and a predetermined organ or a predetermined part may be enlarged in an image by distortion correction using an alpha map. For example, a template corresponding to both eyes of a user may be prepared, and both eyes of the user may be enlarged by distortion correction using an alpha map.

[Modification 4]
In the above-described embodiment, in the fine face correction processing, after the template is adapted to the original image, the distortion correction is performed based on the alpha value included in the template and indicating the degree of correction at each position. Was. However, the present invention is not limited thereto, and the alpha value included in the template may be corrected based on a predetermined condition such as the state of the user, and the distortion may be corrected based on the corrected alpha value. For example, the alpha value included in the template may be corrected based on the presence or absence and the degree of tilt of the user's face in the original image, and the distortion may be corrected based on the corrected alpha value. The case where the alpha value is corrected in this manner will be described below as a fourth modification.

<Correction of alpha value>
FIG. 12 is a schematic diagram for explaining correction of an alpha value in Modification 4 by the imaging device 1. FIG. 12 shows an example in which local distortion correction is performed on each of the left and right face line regions that are predetermined portions included in the face.

  FIG. 12 shows an original image 41 as an example of the original image. In this modification, as described above, the alpha value included in the template is corrected based on the inclination of the user's face. In the present variation, the inclination of the user's face is specified based on depth information (also referred to as depth information) indicating the depth of the user's face in a predetermined area. The depth information is information indicating the distance from the imaging device 1 to the subject (here, the face of the user) at the time of imaging. The depth information can be obtained by, for example, mounting a depth sensor capable of detecting a distance to a subject by infrared rays on the imaging device 1. In addition, the depth information can be obtained by implementing the imaging device 1 with a stereo camera that can detect the distance to the subject based on the principle of triangulation.

  Further, in this modification, the face rotates left and right about the vertical axis of the face (that is, yawing), and the face rotates vertically about the horizontal axis of the face (that is, pitches). In each case, the degree of inclination of the face is specified.

First, a case where the face is rotated left and right about the vertical axis of the face (that is, a case where yawing is performed) will be described as an example.
In this modified example, depth information is obtained together with detection of a face or the like by image analysis. Then, based on the depth information, the degree of inclination (hereinafter, referred to as “yaw direction inclination degree”) of the yawing 43 that rotates left and right around the vertical axis direction 42 of the face shown in FIG. 12 is specified. Note that the vertical axis direction 42 and the yawing 43 and the horizontal axis direction 44 and the pitching 45 described later are not actually displayed together with the original image 41, but are information shown for convenience of explanation.

  The determination of the degree of inclination can be calculated based on the difference between the depth information in two predetermined areas of the face. The predetermined area is not particularly limited as long as it is an area where the degree of yaw tilt can be calculated. In the following, as an example, the inclination degree in the yaw direction is calculated based on the depth information (depth value) under the left cheek and the depth information (depth value) under the right cheek as follows: >.

<Yaw direction inclination degree calculation formula>
yaw = d_l-d_r
However,
d_l: Depth value under the left cheek d_r: Depth value under the right cheek yaw: Degree of inclination in the yaw direction.

  Next, based on the degree of inclination in the yaw direction, the alpha value of the template corresponding to each of the left and right face line regions is corrected. This correction is performed for the purpose of adjusting the balance of the user's face when performing the distortion correction at the subsequent stage. For example, if it can be specified that the user is facing left (that is, right from the viewer who faces the image) based on the degree of inclination in the yaw direction, the degree of distortion correction of the user's left face line (ie, , Alpha value) and increase the degree of distortion correction for the user's right face line. As a result, the right face line of the user who is on the near side in the image is corrected for distortion more than the left face line, so that the right face line of the user becomes a finer face. Therefore, the balance between the left and right faces of the user is adjusted.

  Therefore, in the present modification, the alpha value is corrected so that the distortion correction is performed as described above. For this purpose, the correspondence between the degree of inclination in the yaw direction and the correction rate is set in advance for each template. This setting may be stored, for example, in a table format indicating the correspondence, or may be stored as a predetermined relational expression. Then, when the degree of inclination in the yaw direction is calculated, the corresponding correction rate can be specified by referring to this table or performing an operation using a relational expression.

  FIG. 12 is a graph showing an example of the correspondence between the degree of inclination in the yaw direction and the correction rate. For example, the correction rate for the right side of the face line illustrated in FIG. 12 is set to be lower as the degree of inclination in the yaw direction is lower (that is, as the user is facing the right side). On the other hand, the correction rate for the left face line shown in FIG. 12 is set to be lower as the degree of tilt in the yaw direction is higher (that is, as the user is facing the left side). In this way, after the correction rate is specified based on the set correspondence, the corrected alpha value can be calculated by using the following <yaw direction correction degree correction formula>.

<Yaw direction correction degree correction formula>
Alpha value after correction = α × correction rate / 100

  In this way, by correcting the distortion of the original image based on the corrected alpha value, it is possible to realize a correction that balances the left and right faces of the user as described above. For example, in the original image 41 of FIG. 12, the user is facing left. In this case, by the above-described correction of the alpha value, the correction is made such that the “face line right side (before correction) T2a” is corrected to a higher degree like “the face line right side (after correction) T2b”. In addition, by the above-described correction of the alpha value, the correction is performed such that “the left side of the face line (before correction) T3a” becomes a low correction level like “the left side of the face line (after correction) T3b”. As a result, the right face line of the user who is on the near side in the image is corrected for distortion more than the left face line, so that the right face line of the user becomes a finer face. Therefore, the balance between the left and right faces of the user is adjusted.

  An example in which the distortion value is corrected after the alpha value is corrected based on the yaw direction correction degree in this manner will be described with reference to FIG. FIG. 13 is a schematic diagram illustrating an example of an image subjected to distortion correction after correcting an alpha value in the fine face correction processing. As shown in FIG. 13B, when the distortion correction is performed without correcting the alpha value, the face is turned to the right, but the face is turned to the front in the same manner as when the face is turned to the front. The left and right face lines are evenly corrected. Therefore, an image in which the left face line is emphasized is obtained. On the other hand, as shown in FIG. 13A, when the distortion value is corrected after the alpha value is corrected based on the yaw direction correction degree as described above, the correction of the right face line of the face is performed. , And the face line on the left side of the face is further corrected, so that the face line on the right side of the user becomes a finer face. Therefore, the balance between the left and right faces of the user is adjusted.

  In this manner, in this modification, for example, without performing a complicated image analysis such as detecting the contour of the user's face line, the inclination is calculated from the depth information, and the template is simply corrected. Correction for adjusting the balance of the user's face can be performed. That is, also in this modified example, the correction processing can be realized by a simpler method than in the related art.

Next, a case where the face is rotated up and down around the left-right axis of the face is performed in the same manner as the correction of the alpha value when the face is rotated left and right around the vertical axis of the face (that is, when yawing). A description will be given by taking as an example the correction of the alpha value in the case of pitching.
In this case, depth information is obtained together with the detection of a face or the like by image analysis. Then, based on the depth information, the degree of inclination (hereinafter, referred to as “pitch direction inclination degree”) in the pitching 45 that rotates vertically about the left-right axis direction 44 shown in FIG. 12 is specified. The specification of the degree of inclination can be calculated based on the difference between the depth information in two predetermined areas of the face, similarly to the degree of inclination in the yaw direction described above. The predetermined area is not particularly limited as long as it is an area where the degree of tilt in the pitch direction can be calculated. In the following, as an example, the pitch direction inclination degree is calculated using the following <pitch direction inclination degree calculation formula> based on the eyebrow depth information (depth value) and the upper lip depth information (depth value). .

<Pitch direction inclination degree calculation formula>
pitch = u_c−d_c
However,
u_c: Depth value between eyebrows d_c: Depth value of upper lip pitch: Degree of inclination in the pitch direction.

  Next, based on the degree of inclination in the pitch direction, the alpha value of the template corresponding to each predetermined area of the face is corrected. For example, the alpha value of the template corresponding to the user's chin and the alpha value of the template corresponding to the user's forehead are corrected. This correction is performed for the purpose of adjusting the balance of the user's face when the distortion correction in the subsequent stage is performed, similarly to the above-described degree of tilt in the yaw direction.

  For example, when it is possible to specify that the user is facing upward based on the degree of inclination in the pitch direction, the degree of distortion correction (that is, the alpha value) of the template corresponding to the user's chin is increased. Thus, when the chin is forward, distortion correction is performed so as to make the chin thinner, so that the balance of the user's face is adjusted. On the other hand, if the user can be specified to face downward, the distortion correction degree of the template corresponding to the user's chin is reduced. Thereby, when the chin is being pulled, the distortion is corrected so that the chin remains, so that the balance of the user's face is adjusted in this case as well.

  Therefore, in the present modification, the alpha value is corrected so that the distortion correction is performed as described above. For this purpose, in the same manner as the above-described yaw-direction tilt degree, the correspondence between the pitch-direction tilt degree and the correction rate is set in advance for each template. For example, the correction rate of the chin template is set to be lower as the degree of tilt in the pitch direction is lower (that is, as the user is facing downward). On the other hand, the correction rate of the chin template is set to be higher as the degree of tilt in the pitch direction is higher (that is, as the user faces upward). After the correction rate is specified based on the set correspondence in this way, the corrected alpha value can be calculated by using the following <pitch direction correction degree correction formula>.

<Pitch direction correction degree correction formula>
Alpha value after correction = α × correction rate / 100

  In this way, even when the alpha value is corrected based on the pitch direction correction degree, similarly to the case where the alpha value is corrected based on the yaw direction inclination degree, correction that balances the face of the user is performed. Can be realized.

  An example in which distortion correction is performed after correcting the alpha value based on the pitch direction correction degree in this manner will be described with reference to FIGS. 14 and 15 are schematic diagrams illustrating another example of an image in which distortion has been corrected after the alpha value has been corrected in the fine face correction process.

  As shown in FIG. 14B, when the distortion correction is performed without correcting the alpha value, the face is directed downward and the area of the chin is relatively small in the entire face. The chin is corrected in the same manner as when the face is facing the front. Therefore, the image has no jaw. On the other hand, as shown in FIG. 14A, when the distortion correction is performed after the alpha value is corrected based on the pitch direction correction degree as described above, the correction of the jaw is suppressed. The balance between the entire face and the chin is in place.

  In addition, as shown in FIG. 15B, when the distortion correction is performed without correcting the alpha value, the face is facing upward and the area of the chin is relatively large in the entire face. Regardless, the chin is corrected as in the case where the face is facing front. Therefore, the image has a prominent jaw. On the other hand, as shown in FIG. 15A, when the distortion correction is performed after the alpha value is corrected based on the pitch direction correction degree as described above, the jaw is further corrected. The balance between the entire face and the chin is in place.

  The modification of the alpha value in the present modification has been described above. In this modification, either one of the correction of the alpha value based on the pitch direction correction degree and the correction of the alpha value based on the yaw direction inclination degree may be performed on one template. May go. Further, the alpha value may be corrected based on the correction degree in a different direction for each template.

<Operation of Modification 4>
FIG. 16 is a flowchart illustrating a flow of the fine face correction process in the present modification, which is executed by the imaging device 1 of FIG. 1 having the functional configuration of FIG. Note that the functional blocks in this modification are basically the same as the functional blocks described above with reference to FIG. However, a function for performing the following operation is further provided.

  In step S <b> 11, the image acquiring unit 51 acquires data of a captured image in which the image captured by the imaging unit 16 has been subjected to development processing, or an image to be processed from the image storage unit 61. At this time, in the present modification, the image acquisition unit 51 acquires the depth information corresponding to the original image together with the original image.

In step S12, the face detection unit 52 performs face detection on the image to be processed, and determines whether a face has been detected.
If no face is detected, NO is determined in step S12, and the fine face correction process ends.
If a face is detected, YES is determined in step S12, and the process proceeds to step S13.

  In step S13, the face detection unit 52 detects a predetermined organ and a predetermined part of the face corresponding to the template. At this time, in the present modification, the face detection unit 52 detects the depth information of the detected predetermined organ or predetermined site (for example, the above-mentioned left cheek, right cheek, eyebrows, upper lip). Is also detected.

  In step S14, the matching position specifying unit 53 specifies coordinates in the original image that match the template.

  In step S <b> 21, the matching position specifying unit 53 specifies the degree of inclination based on the depth information of the predetermined organ or the predetermined part detected by the face detection unit 52, as in the <correction of the alpha value> described above. .

  In step S22, the matching position specifying unit 53 corrects the alpha value of each template based on the degree of inclination specified in step S21, as described above <correction of alpha value>.

  In step S15, the alpha map creation unit 54 creates an alpha map based on the alpha value corrected in step S22 and coordinates that match the template.

In step S16, the distortion correction processing unit 55 performs distortion correction using an alpha map.
Then, the fine face correction processing ends.

By the operation described above, an image in which the user's face is balanced as described above with reference to FIGS. 13, 14, and 15 is generated.
In this manner, in this modification, for example, without performing a complicated image analysis such as detecting the contour of the user's face line, the inclination is calculated from the depth information, and the template is simply corrected. Correction for adjusting the balance of the user's face can be performed. That is, also in this modified example, the correction processing can be realized by a simpler method than in the related art.

[Other Modifications]
Further, in the above-described embodiment, the imaging device 1 to which the present invention is applied has been described by taking a digital camera as an example, but is not particularly limited to this.
For example, the present invention can be applied to general electronic devices having an image processing function. Specifically, for example, the present invention is applicable to notebook personal computers, printers, television receivers, video cameras, portable navigation devices, mobile phones, smartphones, portable game machines, and the like.

The above-described series of processing can be executed by hardware or can be executed by software.
In other words, the functional configuration of FIG. 3 is merely an example, and is not particularly limited. That is, it is sufficient that the imaging apparatus 1 has a function capable of executing the series of processes described above as a whole, and what kind of functional block is used to realize this function is not particularly limited to the example of FIG.
Further, one functional block may be constituted by hardware alone, may be constituted by software alone, or may be constituted by a combination thereof.
The functional configuration in the present embodiment is realized by a processor that executes arithmetic processing, and the processor that can be used in the present embodiment includes single processing units, multiprocessors, and various processing units such as multicore processors. In addition to the above, a combination of these various processing apparatuses and processing circuits such as an ASIC (Application Specific Integrated Circuit) and an FPGA (Field-Programmable Gate Array) is included.

When a series of processing is executed by software, a program constituting the software is installed in a computer or the like from a network or a recording medium.
The computer may be a computer embedded in dedicated hardware. Further, the computer may be a computer that can execute various functions by installing various programs, for example, a general-purpose personal computer.

  A recording medium including such a program is constituted not only by the removable medium 31 of FIG. 1 distributed separately from the apparatus main body to provide the program to the user, but also It is composed of a recording medium or the like provided to the user. The removable medium 31 is configured by, for example, a magnetic disk (including a floppy disk), an optical disk, a magneto-optical disk, or the like. The optical disk is composed of, for example, a CD-ROM (Compact Disk-Only Memory), a DVD (Digital Versatile Disk), a Blu-ray (registered trademark) Disc (Blu-ray Disc), and the like. The magneto-optical disk is composed of an MD (Mini-Disk) or the like. The recording medium provided to the user in a state in which the program is incorporated in the apparatus main body in advance includes, for example, the ROM 12 in FIG. 1 in which a program is recorded, the hard disk included in the storage unit 19 in FIG.

  In this specification, the steps of describing a program recorded on a recording medium include, in addition to the processing performed in chronological order according to the order, the processing is not necessarily performed in chronological order. This includes the processing to be executed.

  Although some embodiments of the present invention have been described above, these embodiments are merely examples and do not limit the technical scope of the present invention. The present invention can take various other embodiments, and various changes such as omissions and substitutions can be made without departing from the gist of the present invention. These embodiments and modifications thereof are included in the scope and gist of the invention described in this specification and the like, and are also included in the invention described in the claims and the equivalents thereof.

Hereinafter, the invention described in the claims at the time of filing the application of the present application is additionally described.
[Appendix 1]
Acquiring means for acquiring an image of a predetermined subject,
Storage means for storing a template including a correction degree of each position in the shape corresponding to the correction target region of the predetermined subject, the correction degree changing at each position;
Specifying means for specifying coordinates that match the template in the image,
Correction means for correcting the image based on the coordinates specified by the specifying means and the correction degree included in the template;
An image processing apparatus comprising:
[Appendix 2]
The specifying means includes:
By analyzing the image, coordinates corresponding to a first feature point of the predetermined subject and coordinates corresponding to a second feature point of the predetermined subject are detected,
Identifying coordinates that match the base point in the template based on the coordinates of the detected first feature point, and identifying coordinates that match the contour in the template based on the coordinates of the detected second feature point;
2. The image processing apparatus according to claim 1, wherein
[Appendix 3]
The specifying means includes:
Locating a base point of the template at the coordinates of the first feature point, and specifying coordinates that match the template by rotating the template with the base point of the placed template as a rotation center;
3. The image processing apparatus according to claim 2, wherein
[Appendix 4]
The specifying means includes:
Based on the coordinates of the detected second feature point and the contour in the template, perform enlargement or reduction while maintaining the contour shape of the template.
4. The image processing apparatus according to claim 2 or 3, wherein
[Appendix 5]
The specifying means includes:
The correction degree included in the template is not changed,
4. The image processing apparatus according to claim 4, wherein:
[Appendix 6]
The specifying means specifies the state of the subject in the image, based on the specified state of the subject, corrects the correction degree included in the template,
The correction unit corrects the image based on the correction degree corrected by the specifying unit and the coordinates specified by the specifying unit.
The image processing apparatus according to any one of supplementary notes 1 to 4, wherein:
[Appendix 7]
The specifying unit specifies a degree of inclination of the subject as the state of the subject based on a difference between a depth of the first region of the subject in the image and a depth of a second region of the subject in the image. Do
7. The image processing apparatus according to claim 6, wherein:
[Appendix 8]
The correction means,
As the correction for the image, at least, performing a distortion correction by moving the coordinates of the image at a correction degree corresponding to the correction degree included in the template,
8. The image processing apparatus according to any one of supplementary notes 1 to 7, wherein:
[Appendix 9]
The storage unit stores a plurality of templates corresponding to a plurality of different correction target areas,
The specifying means independently specifies coordinates suitable for the plurality of templates, for each of the plurality of templates,
The image processing apparatus according to any one of supplementary notes 1 to 8, wherein:
[Appendix 10]
The predetermined subject is a face,
The correction target area is a facial organ or a facial part,
The image processing device according to any one of supplementary notes 1 to 9, wherein:
[Supplementary Note 11]
The template according to claims 1 to 10, wherein the degree of correction is set such that a part around the contour within the outline of the template is less corrected than a center part of the template. An image processing device according to any one of the preceding claims.
[Supplementary Note 12]
An obtaining step of obtaining an image of a predetermined subject,
A storage step of storing a template that includes a correction degree of each position in the shape corresponding to the correction target area of the predetermined subject, the correction degree changing at each position;
An identifying step of identifying coordinates in the image that match the template;
A correction step of correcting the image based on the coordinates specified in the specifying step and a correction degree included in the template;
An image processing method comprising:
[Appendix 13]
On the computer,
An acquisition function for acquiring an image of a predetermined subject,
A storage function of storing a template that includes a correction degree of each position in the shape corresponding to the correction target area of the predetermined subject and includes a correction degree that changes at each position;
A specifying function of specifying coordinates that match the template in the image,
A correction function for performing correction on the image based on the coordinates specified by the specific function and a correction degree included in the template;
An image processing program for realizing:

  DESCRIPTION OF SYMBOLS 1 ... Imaging device, 11 ... CPU, 12 ... ROM, 13 ... RAM, 14 ... Bus, 15 ... I / O interface, 16 ... Imaging part, 17 ... Input unit, 18 output unit, 19 storage unit, 20 communication unit, 21 drive, 22 illumination unit, 31 removable media, 51 image acquisition , 52... Face detector, 53... Conforming position specifying unit, 54... Alpha map generator, 55... Distortion correction processing unit, 61. Memory

Claims (13)

Acquiring means for acquiring an image of a predetermined subject,
Storage means for storing a template including a correction degree of each position in the shape corresponding to the correction target region of the predetermined subject, the correction degree changing at each position;
Specifying means for specifying coordinates that match the template in the image,
Correction means for correcting the image based on the coordinates specified by the specifying means and the correction degree included in the template;
An image processing apparatus comprising: The specifying means includes:
By analyzing the image, coordinates corresponding to a first feature point of the predetermined subject and coordinates corresponding to a second feature point of the predetermined subject are detected,
Identifying coordinates that match the base point in the template based on the coordinates of the detected first feature point, and identifying coordinates that match the contour in the template based on the coordinates of the detected second feature point;
The image processing apparatus according to claim 1, wherein: The specifying means includes:
Locating a base point of the template at the coordinates of the first feature point, and specifying coordinates that match the template by rotating the template with the base point of the placed template as a rotation center;
The image processing apparatus according to claim 2, wherein: The specifying means includes:
Based on the coordinates of the detected second feature point and the contour in the template, perform enlargement or reduction while maintaining the contour shape of the template.
The image processing apparatus according to claim 2, wherein: The specifying means includes:
The correction degree included in the template is not changed,
The image processing apparatus according to claim 4, wherein: The specifying means specifies the state of the subject in the image, based on the specified state of the subject, corrects the correction degree included in the template,
The correction unit corrects the image based on the correction degree corrected by the specifying unit and the coordinates specified by the specifying unit.
The image processing apparatus according to claim 1, wherein: The specifying unit specifies a degree of inclination of the subject as the state of the subject based on a difference between a depth of the first region of the subject in the image and a depth of a second region of the subject in the image. Do
The image processing apparatus according to claim 6, wherein: The correction means,
As the correction for the image, at least, performing a distortion correction by moving the coordinates of the image at a correction degree corresponding to the correction degree included in the template,
The image processing apparatus according to claim 1, wherein: The storage unit stores a plurality of templates corresponding to a plurality of different correction target areas,
The specifying means independently specifies coordinates suitable for the plurality of templates, for each of the plurality of templates,
The image processing apparatus according to claim 1, wherein: The predetermined subject is a face,
The correction target area is a facial organ or a facial part,
The image processing apparatus according to claim 1, wherein:   11. The degree of correction of the template is set such that a part around the contour within the outline of the template has a lower degree of correction than a central part of the template. The image processing device according to any one of claims 1 to 4. An obtaining step of obtaining an image of a predetermined subject,
A storage step of storing a template that includes a correction degree of each position in the shape corresponding to the correction target area of the predetermined subject, the correction degree changing at each position;
An identifying step of identifying coordinates in the image that match the template;
A correction step of correcting the image based on the coordinates specified in the specifying step and a correction degree included in the template;
An image processing method comprising: On the computer,
An acquisition function for acquiring an image of a predetermined subject,
A storage function of storing a template that includes a correction degree of each position in the shape corresponding to the correction target area of the predetermined subject and includes a correction degree that changes at each position;
A specifying function of specifying coordinates that match the template in the image,
A correction function for performing correction on the image based on the coordinates specified by the specific function and a correction degree included in the template;
An image processing program for realizing:

Images