JP3773657B2 - Eyebrow deformation system - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to an eyebrow deformation system, and more particularly to an eyebrow deformation system that deforms and displays an eyebrow in a human face image.
[0002]
[Prior art]
The image production method by make-up applies modeling theory, and grasps the characteristics of the shape and how it feels when it is seen. For example, a straight line produces a sharp or static image, and a curve produces a soft or dynamic image. Therefore, by making up the eyebrows linearly or curvilinearly, it is possible to produce a sharp image or a soft image, respectively. Similarly, it is possible to produce a calm image using a descending line, or to produce a robust image using a thick line.
[0003]
It is considered to realize such image production by makeup using image processing technology of a computer system. In the conventional image processing technique of a computer system, the shape is simply deformed linearly or curvedly.
[0004]
[Problems to be solved by the invention]
However, in the conventional image processing technology of a computer system, an operator has to input the details of the eyebrow portion in the face image, the operation of the deformation, the color correction of the deformation portion, etc. There is a problem that it requires specialized knowledge of image processing, and it takes time and effort.
[0005]
The present invention has been made in view of the above points, and provides an eyebrow deformation system that is easy to operate and takes a short time to display a natural eyebrow by deforming and displaying eyebrows in a face image. The purpose is to do.
[0006]
[Means for Solving the Problems]
The invention according to claim 1 is an eyebrow deformation system for deforming and displaying eyebrows in a face image.
Eyebrow template storage means in which a plurality of eyebrow templates representing the shape of the eyebrow are stored in advance;
The eyebrow template selected from the plurality of eyebrow templates is rotated so as to have an eyebrow inclination calculated from the eyes and the corners of the face image, and a line passing through the nose and eyes of the face image and a side passing through the eyebrow head of the face image. The intersection with the axis is the position of the eyebrow head of the selected eyebrow template, and the intersection of the straight line passing through the nose and the corner of the eye and the eyebrow head of the rotated eyebrow template and the line passing through the eyebrow is the eyebrow corner of the selected eyebrow template. An eyebrow template correction means for correcting the rotated eyebrow template by enlarging or reducing the position of the eyebrow template,
Erasing means for erasing the eyebrows from the face image;
Synthesizing means for synthesizing the corrected eyebrow template with the face image from which the eyebrows have been erased.
[0007]
In this way, a plurality of eyebrow templates representing the shape of the eyebrow are prepared, and the eyebrow template selected from the eyebrow template is corrected according to the face image, and the eyebrow template is corrected by deleting the eyebrow from the face image. By synthesizing the eyebrows, it is possible to display natural eyebrows by deforming the eyebrows without spending time with a simple operation of simply selecting a desired eyebrow template.
[0009]
According to a second aspect of the invention, the eyebrows deformation system of claim 1, wherein said erasing means combines the pixels in the vicinity of the eyebrows of the face image portion to be erased eyebrows of the face image.
In this way, since the pixels near the eyebrow are combined with the portion where the eyebrows are erased, even if there is a difference in shape between the eyebrows to be erased and the eyebrow template to be synthesized, the portion where the eyebrows are erased becomes a skin color. It is possible to prevent the scratched face image from being damaged.
[0010]
The invention according to claim 3 is the eyebrow deformation system according to claim 2 ,
A first smoothing unit that performs smoothing on a boundary of a portion where the eyebrows are erased by the erasing unit;
In this way, since the boundary of the portion where the eyebrows are erased is performed, the boundary becomes inconspicuous, and the quality of the synthesized face image can be improved.
[0011]
According to a fourth aspect of the present invention, in the eyebrow deformation system according to any one of the first to third aspects, there is provided second smoothing means for performing smoothing on an eyebrow edge in a face image obtained by synthesizing an eyebrow template by the synthesizing means. .
In this way, since the eyebrow edges in the synthesized face image are smoothed, the boundary of the synthesized eyebrows becomes inconspicuous, and the quality of the synthesized face image can be improved.
[0012]
DETAILED DESCRIPTION OF THE INVENTION
FIG. 1 shows a block diagram of an embodiment of the eyebrow deformation system of the present invention. In the figure, an input device 20, a storage device 30, a display device 40, a printing device 50, and a communication device 60 are connected to a central processing unit (CPU) 10 via a bus 15. The input device 20 includes a keyboard 21, a mouse 22, a digital camera (or a CCD camera) 23, and the storage device 30 includes a RAM 31, a ROM 32, a hard disk device 33, a flexible disk device 34, and the like. The CPU 10 executes various processing programs stored in the storage device 30, stores the results in the storage device 30, displays them on the display device 40, prints them with the printing device 50, and outputs them. In addition to the various processing programs, the storage device 30 stores in advance a plurality of types of eyebrow templates representing the shape of the eyebrows.
[0013]
FIG. 2 shows a main flowchart of an embodiment of the eyebrow deformation system of the present invention. In the figure, in the feature point extraction process in step S10, the face of the subject is imaged and the feature points of the obtained face image are extracted. Next, a desired eyebrow shape to be deformed in step S <b> 12 is selected using a mouse 22 from among a plurality of eyebrow templates displayed on the display device 40.
[0014]
Thereafter, the original left eyebrow of the face image is deleted in step S14, and the left eyebrow of the eyebrow template selected in step S16 is combined with the face image. In step S18, the original right eyebrow of the face image is deleted, and the right eyebrow of the eyebrow template selected in step S20 is combined with the face image. In step S <b> 22, the face image obtained by combining the eyebrows is displayed on the display device 40 and printed out by the printing device 50.
[0015]
FIG. 3 shows a flowchart of an embodiment of the feature point extraction process executed in step S10. In the figure, the face of the subject is imaged by the digital camera 23 of the input device 20 in step S30, and the obtained face image is stored in the hard disk device 33 of the storage device 30. Next, face position recognition processing is performed on the face image in step S32. The face area is labeled by this face position recognition process, and feature points of the face outline are extracted. The face image represents a position in XY coordinates with the upper left corner as the origin, the horizontal direction as the X axis, and the vertical direction as the Y axis.
[0016]
Next, in step S34, face region edge image creation processing is performed to create a face edge image (binary image). In step S36, the edge image is projected onto the Y axis to create an edge histogram. Projection takes place. Thereafter, in step S38, the Y coordinates of each part of the face, that is, the eyebrows, eyes, nose, and mouth are searched from the histogram by Y projection.
[0017]
In step S40, the eye and nose feature points and the feature points associated therewith are extracted at the Y coordinate position including the eyes and nose in the face edge image. In step S42, the mouth feature point and the feature point associated therewith are extracted at the Y coordinate position including the mouth in the face edge image. In step S44, the eyebrow feature point and the feature point associated therewith are extracted at the Y coordinate position including the eyebrow in the face edge image, and the process ends.
[0018]
Here, extraction of eye and nose and eyebrow feature points used for eyebrow deformation will be described in detail. FIG. 4 shows a flowchart of an embodiment of the eye and nose feature point extraction process executed in step S40. In the figure, in step S46, a face image of the Y coordinate area including the eyes and nose is acquired. In the next loop processing in step S47, the threshold value TH is sequentially increased from the lowest value, and in step S48 in the loop, the face image is binarized using the threshold value TH. The reason why binarization is performed by changing the threshold value TH in this way is to detect the pupil accurately.
[0019]
In the next step S49, in the edge image obtained by binarization, a region surrounded by continuous edges is labeled. And it is discriminate | determined whether it is a pupil about each area | region labeled by step S51 in the loop process of step S50. Here, whether or not the region is a pupil is determined based on conditions such as whether the region is substantially circular and has an area within a predetermined range, and whether there is a similar region paired with the region.
[0020]
Next, it is determined whether or not two pupils can be detected in step S52. If a pupil can be detected, the face image is binarized (edge image) using a fixed threshold value in step S54. Thereafter, in step S56, the left eye feature point is extracted from the left eye region centered on the left pupil in the edge image, and in step S58, the right eye feature is extracted in the right eye region centered on the right pupil. Extract points. Note that the left and right eye feature points are the pupils PL and PR, the eye corners EL1 and ER1, the eyes EL4 and ER4, and the upper and lower eye contours EL5, ER5, EL3, and ER3 shown in FIG. Feature points are stored.
[0021]
Thereafter, the process proceeds to step S60. Here, an equilateral triangle having two pupils as two points and one point in the mouth direction is set. In step S62, the point having the largest Y coordinate in the regular triangle in the edge image is extracted and stored as the feature point N2. Next, in step S64, the point with the smallest X coordinate between the feature points EL3, ER3 and the feature point N2 with the Y coordinate is extracted and stored as the feature point N1, and the point with the largest X coordinate is the feature point N3. Extract and store as. All the eye and nose feature points are extracted, and the process is terminated.
[0022]
6 and 7 show a flowchart of an embodiment of the eyebrow feature point extraction process executed in step S44. In FIG. 6, in step S100, it is determined whether or not the Y coordinate of the eyebrows is appropriate. Here, for example, when the Y coordinate of the eyebrow is smaller than the average value of the Y coordinates of the facial feature points F1 and C2, it is determined to be appropriate. If the Y coordinate of the eyebrow is not appropriate, it is set to an appropriate value in step S102.
[0023]
In the next step S102, the X coordinate is displaced from the inner side to the outer side of the left eyebrow in the face area, and loop processing is performed. In step S104 in this loop, the Y coordinate is displaced from the upper end to the lower end direction of the eyebrow area. When a pixel which is an edge from the face edge image and is not a skin color is first detected from the face image in step S106 in the loop, the process proceeds to step S108. In step S108, the detected Y coordinate of the pixel is stored as the upper end of the left eyebrow in the X coordinate at this time. Further, in step S110 in the loop of step S102, the Y coordinate is displaced from the lower end to the upper end direction of the eyebrow region, and loop processing is performed. In step S112 in this loop, the face edge image is an edge, and the face When a non-skin color pixel is first detected from the image, the Y coordinate of the pixel detected in step S114 is stored as the lower end of the left eyebrow in the X coordinate at this time.
[0024]
In the next step S122 in FIG. 7, the X coordinate is displaced from the inner side to the outer side of the right eyebrow in the face area, and a loop process is performed. In step S124 in this loop, the Y coordinate extends from the upper end to the lower end of the eyebrow area. When a pixel that is an edge from the face edge image and is not a skin color is first detected from the face image in step S126 in the loop, the process proceeds to step S128. In step S128, the detected Y coordinate of the pixel is stored as the upper end of the right eyebrow in the X coordinate at this time. Further, in step S122 in the loop of step S122, the Y coordinate is displaced from the lower end to the upper end direction of the eyebrow region to perform loop processing, and in step S132 in this loop, an edge is detected from the face edge image, and the face When a non-skin color pixel is first detected from the image, the Y coordinate of the pixel detected in step S134 is stored as the lower end of the right eyebrow in the X coordinate at this time.
[0025]
After this, in step S144, the highest position (minimum Y coordinate) at the upper ends of the left and right eyebrows is extracted as feature points (points BL2 and BR2 shown in FIG. 5) of the eyebrows. In step S146, the position on the innermost side (center side) of the face at the upper and lower ends of the left and right eyebrows is extracted as the eyebrows head feature points (points BL3 and BR3 shown in FIG. 5). In step S148, the position of the outermost face at the upper and lower ends of the left and right eyebrows is extracted as the feature points of the eyebrows (points BL1 and BR1 shown in FIG. 5). In step S 149, the lowest position (maximum Y coordinate) at the lower ends of the left and right eyebrows is extracted as a feature point of the eyebrow bottom (points BL 4 and BR 4 shown in FIG. 5). Then, this process ends.
[0026]
FIG. 8 shows a flowchart of an embodiment of the eyebrow erasing process executed in steps S14 and S18. In the figure, in step S160, the eyebrow portion (the portion surrounded by the edge image of the eyebrows) of the face image including the eyebrows is masked, and at this time, the color information of the eyebrows is stored. Next, in step S162, the Y coordinate is displaced from the upper end to the lower end of the eyebrow of the face image to perform loop processing, and in step S164 in this loop, the X coordinate is displaced from the inner side to the outer side of the eyebrow to perform loop processing. . In step S166 in this loop, it is determined whether or not the XY coordinates are a mask portion. If the XY coordinates correspond to the mask portion, the process proceeds to step S168.
[0027]
In step S168, skin color pixels for copying are randomly selected from the outer peripheral position of the mask portion in the eyebrow region, and the pixels selected in step S170 are copied to the XY coordinate pixels of the mask portion. As a result, the skin color is combined with the eyebrows, and the eyebrows are erased.
Thereafter, in step S172, the median filter process is performed on the erased portion of the eyebrow to remove noise, and in step S174, the vicinity of the eyebrow edge (by the edge image) of the face image is smoothed. Further, smoothing near the eyebrow head erased in step S176 is performed, and the process ends. In this way, since the boundary of the portion where the eyebrows are erased is performed, the boundary becomes inconspicuous, and the quality of the synthesized face image can be improved.
[0028]
FIG. 9 shows a flowchart of an embodiment of eyebrow compositing processing executed in steps S16 and S20. In the figure, the eyebrow inclination is calculated from the eyebrow head and eyebrow feature points (BR3 and BR1 in the right eyebrow) extracted in step S180, and the eyebrow inclination is set so that the inclination of the selected eyebrow template becomes the calculated eyebrow inclination. Rotate the template. In the next step S182, the color of the eyebrow template is changed to the original color of the eyebrow stored when erasing the eyebrows.
[0029]
In step S184, as shown in FIG. 10A, the intersection of the straight line L1 passing through the nose feature point N3 and the eye feature point ER4 and the Y coordinate (straight line L2) of the original eyebrow feature point BR3 is determined. Set as the eyebrow head TBR3 of the eyebrow template. Further, as shown in FIG. 10B, the intersection of the straight line L3 passing through the feature point N3 of the nose and the feature point ER1 of the eye corner and the straight line L4 passing through the eyebrow head and the eyebrow corner of the rotated eyebrow template is defined as the eyebrow of the template. Set as bottom TBR1. Thereafter, the eyebrow head and the eyebrow bottom of the eyebrow template rotated in step S186 are enlarged or reduced so as to match the size of the eyebrow head TBR3 and the eyebrow tip TBR1. In this way, since the arrangement positions of the eyebrow head and the eyebrow bottom of the eyebrow template selected according to the shape of the nose and eyes of the face image are determined, the eyebrow template can be synthesized in a natural form.
[0030]
Next, in step S188, the Y coordinate is displaced from the upper end to the lower end of the eyebrow of the face image to perform loop processing, and in step S190 in this loop, the X coordinate is displaced from the inner side to the outer side of the eyebrow to perform loop processing. . In step S192 in this loop, the eyebrow template is copied to the face image while performing linear interpolation, and the eyebrow is synthesized.
Thereafter, in step S194, the Y coordinate is displaced from the upper end to the lower end of the eyebrow of the face image to perform loop processing, and in step S196 in this loop, the X coordinate is displaced from the inner side to the outer side of the eyebrow to perform loop processing. . In step S198 in the loop, edge detection is performed, and in step S200, it is determined whether or not the edge is detected. If it is an edge, the process proceeds to step S202, where smoothing is performed using 9 pixels adjacent to the XY coordinates, and the synthesized eyebrows are blended with the face image so that a natural expression can be obtained.
[0031]
In this way, a plurality of eyebrow templates representing the shape of the eyebrow are prepared, and the eyebrow template selected from the eyebrow template is corrected according to the face image, and the eyebrow template is corrected by deleting the eyebrow from the face image. By synthesizing the eyebrows, it is possible to display natural eyebrows by deforming the eyebrows without spending time with a simple operation of simply selecting a desired eyebrow template. In addition, since the eyebrow edges in the synthesized face image are smoothed, the boundary of the synthesized eyebrows can be prevented from standing out unnaturally, and the quality of the synthesized face image can be improved.
[0032]
The storage device 30 corresponds to the eyebrow template storage unit, steps S180 to 186 correspond to the eyebrow template correction unit, steps S160 to 170 correspond to the erasing unit, step S153 corresponds to the combining unit, and steps S172 to S172. 176 corresponds to the first smoothing means, and step S202 corresponds to the second smoothing means.
[0033]
【The invention's effect】
As described above, the invention according to claim 1 includes an eyebrow template storage unit in which a plurality of eyebrow templates representing the shape of the eyebrow are stored in advance,
The eyebrow template selected from the plurality of eyebrow templates is rotated so as to have an eyebrow inclination calculated from the eyes and the corners of the face image, and a line passing through the nose and eyes of the face image and a side passing through the eyebrow head of the face image. The intersection of the axis is the position of the eyebrow head of the selected eyebrow template, and the intersection of the straight line passing through the nose and the corner of the eye and the eyebrow head of the rotated eyebrow template and the line passing through the eyebrow is the eyebrow corner of the selected eyebrow template. An eyebrow template correction means for correcting the rotated eyebrow template by enlarging or reducing the position of the eyebrow template,
Erasing means for erasing the eyebrows from the face image;
Synthesizing means for synthesizing the corrected eyebrow template with the face image from which the eyebrows have been erased.
[0034]
In this way, a plurality of eyebrow templates representing the shape of the eyebrow are prepared, and the eyebrow template selected from the eyebrow template is corrected according to the face image, and the eyebrow template is corrected by deleting the eyebrow from the face image. By synthesizing the eyebrows, it is possible to display natural eyebrows by deforming the eyebrows without spending time with a simple operation of simply selecting a desired eyebrow template.
[0036]
According to a second aspect of the present invention, the erasing unit synthesizes a pixel in the vicinity of the eyebrow of the face image with a portion where the eyebrow of the face image is erased.
In this way, since the pixels near the eyebrow are combined with the portion where the eyebrows are erased, even if there is a difference in shape between the eyebrows to be erased and the eyebrow template to be synthesized, the portion where the eyebrows are erased becomes a skin color. It is possible to prevent the scratched face image from being damaged.
[0037]
Further, the invention described in claim 3 has first smoothing means for performing smoothing on the boundary of the portion where the eyebrows are erased by the erasing means.
In this way, since the boundary of the portion where the eyebrows are erased is performed, the boundary becomes inconspicuous, and the quality of the synthesized face image can be improved.
According to a fourth aspect of the present invention, there is provided second smoothing means for performing smoothing on an eyebrow edge in a face image obtained by synthesizing the eyebrow template by the synthesizing means.
[0038]
In this way, since the eyebrow edges in the synthesized face image are smoothed, the boundary of the synthesized eyebrows becomes inconspicuous, and the quality of the synthesized face image can be improved.
[Brief description of the drawings]
FIG. 1 is a block diagram of an embodiment of an eyebrow deformation system of the present invention.
FIG. 2 is a main flowchart of an embodiment of an eyebrow deformation system of the present invention.
FIG. 3 is a flowchart of an example of feature point extraction processing executed in step S10.
FIG. 4 is a flowchart of an example of eye and nose feature point extraction processing executed in step S40.
FIG. 5 is a diagram showing feature points of a face image.
FIG. 6 is a flowchart of an example of eyebrow feature point extraction processing executed in step S44.
FIG. 7 is a flowchart of an example of eyebrow feature point extraction processing executed in step S44;
FIG. 8 is a flowchart of an embodiment of an eyebrow erasing process executed in steps S14 and S18.
FIG. 9 is a flowchart of an example of eyebrow compositing processing executed in steps S16 and S20.
FIG. 10 is a diagram for explaining how to determine one of an eyebrow head and an eyebrow butt of an eyebrow template.
[Explanation of symbols]
10 Central processing unit (CPU)
20 Input device 21 Keyboard 22 Mouse 23 Digital camera 30 Storage device 31 RAM
32 ROM
33 Hard Disk Device 34 Flexible Disk Device 40 Display Device 50 Printing Device

Claims (4)

In the eyebrow deformation system that deforms and displays the eyebrows in the face image,
Eyebrow template storage means in which a plurality of eyebrow templates representing the shape of the eyebrow are stored in advance;
The eyebrow template selected from the plurality of eyebrow templates is rotated so as to have an eyebrow inclination calculated from the eyes and the corners of the face image, and a line passing through the nose and eyes of the face image and a side passing through the eyebrow head of the face image. The intersection of the axis is the position of the eyebrow head of the selected eyebrow template, and the intersection of the straight line passing through the nose and the corner of the eye and the eyebrow head of the rotated eyebrow template and the line passing through the eyebrow is the eyebrow corner of the selected eyebrow template. An eyebrow template correction means for correcting the rotated eyebrow template by enlarging or reducing the position of the eyebrow template,
Erasing means for erasing the eyebrows from the face image;
An eyebrow deformation system comprising: a combining unit that combines the corrected eyebrow template with the face image from which the eyebrow has been deleted. The eyebrow deformation system according to claim 1 .
The eyebrow deformation system characterized in that the erasing unit synthesizes a pixel in the vicinity of the eyebrow of the face image with a portion where the eyebrow of the face image is erased. The eyebrow deformation system according to claim 2 ,
An eyebrow deformation system comprising: first smoothing means for performing smoothing on a boundary of a portion where the eyebrows are erased by the erasing means. In the eyebrow deformation system according to any one of claims 1 to 3 ,
An eyebrow deformation system comprising: a second smoothing unit that performs smoothing on an eyebrow edge in a face image obtained by combining an eyebrow template with the combining unit.

Images