JP2011008643A - Image creation device, print sticker manufacturing method, program and storage medium - Google Patents

Abstract

PROBLEM TO BE SOLVED: To perform effective image processing on eyes of a subject by a method different from correction of skin color.SOLUTION: In the sticker printer, a face image of a user is captured by use of a camera. The eyes of the user is detected from the captured face image. Fitting processing is performed on the detected eyes to fit an eyelash texture to the detected eyes, to identify an eyelash drawing position of the user in the captured image. A predetermined eyelash image color is added to the identified eyelash drawing position, to generate a display image with an eyelash image, to be displayed on a display 1002.

Description

  The present invention relates to an image generation device including a camera and a display device.

  2. Description of the Related Art Image generation apparatuses that control a camera to photograph a user and correct a captured image are known. Of these, devices such as sticker printing machines and photo sticker creation devices that print images on sticker paper while the user takes a picture with a camera and displays the shot image are popular because they can easily obtain photo stickers. There is. In particular, an apparatus having functions such as performing lightness correction to show the skin color beautifully, putting wigs on the hair, and changing the size of the eyes is popular with female students. For example, Patent Document 1 describes an apparatus that recognizes a peripheral portion of an eye of a photographed subject and sharpens it by performing brightness correction.

JP 2006-228036 A

The first form for solving the above problems is:
An image generation apparatus (for example, a sticker printing machine 1000 in FIG. 1) including a camera (for example, the camera 1001 in FIG. 1) and a display device (for example, the display 1002 in FIG. 1),
Shooting control means (for example, shooting control unit 1211 in FIG. 13; step B1 in FIG. 15) for shooting the user by controlling the camera;
Eyelash drawing position specifying means (for example, eyelash drawing processing unit 1213 in FIG. 13; step B7 in FIG. 15, step C1 in FIG. 16) for specifying the eyelash drawing position in the photographed image taken by the camera;
Eyelash drawing means for drawing a given eyelash image at the eyelash drawing position specified by the eyelash drawing position specifying means in the photographed image (for example, the eyelash drawing processing unit 1213 in FIG. 13; steps B7 and B9 in FIG. 15, FIG. 16 steps C5 to C21),
Display control means (for example, the imaging control unit 1211 in FIG. 13; step B13 in FIG. 15) for causing the display device to display a captured image in which the eyelash image is drawn by the eyelash drawing means;
Is an image generating apparatus.

As another form,
A shooting control step (for example, step B1 in FIG. 15) for controlling the camera and shooting the user;
An eyelash drawing position specifying step (for example, step B7 in FIG. 15 and step C1 in FIG. 16) for specifying an eyelash drawing position in a photographed image taken by the camera;
An eyelash drawing step (for example, steps B7 and B9 in FIG. 15 and steps C5 to C21 in FIG. 16) for drawing a given eyelash image at the eyelash drawing position specified in the eyelash drawing position specifying step in the captured image;
A display control step (for example, step B13 in FIG. 15) for causing the display device to display a captured image in which the eyelash image is drawn in the eyelash drawing step;
A printing control step of printing the photographed image on which the eyelash image is drawn by the printing apparatus (for example, step A7 in FIG. 14);
You may comprise the print sticker manufacturing method containing these.

As another form,
A computer built in a device equipped with a camera and a display device,
Photographing control means for controlling the camera and photographing the user;
Eyelash drawing position specifying means for specifying the eyelash drawing position in the photographed image taken by the camera;
Eyelash drawing means for drawing a given eyelash image at the eyelash drawing position specified by the eyelash drawing position specifying means in the captured image;
Display control means for displaying on the display device a photographed image in which the eyelash image is drawn by the eyelash drawing means;
A program (for example, a seal creation program 1701 in FIG. 13) may be configured. Furthermore, it is also possible to configure a computer-readable storage medium (for example, the storage unit 1700 in FIG. 13) that stores the program.

  According to the first embodiment and the like, the eyelash drawing position is specified in the photographed image taken by the camera, and the eyelash image is drawn at the eyelash drawing position. By drawing an eyelash image, an effect like a mascara or an effect like a false eyelash is exhibited. Thereby, an effect different from the correction of the skin color can be applied to the eye part.

As a second form, the image generation apparatus of the first form is
The eyelash drawing means
Drawing target pixel determining means for determining a drawing target pixel of the eyelash image in the photographed image based on the eyelash drawing position specified by the eyelash drawing position specifying means (for example, the eyelash drawing processing unit 1213 in FIG. 13; FIG. 16). Step C11),
When the color of the drawing target pixel satisfies a predetermined color condition as a condition for allowing coloring of the color of the eyelash image, a coloring unit (for example, FIG. 13) colors the color of the eyelash image on the drawing target pixel. Eyelash drawing processing unit 1213; Steps C15 and C17 in FIG. 16; Yes → Step C19),
You may comprise so that it may have.

  According to this second aspect, when the pixel to be drawn of the eyelash image is determined based on the specified eyelash drawing position and the color of the pixel satisfies a color condition that allows coloring of the color of the eyelash image, The color of the eyelash image is colored. As a result, the image of eyelashes growing from the white of the eye, or the image of eyelashes drawn on the front (front) of the hair taken with the eyes and the front-rear positional relationship between the hair and the eyelashes contradicts each other It is possible to prevent the image from becoming uncomfortable. As the color condition, for example, as the third embodiment, it is preferable that at least a color condition for determining the skin color is set. The color condition for discriminating the skin color may be, for example, setting a color range (RGB value range) that can be taken as the skin color in advance, or from the color around the user's eyes in the captured image. A predetermined color range centered on the sampled / averaged color may be set.

Further, as a fourth form, in the image generation apparatus of the first form,
The eyelash drawing means
Drawing target pixel determining means (for example, eyelash drawing processing unit 1213 in FIG. 13) for determining a drawing target pixel of the eyelash image in the photographed image based on the eyelash drawing position specified by the eyelash drawing position specifying means;
When the color of the drawing target pixel does not satisfy a predetermined color condition as a condition that does not allow coloring of the color of the eyelash image, a coloring unit (for example, FIG. 13) colors the color of the eyelash image on the drawing target pixel. Eyelash drawing processing unit 1213),
You may comprise so that it may have.

  In the case of the fourth embodiment, the same effect as that of the second embodiment is exhibited. As the color condition of the fourth mode, for example, as the fifth mode, it is preferable that at least a color condition for discriminating a white-eye portion is set. The color condition for discriminating the white-eye portion may be set, for example, in advance as a range (RGB value range) that can be taken as the white-eye color, or sampled / averaged from the white-eye portion of the user in the captured image A predetermined color range centered on the color may be set.

Further, as a sixth mode, in the image generation device according to any one of the first to fifth modes,
A printing device (eg, the printing device 1008 of FIG. 1);
Print control means (for example, the print control unit 1217 in FIG. 13) that causes the printing apparatus to print the captured image on which the eyelash image is drawn;
Further comprising
The display screen of the display device is a screen larger than the size of the printing paper printed by the printing device,
The eyelash image includes a printing eyelash image (for example, the printing eyelash texture of FIG. 12), and each cilia is thinner than the printing eyelash image, and / or has a cilia more than the printing eyelash image. The density is low, and there is a drawn display eyelash image (for example, the display eyelash texture of FIG. 12),
The eyelash drawing means
Display eyelash drawing means for drawing the display eyelash image on the captured image (for example, eyelash drawing processing unit 1213 in FIG. 13; step B7 in FIG. 15);
Printing eyelash drawing means for drawing the printing eyelash image on the photographed image (for example, the eyelash drawing processing unit 1213 in FIG. 13; step E3 in FIG. 18);
Have
The display control means causes the display device to display a captured image in which the display eyelash image is drawn by the display eyelash drawing means;
The printing control means causes the printing apparatus to print a captured image in which the printing eyelash image is drawn by the printing eyelash drawing means;
An image generation apparatus may be configured.

  According to the sixth embodiment, the display screen of the display device is a screen larger than the size of the printing paper printed by the printing device. Therefore, the size of the photographed user's face image is larger on the display device than on the printing paper. On the other hand, the eyelash image includes a printing eyelash image, and the thickness of each cilia is smaller than that of the printing eyelash image, and / or the density of the cilia is lower than that of the printing eyelash image, and the display eyelash drawn. There is an image, and what is displayed is an image in which a display eyelash image is drawn, and what is printed is an image in which a print eyelash image is drawn.

  If the thickness and density of each cilia in the print eyelash image are the same to match the display eyelash image, the thickness / density of the eyelashes appears to be appropriate in the displayed face image. The face image that appears is thin / low density. Conversely, if the thickness and density of each cilia in the display eyelash image are the same as the print eyelash image, the thickness / density of the eyelashes appears appropriate in the printed face image. The face image is thicker / higher in density. This is because the display screen of the display device is larger than the size of the printing paper, so that the size of the displayed face image is different from the size of the printed face image. According to this embodiment, such a sense of incongruity does not occur.

Further, as a seventh form, in the image generation apparatus of the sixth form,
Display eyelashes that partially erase the display eyelash image from the captured image controlled to be displayed on the display device in accordance with a user's input operation of an erase locus (for example, the erase operation locus in FIG. 12) on the display screen of the display device. Partial erasing means (for example, the image editing unit 1215 in FIG. 13; step D11 in FIG. 17);
In accordance with the erasing locus, a printing eyelash part erasing unit (for example, the eyelash drawing processing unit 1213 in FIG. 13; step F17 in FIG. 19) for partially erasing the printing eyelash image from the photographed image on which the printing eyelash image is drawn. )When,
Further comprising
The printing control means causes the printing apparatus to print a captured image in which the printing eyelash image is partially erased by the printing eyelash part erasing means;
An image generation apparatus may be configured.

  According to the seventh aspect, the user can partially erase the display eyelash image drawn on the photographed image by an input operation of an erasing locus on the display screen of the display device. On the other hand, the printed eyelash image to be printed is also partially erased in accordance with the erase trajectory input by the user to the display eyelash image. Therefore, the erasing operation for the display eyelash image performed by the user can be reflected on the print eyelash image.

Further, as an eighth aspect, in the image generation apparatus according to any one of the first to seventh aspects,
The eyelash drawing position specifying means specifies the user's eyeline part (for example, eyeline EL in FIG. 24) in the captured image as the eyelash drawing position,
It further comprises eyelash image generation means (for example, eyelash drawing processing unit 1213 in FIG. 13) for generating the eyelash image by deforming a predetermined eyelash pattern image so as to match the shape of the eyeline part.
An image generation apparatus may be configured.

  According to the eighth embodiment, the eyelash pattern image is determined in advance, and the eyelash pattern image is generated by deforming the eyelash pattern image so as to match the shape of the user's eyeline portion in the captured image. Thereby, it becomes possible to classify the eyelash pattern image. For example, eyelash pattern images with different cilia drawing positions (for example, an eyelash pattern image in which cilia are uniformly drawn from the top of the eye to the corner of the eye and an eyelash pattern image in which the cilia is drawn only near the corner of the eye), and eyelashes with different cilia length and density The eyelash pattern image is prepared for each type, such as a pattern image. If the user selects a desired type of eyelash pattern image, an appropriate eyelash image corresponding to the shape of the user's eyeline portion is generated.

  In this case, as a ninth mode, it is more preferable that the eyelash image generation unit generates the eyelash image by scaling the eyelash pattern image according to the length of the eyeline portion.

Further, as a tenth aspect, in the image generation apparatus according to any one of the first to ninth aspects,
Aspect ratio determining means for determining the aspect ratio or degree of the eyes of the user in the captured image,
The eyelash drawing means draws an eyelash image by varying the length and / or density of cilia based on the determination result of the aspect ratio determination means.
An image generation apparatus may be configured.

  According to the tenth aspect, the eyelashes are drawn in a vertical direction with respect to the lateral length of the eyes, such as drawing short eyelashes for users with thin eyes and drawing long eyelashes for users with large eyes. According to the difference in length ratio (so-called fineness / size of eyes), it is possible to draw eyelash images having different lengths and densities of cilia.

Further, as an eleventh aspect, in the image generation apparatus according to any one of the first to tenth aspects,
The eyelash color determining means (for example, the eyelash drawing processing unit 1213 in FIG. 13) further determines the color of the eyelash image based on the color of the eyelash drawing position specified by the eyelash drawing position specifying means and the color in the vicinity thereof. ,
The eyelash image generation means draws the eyelash image in the color determined by the eyelash color determination means;
An image generation apparatus may be configured.

  According to the eleventh aspect, since the color of the eyelash image is determined based on the eyelash drawing position and the color in the vicinity of the eyelash drawing position, it is possible to obtain an eyelash image with a more uncomfortable feeling. Specifically, for example, a method of extracting a dark color color (a color whose brightness satisfies a predetermined dark color condition) in the vicinity of the eyelash drawing position and the vicinity thereof, and setting the average color as the color of the eyelash image is conceivable.

1 is a schematic configuration diagram of a seal printing machine. The figure which shows an example of the display screen of a sticker printing machine. The figure which shows an example of the display screen of a sticker printing machine. The figure which shows an example of the display screen of a sticker printing machine. Explanatory drawing of the layer used by image generation. Explanatory drawing of a fitting process. The figure which shows an example of the drawing image of eyelashes. The figure which shows the example of failure of the drawing image of eyelashes. The figure which shows the example of failure of the drawing image of eyelashes. The figure which shows an example of the screen for graffiti. The figure which shows an example of erase operation of eyelashes. Explanatory drawing of the eyelash texture for a display, and the eyelash texture for printing. The block diagram which shows the function structure of a sticker printing machine. The flowchart which shows the flow of a sticker creation process. The flowchart which shows the flow of a process at the time of imaging | photography. The flowchart which shows the flow of the eyelash drawing process for a display. The flowchart which shows the flow of an image editing process. The flowchart which shows the flow of the process at the time of printing. The flowchart which shows the flow of the eyelash drawing process for printing. 1 is a schematic external view of a portable game device. The figure which shows an example of the display screen of a portable game device. The figure which shows an example of the display screen of a portable game device. The figure which shows an example of the display screen of a portable game device. Explanatory drawing of the modification of an eyelash drawing process. The figure which shows an example of the screen at the time of graffiti in a modification. The figure which shows an example of the screen at the time of graffiti in a modification. The figure which shows the modification of an eyelash texture.

  Hereinafter, preferred embodiments of the present invention will be described with reference to the drawings. In the following, a case where the present invention is applied to a seal printing machine, which is a kind of image generation apparatus, will be described as an example. However, embodiments to which the present invention can be applied are not limited thereto.

1. Schematic Configuration FIG. 1 is a schematic configuration diagram showing the appearance of a sticker printing machine 1000 according to this embodiment. The sticker printing machine 1000 is a kind of image generation apparatus. On the front of the casing having a height higher than the height of the user, a camera 1001, a display 1002, a speaker 1003, an infrared communication unit 1004 for performing infrared communication with the user's mobile phone, and game consideration. A coin insertion slot 1005 for inserting coins and a sticker outlet 1006 for paying out the created sticker paper are provided.

  The camera 1001 is an electronic still camera having an image sensor such as a CCD type or a CMOS type, and is provided at a position where a user standing in front of the housing can be photographed from the front direction.

  The display 1002 is provided from the user's waist to the chest level so that the display surface faces the user. In addition, a touch panel is integrally formed on the display 1002 over the entire display area. The touch panel detects the touch operation position in units of dots constituting the display 1002, for example, based on a detection principle such as a pressure-sensitive type, an optical type, an electrostatic type, or an electromagnetic induction type. The user can perform various operation inputs by touch operations on the display 1002 with the attached touch pen 1007 or fingers.

  The sticker printing machine 1000 includes a system board 1009 on which a CPU, a ROM, a RAM, an IC memory, and the like are mounted, a printing device 1008 that prints an image on sticker paper, and the like. The CPU mounted on the system board 1009 executes various processes based on programs and data read from the IC memory and the like, operation signals from the touch panel, etc., and generates image signals for various screens and sound signals for various sounds. To do. Then, the generated image signal is output to the display 1002 to display a corresponding display screen, and the sound signal is output to the speaker 1003 to output the corresponding sound.

  In the sticker printing machine 1000, a photo sticker in which an image taken by the camera 1001 is printed on a predetermined sticker paper is created. One of the features of the present embodiment is that the eyelash image is automatically drawn on the eyelash portion of the user in the captured image and printed on the sticker paper.

2. Next, the principle of creating a photo sticker in the stick printing machine 1000 will be described. First, the user inserts a predetermined amount of coins (coins) into the coin insertion slot 1005 in the sticker printing machine 1000. Then, the sticker printing machine 1000 displays a display screen 10 as shown in FIG.

  The display screen 10 is a screen for allowing the user to select whether or not to draw eyelashes on the captured image. Two icons, “Yes” and “No”, are displayed along with a message “Do you want to add eyelashes?”. Using the touch pen 1007, the user touches the “Yes” icon when he / she wishes to attach eyelashes, and the “No” icon when he / she does not wish to attach eyelashes.

  When it is detected that the “Yes” icon is touched with the touch pen 1007, the sticker printing machine 1000 displays a display screen 20 as shown in FIG. 3 on the display 1002. The display screen 20 is a screen for allowing the user to select an eyelash mode that is a mode for drawing eyelashes on a captured image. Two icons, “Doll Eye” and “Elegant Eye” are displayed along with the message “Choose Eyelash Mode”.

  “Doll eye” is a mode for drawing eyelashes having a cilia that is longer and denser than other parts near the user's black eye (pupil) in the captured image. On the other hand, the “elegant eye” is a mode for adding eyelashes having cilia that are longer and denser than the other parts near the user's eyes in the captured image.

  When it is detected that any icon on the display screen 20 is touched with the touch pen 1007, the sticker printing machine 1000 displays the display screen 30 as shown in FIG. The display screen 30 is a screen for allowing the user to select the eyelash volume to be added to the captured image. Two icons, “Standard Volume” and “Hyper Volume”, are displayed along with the message “Choose the eyelash volume”.

  When “standard volume” is selected, the eyelashes with a natural volume feeling are drawn on the photographed image as if mascara was applied. On the other hand, when “hypervolume” is selected, long and lashed eyelashes are drawn on the photographed image as if they were put on.

  When it is detected that any icon on the display screen 30 is touched with the touch pen 1007, the sticker printing machine 1000 shifts to the shooting mode and performs processing at the time of shooting. In the photographing process, the sticker printing machine 1000 performs photographing by the camera 1001 after performing a predetermined countdown indicating shutter timing (photographing timing), and stores the finder image at that time as a photographed image. When shooting is finished, the sticker printing machine 1000 performs eyelash drawing processing for drawing eyelashes on the shot image.

  In the present embodiment, the seal printing machine 1000 generates an image using three layers. Specifically, as shown in FIG. 5, the three layers of the basic layer on which the photographed image is drawn, the eyelash layer on which the eyelash image is drawn, and the graffiti layer on which the graffiti image is drawn are sequentially arranged from the back side. The images are generated by using them in a superimposed manner. Each layer has the same size as the photographed image, and the photographed image is drawn on the entire surface of the base layer.

  In each pixel of the base layer, an RGB value representing the color of the user image obtained by shooting is stored, so that a shot image is drawn. In each pixel of the eyelash layer, an RGB value representing the color of the eyelash image to be drawn (hereinafter referred to as “the eyelash image color”) is stored, whereby the eyelash image is drawn. A value representing colorless and transparent is stored in a pixel that is not drawn. Each pixel of the graffiti layer stores an RGB value representing a color colored in a graffiti process, which will be described later, so that a graffiti desired by the user is drawn. Similarly to the eyelash layer, the graffiti layer stores a value indicating colorless and transparent for pixels not drawn.

  In the present embodiment, there are two types of eyelash layers: a display eyelash layer and a printing eyelash layer. Although details will be described later, since the size of the display 1002 is larger than the printed sticker paper or one photo printed on the sticker paper, the display eyelash layer and the print eyelash layer use the eyelash cilia to be drawn. The thickness and density of the are changed.

FIG. 6 is a diagram for explaining eyelash drawing processing.
First, face recognition processing is performed to detect eyes from the captured image. Specifically, for example, a large number of face images serving as dictionary data are prepared and face recognition is performed using a known pattern matching technique, or the contour extraction of the white eye portion is performed based on the color information of each pixel. For example, the eye contour is extracted to recognize the eye portion. The eye part image detection process itself may use a known process. In the present embodiment, as shown in FIG. 6, description will be made on the assumption that the image portion of the eye is detected in an octagonal region with the corner of the eye “A” and the corner of the eye “B” as left and right end points. A line from the top of the eye “A” to the corner of the eye “B” via the vertex “C” of the eye is the upper eyeline, and the opposite side is the lower eyeline. The detection of the eye image part can be called the detection of the eyeline part, and is a process for specifying the position for drawing the eyelashes (the eyelash drawing position). Further, in the present embodiment, the term “eye line” is basically described as indicating the upper eye line.
In the present embodiment, when eye detection fails, the process of drawing eyelashes is not performed.

  On the other hand, the eyelash texture corresponding to the eyelash mode and the eyelash volume is prepared as the eyelash texture. In the present embodiment, there are two types of “doll eye” and “elegant eye” as the eyelash mode, and two types of “standard volume” and “hyper volume” as the eyelash volume. Therefore, four types of eyelash textures, which are combinations of the eyelash mode and the eyelash volume, are prepared in advance for each of the right eye and the left eye.

  In the eyelash texture for the doll eye, the cilia near the black eyes are drawn thicker and longer than the other cilia so that long dark eyelashes are drawn near the user's black eyes. On the other hand, in the eyelash texture for the elegant eye, the cilia near the corner of the eye is drawn longer and thicker than the cilia at other positions so that long and dark eyelashes are drawn near the user's corner. In addition, in comparison with the standard volume eyelash texture, the hypervolume eyelash texture is drawn with a longer length of each cilia and a higher density of the entire cilia.

Each eyelash texture has the same size, although the type of eyelashes drawn is different. Further, three fitting points “X” to “Z” are set in each eyelash texture, and these are also set at the same position on the uv coordinates of the texture. The fitting point is a control point for matching (fitting) the eyelash image to the detected eye shape. The fitting point “X” is a control point for fitting to the position of the eye head “A”, and the fitting point “Y” is a control point for fitting the eye corner “B”. The fitting point “Z” is a control point for fitting to the vertex “C” of the eye.
Although only the right eye is illustrated and described with reference to FIG. 6, the same applies to the left eye.

  The seal printing machine 1000 selects the eyelash texture corresponding to the “lash mode” and the “lash volume” selected by the user from the four types of eyelash textures. Then, to match the relative positional relationship of each fitting point “X” to “Z” of the selected eyelash texture with the relative positional relationship of the vertices “A” to “C” of the detected image portion of the eye, Scale the vertical and horizontal size of the texture.

  Further, by rotating the eyelash texture with the fitting point “X” as a base point, the absolute positional relationship (angle θ) of the fitting points “X” to “Z” is determined from the vertices “A” to “A” of the detected image portion of the eye. The absolute positional relationship (angle θ) of “C” is matched. As a result, the fitting points “X” to “Z” are adjusted to correspond to the vertices “A” to “C” of the image portion of the eye, and texture mapping is ready.

  After performing the fitting process, the eyelash texture is mapped (drawn) to the display eyelash layer, thereby generating an image in which eyelashes are drawn on the eyeline part of the eye as shown in FIG. However, the colors of all pixels related to the eyelashes drawn on the eyelash texture are not drawn on the display eyelash layer. This will be specifically described.

  First, a pixel to be colored in the eyelash image (hereinafter, referred to as “lash drawing target pixel”) is determined. Specifically, when the eyelash texture is projected onto the display eyelash layer, the pixel at the position corresponding to the cilia portion of the eyelash of the eyelash texture is determined as the eyelash drawing target pixel. In order to avoid confusion, the eyelash drawing target pixel is described as a pixel in the eyelash layer. However, since the three layers are overlapped to form one composite image, the eyelash drawing target pixel is divided into three layers. It can be said that it is a common pixel in a composite image.

  The eyelash image is unconditionally drawn on each eyelash drawing target pixel of the display eyelash layer. Then, the eyelash may not be drawn at an appropriate position. For example, FIG. 8 and FIG. 9 are examples. In FIG. 8, the eye image portion was not accurately detected or the octagon was roughly recognized, so that the eye portion was detected to be narrower (narrower) than the original eye image portion, and eyelashes were observed in the white and black eye portions. Has been drawn. This results in an unnatural image as if eyelashes are growing from the white or black eyes.

  In FIG. 9, eyelashes are drawn on the hair. In the captured image, the user's eyes are covered with hair, and as a result of detecting the image portion of the eyes including the portion of the hair that is in the eyes, the eyelashes are drawn on the hair. is there. In this case, an image with a sense of incongruity in which the front-rear positional relationship between the hair and the eyelashes contradicts each other.

  In order to deal with such a problem, in the present embodiment, for each eyelash drawing target pixel included in the photographed image, the color of the pixel is a skin color equivalent color condition that is predetermined as a color condition that allows drawing of eyelashes. It is determined whether or not the above is satisfied. The skin color equivalent color condition is a condition of a color value (RGB value) predetermined as a skin color or a color close to the skin color (hereinafter referred to as “skin color equivalent color”). Since the skin color varies from person to person, or even from the same person, the skin color may differ partially, it is appropriate to give the skin color equivalent color a certain range. The skin color of the subject (user) photographed in the face recognition process may be detected, and a certain color width centered on the skin color may be set as the skin color equivalent color condition.

The eyelash color (hereinafter referred to as “the eyelash image color”) is only the pixel in which the color of the corresponding base layer pixel satisfies the skin color equivalent color condition among the target eyelash drawing target pixels included in the display eyelash layer. Coloring. In other words, only the image portion of the eyelash drawn on the eyelash texture is used as a mask, and a predetermined color (for example, black) is colored as the eyelash image color. The eyelash image color may be, for example, an average color of a dark color (for example, a color whose brightness satisfies a predetermined dark color condition) near the eye image portion detected in the face recognition process or the face recognition process. Thus, the color of the portion determined to be the image portion of the hair may be the same color as the color of the portion, or a color with a slight lightness correction may be used as the eyelash image color. Of course, it is good also as using the color previously colored for the eyelash texture.
As a result, the eyelash image color is not colored in pixels that do not satisfy the skin color equivalent color condition. Therefore, the eyelashes are not drawn in a color other than the skin color equivalent color, for example, white in the white eye part or black part such as hair, and an unnatural image as shown in FIGS. 8 and 9 is generated. It is prevented.

  When the eyelash drawing process is completed, the seal printing machine 1000 combines the captured image of the basic layer, the eyelash image of the display eyelash layer, and the graffiti image of the graffiti layer in this order so as to overlap each other. Then, the sticker printing machine 1000 causes the display 1002 to display a graffiti screen for the user to perform a graffiti operation.

  FIG. 10 is a diagram illustrating an example of a graffiti screen. On the graffiti screen, an image (hereinafter referred to as “display composite image”) 40 in which a basic layer, a display eyelash layer, and a graffiti layer are superimposed, a currently selected graffiti tool 50, and a change of the graffiti tool A button 62 and an end button 64 for instructing the end of the graffiti are displayed.

  The scribble tool 50 is a tool for scribbling the composite image 40 for display, and can be changed to another tool such as a tool for changing the pen color or the pen tip or a stamp tool by a change button 62. The graffiti tool 50 currently selected in FIG. 10 includes a pen tool 51 and an eraser tool 52. The pen tool 51 is a pen tool that can freely draw a line drawing in the composite image for display 40, and displays four types of icons according to the pen thickness.

  The user touches an icon of an arbitrary thickness in the pen tool 51 using the touch pen 1007, and then slides the touch pen 1007 on the composite image for display 40, thereby a predetermined color (for example, black) along the locus. Can be colored. As an internal process, RGB values corresponding to the color of the pen tool 51 are stored in the graffiti layer of the three layers according to the coloring operation locus by the touch pen 1007.

  The eraser tool 52 is also provided with four types of thickness. One characteristic of the present embodiment is that the image erasing operation using the eraser tool 52 extends not only to the graffiti layer but also to the display eyelash layer.

  That is, the user touches an icon of an arbitrary thickness in the eraser tool 52 using the touch pen 1007, and then slides the touch pen 1007 on the display composite image 40, thereby coloring the locus portion. Can be made colorless and transparent. However, the erase operation with the eraser tool does not reach the base layer on which the photographed image is drawn. This is because it is a graffiti for the photographed image. However, the erase operation extends not only to the graffiti layer but also to the display eyelash layer. The eyelashes drawn by the eyelash drawing process are treated as a kind of “doodle” and can be erased at the request of the user. As shown in FIG. 11, when the user performs an erasing operation on the drawn eyelash image, the portion of the erasing operation locus in the eyelash image is erased. As an internal process, an erase operation locus by the touch pen 1007 is stored, and the color information of the erase operation locus on the display eyelash layer and the graffiti layer is changed to colorless and transparent color information. The erasing operation trajectory is stored for use in printing processing.

  When the graffiti ends, the sticker printing machine 1000 performs a printing process, generates a print composite image, and prints on the sticker paper.

FIG. 12 is an explanatory diagram of the principle of generating a print composite image in the present embodiment.
It has been explained that there are four types of eyelash textures, which are combinations of “lash mode” and “lash volume”. However, each of the four types of eyelash texture has two types: display and printing. The difference between the display and printing eyelash texture is the overall thickness and density of each cilia. The fitting point is set at a common position. In the eyelash texture for display, the thickness of each cilia is thinner and the density is lower than that of the eyelash texture for printing.

  The reason is that the display 1002 of the sticker printing machine 1000 has a screen sufficiently larger than the size of the sticker paper to be printed. That is, the size of the user's face image displayed on the display 1002 is sufficiently larger than the face image printed on the sticker paper.

  If the thickness and density of each cilia drawn on the printing eyelash texture are the same according to the display eyelash texture, the facial image displayed on the display 1002 has the thickness / The density looks good, but it looks fine / low in the face image printed on sticker paper. On the contrary, when the thickness and density of each cilia drawn on the display eyelash texture are the same in accordance with the print eyelash texture, the face image printed on the sticker paper has a thickness / Although the density looks appropriate, the face image displayed on the display 1002 appears thicker / higher in density.

  In consideration of this, the eyelash texture for display is drawn with the thickness of each cilia thinner and the density lower than the eyelash texture for printing. The eyelashes drawn on the display eyelash layer described above are eyelash images based on the display eyelash texture.

  The difference between the display eyelash texture and the print eyelash texture may be the difference between both the thickness and density of the cilia, or the difference between either the thickness and density of the cilia. That is, the density of the cilia may be the same, and the thickness of each cilia may be thinner in the display eyelash texture, or the thickness of each cilia is the same and the density of the cilia is in the direction of the display eyelash texture. May be lowered.

  The composite image for printing is an image in which the basic layer on which the photographed image is drawn, the printing eyelash layer on which the eyelash image is drawn based on the printing eyelash texture, and the graffiti layer are overlapped in this order from the back side. . Drawing of the eyelash image on the eyelash layer for printing is performed as follows. That is, the vertical / horizontal expansion / contraction ratio and the rotation angle θ executed in the fitting process described with reference to FIG. 6 are applied to the printing eyelash texture. Similarly to the eyelash drawing process described above, when the printing eyelash texture is projected onto the printing eyelash layer, the pixel corresponding to the cilia portion of the eyelash texture of the printing eyelash texture is set as the eyelash drawing target pixel, and the eyelash Among the drawing target pixels, the eyelash image color is colored to a pixel whose corresponding base layer pixel color condition satisfies the skin color equivalent color condition.

Further, in the printing eyelash layer, the color information of the portion of the erasing operation trajectory input on the graffiti screen is changed to colorless and transparent (see FIG. 12). As a result, even if the display eyelash layer is replaced with the printing eyelash layer, the erase operation once input by the operation is reflected as it is.
Next, the base layer, the printing eyelash layer, and the graffiti layer are stacked in this order from the back side to generate a printing composite image.

3. Functional Configuration FIG. 13 is a block diagram showing an internal configuration of the sticker printing machine 1000. The sticker printing machine 1000 includes an operation input unit 1100, a processing unit 1200, an image display unit 1330, a sound output unit 1340, a communication unit 1400, a photographing unit 1500, a printing unit 1600, a discharge mechanism 1620, and a storage. Part 1700.

  The operation input unit 1100 receives an operation instruction from the user, and outputs an operation signal corresponding to the operation to the processing unit 1200. In FIG. 1, a touch panel formed integrally with the display 1002 corresponds to this.

  The processing unit 1200 performs various arithmetic processes such as control of the entire seal printing machine 1000, image generation, and sound generation. This function is realized by, for example, an arithmetic device such as a CPU (CISC type or RISC type), an ASIC (gate array or the like), or a control program thereof. In FIG. 1, the CPU mounted on the system board 1009 corresponds to this.

  The processing unit 1200 includes a seal creation processing unit 1210 that performs processing related to creation of the photo sticker 100, a display control unit 1230 that controls image display of the image display unit 1330, and various sound such as sound effects and BGM. And a sound generation unit 1240 for generating a sound signal.

  The seal creation processing unit 1210 includes an imaging control unit 1211, an eyelash drawing processing unit 1213, an image editing unit 1215, and a print control unit 1217, and performs various processes related to creation of a photo sticker.

  The shooting control unit 1211 controls shooting (imaging) of the user (subject) by the shooting unit 1500. Specifically, after performing a countdown for instructing the shutter timing, the photographing unit 1500 performs photographing at the shutter timing. Then, the shooting control unit 1211 causes the storage unit 1700 to store the data of the image taken by the shooting unit 1500 as the shot image data 1731.

  The eyelash drawing processing unit 1213 performs a fitting process on the captured image captured by the imaging control unit 1211 to fit the display eyelash texture to the image portion of the user's eyes. Then, the eyelash is drawn on the display eyelash layer by coloring the eyelash image color to the eyelash drawing target pixel of the display eyelash layer. In addition, the eyelash drawing processing unit 1213 performs a fitting process on the printing eyelash texture in the same manner based on the fitting data 1741 executed in the previous fitting process, and applies an eyelash image color to the eyelash drawing target pixel of the printing eyelash layer. By coloring the eyelashes, the eyelashes are drawn on the printing eyelash layer.

  The display eyelash texture data 1721 stores four types of eyelash textures for display by a combination of “lash mode” and “lash volume”. Similarly, the printing eyelash texture data 1723 stores four types of printing eyelash textures based on a combination of “lash mode” and “lash volume”.

  The image editing unit 1215 manages the basic layer, the display eyelash layer, the print eyelash layer, and the graffiti layer, generates a display composite image, and stores the composite image for display in the storage unit 1700 as the display composite image data 1737. .

  Further, the image editing unit 1215 displays a graffiti screen on the image display unit 1330 to perform a graffiti process, and colors the graffiti layer according to the user's coloring operation locus using the pen tool. In addition, the color is erased from the display eyelash layer and the graffiti layer according to the erase operation locus using the eraser tool. The erase operation locus at this time is stored as erase operation locus data 1743.

  The print control unit 1217 generates a composite image for printing and generates a unit seal image. Then, control is performed so that the printing unit 1600 prints an image in which the generated unit seal image is arranged and configured in a predetermined size.

  The image display unit 1330 is a display realized by hardware such as CRT, LCD, ELD, PDP, and HMD, for example, and the display 1002 corresponds to this in FIG.

  The sound generation unit 1240 generates various sounds such as sound effects and BGM, and outputs a sound signal of the generated sound to the sound output unit 1340. The sound output unit 1340 outputs sound based on the sound signal from the sound generation unit 1240. This function is realized by, for example, a speaker. In FIG. 1, the speaker 1003 corresponds to this.

  The communication unit 1400 communicates with an external device such as a mobile phone and transmits / receives data to / from the external device. This function is realized by a wireless communication module conforming to a wireless communication standard such as a wireless LAN, IrDA, Bluetooth (registered trademark), or the like. In FIG. 1, the infrared communication unit 1004 corresponds to this.

  The imaging unit 1500 performs imaging in accordance with the control of the imaging control unit 1211 and outputs the captured image data to the processing unit 1200. This function is realized by a photographing lens or an image sensor such as a CCD or CMOS that converts incident light that has passed through the photographing lens into a digital signal. In FIG. 1, the camera 1001 corresponds to this.

  The printing unit 1600 executes printing on a predetermined sheet (sticker sheet) according to the control of the print control unit 1215. This function is achieved by, for example, an ink jet printer, a laser printer, a sublimation printer, a thermal transfer printer, a fusion printer, a thermal printer, a photographic paper printer, an instant film printer, or a printing device or a paper feeding device, or a driver circuit thereof. Realized. In FIG. 1, the printing apparatus 1008 corresponds to this.

  The discharge mechanism 1620 is a transport mechanism that discharges the sticker sheet on which the print image is printed by the printing unit 1600 as a photo sticker 100 from the sticker outlet 1006 to the outside of the sticker printing machine 1000.

  The storage unit 1700 stores system programs and data for realizing various functions for causing the processing unit 1200 to control the seal printing machine 1000 in an integrated manner, and is used as a work area of the processing unit 1200. Calculation results executed by the 1200 according to various programs, input data input from the operation input unit 1100, and the like are temporarily stored. This function is realized by, for example, various IC memories, hard disks, CD-ROMs, DVDs, MOs, RAMs, VRAMs, and the like. In FIG. 1, the IC memory mounted on the system board 1009 corresponds to this.

  Further, the storage unit 1700 stores a seal creation program 1701 for causing the processing unit 1200 to function as the seal creation processing unit 1210 as a program. The storage unit 1700 stores display eyelash texture data 1721, printing eyelash texture data 1723, and graffiti tool data 1725 as data prepared in advance. As updated data at any time, captured image data 1731, basic layer data 1732, display eyelash layer data 1733, print eyelash layer data 1734, graffiti layer data 1735, display composite image data 1737, , Print composite image data 1739, fitting data 1741, and erasing operation trajectory data 1743 are stored.

4). Processing Flow FIG. 14 is a flowchart showing a flow of a sticker creation process executed in the sticker printing machine 1000 when the sticker creation processing unit 1210 reads and executes the sticker creation program 1701 in the storage unit 1700. This process is started when it is confirmed in the seal printing machine 1000 that a predetermined amount of coins has been inserted into the coin insertion slot 1005.

  First, in the sticker printing machine 1000, the sticker creation processing unit 1210 displays the display screens W10 to W30 illustrated in FIGS. 2 to 4 on the image display unit 1330, and in accordance with a user operation input from the operation input unit 1100, the eyelashes Drawing presence / absence, eyelash mode and eyelash volume are set (step A1). Thereafter, the shooting control unit 1211 performs a shooting process (step A3).

FIG. 15 is a flowchart showing the flow of processing during shooting.
First, the shooting control unit 1211 starts shooting (step B1). That is, the photographing unit 1500 performs photographing at the shutter timing after the countdown is performed, and the photographed image is stored in the storage unit 1700 as photographed image data 1731. The seal creation processing unit 1210 stores data obtained by pasting (drawing) the captured image on the base layer in the storage unit 1700 as base layer data 1732.

  When shooting is finished (step B3; Yes), the shooting control unit 1211 determines whether or not drawing of eyelashes is present (step B5). That is, in step A1, it is determined whether or not the drawing of eyelashes is set according to the user's operation input. If it is determined that there is eyelash drawing (step B5; Yes), the eyelash drawing processing unit 1213 executes display eyelash drawing processing (step B7).

FIG. 16 is a flowchart showing the flow of eyelash drawing processing.
First, the eyelash drawing processing unit 1213 performs eye detection processing (step C1). That is, it tries to detect the image portion of the eye from the captured image.

  If the eyelash drawing processing unit 1213 succeeds in detecting the eye (step C3; Yes), the eyelash mode and eyelash volume selected by the user in step A1 from the display eyelash texture data 1721 stored in the storage unit 1700. The eyelash texture for display corresponding to is read (step C5).

  Then, the eyelash drawing processing unit 1213 performs a fitting process on the display eyelash texture (step C7). Specifically, the positional relationship between the fitting points “X” to “Z” predetermined in the display eyelash texture is made to match each of the eye feature points “A” to “C” detected in Step C1. In addition, enlargement / reduction control of the display eyelash texture and rotation control of the display eyelash texture are performed.

  Thereafter, the eyelash drawing processing unit 1213 stores the enlargement / reduction ratio and the rotation angle of the display eyelash texture as fitting data 1741 in the storage unit (step C9). Then, the eyelash drawing processing unit 1213 performs a process of determining the eyelash drawing target pixel (step C11). Specifically, a pixel corresponding to the colored portion of the eyelash texture of the fitted eyelash texture is selected from the display eyelash layer, and is set as the eyelash drawing target pixel.

  Next, the eyelash drawing processing unit 1213 executes the process of Loop A for each eyelash drawing target pixel determined in step C11 (steps C13 to C21). In the process of Loop A, the eyelash drawing processing unit 1213 performs a coloring permission condition determination process on the eyelash drawing target pixel (step C15). That is, it is determined whether or not the color of the pixel of the base layer corresponding to the eyelash drawing target pixel is a predetermined skin color equivalent color. Is determined to satisfy the coloring permission condition.

  If the eyelash drawing processing unit 1213 determines that the eyelash drawing target pixel satisfies the coloring permission condition (step C17; Yes), the eyelash drawing target pixel is colored with the eyelash image color (step C19). The processing shifts to the next eyelash drawing target pixel. Moreover, if it determines with the said eyelash drawing object pixel not satisfy | filling coloring permission conditions (step C17; No), the eyelash drawing process part 1213 will process to the next eyelash drawing object pixel, without performing step C19. Transition.

  After processing all the eyelash drawing target pixels, the eyelash drawing processing unit 1213 ends the process of loop A (step C21), and ends the display eyelash drawing processing. If it is determined in step C3 that eye detection has failed (step C3; No), the eyelash drawing processing unit 1213 ends the display eyelash drawing processing.

  Returning to the processing at the time of shooting in FIG. 15, after performing the display eyelash drawing process, the seal creation processing unit 1210 displays an image obtained by superimposing the base layer, the display eyelash layer, and the graffiti layer as a display composite image. (Step B9). At this stage, the graffiti layer is a colorless and transparent layer on which nothing is drawn. If it is determined in step B5 that the drawing of the eyelashes is “none” (step B5; No), the imaging control unit 1211 superimposes the basic layer and the graffiti layer to form a composite image for display (step B11). ).

  After the process of step B9 or B11, the seal creation processing unit 1210 displays the display composite image on the image display unit 1330 (step B13). Then, the seal creation processing unit 1210 determines whether or not the composite image for display is OK according to the user's operation input (Step B15). If it is determined that the display composite image is OK (Step B15; Yes), photographing End time processing. When it is determined that the composite image for display is NG (step B15; No), the process returns to step B1 and the photographing process is executed again.

  Returning to the seal creation process of FIG. 14, after performing the photographing process, the image editing unit 1215 performs the image editing process (step A5).

FIG. 17 is a flowchart showing the flow of image editing processing.
First, the image editing unit 1215 causes the image display unit 1330 to display a graffiti screen on which a display composite image, a graffiti tool, and the like are arranged (step D1).

  Next, the image editing unit 1215 determines whether or not the pen tool 51 has been selected by the user (step D3). If it is determined that the pen tool 51 has been selected (step D3; Yes), the image editing unit 1215 sets the graffiti layer according to the pen tool coloring operation locus. A color is colored (step D5).

  Thereafter, the image editing unit 1215 determines whether or not the eraser tool 52 has been selected by the user (step D7). If it is determined that the eraser tool 52 has been selected (step D7; Yes), an erasing operation corresponding to the user's erasing operation is performed. The trajectory is stored in the storage unit 1700 as erase operation trajectory data 1743 (step D9). Then, according to the erase operation locus, the color of the erase operation locus portion is erased from the display eyelash layer and the graffiti layer (step D11). That is, the colored color information is changed to colorless and transparent.

  Then, the image editing unit 1215 determines whether or not to end the graffiti (step D13), and when it is determined that the graffiti is not yet ended (step D13; No), the process returns to step D3. If it is determined that the graffiti is to be terminated (step D13; Yes), the image editing process is terminated.

  Returning to the seal creation process of FIG. 14, after performing the image editing process, the print control unit 1217 performs a printing process (step A7).

FIG. 18 is a flowchart showing the flow of processing during printing.
First, the print control unit 1217 determines whether or not eyelash printing is present (step E1). When it is determined that there is presence (step E1; Yes), the eyelash drawing processing unit 1213 performs printing. An eyelash drawing process is performed (step E3).

FIG. 19 is a flowchart showing the flow of the printing eyelash drawing process.
First, the eyelash drawing processing unit 1213 reads the eyelash texture corresponding to the eyelash mode and the eyelash volume selected by the user in step A1 from the printing eyelash texture data 1723 stored in the storage unit 1700 (step F1).

  Thereafter, the eyelash drawing processing unit 1213 reads the fitting data 1741 stored in the storage unit 1700 (step F3). Then, the eyelash drawing processing unit 1213 determines the eyelash drawing target pixel (step F5). Specifically, the pixel corresponding to the colored portion of the eyelash texture of the fitted eyelash texture is detected from the printing eyelash layer, and determined as the eyelash drawing target pixel.

  Then, the eyelash drawing processing unit 1213 executes the process of loop B for each drawing target pixel (steps F7 to F15). In the process of Loop B, the eyelash drawing processing unit 1213 performs a coloring permission condition determination process on the eyelash drawing target pixel (step F9). That is, it is determined whether or not the color of the pixel of the base layer corresponding to the eyelash drawing target pixel is a predetermined skin color equivalent color. Is determined to satisfy the coloring permission condition.

  When the eyelash drawing processing unit 1213 determines that the eyelash drawing target pixel satisfies the coloring permission condition (step F11; Yes), the eyelash drawing target pixel is colored with the eyelash image color (step F13). The processing shifts to the next eyelash drawing target pixel. If it is determined that the eyelash drawing target pixel does not satisfy the coloring permission condition (step F11; No), the eyelash drawing processing unit 1213 proceeds to the next eyelash drawing target pixel without performing the process of step F13. Migrate processing.

  After processing all the eyelash drawing target pixels, the eyelash drawing processing unit 1213 ends the process of loop B (step F15). Thereafter, the eyelash drawing processing unit 1213 erases the eyelash image color from the printing eyelash layer according to the erase operation locus stored in the erase operation locus data 1743 (step F17). Then, the eyelash drawing processing unit 1213 ends the printing eyelash drawing process.

  Returning to the processing at the time of printing in FIG. 18, after performing the printing eyelash drawing process, the print control unit 1217 generates a print composite image by superimposing the basic layer, the printing eyelash layer, and the graffiti layer, and stores them. The data is stored in the unit 1700 (step E5).

  If it is determined in step E1 that the printing of the eyelashes is “none” (step E1; No), the print control unit 1217 generates a composite image for printing by superimposing the basic layer and the graffiti layer, The data is stored in the storage unit 1700 (step E7).

  After the process of step E5 or E7, the print control unit 1217 generates a seal image by arranging the print composite images in a predetermined arrangement configuration and causes the printing unit 1600 to print them (step E11). Then, the print control unit 1217 ends the printing process.

  Returning to the sticker creation process of FIG. 14, when the printing process is completed, the sticker creation processing unit 1210 causes the discharge mechanism 1620 to discharge the generated photo sticker 100 from the sticker outlet 1006 to the outside of the sticker printing machine 1000. (Step A9). When the above processing is performed, the seal creation processing ends.

5. Operation In the sticker printing machine 1000, a face image of the user is taken using the camera 1001. Then, processing for detecting the user's eyes from the captured face image is performed, and fitting processing for fitting the eyelash texture to the detected eyes is performed, so that the user's eyelash drawing position in the captured image Is identified. Then, a predetermined eyelash image color is colored at the specified eyelash drawing position, whereby a display image on which the eyelash image is drawn is generated and displayed on the display 1002.

  In the present embodiment, a plurality of types of eyelash textures such as an eyelash texture that exerts an effect like a mascara and an eyelash texture that exerts an effect like a false eyelash are prepared in advance, and the eyelash texture selected according to a user's selection instruction operation An eyelash image is drawn using. Therefore, an effect different from the correction of the skin color, such as mascara and false eyelashes, can be applied to the eyes.

  Further, for each eyelash drawing target pixel to be colored with the eyelash image color, a process of determining whether or not the eyelash drawing target pixel satisfies a predetermined coloring permission condition is performed. The coloring permission condition is, for example, that the color of the eyelash drawing target pixel is a skin color equivalent color. Then, the eyelash drawing target pixel that satisfies the coloring permission condition is colored with the eyelash image color, but the eyelash drawing target pixel that does not satisfy the coloring permission condition is not colored.

  Depending on the shape of the user's eyes, when the eyelash texture fitting process is performed, the eyelash drawing portion may enter the eye. In this case, when the eyelash image color is colored in the portion, an image having a sense of incongruity as if eyelashes are growing from the white or black eye portion is generated. In this regard, by coloring the eyelash image color only for pixels corresponding to the skin color, the eyelash image color is not colored in the white eye portion and the black eye portion, and an image with a natural impression can be generated.

  In this embodiment, the eyelash image drawing process is performed by changing the types of the display eyelashes to be displayed on the display 1002 and the printing eyelashes to be printed on the photo sticker. Specifically, two types of eyelash textures are used: a display eyelash texture in which the thickness of each cilia is thin and the density is low, and a printing eyelash texture in which the thickness of each cilia is thick and the density is high. This is because the size of the display 1002 is larger than the size of the photo sticker, so that the size of the face image displayed on the display 1002 is different from the size of the face image printed on the photo sticker.

  Furthermore, in the present embodiment, it is possible to erase the drawn eyelashes by using an eraser tool at the time of graffiti on the image for graffiti. Internally, the eyelash image color is erased from the display image in accordance with the erase locus input operation on the display 1002 using the touch pen 1007. Thus, the user can partially erase the display eyelash image drawn on the photographed image by an input operation of the erasure locus on the display 1002.

  For example, when a couple of men and women have taken pictures, eyelashes are drawn not only on the eyes of women but also on the eyes of men in the processing of the above-described embodiment. However, for men, there are cases in which it is not desired that eyelashes be drawn on their own face images. In that case, the male can erase the eyelashes drawn on his / her face image at the time of graffiti. In addition, as a woman's standpoint, there is a case where it is desired to correct the drawn eyelashes in order to make his / her face image appear more beautiful. Even in such a case, the woman can finely adjust the eyelashes during graffiti.

  Further, in the present embodiment, the eyelash erasing operation trajectory using the eraser tool at the time of graffiti is stored, and the eyelash image color is erased according to the stored erasing operation trajectory in the process of drawing the eyelash on the print image. . Thereby, the erasing operation on the display eyelash image performed by the user at the time of graffiti can be reflected also on the print eyelash image.

6). Modifications Note that embodiments to which the present invention can be applied are not limited to the above-described embodiments, and it is needless to say that the embodiments can be appropriately changed without departing from the spirit of the present invention.

6-1. In the above-described embodiment, the case where the present invention is applied to a sticker printing machine, which is a kind of image generating apparatus, has been described as an example. However, the image generating apparatus to which the present invention can be applied is not limited thereto. is not. For example, the present invention can be applied to a portable game device which is a kind of image generation device.

  FIG. 20 is a schematic external view showing the configuration of the portable game apparatus 1800. The portable game apparatus 1800 includes a direction input key 1802 and a button switch 1804 for a player to input a game operation, a first liquid crystal display 1806, a second liquid crystal display 1808, a speaker 1810, and a control unit 1850. A flip-flop type apparatus main body 1801 that can be opened and closed by a hinge 1814 is provided. Touch panels 1807 and 1809 are provided on the display surfaces of the first liquid crystal display 1806 and the second liquid crystal display 1808. The touch panels 1807 and 1809 can touch and input arbitrary positions on the display screen by touching with a stylus pen 1816 or the like. .

  In addition, the apparatus main body 1801 is provided with a memory card reader 1818 that can read and write data from a memory card 1840 that is a computer-readable storage medium. The memory card 1840 is a storage unit that functions as the storage unit 1700 of the print sealer 1000 described above.

  In addition, an image sensor module 1830 which is a kind of digital photographing unit is mounted on the hinge 1814. Then, a function such as a shutter is assigned to a predetermined switch of the button switch 1804, and the user captures his / her face image, whereby digital image data can be displayed on the displays 1806 and 1808.

  When a face image of the user is captured by the image sensor module 1830, the control unit 1850 performs eyelash drawing processing to draw eyelashes on the captured image, as in the above-described embodiment. That is, a process for detecting eyes from the captured image is performed, and a fitting process for eyelash texture and a coloring process for eyelash image color are performed to generate a display image in which eyelashes are drawn. FIGS. 21 to 23 show examples of display screens displayed on the portable game apparatus 1800. In addition, the portable game device 1800 is not provided with a printing device. Therefore, a program in which the printing process is omitted from the above-described seal creation process is stored in the memory card 1840.

6-2. In the above-described embodiment, for each eyelash drawing target pixel, it is determined whether or not the eyelash drawing target pixel satisfies a predetermined color condition as a condition for allowing coloring of the eyelash image color. It has been described that the eyelash image color is colored when it is satisfied. Specifically, it is determined whether or not the color of the pixel is a skin color equivalent color, and when the color is a skin color equivalent color, the eyelash drawing target pixel is colored with the eyelash image color.

  This processing may be performed as follows. That is, for each eyelash drawing target pixel, it is determined whether or not the eyelash drawing target pixel satisfies a predetermined color condition as a condition that does not allow coloring of the eyelash image color. Color the color. The color condition in this case can be that the color of the eyelash drawing target pixel is white or a color close to it (white equivalent color), or black or a color close to it (black equivalent color). By performing such a process, it is possible to prevent eyelashes from being drawn in the white-eye portion and the black-eye portion, and eyelashes from being drawn over the hair when the eyes are covered with hair.

6-3. Generation of eyelash image In the above-described embodiment, a process of detecting the eye of the user in the captured image (detecting an eyeline portion) is performed, and a process of fitting the eyelash texture is performed on the detected eye. The eyelash drawing position was specified. In addition to this method, it is also possible to analyze the curve shape of the user's eyeline portion in the captured image and draw the eyelashes so as to directly match the shape.

FIG. 24 is an explanatory diagram of eyelash drawing processing in this case.
In the embodiment described above, the eyelash texture in which the cilia are arranged in a curved shape is used as the eyelash texture, but here, the eyelash texture in which the cilia are arranged in a straight line is used. In the eyelash texture, a fitting point “X” is set at the lower right and a fitting point “Y” is set at the lower left.

  First, processing for analyzing the curve shape of the eyeline EL from the captured image is performed. Specifically, for example, image processing such as tone curve processing for increasing the brightness of a captured image and unsharp mask processing for sharpening the captured image is performed. Then, a known edge detection process using, for example, a differential filter is performed on the photographed image after processing, thereby detecting the eyeline EL curve itself from the photographed image and specifying the eyelash drawing position.

  When the eyeline EL curve is detected, the fitting point “X” is matched with the right end point “A” of the eyeline EL, the fitting point “Y” is matched with the left end point “B” of the eyeline EL, and the detected curve is detected. A curve along the shape is defined on the eyelash texture. Then, the eyelash texture is texture-mapped to the eyeline EL based on the coordinates of the eyelash texture on the defined curve. By coloring the eyelash image color to the pixel corresponding to the colored portion of the eyelash of the eyelash texture, an image in which the eyelash is drawn along the curve of the eyeline EL as shown in FIG. 24 can be acquired.

6-4. Eyelash image color In the above-described embodiment, the eyelash image color has been described as the eyelash image color drawn in advance (for example, black). However, the eyelash image color is determined based on the photographed image, and the eyelash image color is determined. You may decide to draw.

  Specifically, a dark color, for example, black or a color close to black, is selected from the boundaries of the image portion of the eye detected from the captured image and the pixels near the curve position of the eyeline EL specified in FIG. (Hereinafter referred to as “black equivalent color”) pixels are extracted. Then, an average color of the extracted pixel colors is calculated and determined as an eyelash image color, and the eyelash is colored using the eyelash image color.

  Further, the eyelash image color may be determined based on the color of the user's hair. Specifically, the hair image portion is recognized from the photographed image, and the average color of each pixel included in the hair image portion is calculated to determine the eyelash image color.

  In addition, in order to prevent the brightness of the eyelash image from being lowered when the eyelash is colored with the average color obtained by the above-described processing, the brightness correction is performed on the average color, so that it is slightly brighter than the average color. The corrected color may be the eyelash image color.

6-5. Addition of eyelashes in graffiti mode The eyelashes may not be drawn when the display image is displayed on the display 1002 at the time of shooting, but may be drawn when the user gives an instruction to add eyelashes when editing the image. . In this case, if it is detected that the user has performed an instruction operation for drawing eyelashes in the image editing process without performing the display eyelash drawing process in the photographing process, the display eyelash drawing process is performed, and the display composite image is displayed. Is generated and displayed on the display 1002.

  25 and 26 are diagrams illustrating an example of a graffiti screen displayed on the display 1002 during graffiti. As shown in FIG. 25, an eyelash addition button 70 is provided in the lower right part of the screen at the time of graffiti. When a touch operation is performed on this button, as shown in FIG. Is displayed.

6-6. Eraser tool The eraser tool has been described as being commonly used for the graffiti layer and the display eyelash layer, but the eraser tool for the graffiti layer and the eraser tool for the eyelash layer are prepared and used separately. It may be possible. In that case, it is possible to edit only the drawn eyelashes.

6-7. Eyelash texture according to the eye size The eyelash texture must be prepared for two types of eyelash textures for display and for printing, corresponding to the four combinations of “lash mode” and “lash volume”. As explained. However, since the size of the user's eyes varies, the eyelash texture having a different cilia length / density according to the size (height / width ratio) of a user with a small eye and a user with a large eye It may be used.

  Specifically, when an eye image portion is detected, the length from the top of the eye “A” to the bottom of the eye “B” and the distance to the vertex “C” of the eye with respect to the line segment (length of the perpendicular) The aspect ratio of the eyes is determined according to the ratio. Then, according to the aspect ratio, the size of the eyes (degrees of the aspect ratio of the eyes) is divided into “thin”, “normal”, and “large” degrees. On the other hand, depending on the degree of “thin”, “normal”, and “large”, prepare the eyelash texture in which the length of the cilia becomes longer and the density is increased from “thin” to “large” . For example, FIG. 27 shows an example of the eyelash texture when only the length of the cilia is varied. The eyelash texture corresponding to the determined degree of eye size is used. In other words, for the display eyelash texture, three types of “narrow”, “normal”, and “large” eyelash textures are prepared corresponding to four combinations of “lash mode” and “lash volume”. Will be. The same applies to the eyelash texture for printing.

  In addition, when the eyelash texture of FIG. 24 described in the section “6-3. Generation of eyelash image” is used, the length of the cilia can be varied by scaling in the vertical direction. For this reason, after determining the size of the user's eyes, the length of the cilia can be varied by scaling the eyelash texture in the vertical direction according to the determination result.

DESCRIPTION OF SYMBOLS 1000 Seal printing machine 1001 Camera 1002 Display 1003 Speaker 1004 Infrared communication part 1005 Coin insertion slot 1006 Seal discharge outlet 1007 Touch pen 1008 Printing apparatus 1009 System board 1100 Operation input part 1200 Processing part 1210 Sticker production process part 1211 Shooting control part 1213 Eyelash drawing process Unit 1215 image editing unit 1217 print control unit 1230 display control unit 1240 sound generation unit 1330 image display unit 1340 sound output unit 1400 communication unit 1500 photographing unit 1600 printing unit 1620 discharge mechanism 1700 storage unit

Claims (14)

An image generation device including a camera and a display device,
Photographing control means for controlling the camera to photograph the user;
Eyelash drawing position specifying means for specifying the eyelash drawing position in a photographed image taken by the camera;
Eyelash drawing means for drawing a given eyelash image at the eyelash drawing position specified by the eyelash drawing position specifying means;
Display control means for displaying on the display device a captured image in which the eyelash image is drawn by the eyelash drawing means;
An image generation apparatus comprising: The eyelash drawing means includes
Drawing target pixel determining means for determining a drawing target pixel of the eyelash image in the captured image based on the eyelash drawing position specified by the eyelash drawing position specifying means;
A coloring means for coloring the drawing target pixel with the color of the eyelash image when the color of the drawing target pixel satisfies a predetermined color condition as a condition for allowing coloring of the color of the eyelash image;
The image generation apparatus according to claim 1, comprising:   The image generation apparatus according to claim 2, wherein at least a color condition for determining a skin color is set in the color condition. The eyelash drawing means includes
Drawing target pixel determining means for determining a drawing target pixel of the eyelash image in the captured image based on the eyelash drawing position specified by the eyelash drawing position specifying means;
Coloring means for coloring the color of the eyelash image to the pixel to be drawn when the color of the pixel to be drawn does not satisfy a predetermined color condition as a condition not allowing the coloration of the color of the eyelash image;
The image generation apparatus according to claim 1, comprising:   The image generation apparatus according to claim 4, wherein at least a color condition for determining a white-eye portion is set in the color condition. A printing device;
Print control means for printing the photographed image on which the eyelash image is drawn by the printing device;
Further comprising
The display screen of the display device is a screen larger than the size of the printing paper printed by the printing device,
The eyelash image includes a printed eyelash image, and the thickness of each cilia is narrower than that of the printing eyelash image, and / or the density of the cilia is lower than that of the printing eyelash image, and the display eyelash drawn. There is an image,
The eyelash drawing means includes
Display eyelash drawing means for drawing the display eyelash image on the captured image;
A printing eyelash drawing means for drawing the printing eyelash image on the photographed image;
Have
The display control means causes the display device to display a captured image in which the display eyelash image is drawn by the display eyelash drawing means,
The printing control unit causes the printing apparatus to print a captured image in which the printing eyelash image is drawn by the printing eyelash drawing unit;
The image generation apparatus as described in any one of Claims 1-5. Display eyelash part erasing means for partially erasing the display eyelash image from the captured image controlled to be displayed on the display device according to an input operation of the user's erasing locus on the display screen of the display device;
A printing eyelash part erasing unit for partially erasing the printing eyelash image from a photographed image on which the printing eyelash image is drawn according to the erasing locus;
Further comprising
The printing control means causes the printing apparatus to print a photographed image in which the printing eyelash image is partially erased by the printing eyelash part erasing means.
The image generation apparatus according to claim 6. The eyelash drawing position specifying means specifies the user's eyeline part in the captured image as the eyelash drawing position,
Further comprising eyelash image generation means for generating the eyelash image by deforming a predetermined eyelash pattern image so as to match the shape of the eyeline part,
The image generation apparatus as described in any one of Claims 1-7. The eyelash image generation means generates the eyelash image by scaling the eyelash pattern image according to the length of the eyeline part.
The image generation apparatus according to claim 8. Aspect ratio determining means for determining the aspect ratio or degree of the eyes of the user in the captured image,
The eyelash drawing means draws an eyelash image by varying the length and / or density of cilia based on the determination result of the aspect ratio determination means.
The image generation apparatus as described in any one of Claims 1-9. Further comprising eyelash color determining means for determining the color of the eyelash image based on the color of the eyelash drawing position specified by the eyelash drawing position specifying means and the vicinity thereof,
The eyelash image generation means draws the eyelash image with the color determined by the eyelash color determination means.
The image generation apparatus as described in any one of Claims 1-10. A shooting control step for shooting the user by controlling the camera;
An eyelash drawing position specifying step for specifying an eyelash drawing position in a captured image taken by the camera;
An eyelash drawing step of drawing a given eyelash image at the eyelash drawing position specified in the eyelash drawing position specifying step in the captured image;
A display control step of causing the display device to display a captured image in which the eyelash image is drawn in the eyelash drawing step;
A printing control step of causing the printing device to print the captured image on which the eyelash image is drawn; and
A method for producing a print sticker. A computer built in a device equipped with a camera and a display device,
Photographing control means for controlling the camera and photographing the user;
Eyelash drawing position specifying means for specifying the eyelash drawing position in the photographed image taken by the camera;
Eyelash drawing means for drawing a given eyelash image at the eyelash drawing position specified by the eyelash drawing position specifying means in the captured image;
Display control means for displaying on the display device a photographed image in which the eyelash image is drawn by the eyelash drawing means;
Program to function as.   A computer-readable storage medium storing the program according to claim 13.

Images