JP5472410B2 - Automatic photo creation device, image processing device, image processing method, and image processing program - Google Patents

Description

  The present invention relates to an automatic photo creation apparatus and an image processing apparatus that capture a user with a camera and output a composite image generated based on the captured image.

  2. Description of the Related Art Conventionally, there is known a photo creation device for a game in which a user is photographed with a camera and the photographed image is output as a photo sticker (or a photo card). Such a play photo creation device is highly playable or entertaining, and therefore, an image to be combined with a photographed image is prepared from a variety of background images and foreground images prepared in advance according to the user's preference. Many are configured so that a user can freely select and draw using a pen-type input means such as a touch pen or a light pen. Since such an action by the user is performed on the photographed image, it is called “graffiti”.

  By the way, it is preferable for the user to obtain a so-called “photographing” good photo sticker. In particular, as the word “small face (kogao)” has become popular in recent years, many women feel that they want to make their faces look smaller, and their faces appear relatively large. Users are more pleased with photo stickers that have a relatively smaller face than photo stickers. Therefore, Patent Document 1 discloses an invention of an automatic photo creation apparatus that can correct a captured image. According to this automatic photo creation device, processing for narrowing the cheek and processing for reducing the face are performed on the captured image based on an instruction from the user.

JP 2005-277772 A

  However, according to the automatic photo creation apparatus disclosed in Patent Document 1, the user himself has to perform an operation for reducing the face in the photographed image. In addition, the joy of the user that his / her face appears small on the photo sticker is greater when there is no such recognition than when there is a recognition that “the process of reducing the face has been applied” It is considered to be.

  Therefore, the present invention makes the user's face appear smaller than the original without imposing an operation burden on the user and without making the user aware of the fact that the process of reducing the face has been performed. It is an object of the present invention to provide an automatic photo creation device capable of outputting a photo sticker (a user's small face).

The present invention relates to an automatic photo creation device and an image processing device for photographing a user who is a subject in a predetermined photographing region and outputting a composite image including a graffiti target image generated based on the photographed image. A user's face in the captured image is detected, and a graffiti target image is generated based on the captured image so that the contour of the user's face in the graffiti target image is smaller than the contour of the user's face in the captured image. At that time, the pixels constituting the user's face are moved based on the positions of both eyes of the user in the captured image . More preferably, a line that divides the user's face in the captured image into a right half and a left half is set as the second reference line, and pixels on the chin side of the user's eyes are positioned in the second reference line. Move toward the reference line.

According to the present invention, a user's face is detected in a photographed image, and a small face process for reducing the contour of the user's face is performed to generate a graffiti target image. Then, a composite image including the graffiti target image is output. Here, when the graffiti target image is generated based on the photographed image, the coordinates of the pixel data of the user's face are moved so as to approach a line that divides the user's face into half halves . For this reason, the width of the user's face in the graffiti target image is shorter than the original . As a result, it is possible to provide the user with a photograph whose face is thinner than the original.

It is a figure which shows the external appearance of the photo production apparatus for games which is an automatic photo production apparatus which concerns on one Embodiment of this invention. It is a front view of the imaging unit in the embodiment. It is a front view of the edit unit in the said embodiment. In the said embodiment, it is a schematic diagram which shows the screen structure of the touchscreen for imaging operations. In the said embodiment, it is a schematic diagram which shows the screen structure of the touch panel for edit operations. It is a block diagram which shows the structure which looked at the principal part of the photo production apparatus for play concerning the said embodiment from the functional surface. It is a flowchart which shows the process sequence of the imaging | photography process in the said embodiment. It is a flowchart which shows the process sequence of the edit process in the said embodiment. It is a flowchart which shows the process sequence of the small face process in the said embodiment. In the said embodiment, it is a figure for demonstrating a small face process. In the said embodiment, it is a figure for demonstrating a small face process. In the said embodiment, it is a figure for demonstrating a small face process. In the said embodiment, it is a figure for demonstrating a small face process. It is a flowchart which shows the process sequence of the imaging | photography process in the modification of the said embodiment. It is a flowchart which shows the process sequence of the brightness compensation process in the said modification. In the modified example, the equation is used for conversion of data from the RGB color space to the Lab color space. In the said modification, it is a figure which shows the tone curve which prescribed | regulated the relationship between the brightness before correction | amendment, and the brightness after correction | amendment.

  Hereinafter, an embodiment of the present invention will be described with reference to the accompanying drawings.

<1. Overall configuration>
FIG. 1 is a diagram showing an appearance of a game photo creation device that is an automatic photo creation device according to an embodiment of the present invention. More specifically, FIG. 1 (a) is an external side view of the play photo creation apparatus as viewed from the side, and FIG. 1 (b) is an external plan view as viewed from above. This play photo creation device includes a shooting room 2 in which a user U enters, a shooting unit 3 that takes a picture of the user U and accepts selection of a background image and a foreground image, and editing including graffiti (drawing operation) by the user U And an editing unit 4 that receives an operation and generates a synthesized image synthesized with the photographed image. FIG. 2 is a front view of the photographing unit 3, and FIG. 3 is a front view of the editing unit 4. Hereinafter, the overall configuration of the game photo creation device according to the present embodiment will be described with reference to FIGS. 1 to 3.

  The photographing room 2 has a substantially rectangular parallelepiped shape, and the photographing unit 3 is arranged along the front surface of the photographing room 2 which is a front surface when viewed from the user U entering the inside. In addition, an opening for allowing the user U to enter and exit and a light-shielding curtain that covers a part or all of the opening are provided in part of the left and right sides of the imaging room 2. In addition, an electric roll curtain device 25 is provided on the rear surface, which is the rear surface when viewed from the user U entering the inside of the photographing room 2. The electric roll curtain device 25 stores a roll curtain with a single color (in this case, blue or green) for chroma key composition processing. The roll curtain is unfolded at the start of photographing and rolled up at the end. Taken. The electric roll curtain device 25 may be configured to appropriately unfold and wind up a plurality of roll curtains each depicting a predetermined pattern, landscape, etc., when the chroma key composition process is not performed. Further, when the floor surface of the photographing room 2 is a subject to be photographed, the single color is also given to the floor surface.

  The photographing unit 3 includes a camera 10 as an imaging unit that photographs the user U, and strobes 11, 12, 13L, 13R, and 14 as illumination units that are arranged at appropriate positions on the top, bottom, left, and right of the camera 10 and emit flash light. And a photographing operation touch panel 20 that is disposed below the camera 10 and receives an operation from the user U and displays a photographed image.

  The camera 10 is typically a digital camera that generates a digital image signal using a CCD (Charge Coupled Device), images the user U, and outputs an image signal representing the captured image. The flashes 11 to 14 emit a flash toward the user U in order to obtain sufficient light for photographing. The shooting operation touch panel 20 is configured to provide an operation screen for accepting various operations by the user U during shooting, and to display an image based on the image signal in real time. FIG. 4 is a schematic diagram illustrating a screen configuration of the shooting operation touch panel 20. As shown in FIG. 4, the shooting operation touch panel 20 has a real-time preview area 201 for displaying a shot image in real time, a pose sample display area 202 for displaying a pose sample, and a graffiti obtained by shooting. A graffiti target image display area 203 for displaying the target image is included.

  In addition, the photographing unit 3 is configured mainly by a computer and incorporates a control device that controls each part, an I / O control device, a network adapter that communicates with the editing unit 4, and the like. A network printer 35 is provided as output means for outputting as a photo sticker (or photo card). Further, the photographing unit 3 is provided with a coin insertion slot 26 on the lower front side and a photo sticker outlet 28 on the side.

  The editing unit 4 is configured around a computer similar to the photographing unit 3 and includes a control device that controls each part and the like, and a network adapter that communicates with the photographing unit 3 and the like. The editing unit 4 is divided into two units 4a and 4b so that two sets of users U can play as shown in FIG. One unit 4a includes an editing operation touch panel 400 including a graffiti area, an area for displaying a menu for graffiti, a tool, and the like, and touch pens 49L and 49R used for operations on the editing operation touch panel 400. Is provided. The other unit 4b has the same configuration. Here, the unit 4a seen in the foreground in FIG. 1A is referred to as “graffiti booth A”, and the unit 4b on the opposite side (the back side in the figure) is referred to as “graffiti booth B”.

  The editing operation touch panel 400 according to the present embodiment has a screen configuration in which each of the units 4a and 4b allows two users U to simultaneously perform graffiti. In this description, components used by the left user U are given reference signs including “L”, and components used by the right user U are assigned reference signs including “R”. It is attached.

  FIG. 5 is a schematic diagram showing a screen configuration of the editing operation touch panel 400. The editing operation touch panel 400 displays an image selected from a plurality of graffiti target images generated based on the photographed image as a component used by each of the left and right users U, and displays a (virtual) graffiti. Graffiti areas 41L and 41R, which can be performed, and tools for the user U to perform graffiti, that is, a set of tools for editing instructions for the (selected) graffiti target image, and displayed as usable Pallets 42L, 42R, mode selection buttons 44L, 44R for switching the types of tools included in the palettes 42L, 42R, and buttons for performing operations related to graffiti other than tools that can be included in the palettes 42L, 42R 45L, 45R. The pallets 42L and 42R are provided with tabs 43L and 43R, and various types of items can be displayed and selected by switching the tabs. Further, the editing operation touch panel 400 includes a timer 46 for displaying the remaining graffiti time as components commonly used by the left and right users, and a plurality of editing graffiti target images as graffiti candidate images. A candidate image thumbnail 47, which is a group of images displayed in a small list, and a guide screen display area 48 for displaying usage instructions for the selected item when a tab or item is selected are included. ing.

  In the editing operation touch panel 400 shown in FIG. 5, the palette mode can be switched to, for example, “pen”, “stamp”, “frame”, and “background” by the mode selection buttons 44L and 44R. . Further, the button groups 45L and 45R, for example, “end of graffiti”, “erase all of graffiti”, “return one”, “eraser”, “background eraser”, “thick pen”, “medium pen”, and “ A function called “thin pen” is provided.

  In the configuration as described above, after shooting in the shooting room 2, the user uses the graffiti booth A of the editing unit 4 or the touch panel 400 for the editing operation of the graffiti booth B to generate based on the shot image. Graffiti is performed on the graffiti target image. The composite image generated by graffiti is printed out from the network printer 35.

<2. Functional configuration>
FIG. 6 is a block diagram showing the configuration of the main part of the play photo creating apparatus according to the present embodiment as seen from the functional aspect. Functionally, the game photo creation device has a shooting processing unit 7 for mainly performing a process of shooting the user U (shooting process), and mainly according to the graffiti operation of the user U on the graffiti target image. An editing processing unit 8 for performing an editing process on the graffiti target image and an output processing unit 9 for performing printing output are included. The imaging processing unit 7 includes a first control unit 70, an imaging unit 71, a first display / operation unit 72, an I / O control unit 73, an illumination unit 74, and a first communication unit 75. The edit processing unit 8 includes a second control unit 80, second display / operation units 81 and 82, and a second communication unit 83. The output processing unit 9 is configured by an output unit 90. The first and second communication units 75 and 83 that are network adapters and the output unit 90 that is the network printer 35 can communicate with each other via a network 6 such as a LAN (Local Area Network).

  The imaging unit 71 corresponds to the camera 10 configured using an imaging element such as a CCD, and captures an image in real time and outputs an image signal representing the image (captured image). This image signal is input to the first control unit 70 and temporarily stored as photographed image data in its internal memory. The captured image data is supplied as a captured image signal from the first control unit 70 to the first display / operation unit 72, and a captured image based on the captured image signal is displayed in real time.

  The first display / operation unit 72 corresponds to the shooting operation touch panel 20 and is used for selecting a background image and a foreground image to be combined with a shot image and determining a layout of a photo to be output. The operation of the person U and the shutter operation are accepted. Signals indicating these operations are input to the first control unit 70 as operation signals. Here, when a predetermined process (corresponding to the selected shooting menu) for shooting the user U is started, guidance for the user U is displayed on the first display / operation unit 72, Thereafter, based on an instruction from the first control unit 70, a flash is emitted from the flashes 11 to 14 in the shooting direction of the camera 10 after a predetermined time of about several seconds has elapsed. At that time, an image signal output from the imaging unit 71 as a signal representing a captured image of the user U is input to the first control unit 70, and a hard disk as a memory or an auxiliary storage device in the first control unit 70. It is stored as captured image data in a device or the like. In addition, a graffiti target image is generated by performing small face processing described later on the captured image data.

  The illumination unit 74 corresponds to the strobes 11, 12, 13 L, 13 R, and 14 disposed at appropriate positions on the top, bottom, left, and right of the camera 10, and is controlled by the I / O control unit 73 based on an instruction from the first control unit 70. Lighting / extinguishing and dimming are controlled. The I / O control unit 73 corresponds to an I / O control device built in the photographing unit 3, and controls the illumination unit 74 based on an instruction from the first control unit 70. Further, an input signal such as a detection signal from a coin detection unit (not shown) described later is transferred to the first control unit 70. The first communication unit 75 corresponds to a network adapter built in the photographing unit 3 and functions as an interface for data transmission / reception.

  The first control unit 70 is built in the photographing unit 3 and corresponds to a control device mainly composed of a computer including a CPU, a memory, a frame buffer, a timer, an auxiliary storage device, and the like, and is stored in an internal memory. When the CPU executes the predetermined program, an instruction is issued to each unit as described above in order to control each unit based on the operation signal or the like input as described above. A graffiti target image is generated based on the photographed image, and the generated graffiti target image is written in the frame buffer and displayed on the first display / operation unit 72. Further, the first control unit 70 generates a composite image that is an image in which a predetermined background image or foreground image is drawn on the graffiti target image. The generated composite image is displayed on the first display / operation unit 72 by being written in the frame buffer. When the shooting and generation of the composite image are completed in this way, the generated composite image is appropriately selected according to the operation input of the user U, and then the second corresponding to the editing unit 4 via the first communication unit 75. To the control unit 80.

  In addition to the above-described components, the photographing unit 3 is further provided with a coin detection unit (not shown) for detecting coins inserted into the coin insertion slot 26 in the photographing unit 3, and the first control unit 70 includes Based on the detection result of the coin detection unit, each unit is controlled so as to allow the user U to play with the present photo creation device, such as shooting, selection of background images and foreground images, and graffiti for a predetermined time. Since the control operation by the coin control unit and the first control unit 70 based on the detection result is the same as that of the conventional game photo creation device and is well known, detailed description thereof will be omitted.

  The second control unit 80 is a built-in editing unit 4 and corresponds to a control device mainly composed of a computer including a CPU, a memory, a frame buffer, an auxiliary storage device, etc., and a predetermined program stored in the internal memory Is executed by the CPU to perform overall control related to the editing process. The second control unit 80 also displays a composite image that is an image obtained by drawing a predetermined image on the composite image based on an operation signal for performing graffiti processing on the composite image sent from the first control unit 70. Generate. The generated composite image is displayed on the corresponding second display / operation unit 81, 82. When the image composition as described above is completed, the composite image is transmitted to the output unit 90. If the output unit 90 is outputting another composite image, a message to that effect is displayed and transmitted after waiting for completion.

  The second display / operation units 81 and 82 correspond to the editing operation touch panel 400 for graffiti, and accept the operation of the user U using the touch pen. The second communication unit 83 corresponds to a network adapter built in the editing unit 4 and functions as an interface for data transmission / reception.

  The output unit 90 corresponds to the network printer 35 incorporated in the photographing unit 3, and displays an image represented by image data image-processed according to the graffiti of the user U received from the second control unit 80 as a photo sticker (or Output as a photo card. The outputted photo sticker or the like is taken out from an outlet 28 provided on the side surface of the photographing unit 3.

  The predetermined program executed in the control device is provided by, for example, a DVD-ROM that is a recording medium on which the program is recorded. That is, a DVD-ROM as a recording medium for the predetermined program is mounted on a DVD-ROM driving device built in the control device as an auxiliary storage device, and the predetermined program is read from the DVD-ROM as an auxiliary storage device. Installed on the hard disk drive. The predetermined program may be provided via a recording medium (CD-ROM or the like) other than a DVD-ROM or a communication line. Then, when a predetermined operation for starting the game photo creation device is performed, the predetermined program installed in the hard disk device is transferred to the memory in the control device and temporarily stored therein, The CPU is executed. Thereby, the control process of each part by the control device is realized.

<3. Processing procedure in a photo creation device for play>
As described above, the photo creation device for play includes the photographing unit 3 and the editing unit 4. The photographing unit 3 performs photographing processing, and the editing unit 4 performs editing processing. It should be noted that when a certain user U is playing with the editing unit 4, another user U can play with the photographing unit 3. That is, this game photo creation device can perform the shooting process and the editing process in parallel. The outline of the processing procedure of the photographing process and the editing process will be described below.

<3.1 Shooting process>
FIG. 7 is a flowchart showing a processing procedure of the photographing process in the present embodiment. When the game photo creation device is not used (when play is not being performed), a demonstration image (demonstration image) is displayed on the shooting operation touch panel 20. When the user U inserts a coin into the coin insertion slot 26 during the display of the demo image, play is started (step S100).

  When the play is started, the first control unit 70 accepts the selection of the shooting mode by the user U (step S110). In this embodiment, an “auto shooting mode” in which shooting is automatically performed for the user U and a “discretion shooting mode” in which the user U selects a background and the like are prepared as shooting modes. It is explained as a thing. In step S <b> 110, the first control unit 70 displays a screen for causing the user U to select one of the modes on the shooting operation touch panel 20 and accepts a selection operation by the user U. Then, the first control unit 70 determines which mode the user U has selected. As a result of the determination, if the auto shooting mode is selected, the process proceeds to step S120, and if the discreet shooting mode is selected, the process proceeds to step S130.

  In step S120, a shooting theme is selected. Specifically, the first control unit 70 displays a screen for allowing the user U to select one (or a plurality) of shooting themes from a plurality of shooting themes prepared in advance. It displays on the touch panel 20 and receives selection operation by the user U. And the 1st control part 70 acquires selection information based on selection operation of user U, and determines the combination of the frame and background used at the time of photography based on the selected photography theme. . Thereafter, the process proceeds to step S121, and photographing is performed. By this shooting, the shot image data is stored in the memory of the first control unit 70.

  After the photographing is completed, the process proceeds to step S123, and face detection processing is performed based on the photographed image data stored in the memory of the first control unit 70. The face detection process is a process for detecting an area representing a part of the face of the user U from the captured image, that is, a process for detecting the contour of the face of the user U. In the present embodiment, the contour of the face of the user U is detected from the captured image using a known face detection process. For example, as disclosed in JP-A-8-63597, color feature values such as brightness, hue, saturation, and chromaticity are obtained from the RGB values of each pixel, and the target pixel is determined based on the feature values. It is determined whether or not the skin color, and edge extraction is performed on the image composed of pixels determined to be the skin color to acquire the edge image, and the edge image and a plurality of face templates prepared in advance are matched. Thus, the contour of the face of the user U can be detected. Further, as disclosed in Japanese Patent Laid-Open No. 2000-48184, after detecting a skin color area, a projection distribution of the skin color area is obtained, a face area candidate is extracted from the projection distribution, and an area for the face area candidate is extracted. The contour of the face of the user U can be detected in consideration of the aspect ratio and whether the face area candidate matches a face pattern prepared in advance. Further, as disclosed in Japanese Patent No. 4121026, the feature amount based on the gradient vector calculated from the size of the edge in the horizontal direction and the vertical direction, and reference data prepared in advance for face identification are used. An identification point may be obtained based on the identification point, and the face of the user U may be detected based on the identification point. After the face detection process ends, the process proceeds to step S125.

  In step S125, small face processing is performed. The small face processing is included in the captured image from the captured image so that the face of the user U in the finally output photo sticker is relatively smaller than the face of the user U in the actual captured image. This is a process of generating a graffiti target image by reducing the contour of the face of the user U who is currently on. A detailed description of the small face processing will be described later. After the small face processing ends, the process proceeds to step S127.

  In step S127, the graffiti target image generated based on the photographed image by the small face processing is displayed on the photographing operation touch panel 20. Specifically, each time the process of step S127 is performed, the graffiti target images are sequentially displayed in the graffiti target image display area 203 of the touch panel for photographing operation 20 illustrated in FIG. Thereafter, the process proceeds to step S129, and the first control unit 70 determines whether or not a predetermined number of images have been taken. As a result of the determination, if the number of images has been shot, the process proceeds to step S140. If the number of images has not been shot, the process returns to step S121.

  In step S130, the user U selects a combination of a frame and a background. Specifically, the first control unit 70 selects any one of a screen for causing the user U to select one of a plurality of frames prepared in advance and a plurality of backgrounds prepared in advance. A screen for allowing the user U to select the background is displayed on the touch panel 20 for photographing operation, and a selection operation by the user U is accepted. And the 1st control part 70 acquires selection information based on user U's selection operation. Thereafter, the process proceeds to step S131, and photographing is performed. By this shooting, the shot image data is stored in the memory of the first control unit 70.

  After shooting, face detection processing (step S133) and small face processing (step S135) are performed, and the graffiti target image generated by these processings is displayed on the shooting operation touch panel 20 (step S137). Note that the details of the processing in step S133, step S135, and step S137 are the same as those in step S123, step S125, and step S127, respectively, and thus description thereof is omitted. After the end of step S137, the process proceeds to step S139, and the first control unit 70 determines whether or not a predetermined time has elapsed since the start of play or since the shooting mode was selected. As a result of the determination, if the predetermined time has elapsed, the process proceeds to step S140, and if the predetermined time has not elapsed, the process returns to step S130.

  In step S140, an image (by the user U) that is an actual graffiti target is selected from a plurality of graffiti target images generated by the small face processing. Specifically, the first control unit 70 displays a list of graffiti target images on the shooting operation touch panel 20 and allows the user U to select the image to be used for graffiti and printing. Accept the operation. Then, the first control unit 70 transfers the image selected by the user U to the second control unit 80 as an actual graffiti target image. After step S140 ends, the process proceeds to step S150. In step S150, a guidance screen is displayed. Specifically, the first control unit 70 displays a screen for guiding the user U to any one of the editing units 4 (4 a or 4 b) on the photographing operation touch panel 20. Thereby, the photographing process is completed.

  In the present embodiment, the face detection means (face detection step) is realized by steps S123 and S133, and the small face processing means (small face processing step) is realized by steps S125 and S135.

<3.2 Editing process>
FIG. 8 is a flowchart showing a processing procedure of editing processing in the present embodiment. After the photographing process described above is completed, the second control unit 80 selects the graffiti target image (the image selected by the user U in step S140 in FIG. 8) sent from the first control unit 70 via the network 6. Obtain (step S200). Thereafter, the timer 46 is set to a predetermined time (time for allowing the user U to doodle) and the countdown is started (step S210).

  After the timer 46 starts counting down, the editing unit 4 accepts a graffiti operation (drawing operation) by the user U (step S220). When the remaining time for allowing the user U to doodle is 60 seconds, the second control unit 80 determines whether or not to extend the doodle time (step S230). Whether or not to extend the graffiti time depends on whether or not the user U who will play next in the unit 4a or 4b is in the shooting room 2 (whether or not the shooting process is performed in the shooting unit 3). Determined. As a result, if it is determined to extend the graffiti time, the process proceeds to step S240. On the other hand, if it is determined that the graffiti time is not extended, the process proceeds to step S250. In step S240, the timer is set to a predetermined time (time for allowing the user U to further doodle), and the countdown is started again. After step S240 ends, the process proceeds to step S250.

  In step S250, as in step S220, the editing unit 4 accepts a graffiti operation by the user U. When the graffiti time reaches 0 (ends), the division pattern of the output photo is selected (step S260). Specifically, the second control unit 80 displays on the editing operation touch panel 400 a screen for allowing the user U to select one of the plurality of division patterns prepared in advance, and uses the division pattern. The selection operation by the person U is received. Then, the second control unit 80 acquires selection information based on the selection operation of the user U. After step S260 ends, the process proceeds to step S270.

  In step S270, the second control unit 80 determines whether or not to register a composite image, which is a graffiti target image for which graffiti has been completed, in a predetermined server. Specifically, the second control unit 80 displays on the editing operation touch panel 400 a screen for allowing the user U to select whether or not to register the composite image in a predetermined server, and the user U selects the screen. Accept the operation. Then, the second control unit 80 determines which user U has selected. If it is determined that registration to the server has been selected, the process proceeds to step S280, and if selection to not register to the server has been performed, the process proceeds to step S290.

  In step S280, the image data of the composite image is transmitted to a predetermined server. After step S280 ends, the process proceeds to step S290. In step S290, a guidance screen is displayed. Specifically, the second control unit 80 displays a screen for guiding the user U to the photo sticker outlet 28 on the editing operation touch panel 400. Thereby, the editing process ends.

<3.3 Small face processing>
FIG. 9 is a flowchart showing a processing procedure of small face processing in the present embodiment. First, the first control unit 70 acquires information (hereinafter referred to as “number data”) indicating how many faces are included in the captured image based on the result of the face detection process described above (step S300). In this small face processing, the processing from step S310 to step S340 is repeated as many times as the number of people based on the number of people data. For example, if the photographed image includes three faces, the processing from step S310 to step S340 is repeated three times, and the faces of the three users U are sequentially processed.

  In step S310, the first control unit 70 extracts, from the captured image, a portion below the eyes from the outline of the face of the user U that is the processing target in the captured image (the extracted data is described below). "Extracted contour data"). Then, it progresses to step S320 and the 1st control part 70 performs the conversion of the coordinate of extraction outline data. This conversion is performed so that the contour of the face of the user U after conversion is smaller than the contour of the face of the user U before conversion. Thereby, for example, the contour of the face of the user U as shown in FIG. 10A is deformed into a shape as shown in FIG.

  After the end of step S320, the process proceeds to step S330, and the first control unit 70 converts the coordinates of the data of the neck portion of the user U in accordance with the transformation of the contour of the user U's face. Thereby, the length of the neck of the user U based on the data after conversion becomes longer than the length of the neck of the user U based on the data before conversion. After step S330 is completed, the process proceeds to step S340, and the first control unit 70 determines whether or not the process for the number of people is completed. As a result of the determination, if the processing for the number of people is completed, the small face processing is completed. If the processing for the number of people is not completed, the process returns to step S310.

  By the way, in the small face processing, the face of the user U in the captured image is reduced as follows, for example. First, a straight line passing through predetermined positions of the right eye and the left eye of the user U in the captured image is set as the first reference line. As shown in FIG. 11, a straight line 51 passing through the uppermost portions of the right eye and the left eye may be used as the first reference line, and a straight line 52 passing through the centers of the right eye and the left eye is used as the first reference line. Alternatively, the straight line 53 passing through the lowermost part of the right eye and the left eye may be used as the first reference line. Of the pixel data forming the contour of the face of the user U, the pixel data located on the chin side of the user U with respect to the first reference line is the extracted contour data. Also, as shown in FIG. 11, a straight line 55 that passes through the center of the forehead of the user U and the center of the jaw in the captured image is taken as the second reference line. Then, the transformation (movement) of the coordinates of the extracted contour data is performed in consideration of the first reference line and the second reference line.

  Hereinafter, assuming that the straight line 53 passing through the lowermost part of the right eye and the left eye of the user U is the first reference line, the coordinate conversion examples (first to third conversion examples) of the extracted contour data are shown. explain. In the first conversion example, as shown in FIG. 12A, the coordinates of each pixel of the extracted contour data are moved closer to the first reference line. Thereby, the captured image as shown in FIG. 11 is converted into an image as shown in FIG. In the second conversion example, as shown in FIG. 12B, the coordinates of each pixel of the extracted contour data are moved closer to the first reference line and closer to the second reference line. The distance moved closer to the reference line is equal to the distance moved closer to the second reference line. As a result, the photographed image as shown in FIG. 11 is converted into an image as shown in FIG. In the third conversion example, as shown in FIG. 12C, the coordinates of each pixel of the extracted contour data are moved closer to the first reference line and closer to the second reference line. The distance moved closer to the reference line is made longer than the distance moved closer to the first reference line. Thereby, the captured image as shown in FIG. 11 is converted into an image as shown in FIG. As described above, there are various ways of converting the coordinates of the extracted contour data.

<4. Effect>
As described above, according to the present embodiment, processing (small face processing) for reducing the contour of the face of the user U is performed on the captured image obtained by capturing, and the image generated thereby is a graffiti target. It is an image. Then, an image in which graffiti is applied to the graffiti target image is output. In the small face processing, pixel data that forms the contour of the face of the user U in the captured image when a straight line passing through predetermined positions of the right eye and left eye of the user U in the captured image is used as the first reference line. Of these, the coordinates of the pixel data located closer to the chin side of the user U than the first reference line are moved closer to the first reference line. Therefore, it is possible to provide the user U with a photograph in which the distance from the eyes to the chin is shorter than the original.

  Further, according to the present embodiment, in the small face processing, the portion of the user U's face that is closer to the chin than the predetermined position of both eyes is moved upward (close to the head) and close to the center (first). 2 conversion example). For this reason, with respect to the face of the user U in the graffiti target image, the distance from the eyes to the chin becomes shorter than the original distance, and the portion on the chin side of the eyes becomes thinner than the original. Accordingly, a photograph in which the distance from the eyes to the chin is shorter than the original and the portion closer to the chin than the eyes is thinner than the original can be provided to the user U.

  Further, according to the present embodiment, in the small face processing, the portion of the user's face contour that is closer to the chin than the predetermined position of both eyes is moved upward (close to the head) and close to the center. The contour is moved so that the moving distance in the center direction becomes larger than the moving distance to (the third conversion example). For this reason, as for the face of the user U in the graffiti target image, the distance from the eyes to the chin becomes shorter than the original distance, and the portion on the chin side of the eyes becomes considerably thinner than the original. Accordingly, it is possible to provide the user with a photograph in which the distance from the eyes to the jaw is shorter than the original and the portion on the jaw side of the eye is much thinner than the original.

  As described above, various methods can be applied to the small face processing method (how to convert the coordinates of the extracted contour data) (see FIGS. 13A to 13C). For this reason, "Photos with a shorter distance from the eyes to the jaw than the original", "Photos with a shorter distance from the eyes to the jaw than the original, and a portion closer to the jaw than the eyes. ”,“ Photos in which the distance from the eyes to the chin is shorter than the original, and the part on the chin side of the eyes is much thinner than the original ”, etc. It can be provided to the user U.

<5. Modification>
Hereinafter, modifications of the embodiment will be described. FIG. 14 is a flowchart showing the processing procedure of the photographing process in the present modification. In this modification, brightness compensation processing (step S124, step S134) is performed at the timing between the face detection processing (step S123, step S133) and the small face processing (step S125, step S135). Here, the brightness compensation process is a user in the graffiti target image generated by the small face process when the brightness of the face of the user U is darker than a predetermined target brightness in the captured image. It is a process to brighten U's face.

  FIG. 15 is a flowchart showing a processing procedure of brightness compensation processing in the present modification. After the start of the brightness compensation process, first, the first control unit 70 converts data from the RGB color space to the Lab color space for the pixel data in the contour of the user U's face detected by the face detection process. Is performed (step S400). This conversion can be performed using the equation shown in FIG. 16 as disclosed in, for example, Japanese Patent Laid-Open No. 2008-99148.

  Next, proceeding to step S410, the first control unit 70 extracts skin color data from the pixel data in the contour of the face of the user U based on the data converted into the Lab color space. In this modification, it is determined that the pixel data in which the a value and the b value are 18 or more and 25 or less is skin color data. Next, proceeding to step S420, the first control unit 70 calculates an average value of brightness (L value) for the skin color data extracted at step S410.

  Next, proceeding to step S430, the first control unit 70 calculates a difference between the average value calculated in step S420 and a predetermined (brightness) target value. Thereafter, the process proceeds to step S440, and the first control unit 70 converts the data indicating the brightness in the pixel data so as to compensate for the difference (between the average value and the target value) calculated in step S430. . This conversion can be performed, for example, according to a tone curve as shown in FIG. 17 that defines the relationship between the brightness before correction and the brightness after correction. When step S440 ends, the brightness compensation process ends.

  According to this modification, when the skin color portion of the face of the user U in the captured image is darker than the target brightness, the brightness of the skin color portion is compensated when the graffiti target image is generated. For this reason, even if the face color of the user U is relatively black, the photograph is generated based on the graffiti target image whose brightness is compensated, so the face of the user U in the photograph becomes clear. Further, even when the amount of light is insufficient at the time of shooting, the brightness of the skin color portion of the user U's face is compensated when generating the graffiti target image, so that the face of the user U in the photograph is clear It becomes.

  Further, according to the present modification, skin color data is extracted from the pixel data within the contour of the face of the user U based on the data converted into the Lab color space. Since the color expression in the Lab color space is close to the human viewing angle, the brightness of the skin color portion of the face of the user U in the captured image is effectively compensated.

  In this modification, the brightness compensation process is performed before the small face process. However, the present invention is not limited to this, and the brightness compensation process may be performed after the small face process. . Further, in this modification, it is determined whether or not each pixel data is a skin color based on the a value and the b value obtained by converting the RGB color space data into the Lab color space data. However, the present invention is not limited to this, and the skin color portion may be extracted by other methods. Furthermore, in this modification, data conversion is performed so as to compensate for the difference between the average brightness value and the target value for the skin color data. However, the present invention is not limited to this, and Data conversion may be performed so as to compensate for a difference between a median value or a value determined based on a histogram (for example, a mode value) and a target value.

<6. Other>
Although the above embodiment has been described on the assumption that one graffiti target image is generated for one captured image, the present invention is not limited to this. In order to generate a plurality of graffiti target images (for example, three images as shown in FIGS. 13A to 13C) for one captured image, and the user U selects one of these images. The above-mentioned means (graffiti target image selection means) may be provided. In this case, an operation by the user U is involved, but a photo sticker showing a reduced face according to the preference of the user U can be provided to the user U.

DESCRIPTION OF SYMBOLS 2 ... Shooting room 3 ... Shooting unit 4 ... Editing unit 10 ... Camera 11-14 ... Strobe 20 ... Shooting operation touch panel 51-53 ... 1st reference line 55 ... 2nd reference line 70 ... 1st control part 71 ... Imaging part (camera)
72. First display / operation unit (touch panel for photographing operation)
73 ... I / O control unit (I / O control device)
74 ... Lighting section (strobe)
75, 83 ... 1st and 2nd communication part (network adapter)
80 ... second control unit 81, 82 ... second display / operation unit (touch panel for editing operation)
400 ... Touch panel for editing operation U ... User

Claims (12)

An automatic photograph provided with an imaging means for photographing a user who is a subject in a predetermined photographing area, and an output means for outputting a composite image including a graffiti target image generated based on a photographed image obtained by the imaging means as a photograph A creation device,
Face detection means for detecting a user's face in the captured image;
Small face processing means for generating the graffiti target image based on the captured image so that the contour of the user's face in the graffiti target image is smaller than the contour of the user's face in the captured image;
The small face processing means includes
A straight line passing through predetermined positions of the user's right eye and left eye in the photographed image is set as a first reference line, and a line that divides the user's face in the photographed image into a right half and a left half is second. Set to the reference line
Of the range detected as the user's face in the photographed image, a pixel in a range that is on the chin side of the first reference line is extracted as an extraction pixel group, and pixels included in the extraction pixel group are An automatic photo creation apparatus, characterized by being moved toward a reference line . 2. The automatic photo creation apparatus according to claim 1, wherein the small face processing means moves a pixel included in the extracted pixel group toward the first reference line. 3. The automatic photo creation apparatus according to claim 1, wherein the small face processing unit moves pixels included in the extracted pixel group based on positions of both eyes of the user in the captured image. Generated by the small face processing means based on a difference between a predetermined target value and a value indicating the brightness of the skin color area among the areas included in the contour of the user's face detected by the face detecting means. that said further comprising: a brightness compensating means for compensating the brightness of skin color region among the regions included in the contour of the face of the user in the graffiti image of the claims 1 to 3 The automatic photo creation apparatus of any one of Claims. The brightness compensator is based on the a and b values obtained by converting the RGB values of the pixel data of the region included in the contour of the user's face detected by the face detector into Lab values. 5. The automatic photo creation apparatus according to claim 4 , wherein a skin color area is detected from an area included in the contour of the face. Face detection means for detecting a human face included in the photographed image;
A small-face processing unit that generates a secondary image based on the captured image with a small outline of the face of the person included in the captured image;
The small face processing means includes
A straight line passing through predetermined positions of the right and left eyes of the person in the photographed image is set as a first reference line, and a line that divides the human face in the photographed image into a right half and a left half is a second reference. Set to line,
Of the range detected as a human face in the photographed image, a pixel in a range located on the chin side of the first reference line is extracted as an extraction pixel group, and pixels included in the extraction pixel group are extracted from the second pixel. An image processing apparatus, wherein the image processing apparatus is moved toward a reference line . A face detection step for detecting a human face included in the captured image;
A small-face processing step for generating a secondary image based on the captured image with a small outline of the face of the person included in the captured image;
In the small face processing step,
A straight line passing through predetermined positions of the right eye and the left eye of the person in the photographed image is set as the first reference line, and a line dividing the person's face in the photographed image into a right half and a left half is a second line. Set to the reference line,
Of the range detected as the person's face in the captured image, pixels in a range that is on the chin side of the first reference line are extracted as an extraction pixel group, and pixels included in the extraction pixel group are the second pixel. An image processing method characterized by being moved toward a reference line . A face detection step for detecting a human face included in the captured image;
An image processing program for causing a computer to execute a small face processing step for generating a secondary image in which a contour of a human face included in the photographed image is reduced based on the photographed image,
In the small face processing step,
A straight line passing through predetermined positions of the right eye and the left eye of the person in the photographed image is set as the first reference line, and a line dividing the person's face in the photographed image into a right half and a left half is a second line. Set to the reference line,
Of the range detected as the person's face in the captured image, pixels in a range that is on the chin side of the first reference line are extracted as an extraction pixel group, and pixels included in the extraction pixel group are the second pixel. An image processing program which is moved toward a reference line . An automatic photograph provided with an imaging means for photographing a user who is a subject in a predetermined photographing area, and an output means for outputting a composite image including a graffiti target image generated based on a photographed image obtained by the imaging means as a photograph A creation device,
Face detection means for detecting a user's face in the captured image;
Small face processing means for generating the graffiti target image based on the captured image so that the contour of the user's face in the graffiti target image is smaller than the contour of the user's face in the captured image;
With
The small face processing means includes
A straight line passing through a predetermined position of the user's right eye and left eye in the captured image is set as a first reference line,
Of the range detected as the user's face in the captured image, the pixels in the range on the chin side of the first reference line are extracted as an extracted pixel group, and the pixels included in the extracted pixel group are moved. An automatic photo creation device characterized by that. Face detection means for detecting a human face included in the photographed image;
Small face processing means for generating a secondary image in which the contour of the face of the person included in the photographed image is reduced based on the photographed image;
With
The small face processing means includes
A straight line passing through a predetermined position of the right eye and left eye of the person in the photographed image is set as a first reference line,
Extracting, as an extracted pixel group, pixels in a range that is located on the chin side of the first reference line in a range detected as a human face in the captured image, and moving the pixels included in the extracted pixel group An image processing apparatus. A face detection step for detecting a human face included in the captured image;
A small face processing step for generating a secondary image based on the photographed image, wherein the contour of the face of the person included in the photographed image is reduced;
With
In the small face processing step,
A straight line passing through predetermined positions of the right eye and left eye of the person in the captured image is set as the first reference line,
Of the range detected as the person's face in the photographed image, the pixels in the range located on the chin side of the first reference line are extracted as the extraction pixel group, and the pixels included in the extraction pixel group are moved. An image processing method. A face detection step for detecting a human face included in the captured image;
A small face processing step for generating a secondary image based on the photographed image, wherein the contour of the face of the person included in the photographed image is reduced;
An image processing program for causing a computer to execute
In the small face processing step,
A straight line passing through predetermined positions of the right eye and left eye of the person in the captured image is set as the first reference line,
Of the range detected as the person's face in the photographed image, the pixels in the range located on the chin side of the first reference line are extracted as the extraction pixel group, and the pixels included in the extraction pixel group are moved. An image processing program.

Images