JP2018022126A - Photograph creation game machine and imaging method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image having a satisfaction level high for a user.SOLUTION: An imaging controlling unit images one or more users as subjects. A display control unit displays a live view image having the user before an imaging timing. In the live view image, a recognition unit determines by face recognition whether at least one of the face of the user and a part of the face is detected. For example, a guide corresponding to the recognition result by the recognition unit is output to the live view image. This technology can be applied to a photograph seal creation device.SELECTED DRAWING: Figure 19

Description

  The present technology relates to a photo creation game machine and a shooting method, and more particularly, to a photo creation game machine and a shooting method capable of providing an image with a high degree of satisfaction for a user.

  Conventionally, a photo sticker installed in an amusement facility or the like is known. The photo sticker machine photographs a user and edits the photographed image according to the user's operation. The photo sticker prints the edited photographed image on sticker paper.

  In recent years, in photo stickers, face recognition is performed on captured images in order to perform appropriate image processing on the captured images. For example, Patent Document 1 describes a photo sticker creation apparatus that displays a recognition result together with a photographed image on a screen for selecting a photographed image to be edited.

  Accordingly, it is possible to prompt the user not to select an image that has failed to be recognized, and it is possible to suppress the output of an image that has failed to be recognized and has not been appropriately subjected to image processing.

  Patent Document 1 discloses that a best position is shown when the target frame set on the recognized face matches the target frame set at an ideal position on the live view screen. It is described that a frame is displayed.

  By displaying the vest-positive frame on the live view screen, the user can easily recognize that he / she is at the optimum position in the shooting space.

  Further, Patent Document 2 describes guiding the faces of subjects close to each other according to the position of the face recognized in the live view image.

JP 2012-74778 A Japanese Patent No. 5304751

  If face recognition is performed on the live view image and the recognition is successful, appropriate image processing can be more reliably performed on the captured image. However, even if the user can easily grasp that he / she is in the optimal position simply by displaying the Vespoy frame on the live view screen or being guided to bring the faces of the subjects closer together, It was not possible to grasp whether or not face recognition was successful.

  As a result, when face recognition fails, an image that has not been appropriately subjected to image processing is output, and an image that is less satisfactory for the user may be provided.

  The present technology has been made in view of such a situation, and makes it possible to provide an image with higher satisfaction for a user.

  A photo creating game machine according to a first aspect of the present technology includes a shooting control unit that shoots one or more users as subjects, a display control unit that displays a live view image of the users before shooting timing, A recognition unit that determines whether or not at least one of the user's face and face parts is detected by face recognition in the live view image, and an output unit that outputs guidance according to a recognition result by the recognition unit With.

  The imaging method according to the first aspect of the present technology shoots one or more users as subjects, displays a live view image showing the user before the shooting timing, and performs face recognition in the live view image. Determining whether or not at least one of the user's face and facial parts is detected, and outputting a guidance according to a recognition result by the recognition unit.

  In the first aspect of the present technology, one or more users are photographed as subjects, a live view image showing the users is displayed before the photographing timing, and the use is performed by face recognition in the live view image. It is determined whether or not at least one of the person's face and facial parts has been detected, and guidance according to the recognition result by the recognition unit is output.

  A photo creating game machine according to a second aspect of the present technology includes a shooting control unit that shoots one or more users as subjects, and a display control unit that displays a live view image of the users before shooting timing. A face size determining unit that determines whether or not a size of the user's face reflected in the live view image is smaller than a predetermined size, and a plurality of the user's faces reflected in the live view image are separated from each other by a predetermined distance. A distance determination unit that determines whether or not the face is smaller than the predetermined size, the first guidance for the user is output, and the faces are separated from the predetermined distance. An output unit that outputs a second guidance for the user.

  The imaging method according to the second aspect of the present technology captures one or more users as subjects, displays a live view image of the user before the imaging timing, and displays the live view image on the user. It is determined whether or not the size of the face of the user is smaller than a predetermined size, and it is determined whether or not the faces of the plurality of users shown in the live view image are separated from each other by a predetermined distance. A step of outputting a first guidance for the user if determined to be smaller than the size, and a step of outputting a second guidance for the user if it is determined that the faces are separated from the predetermined distance; including.

  In the second aspect of the present technology, one or more users are photographed as subjects, a live view image showing the user is displayed before the photographing timing, and the user's face shown in the live view image is displayed. It is determined whether or not the size of the user is smaller than a predetermined size, it is determined whether or not the faces of the plurality of users shown in the live view image are separated from each other by a predetermined distance, and the size of the face is larger than the predetermined size When it is determined that the distance is smaller, the first guidance for the user is output, and when it is determined that the faces are separated from the predetermined distance, the second guidance for the user is output.

  According to the present technology, it is possible to provide an image with higher satisfaction for the user.

It is a perspective view which shows the structural example of the external appearance of the photo sticker production apparatus of this technique. It is a figure explaining a user's movement. It is a perspective view which shows the structural example of an imaging | photography part. It is a front view which shows the structural example of an imaging | photography part. It is a figure which shows the structural example of a background part. It is a figure which shows the structural example of an edit unit. It is a figure which shows the structural example of the right side surface of an edit unit. It is a figure which shows the example of a sticker paper. It is a figure which shows the example of the layer structure of an edit image. It is a block diagram which shows the example of an internal structure of a photograph sticker production apparatus. It is a block diagram which shows the function structural example of the control part of FIG. It is a block diagram which shows the structural example of the imaging | photography process part of FIG. It is a block diagram which shows the structural example of the edit process part of FIG. It is a flowchart explaining a photograph sticker creation game process. It is a flowchart explaining an imaging | photography process. It is a figure which shows the example of the live view display screen at the time of retouch imaging | photography. It is a figure which shows the example of the imaging | photography result confirmation screen at the time of retouch imaging | photography. It is a flowchart explaining a live view image display process. It is a figure which shows the example of the live view display screen at the time of up imaging | photography. It is a figure which shows the example of the live view display screen at the time of up imaging | photography. It is a figure which shows the example of the live view display screen at the time of up imaging | photography. It is a figure which shows the example of an imaging | photography result confirmation screen. It is a figure which shows the example of the live view display screen at the time of whole body imaging | photography. It is a figure which shows the example of the re-photographing object selection screen. It is a figure which shows the example of the selection screen after a reshoot. It is a flowchart explaining an edit process. It is a figure which shows the example of a feeling of fullness selection screen. It is a figure which shows the example of a face color selection screen. It is a figure which shows the example of a makeup feeling selection screen. It is a figure which shows the example of a tear bag selection screen. It is a figure which shows the example of a feeling of fullness selection screen. It is a figure which shows the example of an edit screen. It is a block diagram which shows the other structural example of the imaging | photography process part of FIG. It is a flowchart explaining the live view image display process at the time of up imaging | photography or whole body imaging | photography from the top. It is a figure explaining determination of the size of a face. It is a figure which shows the example of the guidance which prompts a user to approach a camera. It is a figure explaining determination of the distance between faces. It is a figure which shows the example of the guidance which prompts a user to mutually approach. It is a figure which shows the example of OK guidance. It is a flowchart explaining the live view image display process at the time of front whole body imaging | photography. It is a flowchart explaining the live view image display process at the time of up photography in a course for large numbers of people or whole body photography from above. It is a figure explaining determination of the distance of three people's faces. It is a figure explaining determination of the position of a face.

<External structure of photo sticker creation device>
FIG. 1 is a perspective view showing a configuration example of the appearance of the photographic sticker creating apparatus 1.

  The photo sticker creating apparatus 1 is a game machine that provides a photographed image or an edited image. The photo sticker creating apparatus 1 provides an image to the user by printing the image on a sticker sheet or transmitting the image to a server so that the image can be viewed on the user's mobile terminal. .

  The photo sticker creating apparatus 1 is installed in an amusement facility or store. The users of the photo sticker creating apparatus 1 are mainly young women such as high school girls and female university students. In the photo sticker creating apparatus 1, a plurality of users such as two or three people can mainly enjoy the game per group. Of course, one user can enjoy the game.

  In the photo sticker creating apparatus 1, the user performs photographing work with himself / herself as a subject. The user performs an editing operation to synthesize a synthesis image that is a background image or a foreground image (such as a handwritten character or a stamp image) with a photographed image obtained by photographing. As a result, the captured image is edited into a rich image. The user receives a sticker sheet on which an edited image that is an edited image is printed, and ends a series of games.

  The photo sticker creating apparatus 1 is configured by being installed in a state where the photographing unit 11 and the editing unit 12 are in contact with each other. The photographing unit 11 and the editing unit 12 are electrically connected. The photographing unit 11 has a box-shaped housing having a size that allows a person to enter inside.

  The photographing unit 11 includes a photographing unit 21 and a background unit 22. The imaging unit 21 and the background unit 22 are installed separately. A space provided between the imaging unit 21 and the background unit 22 is an imaging space in which a user performs an imaging operation.

  The photographing unit 21 is a device that photographs a user as a subject. A camera or the like is provided in front of the photographing unit 21 facing the photographing space. A front panel 41 is provided on the surface of the imaging unit 21 that contacts the editing unit 12. When the right side when viewed from the user in the imaging space is the right side and the left side is the left side, the right side of the imaging unit 21 is constituted by the side panel 42A, and the left side is constituted by the side panel 42B (FIG. 2). Composed.

  The background portion 22 includes a back panel 51, a side panel 52A, and a side panel 52B (FIG. 2), which are plate-like members. The back panel 51 is located on the back side of the user facing the front. The side panel 52A is attached to the right end of the back panel 51, and the side panel 52B is attached to the left end of the back panel 51.

  The side panel 42A and the side panel 52A are provided on substantially the same plane with a predetermined interval. The upper portions of the side panel 42A and the side panel 52A are connected by a connecting portion 31A that is a plate-like member. Further, the lower portions of the side panel 42A and the side panel 52A are connected by a connecting portion 34 that is a metal member provided on the floor surface.

  Similarly, the side panel 42B and the side panel 52B are provided on substantially the same plane. The upper portions of the side panel 42B and the side panel 52B are connected by a connecting portion 31B. The lower portions of the side panel 42B and the side panel 52B are connected by a connecting portion (not shown) that is a metal member provided on the floor surface.

  An opening formed by being surrounded by the side panel 42A, the connecting portion 31A, and the side panel 52A serves as an entrance / exit of the imaging space. An opening formed by being surrounded by the side panel 42B, the connecting portion 31B, and the side panel 52B also serves as an entrance / exit of the imaging space.

  An opening surrounded by the front panel 41 of the photographing unit 21, the connecting part 31A, the connecting part 31B, and the back panel 51 of the background part 22 is formed above the photographing space. A ceiling lighting unit 32 is provided so as to cover a part of the opening. One end of the ceiling lighting unit 32 is fixed to the connecting portion 31A, and the other end is fixed to the connecting portion 31B.

  A fluorescent lamp is provided inside the ceiling lighting unit 32. The ceiling lighting unit 32 functions as illumination of the shooting space and irradiates the user who is shooting with strobe light.

  The editing unit 12 is a device for editing a captured image. The editing unit 12 is connected to the photographing unit 11 such that one side surface is in contact with the front panel 41 of the photographing unit 21.

  If the configuration of the editing unit 12 shown in FIG. 1 is a front side configuration, a configuration used for editing work is provided on each of the front side and the back side of the editing unit 12. With this configuration, two sets of users can edit simultaneously.

  The front side of the editing unit 12 includes a surface 12A that is a surface perpendicular to the floor surface, and a slope 12B formed above the surface 12A. The slope 12B is provided with a tablet built-in monitor and a touch pen as a configuration used for editing work.

  A columnar support 71A (FIG. 6) that supports one end of the lighting device 73 is provided on the upper right side of the slope 12B. A columnar support 71B that supports the other end of the lighting device 73 is provided on the left side of the inclined surface 12B. A plate-like panel 72 is provided on the right side of the support portion 71A. A support portion 74 that supports the curtain rail 75 is provided on the upper surface of the panel 72.

  A curtain rail 75 is attached above the editing unit 12. The curtain rail 75 is configured by combining three rails 75A to 75C. The three rails 75A to 75C are combined so that the shape when viewed from above becomes a U-shape. One end of the rail 75A and the rail 75B provided in parallel is fixed to the connecting portion 31A and the connecting portion 31B, respectively, and the other end of the rail 75A and the rail 75B is joined to both ends of the rail 75C.

  A curtain is attached to the curtain rail 75 so that the space in front of the editing unit 12 and the space in front of the back are not visible from the outside. A space in front of the editing unit 12 surrounded by the curtain and a space in the rear rear of the editing unit 12 are editing spaces in which the user performs editing operations.

  As will be described later, the right side surface of the editing unit 12 is provided with a discharge port through which printed sticker paper is discharged. The space in front of the right side surface of the editing unit 12 is a print waiting space where the user waits for the printing to end.

<About user movement>
Here, the flow of the photo sticker creation game and the accompanying movement of the user will be described.

  FIG. 2 is a plan view of the photo sticker creating apparatus 1 as viewed from above.

  First, the user takes a photographing space formed between the photographing part 21 and the background part 22 from the entrance / exit G1 as indicated by the white arrow # 1 or from the entrance / exit G2 as indicated by the white arrow # 2. Enter A1. The doorway G1 is an opening between the side panel 42A and the side panel 52A, and the doorway G2 is an opening between the side panel 42B and the side panel 52B. The user performs a shooting operation using a camera or a touch panel monitor provided in the shooting unit 21 in the shooting space A1.

  The user who has finished the shooting work exits the shooting space A1 from the doorway G1 as indicated by the white arrow # 3 and moves to the editing space A2-1, or the doorway G2 as indicated by the white arrow # 4. Exits the shooting space A1 and moves to the editing space A2-2.

  The editing space A2-1 is an editing space on the front side of the editing unit 12. On the other hand, the editing space A2-2 is an editing space on the back side of the editing unit 12. Whether the user moves to the editing space A2-1 or the editing space A2-2 is guided by the screen display of the touch panel monitor of the photographing unit 21 or the like. For example, the free one of the two editing spaces is guided as the destination.

  The user who has moved to the editing space A2-1 or the editing space A2-2 starts editing work. The user of the editing space A2-1 and the user of the editing space A2-2 can perform editing work at the same time.

  After the editing operation is finished, printing of the edited image is started. When printing is started, the user who has finished the editing work in the editing space A2-1 moves from the editing space A2-1 to the print waiting space A3 as indicated by a white arrow # 5. Further, the user who has finished the editing work in the editing space A2-2 moves from the editing space A2-2 to the print waiting space A3 as indicated by the white arrow # 6. The user who has moved to the print waiting space A3 waits for the printing to end.

  When printing is completed, the user receives the sticker paper discharged from the discharge port provided on the right side of the editing unit 12 and finishes a series of photo sticker creation games.

  Next, the configuration of each unit will be described.

<Configuration of the shooting unit>
FIG. 3 is a perspective view illustrating a configuration example of the imaging unit 21, and FIG. 4 is a front view illustrating a configuration example of the imaging unit 21.

  The outer frame of the imaging unit 21 is configured by standing a front panel 41, a side panel 42A, and a side panel 42B on a base unit 43 to which a plurality of casters are attached.

  A substantially box-shaped photographing / display unit 101 is provided in the approximate center of the front panel 41. On the front side of the photographing / display unit 101, a slope 101A that is forwardly lowered, a slope 101B that is slightly inclined forward with respect to the vertical, and a slope 101C that is steeper than the slope 101A are formed in order from the top.

  A circular opening is formed in the approximate center of the inclined surface 101B, and a lens of the camera 111 that photographs a user in the photographing space A1 is provided in the opening. A camera 111, which is a single-lens reflex camera, for example, is attached to the inside of the photographing / display unit 101 so as to be tiltable.

  A touch panel monitor 112 is provided on the slope 101C. The touch panel monitor 112 includes a monitor such as an LCD (Liquid Crystal Display) and a touch panel laminated thereon.

  The touch panel monitor 112 displays a moving image (hereinafter also referred to as a live view image) captured by the camera 111, and displays various GUIs (Graphical User Interfaces) for user operations. It has a function to accept. On the touch panel monitor 112, live view images and captured images captured by the camera 111 are displayed. The captured image is a still image captured by the camera 111 at a predetermined timing.

  As will be described in detail later, the user can shoot a full-body image that is an image of the entire body of the user and an up image that is an image in which the range above the approximate chest is large with the face portion at the center. it can.

  On the front panel 41, a light emitting portion 102L and a light emitting portion 102R are formed symmetrically with the photographing / display unit 101 in between. The light emitting unit 102L and the light emitting unit 102R are configured, for example, by fitting a light-transmitting cover into a narrow groove formed in the front panel 41. A light emitting body such as an LED is provided in the narrow groove formed in the front panel 41.

  As shown in FIG. 4, the left light emitting unit 102L of the photographing / display unit 101 includes narrow linear portions 102L-1 to 102L-3. The linear portion 102L-2 is formed in the vertical direction so that its length is substantially the same as the length in the vertical direction of the photographing / display unit 101. The linear portion 102L-1 is formed obliquely upward and to the right from the upper end of the linear portion 102L-2, and the linear portion 102L-3 is formed obliquely and downward to the right from the lower end of the linear portion 102L-1. Is done.

  On the other hand, the light emitting unit 102R on the right side of the photographing / display unit 101 includes narrow linear portions 102R-1 to 102R-3. The linear portion 102R-2 is formed in the vertical direction so that its length is substantially the same as the length in the vertical direction of the photographing / display unit 101. The linear portion 102R-1 is formed from the upper end of the linear portion 102R-2 toward the upper left diagonal direction, and the linear portion 102R-3 is formed from the lower end of the linear portion 102R-2 toward the lower left diagonal direction. Is done.

  As shown in FIG. 4, the combined shape of the light emitting part 102L and the light emitting part 102R is a substantially hexagonal shape. By emitting light from the light emitting unit 102L and the light emitting unit 102R, it is possible to produce a sharp impression as an impression of the shooting space A1.

  Above the photographing / display unit 101, an upper strobe unit 103 with a curved cover facing the user is installed. The upper strobe unit 103 is fixed to the front panel 41 or the like using a predetermined mounting member. The upper strobe unit 103 irradiates strobe light near the user's face from the front upper side.

  Below the photographing / display unit 101, a foot strobe unit 104 that irradiates the lower body of the user with strobe light in accordance with photographing by the camera 111 is provided. The front surface of the foot strobe unit 104 is constituted by a plate-like member 121 that forms a vertical surface, and the upper surface is constituted by a plate-like member 122 that forms a gentle slope. The plate-like member 121 and the plate-like member 122 are plate-like members having translucency such as milky white acrylic plates.

  On the left and right of the plate-like member 121 of the foot strobe unit 104, a plate-like member 123L and a plate-like member 123R that constitute a vertical surface are provided. The plate-like member 123R is provided with a coin insertion slot through which a user inserts coins.

  On the left and right sides of the plate-like member 122 of the foot strobe unit 104, a plate-like member 124L and a plate-like member 124R that form a gentle slope similar to the plate-like member 122 are provided. The slope formed by the plate-like member 124L and the plate-like member 124R is used for a user to put baggage.

  Although not illustrated, a speaker is provided near the ceiling of the front panel 41, for example. The speaker outputs guidance voice, BGM, sound effects, etc. regarding the shooting work.

<Configuration of background part>
FIG. 5 is a diagram illustrating a configuration example of the background portion 22 on the photographing space A1 side.

  A rear upper strobe 131 is provided at the upper center of the rear panel 51. A rear right strobe 132 is provided on the upper right side (left side in the figure) of the rear panel 51, and a rear left strobe 133 is provided on the upper left side (right side in the figure) of the rear panel 51.

  The rear upper strobe 131 irradiates the user with strobe light from the upper rear side. The rear right strobe 132 irradiates the user with strobe light from the rear right side. The rear left strobe 133 irradiates the user with strobe light from the rear left side.

  The surface on the imaging space A1 side (the front side in the drawing) of the back panel 51 is, for example, a green surface. Although not shown, the surfaces on the photographing space A1 side of the side panels 52A and 52B are also green surfaces, for example.

  The green color of these surfaces is a color for extracting the user's area by chroma processing. Thereby, chroma processing is performed on the captured image, and a background image desired by the user can be combined with the background portion of the captured image. In addition, the color of these surfaces should just be a color which can perform a chroma process, and other colors, such as blue, may be sufficient as it.

<Configuration of editing unit>
FIG. 6 is a diagram illustrating a configuration example of the editing unit 12 on the front side (editing space A2-1 side).

  A tablet built-in monitor 141 is provided substantially at the center of the slope 12B. A touch pen 142A and a touch pen 142B are provided on the left and right of the tablet built-in monitor 141.

  The tablet built-in monitor 141 accepts an operation input using the touch pens 142A and 142B. The tablet built-in monitor 141 displays an editing screen used for editing work. For example, the touch pen 142A is used by a user standing on the left side toward the tablet built-in monitor 141, and the touch pen 142B is used by a user standing on the right side toward the tablet built-in monitor 141. Hereinafter, when it is not necessary to distinguish between the touch pen 142A and the touch pen 142B as appropriate, it is referred to as a touch pen 142.

  FIG. 7 is a diagram illustrating a configuration example of the right side surface of the editing unit 12.

  A seal paper discharge port 151 is provided at the lower part of the right side surface of the editing unit 12. A printer is provided inside the editing unit 12. An image edited by the user of the editing space A2-1 or an image edited by the user of the editing space A2-2 is printed on the sticker paper with a predetermined layout by the printer, and the sticker paper discharge port 151 is discharged.

<Example of sticker paper>
FIG. 8 is a diagram showing an example of a sticker sheet created by the photographic sticker creator 1 having the above-described external configuration.

  On the horizontally long sticker sheet shown in FIG. 8, five edited images 161-1 to 161-5 are printed side by side.

  Of the edited images 161-1 to 161-5, the edited images 161-1 to 161-3 are images generated by editing the up image, and the edited image 161-4 and the edited image 161-5 is an image generated by editing the whole body image.

  The up image and the whole body image have different aspect ratios. For example, the ratio between the horizontal length and the vertical length of the up image is 1: 1.2, and the ratio between the horizontal length and the vertical length of the whole-body image is 3: 4. In the example of FIG. 8, the up image and the whole body image are printed with the same length in the horizontal direction. In practice, a background image, which is an image of a predetermined color or pattern, is synthesized on the background of the user, and a foreground image such as a handwritten character or a stamp image is synthesized on the foreground, respectively. .

  In addition to the model name (LOVERY) and shooting date, an e-mail address, ID, etc. are printed in the margin of the sticker paper. The mail address and ID printed in the blank area are used when viewing an image transmitted from the photo sticker creating apparatus 1 to the server on a portable terminal such as a cellular phone.

  FIG. 9 is a diagram illustrating an example of a layer structure of an edited image.

  The leftmost edited image P1 is an image generated by editing. The edited image P1 is composed of images of each layer of a background image P11, a captured image P12, and a foreground image P13.

  In the example of FIG. 9, the background image P <b> 11 is an image of a wall pattern on which bricks are stacked. The foreground image P13 is an image including a handwritten character image of “Love my friend” and a heart-shaped image. The handwritten character image of “Love my friend” is a pen image input by the user using a pen tool at a position below the center of the captured image P12. The heart-shaped image is a stamp image input by the user using a stamp tool at the upper left position of the captured image P12.

  In this way, the edited image P1 is generated by superimposing the subject area of the photographed image P12 extracted by the chroma process on the background image P11 and superimposing the foreground image P13 thereon. It is also possible to input a pen image or a stamp image on the background image P11 and display it on the background of the subject.

<Internal configuration of photo sticker creation device>
FIG. 10 is a block diagram illustrating an internal configuration example of the photo sticker creating apparatus 1. The same code | symbol is attached | subjected to the same structure as the structure mentioned above. The overlapping description will be omitted as appropriate.

  The control unit 201 is configured by a CPU (Central Processing Unit) or the like. The control unit 201 executes a program stored in a ROM (Read Only Memory) 206 or a storage unit 202 and controls the overall operation of the photo sticker creating apparatus 1. A storage unit 202, a communication unit 203, a drive 204, a ROM 206, and a RAM (Random Access Memory) 207 are connected to the control unit 201. Also connected to the control unit 201 are each configuration of the photographing unit 208, each configuration of the editing units 209A and 209B, and a printer 241.

  The storage unit 202 is a non-volatile storage medium such as a hard disk or a flash memory. The storage unit 202 stores various setting information supplied from the control unit 201. Information stored in the storage unit 202 is appropriately read out by the control unit 201.

  The communication unit 203 is a network interface such as the Internet. The communication unit 203 communicates with an external device according to control by the control unit 201. For example, the communication unit 203 transmits the captured image selected by the user to the server. The image transmitted from the communication unit 203 is allocated and stored in a predetermined storage area in the server, and is displayed or downloaded on a mobile terminal that has accessed the server.

  A removable medium 205 made of an optical disk, a semiconductor memory, or the like is appropriately attached to the drive 204. The programs and data read from the removable medium 205 by the drive 204 are supplied to the control unit 201 and stored in the storage unit 202 or installed.

  The ROM 206 stores programs and data executed by the control unit 201. The RAM 207 temporarily stores data and programs processed by the control unit 201.

  The photographing unit 208 realizes photographing processing for a user in the photographing space A1. The imaging unit 208 includes a coin processing unit 221, an illumination device 222, a camera 111, a touch panel monitor 112, and a speaker 223.

  The coin processing unit 221 detects the insertion of coins into the coin insertion slot. When the coin processing unit 221 detects that a predetermined amount of coins such as 400 yen has been inserted, the coin processing unit 221 outputs an activation signal instructing the start of the game to the control unit 201.

  The illumination device 222 is each illumination device provided in the imaging space A1, such as the upper strobe unit 103 and the foot strobe unit 104. Each of the illumination devices 222 emits light according to an illumination control signal supplied from the control unit 201.

  The camera 111 shoots according to the shutter control by the control unit 201, and outputs captured image data obtained by the shooting to the control unit 201.

  The editing unit 209A realizes editing processing for a user in the editing space A2-1. The editing unit 209A includes a tablet built-in monitor 141, touch pens 142A and 142B, and a speaker 231. The editing unit 209B has the same configuration as the editing unit 209A, and realizes editing processing for a user in the editing space A2-2.

  The tablet built-in monitor 141 displays various screens under the control of the control unit 201 and accepts user operations on the displayed screens. An input signal representing the content of the user's operation is supplied to the control unit 201.

  The printer 241 prints the edited image on the sticker paper stored in the sticker paper unit 242 based on the print data supplied from the control unit 201. The printer 241 discharges the sticker paper on which the edited image is printed with a predetermined layout to the sticker paper discharge port 151.

<Configuration of control unit>
FIG. 11 is a block diagram illustrating a functional configuration example of the control unit 201. At least a part of the functional units shown in FIG. 11 is realized by a predetermined program being executed by the CPU in the control unit 201.

  The control unit 201 includes an imaging processing unit 301, an editing processing unit 302, and a print processing unit 303.

  The imaging processing unit 301 controls each part of the imaging unit 208 to perform imaging processing. The editing processing unit 302 performs editing processing by controlling each of the editing units 209A and 209B. The print processing unit 303 controls the printer 241 to print the edited image on a sticker sheet.

<Configuration example of imaging processing unit>
FIG. 12 is a block diagram illustrating a configuration example of the imaging processing unit 301.

  The imaging processing unit 301 includes a display control unit 311, an audio output control unit 312, an imaging control unit 313, and a recognition unit 314.

  The display control unit 311 controls display on the touch panel monitor 112. For example, the display control unit 311 causes the touch panel monitor 112 to display a live view image captured by the camera 111 or to display a captured image that is a captured result.

  The audio output control unit 312 controls the output of audio from the speaker 223. For example, the audio output control unit 312 causes the speaker 223 to output audio that explains how to proceed with the shooting operation.

  The imaging control unit 313 controls the camera 111 to image a user as a subject and generate a live view image and a captured image.

  The recognition unit 314 determines whether or not at least one of the user's face and face parts (for example, facial organs such as eyes and mouth) is detected in the live view image by face recognition.

<Configuration example of edit processing unit>
FIG. 13 is a block diagram illustrating a configuration example of the editing processing unit 302.

  The edit processing unit 302 includes a display control unit 321, an edit control unit 322, an authentication unit 323, and an image processing unit 324.

  The display control unit 321 controls display on the tablet built-in monitor 141. For example, the display control unit 321 causes the tablet built-in monitor 141 to display a selection screen for selecting the content of image processing performed on the captured image and an editing screen for editing work on the captured image.

  The editing control unit 322 edits the captured image in accordance with a user operation on the editing screen. Specifically, the editing control unit 322 synthesizes the synthesis image selected by the user on the editing screen at a predetermined position of the captured image.

  The authentication unit 323 authenticates the user's face that appears in the photographed image based on the retouch image obtained by the retouch photographing performed by the photographing processing unit 301. Details of the retouch shooting will be described later.

  The image processing unit 324 performs image processing of the selected degree on the captured image based on a selection operation for a selection screen displayed on the tablet built-in monitor 141 for selecting the degree of image processing performed on the captured image. Apply.

<Operation of photo sticker creation device>
Here, with reference to the flowchart of FIG. 14, operation | movement of the photo sticker production apparatus 1 which provides a photo sticker production game is demonstrated. The process of FIG. 14 is started when a predetermined amount of coins is inserted into the coin slot.

  In step S1, the imaging processing unit 301 performs imaging processing. In the photographing process, up photographing and whole body photographing are performed.

  The up-shooting is a shooting for obtaining an up-image that is an image in which the upper range (upper body) of the user is seen from the chest around the face. Whole body photography is photography for obtaining a whole body image that is an image of the whole body of a user. In this embodiment, it is assumed that an image showing the whole body of the user viewed obliquely from above is obtained by whole body photography, but an image showing the whole body of the user viewed from the front may be obtained.

  In step S2, the editing processing unit 302 performs editing processing. In the editing process, the captured image is edited.

  In step S3, the print processing unit 303 executes print processing by controlling the printer 241, and prints the edited image obtained by the edit processing on sticker paper. Note that, when editing is not performed on the captured image obtained by the capturing process, the captured image is printed as it is on the sticker sheet.

  In step S4, the printer 241 discharges the sticker sheet on which the edited image is printed to the sticker paper discharge port 151, and ends the process.

<Shooting process>
Next, the photographing process performed in step S1 of FIG. 14 will be described with reference to the flowchart of FIG.

  When the photographing process is started, the display control unit 311 displays a course selection screen on the touch panel monitor 112 in step S31.

  The course selection screen is a screen used for selection of a course for the number of people to determine the type of shooting performed according to the number of users. The number of people course includes a course for two people where shooting is suitable for two users, and a course for a large number of people where shooting is suitable for three or more users, particularly when there are four or more users. There is. When the two-person course is selected and when the large-number course is selected, the shooting range (angle of view) and size of the obtained shot image are different. The photographed image obtained in the course for large numbers of people is larger than the photographed image obtained in the course for two persons.

  In the following, the processing when the two-person course is selected will be mainly described.

  When any number of people course is selected on the number of people course selection screen, a screen explaining how to perform retouch shooting is displayed on the touch panel monitor 112, and guidance is output. Thereafter, in step S32, the imaging control unit 313 performs retouch imaging for authenticating the user's face.

  Retouch photography is photography performed in order to authenticate a person, specifically, a user's face. Authentication images (hereinafter referred to as retouching images) obtained by retouch photography are not subject to editing in the editing process or printed on sticker paper in the printing process, and are used only for face authentication. Is done.

  For this purpose, in the guidance for retouch shooting, a screen is displayed to prompt the user to take a picture while facing the front, and voice is output.

  When the guidance ends, the shooting control unit 313 starts shooting a moving image by the camera 111, and the moving image obtained by shooting is displayed on the live view display screen.

  FIG. 16 is a diagram illustrating an example of a live view display screen for retouch photography.

  At the top of the Live View display screen, the message “Register each face first! Shoot inside the frame and look forward!” Is displayed, and the image display provided below In the areas 411 and 412, moving images showing two users are displayed in real time. Also, a message “Registered image will not be printed on the sticker!” Is displayed below the image display areas 411 and 412.

  A predetermined range is cut out from the moving image photographed by the camera 111 and used to display the image display areas 411 and 412. The two users adjust the positions of the faces so that the respective faces fit in the image display areas 411 and 412 while confirming the display of the image display areas 411 and 412. Thereby, the accuracy of face authentication by the image for retouching can be ensured.

  After such a live view image is displayed for a predetermined time, a countdown before shooting starts and when shooting timing comes, the shooting control unit 313 performs shooting to acquire a still image as a retouching image. The photographing result of the retouch photographing is displayed on the touch panel monitor 112.

  FIG. 17 is a diagram illustrating an example of a shooting result confirmation screen of retouch shooting.

  In the image display areas 411 and 412 of the shooting result confirmation screen illustrated in FIG. 17, face images of the faces of the two users are displayed as still images (retouch images) obtained by retouch shooting. A message “Registration complete!” Is displayed below the image display areas 411 and 412. The face images of each of the two users are stored in the storage unit 202, and are used for authentication of the user's face shown in the photographed image obtained by five photographings.

  Retouch shooting is performed as described above.

  When a course for a large number of people is selected as the number of people course, the live view display screen and the shooting result confirmation screen for retouch shooting are replaced with a plurality of image display areas 411 and 412 in which two users are respectively shown. It is assumed that only one image display area where the user is shown together is provided.

  When the retouch shooting is completed, a guidance explaining the shooting method is displayed on the touch panel monitor 112, and the guidance is output. Thereafter, a live view display screen is displayed on the touch panel monitor 112.

  When it is time to shoot in this state, in step S33, the shooting control unit 313 performs the first shooting. A photographed image obtained by photographing is displayed on the touch panel monitor 112 as a photographing result. The user can prepare for the next shooting while confirming the first shooting result.

  Then, the second and subsequent shootings are performed. That is, when the second shooting timing is reached, in step S34, the shooting control unit 313 performs the second shooting.

  Similarly, in steps S35 to S37, the imaging control unit 313 performs the third to fifth imaging, respectively.

  Of the first to fifth shootings, up shooting is performed as the first to third shootings, and whole body shooting is performed as the fourth and fifth shootings.

  In each of up-shooting and whole-body shooting, the tilt angle of the camera 111 is controlled by the shooting control unit 313.

  Specifically, assuming that the tilt angle of the camera 111 when the whole body shooting is performed (the state where the optical axis of the lens of the camera 111 is in the horizontal direction) is the default state, the camera 111 is the default when the up shooting is performed. The tilt angle is adjusted so as to tilt forward by a predetermined angle from the above state. Note that, instead of controlling the hardware configuration such as adjusting the tilt angle, an image obtained by adjusting the tilt angle by software (image processing) may be generated.

  In addition, when each of up-shooting and whole-body shooting is performed, not only the tilt angle is controlled, but also the standing position of the user is changed by the guidance output by the audio output control unit 312.

  For example, a label indicating two standing positions, a standing position closest to the camera 111 and a standing position farthest from the camera 111, is attached to the floor surface of the imaging space. When up-shooting is performed, guidance for instructing the user to stand at the closest position from the camera 111 is output. In addition, when whole body imaging is performed, guidance for instructing the user to stand at a position farthest from the camera 111 is output.

  In this way, after the five shootings are completed in steps S33 to S37, the display control unit 311 causes the touch panel monitor 112 to display a re-shooting target selection screen in step S38.

  The re-shooting target selection screen is a screen used to select a re-shooting target image that is a photographic image to be re-taken from among five shot images. On the re-shooting target selection screen, five shot images are displayed, and any one of them is selected, and the re-shooting target image is selected and determined. In addition, a button for selecting not to perform re-shooting is also displayed on the re-shooting target selection screen. By selecting the button, re-shooting is not performed.

  In step S39, it is determined whether or not to perform re-taking according to which of the five captured images and a button for selecting not to perform re-taking is selected.

  If any of the five captured images is selected on the re-shooting target selection screen, it is determined that re-shooting is performed, and the process proceeds to step S40.

  In step S40, the imaging control unit 313 performs imaging (re-imaging).

  After the re-taking is performed, in step S41, the display control unit 311 displays the selection screen after the re-taking on the touch panel monitor 112.

  The selection screen after reshooting is a screen used for selecting either the reshooting target image selected on the reshooting target selection screen or the captured image obtained by reshooting. The re-shooting selection screen displays the re-shooting target image selected on the re-shooting target selection screen and the shot image obtained by the re-shooting, and one of them is selected for final editing. Or an image to be printed is selected and determined.

  When either the recapture target image selected on the recapture target selection screen or the captured image obtained by the recapture is selected on the post-recap selection screen, the process proceeds to step S42.

  On the other hand, if a button for selecting not to retake is selected on the retake target selection screen, it is determined that retake is not performed, steps S40 and S41 are skipped, and the process proceeds to step S42. .

  In step S42, the imaging processing unit 301 guides the user in the imaging space A1 to move to the editing space A2-1 or the editing space A2-2. The guidance for moving to the editing space A2-1 or the editing space A2-2 is performed by displaying a guidance screen on the touch panel monitor 92 of the photographing unit 209 or by outputting sound from the speaker 232.

  The photographing process is performed as described above.

  By the way, on the live view display screen displayed before each shooting timing of the first to fifth shootings (hereinafter referred to as normal shooting) in the shooting process described above, the face and face parts in the live view image are successfully recognized. The guidance display for this is performed.

<Live view image display processing>
Here, with reference to FIG. 18, a live view image display process executed at the time of normal photographing will be described.

  The captured image obtained by successfully recognizing the face and the facial parts is more appropriately subjected to appropriate image processing. Here, in the whole body image, the size of the face with respect to the entire image is smaller than the up image, so even if image processing is performed on the face portion of the whole body image, the user confirms the finish. It's not easy.

  However, in recent years, users have used images taken with a photo sticker on a mobile terminal such as a mobile phone. Some users trim and use the face portion of the whole body image on the mobile terminal. For this reason, it is necessary to perform appropriate image processing not only on the up image but also on the face portion of the whole body image.

  Therefore, it is assumed that the processing in FIG. 18 is executed when either up-shooting or whole-body shooting is performed. Note that the processing in FIG. 18 is not executed when retouch shooting is performed.

  In step S <b> 51, the shooting control unit 313 controls the camera 111 to capture a live view image that is a moving image showing the user from the camera 111. At this time, the display control unit 311 causes the touch panel monitor 112 to display a live view display screen for displaying a live view image.

  In step S52, the display control unit 311 displays the live view image captured from the camera 111 in a predetermined area on the live view display screen displayed on the touch panel monitor 112.

  In step S <b> 53, the recognizing unit 314 determines whether or not the user's face and eyes are detected by face recognition in the live view image captured from the camera 111.

  Here, details of face recognition processing by the recognition unit 314 will be described.

  Specifically, the recognition unit 314 performs face recognition by extracting the contour point of the eye region in the live view image and determining the shape of the region surrounded by the extracted contour point, and the face and eyes are detected. It is determined whether or not. The number and position of the extracted contour points are arbitrary. In this case, for example, it is determined that the recognition of the user who closed the eyes failed and the eyes were not detected. Accordingly, it is possible to prevent the image processing for the eye region, which will be described later, from being unnatural due to being applied to the closed eyes. In the live view image, in addition to the contour point of the eye region, the contour point of the face region may be further extracted, or only the contour point of the face region may be extracted.

  The recognition unit 314 can also perform face recognition by acquiring color information of an area surrounded by contour points extracted from the live view image, and determine whether a face and eyes are detected. For example, if the contour point of the face area is extracted, and the color information of the area surrounded by the extracted contour point of the face area is determined that the color of the area is not a skin color, the face recognition fails. It is determined that no face has been detected. In addition, when it is determined from the color information of the area surrounded by the outline point of the eye area that the color of the area includes colors other than white and black, the face recognition fails and the eyes are not detected. Determined.

  In the present embodiment, as will be described later, it is assumed that image processing for the face area and the eye area is performed on a captured image obtained by successful face recognition in the live view image. However, when only image processing is performed on the face area in the captured image, or only image processing is performed on the eye area, only the face is detected or only the eyes are detected in the live view image. To be.

  Furthermore, the recognition unit 314 may determine whether or not the face recognition is successful based on the face inclination. Even when image processing is performed on a face that is tilted by a predetermined angle or more in the up-down direction or the left-right direction in the captured image, there is a high possibility that the user will not notice it. Therefore, for example, when it is determined that the face is inclined at a predetermined angle or more in the vertical direction or the horizontal direction, the face recognition is determined to have failed. Thereby, unnecessary image processing can be avoided from being performed on the captured image.

  Returning to the description of the flowchart of FIG. 18, in step S54, the display control unit 311 responds to the recognition result (whether the user's face or eyes are detected) on the live view display screen. Display guidance.

  In step S55, the display control unit 311 determines whether or not the shooting timing has come. If it is determined that it is not the shooting timing, the process returns to step S51, and the display of the live view image and the guidance display according to the recognition result are repeated.

  When it is determined that the shooting timing has come, the live view image display process ends, and shooting by the shooting control unit 313 is performed.

  FIG. 19 is a diagram illustrating an example of a live view display screen during up-shooting.

  An image display area 431 is provided in the approximate center of the live view display screen. In the image display area 431, a live view image that is a moving image captured by the camera 111 is displayed. Further, a message “Pause from the camera!” Is displayed below the image display area 431.

  On the lower side of the live view display screen, five thumbnail image display areas 432 to 436 are displayed side by side in the horizontal direction. In the thumbnail image display areas 432 to 436, five model images corresponding to five normal shootings are respectively displayed. Of the five model images displayed in the thumbnail image display areas 432 to 436, the three model images displayed in the thumbnail image display areas 432 to 434 are the up images and are displayed in the thumbnail image display areas 435 and 436. The two model images to be processed are the whole body images.

  Also, among the thumbnail image display areas 432 to 436, the thumbnail image display area in which the model image corresponding to the shooting to be performed is displayed is highlighted with a selection frame. In the example of FIG. 19, a thumbnail image display area 432 in which a model image that the user uses as a sample at the first shooting is displayed is highlighted in a selection frame.

  Here, as described above, face recognition is performed by the recognition unit 314 in the live view image displayed in the image display area 431.

  On the live view display screen, guidance display for prompting the user to succeed in recognition by the recognition unit 314 is performed. Specifically, as shown in FIG. 19, in the live view image displayed in the image display area 431, the portion of the user's face that has been successfully recognized by the recognition unit 314 is displayed as a guidance display. Frames F11 and F12 are displayed. That is, in the example of FIG. 19, the face recognition of each of the two users shown in the live view image has succeeded. As a result, the user reflected in the live view image can grasp that the face is successfully recognized, and tries to maintain the standing position and the face orientation. In the example of FIG. 19, the frame is not displayed when recognition of the user's face shown in the live view image has failed.

  Further, guidance display may be performed for the face of the user who has failed to be recognized by the recognition unit 314. For example, in the example of FIG. 20, among the two users displayed in the live view image displayed in the image display area 431, the left user's face has been successfully recognized and the right user's face has been recognized. The face has failed to recognize.

  In this case, a balloon image B11 including a message indicating that recognition by the recognition unit 314 has failed is displayed near the right user's face shown in the live view image. As a result, the user reflected in the live view image can grasp that the face recognition has failed and try to change the standing position and the face direction so that the face recognition succeeds. .

  As shown in FIG. 21, a balloon image B21 including a message indicating that the recognition by the recognition unit 314 is successful is displayed near the face of the left user who has successfully recognized the face. You may do it.

  FIG. 22 is a diagram illustrating an example of a shooting result confirmation screen.

  In the image display area 431 of the shooting result confirmation screen shown in FIG. 22, a still image showing two users is displayed as a shot image obtained by shooting. Under the image display area 431, a message “I got it like this” is displayed.

  Normal shooting is performed as described above.

  According to the above processing, on the live view display screen, the user can grasp whether or not the face recognition is successful, so that the face recognition failure can be avoided. The orientation and position of the face can be adjusted so as to succeed in recognizing the face while ensuring the degree of freedom of the favorite pose and standing position. As a result, an image that has been appropriately subjected to image processing is output, and it is possible to provide an image that is more satisfactory for the user.

  In addition, when the face or eyes are not detected because the face or eyes are touched by the pose taken by the user, the user can be informed that the face recognition has failed.

  In the above, an example of the live view display screen when up-shooting is performed as normal shooting has been described. However, the above-described live view image display processing is executed also when whole-body shooting is performed.

  FIG. 23 is a diagram illustrating an example of a live view display screen during whole body imaging.

  In the example of FIG. 23, a live view image in which the whole bodies of two users are reflected is displayed in the image display area 431. In the example of FIG. 23, in the live view image displayed in the image display area 431, frames F31 and F32 as guidance displays are provided on the face portion of the user who has been successfully recognized by the recognition unit 314. Is displayed.

  In the example of FIG. 23, as in the example of FIG. 20, a balloon image including a message indicating that the recognition by the recognition unit 314 has failed with respect to the face of the user that has failed recognition by the recognition unit 314. May be displayed, and, similar to the example of FIG. 21, for a user's face that has been successfully recognized by the recognition unit 314, a message that the recognition unit 314 has failed to succeed is included. A balloon image may be displayed.

  In the above, it is assumed that the guidance display prompting the recognition by the recognition unit 314 to be successful is performed on the live view image. However, when shooting has been performed with the recognition failed, a display indicating that the recognition has failed may be displayed on the captured image displayed on the re-shooting target selection screen.

  FIG. 24 is a diagram illustrating an example of a reshooting target selection screen.

  At the top of the re-shooting object selection screen, a message “Please select up to one image you want to retake!” Is displayed, and below that, five shot images 451 to 455 that can be selected as re-shooting targets. And a button 456 operated when re-taking is not performed. In addition, a button 457 that is operated when determining the selected recapture target image is displayed in the lower right corner of the recapture target selection screen.

  When the user selects any of the captured images 451 to 455 and operates the button 457 on the recapture target selection screen, the selected captured image is determined as the recapture target image. On the other hand, when the user operates the button 456, it is determined not to perform re-shooting.

  Also, on the re-shooting target selection screen, among the photographed images 451 to 455, the photographed image obtained by photographing while failing to recognize the face is displayed indicating that the recognition has failed. In the example of FIG. 24, a balloon image B41 including a message indicating that the recognition by the recognition unit 314 has failed is displayed on the captured image 451.

  Accordingly, the user can quickly find a captured image obtained by performing the shooting with the recognition failure, and select the captured image as a recapture target image.

  FIG. 25 is a diagram illustrating an example of a selection screen after reshooting.

  As shown in FIG. 25, a message “Which image do you want to display?” Is displayed above the selection screen after re-shooting, and a photographed image 471 selected as a re-shooting target image and a re-shooting screen. A captured image 472 acquired by photographing is displayed side by side. In addition, a button 473 operated when determining the selection of the photographed image is displayed in the lower right corner of the selection screen after re-taking.

  When the user selects one of the photographed images 471 and 472 and operates the button 473 on the selection screen after re-taking, the selected photographed image is finally selected for editing or printing. Is determined as an image.

  In addition, on the selection screen after re-shooting, the captured image 471 displays a display indicating that the recognition has failed. In the example of FIG. 25, a balloon image B51 including a message indicating that the recognition by the recognition unit 314 has failed is displayed on the captured image 471.

  Thus, the user can prevent the photographed image obtained by photographing with the recognition failure from being finally selected as an image to be edited or printed.

<Example of course for large groups>
In the above, the live view image display processing when the two-person course is selected as the number-of-persons course has been described, but face recognition in the live view image is also performed when the large-number course is selected. .

  In addition, when the course for a large number of people is selected, the face recognition in the live view image is performed with a predetermined number as an upper limit. For example, when the course for a large number of people is selected, the face recognition in the live view image is performed for up to 12 people, and a frame as a guidance display is displayed on the face portion that has been successfully recognized. Therefore, for example, when 13 users are shown in the live view image, face recognition is not performed for any one of the users.

  Here, for all 12 users, when face recognition is successful, 12 frames are displayed in the live view image. In this case, the display of the live view image becomes complicated, It becomes difficult for the user to perform the shooting work.

  Therefore, in this case, nothing may be displayed on the face portion that has been successfully recognized, and a frame may be displayed on the face portion that has failed to be recognized.

  For example, it is assumed that the face is recognized based on the shape of the region surrounded by the extracted contour points and the color information of the region surrounded by the contour points. At this time, even if it is determined that the region surrounded by the contour point is skin color, if it is determined that the shape of the region surrounded by the contour point is not a face, the face recognition is determined to be unsuccessful, and the face A frame is displayed in a portion (region surrounded by contour points). On the other hand, even if it is determined that the shape of the region surrounded by the contour points is a face, if it is determined that the region surrounded by the contour points is not skin-colored, the face recognition is determined to have failed, and the face A frame may be displayed in a portion (region surrounded by contour points).

  Thereby, the display of the live view image is prevented from being complicated, and the user can easily perform the photographing operation.

  In the above description, face recognition in the live view image is performed up to a predetermined number, but recognition may be performed for all faces in the live view image.

  Further, the display of the frame on the face portion that has been successfully recognized in the live view image may be performed with a predetermined number as a lower limit. When the number of faces shown in the live view image is one or two, the user has a high degree of freedom in the pose and standing position, and the face can be easily recognized even if no guidance display is made on the live view image. Therefore, for example, when the number of faces successfully recognized in the live view image is three or more, a frame is displayed on the face part that has been successfully recognized.

  Furthermore, the face recognition in the live view image is performed with the predetermined number as the upper limit, and the frame display on the face portion that has been successfully recognized in the live view image is performed with the predetermined number as the lower limit. It may be.

  Furthermore, when the number of faces successfully recognized in the live view image exceeds a certain number, a frame is displayed on the face portion that failed to be recognized, and there are the number of faces successfully recognized in the live view image. If the number is less than the number, a frame may be displayed on the face portion that has been successfully recognized. This avoids complicated display of the live view image.

  In the above description, the two-person course and the large-number course are provided as the number-of-persons course. However, for example, a four-person course may be provided.

  Even if the four-person course is selected as the number of people course, if all four users' faces are successfully recognized, four frames will be displayed in the live view image. The display of the view image becomes complicated. Therefore, even in this case, it is possible to prevent the display of the live view image from becoming complicated by displaying a frame on the face portion that has failed to be recognized.

  Furthermore, among the five normal shootings, for example, the number of faces successfully recognized in the live view image at the time of the fourth shooting is four, and the recognition was successful in the live view image at the time of the fifth shooting. When the number of faces is three, it is estimated that the recognition of one face has failed in the live view image at the time of the fifth shooting. Therefore, a frame is displayed on the face portion estimated to have failed to be recognized in the live view image at the time of the fifth shooting.

<Example of guidance prompting successful recognition>
In the above description, the frame is displayed as the guidance for prompting the face recognition to be successful when the live view image is displayed. However, sound may be output. Specifically, the audio output control unit 312 causes the speaker 223 to output a sound that prompts the face recognition to be successful.

  In addition, as guidance for prompting the face recognition to be successful, the color and light emission amount of the illumination (strobe light emission and steady light by each strobe unit) in the shooting space may be changed. If it is determined that the image has failed, shooting may not be performed by shutter control of the camera 111.

  Furthermore, the size of the trimming frame for obtaining a photographed image from the original image obtained at the photographing timing and the focal length of the lens of the camera 111 are changed according to the number of faces successfully recognized in the live view image. May be.

  For example, the size of the trimming frame when the three faces are successfully recognized in the live view image is made larger than the size of the trimming frame when the two faces are successfully recognized in the live view image. Further, the focal length when the three faces are successfully recognized in the live view image is made shorter than the focal length when the two faces are successfully recognized in the live view image.

  As a result, the larger the number of faces (number of users), the wider the range that appears in the captured image, so that the overall appearance of the captured image can be avoided, and the balance between the person and the background is good. Composition can be realized.

<Edit processing>
Next, the editing process performed in step S2 of FIG. 14 will be described with reference to the flowchart of FIG. Hereinafter, an example in which a two-person course is selected as the number-of-persons course will be basically described.

  When the editing process is started, in step S71, the display control unit 321 causes the tablet built-in monitor 141 to display a feeling of feeling selection screen. The sensation of sensation selection screen is used to select the sensation of sensation that expresses the appearance of the face exaggerated (looks better) by changing the face size, shape (outline), eye size, etc. It is.

  FIG. 27 is a diagram illustrating an example of a feeling of fullness selection screen.

  The main structure is symmetrically provided on the sense of sensation selection screen. With such a screen configuration, two users can simultaneously select a feeling of excitement.

  27, image display units 511L and 511R are provided above the left and right areas. Retouch images (face images) obtained by retouch photography are displayed on the image display portions 511L and 511R, respectively.

  Selection units 512L and 512R are displayed below the image display units 511L and 511R, respectively. In the center of each of the selection units 512L and 512R, buttons 521-1 to 521-3 for selecting the feeling of the user's face appearing in each of the five up images obtained by the photographing process are provided.

  Each of the buttons 521-1 to 521-3 corresponds to the degree of image processing that gives a feeling of swell. The button 521-1 is a button for selecting “natural” as a feeling of flourishing, the button 521-2 is a button for selecting “recommended” as a feeling of flourishing, and the button 521-3 is a feeling of flourishing Is a button for selecting “Gekimori”.

  When any of the buttons 521-1 to 521-3 is selected, the image processing unit 324 gives a feeling of climax corresponding to the selected button to the face images displayed on the image display units 511 </ b> L and 511 </ b> R. Apply image processing. The user can select a desired feeling of prosperity by confirming changes in the face images displayed on the image display units 511L and 511R.

  An OK button 522 is provided below the buttons 521-1 to 521-3.

  When the OK button 522 is pressed in a state where any of the buttons 521-1 to 521-3 is selected, the degree of image processing that gives a sense of vigor to the entire face area of the user shown in each captured image. The determined degree of image processing is performed on the entire face area of the user determined and captured in each captured image.

  Further, a feeling of overflow OFF button 523 is provided on the left side of the button 521-1. When any of the buttons 521-1 to 521-3 is selected, when the sensation OFF button 523 is pressed, it is displayed on the image display units 511 </ b> L and 511 </ b> R corresponding to the selected button. The image processing to the extent that it is applied to the face image is canceled.

  Note that an exchange button 513 in the shape of a double-headed arrow is provided at a position slightly above the center of the sensation selection screen. There are cases where the left and right positions of the two users heading to the camera 111 in the shooting space and the left and right positions of the two users heading to the tablet built-in monitor 141 in the editing space are different. In this case, one of the users can select the degree of image processing on his / her face without changing the standing position by operating the replacement button 513.

  When the OK button 522 is pressed on the selection feeling screen or the countdown started after the feeling selection screen is displayed ends (the count becomes 0), the process proceeds to step S72.

  In step S72, the display control unit 321 displays a face color selection screen on the tablet built-in monitor 141. The face color selection screen is a screen used to select a user's face color (skin brightness) in a captured image.

  FIG. 28 is a diagram illustrating an example of a face color selection screen.

  The face color selection screen basically has the same configuration as the liveliness selection screen of FIG. 27, and the main configuration is provided symmetrically. With such a screen configuration, two users can simultaneously select face colors.

  In the face color selection screen of FIG. 28, image display portions 511L and 511R are provided above the left and right areas, as in the liveliness selection screen of FIG.

  Selection units 531L and 531R are displayed below the image display units 511L and 511R, respectively. In the center of each of the selection units 531L and 531R, buttons 541-1 to 541-3 are provided for selecting the user's face color in each of the five photographed images obtained by the photographing process.

  Each of the buttons 541-1 to 541-3 corresponds to the degree of image processing for changing the face color. The button 541-1 is a button for selecting “normal skin” as the face color, the button 541-2 is a button for selecting “whitening skin” as the face color, and the button 541-3 is This is a button for selecting “super whitening skin” as the face color.

  When any of the buttons 541-1 to 541-3 is selected, the image processing unit 324 changes the face color displayed on the image display units 511L and 511R to the face color corresponding to the selected button. Change image processing. The user can select a desired face color by confirming changes in the face images displayed on the image display units 511L and 511R.

  An OK button 542 is provided below the buttons 541-1 to 541-3.

  When the OK button 542 is pressed in a state where any of the buttons 541-1 to 541-3 is selected, the degree of image processing for changing the face color with respect to the entire face area of the user shown in each captured image. The determined degree of image processing is performed on the face area of the user that is determined and appears in each captured image.

  When the OK button 542 is pressed on the face color selection screen or the countdown started after the face color selection screen is displayed ends (count becomes 0), the process proceeds to step S73.

  In step S <b> 73, the display control unit 321 displays a makeup feeling selection screen on the tablet built-in monitor 141. The makeup feeling selection screen is a screen used to select a makeup feeling of the user that appears in the photographed image. The feeling of makeup indicates the degree of image processing performed on a facial part to make it appear to be applying makeup. In this example, the darkness of eyes and the colors of cheeks and lips are selected as the feeling of makeup.

  FIG. 29 is a diagram illustrating an example of a makeup feeling selection screen.

  The makeup feeling selection screen also basically has the same configuration as that of the lively feeling selection screen of FIG. 27, and the main configuration is provided symmetrically. With such a screen configuration, two users can simultaneously select a makeup feeling.

  In the makeup feeling selection screen of FIG. 29, image display portions 511L and 511R are provided above the left and right regions, as in the case of the feeling of height selection screen of FIG.

  Selection units 551L and 551R are displayed below the image display units 511L and 511R, respectively. In the center of each of the selection units 551L and 551R, buttons 561-1 to 561-3 are provided for selecting the user's makeup feeling in each of the five photographed images obtained by the photographing process.

  Each of the buttons 561-1 to 561-3 corresponds to the degree of image processing that changes the makeup feeling. The button 561-1 is a button for selecting “natural” as a makeup feeling, the button 561-2 is a button for selecting “recommended” as a makeup feeling, and the button 561-3 is a makeup feeling. It is a button for selecting "firm" as.

  When any of the buttons 561-1 to 561-3 is selected, the image processing unit 324 gives a makeup feeling corresponding to the selected button to the face images displayed on the image display units 511L and 511R. Change image processing. The user can select a desired makeup feeling by confirming changes in the face images displayed on the image display units 511L and 511R.

  An OK button 562 is provided below the buttons 561-1 to 561-3.

  By pressing the OK button 562 in a state where any of the buttons 561-1 to 561-3 is selected, a makeup feeling is applied to the face area, the eye area, and the mouth area of the user shown in each captured image. The degree of image processing to be changed is determined, and the determined degree of image processing is applied to the user's face area, eye area, and mouth area that appear in each captured image.

  When the OK button 562 is pressed on the makeup feeling selection screen or the countdown started after the makeup feeling selection screen is displayed ends (count becomes 0), the process proceeds to step S74.

  In step S74, the display control unit 321 causes the tablet built-in monitor 141 to display a tear bag selection screen. The tear bag selection screen is a screen used to select the user's tear bag makeup (the darkness of the makeup for the bulge portion along the lower eyelid) reflected in the photographed image.

  FIG. 30 is a diagram illustrating an example of a tear bag selection screen.

  The tear bag selection screen also basically has the same configuration as that of the lively feeling selection screen of FIG. 27, and the main configuration is provided symmetrically. With such a screen configuration, two users can simultaneously select tear bag makeup.

  In the tear bag selection screen of FIG. 30, image display portions 511 </ b> L and 511 </ b> R are provided above the left and right regions, as in the lively feeling selection screen of FIG. 27.

  Selection units 571L and 571R are displayed below the image display units 511L and 511R, respectively. In the center of each of the selection units 571L and 571R, buttons 581-1 and 581-2 are provided for selecting the user's tear bag makeup that is shown in each of the five photographed images obtained by the photographing process.

  Each of the buttons 581-1 and 581-2 corresponds to the degree of image processing for changing the tear bag makeup. The button 581-1 is a button for selecting “natural” as a tear bag makeup, and the button 581-2 is a button for selecting “solid” as a tear bag makeup.

  When one of the buttons 581-1 and 581-2 is selected, the image processing unit 324 makes the tear bag makeup corresponding to the selected button on the face image displayed on the image display units 511L and 511R. Apply image processing to change to. The user can select a desired tear bag makeup by confirming changes in the face images displayed on the image display units 511L and 511R.

  An OK button 582 is provided below the buttons 581-1 and 581-2.

  When the OK button 582 is pressed in a state where any of the buttons 581-1 and 581-2 is selected, the degree of image processing for changing the tear bag makeup on the user's eye area shown in each photographed image is increased. The determined degree of image processing is performed on the eye area of the user that is determined and appears in each captured image.

  When the OK button 582 is pressed on the tear bag selection screen or the countdown started after the tear bag selection screen is displayed ends (count becomes 0), the process proceeds to step S75.

  As described above, the feeling of sensation selection screen, face color selection screen, makeup sensation selection screen, and tear bag selection screen provide a retouch function to finish the user's own face in the captured image to the face he wants to be. Displayed as a retouch screen.

  In each retouch screen, the image processing of the degree selected for each user face (retouch image) is performed by the image processing unit 324 for each user face shown in the photographed image as described above. . Specifically, the authentication unit 323 authenticates the user's face that appears in the captured image based on the retouching image stored in the storage unit 202. Then, the image processing unit 324 performs the image processing of the degree selected corresponding to the retouching image, based on the authentication result, the face determined to most closely match the retouching image (face image). Apply to the area.

  The image processing unit 324 performs image processing of the degree selected on the retouch screen on the face of the user who has been successfully recognized by the recognition unit 314. Therefore, in the live view image, when the identification of either one of the two users fails, the face area of the user who has failed to identify in the captured image obtained by the corresponding imaging is displayed. Regardless of the degree of selection, no image processing is performed.

  When a course for a large number of people is selected as the number of people course, a retouch screen as shown in FIG. 31 is displayed.

  FIG. 31 is a diagram showing an example of a feeling of selection selection screen as an example of a retouch screen when a course for a large number of people is selected.

  An image display unit 591 is provided on the upper side of the sense of feeling selection screen in FIG. The image display unit 591 displays a retouch image obtained by retouch shooting. The retouching image here is an image in which a plurality of users are photographed together. In the example of FIG. 31, it is an image in which two users are photographed, but in reality, three or more users are photographed. It is an image.

  A selection unit 592 is displayed below the image display unit 591. In the center of the selection unit 592, buttons 521-1 to 521-3 for selecting the feeling of the user's face appearing in each of the five photographed images obtained by the photographing process, the OK button 522, and the feeling of feeling rich. An OFF button 523 is provided.

  When any one of the buttons 521-1 to 521-3 is selected, the image processing unit 324 changes the selected button to the user's face shown in the retouching image displayed on the image display unit 591. Image processing that gives a corresponding feeling of prosperity is performed. The user can select a desired feeling of swelling by checking the change of the user's face that appears in the retouching image displayed on the image display unit 591.

  Degree of image processing that gives a sense of vigor to the entire face area of all users in each captured image when the OK button 522 is pressed with any of the buttons 521-1 to 521-3 selected Is determined, and the determined degree of image processing is performed on the entire face area of all users in each captured image.

  That is, when a course for a large number of people is selected as the number of people course, the image processing of the degree selected on each retouch screen is performed on the faces of all the users in the captured image by the image processing unit 324 in a lump. The

  Even in this case, the image processing unit 324 performs image processing of the degree selected on the retouch screen on the face of the user who has been successfully recognized by the recognition unit 314. Therefore, in the live view image, when identification of any face among a plurality of users fails, the face area of the user who has failed identification in the captured image obtained by the corresponding imaging is selected. Image processing is not performed regardless of the degree.

  Now, returning to the flowchart of FIG. 26, when the display of each of the above-described retouch screens ends, the process proceeds to step S75.

  In step S75, the display control unit 321 displays a name input screen on the tablet built-in monitor 141.

  The name input screen is a screen used for inputting a name by the user. On the name input screen, a button for selecting hiragana that can be used for inputting a name and a text box for displaying a character string corresponding to the selected button are displayed.

  When the input of the name is completed on the name input screen, the process proceeds to step S76.

  In step S76, the display control unit 321 causes the tablet built-in monitor 141 to display an editing screen.

  The editing screen is a screen used for editing a captured image to be edited. On the editing screen, a captured image to be edited and various editing tools (compositing images) such as a stamp image and a pen image used for editing are displayed.

  The user uses the touch pens 132A and 132B to superimpose a stamp image, a pen image, or the like at a predetermined position on the photographic image to be edited, thereby synthesizing the synthesis image with the photographic image to be edited. Is called.

  That is, in step S77, the editing control unit 322 edits the captured image to be edited in accordance with a user operation.

  When the editing operation is completed, in step S78, the display control unit 321 displays a division number selection screen on the tablet built-in monitor 141.

  The division number selection screen is a screen used to select the number of divisions of sticker paper to be finally output and the sticker layout. The sticker layout is a layout (placement pattern) that determines the number, position, and size of edited images (photographed images) placed on one photo sticker. Here, an image in which an edited image is arranged according to a sticker layout selected by the user, that is, an image printed on sticker paper and finally output is called a layout image.

  Accordingly, on the division number selection screen, for example, each of five photographed images is subjected to image processing by the retouch function, and a plurality of layout images in which edited images are arranged in different seal layouts are displayed. .

  When one of the layout images is selected on the division number selection screen, the process proceeds to step S79.

  In step S79, the display control unit 321 displays the portable transmission image selection screen on the tablet built-in monitor 141.

  The mobile transmission image selection screen is a screen used for selecting a mobile transmission image to be transmitted to a mobile terminal owned by the user. On the mobile transmission image selection screen, in addition to the five photographed images, a mobile-only image dedicated to mobile transmission, which has been subjected to predetermined editing, is displayed.

  When any of the images is selected on the portable transmission image selection screen, the process proceeds to step S80.

  In step S80, the display control unit 321 displays an address input screen on the tablet built-in monitor 141.

  The address input screen is a screen used for inputting a mail address of a mobile terminal owned by the user. On the address input screen, a text box for displaying alphabets and symbols that can be used for the mail address, a button for selecting a carrier of the mobile phone, and a character string corresponding to the selected button is displayed.

  When the input of the e-mail address is completed on the address input screen, the process proceeds to step S81.

  In step S <b> 81, the editing processing unit 302 controls the communication unit 203 so that the portable transmission image selected by the user via the network such as the Internet is sent to the server along with the mail address input by the user. Send to. The server is managed by, for example, the manufacturer of the photo sticker creating apparatus 1.

  The server transmits a mail describing a URL (Uniform Resource Locator) necessary for the mobile terminal to access the server to the mail address input by the user. Then, based on the URL, the mobile terminal owned by the user accesses the server via the network, so that the mobile transmission image transmitted to the server is provided to the mobile terminal.

  In step S82, the editing processing unit 302 guides the user who has finished the editing work to move to the print waiting space A3. Guidance on the movement to the print waiting space A3 is performed by displaying a guidance screen on the tablet built-in monitor 141 of the editing unit 209 or by outputting sound from the speaker 231.

  The editing process is performed as described above.

  According to the above processing, the image processing of the degree selected on the retouch screen is performed on the captured image in which the face has been successfully recognized in the live view image. Therefore, since appropriate image processing is performed on the captured image more reliably, it is possible to provide an image with higher satisfaction for the user.

  Note that, in the above-described processing, when face recognition fails in all live view images at the time of shooting, display of all retouch screens may be skipped.

  In addition, when a retouched image is displayed in addition to a retouching image on the retouch screen, only the captured image corresponding to the live view image that has been successfully recognized is displayed, and face recognition has failed. The captured image corresponding to the live view image may not be displayed. That is, in the live view images at the time of shooting, when the face recognition is successful, all the shot images are displayed on the retouch screen.

<Configuration of edit screen>
Here, the configuration of the editing screen displayed in step S76 of the editing process described above will be described.

  FIG. 32 is a diagram illustrating an example of an editing screen displayed on the tablet built-in monitor 141.

  The editing screen is basically configured by forming areas on the left and right sides of the background selection area 610 that is a vertically long rectangular area. On the background selection area 610, the remaining time of the time limit assigned to the editing work is displayed.

  The background selection area 610 is a display area such as a button used for selecting a background image. The background selection area 610 is always displayed while the editing screen is displayed. The user can select a background image at any time while editing using the editing screen. That is, since the user can select a background image in the editing operation, the user can find an optimal background image for his / her facial expression or pose in the captured image to be edited.

  The area on the left side of the background selection area 610 (hereinafter simply referred to as the left area) is an area used by the user on the left side of the tablet built-in monitor 141. The area on the right side of the background selection area 610 (hereinafter simply referred to as the right area) is an area used by the user on the right side of the tablet built-in monitor 141.

  That is, for the display of the background selection area 610, an operation using either the touch pen 142A or the touch pen 142B is accepted, but for the display of the left area, only an operation using the touch pen 142A is accepted. For the display of the right area, only an operation using the touch pen 142B is accepted.

  When the user on the left side facing the tablet built-in monitor 141 presses the button in the background selection area 610 using the touch pen 142A and selects a background image, the operation is performed using the display in the left area. Reflected in editing.

  On the other hand, when the user on the right side presses the button in the background selection area 610 using the touch pen 142B and selects a background image, the operation is reflected in the editing performed using the display in the right area. .

  The main structure is provided symmetrically in the left area and the right area. Note that the broken lines surrounding each area in FIG. 32 are not actually displayed on the screen.

  A thumbnail display area 611L at the upper center of the left area is a display area for thumbnail images representing captured images. The thumbnail image is generated, for example, by reducing each captured image. A user (a user on the left side of the tablet built-in monitor 141) selects a captured image to be edited by selecting a thumbnail image displayed in the thumbnail display area 611L.

  In the example of FIG. 32, thumbnail images P21 to P25 representing five photographed images are displayed side by side in the thumbnail display area 611L. Further, by selecting the leftmost thumbnail image P21, the first photographed image is to be edited.

  The thumbnail images P21 to P23 are thumbnail images of up images, and the thumbnail images P24 and P25 are thumbnail images of whole body images.

  An editing area 612L formed large in the approximate center of the left area is a display area for a captured image selected as an editing target. The user can edit the captured image displayed in the editing area 612L by selecting an editing tool using the touch pen 142A.

  On the left side of the editing area 612L, a button 613L that is operated when displaying a sample for editing and a button 614L that is operated when creating the above-described portable-only image are displayed.

  Below the editing area 612L, an editing palette 615L that is a horizontally long rectangular area is displayed. The edit palette 615L is an area in which buttons used for selecting various editing tools (contents) such as a pen tool and a stamp tool are displayed.

  On the editing palette 615L, six types of tabs used for selecting the category of the editing tool are displayed. On the right side of the editing palette 615L, six types of tabs used for selecting the type of editing tool are displayed. The user can switch the types and categories of editing tools displayed on the editing palette 615L using these tabs.

  A correction tool selection area 616L is formed on the right side of the editing area 612L. The correction tool selection area 616L cancels all of the buttons used for selecting the eraser tool, the “forward” and “return” buttons operated when returning to the previous, next, or subsequent work, and the already input edits. This is an area in which “from the start” button, etc., which are operated when returning to the state before editing, are displayed side by side. The eraser tool is a tool used when erasing input editing. In the correction tool selection area 616L, a “return 1 set” button is also displayed. By pressing the “return 1 set” button, edits input during a predetermined time can be canceled together.

  On the correction tool selection area 616L, an end button 617L that is operated when the editing process is ended is displayed.

  In the right side area used by the user on the right side facing the tablet built-in monitor 141, the same configuration as that of the left side area described above is arranged symmetrically. In the example of FIG. 32, by selecting the thumbnail image P22 from the thumbnail display area 611R, the second photographed image is to be edited.

  By the way, also on the editing screen, display according to the face recognition result in the live view image is performed.

  Specifically, for the thumbnail image P21 selected in the left area of the editing screen in FIG. 32, a balloon image B61 including a message indicating that recognition by the recognition unit 314 has failed is displayed in the corresponding live view image. ing.

  When such a thumbnail image, that is, a captured image corresponding to a live view image for which face recognition has failed, is selected as an editing target, as shown in the left area of the editing screen in FIG. In 615L, a stamp tool capable of hiding a face or a face part that has not been detected due to recognition failure is displayed. For example, in the example of FIG. 32, a heart-shaped stamp image capable of hiding the face area and a stamp image imitating sunglasses capable of hiding the eye area are displayed in the lower left corner of the editing palette 615L.

  On the other hand, when a captured image corresponding to a live view image that has been successfully recognized as a face is selected as an editing target, it is detected in the editing palette 615R as shown in the right area of the editing screen in FIG. A stamp tool that is combined according to the position of the face or face part is displayed. For example, in the example of FIG. 32, the editing palette 615R imitates glasses that are automatically combined with each position such as the head, eyes, nose, and mouth of a person shown in the photographed image only by the user's selection operation. A stamp image and a stamp image simulating an animal's ear, nose, and mouth are displayed.

<Modification>
Below, the modification of embodiment mentioned above is demonstrated.

(Variation related to shooting process)
When whole body imaging is performed, the user's whole body may be recognized in the live view image. For example, it is assumed that a user's whole body is recognized by generating a person mask image from a distance image acquired by a distance image sensor.

  When image processing (leg length processing) is performed on a whole body image obtained by whole body photography to make the subject's legs slender and long, leg length processing occurs when the left and right legs cross or the knee is bent. May not be applied properly.

  Therefore, when whole body photography is performed, the whole body of the user is recognized in the live view image, and when it is detected that the left and right legs are crossed or the knee is bent, Display and audio output to guide the legs to straighten out. Thereby, the whole body image to which leg length processing was performed appropriately can be provided.

  Further, the smile level of the face detected in the live view image may be measured, and when the smile level exceeds a predetermined value, a frame as a guidance display may be displayed on the face portion.

  For example, in a photo sticker having a pre-course where the above-mentioned photo sticker creation game is provided and an ID photo course where ID photo shooting is performed as a course related to shooting, when the ID photo course is selected, It is not desirable to photograph with a smile. Therefore, the smile level may be measured for the face detected in the live view image, and when the smile level falls below a predetermined value, a frame as a guidance display may be displayed on the face portion.

  Furthermore, a couple course for users of male and female pairs may be provided as a course related to photographing. When a couple course is selected, in the live view image, the degree of adhesion is digitized based on the distance between the users, and the value is displayed on the live view image, and when the value falls below a predetermined value, A guidance display that prompts the users to approach each other may be displayed.

  Further, in the live view image, the size of the face is detected, and when the size of the face exceeds a predetermined size, a display prompting the user to change the standing position to move away from the camera 111 is displayed as a guidance display. You may make it do. Conversely, when the face size is smaller than a predetermined size, a display prompting the user to change the standing position so as to approach the camera 111 may be displayed as the guidance display.

  In the above-described shooting process, in addition to the retouch shooting and the five normal shootings, for example, shooting for generating a moving image may be performed as an image only to be transmitted to the mobile terminal. Here, even during shooting for generating a moving image, guidance display for prompting the user to succeed in face recognition can be performed.

  In the above description, the live view image display process of FIG. 18 is executed when either the up-shooting or the whole-body shooting is performed, but the finish after the image processing is performed is confirmed. However, it may be executed only when up-shooting is performed to obtain an easy-up image.

(Modifications related to editing processing)
In the editing screen of FIG. 32, it is assumed that all five shot images obtained by five shots can be selected as edit targets. However, at the beginning of the editing process, the edit target is selected from the five shot images. A selection screen for selecting a captured image to be displayed may be displayed.

  In this case, only a photographed image obtained by photographing performed with face recognition failure in the live view image may be displayed on the selection screen. As a result, it is possible to more appropriately perform appropriate image processing on the captured image.

  In addition, a photographed image obtained by photographing performed when face recognition has failed in a live view image may be excluded from editing targets without being selected on the selection screen.

  In this case, since the number of captured images to be edited is reduced, there is a possibility that the user may have time for editing work. Therefore, instead of the excluded photographed image, a copy of the photographed image obtained by photographing that has been successfully performed for face recognition may be provided as an editing target.

  Furthermore, when one of the users succeeds in face recognition (eye detection) in the live view image and one of the users fails in face recognition (eye detection) in the live view image in one photographed image. The image processing content that can be selected on the retouch screen and the degree of image processing may be different for each user.

  For example, on the retouch screen, for users who have succeeded in face recognition, the user selects the degree of image processing to change the size and shape of the eyes, and for users who have failed face recognition, The degree of image processing for changing the size is selected. For users who have failed face recognition (eye detection), it is assumed that the extraction of the contour point of the catchlight in the pupil has succeeded, although the extraction of the eye contour point has failed. And

(Modifications related to printing)
A photographed image obtained by photographing performed with face recognition failure in the live view image may be excluded from the print target.

  In addition, even in the case where a captured image obtained by shooting that was performed when face recognition failed in the live view image is included in the print target, the layout image may be arranged as the smallest size image. It is arranged so that it is difficult for the user to see. Here, the position that is not easily noticeable by the user is, for example, the lower right corner of the entire layout image.

  Among seal layouts that determine the layout of images in a layout image, there is a layout in which one edited image (photographed image) selected by the user is arranged in a size larger than other images. In this case, a photographed image obtained by photographing performed with face recognition failure in the live view image may be excluded from the selection target.

(Modifications related to images for mobile transmission)
A photographed image obtained by photographing performed when face recognition has failed in the live view image may be excluded from the selection targets of the images for mobile transmission.

  In addition, when a captured image obtained by shooting performed with face recognition failure in the live view image is included in the selection target of the image for mobile transmission, the image cannot be displayed in an enlarged manner on the mobile terminal. Image processing such as blurring and filtering is performed. As a result, it is possible to prevent the user from clearly confirming the face and eyes that were not detected.

<Other embodiments>
As described above, when the size of the face is smaller than the predetermined size in the live view image, a display that prompts the user to change the standing position so as to approach the camera 111 may be displayed as the guidance display. As a result, a photographed image can be obtained in which the user's face is large and firm.

  On the other hand, when a pair of users who are the main users of the photo sticker creating apparatus 1 perform a shooting operation, if the faces of the users are separated from each other, a captured image with a poor object balance can be obtained. .

  Therefore, in the following, a configuration for obtaining a photographed image in which the user's face is large and firm and the subject has a good balance will be described.

<Another configuration example of the imaging processing unit>
FIG. 33 is a block diagram illustrating another configuration example of the imaging processing unit 301.

  33 includes a display control unit 311, a sound output control unit 312, a shooting control unit 313, a recognition unit 314, a face size determination unit 711, and a distance determination unit 712.

  In the imaging control unit 301 of FIG. 33, the same reference numerals are given to the components having the same functions as those of the imaging control unit 301 of FIG. 12, and description thereof is omitted.

  The face size determination unit 711 determines whether the size of the user's face shown in the live view image is smaller than a predetermined size.

  The distance determination unit 712 determines whether or not the faces of a plurality of users shown in the live view image are separated from each other by a predetermined distance.

<Live view image display processing>
In the present embodiment, three types of shooting are performed in the shooting process: up shooting, whole body shooting from the top, and front whole body shooting. Whole body photography from the top is photography for obtaining an image showing the whole body of the user viewed obliquely from above, and front whole body photography is photography for obtaining an image showing the whole body of the user viewed from the front. The shooting range and camera angle (camera depression angle) of the camera 111 are different for the up-shooting, the whole body shooting from the top, and the frontal whole body shooting, respectively. In each shooting, in addition to actually changing the camera angle, the shot image may be converted into an image with an apparent (pseudo) camera depression angle by image processing.

  Also, which of the three types of shooting described above is performed as a plurality of shootings is determined by a shooting course that is one of the shooting-related courses selected by the user. Specifically, three courses of “up only” course, “up + whole body from above” course, and “up + front whole body” course are prepared as the shooting course. When the “up only” course is selected as the shooting course, all the normal shooting performed five times is the up shooting. When the “up + whole body from above” course is selected as the shooting course, the first to third shootings are up shooting and the fourth and fifth shootings are the whole shooting from the top among the normal shootings performed five times. . In addition, when the “up + front whole body” course is selected as the photographing course, the first to third photographings are up photographing and the fourth and fifth photographings are front whole body photographing among the normal photographing performed five times. .

  Here, first, live view image display processing when up-shooting or whole-body shooting is performed from above will be described with reference to the flowchart of FIG. Here, it is assumed that a course for two people is selected as the number of people course.

  The process of FIG. 34 is executed every time one image is taken, and is started when the narration for causing the user to take a pose determined for each image is completed.

  Here, the processing in steps S111 to S113 in FIG. 34 is basically the same as the processing in steps S51 to S53 in FIG.

  Note that the live view image displayed in step S112 is generated by superimposing a subject area image in each frame constituting a moving image extracted by chroma processing on a predetermined background image.

  In step S113, the above-described face recognition process is performed on the live view image generated in this way.

  In step S114, the face size determination unit 711 determines whether the size of the user's face shown in the live view image is smaller than a predetermined size based on the recognition result by the recognition unit 314. Specifically, the face size determination unit 711 determines whether or not the smallest face size among the faces recognized in the live view image is smaller than a predetermined size.

  For example, as shown in FIG. 35, it is assumed that the faces U1 and U2 of two users are recognized in the live view image 721. First, the face size determination unit 711 obtains distances d1 and d2 between the centers of the left and right eyes detected by face recognition for each of the faces U1 and U2. Next, the face size determination unit 711 determines whether the shorter one of the obtained distances d1 and d2 (distance d1 in the example of FIG. 35) is shorter than the specified distance (specified value). Thus, it is determined whether or not the size of the smallest face (face U1 in the example of FIG. 35) is smaller than a predetermined size.

  It should be noted that the above-mentioned specified value differs between when up-shooting is performed and when whole-body shooting is performed from above. Specifically, in the whole body image obtained from whole body imaging from above (the whole body image from above), the size of the face relative to the entire image is smaller than the up image, so the whole body imaging is performed from above. The specified value is smaller than the specified value when up-shooting is performed.

  In addition to determining whether the size of the face is smaller than a predetermined size based on the distance between the centers of the left and right eyes detected by face recognition, for example, The size of the face is determined based on the distance between the face parts and the predetermined points on the face, such as the distance to the nose, the distance between the left and right mouth corners of the mouth, and the distance between two predetermined face contour points. You may make it determine whether it is smaller than predetermined size. Further, it may be determined whether or not the face size is smaller than a predetermined size based on the detected size of the left and right eyes and the size of the area of the skin color portion of the face.

  Further, the face size determination unit 711 can determine whether the face size is smaller than a predetermined size regardless of the number of faces recognized by the recognition unit 314. Specifically, when the number of recognized faces is one, it is determined whether or not the size of the face is smaller than a predetermined size. When the number of recognized faces is three or more, It is determined whether or not the size of the smallest face among them is smaller than a predetermined size.

  An upper limit may be set for the number of faces detected by face recognition. As a result, the ratio of erroneous face detection can be reduced. Conversely, a lower limit may be provided for the number of faces detected by face recognition.

  In the above description, it is determined whether or not the size of the smallest face among the detected faces is smaller than a predetermined size, but the average value of the detected face sizes (the detected left and right It may be determined whether the average value of the distance between the centers of the eyes is smaller than a predetermined size. Further, whether or not the size of the smallest face among the detected faces is smaller than the predetermined size is determined depending on whether or not the difference between the size of the largest face and the smallest face among the detected faces is larger than a predetermined value. May be determined.

  If it is determined in step S114 that the face size is smaller than the predetermined size, the process proceeds to step S115.

  In step S115, the face size determination unit 711 determines whether or not it is determined for the first time that the face size is smaller than a predetermined size in the up-shooting or the whole-body shooting from above.

  Specifically, when the “up only” course is selected as the shooting course, it is determined whether or not it is determined for the first time that the face size is smaller than a predetermined size in the first to fifth up shooting. Further, when the “up + whole body from above” course is selected as the shooting course, it is determined whether or not it is determined for the first time that the face size is smaller than a predetermined size in the first to third up shots. In the whole body photographing from the fifth time, it is determined whether or not it is determined for the first time that the face size is smaller than the predetermined size. Further, when the “up + front whole body” course is selected as the shooting course, it is determined whether or not it is determined for the first time that the face size is smaller than the predetermined size in the first to third up-shooting.

  If it is determined in step S115 that the face size is determined to be smaller than the predetermined size for the first time, the process proceeds to step S116.

  In step S <b> 116, the display control unit 311 and the audio output control unit 312 output guidance that prompts the user to approach the camera 111.

  For example, in the live view display screen shown in FIG. 36, a model image M71 and a balloon image B71 that urge the user to approach the camera 111 are superimposed below the live view image displayed in the image display area 431. Is displayed. In the balloon image B71, a message “More closer to the camera” is displayed to prompt the user to approach the camera 111. In addition, the speaker 223 displays the model image M71 and the balloon image B71, and simultaneously outputs the voice of the message displayed in the balloon image B71.

  With such guidance, the user can change the standing position to approach the camera 111 and start shooting.

  Note that the guidance prompting the user to approach the camera 111 is not directed to a specific user, but for example, when one of the two users is standing away from the camera 111, the above-described guidance is given. The user can be brought closer to the camera 111 by the guidance.

  On the other hand, if it is not determined in step S115 that the face size is determined to be smaller than the predetermined size for the first time, that is, in up-shooting performed several times or whole body shooting from above, the camera 111 is already urged to approach once. If the guidance is output, the process ends without outputting the guidance again.

  In other words, when the “up only” course is selected as the shooting course, guidance for prompting the user to approach the camera 111 is output once in five up shots performed in the game. Even when it is determined that the face size is smaller than the predetermined size in the up-shooting, the guidance is not output.

  In addition, when the “up + whole body from the top” course is selected as the shooting course, if guidance for urging to approach the camera 111 is output once in the three up shots performed in the game, In subsequent up-shooting, even when it is determined that the face size is smaller than the predetermined size, the guidance is not output. Similarly, if the guidance that prompts the user to approach the camera 111 is output once in the whole body shooting from the top twice in the game, the face size is set to the predetermined size in the whole body shooting from the top after that. Even if it is determined to be smaller, the guidance is not output.

  Therefore, when the “up + up to whole body” course is selected as the shooting course, this guidance is output at most twice. In addition, in the “up + whole body from top” course, if guidance for urging to approach the camera 111 is output once through three up photographing and two whole body photographing, The guidance may not be output. Thereby, a user's discomfort by repeatedly outputting guidance can be suppressed.

  Furthermore, when the “up + front whole body” course is selected as the shooting course, if guidance for prompting the user to approach the camera 111 is output once in three up shots performed in the game, In subsequent up-shooting, even when it is determined that the face size is smaller than the predetermined size, the guidance is not output.

  As described above, in the case of up-shooting performed multiple times or whole-body shooting from above, once the guidance that prompts the user to approach the camera 111 is output, the user usually moves closer to the camera 111 and moves to an appropriate standing position. Therefore, in the subsequent shooting, even if guidance is not output, the user can start shooting at an appropriate standing position.

  If it is determined in step S114 that the face size is not smaller than the predetermined size, the process proceeds to step S117.

  In step S117, the distance determination unit 712 determines whether or not the user's faces shown in the live view image are separated from each other based on the recognition result by the recognition unit 314. Specifically, the distance determination unit 712 determines whether or not the faces are separated from the predetermined distance based on the distance between predetermined points of the faces recognized in the live view image.

  For example, as shown in FIG. 37, it is assumed that faces U1 and U2 of two users are recognized in the live view image 721. First, for each of the faces U1 and U2, the distance determination unit 712 sets the nose positions detected by face recognition as the face center points P1 and P2. Next, the distance determination unit 712 determines whether the distances d3 between the center points P1 and P2 of the faces U1 and U2 are longer than a predetermined value, thereby determining whether or not the faces are separated from each other by a predetermined distance. Determine.

  Here, the above-mentioned specified value is assumed to be the same for the case where the up-shooting is performed and the case where the whole-body shooting is performed from above, but may be different values.

  In the example of FIG. 37, the detected nose position is set as the center point of the face. For example, the detected midpoint of the left and right eyes may be set as the center point of the face. In addition, it is determined whether the distance between the center points of the faces is longer than the specified value, but the size of each face is digitized, and a value obtained by dividing the distance between the center points of the face by the average value ( It may be determined whether the distance between the face center points is normalized by the average size of each face) is longer than a specified value. Furthermore, it may be determined whether the faces are separated from each other based on the distance between arbitrary points other than the center points of the faces.

  If it is determined in step S117 that the faces are separated from each other by a predetermined distance, the process proceeds to step S118.

  In step S118, the distance determination unit 712 determines whether or not it is determined for the first time that the distance between faces is more than a predetermined distance in up-shooting or multiple-body shooting from above.

  Specifically, when the “up only” course is selected as the shooting course, it is determined whether or not it is determined for the first time that the distance between faces is more than a predetermined distance in the first to fifth up shots. The In addition, when the “up + whole body from above” course is selected as the shooting course, it is determined whether or not it is determined for the first time that the distance between faces is more than a predetermined distance in the first to third up-shooting. In the whole body photographing from the fourth and fifth times, it is determined whether or not it is determined for the first time that the distance between the faces is more than a predetermined distance. Further, when the “up + front whole body” course is selected as the shooting course, it is determined whether or not it is determined for the first time that the distance between the faces is more than a predetermined distance in the first to third up shooting. .

  If it is determined in step S118 that it is first determined that the distance between the faces is greater than the predetermined distance, the process proceeds to step S119.

  In step S119, the display control unit 311 and the audio output control unit 312 output guidance that prompts the user to approach each other.

  For example, on the live view display screen shown in FIG. 38, a model image M72 and a balloon image B72 that prompt the user to approach each other are superimposed and displayed below the live view image displayed in the image display area 431. The In the balloon image B72, a message “More sticking to” that prompts the users to approach each other is displayed. In addition, the speaker 223 displays the model image M72 and the balloon image B72, and simultaneously outputs the voice of the message displayed in the balloon image B72.

  With such guidance, the user can change the standing position so as to approach each other and start shooting.

  On the other hand, if it is not determined in step S117 that the faces are determined to be separated from each other for the first time, that is, guidance that prompts them to approach each other once in up-shooting or multiple-body shooting from above. Is output, the process ends without outputting the guidance again.

  In other words, when the “up only” course is selected as the shooting course, if the guidance that prompts them to approach each other is output once in the five up shots performed in the game, the subsequent up shooting is performed. Even if it is determined that the faces are separated from each other by a predetermined distance, the guidance is not output.

  In addition, when the “up + whole body from the top” course is selected as the shooting course, if guidance for prompting them to approach each other once is output in the three up shots performed in the game, the subsequent steps Even when it is determined that the faces are separated from each other by a predetermined distance in the up shooting, the guidance is not output. Similarly, if guidance for urging them to approach each other once is output in the whole body shooting from the top twice in the game, the faces are separated from each other by a predetermined distance in the whole body shooting from then on. Even if it is determined that it is, the guidance is not output.

  Therefore, when the “up + up to whole body” course is selected as the shooting course, this guidance is output at most twice. Note that in the “up + whole body from top” course, if guidance that prompts them to approach each other once is output through three up shots and two full body shots, the guidance will be used in subsequent shots. May not be output. Thereby, a user's discomfort by repeatedly outputting guidance can be suppressed.

  Furthermore, when the “up + frontal whole body” course is selected as the shooting course, in the three up shots performed in the game, once guidance for prompting them to approach each other is output, Even when it is determined that the faces are separated from each other by a predetermined distance in the up-shooting, the guidance is not output.

  In this way, in up-shooting performed multiple times or whole-body shooting from above, once the guidance that prompts them to approach each other is output, the user usually approaches each other and moves to an appropriate standing position. In the subsequent shooting, even if guidance is not output, the user can start shooting at an appropriate standing position.

  In the determination process of step S117, when the number of faces recognized by face recognition is less than 2, that is, when the number of recognized faces is 1 or 0, steps S117 to S119 are skipped, and the process ends. It shall be.

  If it is determined in step S117 that the faces are not separated from each other by a predetermined distance, the process proceeds to step S120.

  In step S120, whether or not it is determined for the first time that the size of the face is not smaller than the predetermined size and the distance between the faces is not more than the predetermined distance in up-shooting or multiple-body shooting from above is performed. Determined.

  Specifically, when the “up only” course is selected as the shooting course, the face size is not smaller than the predetermined size and the distance between the faces is larger than the predetermined distance in the first to fifth up shooting. If not, it is determined whether it is determined for the first time. In addition, when the “up + whole body from above” course is selected as the shooting course, the face size is not smaller than the predetermined size and the distance between the faces is larger than the predetermined distance in the first to third up-shooting. If not, it is determined whether or not it is determined for the first time, and in the whole body photography from the fourth and fifth times, the size of the face is not smaller than the predetermined size, and the distance between the faces is not separated from the predetermined distance, It is determined whether it is determined for the first time. Furthermore, when the “up + front whole body” course is selected as the shooting course, the face size is not smaller than the predetermined size and the distance between the faces is more than the predetermined distance in the first to third up shooting. If not, it is determined whether it is determined for the first time.

  In step S120, when it is determined for the first time that the face size is not smaller than the predetermined size and the distance between the faces is not more than the predetermined distance in up-shooting or multiple-body shooting from above The process proceeds to step S121.

  In step S121, the display control unit 311 and the audio output control unit 312 are appropriate for the distance to the camera 111 and the distance between users, that is, the distance to the camera 111 is close and the distance between users. OK guidance indicating that is close.

  For example, in the live view display screen shown in FIG. 39, a model indicating that the distance to the camera 111 and the distance between users are appropriate below the live view image displayed in the image display area 431. The image M73 and the balloon image B73 are displayed in a superimposed manner. In the balloon image B73, a message “OK !! Good feeling” indicating that the distance from the camera 111 and the distance between the users is appropriate is displayed. In addition, the speaker 223 displays the model image M73 and the balloon image B73, and simultaneously outputs the voice of the message displayed in the balloon image B73.

  With such guidance, the user can start shooting without changing the standing position.

  On the other hand, if it is not determined in step S120 that the face size is not smaller than the predetermined size and the distance between the faces is not greater than the predetermined distance, it is not determined for the first time, that is, up-shooting that is performed a plurality of times or In the case of whole body imaging from above, if the OK guidance has already been output once, the process ends without outputting the guidance again.

  According to the above processing, when the user's face is small, guidance for prompting the user to approach the camera 111 is output, so that the user can approach the camera 111. In addition, when the faces of users are separated from each other, guidance that prompts them to approach each other is output, so that the users can approach each other. As a result, it is possible to provide a photographed image in which the user's face is large and firmly captured and the subject is well balanced.

  In the captured image, even if the user's face is small, the balance between the subject area and the background area is deteriorated, and the degree of completion may be reduced. Therefore, according to the above-described processing, it is first determined whether or not the user's face is small regardless of whether or not the faces of the users are separated (that is, the size of the user's face has priority). Therefore, the user can be surely brought closer to the camera 111, and a reduction in the completeness of the captured image can be avoided.

  In addition, in the captured image, if the users are separated from each other, a part of the captured image may be overlooked from the imaging range, and the degree of completion may be reduced. Therefore, contrary to the processing described above, it is first determined whether or not the faces of the users are separated regardless of whether or not the faces of the users are small (that is, the distance between the users has priority). May be determined). As a result, the users can be brought close to each other reliably, and a reduction in the degree of completion of the captured image can be avoided.

  In the above, live view image display processing when up-shooting or whole-body shooting from above is performed in the two-person course has been described.

  Next, a live view image display process when front whole body imaging is performed will be described with reference to a flowchart of FIG. Here again, it is assumed that a course for two people is selected as the number of people course.

  The processing of FIG. 40 is also executed each time one shooting is performed, and is started when the narration for causing the user to take a pose determined for each shooting is completed.

  Here, the processing in steps S131 to S133 in FIG. 40 is the same as the processing in steps S111 to S113 in FIG. 34, and a description thereof will be omitted.

  Also, the processing in steps S134 to S138 in FIG. 40 is the same as the processing in steps S117 to S121 in FIG.

  In step S135, when the “up + frontal whole body” course is selected as the photographing course, is it determined for the first time that the distance between the faces is more than a predetermined distance in the fourth and fifth frontal whole body photographing? It is determined whether or not.

  In frontal whole body photography, it is preferable to set the user's standing position away from the camera 111. Therefore, in the live view image display process when the front whole body imaging is performed, the determination of the face size is performed as in the live view image display process (FIG. 34) when the whole body imaging is performed from the up shooting or the top ( The determination of the distance to the camera 111 is not performed, and only the determination of the distance between the faces is performed.

  Also, even in front whole body photography that is performed multiple times, once the guidance that prompts them to approach each other is output, the user usually moves closer to each other and moves to an appropriate standing position. Even if guidance is not output, the user can start shooting at an appropriate standing position.

  In the case of whole body photography from above, if the user's standing position is away from the camera 111, the process of FIG. 40 is performed as live view image display processing when whole body photography is performed from above. It may be applied.

  In the above, the live view image display processing when the two-person course is selected as the number of people course has been described, but in the following, the live view image display processing when the large-number course is selected will be described.

  FIG. 41 is a flowchart for describing live view image display processing when up-shooting or whole-body shooting is performed from above in a course for a large number of people.

  The process of FIG. 41 is also executed each time one shooting is performed, and is started when the narration for causing the user to take a pose determined for each shooting is completed.

  Note that the processing in steps S151 through S153 in FIG. 41 is the same as the processing in steps S111 through S113 in FIG.

  Also, the processing in steps S154 to S158 in FIG. 41 is the same as the processing in steps S114 to S116, S120, and S121 in FIG.

  In the course for a large number of people, it is not easy to specify the number of users, and there is a possibility that the faces of the users overlap each other as compared with the course for two people. Therefore, in live view image display processing when up-shooting is performed in a course for a large number of people or whole body shooting from above, live view image display processing is performed when up-shooting is performed in a course for two people or whole body shooting is performed from above ( As shown in FIG. 34), the determination of the distance between the faces is not performed, but only the determination of the face size (distance from the camera 111) is performed.

  Note that in the live view image display processing when front whole body photography is performed in a course for a large number of people, neither the determination of the distance between faces nor the determination of the face size (distance from the camera 111) is performed. And

<Application to re-shooting>
In the re-shooting described above, the type of shooting corresponding to the shot image selected on the re-shooting target selection screen (up shooting / overall body shooting / frontal whole body shooting) is performed.

  Here, the above-described series of processing may be applied also in the live view image display processing when re-taking is performed.

  For example, when the “up only” course is selected as the shooting course and no guidance is output in the first to fifth up shots, the up shot is performed again and the face size is set to a predetermined size. If it is determined that the distance is smaller or the distance between the faces is determined to be greater than the predetermined distance, the corresponding guidance is output.

  In addition, in the state where the “up only” course is selected as the shooting course and the guidance is output once in the first to fifth up shots, the face size is set to a predetermined size when the up shot is performed again. Even if it is determined that the distance is smaller or the distance between the faces is determined to be greater than the predetermined distance, the corresponding guidance is not output.

<Modification>
Hereinafter, modifications of the present embodiment will be described.

(About judgment of face size and distance between faces)
In the above description, the determination of the size of the face and the determination of the distance between the faces are performed based on the recognition result by the recognition unit 314. For example, a mask image that masks the subject area extracted by the chroma process is used. The size of the face and the distance between the faces may be determined based on the generated mask image. Alternatively, a distance image having a distance value indicating a distance from the camera 111 as a pixel value using a distance image sensor may be generated to determine the face size and the distance between faces.

(About judgment in a course for a large number of people)
In the course for a large number of people, the size of the face may be determined with an upper limit of a predetermined number of faces detected in the live view image. It is assumed that no determination is made regarding the size of the face exceeding the upper limit.

  For example, the determination of the face size is performed with the upper limit of four faces detected in the live view image.

  In this case, the average of the detected face sizes of the four people is used as a reference size, and a guidance that prompts the user to approach the camera 111 is output depending on whether the size of each face is separated from the reference size or a predetermined size. Depending on whether or not the maximum size and the minimum size of the detected face sizes of the four people are separated from the predetermined size, guidance for urging to approach the camera 111 is output. You may do it.

  In the course for a large number of people, the number of faces shown in the live view image increases, so that a predetermined frame is displayed only on the face portion determined to be small (separated from the camera 111). It may be.

  Further, without setting the above-described upper limit, the face size may be determined for all the faces detected in the live view image. Thereby, it is possible to avoid such a situation that only one person becomes extremely small in the captured image obtained.

  Further, in contrast to the above-described example, the face size may be determined with a predetermined number of faces detected in the live view image as a lower limit.

  For example, the size of the face is determined with the lower limit of three detected faces in the live view image.

  In this case, when the number of detected faces in the live view image is two or less, the size of the face is not determined. However, if the number of users is small, the degree of freedom of standing position within the shooting range Therefore, even if the guidance is not output, the user can move while confirming the live view image.

  Note that both the upper limit and the lower limit described above may be provided for the number of faces detected in the live view image.

  In the course for a large number of people, since three or more faces are mainly detected in the live view image, the distance between the faces is determined based on the average value of the distances between the center points of the faces. Shall be.

  For example, as shown in FIG. 42, it is assumed that the faces U11, U12, and U13 of three users are recognized in the live view image 731. First, for each of the faces U11, U12, and U13, the position of the nose detected by face recognition is set as the face center points P11, P12, and P13. Next, the average value of the distance d12 between the center points P11 and P12 of the faces U11 and U12, the distance d23 between the center points P12 and P13 of the faces U12 and U13, and the distance d31 between the center points P13 and P11 of the faces U13 and U11. Is calculated. Then, by determining whether the average value is longer than the specified value, it is determined whether the faces are separated from each other by a predetermined distance.

(About countdown before shooting)
When OK guidance is output in shooting performed as normal shooting, the countdown before shooting for that shooting may be accelerated.

  For example, when guidance for urging the camera 111 to approach or guidance for urging the camera 111 to approach each other is output, the subsequent countdown uses “3”, “2”, “1”, “ It is assumed that it starts from “3”, such as “0”.

  On the other hand, when the OK guidance is output, the subsequent countdown does not need to move the standing position, and is assumed to start from “1” such as “1” and “0”.

  Thereby, the time required for shooting can be shortened, and as a result, the turnover rate of the game can be increased.

(About guidance output conditions)
In the live view image display process when frontal full body photography is performed, the determination of the face size is not performed, but only the determination of the distance between the faces is performed. As in the live view image display process (FIG. 34) when it is performed, the face size may be determined and the distance between the faces may be determined.

  Thus, even when the shooting space A1 is wide in the depth direction of the shooting range, the user's standing position can be prevented from being too far from the camera 111.

  Also, in the above, when the “up + whole body from above” course is selected as the shooting course, the guidance is output twice at the maximum. Guidance may be output during shooting. This makes it possible to provide a captured image with a high degree of completeness more reliably.

  It should be noted that an upper limit may be set for the number of times the guidance is output in order to suppress the user's discomfort due to the repeated output of the guidance.

(About other judgments)
In the above, the determination on the size of the face and the determination on the distance between the faces are performed. However, the determination of the smile level of the face, the determination of the position of the face in the shooting range, and the like are performed. Also good.

  For example, when the position of the face in the shooting range is performed, first, as shown in FIG. 43, the center points P21 and P22 of the faces U21 and U22 of the two users in the live view image 741 are specified. Next, it is determined whether or not the center points P21 and P22 of the faces U21 and U22 are present in the rectangular area 741A with the center point of the live view image 741 as a reference. If it is determined that any one of the center points of the face does not exist in the rectangular area 741A, guidance for prompting the user to approach the center of the shooting range is output.

  In the example of FIG. 43, the center point P21 of the face U21 does not exist in the rectangular area 741A, and a model image M81 and a balloon that prompts the user to approach the center of the shooting range below the live view image 741. The image B81 is superimposed and displayed. In the balloon image B81, a message “Take me closer to the middle” to prompt the user to approach the center of the shooting range is displayed. In addition, the speaker 223 displays the voice of the message displayed in the balloon image B81 at the same time as the model image M81 and the balloon image B81 are displayed.

  Thereby, it is possible to prevent the user's face from being completely cut out in the captured image.

  In the example of FIG. 43, when the center point of the face does not exist in the rectangular area 741A, the size of the trimming frame for obtaining the photographed image from the original image obtained at the photographing timing may be increased. .

(Guidance output target)
In the above description, the guidance is output for shooting performed as normal shooting, but the guidance may be output for shooting other than normal shooting.

(1) Reshooting The live view image display process in which guidance is output may be executed only when the reshooting described above is performed. Usually, the user chooses to re-take a photographed image with a higher degree of completion. Therefore, when the re-shooting is performed, the above-described live view image display process is executed, so that a shot image with a high degree of completeness can be reliably provided. Note that it is possible to shorten the time required for the photographing work by preventing the guidance from being output when the normal photographing is performed.

(2) Retouch shooting Live view image display processing in which guidance is output may be executed only when retouch shooting is performed. The retouching image obtained by the retouch shooting is an image used as a sample of image processing to be selected by the user in addition to being used for face authentication. Therefore, when the retouch shooting is performed, the above-described live view image display processing is executed so that a sample with a higher degree of perfection can be presented.

(3) Special shooting There is a photo sticker that performs special shooting after normal shooting is performed in the shooting process.

  Special shooting includes, for example, mobile-only shooting for obtaining a portable-only image that can be transmitted only to a mobile terminal owned by the user, and a collage image obtained by synthesizing shot images obtained by normal shooting. Includes collage photography. As mobile-only shooting, still images are shot for uploading to SNS (Social Networking Service), moving images are shot for a predetermined time, and still images are displayed continuously like flip-flops. A plurality of times of shooting may be performed. According to the special shooting, a special image is provided to the user.

  Here, live view image display processing in which guidance is output may be executed only when such special shooting is performed. Thereby, the completeness of an image with special feeling can be raised.

(4) Others In the above, guidance is output for a live view image. However, guidance may be output for a still image (preview image) displayed as a shooting result after shooting. Good. Specifically, the message “Next, please come a little ahead” is superimposed on the preview image displayed after shooting, or the sound is output. Thereby, the user can consciously correct the distance to the camera 111 and the distance between users at the time of subsequent photographing.

(Example when editing)
In the editing process, various types of image processing are performed on a captured image (an image to be edited) to be edited in accordance with a user's selection operation. At this time, guidance according to the user's selection operation may be output.

  For example, on the editing screen, it is assumed that the type of processing applied to the image to be edited and the degree of blurring of the blurring process applied to the image to be edited can be selected.

  For each type of processing, a combination of multiple image processing that corrects each parameter such as contrast, color balance, tone curve, saturation, transparency, texture, and sharpness of the entire captured image is determined and selected. Image processing corresponding to the type of processing is performed on the captured image.

  The blurring process is a process of reducing sharpness, and the degree of blurring is determined by the intensity of blurring. Increasing the blur intensity decreases the sharpness, and decreasing the blur intensity increases the sharpness.

  Here, when the user selects the type of processing and selects the degree of blurring of the blurring process, guidance for prompting the selection is output.

  For example, when selecting the type of processing, a message such as “Please select the type of processing” prompting selection of the type of processing is superimposed on the image to be edited or is output as audio. .

  In addition, when selecting the degree of blurring for the blurring process, a message such as “Please select the degree of blurring” prompting the selection of the degree of blurring for the blurring process is superimposed on the image to be edited or output as audio. Like that.

  In this way, the corresponding guidance is output for each user selection operation.

  Further, guidance may be output for each type of shooting from which a shot image selected as an image to be edited is obtained.

For example, it is assumed that the course “up + whole body from above” is selected as the photographing course, and three up images and whole body images are obtained from the top two. When the up image is selected for the first time as the image to be edited, the above-described guidance is output. When the up image is selected as the image to be edited thereafter, the guidance is not output. . Further, when the whole body image is selected from the top as the editing target image for the first time, the above-described guidance is output. After that, when the whole body image is selected from the top as the editing target image, the guidance is output. Is not output.

  In the above-described embodiment, the photo sticker creating apparatus 1 is configured to print the obtained photographed image or edited image on the sticker paper and to send it to the server by sending it to the server. . However, the present invention is not limited to this, and it is also possible to adopt a configuration in which a photographed image or an edited image is only provided to a user's mobile terminal by being transmitted to a server without being printed on sticker paper. On the contrary, it is possible to adopt a configuration in which the photographed image or the edited image is only printed on the sticker paper without being transmitted to the server.

  That is, the present technology is not limited to a photo sticker creating apparatus that creates a photo sticker in which a photographed image or an edited image is printed on a sticker paper, but simply a photo creating game machine that provides a game for creating a photographed image or an edited image. Can be applied.

  The series of processes described above can be executed by hardware or can be executed by software. When the above-described series of processing is executed by software, a program constituting the software is installed from a network or a recording medium.

  For example, as shown in FIG. 10, the recording medium is distributed to distribute the program to the administrator of the photo sticker creating apparatus 1 separately from the apparatus main body, and is a magnetic disk (flexible disk) on which the program is recorded. Disk), optical disk (including CD-ROM and DVD), magneto-optical disk, or removable media 205 made of semiconductor memory, etc., but also delivered to the administrator in a pre-installed state in the main unit The ROM 206 in which the program is recorded and the hard disk included in the storage unit 202 are configured.

  In the present specification, the step of describing the program recorded on the recording medium is not limited to the processing performed in chronological order according to the described order, but is not necessarily performed in chronological order. It also includes processes that are executed individually.

  In the above, the print medium is not limited to sticker paper or photographic paper. For example, the print medium is printed on a predetermined size of paper or film, poster paper, a card, or a fabric such as a T-shirt. Also good.

  The embodiments of the present technology are not limited to the above-described embodiments, and various modifications can be made without departing from the gist of the present technology.

DESCRIPTION OF SYMBOLS 1 Photo sticker production apparatus 301 Image pick-up processing part 302 Edit processing part 311 Display control part 312 Audio | voice output control part 313 Image pick-up control part 314 Recognition part 321 Display control part 322 Edit control part 323 Authentication part 324 Image processing part 711 Face size determination part 712 Distance judgment part

Claims (24)

A shooting control unit for shooting one or more users as subjects;
A display control unit that displays a live view image of the user before the shooting timing;
A recognition unit that determines whether at least one of the user's face and face parts is detected by face recognition in the live view image;
A photo creation game machine comprising: an output unit that outputs guidance according to a recognition result by the recognition unit. The photo creation game machine according to claim 1, wherein the output unit outputs a guidance that prompts the user to succeed in recognition by the recognition unit according to a recognition result by the recognition unit. The output unit is configured as a monitor,
The photo creation game machine according to claim 2, wherein the display control unit displays the guidance on the screen displayed on the monitor together with the display of the live view image. The photo creation game machine according to claim 3, wherein the display control unit displays the guidance regarding the user's face that has been successfully recognized by the recognition unit. The photo creation game machine according to claim 3, wherein the display control unit displays the guidance regarding the face of the user that has failed to be recognized by the recognition unit. The output unit is configured as a speaker,
The photo creation game machine according to claim 2, wherein the speaker outputs a sound related to the guidance. The photographing control unit performs one photographing out of a plurality of photographings as authentication photographing for authenticating the user's face,
The recognizing unit recognizes at least one of the face and the face part in the live view image displayed during normal shooting other than the authentication shooting,
The photo creation game machine according to claim 1, wherein the output unit outputs guidance according to a recognition result by the recognition unit. Selection to display a selection screen that accepts selection of the degree of image processing for each of the user's faces that appear in the captured image obtained by the normal shooting based on the result of the authentication shooting A screen display control unit;
An image processing unit that performs the image processing of the degree selected for each face of the user on the selection screen for each face of the user reflected in the captured image;
The photo creation game machine according to claim 7, wherein the selection screen receives selection of the degree of the image processing for the face of the user who has been successfully recognized by the recognition unit. On the selection screen, an authentication image for each user obtained by the authentication shooting is displayed,
The photo creation game machine according to claim 8, wherein the image processing unit performs the image processing of the degree selected for each face of the user on the selection screen on the authentication image of the corresponding user. The image processing unit performs the image processing of the degree selected for each face of the user on the selection screen on the face of the user that has been successfully recognized by the recognition unit and appears in the captured image. The photo creation game machine according to 8 or 9. The photograph according to claim 8, wherein when the number of users is equal to or greater than a predetermined number, the selection screen collectively receives selection of the degree of image processing for all of the user's faces successfully recognized by the recognition unit. Create game machine. Shoot one or more users as subjects,
Before the shooting timing, display a live view image of the user,
In the live view image, it is determined whether or not at least one of the user's face and face parts is detected by face recognition;
An imaging method including a step of outputting guidance according to a recognition result by the recognition unit. A shooting control unit for shooting one or more users as subjects;
A display control unit that displays a live view image of the user before the shooting timing;
A face size determination unit that determines whether or not a size of the user's face reflected in the live view image is smaller than a predetermined size;
A distance determination unit that determines whether or not the plurality of user faces reflected in the live view image are separated from each other by a predetermined distance;
When it is determined that the size of the face is smaller than the predetermined size, the first guidance for the user is output, and when it is determined that the faces are separated from the predetermined distance, the first for the user is output. A photo creation game machine comprising: an output unit that outputs the guidance of 2; The live view image further comprises a recognition unit that determines whether or not the user's face is detected by face recognition,
The face size determination unit determines whether the size of the user's face is smaller than a predetermined size based on the recognition result of the recognition unit;
The photo creation game machine according to claim 13, wherein the distance determination unit determines whether or not the plurality of user faces are separated from each other by a predetermined distance based on a recognition result of the recognition unit. The first guidance is guidance that prompts the user to approach the camera,
The photo creation game machine according to claim 13 or 14, wherein the second guidance is guidance that prompts the users to approach each other. The face size determination unit determines whether or not a size of the user's face is smaller than a predetermined size regardless of whether or not a plurality of the user's faces are separated from each other by a predetermined distance. 15. The photo creation game machine according to any one of 15. The distance determination unit determines whether or not the faces are separated from the predetermined distance when it is determined that the size of at least one of the faces is not smaller than the predetermined size. The photo creation game machine described in 1. The face size determination unit determines whether the face size is smaller than the predetermined size based on a distance between predetermined points of the detected face part. The described photo creation game machine. The distance determination unit determines whether or not the faces are separated from the predetermined distance based on a distance between predetermined points of the plurality of detected faces. The photo creation game machine described in 1. The photograph according to any one of claims 13 to 19, wherein the output unit outputs the first guidance only when it is determined for the first time that the size of the face is smaller than the predetermined size in a plurality of shootings. Create game machine. The output unit outputs the second guidance only when it is determined for the first time that the faces are separated from the predetermined distance in a plurality of shootings. Photo creation game machine. The photograph according to any one of claims 13 to 21, wherein the face size determination unit determines whether or not the size of the face is smaller than the predetermined size only when the face is photographed within a predetermined photographing range or camera angle. Create game machine. The photo creation game according to claim 13, wherein the distance determination unit determines whether or not the faces are separated from the predetermined distance only when a predetermined number of users are photographed. Machine. Shoot one or more users as subjects,
Before the shooting timing, display a live view image of the user,
Determining whether the size of the user's face reflected in the live view image is smaller than a predetermined size;
Determining whether or not the faces of the plurality of users shown in the live view image are separated from each other by a predetermined distance;
If it is determined that the face size is smaller than the predetermined size, a first guidance for the user is output;
When it is determined that the faces are separated from each other by the predetermined distance, an imaging method including a step of outputting a second guidance for the user.

Images