JP7140962B2 - Image processing device, control method, and program - Google Patents

Description

The present technology relates to an image processing device , a control method, and a program, and more particularly to an image processing device , a control method, and a program that enable well-balanced image processing.

Conventionally, a photo sticker creating device is known. A photo sticker creating apparatus photographs a user, makes the user edit the photographed image, and prints the edited image on sticker paper to provide the edited image. A photo sticker creating device is installed in an amusement facility or the like.

The flow of one game provided by the photo sticker creating device is that after taking a picture of the user in the shooting space as the subject, the user is moved to the editing space, and the image is displayed according to the operation performed in the editing space. After editing, the edited image is printed on sticker paper.

By the way, some photo sticker creating devices allow retouching such as making the face appearing in the image smaller or enlarging the eyes after the photograph is taken. Retouching is image processing for recognizing a part to be processed and changing the position, size, and shape of the recognized part.

In addition, there are some that allow content composition such as changing the color of the lips or changing the color of the hair. Content synthesis is performed by generating a mask image based on an image obtained by photographing, and using the generated mask image to synthesize another image (content) such as an image of a predetermined color. For example, when changing the color of the lips, a mask image of the lip area is used to synthesize an image of brightly colored lips.

JP 2013-243648 A

Although it is possible to change the size of a specific part, it is difficult for users who are not used to it to retouch in a well-balanced manner. For example, there is no limit to the number of times the eyes can be enlarged, which sometimes results in an unnatural image.

In addition, when retouching is performed after compositing content using makeup-related content, an image with an impression different from that expected by the user may be produced. For example, if the user enlarges the eyes after performing the operation of synthesizing the eyeline content, the eyes are overemphasized and the balance is lost.

For the system side of the photo sticker creation device, when performing retouching such as changing the shape of the face after compositing makeup content, in order to prevent misalignment etc., it is necessary to adjust the shape after retouching. It is necessary to regenerate the mask image and synthesize makeup content again.

The present technology has been developed in view of such a situation, and enables well-balanced image processing.

An image processing apparatus according to the present technology includes a display control unit that displays an editing screen provided with first information for retouching operation and second information for content composition operation for a photographed image in which a user is a subject; an image processing unit that retouches the photographed image according to the user's operation using the first information ; a synthesizing unit for synthesizing content with a photographed image; and an operation of the user on the editing screen so as not to accept one operation of retouching and synthesizing the content after accepting the other operation. and a control unit that controls acceptance of

In the present technology, an editing screen provided with first information for a retouching operation and second information for a content synthesizing operation for a photographed image in which the user is the subject is displayed, and the first information is used. Acceptance of a user's operation is controlled such that, after accepting one operation of retouching and synthesizing content using the second information, the other operation is not accepted.

According to the present technology, well-balanced image processing can be performed.

It is a perspective view showing an example of composition of appearance of a photograph sticker production device of this art. FIG. 10 is a perspective view of the appearance of the photo sticker creating apparatus as seen from another angle; It is a figure explaining a user's movement. It is a figure which shows the structural example of a prior selection part. It is a figure which shows the structural example of an imaging part. It is a figure which shows the structural example of a background part. It is a figure which shows the structural example of the front of an editing unit. FIG. 4 is a diagram showing a configuration example of a side surface of an editing unit; FIG. 4 is a diagram showing an example of a layer configuration of an edited image; It is a block diagram which shows the structural example of a photo sticker production apparatus. 2 is a block diagram showing a functional configuration example of a photo sticker creating device; FIG. 2 is a block diagram showing a configuration example of an edit processing unit; FIG. FIG. 4 is a diagram showing an example of layers to be subjected to image processing; FIG. 10 is a diagram illustrating an example of restrictions on acceptance of an operation; FIG. 10 is a diagram showing another example of restrictions on acceptance of operations; 4 is a flowchart for explaining game processing of the photo sticker creating device; FIG. 17 is a flowchart for explaining editing processing performed in step S4 of FIG. 16; FIG. FIG. 10 is a diagram showing an example of an edit screen; FIG. 18 is a flowchart for explaining operation reception control processing performed in step S32 of FIG. 17; FIG. FIG. 10 is a diagram illustrating an example of restrictions on acceptance of an operation; FIG. 10 is a diagram showing another example of restrictions on acceptance of operations; It is a figure which shows the example of a display of one part of an edit screen.

<<Example 1>>
<Appearance of photo sticker making device>
1 and 2 are perspective views showing configuration examples of the appearance of the photo sticker creating apparatus 1. FIG.

The photo sticker creation device 1 is a game machine that provides photographed images and edited images. The photo sticker creating apparatus 1 provides images to users by printing the images on sticker paper or making the images viewable on the user's portable terminal. The photo sticker creating device 1 is installed in an amusement facility such as a game center or a store.

Users of the photo sticker creating apparatus 1 are mainly high school girls and young women. In the photo sticker creating device 1, a plurality of users, mainly two or three, can enjoy the game per group. Of course, one user can also enjoy the game in the photo sticker creating apparatus 1 .

In the photo sticker creating apparatus 1, the user himself/herself becomes a subject and performs photographing work. Through editing work, the user combines an image for composition such as handwritten characters or a stamp image with an image selected from the photographed images obtained by photographing. As a result, the photographed image is edited into a colorful image. The user receives the seal paper on which the edited image, which is the edited image, is printed, and ends the series of games.

The photo sticker creation device 1 can be called a photo creation game machine that provides a photo creation game.

As shown in FIG. 1, the photo sticker creating apparatus 1 is basically configured by installing a photographing unit 11 and an editing unit 12 in contact with each other. The photographing unit 11 and the editing unit 12 are electrically connected.

The photographing unit 11 has a box-shaped housing large enough for a person to enter, and is composed of a pre-selection section 20 , a photographing section 21 and a background section 22 . The pre-selection unit 20 is installed on the side of the imaging unit 21 . The space in front of the pre-selection unit 20 becomes the pre-selection space where the pre-selection process is performed. Also, the photographing unit 21 and the background unit 22 are placed apart from each other by a predetermined distance. A space formed between the photographing section 21 and the background section 22 is a photographing space in which photographing processing is performed.

As the pre-selection process, the pre-selection unit 20 performs guidance for introducing games provided by the photo sticker creating apparatus 1, and performs various settings in the photographing process performed in the photographing space. The pre-selection unit 20 is provided with a coin slot into which the user inserts money, a touch panel monitor used for various operations, and the like. The pre-selection unit 20 appropriately guides the user in the pre-selected space to the imaging space according to the availability of the imaging space.

The photographing unit 21 is a device for photographing a user as a subject. The photographing unit 21 is positioned in front of the user entering the photographing space. A camera, a touch panel monitor used for various operations, and the like are provided in front of the photographing unit 21 facing the photographing space.

Assuming that the left side and the right side are the left side and the right side, respectively, as viewed from the user facing the front in the imaging space, the left side of the imaging unit 21 is composed of the side panel 41A, and the right side is composed of the side panel 41B. be done. Further, a front panel 42 constitutes the front of the photographing unit 21 . The preselection section 20 described above is installed on the side panel 41A. Note that the preselection unit 20 may be installed on the side panel 41B.

The background portion 22 is composed of a rear panel 51, side panels 52A, and side panels 52B. The back panel 51 is a plate-like member located on the back side of the user facing the front. The side panel 52A is a plate-like member attached to the left end of the rear panel 51 and having a width narrower than that of the side panel 41A. The side panel 52B is a plate-like member attached to the right end of the rear panel 51 and having a width narrower than that of the side panel 41B.

The side panel 41A and the side panel 52A are provided substantially on the same plane. Upper portions of the side panel 41A and the side panel 52A are connected by a connecting portion 23A, which is a plate-like member. The lower portions of the side panel 41A and the side panel 52A are connected by a connecting portion 23A' which is provided on the floor and is a member made of metal, for example. Similarly, the side panel 41B and the side panel 52B are provided substantially on the same plane. Upper portions of the side panel 41B and the side panel 52B are connected by a connecting portion 23B. The lower portions of the side panel 41B and the side panel 52B are connected by a connecting portion 23B'.

A floor material 27, which is a plate-like member, is laid on the floor of the imaging space. The flooring 27 is printed with a mark used for designating a standing position at the time of photographing.

For example, a green chromakey sheet is attached to the surface of the back panel 51 facing the imaging space. The photo sticker creating apparatus 1 performs chromakey synthesis in the shooting process and the editing process by shooting a chromakey sheet as a background. As a result, the desired background image is synthesized with the user's background in which the sheet is shown.

An opening formed by being surrounded by the side panel 41A, the connecting portion 23A, and the side panel 52A serves as an entrance to the imaging space. An opening formed by being surrounded by the side panel 41B, the connecting portion 23B, and the side panel 52B also serves as an entrance to the imaging space.

A ceiling surrounded by the front of the photographing unit 21, the connecting portion 23A, and the connecting portion 23B is formed above the photographing space. A ceiling strobe unit 24 is provided on a part of the ceiling. One end of the ceiling strobe unit 24 is fixed to the connecting portion 23A, and the other end is fixed to the connecting portion 23B.

The ceiling strobe unit 24 incorporates a strobe that irradiates light toward the inside of the photographing space in accordance with photographing. Inside the ceiling strobe unit 24, a fluorescent lamp is provided in addition to the strobe. As a result, the ceiling strobe unit 24 also functions as lighting for the shooting space.

The editing unit 12 is a device for editing a captured image. The editing unit 12 is connected to the photographing unit 11 so that one side thereof contacts the front panel 42 of the photographing section 21 .

If the configuration of the editing unit 12 shown in FIG. 1 is taken as the configuration on the front side, the configuration used in the editing work is provided on each of the front side and the back side of the editing unit 12 . This configuration allows two sets of users to perform editing work at the same time.

The front side of the editing unit 12 is composed of a surface 61 and an inclined surface 62 formed above the surface 61 . The surface 61 is a surface that is perpendicular to the floor surface and substantially parallel to the side panel 41A of the imaging unit 21 . A built-in tablet monitor and a touch pen are provided on the slope 62 as components used for editing work. A columnar support portion 63A that supports one end of the lighting device 64 is provided on the left side of the slope 62 . A columnar support portion 63B that supports the other end of the lighting device 64 is provided on the right side of the slope 62 . A support portion 65 for supporting the curtain rail 26 is provided on the upper surface of the support portion 63A.

A curtain rail 26 is attached above the editing unit 12 . The curtain rail 26 is configured by combining three rails 26A to 26C. The three rails 26A to 26C are combined to form a U-shape when viewed from above. One ends of rails 26A and 26B provided in parallel are fixed to connecting portions 23A and 23B, respectively, and the other ends of rails 26A and 26B are joined to both ends of rail 26C, respectively.

A curtain is attached to the curtain rail 26 so that the space in front of the front and the space in front of the back of the editing unit 12 are not visible from the outside. The space in front of the editing unit 12 surrounded by the curtain and the space behind the editing unit 12 become an editing space in which the user performs editing work.

As will be described later, the left side of the editing unit 12 is provided with a discharge port through which the printed sticker paper is discharged. A space in front of the left side of the editing unit 12 becomes a print waiting space where the user waits for the printed sticker paper to be discharged.

<Regarding the movement of users>
Here, the flow of the photo sticker creation game and the movement of the user associated therewith will be described.

FIG. 3 is a top plan view of the photo sticker creating apparatus 1. FIG.

The user inserts money into the coin slot in the pre-selection space A0, which is the space in front of the pre-selection section 20. FIG. Next, the user makes various settings according to the screen displayed on the touch panel monitor. The user performs, for example, selection of a course related to photographing to be performed in the photographing space as a pre-selection work.

After completing the pre-selection work, the user enters the imaging space A1 formed between the imaging unit 21 and the background unit 22 from the entrance G1 between the side panel 41A and the side panel 52A, as indicated by the white arrow #1. to go into. A user uses a camera, a touch panel monitor, or the like provided in the photographing unit 21 to perform photographing work.

The user who has completed the photographing work exits the photographing space A1 from the doorway G1 as indicated by the white arrow #2 and moves to the editing space A2-1, or enters the doorway G2 as indicated by the white arrow #3. , exits the photographing space A1 and moves to the editing space A2-2.

The editing space A2-1 is an editing space on the front side of the editing unit 12. FIG. On the other hand, the editing space A2-2 is the editing space behind the editing unit 12. FIG. To which of the editing space A2-1 and the editing space A2-2 the user should move is guided by the screen display of the touch panel monitor of the photographing unit 21 or the like. For example, the empty one of the two editing spaces is guided as the destination. The user who has moved to the editing space A2-1 or the editing space A2-2 starts editing work. The user of the editing space A2-1 and the user of the editing space A2-2 can perform editing work at the same time.

After the editing work is completed, printing of the edited image is started. When printing starts, the user who has finished the editing work in the editing space A2-1 moves from the editing space A2-1 to the print waiting space A3 as indicated by the white arrow #4. Also, the user who has finished the editing work in the editing space A2-2 moves from the editing space A2-2 to the print waiting space A3 as indicated by the white arrow #5.

The user who has moved to the print waiting space A3 waits for the end of printing. When the printing is finished, the user receives the sticker paper ejected from the ejection port provided on the right side of the editing unit 12, and finishes the series of photo sticker creating games.

<Configuration of pre-selection unit>
FIG. 4 is a diagram showing a configuration example of the pre-selection unit 20. As shown in FIG.

A touch panel monitor 71 is provided in the pre-selection unit 20 . The touch panel monitor 71 is composed of a monitor such as an LCD (Liquid Crystal Display) and a touch panel stacked thereon. The touch panel monitor 71 has a function of displaying various GUIs (Graphical User Interfaces) and receiving a user's selection operation. The touch panel monitor 71 displays a screen used for selecting the number of users.

A speaker 72 is provided below the touch panel monitor 71 . The speaker 72 outputs a guidance voice for pre-selection processing, BGM (Background Music), sound effects, and the like. Adjacent to the speaker 72, a coin insertion/return opening 73 for inserting coins by the user is provided.

<Structure of shooting unit>
FIG. 5 is a diagram showing a configuration example of the front of the photographing unit 21. As shown in FIG. The imaging unit 21 is configured to be surrounded by the side panel 41A, the side panel 41B, and the front panel 42. As shown in FIG.

A camera unit 81 is provided slightly above the center of the front panel 42 . A camera 91 , a touch panel monitor 92 , and a front strobe 93 are provided in front of the camera unit 81 .

The camera 91 is, for example, a single-lens reflex camera, and is attached inside the camera unit 81 so that the lens is exposed. The camera 91 has an imaging device such as a CCD (Charge Coupled Device) image sensor or a CMOS (Complementary Metal Oxide Semiconductor) image sensor, and photographs a user in the imaging space A1. The camera 91 functions as a photographing unit that photographs a user as a subject.

A moving image captured by the camera 91 (hereinafter also referred to as a live view image) is displayed on the touch panel monitor 92 in real time. A still image captured by the camera 91 at a predetermined timing, such as when an instruction to shoot is given, is saved as a shot image.

A touch panel monitor 92 is provided below the camera 91 . The touch panel monitor 92 is composed of a monitor such as an LCD and a touch panel laminated thereon. The touch panel monitor 92 has a function as a live view monitor that displays moving images captured by the camera 91, and a function that displays various GUIs and receives user selection operations.

A front strobe 93 is provided above the camera 91 . The front strobe 93 has a crescent-shaped light-emitting surface made of a milky white acrylic plate. The front strobe 93 emits light in synchronization with photographing by the camera 91 to illuminate the vicinity of the user's face as a subject from the front.

Above the camera unit 81, an upper strobe unit 82 is installed with its curved light-emitting surface facing the user. The upper strobe unit 82 irradiates the user's face and upper body with light from above the front of the user.

Below the camera unit 81, a foot strobe unit 83 is provided to irradiate the user's lower body and feet with light.

The upper strobe unit 82 and the foot strobe unit 83 are composed of a fluorescent lamp and lighting equipment capable of strobe light emission. The upper strobe unit 82 and the foot strobe unit 83 always illuminate the shooting space in order to ensure the operability and safety of the user, and emit strobe light at the shooting timing to improve the image quality of the shot image. A user, who is a subject, is irradiated with light.

A box-shaped portion 84 is provided between the foot strobe unit 83 and the side panel 41A. A box-shaped portion 85 is provided between the foot strobe unit 83 and the side panel 41B. The upper surfaces of the box-shaped parts 84 and 85 form a surface substantially horizontal with the installation surface of the photo sticker creating apparatus 1, and are used as a luggage storage area for the user who performs the photographing work to place luggage and the like.

Although not shown, a speaker is provided on the front panel 42, for example, near the ceiling. The speaker outputs guide voices, BGM, sound effects, etc. for the shooting process.

<Composition of the background part>
FIG. 6 is a diagram showing a configuration example of the background portion 22 on the photographing space A1 side.

A rear upper strobe 101 is installed above the rear panel 51 . The upper back strobe 101 irradiates the user with light from above the back.

In the drawing, a rear left strobe 102 is installed on the left side of the rear panel 51 . The back left strobe 102 irradiates the user from the right side of the back. In the drawing, a rear right strobe 103 is installed on the right side of the rear panel 51 . The back-right strobe 103 irradiates the user from the back-left side.

A chromakey sheet 121 is attached to the surface of the back panel 51 facing the shooting space A1 (the front side in the figure). The color of the chromakey sheet 121 is, for example, green.

Although not shown, a chromakey sheet similar to the chromakey sheet 121 is attached to the lower side (floor surface side) of the side panels 52A and 52B on the imaging space A1 side.

<Structure of editing unit>
FIG. 7 is a diagram showing a configuration example of the front side (editing space A2-1 side) of the editing unit 12. As shown in FIG.

A tablet built-in monitor 131 is provided approximately in the center of the slope 62 . A touch pen 132A is provided on the left side of the tablet built-in monitor 131 . A touch pen 132B is provided on the right side of the tablet built-in monitor 131 .

The tablet built-in monitor 131 is configured by laminating a tablet and a display. The tablet enables operation input using touch pen 132A or touch pen 132B. The tablet built-in monitor 131 displays, for example, an editing screen used for editing work. When two users simultaneously perform editing work, the touch pen 132A is used by the user on the left side of the tablet built-in monitor 131, and the touch pen 132B is used by the user on the right side of the tablet built-in monitor 131. .

An operation using the touch pen 132A and an operation using the touch pen 132B are distinguished. As appropriate, the display on the left side of the tablet built-in monitor 131 can only be operated using the touch pen 132A, and the display on the right side can only be operated using the touch pen 132B. Hereinafter, when there is no need to distinguish between the touch pen 132A and the touch pen 132B, the touch pen 132A and the touch pen 132B are collectively referred to as the touch pen 132.

FIG. 8 is a diagram showing a configuration example of the left side of the editing unit 12. As shown in FIG.

A seal paper discharge port 161 is provided on the lower side of the left side of the editing unit 12 . A printer is provided inside the editing unit 12 . The printer prints an image of the user of the editing space A2-1 or an image of the user of the editing space A2-2 in a predetermined layout on a sticker sheet, which is discharged from the sticker paper outlet 161. FIG.

<Layer structure of edited image>
FIG. 9 is a diagram showing an example of the layer structure of an edited image.

The edited image P1 at the left end is an image generated by editing. The edited image P1 is basically composed of images of layers of a background image P11, a person image P12, and a foreground image P13. As will be described later, layers to be subjected to various types of image processing are also set appropriately.

In the example of FIG. 9, the background image P11 is an image of a pattern of a wall made of stacked bricks. The foreground image P13 is an image including an image of handwritten characters "Love my friend" and a heart-shaped image. The handwritten character image of "Love my friend" is a pen image input by the user using the pen tool. The heart-shaped image is a stamp image input by the user using the stamp tool.

Thus, the edited image P1 is generated by superimposing the person image P12, which is the image of the person's area extracted from the photographed image by chromakey processing, on the background image P11, and superimposing the foreground image P13 thereon. It is also possible to input a pen image or a stamp image onto the background image P11 and display it in the background of the subject. A foreground image or a background image may be combined with a photographed image, or may not be combined depending on a user's operation even after an editing process.

In the example of FIG. 9, an image showing a large area above the user's chest is shown as the person image P12. In the photographing process, a close-up image, which is a photographed image in which the face is enlarged in this way, is photographed. Shooting of the close-up image is performed, for example, multiple times.

In the photographing process, in addition to the close-up image, an image showing a wide range from the top of the head to the vicinity of the knee is also photographed. This image is used, for example, to keep the coordination of clothes. Hereinafter, an image with a wider shooting range than the close-up image, which is used for keeping the coordination of clothes as appropriate, will be referred to as a coordinated image.

A whole-body image that captures the whole body of the user, which has a wider imaging range than the coordinated image, may be captured.

<Internal configuration of photo sticker creation device>
FIG. 10 is a block diagram showing a configuration example of the photo sticker creating apparatus 1. As shown in FIG. In FIG. 10, the same reference numerals are given to the same configurations as those described above. Duplicate explanations will be omitted as appropriate.

As shown in FIG. 10, the photo sticker creating apparatus 1 has an external input/output interface 303 for a PC (Personal Computer) section 301, a preselection section 302, a photographing section 304, editing sections 305A and 305B, and a printing section 306. It is configured by being connected via The PC section 301 is accommodated inside the housing of the editing unit 12, for example.

A CPU (Central Processing Unit) 311 , a ROM (Read Only Memory) 312 , and a RAM (Random Access Memory) 313 constituting the PC section 301 are interconnected by a bus 314 .

A CPU 311 executes a predetermined program and controls the overall operation of the photo sticker creating apparatus 1 . The ROM 312 stores programs and data executed by the CPU 311 . A RAM 312 temporarily stores data and programs processed by the CPU 311 .

An input/output interface 315 is further connected to the bus 314 . A storage unit 316 , a communication unit 317 and a drive 318 are connected to the input/output interface 315 .

The storage unit 316 is a nonvolatile storage medium such as a hard disk or flash memory. The storage unit 316 stores various setting information and the like supplied from the CPU 311 . Information stored in the storage unit 316 is appropriately read by the CPU 311 .

A communication unit 317 is an interface of a network such as the Internet. A communication unit 317 communicates with an external device under the control of the CPU 311 . The communication unit 317 transmits the image selected by the user to, for example, a management server managed by the manufacturer of the photo sticker creating apparatus 1 . The image transmitted from the communication unit 317 is stored in a predetermined storage area, displayed on the mobile terminal accessing the management server, or downloaded to the mobile terminal.

A removable medium 319 such as an optical disk or a semiconductor memory is mounted in the drive 318 as appropriate. Programs and data read from the removable medium 319 by the drive 318 are supplied to the CPU 311 and stored in the storage unit 316 or installed.

The external input/output interface 303 is connected to the input/output interface 315 . Each unit is controlled by the PC unit 301 via the external input/output interface 303 .

The preselection unit 302 performs preselection processing for users in the preselection space A0. The preselection unit 302 is composed of a touch panel monitor 71 , a speaker 72 and a coin processing unit 321 .

The touch panel monitor 71 displays various selection screens under the control of the PC unit 301 and accepts user operations on the selection screens. An input signal representing the content of the user's operation is supplied to the PC unit 301, and various settings are performed.

The coin processing unit 321 detects insertion of coins into the coin insertion/return port 73 . When the coin processing unit 321 detects that a predetermined amount of coins has been inserted, the coin processing unit 321 outputs an activation signal instructing the start of the game to the PC unit 301 .

The photographing unit 304 performs photographing processing for users in the photographing space A1. The photographing unit 304 is composed of an illumination device 331 , a camera 91 , a touch panel monitor 92 and a speaker 332 .

The lighting device 331 is each strobe unit in the shooting space A1 and emits light according to the lighting control signal supplied from the PC section 301 .

The camera 91 performs photographing according to shutter control by the PC section 301 and outputs image data obtained by photographing to the PC section 301 .

The editing unit 305A performs editing processing for users in the editing space A2-1. The editing unit 305A is composed of a tablet built-in monitor 131, touch pens 132A and 132B, and a speaker 341. FIG. The editing section 305B has the same configuration as the editing section 305A, and performs editing processing for users in the editing space A2-2.

The tablet built-in monitor 131 displays an edit screen under the control of the PC unit 301 and accepts a user's operation on the edit screen. An input signal representing the content of the user's operation is supplied to the PC unit 301, and the captured image to be edited is edited.

Printing unit 306 is configured to include printer 351 . A sticker paper unit 352 is attached to the printer 351 .

The printer 351 drives a built-in head 361 and prints an image on a roll-shaped sticker paper 363 stored in the sticker paper unit 352 based on print data supplied from the PC section 301 . The printer 351 also cuts the sticker paper 363 on which the image is printed by the cutter 362 at a predetermined cut position, and discharges the cut paper to the sticker paper outlet 161 .

<Example of functional configuration of photo sticker creation device>
FIG. 11 is a block diagram showing a functional configuration example of the photo sticker creating apparatus 1. As shown in FIG. At least some of the functional units shown in FIG. 11 are implemented by the CPU 311 executing a predetermined program.

In the photo sticker creating apparatus 1, a pre-selection processing unit 401, a photographing processing unit 402, an editing processing unit 403, and a printing processing unit 404 are realized.

The pre-selection processing unit 401 executes pre-selection processing by controlling each unit of the pre-selection unit 302 . Selection of the number of users, selection of a background image, and the like are performed by the pre-selection process. Information representing the content selected by the user is supplied to the imaging processing unit 402 .

The imaging processing unit 402 performs imaging processing by controlling each unit of the imaging unit 304 . The shooting processing unit 402 outputs the data of the shot image obtained by performing the shooting processing to the editing processing unit 403 . The shooting processing unit 402 supplies the editing processing unit 403 with data of a plurality of captured images obtained by performing shooting a plurality of times.

The editing processing unit 403 performs editing processing by controlling each unit of the editing units 305A and 305B. After editing the captured image, the edit processing unit 403 generates print data by arranging the edited image in a predetermined layout, and outputs the print data to the print processing unit 404 .

The print processing unit 404 outputs print data to the printer 351 to print the edited image.

<Detailed configuration of editing processing unit>
FIG. 12 is a block diagram showing a configuration example of the edit processing unit 403. As shown in FIG.

The edit processing unit 403 receives data from a captured image acquisition unit 411 , an edit target image buffer 412 , a display control unit 413 , an edit control unit 414 , a content composition unit 415 , a retouch processing unit 416 , a reception control unit 417 , and a print data generation unit 418 . Configured.

The captured image acquisition unit 411 acquires the captured image output from the image capturing processing unit 402 . For example, five close-up images and one coordinated image are acquired. The captured image selected as the editing target is transferred to the editing target image buffer 412, for example.

The edit target image buffer 412 stores the shot image to be edited. The shot image to be edited stored in the edit target image buffer 412 is not only edited (scribbled) by the editing control unit 414, but also content synthesized by the content synthesizing unit 415 and retouched by the retouch processing unit 416 as appropriate. be done.

The edited image buffer 412 stores captured images that have been subjected to image processing such as editing, content composition, and retouching. When the editing process is finished, the captured image that has been subjected to image processing such as editing, content composition, and retouching is supplied to the print data generation unit 418 as an edited image.

FIG. 13 is a diagram illustrating an example of layers to be subjected to image processing.

As shown in FIG. 13, layers L1 to L5 are set for each captured image in order from the bottom.

A layer L1 is a layer in which a background image is arranged. For example, the background image selected in the pre-selection process is placed on layer L1.

A layer L2 is a layer in which a person image is arranged. Retouching by the retouching processing unit 416 is performed on the layer L2.

Layer L3 is a layer used for content synthesis. Makeup-related content and body-related content, which will be described later, are arranged on layer L3 by the content synthesizing unit 415 .

Layer L4 is a layer used for editing. An image for synthesis using an editing tool such as a pen tool or a stamp tool is arranged on layer L4 by the editing control unit 414 .

The layer L5 is a layer to be subjected to frame image synthesis and filtering. Image processing such as synthesis of frame images and filter processing is also performed by the editing control unit 414 in accordance with the user's operation, for example.

In the edit target image buffer 412, the shot image to be edited is stored in a form having such a layer structure. The captured image stored in the editing target image buffer 412 is read by the display control unit 413 and used for the display of the editing screen.

Returning to the description of FIG. 12 , the display control unit 413 controls display on the tablet built-in monitor 131 . For example, the display control unit 413 causes the tablet built-in monitor 131 to display an editing screen used for editing the captured image. On the editing screen, it is possible not only to edit the shot image, but also to perform content composition and retouching.

The display control unit 413 switches the display of the tablet built-in monitor 131 according to the user's operation. Information representing the content of the user's operation is supplied from the reception control unit 417 .

The edit control unit 414 edits the captured image stored in the edit target image buffer 412 according to the user's operation. Information representing the content of the user's operation is supplied from the reception control unit 417 . The editing control unit 414 arranges an image for synthesis corresponding to an editing tool such as a pen tool or a stamp tool on the layer L4 of the shot image to be edited.

The content synthesizing unit 415 performs content synthesizing on the captured images stored in the edit target image buffer 412 in accordance with the user's operation. Content synthesis is a process of synthesizing another image (content) in a specific area of a subject captured in a captured image using a mask image generated based on the captured image. The shape, size, etc. of the content used for synthesis are appropriately changed according to the shape and size of the area to be synthesized.

For example, make-up content includes lip content. When the lip content is selected, the content synthesizing unit 415 recognizes the lips of the person in the captured image and generates a mask image that masks the area other than the lip area. The content synthesizing unit 415 performs content synthesis using the lip content by synthesizing the lip image of the color selected by the user with the lip region of the person using the mask image.

Content used for content synthesis includes close-up image content and full-body image content. The content for the whole-body image may be used for content synthesis targeting the coordinated image.

Content for close-up images includes makeup-based content and stick-on content, as shown below.

Makeup-related content ・Eyebrow mascara ・Eyeshadow ・Eyeliner ・Colored contact lenses ・Pseudo catch light ・False eyelashes ・Tear bags ・Lower eyelashes ・Highlights ・Shading ・Cheeks ・Lip ・Skin color (whitening, blackening)
・Skin quality (matte, glossy)
・Hair color ・Angel ring

The eyebrow mascara is content for synthesizing an eyebrow image of a predetermined color on the eyebrow portion. The eye shadow is content for synthesizing an eye shadow image of a predetermined color on the eyelid portion. The eyeline is content that synthesizes an eyeline image of a predetermined color on the edge of the eye.

A color contact lens is content for synthesizing an image of a contact lens of a predetermined color on the pupil of the eye. The pseudo catchlight is content that synthesizes an image of a catchlight with a predetermined shape on the pupil of the eye. False eyelashes is content that synthesizes an image of false eyelashes of a predetermined length with the upper eyelashes.

The lacrimal sac is content that synthesizes a predetermined lacrimal sack image with the lacrimal sack portion. The lower eyelashes is content for synthesizing an image of false eyelashes of a predetermined length with the lower eyelashes. Highlight is content that synthesizes an image that expresses light on the bridge of the nose and forehead.

Shading is content that synthesizes an image that expresses shadows on the bridge of the nose, forehead, chin, and the like. Cheek is content that synthesizes a cheek image with a cheek portion. As described above, the lip is content that synthesizes the lip image with the lip portion.

The skin color is content for synthesizing an image of a predetermined skin color with the skin part. The skin texture is content for synthesizing an image representing a predetermined texture of the skin on the skin portion. The hair color is content for synthesizing an image of hair of a predetermined color with the hair portion. An angel's ring is content that synthesizes an image of an angel's ring above the head.

Attached content ・Sunglasses ・Glasses ・Nose ・Beard ・Mouth

Sunglasses as sticky content is content that recognizes the user's eyes and synthesizes an image of sunglasses superimposed on the eyes. Glasses are content that recognizes the eyes of the user and synthesizes an image of predetermined glasses by superimposing them on the eyes. The nose is content that recognizes the user's nose and synthesizes a predetermined nose image over the nose.

The whisker is content that recognizes the user's mouth and synthesizes a predetermined whisker image around the mouth. Mouth is content that recognizes the user's mouth and synthesizes a predetermined mouth image over the mouth.

Full-body image content includes full-body content and attached content as follows.

Full-body content ・Aura ・Frame (type that avoids the subject)

The whole-body content is synthesized according to the position and shape of the user's region in the captured image. Auras and frames are content that composite images around the user's body.

Attached content ・Ears ・Hat ・Wig ・Tail ・Wings ・Angel's ring ・Shoes
·clothes

The ear as sticky content is content that recognizes the user's head and synthesizes a predetermined ear image superimposed on the head. A hat is content that recognizes the user's head and synthesizes a predetermined hat image over the head. A wig is content that recognizes the user's head and synthesizes a predetermined wig image over the head.

The tail is content that recognizes the user's whole body and synthesizes a predetermined tail image overlaid near the buttocks. The wing is content that recognizes the user's shoulder and synthesizes a predetermined wing image over the shoulder. The ring of angels is content that recognizes the user's head and synthesizes an image of the ring of angels above the head.

"Shoes" is content that recognizes the user's feet and synthesizes an image of a predetermined pair of shoes over the user's feet. The palm-on-the-palm system is content that recognizes the user's hand and synthesizes a predetermined image by placing it on the hand. Clothing is content that recognizes the user's whole body and synthesizes a predetermined clothing image over the whole body.

The content synthesizing unit 415 functions as a synthesizing processing unit that performs such various content syntheses according to user operations received by the reception control unit 417 . Information representing the content of the user's operation is supplied from the reception control unit 417 . The content synthesizing unit 415 arranges makeup-related content selected by the user on the layer L3 of the shot image to be edited.

The retouch processing unit 416 retouches the captured image stored in the edit target image buffer 412 according to the user's operation. Retouching is image processing for recognizing a specific part of a person in a captured image and changing the shape, size, and position of the recognized part.

Retouching includes close-up image retouching and full-body image retouching.

Retouching for close-up images Contour processing Small face processing Eye size processing Eye shape processing Nostril processing Raising the corners of the mouth Mouth size processing Skin color processing Moving parts (eyes) move it toward the temple, etc.)
・Eyebrow (change the shape of the eyebrows)

The contour processing is processing for recognizing the user's face in the captured image and changing the contour. Face-slimming processing is processing for recognizing a user's face in a captured image and making the face smaller. The eye size processing is processing for recognizing the user's eyes in the captured image and enlarging the eyes.

The eye shape processing is processing for recognizing the user's eyes in the captured image and changing the eye shape. The nostril processing is processing for recognizing the user's nose in the captured image and making the nose smaller. The process of raising the corners of the mouth is a process of recognizing the user's mouth in the photographed image and raising the corners of the mouth.

Mouth size processing is processing for recognizing the user's mouth in the captured image and changing the size of the mouth. The skin color processing is processing for recognizing the skin color of the user appearing in the captured image and changing the skin color. The movement of parts is a process of recognizing the parts (parts) of the user's face shown in the captured image and moving the parts. For example, movements of parts include upward and downward movements of the eyes, inward movements, and movements of the mouth. Eyebrow is a process of recognizing the user's eyebrows in a captured image and changing the shape of the eyebrows.

Retouching for full-body images ・Leg length processing ・Leg thinness processing ・Height processing ・Upper arm processing ・Constriction processing ・Neck length processing ・Waist position raising processing・Nail length processing

The leg lengthening process is a process of recognizing the legs of the user appearing in the captured image and lengthening the legs. The thin leg processing is processing for recognizing the leg of the user in the photographed image and thinning the leg. The height processing is processing for recognizing the whole body of the user appearing in the photographed image and increasing the height of the user.

The upper arm processing is processing for recognizing the user's upper arms in the captured image and thinning the upper arms. Constriction processing is processing for recognizing the waist of the user in the photographed image and emphasizing the constriction. Neck length processing is processing for recognizing the user's neck in the captured image and lengthening the neck.

The process of raising the waist position is a process of recognizing the waist of the user in the captured image and raising the waist position. The finger thinning process is a process of recognizing the user's hand in the captured image and thinning the finger. The process of deforming the nail shape is a process of recognizing the user's hand in the photographed image and deforming the shape of the nail. The nail length processing is processing for recognizing the user's hand in the photographed image and changing the length of the nail.

The retouching processing unit 416 functions as an image processing unit that performs such various retouching processes according to user operations received by the reception control unit 417 . Information representing the content of the user's operation is supplied from the reception control unit 417 . A person image in which the shape, size, position, etc. of a specific part has been modified by the retouch processing unit 416 is arranged as an image of the layer L2 that constitutes the shot image to be edited.

The reception control unit 417 controls reception of user operations on the editing screen.

For example, when an editing operation is performed using a predetermined editing tool, the reception control unit 417 receives the editing operation by the user and outputs information representing the details of the user's operation to the editing control unit 414 .

Further, when predetermined content to be used for content synthesis is selected, reception control unit 417 receives an operation by the user and outputs information representing the content of the user's operation to content synthesis unit 415 .

When a predetermined type of retouching is selected, the reception control unit 417 receives an operation by the user and outputs information representing the content of the user's operation to the retouching processing unit 416 .

Here, the acceptance control unit 417 does not accept a retouching operation for a certain captured image if content composition for the same captured image is performed first.

FIG. 14 is a diagram illustrating an example of restrictions on acceptance of operations.

In FIG. 14, the first operation represents the operation performed earlier, and the second operation represents the operation performed after the first operation.

As shown in the upper part of FIG. 14 , when content composition using makeup-related content is performed as the first operation, and retouching regarding the face is performed as the second operation, the second operation is not accepted. . Face retouching includes the above-described close-up image retouching.

Information indicating that the operation has not been accepted is output from the acceptance control unit 417 to the display control unit 413 and reflected in the display of the edit screen, for example.

For example, the following control is performed in response to the second operation being performed.
(1) Buttons used for face retouching operations are grayed out.
(2) Even if a button used for face retouching operation is pressed, the operation is not reflected in the shot image to be edited.
(3) Display a pop-up message notifying that face-related retouching operations cannot be performed.

Further, as shown in the lower part of FIG. 14 , in the case where content composition using whole-body content is performed as the first operation, and retouching regarding the whole body is performed as the second operation, the second operation is accepted. can't Retouching for the whole body includes retouching for the whole-body image described above.

Information indicating that the operation has not been accepted is output from the acceptance control unit 417 to the display control unit 413 and reflected in the display of the edit screen, for example.

For example, the following control is performed in response to the second operation being performed.
(1) Buttons used for retouching the whole body are grayed out.
(2) Even if a button used for retouching the whole body is pressed, the operation is not reflected in the captured image to be edited.
(3) Display a pop-up message notifying that retouching operations related to the whole body cannot be performed.

In this way, by restricting acceptance of retouching after content composition is performed, the user can perform well-balanced image processing on the captured image.

For example, if retouching to enlarge the eyes can be performed after the operation of synthesizing eyeliner content, the enlargement of the eyes may result in an unbalanced image, resulting in an unnatural image. However, it is possible to prevent such a thing.

Also, when changing the shape of the face after compositing makeup-related content, the balance will be lost unless the position and shape of the composited content are changed in accordance with the change in the shape of the face. there is a possibility. When changing the position or shape of the content, it is necessary to regenerate the mask image based on the image after the shape of the facial part has been changed. This avoids such a burden. That is, it is possible to reduce the processing load of the photo sticker creating apparatus 1 .

FIG. 15 is a diagram illustrating an example of restrictions on acceptance of operations.

As shown in the upper part of FIG. 15 , when content composition using makeup-based content is performed as the first operation, and retouching of the whole body is performed as the second operation, the second operation can be accepted. can be

Further, as shown in the lower part of FIG. 15 , in the case where content synthesis using whole-body content is performed as the first operation, and retouching regarding the face is performed as the second operation, the second operation is accepted. may be made available.

In this way, whether or not to restrict acceptance of retouching is set according to whether the retouching target is the whole body or the face.

Returning to the description of FIG. 12, the print data generation unit 418 reads out the captured image stored in the edit target image buffer 412 after finishing the editing, and acquires it as an edited image. The print data generation unit 418 generates print data by arranging the edited images in a predetermined layout, and outputs the print data to the print processing unit 404 for printing.

<Operation of photo sticker creation device>
Here, the operation of the photo sticker creating apparatus 1 that provides the photo sticker creating game will be described with reference to the flowchart of FIG.

In step S1, the pre-selection processing unit 401 determines whether or not a predetermined amount of coins has been inserted based on the activation signal supplied from the coin processing unit 321, and waits until it is determined that coins have been inserted. do. If it is determined in step S1 that a coin has been inserted, the process proceeds to step S2.

In step S<b>2 , the pre-selection processing unit 401 performs pre-selection processing by controlling the pre-selection unit 302 . By the pre-selection process, for example, selection of the number of users, selection of a background image to be combined with the captured image, and the like are performed. Any selection is made using the screen displayed on the touch panel monitor 71 .

In step S<b>3 , the imaging processing unit 402 performs imaging processing by controlling the imaging unit 304 . For example, the photographing processing unit 402 displays a live view screen on the touch panel monitor 92 based on the moving image captured by the camera 91, and photographs the user in the photographing space A1 as a subject.

In step S4, the editing processing unit 403 controls the editing unit corresponding to the editing space to which the user who has completed the shooting process is moved, out of the editing space A2-1 and the editing space A2-2. , edit processing. For example, the editing processing unit 403 causes the tablet built-in monitor 131 to display an editing screen, and allows the user to edit the photographed image obtained by the photographing process.

In step S<b>5 , the print processing unit 404 performs print processing by controlling the printer 351 . The print processing unit 404 outputs the print data generated by the editing processing unit 403 to the printer 351 to print it on the sticker paper 363 .

When printing is completed, the printer 351 ejects the sticker paper 363 cut by the cutter 362 to the sticker paper outlet 161 in step S6.

Each group of users of the photo sticker creating apparatus 1 is provided with a photo creating game consisting of a series of processes as described above. When another group starts using the photo sticker creating device 1 following a certain group, the photo creating game of the group that started using the photo sticker creating device 1 first and the photo creating game of the group that started later. will be performed in parallel as appropriate.

<Editing process>
Next, the editing process performed in step S4 of FIG. 16 will be described with reference to the flowchart of FIG.

In step S31, the display control unit 413 causes the tablet built-in monitor 131 to display an edit screen.

FIG. 18 is a diagram showing an example of an edit screen displayed on the tablet built-in monitor 131. As shown in FIG.

The edit screen is basically configured by symmetrically providing main components. The left half area is used by the user on the left side of the tablet built-in monitor 131 . The right half area is used by the user on the right side of the tablet built-in monitor 131 . With such an editing screen configuration, mainly two users can perform editing work at the same time. It should be noted that the dashed lines surrounding each area on the edit screen shown in FIG. 18 are not actually displayed on the screen.

A thumbnail display area 511L is provided above the center of the left half area of the editing screen. The thumbnail display area 511L is a display area for thumbnail images representing captured images. A thumbnail image is generated, for example, by reducing each of the shot images combined with the background image. The user (the user on the left side of the tablet built-in monitor 131) selects a captured image to be edited by selecting a thumbnail image displayed in the thumbnail display area 511L.

In the example of FIG. 18, five close-up thumbnail images P51 to P55 and one full-body thumbnail image P56 are displayed side by side in the thumbnail display area 511L.

An editing area 512L is provided substantially in the center of the left half area. The editing area 512L is a display area for a captured image to be edited, selected using a thumbnail image. The user can select an editing tool using the touch pen 132A and edit the captured image displayed in the editing area 512L.

A horizontally long editing palette 513L is displayed below the editing area 512L. The editing palette 513L includes buttons for selecting various editing tools (compositing images) such as a pen tool (pen image) and a stamp tool (stamp image) that are combined with the captured image displayed in the editing area 512L. This is the displayed area.

Above the editing palette 513L, five types of tabs used for selecting categories of editing tools are displayed. Six types of tabs used for selecting the type of editing tool are displayed on the right side of the editing palette 513L. The user can switch the types and categories of editing tools displayed on the editing palette 513L using these tabs.

A manual retouching area 514L and a makeup palette 515L are provided to the left of the editing area 512L.

The manual retouching area 514L is a display area for information used for retouching the captured image to be edited.

In the example of FIG. 18, a "slightly enlarged" button 521, a "largely enlarged" button 522, and a "from the beginning" button 523 used for retouching the size of the user's eyes are displayed in the manual retouching area 514L. there is

A “small enlargement” button 521 is a button operated when the eyes are enlarged at a predetermined magnification. The “larger enlargement” button 522 is a button operated when the eyes are enlarged at a magnification higher than that of the “smaller enlargement” button 521 . A "from the beginning" button 523 is a button operated to restore the enlarged eye size to the original size.

The makeup palette 515L is a display area of information used for content composition of the captured image to be edited.

In the example of FIG. 18, mascara button 531, lip button 532, and cheek button 533, which are used to select makeup-related content, are displayed in makeup palette 515L. A color selection button 534 used for color selection is displayed below makeup palette 515L.

The mascara button 531 is a button that is operated when performing content synthesis using eyebrow mascara, which is makeup-related content. The rip button 532 is a button that is operated when synthesizing content using rip. The cheek button 533 is a button that is operated when synthesizing content using cheek.

After operating the mascara button 531 , for example, the user can select the desired color of the eyebrow mascara using the color selection button 534 .

Thus, the editing screen is provided with an editing palette 513L used for editing operations, a manual retouching area 514L used for retouching operations, and a makeup palette 515L used for content synthesis.

The user can apply any image processing such as editing, retouching, and content composition to the captured image to be edited at any timing.

A correction tool selection area 516L is formed to the right of the edit area 512L. The correction tool selection area 516L is an area in which a button used to select an eraser tool, a "back" button, a "forward" button, a "from the beginning" button, etc. are displayed vertically.

The eraser tool is a tool used when erasing a synthesized editing tool. The "back" button and the "forward" button are buttons that are operated to return or advance the selection operation of the editing tool by one, and the "from the beginning" button cancels all the selection operations of the editing tool. This button is operated when returning to a state in which none of the editing has been performed. A "1set back" button is also displayed in the correction tool selection area 516L. By pressing the "Return 1set" button, it is possible to collectively erase the editing tools synthesized during a predetermined period of time.

On the right side of the correction tool selection area 516L, a creation button 517L that is operated when creating an image for mobile transmission is displayed. The characters "mobile phone bonus" displayed on the edit screen of FIG. 18 represent images for mobile transmission. The image for portable transmission is an image to be transmitted to the management server and is not printed on the seal paper 363 . When the creation button 517L is pressed, the creation screen of the image for mobile transmission is displayed superimposed on the edit screen.

An end button 518L is displayed above the correction tool selection area 516L to be operated when the editing process is to be ended.

In the right half area used by the user on the right side of the tablet built-in monitor 131, the same configuration as the left half area described above is arranged symmetrically.

In the example of FIG. 18, buttons used for eye size retouching are displayed, but buttons used for other types of retouching described above may be displayed on the editing screen.

In addition, although the buttons used for synthesizing the contents of eyebrow mascara, lips, and cheeks are displayed, the buttons used for synthesizing other types of contents described above may be displayed on the editing screen. .

The display of the button used for content composition and the button used for retouching may be switched according to the angle of view of the captured image selected as the edit target.

For example, when a close-up image is selected as a shot image to be edited, a button used for composition of content for the close-up image and a button used for retouching the close-up image are displayed. Along with these buttons, a button used for synthesizing content for the whole body image and a button used for retouching the whole body image may be displayed.

Also, when a whole-body image is selected as the shot image to be edited, a button used for synthesizing content for the whole-body image and a button used for retouching the whole-body image are displayed. Along with these buttons, a button used for synthesizing content for close-up images and a button used for retouching for close-up images may be displayed.

Returning to the description of FIG. 17, after the editing screen is started to be displayed, an operation acceptance control process is performed in step S32. The operation acceptance control process is a process of controlling acceptance of an image processing operation for a captured image to be edited. Details of the operation acceptance control process will be described later with reference to the flowchart of FIG.

In step S<b>33 , image processing is performed according to the operation accepted by the acceptance control unit 417 .

That is, the editing control unit 414 edits the captured image according to the user's operation.

Also, the content synthesizing unit 415 performs content synthesizing on the captured image according to the user's operation.

When the reception control unit 417 receives a retouch operation, the retouch processing unit 416 retouches the captured image in accordance with the user's operation.

In step S34, the display control unit 413 determines whether or not the editing is finished. When the end button 518L, 518R is pressed, or when the time limit has passed, it is determined that the editing is finished. If it is determined in step S34 that editing is not finished, the process returns to step S31 to continue editing using the display of the editing screen.

On the other hand, if it is determined in step S34 that the editing is finished by pressing the end buttons 518L, 518R or the like, the display control unit 413 causes the tablet built-in monitor 131 to display a division number selection screen in step S35.

The division number selection screen is a screen used to select a seal layout. A sticker layout is a layout (arrangement pattern) that determines the number, positions, and sizes of edited images to be arranged on one photo sticker. Edited images arranged according to the sticker layout selected by the user are printed on the sticker paper 363 .

In step S36, the print data generation unit 435 generates print data by arranging the edited image in the layout selected by the user. The print data is supplied to the print processing unit 404 and used to print the edited image.

In step S37, the display control unit 413 displays a guide screen on the tablet built-in monitor 131 to guide the user who has finished the editing work to move to the print waiting space A3. Guidance to move to the print waiting space A3 is also provided by outputting a voice from the speaker 341. FIG. After that, the process returns to step S4 in FIG. 16 and the subsequent processes are performed.

<Operation reception control processing>
Next, the operation acceptance control process performed in step S32 of FIG. 17 will be described with reference to the flowchart of FIG. The process of FIG. 19 is started when the user performs a predetermined operation using the display of the editing screen.

In step S51, the reception control unit 417 determines whether or not the operation performed by the user is an editing operation.

If it is determined in step S51 that the operation performed by the user is an editing operation, the acceptance control unit 417 accepts the editing operation in step S52. Information representing the content of the editing operation received here is supplied from the reception control unit 417 to the editing control unit 414 .

On the other hand, if it is determined in step S51 that the operation performed by the user is not an editing operation, in step S53 the reception control unit 417 determines whether or not the operation performed by the user is an operation for synthesizing content. judge.

If it is determined in step S53 that the operation performed by the user is an operation for synthesizing content, the acceptance control unit 417 accepts an operation for synthesizing content in step S52. Information representing the details of the content synthesis operation received here is supplied from the reception control unit 417 to the content synthesis unit 415 .

On the other hand, if it is determined in step S53 that the operation performed by the user is not content composition operation, that is, if it is determined that the operation performed by the user is retouching operation, the process proceeds to step S54. proceed to

In step S54, the reception control unit 417 determines whether or not content composition is performed first.

If it is determined in step S54 that content composition has not yet been performed, the acceptance control unit 417 accepts a retouching operation in step S52. Information representing the content of the retouching operation accepted here is supplied from the acceptance control unit 417 to the retouching processing unit 416 .

On the other hand, if it is determined in step S54 that content synthesis has been performed first, the acceptance control unit 417 does not accept retouching operations in step S55.

If an editing operation, a content composition operation, or a retouching operation is accepted in step S52, or if a retouching operation is not accepted in step S55, the process returns to step S32 in FIG. 17, and the subsequent processes are performed. will be

As described above, by restricting acceptance of retouching after content composition is performed, the user can apply well-balanced image processing to the captured image.

In addition, since the mask image needs not be regenerated, the processing load on the photo sticker creating apparatus 1 can be reduced.

<<Modification 1>>
・Example of restricting acceptance of content composition after retouching Although it was assumed that acceptance of retouching after content composition was performed was restricted, on the contrary, acceptance of content composition after retouching was restricted. You may do so.

FIG. 20 is a diagram illustrating an example of restrictions on acceptance of operations.

As shown in the upper part of FIG. 20 , when face retouching is performed as the first operation and content composition using makeup-related content is performed as the second operation, the second operation is not accepted. .

Information indicating that the operation has not been accepted is output from the acceptance control unit 417 to the display control unit 413 and reflected in the display of the edit screen, for example.

For example, the following control is performed in response to the second operation being performed.
(1) Buttons used for operating makeup-related content are displayed grayed out.
(2) Even if a button used for operating makeup-related content is pressed, the operation is not reflected in the shot image to be edited.
(3) Pop-up display of a message notifying that operation of makeup-related content cannot be performed.

Further, as shown in the lower part of FIG. 20, when retouching related to the whole body is performed as the first operation, and content composition using whole-body content is performed as the second operation, the second operation is accepted. can't

Information indicating that the operation has not been accepted is output from the acceptance control unit 417 to the display control unit 413 and reflected in the display of the edit screen, for example.

For example, the following control is performed in response to the second operation being performed.
(1) Buttons used for operating content for the whole body are grayed out.
(2) Even if a button used to operate contents of the whole body is pressed, the operation is not reflected in the shot image to be edited.
(3) Pop-up display of a message notifying that operation of system content cannot be performed.

In this way, by restricting the acceptance of content composition after retouching, the user can apply well-balanced image processing to the captured image.

<<Modification 2>>
- Example of restricting retouching after editing Acceptance of retouching after editing may be restricted.

Acceptance of post-editing retouching for close-up images is limited, for example, in the following cases.

(1) Acceptance of retouching after editing near the face is restricted.
(2) If a frame is placed near the face or if there is an area to be filtered, acceptance of subsequent retouching is restricted. If the image processing targeting the layer L5 is performed first, acceptance of retouching is restricted.

By restricting acceptance of retouching, it is possible to prevent deformation of editing results, frame synthesis results, and filter processing results.

Acceptance of post-editing retouching for whole-body images is limited, for example, in the following cases.

(1) Acceptance of retouching after editing is performed on the vicinity of the whole body is restricted.
(2) If a frame is placed near the whole body or if there is an area to be filtered, acceptance of subsequent retouching is restricted.

<<Modification 3>>
- Example of restricting editing after retouching Acceptance of editing after retouching may be restricted.

Further, acceptance of content synthesis after editing may be restricted, or acceptance of editing after content synthesis may be restricted.

<<Modification 4>>
- When the Second Operation is Exceptionally Accepted A condition for exceptionally accepting the operation of synthesizing content as the second operation may be set in advance.

For example, if the following conditions are satisfied, the content composition operation as the second operation is exceptionally accepted.

(1) If there is no change in the shape of the person region of the mask image before and after retouching for a close-up image, an operation for synthesizing contents relating to the whole body is accepted.

For example, even if retouching is performed to enlarge the eyes, the shape of the person region of the mask image used for synthesizing the whole-body content is not changed. In this case, it is possible to accept an operation for synthesizing contents relating to the whole body as the second operation, such as an aura.

(2) If there is no change in the shape of the region of the face of the mask image before and after retouching for the full-body image, the makeup-related content composition operation is accepted.

For example, even when retouching is performed to lengthen the legs, the shape of the face region of the mask image used for composition of makeup content is not changed. In this case, an operation for synthesizing makeup-related content as a second operation, such as blushing, can be accepted.

Since the degree of freedom of operation is high, the user can improve the completeness of editing.

- Example in which acceptance of the second operation is restricted according to the type of the first operation Fig. 21 is a diagram showing an example of restrictions on acceptance of the operation.

As shown in FIG. 21, whether or not to accept the second operation may be determined in advance according to the type of the first operation, regardless of the type of the second operation.

As shown in the second and third stages, when face retouching is performed as the first operation, all the second operations may be accepted or may not be accepted.

As shown in the 4th and 5th rows, when the whole body is retouched as the first operation, all the second operations may be accepted or may not be accepted.

As shown in the 6th and 7th rows, when makeup-related content composition is performed as the first operation, all the second operations may be accepted or may not be accepted.

As shown in the eighth and ninth levels, when the whole-body content composition is performed as the first operation, the second operation may be accepted entirely or may not be accepted.

<<Modification 5>>
Restrictions on acceptance when there are multiple users FIG. 22 is a diagram showing a display of part of the edit screen.

In the example of FIG. 22, it is possible to designate a person to be subjected to content synthesis or retouching from among persons appearing in the captured image.

In the case where the shot image to be edited is a close-up image, when a person to be subjected to content composition or retouching is specified, a frame image F11 is displayed on the face of the specified person as shown in FIG. . The frame image F11 represents a person who is subject to content composition or retouching.

In this way, when multiple users appear in the captured image and it is possible to designate a person for content composition or retouching, acceptance of retouching as the second operation is as follows. limited by

(1) When content is synthesized for one of a plurality of users appearing in a captured image, acceptance of retouching operations for all users is restricted.

That is, when two users are photographed, and when content is synthesized for one user, retouching operations for one user and retouching operations for the other user are performed. Acceptance of both retouching operations is restricted.

(2) When content synthesis is performed for one of multiple users appearing in a captured image, retouching is performed for the same user as the target user for content synthesis. Acceptance of operations is restricted.

In other words, when two users are photographed, and content is synthesized for one of the users, acceptance of only retouching operations for one of the users is restricted, and use of the other user is restricted. A retouching operation targeting a person is accepted. When there are multiple users, intended processing can be performed for each user.

Similarly, when the first operation is retouching and the second operation is content synthesis, acceptance of content synthesis may be restricted.

<<Modification 6>>
Screen Display Example (1) After a retouching operation is performed, the display of the editing screen may be switched so as to display buttons and the like used for operating makeup-related content.

(2) After the retouching operation is performed, the display of the editing screen may be switched so as to display buttons and the like used for operating the contents of the whole body system.

<<Modification 7>>
Although the case of restricting acceptance of operations performed in the photo sticker creating apparatus 1 has been described, acceptance of operations in a portable terminal such as a smartphone may be similarly restricted. An application that is downloaded from the management server and used for image processing of captured images is installed in the mobile terminal of the user.

(1) Example in which retouching is performed in the photo sticker creation device 1 and content synthesis is performed on a mobile terminal

Since retouching is performed as the first operation, second operations such as compositing frames, accepting filter processing, accepting edits, compositing makeup-related content, and compositing background images on the mobile terminal are accepted. Each may be restricted.

(2) Example in which contents are synthesized in the photo sticker creation device 1 and retouched on a mobile terminal

Since content composition is performed as the first operation, reception of second operations such as retouching on the face and retouching on the whole body on the mobile terminal may be restricted.

Captured images captured by mobile devices are layered and compressed, so if retouching is applied to the vicinity of the content or background, the content will be distorted and the image will look strange. become. By limiting acceptance of retouching, it is possible to prevent the image from becoming such a strange image.

(3) Both content synthesis and retouching of images taken by the photo sticker creating apparatus 1 may be performed on the mobile terminal. In this case, it is possible to restrict acceptance of the second operation as described above.

(4) The image to be transmitted to the mobile terminal may be any of a photographed image, a preview image, a retouched image, an edited image, and an image for mobile transmission.

<<Modification 8>>
When one operation is accepted and then the other operation is performed, the first operation may be canceled and the other operation performed later may be accepted.

・When content composition is performed after retouching (1) If makeup-type content composition is selected after close-up image retouching is performed, the previously accepted close-up image retouching is canceled, and is performed later. Accepts composition of selected makeup-based content.
(2) When whole-body content synthesis is selected after whole-body image retouching is performed, the previously accepted full-body image retouching is canceled, and the subsequently selected whole-body content synthesis is accepted.
(3) If content synthesis is selected after retouching, the previously accepted retouching is cancelled, and the subsequently selected content synthesis is accepted.

Contents of Control When canceling retouching as the first operation and accepting content composition as the second operation, for example, the following control is performed.
(1) When a button for synthesizing makeup-related content is pressed, retouching for a close-up image previously accepted is automatically canceled.
(2) Pop-up display a message notifying that the retouching for the close-up image previously accepted is to be cancelled.
(3) When the button for synthesizing the contents of the whole body system is pressed, the retouching for the whole body image previously received is automatically canceled.
(4) Pop-up display a message notifying that the previously accepted retouching for the whole-body image is to be cancelled.
(5) When the content composition button is pressed, the previously received retouching is automatically canceled.
(6) Display a pop-up message notifying that the previously accepted retouch will be cancelled.
(7) Display a pop-up message for allowing the user to select whether or not to cancel the previously accepted retouch.

・When retouching is performed after content composition (1) If retouching for a close-up image is selected after makeup-related content composition is performed, the previously received makeup-type content composition is canceled, and then retouched later. Accepts retouching for the selected close-up image.
(2) When retouching for a whole-body image is selected after synthesizing contents of the whole-body system, the composition of contents of the whole-body system previously accepted is canceled, and retouching for the whole-body image selected later is accepted.
(3) If retouching is selected after content synthesis is performed, the previously received content synthesis is cancelled, and the retouching selected later is accepted.

Contents of Control When canceling content synthesis as the first operation and accepting retouching as the second operation, for example, the following control is performed.
(1) When the close-up image retouching button is pressed, the composition of makeup-related content received in advance is automatically canceled.
(2) Pop-up display a message notifying that the composition of makeup-related content that has been previously accepted will be cancelled.
(3) When a button for retouching a whole-body image is pressed, the system automatically cancels the received synthesis of whole-body content.
(4) Pop-up display a message notifying that the previously accepted whole-body content synthesis is to be cancelled.
(5) When the retouch button is pressed, the previously received content composition is automatically canceled.
(6) Display a pop-up message notifying that the previously received content synthesis is cancelled.
(7) Display a pop-up message for allowing the user to select whether or not to cancel the previously received content synthesis.

<<other examples>>
The series of processes described above can be executed by hardware or by software. When a series of processes is executed by software, a program that constitutes the software is installed in a computer built into dedicated hardware or a general-purpose personal computer.

The program to be installed is provided by being recorded on removable media 319 shown in FIG. Alternatively, it may be provided via a wired or wireless transmission medium such as a local area network, the Internet, or digital broadcasting. The program can be pre-installed in the ROM 312 or storage unit 316 .

The program executed by the computer may be a program that is processed in chronological order according to the order described in this specification, or may be executed in parallel or at a necessary timing such as when a call is made. It may be a program in which processing is performed.

1 photo sticker creation device, 91 camera, 92 touch panel monitor, 131 tablet built-in monitor, 402 photographing processing unit, 403 editing processing unit, 411 photographed image acquisition unit, 412 edit target image buffer, 413 display control unit, 414 editing control unit, 415 content synthesizing unit, 416 retouch processing unit, 417 acceptance control unit

Claims (11)

Photographed image with the user as the subjecta display control unit for displaying an editing screen provided with first information for retouching operation and second information for content synthesizing operation;
using the first information an image processing unit that retouches the captured image in accordance with the user's operation;
using the second information a synthesis processing unit that synthesizes content with the captured image in accordance with the user's operation;
After accepting one operation of retouching and synthesizing the content, the other operation is not accepted,for the edit screena control unit that controls acceptance of the user's operation;
haveImage processing device. The control unit receives an operation of synthesizing the content with at least a partial area of the face of the user captured in the captured image, and then performs the retouching on the shape of the face or the shape of the part of the face. The image processing apparatus according to claim 1, wherein the operation of is not accepted. After receiving the retouching operation regarding the shape of the user's face or the shape of the part of the face appearing in the captured image, the control unit synthesizes the content with at least a part of the face. The image processing apparatus according to claim 1, wherein an operation to do is not accepted. The control unit, after receiving an operation of synthesizing the content with at least a partial region of the whole body of the user captured in the captured image, performs the retouching on the shape of the whole body or the shape of the part of the whole body. The image processing apparatus according to claim 1, wherein the operation of is not accepted. The control unit synthesizes the content with at least a partial region of the whole body after receiving the retouching operation regarding the shape of the user's whole body or the shape of the part of the whole body appearing in the captured image. The image processing apparatus according to claim 1, wherein an operation to do is not accepted. When the control unit receives the one operation targeting one of the plurality of users appearing in the photographed image, the control unit performs the other operation targeting all the users. 6. The image processing apparatus according to any one of claims 1 to 5, wherein the image processing apparatus does not accept the . When receiving the one operation targeting one of the plurality of users appearing in the photographed image, the control unit is configured to: 6. The image processing apparatus according to any one of claims 1 to 5, wherein the other operation directed to the same user is not accepted. The display control unit If one of the operations is accepted, the other operation cannot be accepted.Editing screento control the display of
Any one of claims 1 to 7Image processing device. The image processing apparatus according to claim 8, wherein the display control unit grays out the display used for the other operation, or pops up information indicating that the other operation cannot be accepted. The image processing device
Photographed image with the user as the subjectdisplaying an editing screen provided with first information for retouching operations and second information for content synthesizing operations;
using the first information retouch andusing the second informationso as not to accept the other operation after accepting one operation of content synthesis,for the edit screenControl acceptance of operations by said user
control method. to the computer,
Photographed image with the user as the subjectdisplaying an editing screen provided with first information for retouching operations and second information for content synthesizing operations;
using the first information retouch andusing the second informationso as not to accept the other operation after accepting one operation of content synthesis,for the edit screenControl acceptance of operations by said user
A program that causes an action to take place.

Images