JP2014002445A - Image process device, method and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a highly satisfactory image for a user.SOLUTION: An image process device comprises: a photographing image acquisition part that acquires a photographing image captured by photographing a subject as a user; a movement area setting part that sets a part of a subject area in the photographing image to a movement area in which the part thereof is caused to move in a direction toward an inner side of the subject area; and an area movement process part that moves an area set in the movement area in the direction toward the inner side of the subject area in the photographing image and outputs the photographing image in which the movement area is moved in the direction toward the inner side of the subject area. The present invention is applicable to, for example, a photographic seal preparation device capable of photographing an entire body of the subject.

Description

  The present invention relates to an image processing apparatus and method, and a program, and more particularly, to an image processing apparatus and method, and a program capable of providing an image with a high degree of satisfaction for a user.

  There is known a photo sticker that takes a picture of a user, causes the user to edit the shot image, and prints and provides the edited image on sticker paper. Such a photo sticker is installed in an entertainment facility.

  The flow of one game provided by the photo-seal machine usually shoots a user in the shooting space as a subject, moves the user to the editing space, edits the image according to operations performed in the editing space, The edited image, which is the edited image, is printed on a sticker sheet and discharged.

  In recent years, after editing work, during the waiting time until the edited image is printed on the sticker paper, the photographed image or edited image to be printed on the sticker paper is transmitted and stored to the server at the customer service. By accessing the server from the mobile terminal owned by the user and acquiring the stored image, the user can use the captured image or the edited image on the mobile terminal.

  In order to improve the appearance of the subject in the image provided to the user in such a series of game flows, it has been proposed to perform various image processing on the captured image.

  For example, there is a photo sticker that corrects the body shape of a subject by dividing a photographed image into a grid at regular intervals and combining the partial images in each grid by changing the vertical or horizontal enlargement / reduction magnification. (For example, refer to Patent Document 1).

  Also in fields other than photo stickers, the area of the human area in the input image is compressed in the normal direction with respect to the body axis of the subject on the input image, and the area of the background area in the input image is expanded in the normal direction. Thus, a technique for slimming the appearance of a person in an image has been proposed (see, for example, Patent Document 2).

JP 2000-267156 A JP 2011-18199 A

  However, according to Patent Documents 1 and 2 described above, although the subject is generally thin in the image, it is not always a well-balanced body shape that satisfies the user.

  The present invention has been made in view of such a situation, and makes it possible to provide an image with higher satisfaction for the user.

  An image processing apparatus according to an aspect of the present invention includes a captured image acquisition unit that acquires a captured image obtained by capturing a subject, and a direction in which at least a part of the subject region in the captured image is directed toward the inside of the subject region. A moving area setting means for setting a moving area to be moved, an area moving processing means for moving the area set as the moving area in the captured image in a direction toward the inside of the subject area, and the moving area. Output means for outputting the photographed image moved in a direction toward the inside of the subject area.

  The moving area setting means can set an area in the vicinity of the boundary between the subject area and the background area in the captured image as the moving area.

  Based on the captured image, subject mask image generation means for generating a subject mask image that transmits the subject region of the captured image, and blur processing that performs a blurring process on the subject mask image to generate a blurred subject mask image A processing unit and a gradient image generation unit that generates a gradient image of the blurred subject mask image from the pixel value of the blurred subject mask image are further provided, and the moving region setting unit includes a region having a gradient in the gradient image. At least a part of the area of the captured image corresponding to can be set as the movement area.

  The movement area setting means can set almost all areas of the subject area in the captured image as the movement area.

  Based on the captured image, subject mask image generation means for generating a subject mask image that masks an area other than the subject area of the captured image, and a distance conversion process is performed on the subject mask image, and a distance converted image is obtained. Distance conversion image generation means for generating, and gradient image generation means for generating a gradient image of the distance conversion image from the pixel values of the distance conversion image are further provided, and the moving region setting means includes a gradient in the gradient image. At least a part of the area of the captured image corresponding to a certain area can be set as the movement area. According to the distance conversion process, a portion of the subject area that is far from the background region, that is, a skeleton is extracted, and almost all the region of the subject area is moved toward the skeleton. It is possible to make the subject as a whole thin while maintaining it.

  Further provided is a face mask image generating means for generating a face mask image for masking a face area of the subject area of the photographed image based on the photographed image, and the moving area setting means is based on the face mask image. The predetermined area excluding the face in the subject area can be set as the moving area.

  The movement area setting means can set a lower body area of the subject area in the captured image as the movement area.

  Based on the captured image, subject mask image generating means for generating a subject mask image for masking an area other than the subject area of the captured image, and calculating the center of gravity of the subject area from the subject mask image Means for generating a band area mask image in which a band area of a predetermined width is set from the position of the center of gravity of the photographed image toward the foot of the subject area, and the moving area setting is provided. The means can set at least a part of the area of the photographed image corresponding to the band area in the band area mask image as the moving area.

  In the captured image, the area movement processing means moves the area set as the moving area in a direction toward the inside of the subject, and sets an area inside the area set as the moving area to the inside. The moving area can be reduced in the moving direction, and an area outside the area set as the moving area can be enlarged in the moving direction of the moving area.

  In the image processing method according to one aspect of the present invention, the image processing apparatus acquires a captured image obtained by capturing a subject, and at least a part of the subject region in the captured image is directed toward the inside of the subject region. A moving area to be moved in the direction, and in the captured image, the area set as the moving area is moved in a direction toward the inside of the subject area, and the moving area is in a direction toward the inside of the subject area. Outputting the moved photographed image.

  A program according to an aspect of the present invention is a program that causes a computer to acquire a captured image obtained by capturing a subject, and to move at least a part of the subject region in the captured image in a direction toward the inside of the subject region. The captured image in which the region set as the moving region is moved in the direction toward the inside of the subject region, and the moving region is moved in the direction toward the inside of the subject region. The process including the step of outputting is executed.

  In one aspect of the present invention, a photographed image obtained by photographing a subject is acquired, and at least a part of the subject region in the photographed image is set as a moving region that moves in a direction toward the inside of the subject region. In the image, the region set as the moving region is moved in the direction toward the inside of the subject region, and a captured image in which the moving region is moved in the direction toward the inside of the subject region is output.

  According to the present invention, it is possible to provide an image with higher satisfaction for the user.

It is a perspective view which shows the structural example of the external appearance of the photograph sticker production apparatus which concerns on one embodiment of this invention. It is the perspective view which looked at the external appearance of the photograph sticker production apparatus from another angle. It is a figure explaining a user's movement. It is a figure which shows the structural example of an imaging | photography part. It is a figure which shows the structural example of a background part. It is a figure which shows the structural example of an edit part. It is a figure which shows the structural example of a customer service part after the fact. It is a block diagram which shows the example of an internal structure of a photograph sticker production apparatus. It is a functional block diagram which shows the detailed structural example of a control part. It is a flowchart explaining the example of the flow of photograph sticker creation game processing. It is a block diagram which shows the function structural example of an image processing apparatus. It is a flowchart explaining a subject slimming process. It is a figure explaining the image produced | generated in the subject slimming process of FIG. It is a flowchart explaining a face mask image generation process. It is a figure explaining the production | generation of a face mask image. It is a figure which shows the gradient image by which the face mask image was multiplied. It is a figure explaining area movement processing. It is a figure explaining the detail of an area | region movement process. It is a figure which shows the result of a subject slimming process. It is a block diagram which shows the other function structural example of an image processing apparatus. It is a flowchart explaining a subject leg length process. It is a figure explaining calculation of the gravity center position of a photographic subject mask image. It is a figure explaining the production | generation of a belt area | region mask image. It is a figure which shows the result of a subject leg length process. It is a block diagram which shows the further another function structural example of an image processing apparatus. It is a flowchart explaining a subject slimming process. It is a figure explaining the image produced | generated in the subject slimming process of FIG. It is a figure which shows the gradient image by which the face mask image was multiplied. It is a figure which shows the result of a subject slimming process.

  Hereinafter, specific embodiments to which the present invention is applied will be described in detail with reference to the drawings.

[External structure of photo sticker creation device]
FIG. 1 is a perspective view showing a configuration example of the appearance of a photo sticker creating apparatus 1 as an image generating apparatus.

  The photo sticker creating apparatus 1 is a game machine that allows a user to perform operations such as shooting and editing as a game and receives a price from the user instead of providing a shot image or an edited image. The photo sticker creating apparatus 1 is installed in a store such as a game center. There may be one user or a plurality of users.

  A user who plays in the game provided by the photo sticker creating apparatus 1 pays the price, takes a picture of himself as a subject, and selects an image to be edited selected from the shot images obtained by shooting. Editing is performed using an editing function for synthesizing a background image or foreground image, a handwritten line drawing, or a stamp image, thereby making the photographed image rich and colorful. The user receives the sticker sheet on which the edited image is printed and ends the series of games.

  As shown in FIG. 1, the photo sticker creating apparatus 1 mainly includes a photographing unit 11, an editing unit 12, and a subsequent customer service unit 13. The photographing unit 11 and the editing unit 12 are installed in contact with each other, and the editing unit 12 and the subsequent customer service unit 13 are installed in contact with each other.

  The photographing unit 11 includes a photographing unit 21 and a background unit 22. The photographing unit 21 and the background unit 22 are installed with a predetermined distance apart, and photographing processing is performed in a photographing space that is a space formed between the photographing unit 21 and the background unit 22.

  The photographing unit 21 is a device that allows a user to perform photographing processing. The photographing unit 21 enters the photographing space and is located in front of the user who is performing the photographing process. The front panel 41 of the photographing unit 21 that constitutes a surface facing the photographing space in front is provided with a touch panel monitor used by the user during photographing processing. When the left side is the left side and the right side is the right side as viewed from the user in the imaging space, the left side of the imaging unit 21 is configured by the side panel 42A, and the right side is configured by the side panel 42B.

  The background unit 22 is attached to the left end of the rear panel 51 and the rear panel 51, which are plate-like members positioned on the rear side of the user who is performing the photographing process facing the front, and is a plate having a narrower width than the side panel 42A. The side panel 52A (not shown) is a plate-like member that is attached to the right end of the rear panel 51 and is narrower than the side panel 42B.

  The side panel 42A constituting the left side surface of the photographing unit 21 and the side panel 52A of the background unit 22 are provided on substantially the same plane, and their upper portions are connected by a connecting part 23A that is a plate-like member. Further, the side panel 42B constituting the right side surface of the photographing unit 21 and the side panel 52B of the background portion 22 are provided on substantially the same plane, and their upper portions are connected by a connecting portion 23B which is a plate-like member.

  An opening formed by being surrounded by the side panel 42 </ b> A of the photographing unit 21, the connecting part 23 </ b> A, and the side panel 52 </ b> A of the background part 22 becomes an entrance G <b> 1 of the photographing space. Although not shown, an opening formed by being surrounded by the side panel 42B of the photographing unit 21, the connecting part 23B, and the side panel 52B of the background 22 is also an entrance G2 of the photographing space.

  The background curtain unit 25 is provided above the background portion 22 so as to be supported by the back panel 51, the side panel 52A, and the side panel 52B. The background curtain unit 25 stores a plurality of winding-type background curtains having different colors or patterns and used for the background. The background curtain unit 25 operates in conjunction with photographing by a camera or the like provided in the photographing unit 21. At the time of photographing, for example, the curtain of the color selected by the user is lowered and the other curtains are wound up.

  In addition, as a background curtain accommodated in the background curtain unit 25, you may prepare the raising / lowering curtain which uses several curtains as one curtain for curtains. In addition, a black curtain is pasted on the rear panel 51, which is the back of the shooting space, and a plurality of background images for synthesis are prepared, and the background image desired by the user is captured in the shooting process and editing process. It may be possible to synthesize it into parts.

  An opening surrounded by the front panel 41 of the photographing unit 21, the connecting part 23A, the connecting part 23B, and the background curtain unit 25 is formed above the photographing space, and the ceiling strobe unit 24 covers the part of the opening. Provided. One end of the ceiling strobe unit 24 is fixed to the connecting portion 23A, and the other end is fixed to the connecting portion 23B. The ceiling strobe unit 24 has a built-in strobe that emits light toward the photographing space in accordance with photographing. A fluorescent lamp is provided inside the strobe constituting the ceiling strobe unit 24, and also functions as illumination of the photographing space.

  The editing unit 12 is a device that allows a user to perform editing processing, which is processing for editing an image obtained by shooting processing. The editing unit 12 is provided so as to be connected to the photographing unit 11 so that one side surface thereof is in contact with the back surface of the front panel 41 of the photographing unit 21. The editing unit 12 is provided with a configuration such as a tablet built-in monitor used by the user during editing processing.

  If the configuration of the editing unit 12 shown in FIG. 1 is a front-side configuration, the editing unit 12 is used for editing processing on the front side and the back side so that two sets of users can perform editing processing simultaneously. Each configuration is provided.

  The front side of the editing unit 12 is a surface perpendicular to the floor surface, and is composed of a surface 71 that is substantially parallel to the side panel 42A, and a slope 72 formed above the surface 71. The configuration used is provided on the slope 72. On the left side of the slope 72, a support portion 73A that has a columnar shape and supports one end of the lighting device 74 is provided. A support portion 75 that supports the curtain rail 26 is provided on the upper surface of the support portion 73A. A support portion 73 </ b> B (FIG. 2) that supports the other end of the lighting device 74 is also provided on the right side of the slope 72.

  A curtain rail 26 is attached above the editing unit 12. The curtain rail 26 is configured by combining three rails 26A to 26C so that the shape when viewed from above is a U-shape. One end of the rails 26A and 26B provided in parallel is fixed to the connecting portion 23A and the connecting portion 23B, respectively, and both ends of the remaining one rail 26C are joined to the other end.

  A curtain is attached to the curtain rail 26 so that the inside of the space in front of the editing unit 12 and the space in front of the back cannot be seen from the outside. The space in front of the editing unit 12 and the space in front of the back of the editing unit 12 surrounded by the curtain attached to the curtain rail 26 are editing spaces in which the user performs editing processing.

  FIG. 2 is a perspective view of the photo sticker creating apparatus 1 as seen from another angle.

  The subsequent customer service unit 13 is a device that allows the user to perform the subsequent customer service process. Subsequent customer service processing includes processing for transmitting a photographed image or edited image to a mobile terminal such as a mobile phone, processing for allowing a user to play a mini game, processing for answering a questionnaire, and the like.

  As shown in FIG. 2, the post-service unit 13 has a plate-shaped housing having a predetermined thickness as a part thereof, and the remaining part is provided integrally with the left side surface of the editing unit 12. On the front side of the post-service unit 13, there are provided a tablet built-in monitor used by the user at the time of post-service processing, a discharge port through which sticker paper on which a photographed image or an edited image is printed is discharged. The space in front of the front-end customer service unit 13 becomes a rear-end customer service space where a user waiting for printing on the sticker paper to perform post-service processing.

  Here, the flow of the photo sticker creation game and the accompanying movement of the user will be described with reference to FIG. FIG. 3 is a plan view of the photo sticker creating apparatus 1 as viewed from above.

  The user of the photo sticker creating apparatus 1 is formed between the photographing part 21 and the background part 22 from the entrance G1 as indicated by the white arrow # 1 or from the entrance G2 as indicated by the white arrow # 2. The photographing space A1 is entered, and photographing processing is performed using a camera or a touch panel monitor provided in the photographing unit 21.

  The user who has finished the shooting process exits the shooting space A1 and moves to the editing space A2-1 using the doorway G1 as indicated by the white arrow # 3, or moves to the editing space A2-1 as indicated by the white arrow # 4. To leave the shooting space A1 and move to the editing space A2-2.

  The editing space A2-1 is an editing space on the front side of the editing unit 12, and the editing space A2-2 is an editing space on the back side of the editing unit 12. Which of the editing space A2-1 and the editing space A2-2 is moved is guided by the screen display of the touch panel monitor of the photographing unit 21 or the like. For example, the empty space of the two editing spaces is guided as the destination. The user who has moved to the editing space A2-1 or the editing space A2-2 starts editing processing. The user of the editing space A2-1 and the user of the editing space A2-2 can perform editing processing at the same time.

  After the editing process is finished, printing of an image selected from the photographed image and the edited image is started. During printing of the image, the user who has finished the editing process moves from the editing space A2-1 to the subsequent customer service space A3 as indicated by the white arrow # 5 if the editing process has been performed in the editing space A2-1. Then, the customer service is processed after the fact. In addition, when the editing process is completed in the editing space A2-2, the user who has finished the editing process moves from the editing space A2-2 to the post-service area A3 as indicated by the white arrow # 6. Post service processing is performed in the customer service space A3.

  When the printing of the image is finished, the user receives the sticker paper from the outlet provided in the post-service section 13 and finishes a series of photo sticker creation games.

  Next, the configuration of each device will be described.

[Configuration of shooting section]
FIG. 4 is a diagram illustrating a configuration example of the front of the photographing unit 21. The photographing unit 21 is configured by attaching a front panel 41, a side panel 42A, and a side panel 42B to a base unit 43 having a box shape.

  The front panel 41 is positioned in front of the user who performs the shooting process in the shooting space A1, and the side panel 42A and the side panel 42B are positioned on the left side and the right side of the user who performs the shooting process in the shooting space A1, respectively.

  An imaging / display unit 81 is provided in the approximate center of the front panel 41. The photographing / display unit 81 includes a camera 91, a front strobe 92, and a touch panel monitor 93.

  The camera 91 is configured by an image sensor such as a CCD (Charge Coupled Device), and photographs a user in the photographing space A1. The moving image captured by the camera 91 is displayed on the touch panel monitor 93 in real time. An image captured by the camera 91 at a predetermined timing such as when a user gives an instruction to shoot is saved as a photographic image (still image).

  The front strobe 92 provided with a light emitting surface so as to surround the camera 91 emits light in accordance with the photographing by the camera 91 like other strobes, and irradiates the vicinity of the user's face as a subject from the front.

  A touch panel monitor 93 provided on the lower side of the camera 91 includes a monitor such as an LCD (Liquid Crystal Display) and a touch panel laminated thereon. The touch panel monitor 93 has a function as a live view monitor that displays a moving image captured by the camera 91 and a function that displays various GUI (Graphical User Interface) images and accepts a user's selection operation through the touch panel. ing. On the touch panel monitor 93, a still image as a photographing result and a moving image after an image such as a background is combined are displayed as appropriate.

  On the front panel 41, an upper strobe 82 is installed on the upper side with respect to the position of the photographing / display unit 81. A left strobe 83 is installed on the left side and a right strobe 84 is installed on the right side.

  The upper strobe 82 irradiates the user from the upper front. The left strobe 83 irradiates the user from the left front, and the right strobe 84 irradiates the user from the right front.

  The base unit 43 is also provided with a foot strobe 86 that irradiates the user's foot. Fluorescent lamps are provided inside the upper strobe 82, the left strobe 83, the right strobe 84, and the foot strobe 86, and together with the fluorescent lights inside the strobe constituting the ceiling strobe unit 24, the inside of the photographing space A1. Used as lighting. By adjusting the light emission amount of each fluorescent lamp, or by adjusting the number of fluorescent lamps to emit light, the brightness in the shooting space A1 depends on the content of the shooting process performed by the user. Adjust as appropriate.

  On the right side of the foot strobe 86, a coin insertion / return port 87 through which a user inserts money is provided.

  Spaces 43A and 43B formed on the left and right sides of the upper surface of the base portion 43 are used as a luggage storage place for a user who performs a photographing process to place baggage and the like. For example, near the ceiling of the front panel 41, a speaker for outputting sound such as guidance sound for shooting processing, BGM (Back Ground Music), and sound effects is also provided.

[Configuration of the background]
FIG. 5 is a diagram illustrating a configuration example of the background portion 22 on the photographing space A1 side.

  As described above, the background curtain unit 25 is provided above the back panel 51. Near the center of the background curtain unit 25, a back center strobe 101 that irradiates a user who is performing a photographing process in the photographing space A1 from the rear center is attached.

  A rear left strobe 102 that irradiates a user who is performing a photographing process in the photographing space A1 from the left rear is provided above the rear panel 51 and on the doorway G1 side. Further, a rear right strobe 103 that irradiates a user who is performing a photographing process in the photographing space A1 from the right rear is provided above the rear panel 51 and on the entrance / exit G2 side.

[Configuration of editorial department]
FIG. 6 is a diagram illustrating a configuration example of the editing unit 12 on the front side (editing space A2-1 side).

  A tablet built-in monitor 131 is provided at substantially the center of the slope 72. A touch pen 132A is provided on the left side of the tablet built-in monitor 131, and a touch pen 132B is provided on the right side.

  The tablet built-in monitor 131 is configured by superimposing a tablet capable of operation input using the touch pen 132A or 132B on a monitor such as an LCD. On the tablet built-in monitor 131, for example, an editing screen that is a screen used for editing a captured image selected as an image to be edited is displayed. When two people perform editing work simultaneously, the touch pen 132A is used by one user, and the touch pen 132B is used by the other user.

  On the left side of the editing unit 12, a post-service unit 13 is provided.

[Composition of the customer service department]
FIG. 7 is a diagram illustrating a configuration example of the front side of the post-service unit 13.

  A monitor 161 with a built-in tablet is provided at the upper center of the post-service section 13. The tablet built-in monitor 161 is configured by superimposing a tablet capable of operation input with a user's finger or the like on a monitor such as an LCD. On the tablet built-in monitor 161, a screen used for post-service processing such as a mini game is displayed.

  A speaker 162 is provided below the tablet built-in monitor 161, and a sticker paper discharge port 163 is provided below the speaker 162.

  The speaker 162 outputs sound such as guidance voice, background music, and sound effects for the post-service processing.

  In the sticker paper outlet 163, sticker paper on which an image created by the user of the editing space A2-1 is edited or printed by the user of the editing space A2-2 is edited. The sticker paper on which the created image is printed is discharged. A printer is provided inside the editing unit 12, and printing of images is performed by the printer.

[Internal configuration of photo sticker creation device]
Next, the internal configuration of the photo sticker creating apparatus 1 will be described. FIG. 8 is a block diagram illustrating an internal configuration example of the photo sticker creating apparatus 1. The same code | symbol is attached | subjected to the same structure as the structure mentioned above. The overlapping description will be omitted as appropriate.

  The control unit 201 includes a CPU (Central Processing Unit) and the like, executes a program stored in a ROM (Read Only Memory) 206 and a storage unit 202, and controls the overall operation of the photo sticker creating apparatus 1. A storage unit 202, a communication unit 203, a drive 204, a ROM 206, and a RAM (Random Access Memory) 207 are connected to the control unit 201. The control unit 201 is also connected with each configuration of the photographing unit 208, the editing unit 209, and the post-service unit 210.

  The storage unit 202 includes a non-volatile storage medium such as a hard disk or a flash memory, and stores various setting information supplied from the control unit 201. Information stored in the storage unit 202 is appropriately read out by the control unit 201.

  A communication unit 203 is an interface of a network such as the Internet, and communicates with an external device according to control by the control unit 201.

  A removable medium 205 made of an optical disk, a semiconductor memory, or the like is appropriately attached to the drive 204. Computer programs and data read from the removable medium 205 by the drive 204 are supplied to the control unit 201 and stored in the storage unit 202 or installed.

  The ROM 206 stores programs and data executed by the control unit 201. The RAM 207 temporarily stores data and programs processed by the control unit 201.

  The imaging unit 208 includes a coin processing unit 221, a background control unit 222, a lighting device 223, a camera 91, a touch panel monitor 93, and a speaker 224 that are configured to perform imaging processing in the imaging space A1.

  The coin processing unit 221 detects the insertion of coins into the coin insertion / return port 87. When the coin processing unit 221 detects that a predetermined amount of coins such as 400 yen has been inserted, the coin processing unit 221 outputs an activation signal indicating that to the control unit 201.

  The background control unit 222 raises and lowers the background curtain stored in the background curtain unit 25 according to the background control signal supplied from the control unit 201. The background curtain may be manually selected by the user.

  The illumination device 223 is a fluorescent lamp provided inside each strobe in the imaging space A1, and emits light according to an illumination control signal supplied from the control unit 201. As described above, in the shooting space A 1, in addition to the strobe of the ceiling strobe unit 24, the upper strobe 82, the left strobe 83, the right strobe 84, the foot strobe 86 and the background unit 22 provided in the photographing unit 21 are provided. A back center strobe 101, a back left strobe 102, and a back right strobe 103 are provided.

  Further, the lighting device 223 adjusts the brightness in the shooting space A1 according to the stage of the shooting work performed by the user by adjusting the light emission amount according to the control by the control unit 201.

  The camera 91 shoots according to the control by the control unit 201 and outputs an image obtained by the shooting to the control unit 201.

  The editing unit 209A includes a tablet built-in monitor 131, touch pens 132A and 132B, and a speaker 231 provided on the front side of the editing unit 12 as a configuration for performing editing processing in the editing space A2-1. The editing unit 209B has the same configuration as the editing unit 209A, and performs editing processing in the editing space A2-2.

  The tablet built-in monitor 131 displays an editing screen according to control by the control unit 201, and detects a user operation on the editing screen. A signal representing the content of the user's operation is supplied to the control unit 201, and the captured image to be edited is edited.

  The subsequent customer service unit 210 includes a tablet built-in monitor 161 that is configured to perform the subsequent customer service in the subsequent customer service space A3, a speaker 162, a printer 241 that is configured to perform the printing process, and a sticker sheet unit 242.

  The printer 241 prints the photographed image or the edited image obtained by the editing process on the sticker paper stored in the sticker paper unit 242 attached to the printer 241 and discharges it to the sticker paper discharge port 163.

[Configuration of control unit]
Next, the control unit 201 will be described. FIG. 9 shows a configuration example of functional blocks realized by the control unit 201 executing a program stored in the ROM 206 or the like.

  The control unit 201 performs a process related to a charge input when starting the photo sticker creation game and a process related to a photographing work process of the photo sticker creation game such as photographing a user group. An editing processing unit 302 that performs processing related to the editing work process of the photo sticker creation game such as graffiti editing, a printing processing unit 303 that performs processing related to the printing process of the photo sticker creation game such as printing of sticker paper, and editing work. A post-service processing unit 304 is provided for performing processing related to the post-service process of the photo sticker creation game that serves the finished user group.

  That is, the control unit 201 controls processing related to each process of the photo sticker creation game.

[Operation of photo sticker creation device]
Next, processing of the photo sticker creating apparatus 1 that provides a photo sticker creating game will be described with reference to the flowchart of FIG.

  In step S1, the imaging processing unit 301 of the photo sticker creating apparatus 1 determines whether or not a predetermined amount of coins has been inserted based on a signal supplied from the coin processing unit 221 until it is determined that the coins have been inserted. stand by.

  If it is determined in step S1 that a coin has been inserted, in step S2, the imaging processing unit 301 controls the imaging unit 208 to display a moving image of the subject captured by the camera 91 on the touch panel monitor 93 in a live view. Then, a photographing process for photographing a user in the photographing space A1 as a subject is performed. In the photographing process, for example, whole body photographing for photographing the whole body of the user and up photographing for photographing the user's face or upper body are performed.

  In step S3, the imaging processing unit 301 guides the user in the imaging space A1 to move to the editing space A2-1 or the editing space A2-2. The guidance for movement to the editing space A2-1 or the editing space A2-2 is performed by displaying a screen on the touch panel monitor 93 of the photographing unit 208 or by outputting sound from the speaker 224.

  In step S4, the editing processing unit 302 controls the editing unit 209 corresponding to the editing space of the editing space A2-1 and the editing space A2-2 that is the destination of the user who has finished the shooting process, Edit processing. Specifically, the editing processing unit 302 causes the user to select a synthesis image to be synthesized with the captured image selected as the editing target image, synthesizes the selected synthesis image with the captured image, and obtains the obtained synthesis. The image is edited according to the editing operation by the user.

  In step S5, the editing processing unit 302 guides the user who has finished the editing process in the editing space A2-1 or the editing space A2-2 to move to the post-service space A3. The guidance of the movement to the post-service space A3 is performed by displaying a screen on the tablet built-in monitor 131 or by outputting sound from the speaker 231.

  In step S <b> 6, the print processing unit 303 performs (starts) a print process in which the image selected by the user is output to the printer 241 and printed on a sticker sheet.

  Further, when the post-service processing start button displayed on the tablet built-in monitor 161 is selected, in step S7, the post-service processing unit 304 performs post-service processing for a user who is waiting for completion of printing. Specifically, the post-service processing unit 304 performs a mobile transmission game process for transmitting a captured image or an edited image to the mobile terminal as the post-service processing, and directly inputs the e-mail address of the mobile terminal. Display on the tablet built-in monitor 161 a selection screen for a transmission method that allows a user to select whether to transmit an image to a portable terminal by transmitting to a portable terminal or by communicating with a non-contact type IC. The image is transmitted according to the operation by the user.

  When printing is completed, in step S8, the customer service processing unit 304 displays that the printing is completed on the tablet built-in monitor 161, and the printer 241 discharges the sticker paper on which the image is printed to the sticker paper discharge port 163. The process is terminated.

  In addition, the photo sticker creating apparatus 1 can improve the appearance of the subject in the captured image by performing predetermined image processing on the captured image.

[Functional configuration example of image processing apparatus]
Here, a functional configuration example of an image processing apparatus that performs image processing for improving the appearance of a subject in a captured image will be described with reference to FIG. The image processing apparatus shown in FIG. 11 is realized, for example, in the photographing processing unit 301 of the photographic sticker 1 described above.

  The image processing apparatus in FIG. 11 includes a captured image acquisition unit 411, a subject mask image generation unit 412, a blur processing unit 413, a gradient image generation unit 414, a face mask image generation unit 415, a movement region setting unit 416, and a region movement processing unit. 417.

  The captured image acquisition unit 411 acquires a captured image that is an image captured by the camera 91 and supplies the captured image to the subject mask image generation unit 412, the face mask image generation unit 415, and the region movement processing unit 417.

  The subject mask image generation unit 412 separates a subject region that is a subject region in the photographed image and a background region that is a background region other than the subject region from the photographed image supplied from the photographed image acquisition unit 411, A subject mask image is generated that has undergone binarization processing in which the pixels in the region are white pixels and the pixels in the background region are black pixels. The generated subject mask image is supplied to the blur processing unit 413. The subject mask image generated by the subject mask image generation unit 412 is a mask image that masks a region other than the subject region, that is, the background region.

  The blurring processing unit 413 performs a filtering process having a blurring effect on the subject mask image supplied from the subject mask image generating unit 412, and supplies the resulting blurred subject mask image to the gradient image generating unit 414. To do.

  The gradient image generation unit 414 generates a gradient image from the pixel value of the blurred subject mask image supplied from the blur processing unit 413 and supplies the gradient image to the movement region setting unit 416. The gradient image is an image indicating the degree of change in the pixel value of the blurred subject mask image with respect to a predetermined direction.

  The face mask image generation unit 415 divides a face area, which is the face area of the subject in the photographed image, from a photographed image supplied from the photographed image acquisition unit 411, and an area other than the face area. The face mask image is generated by performing binarization processing with the white pixels as white pixels and the pixels in the face area as black pixels. The generated face mask image is supplied to the movement area setting unit 416. The face mask image generated by the face mask image generation unit 415 is a mask image that masks the face area.

  Based on the gradient image supplied from the gradient image generation unit 414 and the face mask image supplied from the face mask image generation unit 415, the movement area setting unit 416 converts at least a part of the subject area in the captured image to the subject. The movement area to be moved in the direction toward the inner side of the area is set, and information representing the movement area is supplied to the area movement processing unit 417.

  Based on the information supplied from the movement area setting unit 416, the area movement processing unit 417 determines the area set as the movement area in the captured image supplied from the captured image acquisition unit 411 in a direction toward the inside of the subject. And the result is supplied to the touch panel monitor 93. The touch panel monitor 93 outputs (displays) the image supplied from the area movement processing unit 417 as a still image as a photographing result. The output captured image is a target of editing processing by the editing processing unit 302.

[Subject slimming process]
Next, the subject slimming process for improving the appearance of the subject in the captured image, which is executed by the image processing apparatus in FIG. 11, will be described with reference to the flowchart in FIG.

  In step S <b> 11, the captured image acquisition unit 411 acquires a captured image that is an image captured by the camera 91. Specifically, a photographed image obtained by photographing a person standing in front of the wall surface with the background curtain stored in the background curtain unit 25 as a background is acquired. Note that the acquired captured image is a whole body image obtained by performing whole body imaging.

  In step S <b> 12, the subject mask image generation unit 412 performs a chroma process, a skin region detection process, and the like on the captured image acquired by the captured image acquisition unit 411, so that the subject region and the background region are detected in the captured image. The subject mask image subjected to binarization processing is generated.

  Specifically, as shown first from the left in FIG. 13, a captured image 431 in which a synthesis image such as a background image is not synthesized, and the background color is the color of the curtain for chromach. Is acquired by the captured image acquisition unit 411, the subject mask image generation unit 412 sets the pixel value of the pixel in the subject region to (R, G, B) = (255,255,255) for the captured image 431 (ie, , Subject area pixel is white pixel), background area pixel value is (R, G, B) = (0,0,0) (that is, background area pixel is black pixel), A subject mask image 432 shown second from the left in FIG. 13 is generated. In the subject mask image 432, the amount of change in the pixel value is large at the boundary between the subject region and the background region.

  In step S13, the blurring processing unit 413 performs blurring processing for blurring the subject mask image 432 supplied from the subject mask image generating unit 412 using, for example, a low-pass filter. As a result, a blurred subject mask image 433 shown third from the left in FIG. 13 is obtained and supplied to the gradient image generation unit 414. In the blurred subject mask image 433 in FIG. 13, the change amount of the pixel value is smoothed at the boundary between the subject region and the background region. Note that the width of the moving area, which will be described later, can be adjusted by the degree of blurring in this blurring process.

  In step S14, the gradient image generation unit 414 determines the gradient image in the X direction (horizontal right direction in FIG. 13) and the Y direction (FIG. 13) from the pixel value of the blurred subject mask image supplied from the blur processing unit 413. A gradient image in the middle and vertical direction is generated.

  Specifically, first, the gradient image generation unit 414 sets pixel values of pixels (that is, black pixels) whose pixel values are smaller than (R, G, B) = (128,128,128) in the blurred subject mask image 433, for example. As (R, G, B) = (128, 128, 128), the background area of the blurred subject mask image 433 is made gray. Next, the gradient image generation unit 414 performs spatial first-order differentiation in the X direction on the blurred subject mask image 433 using, for example, a Sobel filter, and is shown in the fourth upper stage from the left in FIG. An X-direction gradient image 434X is generated, and a spatial first-order differentiation is performed in the Y direction to generate a Y-direction gradient image 434Y shown in the fourth lower stage from the left in FIG.

  In the X-direction gradient image 434X in FIG. 13, a region (black region) shown in black indicates a region in which the gradient (gradient) of change in the pixel value with respect to the X direction is positive in the blurred subject mask image 433, and is shown in white. The region to be displayed (white region) indicates a region where the gradient (gradient) of the change in the pixel value with respect to the X direction in the blurred subject mask image 433 is negative.

  In the Y-direction gradient image 434Y of FIG. 13, the black region indicates a region in which the gradient (gradient) of the change in the pixel value with respect to the Y direction is positive in the blurred subject mask image 433, and the white region indicates the blurred subject. In the mask image 433, an area in which the gradient (gradient) of the change of the pixel value with respect to the Y direction is negative is shown.

  That is, in the X-direction gradient image 434X and the Y-direction gradient image 434Y of FIG. 13, the black region and the white region are regions having a gradient of pixel values in the blurred subject mask image 433, and the other regions are the blurred subject mask. In the image 433, a region having no pixel value gradient is shown.

  Returning to the flowchart of FIG. 12, in step S15, the face mask image generation unit 415 executes a face mask image generation process to generate a face mask image.

  Here, the details of the face mask image generation processing will be described with reference to the flowchart of FIG.

  In step S <b> 31, the face mask image generation unit 415 performs face recognition processing on the captured image acquired by the captured image acquisition unit 411.

  In step S <b> 32, the face mask image generation unit 415 generates a face mask image that masks the face area of the subject in the photographed image based on the result of face recognition processing on the photographed image.

  Specifically, as shown first from the left in FIG. 15, when face authentication processing is performed on the captured image 431 and, as a result, rectangular face regions f1 and f2 are obtained, a face mask is obtained. The image generation unit 415 obtains a circle (or an ellipse) inscribed in the face regions f1 and f2, and generates a face mask image 451 as shown second from the left in FIG. In the face mask image 451, the face mask m1 is set from the face area f1, and the face mask m2 is set from the face area f2.

  Returning to the flowchart of FIG. 14, in step S <b> 33, the face mask image generation unit 415 performs a blurring process for blurring the face mask image 451 using, for example, a low-pass filter. As a result, a blurred face mask image 452 obtained by performing the blurring process on the face masks m1 ′ and m2 ′ shown third from the left in FIG. 15 is obtained and supplied to the moving region setting unit 416. After step S33, the process returns to step S15 in the flowchart of FIG.

  In step S <b> 16, the moving area setting unit 416 multiplies the gradient image supplied from the gradient image generation unit 414 by the blurred face mask image supplied from the face mask image generation unit 415. As a result, an X-direction gradient image 471X in which the black region and the white region are deleted from the region corresponding to the subject's face shown on the left side of FIG. 16, and the region corresponding to the subject's face shown on the right side of FIG. A Y-direction gradient image 471Y from which the black area and the white area are deleted is obtained.

  Here, in the captured image, regions corresponding to the black region and the white region in the X-direction gradient image 471X and the Y-direction gradient image 471Y are regions including the boundary between the subject region and the background region.

  Based on the X-direction gradient image 471X and the Y-direction gradient image 471Y obtained in this way, the moving area setting unit 416, in the captured image, an area corresponding to at least a part of the black area and the white area, specifically, Then, a region near the boundary between the subject region and the background region is set as a moving region that moves in the direction toward the inside of the subject region, and information representing the moving region is supplied to the region movement processing unit 417. Here, the direction toward the inside of the subject area refers to a direction away from the background area in the subject area.

  In step S <b> 17, the area movement processing unit 417 determines, based on the information supplied from the movement area setting unit 416, the subject area and the background of the image corresponding to the black area and the white area of the gradient image in the captured image. An image in a region near the boundary with the region is moved by a predetermined movement amount in a direction toward the inside of the subject.

  FIG. 17 is a diagram illustrating an area corresponding to a part of the black area of the X-direction gradient image 471X in the captured image. In FIG. 17, an area sandwiched between the line a and the line a ′ is an area corresponding to a part of the black area of the X-direction gradient image 471X, and the line b is the boundary between the subject area and the background area in the captured image. Is shown. Thus, in the captured image, the region corresponding to the black region of the X-direction gradient image 471X straddles the subject side and the background side across the boundary.

  In the example of FIG. 17, in the captured image, an area in the vicinity of the line b (the boundary between the subject area and the background area) among the areas sandwiched between the line a and the line a ′ (area corresponding to the black area) is moved. The area movement processing unit 417 moves the image of the movement area in the X direction (horizontal right direction in FIG. 17).

  Subsequently, in step S18, the region movement processing unit 417 enlarges the image of the region outside the image of the moved region among the images of the region corresponding to the black region and the white region of the gradient image in the captured image, The image in the area inside the image of the moved area is reduced.

  That is, in the example of FIG. 17, in the captured image, an area and a line moved in the X direction (right direction in FIG. 17) among areas sandwiched between the line a and the line a ′ (area corresponding to the black area). A region sandwiched between a and a is expanded in the X direction (horizontal direction), and a region sandwiched between the region moved in the X direction and the line a ′ is contracted in the X direction (horizontal direction).

  Here, with reference to FIG. 18, the movement and enlargement / reduction processing of the area described in FIG. 17 will be described in more detail.

  FIG. 18A shows a pixel block in the vicinity of a line b (a boundary between the subject area and the background area) set as a movement area by the X-direction gradient image 471X. Here, the pixel block is a collection of N × N pixels, where N is an arbitrary integer. In FIG. 18, attention is paid to one column of pixel blocks arranged in the X direction (horizontal direction). That is, in FIG. 18A, two pixel blocks centered on the line b are set in the movement region.

  FIG. 18B shows a state of movement of the pixel block set in the movement area. In FIG. 18B, the pixel blocks in the movement region in FIG. 18A are moved in the X direction (horizontal right direction in the figure) by a predetermined movement amount (for example, three pixel blocks). As a result, the line b (the boundary between the subject region and the background region) is moved in the right direction in the drawing, that is, inside the subject.

  FIG. 18C shows a state of enlargement / reduction of pixel blocks other than the moving region. In FIG. 18C, the pixel block outside (left side in the figure) outside the pixel block in the moving area shown by shading is enlarged in the X direction (horizontal direction) and inside the pixel block in the moving area shown by shading (see FIG. 18C). The middle right pixel block is reduced in the X direction (horizontal direction). The enlargement ratio of the pixel block to be enlarged increases as the distance from the background side (line a side) and the pixel block side (line b side) of the moving region increases. Further, the reduction ratio of the pixel block to be reduced increases as the distance from the subject side (line a ′ side) and the pixel block side (line b side) of the moving region increases.

  In the example of FIGS. 17 and 18, the movement and enlargement / reduction processing of the region corresponding to the black region of the X-direction gradient image 471X in the captured image has been described. However, the processing corresponds to the white region of the X-direction gradient image 471X. In the area to be moved, the pixel block in the moving area is moved in the −X direction (horizontal left direction). Also, the pixel block outside (right side) of the pixel block in the moving area is enlarged in the X direction (horizontal direction), and the pixel block inside (left side) of the pixel block in the moving area is reduced in the X direction (horizontal direction). Is done.

  Further, in the area corresponding to the black area of the Y-direction gradient image 471Y, the pixel block in the moving area is moved in the Y direction (vertically downward). Further, the pixel block outside (upper) from the pixel block in the moving area is enlarged in the Y direction (vertical direction), and the pixel block inside (lower) from the pixel block in the moving area is expanded in the Y direction (vertical direction). Reduced.

  In the region corresponding to the white region of the Y-direction gradient image 471Y, the pixel block in the moving region is moved in the −Y direction (vertical upward direction). In addition, the pixel block outside (lower) the pixel block in the moving area is enlarged in the Y direction (vertical direction), and the pixel block inside (upper) from the pixel block in the moving area is expanded in the Y direction (vertical direction). Reduced.

  In this way, in the captured image 431, the boundary portion of the subject excluding the face moves in the direction toward the inside of the subject (the direction away from the background), so that the subject as shown in FIG. A captured image 491 in which the body is slightly smaller (thinned) is displayed on the touch panel monitor 93.

  Note that the area movement process in step S17 and the area enlargement / reduction process in step S18 may be performed simultaneously (in parallel).

  According to the above processing, the area in the vicinity of the boundary between the subject area and the background area excluding the face in the photographed image is moved in the direction toward the inside of the subject area. The subject's body becomes smaller (thinner). This makes it possible to obtain a natural whole body image that looks cute and cute without making the face too small or too tall, and provides a more satisfying image for the user. It becomes possible.

  In the above, the subject's body is made smaller in the captured image to improve the appearance of the subject, but in the following, the subject's appearance is improved by making the subject's legs longer. An example will be described.

[Another functional configuration example of the image processing apparatus]
FIG. 20 shows an example of the functional configuration of an image processing apparatus that performs image processing for improving the appearance of a subject in a captured image by lengthening the leg of the subject.

  The image processing apparatus in FIG. 20 includes a captured image acquisition unit 511, a subject mask image generation unit 512, a center of gravity calculation unit 513, a band area mask image generation unit 514, a movement area setting unit 515, and an area movement processing unit 516. .

  In the image processing apparatus of FIG. 20, the captured image acquisition unit 511 and the subject mask image generation unit 512 have the same functions as the captured image acquisition unit 411 and the subject mask image generation unit 412 in the image processing apparatus of FIG. Therefore, the description is omitted.

  The center-of-gravity calculation unit 513 calculates the position of the center of gravity of the subject area from the subject mask image supplied from the subject mask image generation unit 512, and sends position information (coordinates) representing the position of the center of gravity to the band region mask image generation unit 514. Supply.

  The band area mask image generation unit 514 uses the position information supplied from the centroid calculation unit 513 to set a band area having a predetermined width from the centroid position of the subject area toward the leg of the subject area in the photographed image. A mask image is generated and supplied to the movement area setting unit 515.

  Based on the band area mask image supplied from the band area mask image generation unit 514, the movement area setting unit 515 is a movement area that moves at least a part of the subject area in the captured image in a direction toward the inside of the subject area. The information indicating the movement area is set and supplied to the area movement processing unit 516.

  Based on the information supplied from the movement area setting unit 515, the area movement processing unit 516 moves the area set as the movement area in the captured image supplied from the captured image acquisition unit 511 in the direction toward the inside of the subject. And supplied to the touch panel monitor 93. The touch panel monitor 93 outputs (displays) the image supplied from the area movement processing unit 417 as a still image as a photographing result.

[Subject leg length processing]
Next, the subject leg length process for improving the appearance of the subject of the captured image, which is executed by the image processing apparatus of FIG. 20, will be described with reference to the flowchart of FIG.

  Note that the processing of steps S111 and S112 in the flowchart of FIG. 21 is the same as the processing of steps S11 and S12 in the flowchart of FIG.

  That is, in step S113, the center-of-gravity calculation unit 513 calculates the position of the center of gravity of the subject area from the subject mask image supplied from the subject mask image generation unit 512, and obtains position information (coordinates) representing the position of the center of gravity as the band region. This is supplied to the mask image generation unit 514.

  Specifically, when the subject mask image 531 as shown on the left side of FIG. 22 is generated by the subject mask image generation unit 512, the centroid calculation unit 513 uses a predetermined calculation formula to calculate the right side of FIG. As shown in FIG. 4, the coordinates g (gx, gy) of the center of gravity of the subject area (white pixel area) of the subject mask image 531 are calculated.

  As shown in FIG. 22, when there are a plurality of subjects, the centroid position is calculated for each subject area of the subject, and the average is set as the final centroid position. Further, the barycentric position may be calculated only in the Y direction (vertical direction).

  In step S114, the band area mask image generation unit 514 uses the position information supplied from the center of gravity calculation unit 513 to generate a band area having a predetermined width from the position of the center of gravity of the subject area toward the leg of the subject area. A set band area mask image is generated and supplied to the movement area setting unit 515.

  Specifically, first, the band area mask image generation unit 514 prepares a black area image 551 having the same size as the photographed image and only the black area as shown first from the left in FIG. Next, the band area mask image generation unit 514 puts the coordinates g (gx, gy) of the center of gravity of the subject area of the subject mask image 531 on the black area image 551 as shown in the second from the left in FIG. A band area mask image 552 is set, in which a band area (white area) having a predetermined width is set from the coordinates g (gx, gy) to the legs of the subject area, that is, the lower side in the figure. In the band area mask image 552, the amount of change in pixel value is large at the boundary between the band area (white area) and other areas.

  In step S115, the moving area setting unit 515 performs a blurring process for blurring the band area mask image 552 supplied from the band area mask image generation unit 514 using, for example, a low-pass filter. Further, the moving area setting unit 515, for example, in the band area mask image 552 that has been subjected to the blurring process, sets the pixel value of a pixel (that is, a black pixel) whose pixel value is smaller than (R, G, B) = (128,128,128). As (R, G, B) = (128,128,128), the black region of the band region mask image 552 subjected to the blurring process is made gray. As a result, a blurred band region mask image 553 shown third from the left in FIG. 23 is obtained. In the blurred band region mask image 553 of FIG. 23, the amount of change in pixel value is smoothed at the boundary between the band region (white region) and other regions. Note that the width of the moving region can be adjusted according to the degree of blurring in this blurring process.

  Then, the moving area setting unit 515 has a predetermined pixel value (for example, (R, G, B) sandwiched between the broken line a and the broken line a ′ of the blurred band area mask image 553 shown in FIG. ) = (128,128,128)) larger area, that is, an area corresponding to at least a part of an area closer to white than a certain color (hereinafter simply referred to as a white area), specifically, an area of the lower half of the subject area. A movement area to be moved in the direction toward the inside of the area is set, and information representing the movement area is supplied to the area movement processing unit 516. Here, the direction toward the inside of the subject area refers to the direction toward the center of gravity of the subject area.

  In step S116, the area movement processing unit 516, based on the information supplied from the movement area setting unit 515, the center of the band area in the image corresponding to the white area of the blurred band area mask image in the captured image. Move the image near the line above the subject.

  Specifically, as shown on the right side of FIG. 24, in the captured image 431, among the areas sandwiched between the broken line a and the broken line a ′ corresponding to the white area of the blurred band area mask image 553, The area near the direction center line b is set as a movement area, and the area movement processing unit 516 moves the image of the movement area in the −Y direction (vertical upward direction in FIG. 24).

  Subsequently, in step S117, the region movement processing unit 516 enlarges the image of the region below the image of the moved region in the captured image in the region corresponding to the white region of the blurred band region mask image. The image in the region above the moved region image is reduced.

  That is, in the example of FIG. 24, in the captured image, an area moved in the −Y direction (vertical upward direction in FIG. 24) among areas sandwiched between the broken line a and the broken line a ′ (area corresponding to the white area). And the area between the broken line a is expanded in the Y direction (vertical direction), and the area between the area moved in the −Y direction and the broken line a ′ is reduced in the Y direction (vertical direction).

  The details of the area movement and enlargement / reduction processing described with reference to FIG. 24 are basically the same as the processing described with reference to FIG.

  In this way, in the captured image 431, the leg of the subject moves in the direction (upward) from the waist to the inside of the subject (upward), so that the subject's legs are long and the torso as shown in FIG. A captured image 571 in which is shortened is displayed on the touch panel monitor 93.

  According to the above process, the lower half of the subject area in the captured image, specifically, the waist to thigh area is moved in the direction toward the inside of the subject area (upward). The subject's legs are long and the torso is short. As a result, a full-body image with a good subject style can be obtained, and an image with a higher degree of satisfaction for the user can be provided.

  Note that, by performing face recognition processing on the captured image, the face of the subject is included in the captured image in a region sandwiched between the broken line a and the broken line a ′ corresponding to the white region of the blurred band region mask image 553. In some cases, it is possible not to perform the process of area movement and enlargement / reduction. Thereby, when a user who is a subject is bent at the time of photographing, it is possible to avoid the subject's face from being significantly deformed in the photographed image.

  Further, in the above, the region movement and enlargement / reduction processing is performed on the waist to thigh region of the subject region, but the region of the subject region at the foot (near the toe) The movement and enlargement / reduction processing may be performed. Specifically, the area near the toe of the subject area is moved upward as a moving area, the area below the moving area is enlarged, and the area above the moving area is reduced. As a result, a captured image in which the foot (shoes) of the subject is small can be obtained.

  In addition, when there are a plurality of subjects such as two people, the positions of the toes in the Y direction (vertical direction) may differ depending on the height and standing position of each subject. Therefore, the positions of the feet of each subject are detected. Thus, a band area mask image in which an oblique band area is set so that each step is included may be generated.

  In the above-described processing, when the user who is the subject raises one foot and stands on one foot during shooting, the size of the foot that is on the floor and the foot that is raised are different. Therefore, for example, when the position of the foot of the subject in the captured image is more than a predetermined distance, the region movement and enlargement / reduction processing may not be performed.

  In addition, as described with reference to FIG. 23, when the band region is set based on the position of the center of gravity of the subject region of the subject mask image 531, the region near the belly of the subject region moves reliably, but the waist region The region near the base of the leg may not move. Therefore, the belt region is set with reference to the position moved by a predetermined distance from the center of gravity of the subject region to the leg side (Y direction), so that the region near the base of the waist and legs is surely moved. Also good. Thereby, in the captured image, the leg of the subject can be lengthened and the trunk can be shortened more reliably.

  In the above, the moving area of the captured image is an area corresponding to at least a part of an area where the pixel value is larger than a predetermined value (an area between the broken line a and the broken line a ′) in the blurred band area mask image. However, by increasing the width of the band area in the band area mask image, the area sandwiched between the broken line a and the broken line a ′ is widened, and the area to be moved and enlarged / reduced in the captured image is widened. You may do it. As a result, the difference between the area subjected to the movement and enlargement / reduction processing and the other area can be smoothed, and a more natural captured image can be obtained.

  Further, the same effect as described above can be obtained by increasing the degree of blurring in the blurring process for the band area mask image (increasing the width of the blur) without increasing the width of the band area in the band area mask image. Can do.

  The subject slimming process in FIG. 12 and the subject leg length process in FIG. 21 may be performed for each captured image, or each process may be performed for one captured image. You may be made to be.

[Another functional configuration example of the image processing apparatus]
Next, still another functional configuration example of the image processing apparatus that performs image processing for improving the appearance of a subject in a captured image will be described with reference to FIG.

  The image processing apparatus in FIG. 25 includes a captured image acquisition unit 611, a subject mask image generation unit 612, a distance conversion image generation unit 613, a gradient image generation unit 614, a face mask image generation unit 615, a movement region setting unit 616, and region movement. The processing unit 617 is configured.

  25, the captured image acquisition unit 611, the subject mask image generation unit 612, the gradient image generation unit 614, the face mask image generation unit 615, the movement region setting unit 616, and the region movement processing unit 617 include Basically, the captured image acquisition unit 411, the subject mask image generation unit 412, the gradient image generation unit 414, the face mask image generation unit 415, the movement area setting unit 416, and the area movement processing unit 417 in the image processing apparatus of FIG. Since it has the same function, its description is omitted.

  The distance conversion image generation unit 613 performs distance conversion processing on the subject mask image supplied from the subject mask image generation unit 412, and supplies the distance conversion image obtained as a result to the gradient image generation unit 614.

[Subject slimming process]
Next, the subject slimming process for improving the appearance of the subject of the photographed image, which is executed by the image processing apparatus of FIG. 25, will be described with reference to the flowchart of FIG.

  Note that the processing in steps S211 and S212 in the flowchart in FIG. 26 is the same as the processing in steps S11 and S12 in the flowchart in FIG.

  That is, in steps S211 and S212, a captured image 631 as shown first from the left in FIG. 27 is acquired, and a subject mask image 632 shown second from the left in FIG. 27 is generated.

  In step S213, the distance conversion image generation unit 613 performs a distance conversion process on the subject mask image 632 supplied from the subject mask image generation unit 612. Specifically, the distance conversion image generation unit 613 obtains the distance from the pixel (black pixel) in the closest background region for each pixel (white pixel) in the subject region of the subject mask image 632, and calculates the distance as a pixel value. An image is generated. As a result, a distance conversion image 633 shown third from the left in FIG. 27 is obtained and supplied to the gradient image generation unit 614. In the distance conversion image 633 in FIG. 27, the pixels that are farther from the background area are closer to white (the pixel value is larger). In general, according to the distance conversion process, the skeleton of an object can be extracted. That is, the distance conversion image 633 can be said to be an image representing the skeletons of two subjects.

  In step S214, the gradient image generation unit 614 generates an X-direction gradient image and a Y-direction gradient image from the pixel values of the distance conversion image supplied from the distance conversion image generation unit 613. Specifically, the gradient image generation unit 614 performs spatial first-order differentiation in the X direction on the distance conversion image 633 using, for example, a Sobel filter, so that the gradient image generation unit 614 is shown in the fourth upper stage from the left in FIG. X direction gradient image 634X is generated, and spatial first order differentiation is performed in the Y direction to generate a Y direction gradient image 634Y shown in the fourth lower row from the left in FIG.

  Also in the X-direction gradient image 634X of FIG. 27, the black region indicates a region in which the gradient (gradient) of the change of the pixel value with respect to the X direction is positive in the distance conversion image 633, and the white region indicates the distance conversion image 633. In FIG. 6, the region in which the gradient (gradient) of the change of the pixel value with respect to the X direction is negative is shown.

  In the Y-direction gradient image 634Y of FIG. 27, the black area indicates an area in which the gradient (gradient) of the change in the pixel value with respect to the Y direction is positive in the distance conversion image 633, and the white area indicates the distance conversion. In the image 633, an area in which the gradient (gradient) of the change of the pixel value with respect to the Y direction is negative is shown.

  That is, in the X-direction gradient image 634X and the Y-direction gradient image 634Y in FIG. 27, the black region and the white region are regions having pixel value gradients in the distance conversion image 633, and the other regions are the distance conversion image 633. In FIG. 6, a region without a gradient of pixel values is shown.

  Returning to the flowchart of FIG. 26, in step S215, the face mask image generation unit 615 executes a face mask image generation process to generate a face mask image. In step S215, the face mask image generation process described with reference to the flowchart of FIG. 14 is executed, and thus the description thereof is omitted.

  In step S <b> 216, the moving region setting unit 616 multiplies the gradient image supplied from the gradient image generation unit 614 by the blurred face mask image supplied from the face mask image generation unit 615. As a result, an X-direction gradient image 651X in which the black region and the white region are deleted from the region corresponding to the subject's face shown on the left side of FIG. 28, and the region corresponding to the subject's face shown on the right side of FIG. A Y-direction gradient image 651Y from which the black area and the white area are deleted is obtained.

  Here, in the captured image, the regions corresponding to the black region and the white region in the X-direction gradient image 651X and the Y-direction gradient image 651Y are regions including substantially all of the subject region.

  In the X-direction gradient image 651X and the Y-direction gradient image 651Y obtained in this way, the moving area setting unit 616 includes, in the captured image, an area corresponding to at least a part of the black area and the white area, specifically, the subject. Substantially all of the areas are set as moving areas that move in the direction toward the inside (skeleton) of the subject, and information representing the moving areas is supplied to the area movement processing unit 617.

  In step S217, based on the information supplied from the movement region setting unit 616, the region movement processing unit 617 is an image in the vicinity of the boundary among the images of the regions corresponding to the black region and the white region of the gradient image in the captured image. Is moved in the direction toward the inside of the subject.

  More specifically, at this time, in the captured image, an area corresponding to an area excluding an edge portion in the X direction among black areas of the X-direction gradient image 651X is set as a movement area. Moves the image of the moving area in the X direction (horizontal right direction in FIG. 28). In the captured image, an area corresponding to an area excluding an edge portion in the X direction is set as a movement area among white areas of the X-direction gradient image 651X, and the area movement processing unit 617 displays an image of the movement area. Are moved in the -X direction (horizontal left direction in FIG. 28).

  Similarly, in the captured image, an area corresponding to an area excluding an edge portion in the Y direction is set as a movement area in the black area of the Y-direction gradient image 651Y, and the area movement processing unit 617 sets the movement area. The image is moved in the Y direction (vertically downward in FIG. 28). In the captured image, an area corresponding to an area excluding an edge portion in the Y direction is set as a movement area in the white area of the Y-direction gradient image 651Y, and the area movement processing unit 617 displays the image of the movement area. Are moved in the -Y direction (vertically upward in FIG. 28).

  Subsequently, in step S218, the region movement processing unit 617 enlarges the image of the region outside the image of the moved region among the images of the region corresponding to the black region and the white region of the gradient image in the captured image, The image in the area inside the image of the moved area is reduced.

  More specifically, the area movement processing unit 617 expands the area outside the moving area in the X direction (horizontal direction) in the captured image, in the area corresponding to the black area of the X-direction gradient image 651X. The inner area is reduced in the X direction (horizontal direction). In addition, the region movement processing unit 617 enlarges a region outside the moving region in the X direction (horizontal direction) among the regions corresponding to the white region of the X-direction gradient image 651X in the captured image, and moves the region inside the moving region. The area is reduced in the X direction (horizontal direction).

  Similarly, the area movement processing unit 617 enlarges the area outside the moving area in the Y direction (vertical direction) among the areas corresponding to the black area of the Y-direction gradient image 651Y in the captured image, and moves the area inside the moving area. Is reduced in the Y direction (vertical direction). In addition, the area movement processing unit 617 enlarges an area outside the moving area in the Y direction (vertical direction) among areas corresponding to the white area of the Y-direction gradient image 651Y in the captured image, and moves the area inside the moving area. The area is reduced in the Y direction (vertical direction).

  In this way, in the captured image 631, almost all the part of the subject excluding the face moves in the direction toward the inside of the subject (the direction away from the background), so that the subject as shown in FIG. A captured image 671 in which the body is entirely small (thinned) is displayed on the touch panel monitor 93.

  According to the above processing, almost all of the subject area excluding the face in the photographed image is moved in the direction toward the inside of the subject area. Therefore, in the photographed image, the body of the subject excluding the face Becomes smaller (thinner) as a whole. As a result, it is possible to obtain a whole body image with a good style, in which the waistline is particularly emphasized, and as a result, it is possible to provide an image that is more satisfactory for the user.

  In particular, the distance conversion process extracts a portion of the subject area that is far from the background region, that is, a skeleton, and moves almost all of the subject area toward the skeleton. It is possible to make the subject as a whole thin while maintaining the skeleton.

  The subject slimming process in FIG. 26 and the subject leg length process in FIG. 21 may be performed for each captured image, or each process may be performed for one captured image. You may be made to be.

  Furthermore, the subject slimming process of FIG. 26 may be further performed on the photographed image subjected to the subject slimming process of FIG. This makes it possible to improve the subject naturally and surely in the captured image.

  In the above-described processing, when the subject mask image is generated, if the subject's clothes or personal belongings include substantially the same color as the blackboard background curtain, the portion of the subject mask image is a black pixel. However, when the area of the black pixel area is smaller than a predetermined area and all the surroundings are white pixels, the black pixels may be replaced with white pixels.

  In addition, the above-described processing is performed on a captured image as a still image obtained by shooting. However, a live view image captured by the camera 91 and displayed as a moving image on the touch panel monitor 93 is displayed. It may also be performed on the other hand.

  As for the image processing (gradient image generation, distance conversion, center of gravity position calculation, etc.) performed in the above-described processing, for example, ““ New Image Analysis Handbook ”The University of Tokyo Press” or ““ Digital Image Processing ” This can be done by using the method disclosed in the “Image Information Education Promotion Association”.

  By the way, the above-described series of processes can be executed by hardware or can be executed by software. When the above-described series of processing is executed by software, a program constituting the software is installed from a network or a recording medium.

  For example, as shown in FIG. 8, the recording medium is distributed to distribute the program to the administrator of the photo sticker creating apparatus 1 separately from the apparatus main body, and is a magnetic disk (flexible disk) on which the program is recorded. Disk), optical disk (including CD-ROM and DVD), magneto-optical disk (including MD), or removable media 205 made of semiconductor memory, etc., but also pre-installed in the main unit The ROM 206 that stores the program and the hard disk included in the storage unit 202 are distributed to the administrator.

  In the present specification, the step of describing the program recorded on the recording medium is not limited to the processing performed in chronological order according to the described order, but is not necessarily performed in chronological order. It also includes processes that are executed individually.

  In the above, the printing medium is not limited to sticker paper or photographic paper. For example, printing on a predetermined size of paper or film, poster paper, a card such as a telephone card, or a cloth such as a T-shirt. You may make it do.

DESCRIPTION OF SYMBOLS 1 Photo sticker production apparatus 11 Shooting unit 12 Editing part 13 Subsequent customer service part 201 Control part 208 Imaging | photography part 209 Editing part 210 Subsequent customer service part 301 Shooting process part 302 Edit process part 303 Print process part 304 Post customer service process part 411 Photographed image acquisition part 412 Subject mask image generation unit 413 Blur processing unit 414 Gradient image generation unit 415 Face mask image generation unit 416 Movement region setting unit 417 Region movement processing unit

Claims (11)

Photographed image acquisition means for acquiring a photographed image obtained by photographing a subject;
Moving region setting means for setting a moving region for moving at least a part of the subject region in the captured image in a direction toward the inside of the subject region;
An area movement processing means for moving an area set as the movement area in a direction toward the inside of the subject area in the captured image;
An image processing apparatus comprising: an output unit that outputs the captured image in which the moving area moves in a direction toward the inside of the subject area. The image processing apparatus according to claim 1, wherein the moving area setting unit sets an area in the vicinity of a boundary between the subject area and the background area in the captured image as the moving area. Subject mask image generation means for generating a subject mask image that transmits the subject region of the photographed image based on the photographed image;
Blur processing means for performing blur processing on the subject mask image and generating a blurred subject mask image;
A gradient image generation unit that generates a gradient image of the blurred subject mask image from the pixel value of the blurred subject mask image;
The image processing apparatus according to claim 2, wherein the moving region setting unit sets at least a part of the region of the captured image corresponding to a region having a gradient in the gradient image as the moving region. The image processing apparatus according to claim 1, wherein the movement area setting unit sets substantially all areas of the subject area in the captured image as the movement area. Subject mask image generation means for generating a subject mask image for masking an area other than the subject area of the photographed image based on the photographed image;
Distance conversion image generation means for performing a distance conversion process on the subject mask image and generating a distance conversion image;
Gradient image generation means for generating a gradient image of the distance conversion image from the pixel value of the distance conversion image,
The image processing apparatus according to claim 4, wherein the moving region setting unit sets at least a part of the region of the captured image corresponding to a region having a gradient in the gradient image as the moving region. A face mask image generating unit configured to generate a face mask image for masking a face area of the subject area of the photographed image based on the photographed image;
The image processing according to claim 1, wherein the moving area setting unit sets the predetermined area excluding the face in the subject area as the moving area based on the face mask image. apparatus. The image processing apparatus according to claim 1, wherein the moving area setting unit sets a lower body area of the subject area in the captured image as the moving area. Subject mask image generation means for generating a subject mask image for masking an area other than the subject area of the photographed image based on the photographed image;
Centroid calculating means for calculating the centroid position of the subject area from the subject mask image;
A band area mask image generating means for generating a band area mask image in which a band area of a predetermined width is set from the center of gravity position of the captured image toward the foot of the subject area;
The image processing apparatus according to claim 7, wherein the movement area setting unit sets at least a part of the area of the captured image corresponding to the band area in the band area mask image as the movement area. The area movement processing means moves the area set as the moving area in the captured image in a direction toward the inside of the subject and moves the area inside the area set as the moving area. The image processing apparatus according to claim 1, wherein the image processing apparatus reduces in the moving direction of the area and expands an area outside the area set as the moving area in the moving direction of the moving area. In the image processing method of the image processing apparatus,
The image processing apparatus is
Get a shot image obtained by shooting the subject,
Setting at least a part of the subject area in the captured image as a moving area that moves in a direction toward the inside of the subject area;
In the captured image, the region set as the moving region is moved in a direction toward the inside of the subject region,
An image processing method including a step of outputting the captured image in which the moving area moves in a direction toward the inside of the subject area. On the computer,
Get a shot image obtained by shooting the subject,
Setting at least a part of the subject area in the captured image as a moving area that moves in a direction toward the inside of the subject area;
In the captured image, the region set as the moving region is moved in a direction toward the inside of the subject region,
A program for executing a process including a step of outputting the captured image in which the moving area moves in a direction toward the inside of the subject area.

Images