JP5013203B2 - Photo sticker creation apparatus and method, and program - Google Patents

Description

  The present invention relates to a photo sticker creation apparatus and method, and a program, and more particularly, to a photo sticker creation apparatus and method, and a program that allow a user to take a picture while having more enjoyment.

  One game provided by a conventional photo sticker creation device installed in an amusement (playing) facility, etc., is to shoot a subject, edit a photographed image (doodle), edit edited image or sticker of a photographed image. The process ends with printing on paper.

  In such a photo sticker creation apparatus, various edits can be performed on a captured image in which the user himself / herself is photographed, but the captured image to be edited before editing is performed. If the image itself is not according to the user's preference, editing will be performed without satisfaction. That is, editing cannot be enjoyed, and further, satisfaction with the game is lacking.

  Therefore, in recent years, various methods have been used to capture satisfactory captured images, and to enable users to perform interesting shooting operations. A photo sticker creating apparatus capable of acquiring a photographed image has been developed.

  For example, there is a photo sticker creation apparatus that combines a foreground image or background image selected by a user with a moving image of a subject captured by a camera at the time of shooting, and displays the combined image in a live view. The user can take a picture while viewing his / her own image combined with the foreground image and the background image.

  Patent Document 1 discloses a technique for combining a background image with a region other than a subject region in a photographed image and displaying an image obtained by further combining a foreground image with the composite image for live view display. ing.

  Japanese Patent Application Laid-Open No. 2004-228561 discloses a technique for synthesizing a moving image of a subject being captured and displaying it in a live view in an area designated by a user among photographed images taken once.

  Patent Document 3 discloses a technique for detecting a specific part of a subject from a photographed image and combining and displaying a foreground moving image such as a predetermined object based on the detected position of the specific part. .

  In Patent Document 4, a frame image is displayed as a moving image, and the frame image is stopped in accordance with a user's designation so that the user can synthesize a subject image in which he / she is photographed and a preferred frame image. A technology that can perform the above is disclosed.

  Patent Documents 5 and 6 disclose a technique in which a moving image of a subject is displayed, and the displayed moving image can be edited using a touch pen or the like.

JP 2004-320077 A JP 2007-241199 A JP 2000-92481 A Japanese Patent Laid-Open No. 10-282571 JP 2002-94906 A JP 2002-125176 A

  However, according to the technique disclosed in Patent Document 1, since the composition image (foreground image / background image) used to compose the subject image is a still image, such a still image is composed. Even if you can see your own image combined with the work image, the shooting work is not interesting.

  Further, according to the technique disclosed in Patent Document 2, since it is necessary to perform shooting twice or to specify a region for synthesizing a moving image, the operation becomes complicated for the user and the operability is improved. It will decline.

  According to the technique disclosed in Patent Document 3, for example, when a foreground image is synthesized in response to finger detection, when a finger is not captured in a captured image (when it is different from a sample image of a capturing pose presented at the time of capturing) ), The foreground image will not be displayed, and in such a case, it will be uninteresting.

  According to the technique disclosed in Patent Document 4, when acquiring a frame image of a still image, if the timing of the frame image that is a moving image does not match the timing of shooting, The frame image intended by the user cannot be obtained, and as a result, the composite image becomes an unsatisfactory image.

  According to the techniques disclosed in Patent Documents 5 and 6, editing can be performed while watching a moving image. However, since it is necessary to operate a touch pen or the like to perform the editing, the operation is difficult for the user. It becomes complicated and the operability deteriorates.

  As described above, according to the conventional technology, since the operation is complicated for the user, the operability is deteriorated, or the photographed image that is not interesting and is not satisfactory is provided. doing. Therefore, the user's satisfaction with the game is inevitably lowered.

  It is preferable that the moving image presented to the user during the shooting operation by live view display is interesting.

  The present invention has been made in view of such a situation, and enables a user to take a picture while having more fun.

  According to another aspect of the present invention, there is provided a photo sticker creating apparatus that photographs a user as a subject, causes the user to edit the image obtained by photographing, and the image obtained by photographing or the edited image. In a photo sticker creating apparatus that prints an image on a sticker sheet to create a photo sticker and provides the created photo sticker to a user, a photographing unit that captures a moving image that the user shows as a subject, and the photographing unit captures the moving image Creating means for creating a mask image that separates a subject area and a background based on respective frames that are still images arranged in time-series order that constitute a moving image, and based on a difference between the mask images created by the creating means A motion detection unit that detects a range in which the subject has moved every predetermined period, and a foreground that is a foreground of a captured image according to the range detected by the motion detection unit A synthesizing unit that adjusts the density of the pixels of the image and synthesizes the foreground image in which the density of each pixel is adjusted as a foreground of a subject appearing in each frame constituting the moving image captured by the photographing unit; Display control means for displaying a synthesized moving image obtained by synthesizing during imaging of an image to be edited.

  The synthesizing means can be adjusted to increase the density of pixels in the same range as the range detected by the motion detecting means among the pixels of the foreground image.

  The synthesizing means may be further adjusted so as to lower the density of pixels in the foreground image other than the pixels within the same range as the range detected by the motion detecting means.

  The creating means can be a means for creating a mask image by detecting an area of a subject by a chroma processing or a skin color area detecting process.

  A user can take a picture while watching a synthesized moving image in which a foreground image in which a pattern or a character rises or disappears according to his / her movement is synthesized. Further, the user can display such a synthesized moving image without performing a complicated operation.

  That is, the user can more easily and enjoy shooting, thereby obtaining satisfaction.

  According to another aspect of the present invention, there is provided a photo sticker creation method or program that captures a moving image that is captured by a user as a subject, and sets the region of the subject based on each frame that is a still image arranged in chronological order that constitutes the captured moving image. A mask image that separates the background is created, and based on the difference between the created mask images, a range of movement of the subject is detected every predetermined period, and the foreground image that is the foreground of the captured image is detected according to the detected range. The foreground image in which the density of each pixel is adjusted and the density of each pixel is adjusted is synthesized as the foreground of the subject appearing in each frame constituting the captured moving image, and the synthesized moving image obtained by synthesizing is combined. A step of displaying the image to be edited during shooting.

  In one aspect of the present invention, a moving image that is captured as a subject by a user is captured, and the region and background of the subject are separated based on each frame that is a still image arranged in chronological order that constitutes the captured moving image. A mask image is created. Further, based on the difference between the created mask images, a range in which the subject has moved is detected every predetermined period, and the pixel density of the foreground image that is the foreground of the captured image is adjusted according to the detected range. . Further, the foreground image in which the density of each pixel is adjusted is synthesized as the foreground of the subject appearing in each frame constituting the captured moving image, and the synthesized moving image obtained by synthesizing is synthesized. Displayed during shooting of the image.

  According to the present invention, the user can take a picture while having more fun.

  FIG. 1 is a diagram showing an embodiment of a photo sticker creating apparatus to which the present invention is applied.

  The photo sticker creating apparatus 1 shown in FIG. 1 enhances the amusement of work such as shooting and editing, allows the user to perform such work as a game (play service), and takes the shot / edited images as photo stickers. It is a game device (play service providing device) that receives money from the user instead of providing it to the user as data. The photo sticker creating apparatus 1 is usually installed in a game center or a store in a sightseeing spot.

  In the photo sticker creating game executed by the photo sticker creating device 1, the user inputs the price of the game, shoots himself / herself using the camera provided in the photo sticker creating device 1, and the photographed image is taken. Then, editing is performed using an editing function (doodle editing function) such as pen image input or stamp image input, and the photographed image is designed to be rich and colorful. After the game ends, the user receives a photo sticker or the like on which the edited image is printed as a product.

  As will be described in detail later, in the photo sticker creating apparatus 1, during the shooting work in a series of photo sticker creating games, the movement of the user as the subject is detected from the moving image captured by the camera, and the live view display is performed. In the entire moving image, a pattern or the like is synthesized in a range where the motion is detected.

  Since a pattern or the like is displayed in a different range according to the movement, the user can enjoy shooting while watching a moving image displayed in a live view that changes according to his / her movement.

  As shown in FIG. 1, the housing 10 of the photo sticker creating apparatus 1 is roughly divided into four units: a pre-service unit 11, a photographing unit 12, an editing unit 13, and a ceiling strobe unit 14.

  The pre-service unit 11 is a unit that accepts a photo sticker creation game and outputs content (demonstration) that guides (explains) the flow and play of the photo sticker creation game with video and audio.

  The photographing unit 12 has a function of photographing a subject (such as a user), and the subject is photographed in an internal space (photographing space) described later. The photographing unit 12 is roughly divided into two units, a front unit 12A and a rear unit 12B.

  The front unit 12A is a unit on the front side of the shooting space, and performs processing related to shooting of the user in the shooting space and accepts operations related to the user's photo sticker creation game. The rear unit 12B is a unit on the side that is the rear side of the shooting space, and is a unit that plays the role of the user when the front unit 12A takes a picture. The interior of the rear unit 12B is a cavity (space), and is configured as a part of a space (photographing space described later) used when the user performs photographing. Inside the rear unit 12B, for example, a movable curtain (background curtain) used for the background, a stage used for the user to pose, and the like may be provided.

  The editing unit 13 is provided with a configuration (for example, a monitor, a touch panel, and a touch pen) for the user to perform graffiti editing or the like on the captured image. The editing unit 13 composed of one frame is provided with a configuration for performing graffiti editing or the like on each of the two surfaces so that two sets of users can perform editing work simultaneously. That is, the photo sticker creation apparatus 1 can execute a plurality of photo sticker creation games in parallel. The editing unit 13 has a configuration for performing two graffiti editing and the like so that a plurality of sets of users can simultaneously perform graffiti editing in order to improve the rotation rate of the photo sticker creation game. Is provided.

  The ceiling strobe unit 14 provided above the photographing space incorporates a flash that emits strobe light toward the photographing space in accordance with the photographing timing. The ceiling strobe unit 14 further includes a fluorescent lamp, and functions as illumination that constantly illuminates the photographing space (the user in the photographing space). The photographing space is simply configured as a closed space as will be described later. Therefore, it is difficult for light to enter from the outside, and the user's operability may be deteriorated because it tends to be dark. The ceiling strobe unit 14 illuminates the shooting space with the illumination, thereby avoiding such a fear and improving operability. Such illumination is also provided in the front unit 12A and the rear unit 12B.

  FIG. 2 is a perspective view of the photo sticker creating apparatus 1 of FIG. 1 as viewed from the editing unit 13 side.

  As shown in FIG. 2, the editing unit 13 is installed adjacent to the surface of the front unit 12A opposite to the surface on the rear unit 12B side (that is, the surface on the imaging space side). A first editing interface 13A and a second editing interface 13B are provided on two side surfaces of the editing unit 13 that are orthogonal to the front unit 12A. That is, the editing unit 13 allows a plurality of sets of users who perform the photo sticker creation game to perform editing operations with the editing unit 13 facing each other in a direction orthogonal to the direction connecting the front unit 12A and the rear unit 12B. Two editing interfaces are provided so that this can be done. Details of the configuration of these editing interfaces will be described later.

  Also, on the surface of the editing unit 13 that faces the photographing unit 12, a photo sticker discharge unit 13C that is provided to the user by discharging the photo sticker created by playing the photo sticker making game is provided. ing.

  FIG. 3 is an external view showing an installation example of the photo sticker creating apparatus 1.

  When the photo sticker creating apparatus 1 is installed in a game center or the like, for example, as shown in FIG. 3, a curtain 21 is provided so as to surround the photographing unit 12 and the editing unit 13, and a photographing space where photographing work is performed or The editing space where the editing work is performed is simply enclosed, and the closeability of each space is improved. In other words, for example, the curtain 21 stabilizes the environment in each space, thereby contributing to high-quality shooting, easy viewing of the editing screen, etc., or blocking the field of view from outside each space, It is possible to perform operations such as shooting and editing without being aware of the line of sight.

  The outer surface of the curtain 21 is usually used as a place for posting advertisements, and a flashy design image representing the model name and model image of the photo sticker creating apparatus 1 is printed so as to attract attention at the installation location. Is done.

  The photo sticker creating apparatus 1 as described above causes each set of users to play a photo sticker creating game. The photo sticker creating apparatus 1 can simultaneously execute a plurality of photo sticker creating games. That is, the photo sticker creating apparatus 1 can cause a plurality of sets to play a photo sticker creating game at the same time. At this time, each group of users performs each work of the photo sticker creation game while moving through a plurality of spaces provided in the photo sticker creation device 1, and advances the game.

  The flow of the photo sticker creation game and the accompanying movement of the space will be described with reference to FIG.

  FIG. 4 is a plan view of the photo sticker creating apparatus 1 as viewed from above.

  As shown in FIG. 4, the user starts the photo sticker creation game by charging the pre-service unit 11 in the pre-service area 31, which is the area around the front of the pre-service unit 11 (user A). ). The user enters from the side of the pre-serving unit 11 if the shooting space 32 for performing the shooting operation, which is formed between the front unit 12A and the rear unit 12B and is indicated by the dotted line in the drawing, is not in use. (User B). If the previous group of users is using the shooting space 32 (if shooting is in progress), the user waits in the pre-service area 31 (the pre-service unit 11 instructs the standby).

  The pre-service unit 11 can also be provided on the opposite side of the front unit 12A as indicated by the dotted line. In this case, the user starts the photo sticker creation game from the one with the pre-service unit 11 Then (user A ′), after the pre-service, the user enters the shooting space 32 from the side opposite to the above side (user B).

  A user (user B) who enters the shooting space 32 performs a shooting operation of shooting himself or herself using a camera or monitor provided in front of the front unit 12A.

  When the photographing work is completed, the user selects a keep image to be kept for editing and printing from the obtained photographed images, and follows the displayed movement guide from the photographing space 32 where the photographing work is performed. Move to the editing space indicated by the dotted line.

  A first editing space 33A is provided on the left side of the editing unit 13 as viewed from the front unit 12A (in front of the first editing interface 13A), and a second editing space 33B is provided on the right side (in front of the second editing interface 13B). ing. That is, in this photo sticker creation apparatus 1, a total of two sets of users, the user (user C) of the first editing space 33A and the user (user C ') of the second editing space 33B, simultaneously edit. Work can be done. A user (user B) of the shooting space 32 is guided to a free one of the two editing spaces. When the user moves to the first editing space 33A or the second editing space 33B, the user performs a graffiti editing work. Hereinafter, when it is not necessary to distinguish the first editing space 33A and the second editing space 33B, they are referred to as an editing space 33.

  Thereafter, when the graffiti editing is completed, the user (user C or user C ′) next performs print setting work and sets the number of divisions and the like. When printing is started, the user plays a mini game during the printing. Then, when printing is completed, it moves to the printed matter receiving area 34 in front of the photo sticker discharge unit 13C (user D), receives the discharged photo sticker paper, and ends the photo sticker creation game.

  Next, the configuration of each unit will be described.

  FIG. 5 is a diagram illustrating a configuration example of the advance customer service unit 11.

  As shown in FIG. 5, the customer service unit 11 accepts a price for a photo sticker creation game inserted by the user, returns a change to the user, etc., and a photo sticker creation game. And a monitor 42 for displaying a guide or demonstration screen of the game method and a speaker 43 for outputting sound corresponding to the image displayed on the monitor 42, sound effects, and the like.

  Since the image displayed on the monitor 42 and the sound output from the speaker 43 are output to the outside of the housing 10 (outside the area surrounded by the curtain 21), use before the photo sticker creation game is performed. In addition to deepening the understanding of how to play the photo sticker creation game of the person who is going to be a photographer, advertisement (invitation) to the people around the photo sticker creation device 1 who are not playing the photo sticker creation game It also has an effect.

  When the user inserts a predetermined price into the coin insertion / return slot 41 to play the photo sticker creation game, the user is directed to the monitor 42 on the monitor 42 when the shooting space 32 is free. A guidance screen is displayed, and a sound corresponding to the guidance screen is output from the speaker 43. When the shooting space 32 is in use, a guidance screen is displayed on the monitor 42 to prompt the user to stand by on the spot (pre-service area 31), and the corresponding sound is output from the speaker 43. Is done.

  As described above, the pre-service unit 11 may be installed on the right side of the front unit 12A as shown in FIG. 5, or on the left side of the front unit 12A as shown in FIG. May be.

  FIG. 6 is a diagram illustrating a configuration example of the front unit 12A.

  A front surface 12A-1 shown in FIG. 6 is a side surface facing the imaging space 32, and is a surface positioned in front of the user who performs the imaging operation. That is, the user basically performs a photographing operation in the photographing space 32 toward the front surface 12A-1 (facing the front unit 12A side).

  On the front surface 12A-1, a camera 51 that shoots a subject in the shooting space 32, for example, an acquired image or a shooting result obtained (photographed) in real time (immediately) after being photoelectrically converted by the camera 51. In addition to displaying images related to shooting operations, such as GUI images for shooting images, guidance regarding shooting operations, and GUI (Graphical User Interface) images for selecting background images, etc., and user instructions via a touch panel superimposed on the screen Touch panel monitor 52 that illuminates, illumination device 53-1 to illumination device 53-8 that illuminates the imaging space 32 or emits flash light in accordance with the imaging timing of the camera 51, and baggage of the user during the imaging operation For example, the luggage storage area 54-1 and the luggage storage area 54-2 for placing the sound, etc. Speakers 55-1 and a speaker 55-2 is provided is.

  The camera 51 is configured by an imaging device using a CCD (Charge Coupled Device) or the like, and photographs a user in the shooting space 32 in the shooting process of the photo sticker creation game. For example, a camera using a CMOS (Complementary Metal Oxide Semiconductor) sensor may be used as the camera 51 instead of the CCD. Of course, any camera other than these may be used as long as the user can be photographed and digital image data can be obtained. Further, the number of cameras 51 and the installation position are arbitrary.

  The touch panel monitor 52 is configured by an LCD (Liquid Crystal Display) or the like, and includes an acquired image of the camera 51, a composite image to be combined with the captured image, a captured image stored as a captured result, or a guide image that guides the procedure of the capturing operation. This is a monitor that displays an image relating to a shooting operation. A colorless and transparent touch panel (for example, a pressure-sensitive type or an electromagnetic induction type) is superimposed on the screen of the touch panel monitor 52. For example, the position information (user instruction) is obtained by tapping with a user's finger or the like. Can be entered.

  The illuminating devices 53-1 to 53-8 are configured by a plurality of illuminating devices that can emit fluorescent light and strobe light, and in order to brighten the photographic space 32 and ensure operability and safety, the photographic space 32 is provided. In order to improve the image quality of the photographed image, the flash is emitted at a timing corresponding to the photographing timing, and the user who is photographed is irradiated with light from various angles. Hereinafter, the lighting devices 53-1 to 53-8 are referred to as the lighting devices 53 when it is not necessary to distinguish them from each other for explanation.

  The luggage storage area 54-1 and the luggage storage area 54-2 are shelves that are used as storage locations for the user's baggage at the time of photographing. Both the luggage storage area 54-1 and the luggage storage area 54-2 do not interfere with shooting, that is, do not interfere with the user taking a shooting pose, and the illumination device 53 emits light. It is provided at a position that does not interfere with The user can perform photographing work more comfortably by placing the baggage in the luggage storage area 54-1 or the luggage storage area 54-2. Hereinafter, when it is not necessary to distinguish between the luggage storage area 54-1 and the luggage storage area 54-2, they are simply referred to as the luggage storage area 54.

  The speaker 55-1 and the speaker 55-2 appropriately output sound such as guidance sound, BGM (Back Ground Music), and sound effects in the shooting work of the photo sticker creation game. When it is not necessary to distinguish the speaker 55-1 and the speaker 55-2 from each other, they are referred to as a speaker 55. Note that the number and installation positions of the speakers 55 are arbitrary.

  Next, the configuration of the editing unit 13 will be described.

  FIG. 7 is a diagram showing a configuration example of the left side of the editing unit 13 when viewed from the photographing unit 12 side (configuration example on the first editing space 33A side).

  In FIG. 7, a first editing interface 13A is provided on the left side surface. On the upper side, an illuminating device 61 is provided for illuminating the first editing space 33A that is darkened because it is partitioned by the curtain 21 or the like. The illuminating device 61 allows the photo sticker creating apparatus 1 to suppress a reduction in the user's editing work efficiency and improve the safety of the first editing space 33A. In the example of FIG. 7, the upper portion of the editing unit 13 on the first editing interface 13 </ b> A where the lighting device 61 is provided penetrates to the second editing space 33 </ b> B side. The first editing space 33A and the second editing space 33B are shared. That is, the illumination device 61 illuminates not only the first editing space 33A but also the second editing space 33B, thereby improving safety. Note that the installation position and the number of the illumination devices 61 are arbitrary.

  The first editing interface 13A is provided with a tablet built-in monitor 62, two touch pens (touch pen 63-1 and touch pen 63-2), and a speaker 64.

  The tablet built-in monitor 62 displays a GUI image (doodle editing input screen) and the like for editing a photographed image obtained by photographing work in the photographing space.

  The tablet built-in monitor 62 is a monitor having a tablet capable of inputting position information with the touch pen 63-1 or the touch pen 63-2 and a display device such as an LCD capable of displaying an image. Here, the tablet is, for example, a pressure-sensitive type or electromagnetic induction type input device, is colorless and transparent, and is installed so as to be superimposed on the display screen of the display device. That is, the tablet built-in monitor 62 not only displays a GUI image or the like by the display device but also accepts an input operation from the user by the tablet.

  Touch pens 63-1 and 63-2 corresponding to the touch panel superimposed on the screen are detachably fixed to the editing unit 13 by a predetermined attachment near the left and right of the tablet built-in monitor 62. By using these touch pens, two users in the same group can play the editing process of the photo sticker creation process at the same time. In the following description, the touch pen 63-1 and the touch pen 63-2 are simply referred to as the touch pen 63 when it is not necessary to distinguish them from each other.

  The touch pen 63 is also electrically connected to a system installed in the editing unit 13 by wire (or wireless). The touch pen 63 operates in cooperation with a colorless and transparent tablet superimposed on the display screen of the display device of the tablet built-in monitor 62. When the user operates these touch pens 63 and taps the tablet (the touch pen 63 and the display screen of the tablet built-in monitor 62 are brought into contact (or close proximity)), these are inputs that accept input from the user. Acts as a means. The number of touch pens 63 to be installed, the installation position, the shape, and the like are arbitrary.

  The speaker 64 outputs, for example, sound related to editing work of the photo sticker creation game, such as guidance sound, sound effects, or BGM. The number, design, shape, etc. of the speakers 64 are arbitrary.

  Note that the configuration of the editing unit 13 on the second editing space 33B side is basically the same as the configuration of the first editing space 33A side shown in FIG. In the following, the configuration example shown in FIG. 7 is also applied to the case where the configuration of the editing unit 13 on the second editing space 33B side is described.

  FIG. 8 is a diagram illustrating a configuration example of the photo sticker discharge unit 13 </ b> C of the editing unit 13.

  As shown in FIG. 8, the photo sticker discharging unit 13C has a photo sticker created by a user who has performed editing work in the first editing space 33A (that is, a photo created by operating the first editing interface 13A). And a photo sticker discharge port 71 through which a photo sticker created by a user who has performed editing work in the second editing space 33B (that is, a photo sticker created by operating the second editing interface 13B) is discharged. Is provided.

  A printer is installed inside the editing unit 13, and a printing image created by operating the first editing interface 13A or the second editing interface 13B is printed on sticker paper by the printer. The photo sticker output from the printer is discharged from the photo sticker discharge port 71.

  Next, the internal configuration of the photo sticker creating apparatus 1 will be described.

  FIG. 9 is a block diagram illustrating a hardware configuration example of the photo sticker creating apparatus 1. The same parts as those described above are denoted by the same reference numerals and description thereof is omitted.

  In FIG. 9, the photo sticker creating apparatus 1 includes a control device 101 that is a processing unit that controls the entire apparatus of the photo sticker creating device 1. The control device 101 includes a storage unit 102, a communication unit 103, A drive 104, a ROM (Read Only Memory) 106, a RAM (Random Access Memory) 107, a pre-serving section 111, a photographing section 112, an editing section 113, and a printer 114 are respectively connected via a predetermined bus.

  The storage unit 102 has a non-volatile storage medium such as a hard disk or a flash memory, and stores various setting information supplied from the control device 101 in the storage medium or is recorded in the storage medium. The setting information is read out and supplied to the control device 101. Note that this storage medium may be anything as long as it can store information.

  The communication unit 103 is connected to another communication device (not shown) via an external network (not shown) such as the Internet or a public telephone line network, or simply via a communication cable (not shown). And is controlled by the control device 101 to communicate with other communication devices such as a user's mobile phone, a user's personal computer, or a central management server. For example, the communication unit 103 transmits transmission information supplied from the control device 101 to another communication device, or supplies reception information supplied from another communication device to the control device 101.

  The drive 104 includes a removable medium such as a magnetic disk (including a flexible disk), an optical disk (including a CD-ROM (Compact Disc-Read Only Memory), a DVD (Digital Versatile Disc)), a magneto-optical disk, and a semiconductor memory. The computer program and data read out from the computer 105 and the data read from them are supplied to the control device 101 as needed, and stored or installed in the storage unit 102 or the like.

  The ROM 106 stores programs and data executed in the control device 101 in advance, and the ROM 106 supplies the programs and data to the control device 101 based on instructions from the control device 101. The RAM 107 temporarily holds data and programs processed by the control device 101.

  The prior customer service unit 111 uses the prior customer service unit 11 to perform customer service for a user before photographing. For example, the pre-service section 111 displays a demonstration screen of a photo sticker creation game, accepts the insertion of a price (coin) for the photo sticker creation game, guides movement into the shooting space 32, or takes the shooting space 32. When the is in use, the user is made to wait. The customer service unit 111 includes a coin processing unit 121 in addition to the monitor 42 and the speaker 43 described above. When a coin is inserted into the coin insertion / return port 41, the coin processing unit 121 performs processing related to payment for the photo sticker creation device 1 by the user (game participation acceptance, etc.), charging processing, movement or standby guidance, etc. Is performed by the monitor 42 and the speaker 43.

  The photographing unit 112 is a block related to the photographing unit 12 and includes a background curtain 131, a lighting device 53, a camera 51, a touch panel monitor 52, and a speaker 55.

  The background curtain 131 is a unit having a plurality of elevating curtains used as a background and a control unit for controlling the curtains used as a background in the process of shooting a photo sticker creation game. For example, there are many types of curtains with different colors, patterns, patterns, and the like. Instead of an elevating curtain, a hanging curtain may be prepared.

  The lighting device 53 is turned on, turned off, or emits flash light based on the lighting control signal supplied from the control device 101.

  The camera 51 is controlled by the control device 101 to shoot a subject (user) in the shooting space 32 in the shooting work process of the photo sticker creation game. For example, the camera 51 captures a moving image under the control of the control device 101 and supplies the acquired moving image data to the control device 101. Further, the camera 51 acquires a captured image (still image) by operating a mechanical shutter based on a shutter control signal supplied from the control device 101, and supplies the acquired still image data to the control device 101. To do.

  The touch panel monitor 52 has a captured image display unit 132, and displays an image based on the RGB signal supplied from the control device 101 on the captured image display unit 132. The touch panel monitor 52 also includes a colorless and transparent touch panel (not shown) superimposed on the surface of the captured image display unit 132. When the user taps (contacts or approaches) the touch panel with his / her finger or the like, the touch panel monitor 52 supplies the position information to the control device 101. The control device 101 analyzes the position information, identifies and accepts information and instructions input by the user with respect to the display image.

  The speaker 55 outputs, for example, sound of a sound signal supplied from the control device 101 (shooting work guidance sound, sound effect, BGM, or the like).

  The editing unit 113 is a block related to the editing unit 13, and includes a first editing unit 113A corresponding to the first editing interface 13A and a second editing unit 113B corresponding to the second editing interface 13B. The first editing unit 113A and the second editing unit 113B have the same configuration, and the following description is given with respect to the first editing unit 113A, but is also applicable to the second editing unit 113B.

  As described with reference to FIG. 7, the first editing unit 113 </ b> A includes the tablet built-in monitor 62, the touch pen 63-1 and the touch pen 63-2, and the speaker 64.

  The tablet built-in monitor 62 includes an editing monitor 141 composed of a display device such as an LCD and a colorless and transparent tablet 142 superimposed on the display screen of the editing monitor 141. The tablet built-in monitor 62 displays on the editing monitor 141 an image relating to editing, such as an editing input GUI and a captured image, corresponding to the RGB signal supplied from the control device 101. When the user operates the touch pen 63-1 or the touch pen 63-2 and taps (contacts or approaches) the tablet 142, the tablet built-in monitor 62 supplies the position information to the control device 101. The control device 101 analyzes the position information, identifies and accepts information and instructions input by the user on the display image (GUI or the like). That is, editing input for a captured image (a composite image created using the captured image) is performed using these configurations.

  The speaker 64 outputs the sound of the sound signal supplied from the control device 101 (the sound related to the editing work process of the photo sticker creation game, such as the editing work guidance sound, the sound effect, or the BGM).

  The printer 114 has two heads (a head 151 and a head 152) that print the result of editing work performed by the editing unit 113 on sticker paper. When the printer 114 obtains the result of editing work performed by the first editing unit 113A from the control device 101, the printer 114 obtains the sticker paper 162 from the sticker paper unit 161 and prints the result of editing work on the sticker paper 162 in the head 151. . Further, when the printer 114 acquires the result of the editing work performed by the second editing unit 113B from the control device 101, the printer 114 acquires the sticker paper 162 from the sticker paper unit 161, and the editing result is converted into the sticker paper 162 by the head 152. Print.

  When the printing is finished, the printer 114 discharges the printed sticker paper 162 as a photo sticker from the photo sticker discharge port 71 and provides it to the user.

  FIG. 10 is a block diagram illustrating a functional configuration example of the control device 101.

  FIG. 10 shows a functional configuration related to creation of a composite moving image to be presented by a live view display to a user who is performing a shooting operation. In addition to the configuration shown in FIG. 10, a configuration for performing photographing processing, a configuration for performing editing processing, a configuration for performing printing processing, and the like are realized in the control apparatus 101.

  At least a part of the configuration shown in FIG. 10 is realized by executing a predetermined program by the control device 101.

  As shown in FIG. 10, in the control apparatus 101, a mask image creation unit 201, a difference extraction unit 202, a synthesis unit 203, and a display control unit 204 are realized. When the user starts the shooting operation, shooting of a moving image in which the user appears as a subject is started, and the shot moving image is supplied from the camera 51 to the mask image creating unit 201 and the combining unit 203. When moving images taken by the camera 51 are arranged on the time axis, it can be considered that the moving images are continuous still images.

  The mask image creation unit 201 performs a chroma process, a skin color area detection process, and the like on each frame, which is a still image arranged in time series constituting the moving image supplied from the camera 51, to thereby determine the area of the subject. A mask image, which is data that separates the background area, is created. For example, in the moving image supplied from the camera 51, a chroma curtain is reflected as the background of the subject, and the chroma processing can be performed for each frame.

  FIG. 11 is a diagram illustrating an example of a moving image captured by the camera 51.

In the example of FIG. 11, frames f _{1 to} f ₃ are shown as frames constituting a moving image taken by the camera 51. The horizontal direction represents the time direction.

In the frame f ₁ , the body of the subject is slightly on the right side with respect to the center of the frame, and the right arm of the subject is tilted downward. Further, in the frames f ₂ and f ₃ , the body of the subject is at the same position as the position in the frame f ₁ , and the right arm of the subject is in a horizontally oriented state or in a state in which the subject is raised obliquely.

  That is, FIG. 11 shows an example in which the user who is facing the front of the body to the camera 51 moves the right arm from obliquely downward to obliquely during photographing.

  When a moving image composed of such frames is supplied from the camera 51, the mask image creating unit 201 creates a mask image as shown in FIG. The number of vertical and horizontal pixels of the mask image is the same as the number of vertical and horizontal pixels of each frame constituting the moving image captured by the camera 51.

The mask images m _{1 to} m _{3 in} FIG. 12 are mask images created from the frames f _{1 to} f ₃ in FIG. 11, respectively, and a black area represents a subject area. Among the pixels of the entire mask image, the pixel value of the pixel corresponding to the blacked area is represented by RGB = (0, 0, 0), and the pixel value of the pixel corresponding to the other area is RGB = ( 255, 255, 255).

  The mask image creation unit 201 outputs the mask image created in this way to the difference extraction unit 202.

  The difference extraction unit 202 extracts a range in which the subject has moved within a predetermined period based on the mask image supplied from the mask image creation unit 201.

  FIG. 13 is a diagram illustrating an example of a range extracted by the difference extraction unit 202.

For example, when the mask images m _{1 to} m ₃ in FIG. 12 are created by the mask image creation unit 201, the difference extraction unit 202 obtains a difference between the mask images, and there is a difference between the mask images. A range A ₁ indicated by hatching 13 is extracted as a range in which the subject has moved. When the control device 101 has a function for recognizing the hand of the subject, only the range in which the hand has moved may be extracted as the range in which the subject has moved.

  The difference extraction unit 202 outputs information representing the range in which the subject has moved to the synthesis unit 203.

  The synthesizing unit 203 has a work area for holding count values corresponding to all pixels of a frame constituting a moving image to be displayed in live view. The count value is a value representing the density of each pixel of the foreground image to be combined with the subject image, and the density of each pixel of the foreground image is adjusted so as to have a density corresponding to the set count value. Is done. The number of vertical and horizontal pixels of each frame constituting a moving image to be displayed in live view, the number of vertical and horizontal pixels of each frame constituting the moving image taken by the camera 51, and the number of vertical and horizontal pixels of the foreground image are the same. And

  FIG. 14 is a diagram illustrating an example of the foreground image.

  FIG. 14 shows an example of a foreground image in which heart patterns are arranged on the whole. The foreground image may be a still image or a moving image. Moreover, not only a pattern (image) but also a character may be displayed.

  For example, all the count values of the work area are set to 0 in the initial state at the start of the photographing work. Since all the count values are 0, in the initial state, the foreground image is not seen on the synthesized moving image, and the moving image taken by the camera 51 is seen as it is.

  Further, based on the information supplied from the difference extraction unit 202, the synthesis unit 203 increases the count value corresponding to the pixels of the foreground image within the same range as the range in which the subject has moved among all the count values in the work area. On the other hand, the count value corresponding to the pixels of the foreground image within the same range as the range where the subject has not moved (the range where the subject is stationary) is decreased. As a result, the count value corresponding to the pixels of the foreground image within a range where there is continuous movement exceeds a certain value.

  The synthesizing unit 203 synthesizes the foreground image in which the density of each pixel is adjusted according to the count value by superimposing the foreground image on the moving image captured by the camera 51, and displays the synthesized moving image obtained by synthesizing the display control unit 204. Output to. Only the pixels of the foreground image whose count value exceeds a certain value may be adjusted so as to have a density that is visible on the composite image.

  FIG. 15 is a diagram illustrating an example of one frame constituting a moving image captured by the camera 51.

For example, a moving image including a frame as shown in FIG. 15 is supplied from the camera 51, and a range A ₁ as shown in FIG. 13 is extracted by the difference extraction unit 202 and is within the same range as the range A _1. When the pixel density of the foreground image is adjusted so as to have a density that can be visually recognized on the synthesized image, a synthesized image of one frame as shown in FIG.

In the example of FIG. 16, the foreground image (heart pattern) in the same range as the range A ₁ is synthesized as the foreground of the subject region extracted from the image of FIG. A range A ₁ indicated by a dotted line in FIG. 16 represents the same range as the range A _{1 in} FIG. 13 indicated by hatching.

  Since the count value is updated according to the movement of the subject and the density of each pixel of the foreground image is adjusted, the foreground image in the same range as the subject moved appears on the synthesized moving image, The foreground image in the same range as the range where the subject was stationary appears to disappear gradually.

  Thereby, the function which can make a pattern, a character, etc. appear by imitating like wiping a window with a elephant is realized. In the following, the function that makes an image or a character appear within the range in which the body is moving is referred to as a “zokin” function.

  Returning to the description of FIG. 10, the display control unit 204 displays the synthesized moving image supplied from the synthesizing unit 203 on the captured image display unit 132 of the touch panel monitor 52 in the imaging space 32. The user can perform the shooting operation while watching the synthesized moving image displayed in the live view.

  Here, processing of the photo sticker creating apparatus 1 having the above-described configuration will be described.

  First, the flow of the photo sticker creation game will be described with reference to the flowchart of FIG.

  A user who performs a photo sticker creation game first inserts a coin into the coin insertion / return port 41 of the pre-service unit 11 in the pre-service area 31 and pays for the game. The coin processing unit 121 accepts coin insertion and supplies an activation signal to the control device 101.

  In step S1, the control device 101 determines whether or not a price has been inserted based on the activation signal, and proceeds to step S2 if it is determined that a valid price has been inserted. The user who has entered the price moves to the shooting space 32 according to a guidance demonstration screen displayed on the monitor 42 of the advance customer service section 111 and starts shooting work.

  In step S <b> 2, the control device 101 controls each unit of the photographing unit 112 and executes a photographing process that is a process related to the photographing work of the photo sticker creation game performed in the photographing space 32. The photographing process will be described with reference to the flowchart of FIG.

  In step S <b> 3, the control device 101 performs a movement guidance process that prompts the user in the shooting space 32 who has finished the shooting operation to move to the first editing space 33 </ b> A or the second editing space 33 </ b> B by an image or sound. .

  In step S <b> 4, the control device 101 performs editing processing, which is processing related to editing work of the photo sticker creation game, on the user who has moved to the editing space 33.

  When the editing process is completed, in step S5, the control device 101 controls the printer 114 to perform a printing process for printing the edited image obtained by the editing work on a sticker sheet.

  In step S <b> 6, the control apparatus 101 performs post-service processing for providing services such as a mini game and image transfer to a user who is waiting for completion of printing.

  When the subsequent customer service process is completed, in step S7, the control device 101 performs movement guidance for moving the user of the editing space 33 to the printed material receiving area.

  When the printing process is completed, in step S8, the control device 101 controls the printer 114 to discharge the printed sticker paper from the photo sticker discharge port 71, and the photo sticker creation game process is finished.

  Next, with reference to the flowchart of FIG. 18, the imaging process performed in step S2 of FIG. 17 will be described.

  In step S <b> 11, the control device 101 causes the captured image display unit 132 to display the moving image captured by the camera 51 and starts live view.

  In step S <b> 12, the control device 101 adjusts the image and brightness of the camera 51 in accordance with a user input to the touch panel monitor 52.

  In step S13, the control apparatus 101 determines whether or not the user has instructed the start of the elephant function. The touch panel monitor 52 displays a button operated when instructing the start of the elephant function.

  If the button displayed on the touch panel monitor 52 has been operated and it is determined in step S13 that the start of the elephant function has been instructed, in step S14, the combining unit 203 of the control device 101 has adjusted the density of each pixel. The foreground image is combined as the foreground of the subject shown in the moving image supplied from the camera 51, and the combined moving image obtained by combining is output to the display control unit 204. The processing of the elder function for adjusting the density of each pixel of the foreground image, which is performed in parallel with the processing of FIG. 18, will be described with reference to the flowchart of FIG.

  In step S <b> 15, the display control unit 204 causes the captured image display unit 132 to display the combined moving image supplied from the combining unit 203. As a result, a synthesized moving image composed of images as shown in FIG. 16 is displayed on the captured image display unit 132.

  When it is instructed to start photographing, or when it is determined in step S13 that the start of the elephant function is not instructed, the control device 101 displays a countdown indicator on the touch panel monitor 52 in step S16.

  When the remaining time until shooting represented by the countdown indicator becomes 0 second, in step S17, the control device 101 performs shooting and stores one frame of the synthesized moving image at the shooting timing as a shot image.

  In step S18, the control device 101 determines whether or not shooting has been performed a predetermined number of times, and if it is determined that shooting has not been performed, that is, that the number of possible shots remains, the process returns to step S13. The subsequent processing is repeated.

  On the other hand, if it is determined in step S18 that shooting has been performed a predetermined number of times, in step S19, the control device 101 gives the user an image to be edited (keep image) from among a plurality of shot images. Select and save the selected captured image. Thereafter, the process returns to step S2 in FIG. 17, and the subsequent processing is performed.

  Next, processing of the elephant function will be described with reference to the flowchart of FIG.

  In step S31, the composition unit 203 of the control apparatus 101 sets all the count values of the work areas corresponding to the pixels of the foreground image as 0.

  In step S <b> 32, the mask image creation unit 201 creates a mask image by performing a process such as chroma for each frame constituting the moving image supplied from the camera 51. The mask image creation unit 201 outputs the created mask image to the difference extraction unit 202.

  In step S33, the difference extraction unit 202 obtains a difference between the mask images and extracts a range in which the subject has moved. The difference extraction unit 202 outputs information representing the range in which the subject has moved to the synthesis unit 203.

  In step S <b> 34, based on the information supplied from the difference extraction unit 202, the synthesis unit 203 increases the count value corresponding to the foreground image pixels in the same range as the range in which the subject has moved, and other foreground images. The count value is updated by decreasing the count value corresponding to the pixel.

  In step S35, the composition unit 203 adjusts the density of each pixel of the foreground image according to the updated count value, and repeats the processing from step S32 onward. The processing after step S32 is repeated while the live view display by the elephant function is being performed.

  Through the above processing, the user can enjoy shooting while looking at the foreground image that rises and disappears according to his / her own movement and stoppage.

  In the above, the foreground image in the initial state is transparent, and the foreground image is not visible on the synthesized moving image displayed in the live view. However, an image including at least a part of the translucent portion is used as the foreground image. You may be made to do. The part of the user's body that overlaps the translucent part looks like it was seen through the polished glass on the composite video.

  FIG. 20 is a diagram illustrating an example of one frame constituting a synthesized moving image displayed in live view during a shooting operation.

In the entire frame shown in FIG. 20, a range A ₁₁ indicated by hatching is a semi-transparent region. The area of the foreground image other than the range A ₁₁ is a transparent area, and the vicinity of the head of the subject overlapping this area can be clearly seen on the synthesized moving image. The frame shown in FIG. 20 is also created by the synthesis unit 203.

In the example of FIG. 20, a circle R ₁ is displayed in the range A ₁₁ . During the photographing operation, for example, guidance such as “Move your hand within a circle” is presented to the user by voice or screen display.

When the user who has confirmed this guidance moves the left hand or the like, the fact that the subject has moved within the range corresponding to the circle R ₁ is extracted as described above based on the moving image taken by the camera 51. A predetermined image such as a character is gradually displayed from the extracted part of the motion (the hidden image gradually appears). Finally, when movement is extracted over the entire range corresponding to the circle R ₁ , as shown in FIG. 21, the display of the translucent area A ₁₁ is switched as a transparent area, The entire image P ₁ that is the character image is displayed at the position where the circle R ₁ was displayed. That is, the circle R ₁ represents that some image is hidden in the portion.

Note that when the motion is extracted over the entire range corresponding to the circle R ₁ , the entire image P ₁ is not displayed, but a hidden image as shown by hatching in FIG. entire image P ₁ may also be displayed when the motion is extracted at the portion corresponding to the predetermined range of P _1. When the image of the pig's face as shown in FIG. 22 is prepared as the image P ₁ , this allows the user to move the hand within a range corresponding to a portion other than one ear of the pig's face, for example. The entire face can be displayed.

  Even when the face of the subject is detected for each frame constituting the moving image taken by the camera 51 and the movement of the subject is extracted on the face range, May not display the foreground image.

In the example of FIG. 20, the foreground image includes a semi-transparent region in part, but the whole may be transparent. When the whole is transparent, it is not known where the image of the character or the like is hidden on the synthesized moving image displayed in the live view. Therefore, as shown in FIG. May be specified. In FIG. 23, a range A ₁₁ indicated by a dotted line is a transparent region. Also, the circle R ₁ indicated by the dotted line is not actually displayed.

  Further, the density of the pixels of the foreground image within a different range may be increased instead of increasing the density of the pixels of the foreground image within the same range as the range in which the subject moves the body.

  FIG. 24 is a diagram showing an example of one frame constituting the synthesized moving image.

FIG. 24 shows a state where the density of each pixel of the foreground image is so low that it cannot be seen on the synthesized moving image. In such a state, when the user moves his hand in the range indicated by the white arrow in FIG. 24 and this is extracted, the foreground image is displayed in the same range as the white arrow according to the method described above. In this example, as shown in FIG. 25, the pattern of the foreground image is displayed in a range A ₂₁ outside the subject, which is different from the range where the hand is moved. The The foreground image is hidden by the foreground image by displaying the pattern of the foreground image at a position on the extended line of the line connecting the position where the motion is extracted and the position of the face detected from the mask image (face mask). Can be prevented.

  As a method of extracting the range in which the subject has moved, a plurality of mask images are created from the moving image captured by the camera 51, and the color difference of the pixels in the region of the subject in the mask image is obtained between the respective mask images. A range having a color difference may be extracted as a range in which the subject has moved. When the mask images are arranged on the time axis, the difference between the color in the subject area in the (n−1) th mask image and the color in the subject area in the nth mask image is obtained, and the color difference A certain range is extracted as a range in which the subject has moved.

  Next, another function for extracting a moving range of a subject and displaying a live image different in accordance with the extracted range during a shooting operation will be described.

  At the start of this function, the density of the pixels in the subject area extracted from the frame constituting the moving picture taken by the camera 51 is set to a density (transparent) that cannot be seen from the synthesized moving picture displayed in the live view. Adjusted.

  In this state, when the range in which the subject has moved is extracted as described above, the camera 51 increases the density of pixels within the same range as the range in which the subject has moved, and decreases the density of other pixels. The density of each pixel in the subject area extracted from each frame of the captured moving image is adjusted by the synthesis unit 203. As a result, a moving image in which the appearance of the subject appears or disappears according to the motion is created and displayed in live view.

  That is, this function does not cause an image to be displayed or disappeared by adjusting the density of each pixel of the foreground image, but the subject extracted from each frame of the moving image captured by the camera 51. By adjusting the density of each pixel in this area, the appearance of the subject is displayed or disappears.

  Hereinafter, the function of displaying or erasing the subject on the synthesized moving image in this way is referred to as a “transparent human” function.

  FIG. 26 is a diagram illustrating another example of a moving image captured by the camera 51.

In the example of FIG. 26, frames f _{11 to} f ₁₃ are shown as frames constituting a moving image photographed by the camera 51. The horizontal direction represents the time direction.

In the frame f _11, the body of the subject is on the left when on the basis of the center of the frame, the frame f _12, the body of the subject is approximately in the center of the frame. In the frame f _13, the body of the subject is on the right when on the basis of the center of the frame.

  FIG. 27 is a diagram illustrating an example of a mask image created by the mask image creation unit 201 based on the moving image of FIG.

Mask images m _{11 to} m _{13 in} FIG. 27 are mask images created from the frames f _{11 to} f _{13 in} FIG. 26, respectively. The difference extraction unit 202 obtains a difference between mask images as shown in FIG. 27, and extracts a range in which the subject has moved within a predetermined period.

  FIG. 28 is a diagram illustrating an example of a range extracted by the difference extraction unit 202.

For example, based on the mask images m _{11 to} m ₁₃ in FIG. 27, a range A ₂₁ indicated by hatching in FIG. 28 that is different between the mask images is extracted as a range in which the subject has moved. When the range A ₂₁ as shown in FIG. 28 is extracted by the difference extraction unit 202, the synthesis unit 203 extracts the frame A constituting the moving image photographed by the camera 51 by updating the count value. A moving image is created by increasing the density of the pixels within the same range as the range A ₂₁ among the pixels in the subject area and decreasing the density of the other pixels. Here, the count value represents the density of the pixels in the subject area extracted from the frame constituting the moving image photographed by the camera 51.

  FIG. 29 is a diagram illustrating an example of one frame constituting a moving image displayed in live view by the transparent human function.

As shown in FIG. 29, when a subject is shown in the center of a frame constituting a moving image photographed by the camera 51, the subject is inside a range A ₂₁ indicated by a dotted line. Among them, when the density of the pixels within the same range as the range A ₂₁ increases, the whole body of the subject is displayed on the moving image. Note that a range A ₂₁ indicated by a dotted line in FIG. 29 is not displayed on the moving image. When it is extracted that the subject is stationary for a predetermined time, the appearance of the subject displayed on the moving image gradually disappears as the count value is subtracted.

  In this way, the user can perform shooting work while having more enjoyment by making the appearance appear or disappear according to the movement. Depending on the timing of shooting, the user can take an image in which a part of his / her body has disappeared and select it as an image to be edited.

  It should be noted that here, the moving range of the subject is extracted, and the appearance of the subject is displayed on the moving image by adjusting the pixel density of the region of the subject appearing in the frame constituting the moving image captured by the camera 51. Although the density of the translucent foreground image is adjusted according to the range of movement of the subject, the appearance of the subject may be displayed or erased on the synthesized moving image. . In this case, among the pixels of the translucent foreground image, pixels within the same range as the range of movement of the subject are transparent, and the shape of the subject is displayed on the synthesized moving image.

  In the above, it is assumed that the foreground image is displayed within the same range as the subject moved, or the subject is displayed, but on the contrary, the range where there is no movement, that is, the range where the subject is stationary. The foreground image may be displayed within the same range, or the appearance of the subject may be displayed.

  FIG. 30 is a diagram illustrating still another example of a moving image captured by the camera 51.

In the example of FIG. 30, frames f _{21 to} f ₂₃ are shown as frames constituting a moving image taken by the camera 51. The horizontal direction represents the time direction.

In the frame f ₂₁ , the body of the subject is on the left side with respect to the center of the frame, and the body is tilted so that only the head is at the center of the frame. In the frame f _22, the body of the subject is approximately in the center of the frame. In the frame f _23, the body of the subject is on the right when on the basis of the center of the frame, the head only is the body is inclined such that the center of the frame. The position of the head is substantially the same in any of the frames f _{21 to} f ₂₃ .

  FIG. 31 is a diagram illustrating an example of a mask image created by the mask image creation unit 201 based on the moving image of FIG.

The mask image m ₂₁ to m ₂₃ in FIG. 31, respectively, is a mask image created from the frame f ₂₁ to f ₂₃ of Figure 30. In the difference extraction unit 202, a difference between mask images as shown in FIG. 31 is obtained, and a range where the subject is stationary within a predetermined period is extracted.

  FIG. 32 is a diagram illustrating an example of a range extracted by the difference extraction unit 202.

For example, based on the mask images m _{21 to} m ₂₃ in FIG. 31, a range A ₃₁ indicated by hatching in FIG. 32 that has no difference between the mask images is extracted as a range in which the subject is stationary. When the range A ₃₁ as shown in FIG. 32 is extracted by the difference extraction unit 202, the synthesis unit 203 extracts the frame A constituting the moving image captured by the camera 51 by updating the count value. A moving image is created by increasing the density of the pixels within the same range as the range A ₃₁ among the pixels of the subject area thus set and decreasing the density of the other pixels. Again, the count value represents the pixel density of the subject area extracted from the frame constituting the moving image taken by the camera 51.

  FIG. 33 is a diagram showing an example of one frame constituting a moving image photographed by the camera 51, and FIG. 34 is a diagram showing an example of one frame constituting a moving image displayed in live view by the transparent human function. It is.

34. As shown in FIG. 34, the density of the pixels in the same range as the range A ₃₁ among the pixels in the subject area shown in the frame of FIG. 33 constituting the moving image taken by the camera 51 increases. Only the head is displayed on the moving image.

  In this way, the user can also perform shooting work while having more enjoyment by displaying an image in a stationary range.

  It should be noted that here, a stationary range of the subject is extracted, and the shape of the subject is displayed on the moving image by adjusting the density of pixels in the region of the subject appearing in the frame constituting the moving image captured by the camera 51. In this case, the density of the semi-transparent foreground image is adjusted according to the range where the subject is stationary, so that the subject's appearance is displayed or erased on the composite moving image. You may do it. In this case, among the pixels of the translucent foreground image, pixels within the same range as the range where the subject is stationary are transparent, and the appearance of the subject is displayed on the synthesized moving image.

  The series of processes described above can be executed by hardware or can be executed by software. When a series of processing is executed by software, a program constituting the software executes various functions by installing a computer incorporated in dedicated hardware or various programs. It can be installed on a general-purpose personal computer or the like.

  The program to be installed is recorded on the removable medium 105 shown in FIG. 9 formed of an optical disc (CD-ROM (Compact Disc-Read Only Memory), DVD (Digital Versatile Disc), etc.), semiconductor memory, or the like, or locally. It is provided via a wired or wireless transmission medium such as an area network, the Internet, or digital broadcasting. The program can be installed in advance in the storage unit 102 or the ROM 106.

  The program executed by the computer may be a program that is processed in time series in the order described in this specification, or in parallel or at a necessary timing such as when a call is made. It may be a program for processing.

  The embodiments of the present invention are not limited to the above-described embodiments, and various modifications can be made without departing from the scope of the present invention.

It is a perspective view which shows one Embodiment of the photographic sticker production apparatus to which this invention is applied. It is a perspective view from the other angle of the photograph sticker production apparatus of FIG. It is a perspective view which shows the example of installation of the photograph sticker production apparatus of FIG. It is a figure explaining a user's space movement. It is a figure which shows the structural example of a prior customer service unit. It is a figure which shows the structural example of the front of a front unit. It is a figure which shows the structural example of the side surface of an edit unit. It is a figure which shows the structural example of the other side of an edit unit. It is a block diagram which shows the example of an internal structure of a photograph sticker production apparatus. It is a block diagram which shows the function structural example of a control apparatus. It is a figure which shows the example of the moving image image | photographed with the camera. It is a figure which shows the example of a mask image. It is a figure which shows the example of the range of a to-be-photographed object's movement. It is a figure which shows the example of a foreground image. It is a figure which shows the example of 1 frame which comprises the moving image image | photographed with the camera. It is a figure which shows the example of 1 frame which comprises a synthesized moving image. It is a flowchart explaining the flow of a photo sticker creation game. It is a flowchart explaining the imaging | photography process performed in step S2 of FIG. It is a flowchart explaining the process of an elephant function. It is a figure which shows the other example of 1 frame which comprises a synthetic | combination moving image. It is a figure which shows the further another example of 1 frame which comprises a synthetic | combination moving image. It is a figure which shows the other example of a foreground image. It is a figure which shows the example of 1 frame which comprises a synthesized moving image. It is a figure which shows the other example of 1 frame which comprises a synthetic | combination moving image. It is a figure which shows the further another example of 1 frame which comprises a synthetic | combination moving image. It is a figure which shows the other example of the moving image image | photographed with the camera. It is a figure which shows the other example of a mask image. It is a figure which shows the example of the range of a to-be-photographed object's movement. It is a figure which shows the example of 1 frame which comprises a moving image. It is a figure which shows the further another example of the moving image image | photographed with the camera. It is a figure which shows the further another example of a mask image. It is a figure which shows the example of the range of a to-be-photographed object's movement. It is a figure which shows the example of 1 frame which comprises the moving image image | photographed with the camera. It is a figure which shows the example of 1 frame which comprises a moving image.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Photo sticker production apparatus 12 Imaging unit 13 Editing unit 101 Control apparatus 201 Mask image creation part 202 Difference extraction part 203 Composition part 204 Display control part

Claims (5)

Take a picture of the user as a subject, let the user edit the image obtained by shooting, and create a photo sticker by printing the image obtained by shooting or the edited image on a sticker sheet In the photo sticker creation device that provides the user with the photo sticker created,
Photographing means for capturing a moving image reflected by the user as a subject;
Creating means for creating a mask image that separates the area of the subject from the background based on the respective frames that are still images arranged in chronological order constituting the moving image captured by the photographing means;
A motion detecting means for detecting a range in which the subject has moved every predetermined period based on the difference between the mask images created by the creating means;
According to the range detected by the motion detection means, the density of the pixels of the foreground image, which is the foreground of the photographed image, is adjusted, and the foreground image in which the density of each pixel is adjusted is taken as the moving image captured by the photography means. Compositing means for compositing as a foreground of a subject reflected in each frame constituting the;
A photo sticker creating apparatus comprising: display control means for displaying a synthesized moving image obtained by being synthesized by the synthesizing means during photographing of an image to be edited. The photo sticker creating apparatus according to claim 1, wherein the synthesizing unit adjusts the density of pixels in the same range as the range detected by the motion detecting unit among the pixels of the foreground image. The photo sticker creation according to claim 2, wherein the synthesizing unit further adjusts the density of pixels in the foreground image other than the pixels within the same range as the range detected by the motion detecting unit. apparatus. Take a picture of the user as a subject, let the user edit the image obtained by shooting, and create a photo sticker by printing the image obtained by shooting or the edited image on a sticker sheet In the photo sticker creation method of the photo sticker creation device that provides the created photo sticker to the user,
A user captures a moving image that appears as a subject,
Create a mask image that separates the subject area and the background based on each frame that is a still image arranged in chronological order that constitutes the captured moving image,
Based on the difference between the created mask images, detect the range in which the subject moved every predetermined period,
According to the detected range, the density of the pixels of the foreground image that is the foreground of the captured image is adjusted, and the foreground of the subject that appears in each frame constituting the captured moving image is obtained by adjusting the density of each pixel. Synthesized as
A method for creating a photo sticker, comprising: displaying a synthesized moving image obtained by synthesizing during shooting of an image to be edited. Take a picture of the user as a subject, let the user edit the image obtained by shooting, and create a photo sticker by printing the image obtained by shooting or the edited image on a sticker sheet In a program for causing a computer to execute the processing of the photo sticker creating apparatus that provides the user with the photo sticker created,
A user captures a moving image that appears as a subject,
Create a mask image that separates the subject area and the background based on each frame that is a still image arranged in chronological order that constitutes the captured moving image,
Based on the difference between the created mask images, detect the range in which the subject moved every predetermined period,
According to the detected range, the density of the pixels of the foreground image that is the foreground of the captured image is adjusted, and the foreground of the subject that appears in each frame constituting the captured moving image is obtained by adjusting the density of each pixel. Synthesized as
A program including a step of displaying a synthesized moving image obtained by synthesizing during shooting of an image to be edited.

Images