JP5076911B2 - Image photographing and editing apparatus and method, and program - Google Patents

Description

The present invention relates to an image capturing / editing apparatus and method, and a program, and more particularly, to an image capturing / editing apparatus and method, and a program that enable a user to perform capturing while enjoying the image .

  One game provided by a conventional photo sticker creating apparatus installed in an entertainment (game) facility, etc., is to shoot a subject, edit (doodle) the shot image, an edited image, or a sticker sheet of the shot image It ends with the flow of printing.

  In such a photo sticker creation apparatus, various edits can be performed on a captured image in which the user himself / herself is photographed, but the captured image to be edited before editing is performed. If the image itself is not according to the user's preference, editing will be performed without satisfaction. That is, editing cannot be enjoyed, and further, satisfaction with the game is lacking.

  Therefore, in recent years, various methods have been used to capture satisfactory captured images, and to enable users to perform interesting shooting operations. A photo sticker creating apparatus capable of acquiring a photographed image has been developed.

  For example, there is a photo sticker creation apparatus that combines a foreground image or background image selected by a user with a moving image of a subject captured by a camera at the time of shooting, and displays the combined image in a live view. The user can take a picture while viewing his / her own image combined with the foreground image and the background image.

  Patent Document 1 discloses a technique for combining a background image with a region other than a subject region in a photographed image and displaying an image obtained by further combining a foreground image with the composite image for live view display. ing.

  Japanese Patent Application Laid-Open No. 2004-228561 discloses a technique for synthesizing a moving image of a subject being captured and displaying it in a live view in an area designated by a user among photographed images taken once.

  Patent Document 3 discloses a technique for detecting a specific part of a subject from a photographed image and combining and displaying a foreground moving image such as a predetermined object based on the detected position of the specific part. .

  In Patent Document 4, a frame image is displayed as a moving image, and the frame image is stopped in accordance with a user's designation so that the user can synthesize a subject image in which he / she is photographed and a preferred frame image. A technology that can perform the above is disclosed.

  Patent Documents 5 and 6 disclose a technique in which a moving image of a subject is displayed, and the displayed moving image can be edited using a touch pen or the like.

JP 2004-320077 A JP 2007-241199 A JP 2000-92481 A Japanese Patent Laid-Open No. 10-282571 JP 2002-94906 A JP 2002-125176 A

  However, according to the technique disclosed in Patent Document 1, since the composition image (foreground image / background image) used to compose the subject image is a still image, such a still image is composed. Even if you can see your own image combined with the work image, the shooting work is not interesting.

  Further, according to the technique disclosed in Patent Document 2, since it is necessary to perform shooting twice or to specify a region for synthesizing a moving image, the operation becomes complicated for the user and the operability is improved. It will decline.

  According to the technique disclosed in Patent Document 3, for example, when a foreground image is synthesized in response to finger detection, when a finger is not captured in a captured image (when it is different from a sample image of a capturing pose presented at the time of capturing) ), The foreground image will not be displayed, and in such a case, it will be uninteresting.

  According to the technique disclosed in Patent Document 4, when acquiring a frame image of a still image, if the timing of the frame image that is a moving image does not match the timing of shooting, The frame image intended by the user cannot be obtained, and as a result, the composite image becomes an unsatisfactory image.

  According to the techniques disclosed in Patent Documents 5 and 6, editing can be performed while watching a moving image. However, since it is necessary to operate a touch pen or the like to perform the editing, the operation is difficult for the user. It becomes complicated and the operability deteriorates.

  As described above, according to the conventional technology, since the operation is complicated for the user, the operability is deteriorated, or the photographed image that is not interesting and is not satisfactory is provided. doing. Therefore, the user's satisfaction with the game is inevitably lowered.

  The present invention has been made in view of such a situation, and enables a user to take a picture while having more fun.

An image capturing / editing device according to one aspect of the present invention captures a user as a subject, causes the user to edit an image obtained by capturing, and the image obtained by capturing or the edited image. In an image photographing / editing apparatus for creating an image, a photographing unit that captures a moving image that is captured as a subject by a user, and a subject in each frame that is a still image arranged in a time-series order that constitutes the moving image captured by the photographing unit. A region detecting unit for detecting a region, a motion detecting unit for detecting a speed of movement of the subject for each predetermined period based on the region of the subject detected by the region detecting unit, and the motion detecting unit depending on the speed of movement of the subject, and creating means for creating an image with different sizes of a plurality ordered predetermined pattern as synthesis image, created by the creation means The composition image is synthesized as a foreground or background of a subject shown in the moving image captured by the photographing means, and the synthesized moving image obtained by the composition is synthesized as an object to be edited. Display control means for displaying the image to be captured.

  The user can take a picture while viewing a synthesized moving image in which a moving image in which he / she appears and a synthesis image created according to the content of his / her movement are synthesized. Further, the user can display the synthesized moving image without performing complicated operations.

  That is, the user can more easily and enjoy shooting, thereby obtaining satisfaction.

  The area detecting means may be a means for detecting a subject area by a chroma processing, a skin color area detecting process, or the like.

The creating means can create the composition image having a different number of patterns to be displayed or different color densities according to the speed of movement of the subject detected by the motion detecting means.

An image capturing / editing method or program according to an aspect of the present invention detects a region of a subject in each frame, which is a still image arranged in chronological order that constitutes the captured moving image, by capturing a moving image that the user shows as a subject. Based on the detected area of the subject, the speed of movement of the subject for each predetermined period is detected, and the size of the predetermined pattern arranged in plural is changed according to the detected speed of movement of the subject. An image is created as a synthesis image, and the created synthesis image is synthesized as a foreground or background of a subject shown in the captured moving image, and the synthesized moving image obtained by the synthesis is an image to be edited A step of displaying during shooting.

In one aspect of the present invention, a moving image in which a user appears as a subject is captured, and a region of the subject in each frame, which is a still image arranged in chronological order that constitutes the captured moving image, is detected. Also, based on the detected area of the subject, the speed of movement of the subject for each predetermined period is detected, and a plurality of predetermined pattern sizes arranged in accordance with the detected speed of movement of the subject. An image in which is changed is created as a composition image. The created composite image is combined as the foreground or background of the subject appearing in the captured moving image, and the combined moving image obtained by combining is displayed during shooting of the image to be edited. The

  According to one aspect of the present invention, a user can perform shooting while having more fun.

  FIG. 1 is a diagram showing an embodiment of a photo sticker creating apparatus to which the present invention is applied.

  The photo sticker creating apparatus 1 shown in FIG. 1 enhances the amusement of work such as shooting and editing, allows the user to perform such work as a game (play service), and takes the shot / edited images as photo stickers. It is a game device (play service providing device) that receives money from the user instead of providing it to the user as data. The photo sticker creating apparatus 1 is usually installed in a game center or a store in a sightseeing spot.

  In the photo sticker creating game executed by the photo sticker creating device 1, the user inputs the price of the game, shoots himself / herself using the camera provided in the photo sticker creating device 1, and the photographed image is taken. Then, editing is performed using an editing function (doodle editing function) such as pen image input or stamp image input, and the photographed image is designed to be rich and colorful. After the game ends, the user receives a photo sticker or the like on which the edited image is printed as a product.

  As will be described in detail later, in the photo sticker creating apparatus 1, the content of the movement of the user as the subject is detected from the moving image captured by the camera during the shooting operation in the series of photo sticker creating games, and the detection is performed. The moving image for synthesis created according to the content of the moved motion is combined with the moving image being captured, and the combined moving image is displayed in live view.

  Since the moving image for synthesis according to the magnitude of the movement and the speed of the movement is combined with the moving image of the subject, the user views the moving image displayed in the live view by changing according to his / her movement. You can enjoy shooting while.

  As shown in FIG. 1, the housing 10 of the photo sticker creating apparatus 1 is roughly divided into four units: a pre-service unit 11, a photographing unit 12, an editing unit 13, and a ceiling strobe unit 14.

  The pre-service unit 11 is a unit that accepts a photo sticker creation game and outputs content (demonstration) that guides (explains) the flow and play of the photo sticker creation game with video and audio.

  The photographing unit 12 has a function of photographing a subject (such as a user), and the subject is photographed in an internal space (photographing space) described later. The photographing unit 12 is roughly divided into two units, a front unit 12A and a rear unit 12B.

  The front unit 12A is a unit on the front side of the shooting space, and performs processing related to shooting of the user in the shooting space and accepts operations related to the user's photo sticker creation game. The rear unit 12B is a unit on the side that is the rear side of the shooting space, and is a unit that plays the role of the user when the front unit 12A takes a picture. The interior of the rear unit 12B is a cavity (space), and is configured as a part of a space (photographing space described later) used when the user performs photographing. Inside the rear unit 12B, for example, a movable curtain (background curtain) used for the background, a stage used for the user to pose, and the like may be provided.

  The editing unit 13 is provided with a configuration (for example, a monitor, a touch panel, and a touch pen) for the user to perform graffiti editing or the like on the captured image. The editing unit 13 composed of one frame is provided with a configuration for performing graffiti editing or the like on each of the two surfaces so that two sets of users can perform editing work simultaneously. That is, the photo sticker creation apparatus 1 can execute a plurality of photo sticker creation games in parallel. The editing unit 13 has a configuration for performing two graffiti editing and the like so that a plurality of sets of users can simultaneously perform graffiti editing in order to improve the rotation rate of the photo sticker creation game. Is provided.

  The ceiling strobe unit 14 provided above the photographing space incorporates a flash that emits strobe light toward the photographing space in accordance with the photographing timing. The ceiling strobe unit 14 further includes a fluorescent lamp, and functions as illumination that constantly illuminates the photographing space (the user in the photographing space). The photographing space is simply configured as a closed space as will be described later. Therefore, it is difficult for light to enter from the outside, and the user's operability may be deteriorated because it tends to be dark. The ceiling strobe unit 14 illuminates the shooting space with the illumination, thereby avoiding such a fear and improving operability. Such illumination is also provided in the front unit 12A and the rear unit 12B.

  FIG. 2 is a perspective view of the photo sticker creating apparatus 1 of FIG. 1 as viewed from the editing unit 13 side.

  As shown in FIG. 2, the editing unit 13 is installed adjacent to the surface of the front unit 12A opposite to the surface on the rear unit 12B side (that is, the surface on the imaging space side). A first editing interface 13A and a second editing interface 13B are provided on two side surfaces of the editing unit 13 that are orthogonal to the front unit 12A. That is, the editing unit 13 allows a plurality of sets of users who perform the photo sticker creation game to perform editing operations with the editing unit 13 facing each other in a direction orthogonal to the direction connecting the front unit 12A and the rear unit 12B. Two editing interfaces are provided so that this can be done. Details of the configuration of these editing interfaces will be described later.

  Also, on the surface of the editing unit 13 that faces the photographing unit 12, a photo sticker discharge unit 13C that is provided to the user by discharging the photo sticker created by playing the photo sticker making game is provided. ing.

  FIG. 3 is an external view showing an installation example of the photo sticker creating apparatus 1.

  When the photo sticker creating apparatus 1 is installed in a game center or the like, for example, as shown in FIG. 3, a curtain 21 is provided so as to surround the photographing unit 12 and the editing unit 13, and a photographing space where photographing work is performed or The editing space where the editing work is performed is simply enclosed, and the closeability of each space is improved. In other words, for example, the curtain 21 stabilizes the environment in each space, thereby contributing to high-quality shooting, easy viewing of the editing screen, etc., or blocking the field of view from outside each space, It is possible to perform operations such as shooting and editing without being aware of the line of sight.

  The outer surface of the curtain 21 is usually used as a place for posting advertisements, and a flashy design image representing the model name and model image of the photo sticker creating apparatus 1 is printed so as to attract attention at the installation location. Is done.

  The photo sticker creating apparatus 1 as described above causes each set of users to play a photo sticker creating game. The photo sticker creating apparatus 1 can simultaneously execute a plurality of photo sticker creating games. That is, the photo sticker creating apparatus 1 can cause a plurality of sets to play a photo sticker creating game at the same time. At this time, each group of users performs each work of the photo sticker creation game while moving through a plurality of spaces provided in the photo sticker creation device 1, and advances the game.

  The flow of the photo sticker creation game and the accompanying movement of the space will be described with reference to FIG.

  FIG. 4 is a plan view of the photo sticker creating apparatus 1 as viewed from above.

  As shown in FIG. 4, the user starts the photo sticker creation game by charging the pre-service unit 11 in the pre-service area 31, which is the area around the front of the pre-service unit 11 (user A). ). The user enters from the side of the pre-serving unit 11 if the shooting space 32 for performing the shooting operation, which is formed between the front unit 12A and the rear unit 12B and is indicated by the dotted line in the drawing, is not in use. (User B). If the previous group of users is using the shooting space 32 (if shooting is in progress), the user waits in the pre-service area 31 (the pre-service unit 11 instructs the standby).

  The pre-service unit 11 can also be provided on the opposite side of the front unit 12A as indicated by the dotted line. In this case, the user starts the photo sticker creation game from the one with the pre-service unit 11 Then (user A ′), after the pre-service, the user enters the shooting space 32 from the side opposite to the above-mentioned side (user B).

  A user (user B) who enters the shooting space 32 performs a shooting operation of shooting himself or herself using a camera or monitor provided in front of the front unit 12A.

  When the photographing work is completed, the user selects a keep image to be kept for editing and printing from the obtained photographed images, and follows the displayed movement guide from the photographing space 32 where the photographing work is performed. Move to the editing space indicated by the dotted line.

  A first editing space 33A is provided on the left side of the editing unit 13 as viewed from the front unit 12A (in front of the first editing interface 13A), and a second editing space 33B is provided on the right side (in front of the second editing interface 13B). ing. That is, in this photo sticker creation apparatus 1, a total of two sets of users, the user (user C) of the first editing space 33A and the user (user C ') of the second editing space 33B, simultaneously edit. Work can be done. A user (user B) of the shooting space 32 is guided to a free one of the two editing spaces. When the user moves to the first editing space 33A or the second editing space 33B, the user performs a graffiti editing work. Hereinafter, when it is not necessary to distinguish the first editing space 33A and the second editing space 33B, they are referred to as an editing space 33.

  Thereafter, when the graffiti editing is completed, the user (user C or user C ′) next performs print setting work and sets the number of divisions and the like. When printing is started, the user plays a mini game during the printing. Then, when printing is completed, it moves to the printed matter receiving area 34 in front of the photo sticker discharge unit 13C (user D), receives the discharged photo sticker paper, and ends the photo sticker creation game.

  Next, the configuration of each unit will be described.

  FIG. 5 is a diagram illustrating a configuration example of the advance customer service unit 11.

  As shown in FIG. 5, the customer service unit 11 accepts a price for a photo sticker creation game inserted by the user, returns a change to the user, etc., and a photo sticker creation game. And a monitor 42 for displaying a guide or demonstration screen of the game method and a speaker 43 for outputting sound corresponding to the image displayed on the monitor 42, sound effects, and the like.

  Since the image displayed on the monitor 42 and the sound output from the speaker 43 are output to the outside of the housing 10 (outside the area surrounded by the curtain 21), use before the photo sticker creation game is performed. In addition to deepening the understanding of how to play the photo sticker creation game of the person who is going to be a photographer, advertisement (invitation) to the people around the photo sticker creation device 1 who are not playing the photo sticker creation game It also has an effect.

  When the user inserts a predetermined price into the coin insertion / return slot 41 to play the photo sticker creation game, the user is directed to the monitor 42 on the monitor 42 when the shooting space 32 is free. A guidance screen is displayed, and a sound corresponding to the guidance screen is output from the speaker 43. When the shooting space 32 is in use, a guidance screen is displayed on the monitor 42 to prompt the user to stand by on the spot (pre-service area 31), and the corresponding sound is output from the speaker 43. Is done.

  As described above, the pre-service unit 11 may be installed on the right side of the front unit 12A as shown in FIG. 5, or on the left side of the front unit 12A as shown in FIG. May be.

  FIG. 6 is a diagram illustrating a configuration example of the front unit 12A.

  A front surface 12A-1 shown in FIG. 6 is a side surface facing the imaging space 32, and is a surface positioned in front of the user who performs the imaging operation. That is, the user basically performs a photographing operation in the photographing space 32 toward the front surface 12A-1 (facing the front unit 12A side).

  On the front surface 12A-1, a camera 51 that shoots a subject in the shooting space 32, for example, an acquired image or a shooting result obtained (photographed) in real time (immediately) after being photoelectrically converted by the camera 51. In addition to displaying images related to shooting operations, such as GUI images for shooting images, guidance regarding shooting operations, and GUI (Graphical User Interface) images for selecting background images, etc., and user instructions via a touch panel superimposed on the screen Touch panel monitor 52 that illuminates, illumination device 53-1 to illumination device 53-8 that illuminates the imaging space 32 or emits flash light in accordance with the imaging timing of the camera 51, and baggage of the user during the imaging operation For example, the luggage storage area 54-1 and the luggage storage area 54-2 for placing the sound, etc. Speakers 55-1 and a speaker 55-2 is provided is.

  The camera 51 is configured by an imaging device using a CCD (Charge Coupled Device) or the like, and photographs a user in the shooting space 32 in the shooting process of the photo sticker creation game. For example, a camera using a CMOS (Complementary Metal Oxide Semiconductor) sensor may be used as the camera 51 instead of the CCD. Of course, any camera other than these may be used as long as the user can be photographed and digital image data can be obtained. Further, the number of cameras 51 and the installation position are arbitrary.

  The touch panel monitor 52 is configured by an LCD (Liquid Crystal Display) or the like, and includes an acquired image of the camera 51, a composite image to be combined with the captured image, a captured image stored as a captured result, or a guide image that guides the procedure of the capturing operation. This is a monitor that displays an image relating to a shooting operation. A colorless and transparent touch panel (for example, a pressure-sensitive type or an electromagnetic induction type) is superimposed on the screen of the touch panel monitor 52. For example, the position information (user instruction) is obtained by tapping with a user's finger or the like. Can be entered.

  The illuminating devices 53-1 to 53-8 are configured by a plurality of illuminating devices that can emit fluorescent light and strobe light, and in order to brighten the photographic space 32 and ensure operability and safety, the photographic space 32 is provided. In order to improve the image quality of the photographed image, the flash is emitted at a timing corresponding to the photographing timing, and the user who is photographed is irradiated with light from various angles. Hereinafter, the lighting devices 53-1 to 53-8 are referred to as the lighting devices 53 when it is not necessary to distinguish them from each other for explanation.

  The luggage storage area 54-1 and the luggage storage area 54-2 are shelves that are used as storage locations for the user's baggage at the time of photographing. Both the luggage storage area 54-1 and the luggage storage area 54-2 do not interfere with shooting, that is, do not interfere with the user taking a shooting pose, and the illumination device 53 emits light. It is provided at a position that does not interfere with The user can perform photographing work more comfortably by placing the baggage in the luggage storage area 54-1 or the luggage storage area 54-2. Hereinafter, when it is not necessary to distinguish between the luggage storage area 54-1 and the luggage storage area 54-2, they are simply referred to as the luggage storage area 54.

  The speaker 55-1 and the speaker 55-2 appropriately output sound such as guidance sound, BGM (Back Ground Music), and sound effects in the shooting work of the photo sticker creation game. When it is not necessary to distinguish the speaker 55-1 and the speaker 55-2 from each other, they are referred to as a speaker 55. Note that the number and installation positions of the speakers 55 are arbitrary.

  Next, the configuration of the editing unit 13 will be described.

  FIG. 7 is a diagram showing a configuration example of the left side of the editing unit 13 when viewed from the photographing unit 12 side (configuration example on the first editing space 33A side).

  In FIG. 7, a first editing interface 13A is provided on the left side surface. On the upper side, an illuminating device 61 is provided for illuminating the first editing space 33A that is darkened because it is partitioned by the curtain 21 or the like. The illuminating device 61 allows the photo sticker creating apparatus 1 to suppress a reduction in the user's editing work efficiency and improve the safety of the first editing space 33A. In the example of FIG. 7, the upper portion of the editing unit 13 on the first editing interface 13 </ b> A where the lighting device 61 is provided penetrates to the second editing space 33 </ b> B side. The first editing space 33A and the second editing space 33B are shared. That is, the illumination device 61 illuminates not only the first editing space 33A but also the second editing space 33B, thereby improving safety. Note that the installation position and the number of the illumination devices 61 are arbitrary.

  The first editing interface 13A is provided with a tablet built-in monitor 62, two touch pens (touch pen 63-1 and touch pen 63-2), and a speaker 64.

  The tablet built-in monitor 62 displays a GUI image (doodle editing input screen) and the like for editing a photographed image obtained by photographing work in the photographing space.

  The tablet built-in monitor 62 is a monitor having a tablet capable of inputting position information with the touch pen 63-1 or the touch pen 63-2 and a display device such as an LCD capable of displaying an image. Here, the tablet is, for example, a pressure-sensitive type or electromagnetic induction type input device, is colorless and transparent, and is installed so as to be superimposed on the display screen of the display device. That is, the tablet built-in monitor 62 not only displays a GUI image or the like by the display device but also accepts an input operation from the user by the tablet.

  Touch pens 63-1 and 63-2 corresponding to the touch panel superimposed on the screen are detachably fixed to the editing unit 13 by a predetermined attachment near the left and right of the tablet built-in monitor 62. By using these touch pens, two users in the same group can play the editing process of the photo sticker creation process at the same time. In the following description, the touch pen 63-1 and the touch pen 63-2 are simply referred to as the touch pen 63 when it is not necessary to distinguish them from each other.

  The touch pen 63 is also electrically connected to a system installed in the editing unit 13 by wire (or wireless). The touch pen 63 operates in cooperation with a colorless and transparent tablet superimposed on the display screen of the display device of the tablet built-in monitor 62. When the user operates these touch pens 63 and taps the tablet (the touch pen 63 and the display screen of the tablet built-in monitor 62 are brought into contact (or close proximity)), these are inputs that accept input from the user. Acts as a means. The number of touch pens 63 to be installed, the installation position, the shape, and the like are arbitrary.

  The speaker 64 outputs, for example, sound related to editing work of the photo sticker creation game, such as guidance sound, sound effects, or BGM. The number, design, shape, etc. of the speakers 64 are arbitrary.

  Note that the configuration of the editing unit 13 on the second editing space 33B side is basically the same as the configuration of the first editing space 33A side shown in FIG. In the following, the configuration example shown in FIG. 7 is also applied to the case where the configuration of the editing unit 13 on the second editing space 33B side is described.

  FIG. 8 is a diagram illustrating a configuration example of the photo sticker discharge unit 13 </ b> C of the editing unit 13.

  As shown in FIG. 8, the photo sticker discharging unit 13C has a photo sticker created by a user who has performed editing work in the first editing space 33A (that is, a photo created by operating the first editing interface 13A). And a photo sticker discharge port 71 through which a photo sticker created by a user who has performed editing work in the second editing space 33B (that is, a photo sticker created by operating the second editing interface 13B) is discharged. Is provided.

  A printer is installed inside the editing unit 13, and a printing image created by operating the first editing interface 13A or the second editing interface 13B is printed on sticker paper by the printer. The photo sticker output from the printer is discharged from the photo sticker discharge port 71.

  Next, the internal configuration of the photo sticker creating apparatus 1 will be described.

  FIG. 9 is a block diagram illustrating a hardware configuration example of the photo sticker creating apparatus 1. The same parts as those described above are denoted by the same reference numerals and description thereof is omitted.

  In FIG. 9, the photo sticker creating apparatus 1 includes a control device 101 that is a processing unit that controls the entire apparatus of the photo sticker creating device 1. The control device 101 includes a storage unit 102, a communication unit 103, A drive 104, a ROM (Read Only Memory) 106, a RAM (Random Access Memory) 107, a pre-serving section 111, a photographing section 112, an editing section 113, and a printer 114 are respectively connected via a predetermined bus.

  The storage unit 102 has a non-volatile storage medium such as a hard disk or a flash memory, and stores various setting information supplied from the control device 101 in the storage medium or is recorded in the storage medium. The setting information is read out and supplied to the control device 101. Note that this storage medium may be anything as long as it can store information.

  The communication unit 103 is connected to another communication device (not shown) via an external network (not shown) such as the Internet or a public telephone line network, or simply via a communication cable (not shown). And is controlled by the control device 101 to communicate with other communication devices such as a user's mobile phone, a user's personal computer, or a central management server. For example, the communication unit 103 transmits transmission information supplied from the control device 101 to another communication device, or supplies reception information supplied from another communication device to the control device 101.

  The drive 104 includes a removable medium such as a magnetic disk (including a flexible disk), an optical disk (including a CD-ROM (Compact Disc-Read Only Memory), a DVD (Digital Versatile Disc)), a magneto-optical disk, and a semiconductor memory. The computer program and data read out from the computer 105 and the data read from them are supplied to the control device 101 as needed, and stored or installed in the storage unit 102 or the like.

  The ROM 106 stores programs and data executed in the control device 101 in advance, and the ROM 106 supplies the programs and data to the control device 101 based on instructions from the control device 101. The RAM 107 temporarily holds data and programs processed by the control device 101.

  The prior customer service unit 111 uses the prior customer service unit 11 to perform customer service for a user before photographing. For example, the pre-service section 111 displays a demonstration screen of a photo sticker creation game, accepts the insertion of a price (coin) for the photo sticker creation game, guides movement into the shooting space 32, or takes the shooting space 32. When the is in use, the user is made to wait. The customer service unit 111 includes a coin processing unit 121 in addition to the monitor 42 and the speaker 43 described above. When a coin is inserted into the coin insertion / return port 41, the coin processing unit 121 performs processing related to payment for the photo sticker creation device 1 by the user (game participation acceptance, etc.), charging processing, movement or standby guidance, etc. Is performed by the monitor 42 and the speaker 43.

  The photographing unit 112 is a block related to the photographing unit 12 and includes a background curtain 131, a lighting device 53, a camera 51, a touch panel monitor 52, and a speaker 55.

  The background curtain 131 is a unit having a plurality of elevating curtains used as a background and a control unit for controlling the curtains used as a background in the process of shooting a photo sticker creation game. For example, there are many types of curtains with different colors, patterns, patterns, and the like. Instead of an elevating curtain, a hanging curtain may be prepared.

  The lighting device 53 is turned on, turned off, or emits flash light based on the lighting control signal supplied from the control device 101.

  The camera 51 is controlled by the control device 101 to shoot a subject (user) in the shooting space 32 in the shooting work process of the photo sticker creation game. For example, the camera 51 captures a moving image under the control of the control device 101 and supplies the acquired moving image data to the control device 101. Further, the camera 51 acquires a captured image (still image) by operating a mechanical shutter based on a shutter control signal supplied from the control device 101, and supplies the acquired still image data to the control device 101. To do.

  The touch panel monitor 52 has a captured image display unit 132, and displays an image based on the RGB signal supplied from the control device 101 on the captured image display unit 132. The touch panel monitor 52 also includes a colorless and transparent touch panel (not shown) superimposed on the surface of the captured image display unit 132. When the user taps (contacts or approaches) the touch panel with his / her finger or the like, the touch panel monitor 52 supplies the position information to the control device 101. The control device 101 analyzes the position information, identifies and accepts information and instructions input by the user with respect to the display image.

  The speaker 55 outputs, for example, sound of a sound signal supplied from the control device 101 (shooting work guidance sound, sound effect, BGM, or the like).

  The editing unit 113 is a block related to the editing unit 13, and includes a first editing unit 113A corresponding to the first editing interface 13A and a second editing unit 113B corresponding to the second editing interface 13B. The first editing unit 113A and the second editing unit 113B have the same configuration, and the following description is given with respect to the first editing unit 113A, but is also applicable to the second editing unit 113B.

  As described with reference to FIG. 7, the first editing unit 113 </ b> A includes the tablet built-in monitor 62, the touch pen 63-1 and the touch pen 63-2, and the speaker 64.

  The tablet built-in monitor 62 includes an editing monitor 141 composed of a display device such as an LCD and a colorless and transparent tablet 142 superimposed on the display screen of the editing monitor 141. The tablet built-in monitor 62 displays on the editing monitor 141 an image relating to editing, such as an editing input GUI and a captured image, corresponding to the RGB signal supplied from the control device 101. When the user operates the touch pen 63-1 or the touch pen 63-2 and taps (contacts or approaches) the tablet 142, the tablet built-in monitor 62 supplies the position information to the control device 101. The control device 101 analyzes the position information, identifies and accepts information and instructions input by the user on the display image (GUI or the like). That is, editing input for a captured image (a composite image created using the captured image) is performed using these configurations.

  The speaker 64 outputs the sound of the sound signal supplied from the control device 101 (the sound related to the editing work process of the photo sticker creation game, such as the editing work guidance sound, the sound effect, or the BGM).

  The printer 114 has two heads (a head 151 and a head 152) that print the result of editing work performed by the editing unit 113 on sticker paper. When the printer 114 obtains the result of editing work performed by the first editing unit 113A from the control device 101, the printer 114 obtains the sticker paper 162 from the sticker paper unit 161 and prints the result of editing work on the sticker paper 162 in the head 151. . Further, when the printer 114 acquires the result of the editing work performed by the second editing unit 113B from the control device 101, the printer 114 acquires the sticker paper 162 from the sticker paper unit 161, and the editing result is converted into the sticker paper 162 by the head 152. Print.

  When the printing is finished, the printer 114 discharges the printed sticker paper 162 as a photo sticker from the photo sticker discharge port 71 and provides it to the user.

  FIG. 10 is a block diagram illustrating a functional configuration example of the control device 101.

  FIG. 10 shows a functional configuration related to creation of a composite moving image to be presented by a live view display to a user who is performing a shooting operation. In addition to the configuration shown in FIG. 10, a configuration for performing photographing processing, a configuration for performing editing processing, a configuration for performing printing processing, and the like are realized in the control apparatus 101.

  At least a part of the configuration shown in FIG. 10 is realized by executing a predetermined program by the control device 101.

  As shown in FIG. 10, the control device 101 implements an area detection unit 201, a motion detection unit 202, a composition image creation unit 203, a composition unit 204, and a display control unit 205. When the user starts shooting, shooting of a moving image in which the user is shown as a subject is started, and the shot moving image is supplied from the camera 51 to the area detection unit 201 and the combining unit 204.

  The area detection unit 201 performs a chromach process, a skin color area detection process, and the like on each frame that is a still image arranged in time series constituting the moving image supplied from the camera 51, thereby performing subject detection in each frame. Detect the area. For example, in the moving image supplied from the camera 51, a chroma curtain is reflected as a user's background, and chroma frames can be processed for each frame.

  FIG. 11 is a diagram illustrating an example of a region detected by the region detection unit 201.

In the example of FIG. 11, frames f _{1 to} f ₃ are shown as frames constituting a moving image taken by the camera 51. The horizontal direction represents the time direction.

As shown in FIG. 11, for example, the area A ₁ is detected from the frame f ₁ , the area A ₂ is detected from the frame f _2, and the area A ₃ is detected from the frame f ₃ . In the frame f _1, a region A ₁ in which the subject is reflected slightly on the right when on the basis of the center of the frame, the frame f _2, the area A ₂ the object is reflected is approximately in the center of the frame. In the frame f ₃ , the area A ₃ where the subject is shown is slightly on the left side with respect to the center of the frame.

  That is, FIG. 11 shows an example in which the user who faces the front of the body to the camera 51 moves the body from left to right during the photographing work.

  The region detection unit 201 outputs information on the subject region detected from each frame in this way to the motion detection unit 202.

  The motion detection unit 202 detects the content of the subject's motion based on the information supplied from the region detection unit 201. Specifically, based on the information supplied from the region detection unit 201, the motion detection unit 202 detects a range in which the subject has moved in the plane of the captured image for each predetermined period, and the detected range. Find the area.

  FIG. 12 is a diagram illustrating an example of a range detected by the motion detection unit 202.

For example, when the areas A _{1 to} A _{3 in} FIG. 11 are detected from the frames f _{1 to} f ₃ , the motion detection unit 202 obtains a difference between the areas A _{1 to} A _3. A range indicated by hatching in FIG. 12 having a difference between them is detected as a range in which the subject has moved. If the number of pixels in the frame of FIG. 12 is the same as the number of pixels in each frame of FIG. 11, the hatched range is detected as a pixel in at least _one of the regions A _{1 to} A _3. This is a range of pixels.

Note that, based on information supplied from the region detection unit 201, the motion detection unit 202 may detect the speed of movement of the subject for each predetermined period. The speed of movement of the subject can be obtained from, for example, the amount of movement of the areas A _{1 to} A ₃ and the shooting times of the frames f _{1 to} f _{3 shown} in FIG.

  The motion detection unit 202 outputs information indicating the range in which the subject has moved or information indicating the speed of movement of the subject to the composition image creation unit 203 as information indicating the content of the subject's movement. Both the range in which the subject has moved and the speed of movement of the subject may be detected by the motion detection unit 202, and information representing them may be supplied to the composition image creation unit 203.

  The compositing image creation unit 203 creates a compositing image corresponding to the information representing the content of the motion of the subject supplied from the motion detection unit 202. The composition image created by the composition image creation unit 203 is an image used to compose a moving image captured by the camera 51, and may be either a moving image or a still image.

  For example, the composition image creation unit 203 creates an image that displays a predetermined image in the same range as the range of movement of the subject represented by the information supplied from the motion detection unit 202.

  FIG. 13 is a diagram illustrating an example of a composition image.

  In the example of FIG. 13, an object O that displays the same range as the range shown by hatching in FIG. 12 in a predetermined color is displayed in one frame. Of the frame shown in FIG. 13, the portion other than the portion where the object O is displayed is transparent.

  The composition image creation unit 203 outputs the composition image created according to the content of the subject movement in this way to the composition unit 204.

  The synthesizing unit 204 is obtained by synthesizing and synthesizing the synthesizing image created by the synthesizing image creating unit 203 as a foreground or background of a subject reflected in the moving image supplied from the camera 51. The synthesized moving image is output to the display control unit 205.

  FIG. 14 is a diagram illustrating an example of one frame constituting the synthesized moving image.

  In the example of FIG. 14, a region U that is a subject region is extracted from one frame of a moving image supplied from the camera 51 by chroma processing or the like, and the object shown in FIG. 13 is used as the background of the extracted region U. A composition image for displaying O is synthesized.

  When the object O displays a predetermined color as described above, it is combined as the background of the area U as shown in FIG. 14, so that the combined image has the object O around the subject. It will be displayed. The function of displaying a predetermined color or the like so as to color the periphery of the subject is called an “aura” function.

  Since the size of the object O is the same as the range in which the subject has moved, the “aura” function is realized as described above. As the subject moves more greatly, the range of the aura (the range of the object O) also increases. It will be displayed widely.

  FIG. 15 is a diagram illustrating an example of one frame constituting the synthesized moving image when the range in which the subject has moved is narrow.

For example, when the range in which the subject has moved is narrower than the range described with reference to FIGS. 11 and 12, the range of the object O is also narrower than that of FIG. 14 as shown in FIG. The width W ₂ indicated by the white arrow in FIG. 15 is narrower than the width W ₁ indicated by the white arrow in FIG.

  The object O may be gradually reduced and displayed as time passes. As a result, it is possible to realize a display in which the aura disappears from a distance from the subject.

  The synthesizing unit 204 synthesizes a synthesizing image with each frame constituting the moving image supplied from the camera 51 and outputs the synthesized moving image obtained by synthesizing to the display control unit 205. A combined mask image is used as appropriate for a captured image (each frame constituting a moving image supplied from the camera 51) to be combined with the combined image. For example, a subject area is extracted from the photographed image by using a synthesis mask image, and a synthesis image is synthesized with the extracted subject area.

  The display control unit 205 displays the synthesized moving image supplied from the synthesizing unit 204 on the captured image display unit 132 of the touch panel monitor 52 in the imaging space 32. The user can perform the shooting operation while watching the synthesized moving image displayed in the live view.

  In the above, the case where an image for displaying an aura is created as a composition image according to the range in which the subject has moved has been described, but depending on the range in which the subject has moved and the speed of movement of the subject, A composition image with a different pattern or a composition image with a different color may be created.

  When images for composition with different patterns are created, for example, when the area of the range in which the subject has moved is large or the movement speed of the subject is slow, the number of patterns to be displayed may be large as shown in FIG. As shown in FIG. 17, a composition image having a large pattern is created. On the other hand, when the area of the moving range of the subject is small or the subject moves fast, the number of patterns to be displayed is small as shown in FIG. 18 or the size of the pattern is large as shown in FIG. A composite image with a small size is created.

  In this way, when the pattern is a heart pattern, a composite image that displays a heart pattern with a different size as the background or foreground of the subject, or a different number of heart patterns according to the content of the subject's movement, Or a composite image to be displayed as a foreground.

  Also, when a composition image with different colors is created, for example, when the area of the moving range of the subject is large or the subject moves slowly, a composition image with a low density of the displayed color is created. The On the contrary, when the area of the moving range of the subject is small or the moving speed of the subject is fast, a composite image having a dark color density to be displayed is created.

  There is no limitation on the combination of the content of the movement of the subject and the content of the composition image, and any combination other than such a combination may be set.

  Here, processing of the photo sticker creating apparatus 1 having the above-described configuration will be described.

  First, the flow of the photo sticker creation game will be described with reference to the flowchart of FIG.

  A user who performs a photo sticker creation game first inserts a coin into the coin insertion / return port 41 of the pre-service unit 11 in the pre-service area 31 and pays for the game. The coin processing unit 121 accepts coin insertion and supplies an activation signal to the control device 101.

  In step S1, the control device 101 determines whether or not a price has been inserted based on the activation signal, and proceeds to step S2 if it is determined that a valid price has been inserted. The user who has entered the price moves to the shooting space 32 according to a guidance demonstration screen displayed on the monitor 42 of the advance customer service section 111 and starts shooting work.

  In step S <b> 2, the control device 101 controls each unit of the photographing unit 112 and executes a photographing process that is a process related to the photographing work of the photo sticker creation game performed in the photographing space 32. The photographing process will be described with reference to the flowchart of FIG.

  In step S <b> 3, the control device 101 performs a movement guidance process that prompts the user in the shooting space 32 who has finished the shooting operation to move to the first editing space 33 </ b> A or the second editing space 33 </ b> B by an image or sound. .

  In step S <b> 4, the control device 101 performs editing processing, which is processing related to editing work of the photo sticker creation game, on the user who has moved to the editing space 33.

  When the editing process is completed, in step S5, the control device 101 controls the printer 114 to perform a printing process for printing the edited image obtained by the editing work on a sticker sheet.

  In step S <b> 6, the control apparatus 101 performs post-service processing for providing services such as a mini game and image transfer to a user who is waiting for completion of printing.

  When the subsequent customer service process is completed, in step S7, the control device 101 performs movement guidance for moving the user of the editing space 33 to the printed material receiving area.

  When the printing process is completed, in step S8, the control device 101 controls the printer 114 to discharge the printed sticker paper from the photo sticker discharge port 71, and the photo sticker creation game process is finished.

  Next, the photographing process performed in step S2 of FIG. 20 will be described with reference to the flowchart of FIG.

  In step S <b> 11, the control device 101 causes the captured image display unit 132 to display the moving image captured by the camera 51 and starts live view.

  In step S <b> 12, the control device 101 adjusts the image and brightness of the camera 51 in accordance with a user input to the touch panel monitor 52.

  In step S <b> 13, the control apparatus 101 determines whether or not the user has instructed the creation of the composition image. On the touch panel monitor 52, for example, a button operated when instructing creation of a composition image such as an aura is displayed.

  If the button displayed on the touch panel monitor 52 has been operated and it is determined in step S13 that an instruction to start creating a composition image has been issued, the control apparatus 101 starts creating a composition image in step S14. To do. Thereafter, the process of creating the composition image is performed in parallel with the process of FIG. The process of creating the composition image will be described with reference to the flowchart of FIG.

  In step S <b> 15, the combining unit 204 of the control apparatus 101 combines the generated combining image as a foreground or background of a subject reflected in the moving image supplied from the camera 51, and combining the resultant moving image. The image is output to the display control unit 205. In addition, the display control unit 205 causes the captured image display unit 132 to display the synthesized moving image supplied from the synthesis unit 204. As a result, a synthesized moving image composed of images as shown in FIG. 14 is displayed on the captured image display unit 132.

  When it is instructed to start shooting, or when it is determined in step S <b> 13 that creation of a compositing image has not been instructed, the control device 101 displays a countdown indicator on the touch panel monitor 52 in step S <b> 16.

  When the remaining time until photographing represented by the countdown indicator becomes 0 second, in step S17, the control device 101 performs photographing, and a composite image obtained by combining the subject image and the composite image at the photographing timing is displayed. Save as a shot image.

  In step S <b> 18, the control device 101 ends the process of creating a composition image so far.

  In step S19, the control device 101 causes the captured image display unit 132 to display the composite image acquired and stored by shooting on the captured image display unit 132, and allows the user to confirm.

  In step S20, the control device 101 determines whether or not shooting has been performed a predetermined number of times. If it is determined that shooting has not been performed, that is, the number of possible shots remains, the process returns to step S13. The subsequent processing is repeated.

  On the other hand, if it is determined in step S20 that shooting has been performed a predetermined number of times, in step S21, the control device 101 causes the user to select an image to be edited (keep image) from the shot images. Save the selected captured image. Then, it returns to step S2 of FIG. 20, and the process after it is performed.

  Next, with reference to the flowchart of FIG. 22, a process that is started when an instruction to create a composition image is given.

  In step S <b> 31, the area detection unit 201 of the control device 101 detects the area of the subject in each frame by performing a chroma process or the like on each frame constituting the moving image supplied from the camera 51. The area detection unit 201 outputs information on the area of the subject to the motion detection unit 202.

  In step S <b> 32, the motion detection unit 202 detects the content of the subject motion, that is, the range in which the subject has moved or the speed of the subject motion based on the information supplied from the region detection unit 201. The motion detection unit 202 outputs information representing the content of the detected subject motion to the composition image creation unit 203.

  In step S <b> 33, the compositing image creation unit 203 creates a compositing image according to the information representing the content of the subject motion supplied from the motion detection unit 202. The composition image created here is composed with the moving image photographed by the camera 51 in step S15 in FIG. The process of FIG. 22 is repeated until the creation of the composition image is completed.

  Through the above processing, the user sees an interesting moving image as compared to a case where the image combined with his / her image is a still image and a combined image in which such a still image is combined is displayed. You can do shooting work.

  Further, the user can start displaying such a synthesized moving image only by instructing synthesis of an image without performing a complicated operation such as selecting a region to be synthesized.

  In the above, the range and speed of movement of the user are detected from the image captured by the camera 51, and the composition image changes according to the area and speed of the detected range. When the control device 101 is equipped with a function for recognizing facial expressions such as a smile or an angry face, an image for synthesis corresponding to the facial expression may be created.

  For example, when it is detected that the user's face is smiling, a background image with a pink color scheme is created and synthesized as the background of the user. When it is detected that the user's face is an angry face (unfriendly face), a background image with a blue color scheme is created and synthesized as the user's background. Not only the background color but also the pattern and pattern may be different.

  For example, the facial expression is prepared in advance by using features extracted from each detected face area by detecting the face area of the user from each frame constituting the moving image taken by the camera 51 and image processing. Detected by matching with features of smiles and angry faces.

  The series of processes described above can be executed by hardware or can be executed by software. When a series of processing is executed by software, a program constituting the software executes various functions by installing a computer incorporated in dedicated hardware or various programs. It can be installed on a general-purpose personal computer or the like.

  The program to be installed is recorded on the removable medium 105 shown in FIG. 9 formed of an optical disc (CD-ROM (Compact Disc-Read Only Memory), DVD (Digital Versatile Disc), etc.), semiconductor memory, or the like, or locally. It is provided via a wired or wireless transmission medium such as an area network, the Internet, or digital broadcasting. The program can be installed in advance in the storage unit 102 or the ROM 106.

  The program executed by the computer may be a program that is processed in time series in the order described in this specification, or in parallel or at a necessary timing such as when a call is made. It may be a program for processing.

  The embodiments of the present invention are not limited to the above-described embodiments, and various modifications can be made without departing from the scope of the present invention.

It is a perspective view which shows one Embodiment of the photographic sticker production apparatus to which this invention is applied. It is a perspective view from the other angle of the photograph sticker production apparatus of FIG. It is a perspective view which shows the example of installation of the photograph sticker production apparatus of FIG. It is a figure explaining a user's space movement. It is a figure which shows the structural example of a prior customer service unit. It is a figure which shows the structural example of the front of a front unit. It is a figure which shows the structural example of the side surface of an edit unit. It is a figure which shows the structural example of the other side of an edit unit. It is a block diagram which shows the example of an internal structure of a photograph sticker production apparatus. It is a block diagram which shows the function structural example of a control apparatus. It is a figure which shows the example of the area | region of a to-be-photographed object. It is a figure which shows the example of the range of a to-be-photographed object's movement. It is a figure which shows the example of the image for a synthesis | combination. It is a figure which shows the example of 1 frame which comprises a synthesized moving image. It is a figure which shows the other example of 1 frame which comprises a synthetic | combination moving image. It is a figure which shows the further another example of 1 frame which comprises a synthetic | combination moving image. It is a figure which shows the example of 1 frame which comprises a synthesized moving image. It is a figure which shows the other example of 1 frame which comprises a synthetic | combination moving image. It is a figure which shows the further another example of 1 frame which comprises a synthetic | combination moving image. It is a flowchart explaining the flow of a photo sticker creation game. It is a flowchart explaining the imaging | photography process performed in step S2 of FIG. It is a flowchart explaining a composition image creation process.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Photo sticker production apparatus 12 Image pick-up unit 13 Editing unit 101 Control apparatus 201 Area | region detection part 202 Motion detection part 203 Composition image creation part 204 Composition part 205 Display control part

Claims (4)

In an image photographing / editing apparatus for photographing a user as a subject, causing the user to edit the image obtained by photographing, and creating the image obtained by photographing or the edited image,
Photographing means for capturing a moving image reflected by the user as a subject;
A region detecting unit for detecting a region of a subject in each frame which is a still image arranged in chronological order constituting a moving image captured by the photographing unit;
Motion detection means for detecting the speed of movement of the subject for each predetermined period based on the area of the subject detected by the area detection means;
Creating means for creating, as an image for synthesis, an image in which the size of a predetermined pattern arranged in plural is changed according to the speed of movement of the subject detected by the motion detecting means;
Synthesizing means for synthesizing the composition image created by the creating means as a foreground or background of a subject shown in the moving image captured by the photographing means;
An image photographing / editing apparatus comprising: display control means for displaying a synthesized moving image obtained by being synthesized by the synthesizing means during photographing of an image to be edited. The creation means creates the image for synthesis having a different number of patterns to be displayed or different color densities according to the speed of movement of the subject detected by the motion detection means.
The image photographing / editing apparatus according to claim 1 . Image capturing / editing of an image capturing / editing apparatus that captures a user as a subject, edits the captured image, and creates the image obtained by capturing or the edited image. In the method
A user captures a moving image that appears as a subject,
Detect the area of the subject in each frame that is a still image arranged in chronological order constituting the captured moving image,
Based on the detected area of the subject, the speed of movement of the subject for a predetermined period is detected,
Depending on the detected speed of movement of the subject, create a composite image that changes the size of the predetermined pattern that is arranged,
The created composition image is composed as a foreground or background of a subject reflected in the captured moving image,
An image shooting / editing method including a step of displaying a synthesized moving image obtained by synthesizing during shooting of an image to be edited. The processing of the image photographing / editing apparatus for photographing the user as a subject, causing the user to edit the image obtained by photographing, and creating the image obtained by photographing or the edited image is performed by the computer. In the program to be executed
A user captures a moving image that appears as a subject,
Detect the area of the subject in each frame that is a still image arranged in chronological order constituting the captured moving image,
Based on the detected area of the subject, the speed of movement of the subject for a predetermined period is detected,
Depending on the detected speed of movement of the subject, create a composite image that changes the size of the predetermined pattern that is arranged,
The created composition image is composed as a foreground or background of a subject reflected in the captured moving image,
A program including a step of displaying a synthesized moving image obtained by synthesizing during shooting of an image to be edited.

Images