JP2021078133A - Imaging device for gaming, image processing device, and image processing method - Google Patents

Abstract

To provide an imaging device for gaming and an image processing method with which it is possible to image and edit a moving-image following a simple procedure, with the time and effort of a user eliminated.SOLUTION: In an imaging device 100 for gaming, a display control unit 135 displays a sample moving-image on a screen for showing a sample of subject motion during imaging of a moving-image to a user, and after terminating the display of the sample moving-image and switching to the real-time display of a user's video imaged by an imaging unit 132, an image acquisition unit 133 images the moving-image of the user and repeats a series of processes of displaying the sample moving-image and imaging the moving-image of the user, thereby acquiring a plurality of moving-images. A moving-image processing unit 144 sets, to a template moving-image in which a three-dimensional object is displayed, a moving-image for display with which a plurality of moving-image frames are buried in each of a plurality of frame shapes included in the three-dimensional object and reproduced.SELECTED DRAWING: Figure 8

Description

The present invention relates to a game photographing apparatus and an image processing method, and particularly to an apparatus and a method for processing moving image shooting and editing.

Conventionally, a game shooting device and a game capable of shooting a subject including a plurality of users multiple times with a camera, outputting the shot image as a photo print sheet, or transmitting the shot image to an external device via a network. Photography services are known. Many such amusement photography devices have been proposed that can perform the following editing processes according to the user's preference. First, chroma key processing is performed on the captured image of the subject to generate a subject image from which the subject portion is extracted. Then, after synthesizing the background image selected by the user with this subject image, editing processing such as making corrections to the eye color and size, skin color, facial contour, etc., and superimposing characters and patterns. After going through (commonly referred to as "graffiti"), it produces the final image for printing. In addition, by providing such an editing function so that multiple users who have taken pictures together can operate it with a touch pen or the like at the same time, friends can enjoy it together from shooting to editing, which is gaining popularity.

Japanese Unexamined Patent Publication No. 2015-555838

In recent years, in the above-mentioned amusement photographing apparatus, an apparatus capable of photographing not only a still image but also a moving image has been developed (see, for example, Patent Document 1). However, in the case of moving images, unlike still images, shooting does not end in an instant, so the inventor of the present application has come to recognize that more efficient procedures and effective procedures are required to reduce waiting time and shooting failures. ..

The present invention has been made in view of these problems, and an object of the present invention is to provide a game shooting device and an image processing method capable of shooting a moving image and editing the moving image in a simple procedure without the trouble of a user. ..

In order to solve the above problems, the amusement photography apparatus according to an embodiment of the present invention includes a photographing space in which a user is photographed as a subject, an imaging unit that captures an image including the user as a subject in the photographing space, and imaging. An image acquisition unit that acquires the user's moving image captured by the unit, a display control unit that displays a model video on the screen to show the user a model of the subject's movement during movie shooting, and editing processing for the moving image. It is provided with a moving image processing unit for executing the above and an output processing unit for executing the output processing of moving images. The display control unit switches to the real-time display of the user's image captured by the imaging unit after the display of the model moving image is completed, and the image acquisition unit captures the user's moving image after switching to the real-time display and controls the display. The unit and the image acquisition unit acquire a plurality of moving images by repeating the process from the display of the model video to the shooting of the user's moving image, and the video processing unit receives the template video in which the three-dimensional object is displayed. A display moving image is set by embedding a plurality of moving image frames in each of the plurality of frame shapes included in the three-dimensional object and played back, and the output processing unit displays the set display moving image on the screen. ..

Here, the "output process" may include either displaying the moving image on the screen or uploading the moving image to the server for the user to download the moving image. According to this aspect, it is possible to shoot a movie while the user remembers the model of the operation, and by repeating the shooting in a form that does not leave a time interval between the provision of the model and the shooting as much as possible, it is ideal for multiple times. It is possible to easily have the user acquire a moving image that summarizes the moving images obtained by moving image shooting.

Another aspect of the present invention is also a game photographing device. This device includes a shooting space in which a user is taken as a subject, an imaging unit that captures an image including the user as a subject in the shooting space, and an image acquisition unit that acquires a moving image of the user captured by the imaging unit. , A moving image processing unit that executes editing processing on a moving image and an output processing unit that executes output processing of a moving image are provided. The image acquisition unit shoots a moving image at the first frame rate, and the moving image processing unit sets the shot moving image as a display moving image to be reproduced at a second frame rate higher than the first frame rate. Then, the output processing unit displays the set display moving image on the screen.

The first frame rate can also be lower than usual.
Here, "lower than usual" may mean lower than the frame rate of general moving image shooting, or at least lower than "second frame rate". According to this aspect, the motion is reproduced like a fast forward, which is faster than the actual operation speed of the object to be photographed. Therefore, it is easy to create a moving image in which the user's awkward operation is not noticeable in the amusement photographing device. Can be provided. In addition, by shooting at a frame rate lower than the frame rate during playback, it is possible to shoot in a form that eliminates wasted frames from the beginning, reducing the data size of moving images and reducing the processing load on the device. Processing efficiency can be improved.

Yet another aspect of the present invention is an image processing method. This method includes a step of capturing an image including the user as a subject in a predetermined shooting space provided in the game shooting device as a space in which the user is taken as a subject, and a step of acquiring a moving image of the user to be captured. , The step of embedding the frame of the acquired moving image in the image frame included in the predetermined template video and setting the moving image to be played back as the moving image for display, and embedding the frame of the moving image in the template video and this video in the first frame It includes a step of setting a moving image in which an image showing the characteristics of the image is inserted as a moving image for transmission.

As the "image showing the characteristics of this moving image", an image in which a character string including the name of the template video or the video theme or a motif such as a pattern or a picture symbolizing the video theme is displayed may be used. Of the moving images, a frame in which a person is detected by person detection may be used. The process of "embedding the acquired moving image frame in the image frame included in the predetermined template video" may be executed by the game shooting device or by the management server that causes the user to download the moving image. It may be executed on the user's mobile terminal. According to this aspect, since the first frame of the moving image includes an image in which the characteristics of the moving image appear, the first frame is displayed when a list of moving images is displayed or when thumbnails are displayed on SNS (Social Networking Service). It is automatically displayed as a thumbnail still image. As a result, what happens to the moving image when it is displayed in thumbnail format, for example, an image with no features becomes the first frame, or an image with nothing in it, such as a solid black image, becomes the first frame. It is possible to avoid a situation in which a clue as to whether or not the image is a moving image is not displayed. In addition, it is possible to easily present information that the person who sees the thumbnail wants to play.

It should be noted that any combination of the above components and the conversion of the expression of the present invention between methods, devices, systems, computer programs, data structures, recording media and the like are also effective as aspects of the present invention.

According to the present invention, it is possible to provide a game shooting device and an image processing method capable of shooting a moving image and editing the moving image in a simple procedure without the user's trouble.

It is a figure which shows typically the whole structure of the photography system. It is a perspective view of a game photography device seen from the left front side. It is a top view of the game photographing apparatus shown in FIG. It is a perspective view of the photographing housing. It is a perspective view of an editing housing. It is a front view of the output housing. It is a block diagram which shows typically the hardware composition of the amusement photography apparatus. It is a functional block diagram which shows typically the functional structure of a game photographing apparatus. It is a flowchart which shows the process from the preset setting to the main shooting. It is a flowchart which shows the process of correction and editing. It is a flowchart which shows the process of output processing. It is a figure which shows the screen example which makes a user select the content of moving image shooting in the preset. It is a figure which shows the example of the guidance screen which displays before the start of moving image shooting. It is a figure which shows the screen transition of the first half of a moving image shooting process. It is a figure which shows the screen transition of the latter half of a moving image shooting process. It is a figure which shows the example of the template moving image using the frame image of a three-dimensional object. It is a figure which shows the example of the guidance screen which guides the provision of an edited video. It is a figure which shows the example of the initial state of the editing screen displayed on the editing monitor. It is a figure which shows the example of the state in which video editing is started on the edit screen. It is a figure which shows the example of the state which selected the still image to embed in the template moving image on the edit screen. It is a figure which shows the example of performing graffiti on the template moving image on the edit screen.

(First Embodiment)
<Overview of photography system>
FIG. 1 is a diagram schematically showing the overall configuration of the photography system 600. The photography system 600 includes a game photography device 100, a management server 300, an SNS server 400, and a plurality of terminal devices 500 connected via a network 200 such as the Internet. In this figure, two terminal devices, a terminal device 500a and a terminal device 500b, are shown, but the number of terminal devices 500 is not limited to two and may be three or more. Hereinafter, in the present specification, unless each terminal device 500 is particularly distinguished, it is simply collectively referred to as "terminal device 500".

The game photographing device 100 is a device including a camera for taking a picture of a subject including a user and an electronic device provided with a user interface for editing the taken photographic image. When the user takes a picture with the game shooting device 100, the game shooting device 100 associates the photographic image group including the user with the subject, the play ID, and the e-mail address input by the game shooting device 100. It is transmitted to the management server 300. The play ID is identification information uniquely assigned to each play, and is unique identification information for identifying the model and play in the store where the play was performed. The photographic image group refers to a plurality of captured images created and grouped in a predetermined play. Hereinafter, in the present specification, the data in which the play ID, the photographic image group, and the e-mail address are associated with each other will be referred to as "content data". In addition, the term "photographed image" in the present specification includes not only the image obtained by shooting but also an image that has been corrected or edited after shooting, or has been resized or changed in image quality for output. There may be. Similarly, the term "output image" may include an image for screen display, an image for printing, and an image for transmission, depending on each specification such as hardware specifications, software specifications, and communication specifications. It may include all those that have undergone resizing or image quality changes.

The management server 300 is a server that manages user information such as content data (play ID, photographic image group, e-mail address input by the amusement photography device 100) acquired from the amusement photography device 100, and usage history of the photography service. Is. The management server 300 transmits the content access information for accessing the photographic image group included in the content data to the e-mail address included in the content data. As a result, the terminal 500 owned by each user can acquire the mail in which the content access information is described. The content access information is, for example, a URL (Uniform Resource Locator) for accessing the management server 300, and the play ID is included in this URL.
By accessing the URL of this e-mail, the mobile terminal 500 can acquire a group of photographic images associated with the play ID included in the URL.

The user accesses the management server 300 based on the content access information by using a dedicated information presentation application designed for the user of the game photographing device 100 and installed in the user's own terminal device 500, and the user owns the terminal. The photographic image group is downloaded to the device 500. The user can also edit the photographic image group on the terminal device 500. The information presentation application can be downloaded from the management server 300 via the network 200. A moving image taken by the user with the game photographing device 100 is uploaded and saved in the management server 300. The user can share the moving image stored in the management server 300 on the SNS provided by the SNS server 400. That is, the URL of the own moving image stored in the management server 300 is shared on the SNS server 400, or the user downloads the moving image to his / her own mobile terminal or computer from the management server 300 and then downloads the moving image. You may post the moving image by sending it to the SNS server 400 and share it with other users. At this time, the first frame of this moving image is automatically selected and displayed as a thumbnail on the SNS.

<Appearance of amusement photography equipment>
FIG. 2 is a perspective view of the game photographing device 100 as viewed from the left front side. FIG. 3 is a plan view of the game photographing device 100 shown in FIG. 2 as viewed from above. In the following, other drawings will be described with reference to the front-back, left-right directions shown in FIG. As shown in FIGS. 2 and 3, the amusement photographing apparatus 100 includes a photographing unit 50 on the front side and an editing unit 51 on the rear side, and exhibits a substantially rectangular parallelepiped shape long in the front-rear direction. On the left side or right side of the shooting unit 50, a presetting unit 40 for inserting coins and presetting the shooting contents including the number of photographers is provided, and at the rear end of the editing unit 51, output of the edited photo and the like are output. An output unit 52 is provided for this purpose.

The shooting unit 50 has a shooting housing 11 having a built-in camera for shooting a user, and a background wall portion 12 arranged behind the shooting housing 11 via a shooting space 10. The gap between the side surface of the photographing housing 11 and the side surface of the background wall portion 12 constitutes the first entrances 101a and 101b which are entrances and exits to the photographing space 10. The preset unit 40 includes a touch panel 41 that displays setting items of shooting contents including the number of photographers and receives instructions from the user, and a coin insertion unit 42 that includes a coin insertion slot and a coin insertion slot and a return slot. In the figure, the housing of the preset unit 40 is detachably attached to the left side surface of the photographing unit 50, but if it is attached to either the left or right side surface depending on the layout of the store where the game photographing device 100 is installed. Good. The user enters the shooting space 10 from the entrance on the side where the preset unit 40 is attached. The editing unit 51 has an editing housing 21 arranged adjacent to the back of the background wall unit 12. The editing housing 21 is provided with a pair of operation surfaces facing in the left-right direction. The user can enter the editing spaces 20a and 20b for inputting to each operation surface via the second entrances 201a and 201b. The output unit 52 has an output housing 31 that is in contact with the rear end surface of the editing housing 21. The operation surface of the output housing 31 faces the rear end side so that the user operates from the rear end side of the game photographing device 100. Therefore, the space on the rear end side of the output housing 31 becomes the output space 30 for the user to perform the output operation. As a modification, the game photographing device 100 may be configured without the preset unit 40, or the game photographing device 100 may be configured without the editing unit 51 or the editing space 20. Good.

The side surfaces of the upper ends of the photographing housing 11 and the background wall portion 12 are connected to each other by a ceiling frame 13 extending in the front-rear direction. Inside the frame of the ceiling frame 13, a ceiling lighting unit 14 that covers the photographing space 10 from above and illuminates and irradiates the flash is provided. On the left side surface or the right side surface of the photographing housing 11, a posting space 15 for posting an explanatory note including how to use the amusement photographing apparatus 100 is provided. A display unit 16 indicating whether or not the photographing housing 11 is in use is provided above the posting space 15.

The display unit 16 includes an indicator light (not shown) that emits light or is turned off. The display unit 16 indicates to the user that the photographing unit 50 is in use by emitting light from the indicator lamp, and indicates to the user that the photographing unit 50 is vacant by turning off the light. The game photographing device 100 is formed in a triangular shape in a plan view in which the display unit 16 projects outward in the left-right direction, but the shape is not particularly limited and may be another shape such as a convex curve shape. ..

The background wall portion 12 is formed of a plate-shaped member having substantially the same width and height as the photographing housing 11, and extends forward from the main body plate 121 facing the front surface of the photographing housing 11 and the left and right ends of the main body plate 121. It has a pair of side plates 122 provided. Therefore, the background wall portion 12 is formed in a U-shape in a plan view as a whole. In the main body plate 121, the surface facing the photographing housing 11 is colored with a single color (for example, blue or green) for chroma key processing. Regarding coloring, in addition to coloring the main body plate 121 directly with paint or the like, a colored cloth, plate or the like may be attached. Although not shown, the floor surface of the photographing space 10 is colored with a single color for chroma key processing in the same manner as the main body plate 121 so as to be continuous with the main body plate 121. As described above, the first entrances 101a and 101b for entering and exiting the photographing space 10 are configured from the rear end edge of the photographing housing 11, the lower end edge of the ceiling frame 13, and the front end edge of the side plate 122. ..

The editing housing 21 is a face-to-face type in which two sets of users can edit from both the left and right sides at the same time, and operation surfaces are provided on each of the left and right sides. The length of the editing housing 21 in the left-right direction is shorter than the width of the background wall portion 12 in the same direction. The front end surface of the editing housing 21 is connected to the central portion of the back surface of the background wall portion 12. The spaces on both the left and right sides of the editing housing 21 are the editing spaces 20a and 20b where two sets of users perform editing work, respectively. Above the editing housing 21, a substantially U-shaped curtain frame 22 is provided in a plan view. The curtain frame 22 includes a pair of side frames 221 extending in the front-rear direction and a rear frame 222 extending in the left-right direction connecting the rear ends of the side frames 221. Although not shown, a computer equipped with a CPU (Central Processing Unit) such as a PC (Personal Computer) is installed inside the editing housing 21.

The front end of each side frame 221 is extended in the front-rear direction to the rear end of the ceiling frame 13, and the central portion of the rear frame 222 is supported by a support member 23 provided at the rear upper end of the editing housing 21. Will be done. The side frames 221 on one of the left and right sides and the approximately half the length of the rear frame 222 form upper frame portions of the second entrances 201a and 201b for entering and exiting the editing spaces 20a and 20b.

<Shooting housing>
FIG. 4 is a perspective view of the photographing housing. A front lighting unit 115 that brightly illuminates the shooting space 10 from the front and irradiates a flash is provided on the indoor side surface of the shooting housing 11. The front lighting unit 115 is composed of a unit portion 115A on the upper center side and a unit portion 115B on the lower center side. The unit portion 115A and the unit portion 115B on the upper center are integrally connected and formed in a concave curved shape. At the center of the unit portion 115A, a front camera 111 for shooting a subject mainly from the front in close-up is mounted. A touch panel type front monitor 113 is provided below the front camera 111, and an image captured by the front camera 111 and guidance on the shooting method are displayed, and a user's touch operation input according to the guidance is received. On the right side of the front monitor 113, a side camera 112 for shooting the whole body of the user as a subject from the side at a wide angle is mounted. Below the side camera 112, a side monitor 114 for displaying an image captured by the side camera 112 is provided. As a modification, when the preset unit 40 is not installed outside the photographing space 10, the coin insertion unit 42 may be provided below the front monitor 113 or the side monitor 114. Further, although the side camera 112 is not a touch panel type, in the modified example, it may be configured as a touch panel type like the front monitor 113.

<Editing housing>
FIG. 5 is a perspective view of the editing housing. The editing housing 21 has a control box 211 on the lower side and an editing base 212 provided on the control box 211. Various control devices such as a main computer unit and an I / O controller are built in the control box 211, and these control devices control the overall operation of the game photographing device 100.

The editing table portion 212 is formed by a chevron-shaped inclined surface facing in the left-right direction, and an operation surface for performing editing work is installed on each inclined surface. Specifically, an editing monitor 213 composed of a liquid crystal monitor with a built-in tablet is provided on each operation surface, and two touch pens 214A and 214B are connected to the left and right sides of each editing monitor 213. The editing monitor 213 is a touch panel, and the user touches the pen tips of the touch pens 214A and 214B to the screen of the editing monitor 213 to perform selection input for selecting an icon displayed on the screen and handwriting of characters and drawings. You can input by handwriting when you do. Although FIG. 5 shows the editing monitor 213 on the left side, the editing monitor 213 on the right side has the same structure as the left side. Further, the control device of the control box 211 can individually perform parallel processing of processing corresponding to the input information to the editing monitors 213 on both the left and right sides.

<Output housing>
FIG. 6 is a front view of the output housing. The output housing 31 is a rectangular housing that is in contact with the rear end portion of the editing housing 21, and an operation surface is provided on a surface facing the rear end side. A touch panel type liquid crystal monitor 311 is provided in the upper center of the operation surface, and an explanation of output and the like are displayed on the liquid crystal monitor 311.

A non-contact communication unit 312 is provided below the liquid crystal monitor 311 and communicates with a nearby non-contact IC (Integrated Circuit) to read data from the non-contact IC or write data to the non-contact IC. .. For example, FeliCa (registered trademark) and the like can be used. A sticker paper discharge port 313 is provided under the non-contact communication unit 312. A photo print sheet on which an image created by performing an editing process in the editorial unit is printed is discharged to the sticker paper discharge port 313. Therefore, a printer is built in the output housing 31, and the printer prints a photo printing sheet. A speaker 314 is provided under the sticker paper discharge port 313, and outputs sounds such as guidance voice, BGM, and sound effect in the output process. In the present embodiment, the output housing 31 is separate from the editing housing 21, but these can also be configured as an integrated housing.

<Hardware configuration of amusement photography equipment>
FIG. 7 is a block diagram schematically showing the hardware configuration of the game photographing device 100. In the game photographing device 100, the control unit 53, the storage unit 54, the photographing unit 50, the preset unit 40, the editing unit 51, and the output unit 52 are electrically connected via the bus X. The control unit 53 includes a known CPU (Central Processing Unit), ROM (Read Only Memory), and RAM (Random Access Memory). The control unit 53 executes a program stored in the storage unit 54 and controls the overall operation of the game photographing device 100. A communication unit 55 and an external drive 56 are also connected to the bus X. The control unit 53, the storage unit 54, the communication unit 55, and the external drive 56 may be configured by a known computer, and such a computer is built in the control box 211 of the editing housing 21 described above.

The storage unit 54 is composed of a non-volatile storage medium such as a hard disk or a flash memory, and stores various setting information transmitted from the control unit 53. The information stored in the storage unit 54 is appropriately read out by the control unit 53. The storage unit 54 stores various image data used for image composition and editing, and data for image processing. For example, background image data that is the background of the captured image is stored. The ROM of the control unit 53 stores data such as a program such as firmware and setting information, and the RAM temporarily stores the data and program to be processed by the control unit 53. In the following, RAM and ROM may be collectively referred to as memory.

The communication unit 55 is an interface for connecting to a network such as the Internet, and communicates with an external device under the control of the control unit 53. The external drive 56 executes reading and writing to removable media such as an optical disk and a semiconductor memory. For example, the computer program or data read from the removable media by the external drive 56 is transmitted to the control unit 53, stored in the storage unit 54, or installed as a program.

The preset unit 40 is a device for receiving instructions from the user such as shooting conditions and shooting contents including the number of shots from the user prior to shooting, and includes a touch panel 41 and a coin insertion unit 42. The touch panel 41 is a known touch panel, and as a display device, displays a guide to the user and receives an instruction based on the user's touch operation. The coin insertion unit 42 detects the insertion of coins into the coin insertion slot. When the coin insertion unit 42 detects that a predetermined amount of coins (play fee) required for play has been inserted, the coin insertion unit 42 outputs an activation signal indicating that fact to the control unit 53. Instead of the coin insertion unit 42, a device that accepts payment of a play fee by electronic money or a credit card and outputs the activation signal to the control unit 53 may be provided.

The shooting unit 50 is composed of a lighting control unit 18, a front camera 111, a side camera 112, a front monitor 113, a side monitor 114, and a speaker 19 in order to perform shooting processing in the shooting space 10.

The front camera 111 and the side camera 112 take a picture of the subject under the control of the control unit 53. The front camera 111 and the side camera 112 output a dynamic image of the subject obtained by imaging or a still image of the subject obtained by shooting to the control unit 53. As described above, the front camera 111 mainly shoots a still image and a moving image of a subject from the front in close-up, and the side camera 112 shoots the whole body of the user who is the subject from the side at a wide angle. It is for. Therefore, in order to obtain the optimum image, the cameras 111 and 112 are set in advance with different ideal focal lengths, angles of view, trimming ranges, etc., depending on the number of photographers. The cameras 111 and 112 output the image of the subject as an image signal in real time. The image signals output from the cameras 111 and 112 are input to the control unit 53 and temporarily stored in the internal memory thereof. In the above example, since the front camera 111 is for individual authentication shooting or close-up shooting, which will be described later, a zoom lens capable of changing the focal length within a range that does not cover the entire body of the user who is the subject is used. In the modified example, a zoom lens that includes the focal length on the wider angle side is used as the lens of the front camera 111, and by changing the focal length, both close-up photography and full-body photography may be performed. .. When performing full-body photography, the focal length of the zoom lens may be shorter than in close-up photography.

The control unit 53 functions as a display control unit, transmits an image signal of a subject image captured by the front camera 111 to the front monitor 113, receives a user's operation input to the front monitor 113 as a touch panel, and receives a user's operation input to the front monitor 113 as a touch panel to receive a side camera. The image signal of the subject image captured by 112 is transmitted to the side monitor 114. At this time, the control unit 53 may transmit an image or a still image in which a background image is synthesized with the subject image by chroma key composition to the front monitor 113 or the side monitor 114, or a correction process for changing the size and contour of the eyes. The video or still image may be transmitted to the front monitor 113 or the side monitor 114. The captured image data saved as a still image (hereinafter, may be simply referred to as “captured image”) is high-resolution still image data, and the captured image data for displaying a live image (“through image data”). ”) Is often low-resolution moving image data, but may be high-resolution moving image data.

The front monitor 113 displays the live image being captured by the front camera 111 in real time. The front monitor 113 also displays captured images as still images captured by the front camera 111 and the side camera 112. The side monitor 114 displays a live image being captured by the side camera 112 in real time, and is not a touch panel type monitor unlike the front monitor 113, but may be configured by a touch panel type monitor. Further, the side camera 112 and the side monitor 114 are not indispensable configurations, and only the front camera 111 and the front monitor 113 may be used.

The illumination control unit 18 controls the flash emitted by the front illumination unit 115 in the photographing space 10 according to the illumination control signal supplied from the control unit 53. The lighting control unit 18 controls the electric lamps (LEDs, fluorescent lamps, etc.) provided inside the ceiling lighting unit 14 and the front lighting unit 115, so that the brightness in the shooting space 10 is adjusted according to the stage of the shooting process by the user. To adjust. The speaker 19 outputs sound according to the stage of the shooting process by the user.

The editing units 51a and 51b are provided for editing processing in the editing spaces 20a and 20b, respectively. Since the functions and configurations of the editorial unit 51a and the editorial unit 51b are the same, only the editorial unit 51a will be described below, and the description of the editorial unit 51b will be omitted.

The editorial unit 51a is composed of an editing monitor 213 as a tablet built-in monitor, two touch pens 214A and 214B, and a speaker 215. The editing monitor 213 displays the editing screen under the control of the control unit 53, and detects the user's editing operation on the editing screen. That is, when the editing monitor 213 is touched by the touch pens 214A and 214B, the user detects the operation and which of the touch pens 214A and 214B is touched, and transmits the operation signal to the control unit 53. As a result, the user edits the composite image to be edited. Specifically, for example, so-called graffiti such as characters, patterns, and figures are input to the image displayed on the editing monitor 213 with an input device such as touch pens 214A and 214B. As a result, an output image in which graffiti is superimposed on the composite image is generated. The image data based on the output image thus generated is transmitted to the output unit 52.

The output unit 52 executes the output process in the output space 30. The output unit 52 includes a touch panel type liquid crystal monitor 311, a non-contact communication unit 312, and a speaker 314. In addition, a printer 34 for printing an output image and a sticker paper unit 35 are also provided. The printer 34 prints a composite image or an output image obtained by correction processing or editing processing on the sticker paper housed in the sticker paper unit 35 mounted on the printer 34, and discharges the composite image to the sticker paper discharge port 313. To do. The non-contact communication unit 312 is a wireless communication unit that receives and transmits data to and from a mobile terminal. For example, a short-range wireless communication module such as Bluetooth (trademark) or NFC, or a wireless network communication module such as wireless LAN can be used. included. The image data created by the editorial unit 51 is transmitted by the communication unit 55 to an external image storage server via the network.

<Functional configuration of a game shooting device>
FIG. 8 is a functional block diagram schematically showing the functional configuration of the game photographing device 100. The game photographing device 100 includes a presetting unit 40, an imaging processing unit 130, a display control unit 135, a data generation unit 140, a data storage unit 150, and an output processing unit 160.

This figure shows a functional configuration for realizing the game photographing device 100, and other configurations are omitted. In this figure, each element described as a functional block that performs various processes can be configured by the hardware shown in FIG. 7, such as a camera, a CPU, a main memory, and another LSI (Large Scale Integrated circuit). Further, in terms of software, it is realized by the CPU executing a program or the like loaded in the RAM. Therefore, it is understood by those skilled in the art that these functional blocks can be realized in various forms by hardware only, software only, or a combination thereof, and is not limited to any of them.

Prior to shooting, the preset unit 40 presets the shooting contents such as the number of shots of the user to be the subject, the name and group name, the favorite content set, and the favorite moving image theme based on the user's instruction. As the number of photographers preset by the preset unit 40, any one of "2", "3", and "large number" is set by the user.

The photographing processing unit 130 includes an individual authentication unit 131, an imaging unit 132, and an image acquisition unit 133. The imaging unit 132 uses the front camera 111 or the side camera 112 as a subject for one or more users according to the control by the individual authentication unit 131 or the image acquisition unit 133 based on the play start instruction received from the preset unit 40. Take an image. The individual authentication unit 131 acquires a photographed image for authentication, detects a person for the acquired photographed image for authentication, and executes person authentication. The individual authentication unit 131 executes person detection for detecting a person from the acquired photographed image for authentication. The person detection in the present embodiment is, for example, face detection. The individual authentication unit 131 detects the face region by extracting the skin color region and organs such as eyes and nose from the photographed image for authentication. The individual authentication unit 131 detects a feature amount such as the distance, arrangement, and size of organs such as eyes and nose from the detected face area, and executes person authentication that identifies the feature of each person based on the feature amount. .. The individual authentication unit 131 temporarily stores the photographed image for authentication in the data storage unit 150.

The image acquisition unit 133 captures and acquires a photographed image for the user's main shooting a plurality of times with respect to the subject imaged by the image pickup unit 132. In this example, the image acquisition unit 133 executes the main shooting at least five times and generates five shot images. The image acquisition unit 133 stores the acquired plurality of captured images for the main shooting in the data storage unit 150 as a group of photographic images. More specifically, the image acquisition unit 133 assigns the same play ID to the acquired plurality of captured images for main shooting, and assigns a unique image identifier to each captured image in order to identify the captured image. To do.

The image acquisition unit 133 captures and acquires a moving image of the user a plurality of times for the subject imaged by the image pickup unit 132. The image acquisition unit 133 executes, for example, three moving images, and acquires three moving images.

The display control unit 135 causes the image pickup unit 132 to display an image being photographed (hereinafter referred to as “live view”) on the screen of the front monitor 113 or the side monitor 114 in real time. Prior to the start of movie shooting, the display control unit 135 displays a model movie for showing the user a model of the subject movement during movie shooting on the screen of the front monitor 113. After finishing the display of the model moving image, the display control unit 135 switches to the live view of the user's image captured by the imaging unit 132. After switching to the live view, the image acquisition unit 133 takes a moving image of the user. By repeating the process from the display of the model moving image to the shooting of the moving image of the user three times, three moving images are acquired. The image acquisition unit 133 stores the acquired moving image in the data storage unit 150 in association with the play ID. The procedure for shooting a moving image will be described later.

The data generation unit 140 includes a background processing unit 141, a correction processing unit 142, an editing processing unit 143, and a moving image processing unit 144. The background processing unit 141 synthesizes a subject and a background image for each of the photographed image for authentication, the plurality of photographed images for main shooting, and the plurality of moving images stored in the data storage unit 150 by a known chroma key composition technique. To do. That is, the background processing unit 141 converts the background information (chroma key mask) for distinguishing the subject in the captured image from the background area other than the captured image into a binarized image or a grayscale image, or data obtained by compressing them. And use the background information to acquire an image of the subject extracted from the background. Regarding the captured image for authentication, the background processing unit 141 synthesizes an image of only the subject with a predetermined background image, for example, a background image in which a single achromatic color such as white, gray, or black is applied to one surface. Regarding the captured image for the main shooting, the background processing unit 141 synthesizes an image of only the subject with a preset background image of the user's preference. The image of only the subject and the composite image combined with the background image are stored in the data storage unit 150. For the captured image for the main shooting, another background image can be combined with the image of only the subject in a later editing process to regenerate a new composite image. The background processing unit 141 may perform a process of extracting a subject in real time from the live view image being photographed by the imaging unit 132, synthesizing the background image, and displaying it by the display control unit 135. Further, the background processing unit 141 also processes the above-mentioned chroma key composition on the moving image acquired by the image acquisition unit 133 in the same manner to generate a combined moving image.

The correction processing unit 142 executes person detection and person authentication in the captured image for the main shooting, and also executes correction for the captured image for the main shooting. The person detection in the captured image for the main shooting is, for example, face detection similar to the person detection in the captured image for authentication. The correction processing unit 142 detects the face region by extracting the skin color region and organs such as eyes and nose from the captured image for the main shooting. The person authentication in the photographed image for the main shooting includes the same processing as the person authentication in the photographed image for authentication and the person identification processing. More specifically, first, the feature amount for each person such as the distance, arrangement, and size of organs such as eyes and nose is detected from the face area detected by the person detection in the photographed image for the main shooting. The feature amount for each person is collated with the feature amount for each user specified in advance by person authentication in the photographed image for authentication and temporarily stored in the data storage unit 150, and based on the collation result, for the main shooting. The process of identifying which user is the person included in the captured image of is "person identification". For example, when there are two types of feature amount information for each user that are specified and stored in advance, A and B, the feature amount for each person specified from the captured image for the main shooting is relative to either A or B, respectively. Check if it is close to. As a result, which user is specified for each person included in the captured image for the main shooting.

The correction for the captured image for the main shooting is a process of changing the color and size of the eyes, the contour of the face, the color and texture of the skin, etc. Executed for the captured image for capture. The correction processing method differs depending on the number of photographers and whether the image is a close-up image or a full-body image. When the preset number of photographers is 3 or less, individual correction is executed for each person by the correction method selected according to the preference preset for each user. When the number of people to be photographed exceeds three, the correction processing unit 142 executes the same correction for all the people included in the photographed image for the main shooting.

The editing processing unit 143 executes an editing process called graffiti, and at the same time, plays for identifying a photographic image group including a plurality of captured images stored in the data storage unit 150 and a user specified by the individual authentication unit 131. Generate content data associated with the ID. The content data may include background information acquired by the background processing unit 141 and identification information of the game photographing device 100. The editing processing unit 143 generates content data each time the image acquisition unit 133 captures a series of photographic image groups (for each play). The content data can be uniquely specified by using the play ID generated by the image acquisition unit 133.

The editing processing unit 143 sets a captured image that has been combined with the background image by the background processing unit 141 and corrected by the correction processing unit 142 as the image to be edited, and executes the editing process on the image to be edited. The editing process can be executed individually for the plurality of images to be edited, and can be executed simultaneously for the plurality of images to be edited based on the respective instructions from the plurality of users. The instructions from a plurality of users specify the position of the user's editing operation on the editing monitor 213 using the touch pen 214, for example, the selection operation for selecting the editing function to be used or the editing target image to be edited. It is input by the input operation. The editing processing unit 143 executes graffiti processing for superimposing stamp images and characters on the image to be edited and determining the print layout on the image to be edited, generates a plurality of output images, and stores the images for output. Store in 150.

The moving image processing unit 144 executes editing processing on a plurality of moving images acquired by the image acquisition unit 133 and stored in the data storage unit 150, and generates a display moving image and a transmission moving image to be output. First, the video processing unit 144 reads out the template video corresponding to the video theme selected by the user in the preset from the data storage unit 150, and three shots are taken in each of the three or more image frames included in the template video. Edited moving images are set by embedding the moving images of the above once or multiple times, respectively. The edited moving image is also set as a display moving image or a transmission moving image. The three image frames included in the template moving image may be arranged two-dimensionally or three-dimensionally. In the latter case, a three-dimensional object is displayed in the template video, and a plurality of frame shapes included in the three-dimensional object form a screen frame, and a plurality of moving image frames are embedded in each screen frame for playback. Set the edited video with. At this time, the moving image processing unit 144 merely sets the template moving image and the embedded moving image to be reproduced at the same time with a plurality of moving images embedded in the template moving image. However, as a modification, the moving image may be embedded in the template moving image and converted into new moving image data.

When converting the edited moving image into new moving image data, the moving image processing unit 144 inserts an image in which the characteristics of the moving image appear in the first frame thereof. When displaying a list of moving images in a thumbnail format on a mobile terminal or PC application or a screen on an SNS, the first frame is usually automatically set as a thumbnail in order to show the contents of the moving image as an image. However, when creating a moving image, for example, when the title or the main part is created so as to fade in, the first frame may be all black or all white. Alternatively, in many cases, an image having uncharacteristic content, such as displaying only a part of the title or the subject, is the first frame. Therefore, a characteristic image is inserted as the first frame so that the contents can be grasped to some extent at a glance and the information that the person who sees the thumbnail wants to play can be easily presented. "Images that show the characteristics of this moving image" include character strings such as the name of the template video or the video theme, the model name of the game shooting device 100, the user's name, and the patterns and pictures that symbolize the video theme. Use an image in which the motif of is displayed. Alternatively, the edited video may be subjected to person detection, the frame in which the user's image is detected may be extracted, and the frame may be inserted into the first frame.

The output processing unit 160 includes a print processing unit 161, a wireless output unit 162, and a transmission output unit 163. The print processing unit 161 prints a plurality of output images stored in the data storage unit 150 on the sticker paper by the printer 34. The wireless output unit 162 transmits a plurality of output images to the user's terminal via the non-contact communication unit 312. The transmission / output unit 163 transmits all the output images to the management server 300 via the network. The transmission output unit 163 receives personally identifiable information such as an e-mail address input by the user on the keyboard displayed on the touch panel after the end of the play. The transmission output unit 163 can upload the output image to the management server 300 in association with the input e-mail address and play ID, and allow the user to download the output image later. The transmission output unit 163 transmits the display moving image set by the moving image processing unit 144 to the display control unit 135, so that the display moving image is displayed on a display device such as a liquid crystal monitor 311. The transmission output unit 163 uploads the transmission moving image or the captured moving image set by the moving image processing unit 144 to the management server 300 in association with the input e-mail address and play ID, and causes the user to download it later. be able to. The management server 300 stores an output image and a moving image for transmission (or a moving image taken) in a form corresponding to the e-mail address and the play ID, and the play ID associated with the e-mail address is addressed to the e-mail address. Is transmitted as the content access information by the URL including the argument. By accessing the URL of the received content access information with a terminal device 500 such as a mobile phone, the user can open a page on which the image for output or the moving image for transmission in a predetermined image acquisition website can be downloaded. It can be downloaded. In addition, a play ID is printed on the sticker paper together with the output image, and the user can download his / her own output image or transmission moving image by inputting the play ID into a predetermined image acquisition website. Can be opened and downloaded. When the user has registered his / her e-mail address as a free member, he / she can open a page on which the image selected by himself / herself can be acquired from the plurality of output images uploaded to the management server 300. When the user has registered his / her e-mail address as a paid member, he / she can open a page where all of the plurality of output images uploaded to the management server 300 can be acquired. Further, in the case of the specification that the transmission output unit 163 uploads the captured moving image to the management server 300 instead of the transmitting moving image, the information indicating the moving image theme selected by the user is uploaded together with the captured moving image. As a result, even in the management server 300 or in the dedicated information presentation application on the terminal device 500 that acquires information from the management server 300, the moving image can be embedded in the template video corresponding to the video theme selected by the user. A moving image for display can be configured. In that case, the management server 300 or the dedicated information presentation application on the terminal device 500 that acquires information from the management server 300 stores data such as template moving images and background music corresponding to each moving image theme in advance.

FIG. 9 is a flowchart showing the process from the preset setting to the main shooting. First, the user touches the touch panel 41 of the preset unit 40 to start, and the user inputs the selection of the number of photographers, the name, the selection of the background image, and the selection of the video theme as presets by the touch operation (S100). .. When the "two-person course" is selected (S101), a guide to the two users to take a picture for authentication is displayed on the front monitor 113 (S104), and the front camera 111 of the two users Authentication shooting is executed (S105). When the "three-person course" is selected (S102), a guide to the three users to take an authentication photograph is displayed on the front monitor 113 (S107), and the front camera 111 of the three users Authentication shooting is executed (S108). When "Large number course" is selected (S103), authentication photography is not performed (S104, S105, S107, S108 are skipped). After the pre-setting and authentication shooting, the close-up image is shot three times by the front camera 111 as the main shooting (S106).

After the close-up shooting, the guidance for guiding to the side monitor 114 on the side is displayed on the front monitor 113 to direct the user toward the side monitor 114 (S110), and the guidance regarding the standing position of the user is displayed on the side monitor 114. (S111), and the user's whole body image is taken twice by the side camera 112 as the main shooting (S112). The guidance for guiding to the front monitor 113 is displayed on the side monitor 114 again to direct the user toward the front monitor 113 (S113), the guidance for movie shooting is displayed on the front monitor 113 (S114), and the movie shooting is performed. After displaying a sample movie as a model as a tutorial, a series of steps of shooting a movie with the front camera 111 is repeated three times (S116). This series of processes will be described later using a screen example.

Referencing the remaining time of play at this point (S120), if there is remaining time (S121), provide the user with a bonus shot shot, return to S120 (S122), and if there is no remaining time (S123), The guidance to guide to the graffiti booth (editing space 20) is displayed on the front monitor 113 (S124), the guide to which of the two editing spaces 20a and 20b should be moved (S125), and the movement to the graffiti booth (S126). ).

FIG. 10 is a flowchart showing the process of correction and editing. After displaying the correction (also referred to as "retouch") guidance screen on the editing monitor 213 (S200), if a two-person course has been selected in advance (S201), "taste selection" (S202), "taste selection", " The user can select the preference for the three items of "finish selection" (S203) and "brightness selection" (S204), display "preparing for graffiti", and make corrections based on the preference selection of the captured image (S106). It is applied to the three close-up images acquired in (S205) and the two whole-body images acquired in S107). If the 3-person course is selected in advance (S210), the user can select the preference for the two items "taste selection" (S211) and "brightness selection" (S212), and select "preparing for graffiti". Along with displaying the image, correction based on a preference selection is applied to the captured image for the main shooting (S205). In the case of 3 people, it takes more time to select individual preferences than in the case of 2 people, so the number of correction items is reduced compared to the case of 2 people. If a large number of courses are selected in the preset (S220), the user can select the preference for only one item of "eye size, brightness selection" (S221), and "preparing graffiti" is displayed. At the same time, correction based on the selection of preference is applied to all the captured images for the main shooting (S205). In the case of a large-group course, there is not enough time to select the preferences for the number of people, so the purpose is to select only one item that can be specified for all at once, rather than selecting individually for each user.

After shifting from "preparing graffiti" to the main part of graffiti (S230), editing processing is performed on a plurality of corrected images for main shooting based on instructions from the user, and then as a print layout. The division layout selection and the selection of the photograph to be printed large are set based on the user's instruction (S232), the movement guidance to the output space 30 is displayed on the editing monitor 213 (S233), and the output space 30 (“exit corner””. (Also also referred to as) (S234).

FIG. 11 is a flowchart showing the process of output processing. A screen (sending image selection screen) for the first user to select one from a plurality of output images is displayed (S302). Then, a screen (e-mail address input screen) for inputting the e-mail address of the first user is displayed (S303). Subsequently, the transmission image selection screen for the second user is displayed (S304), and the e-mail address input screen for the second user is displayed (S305). If there is a third user (Y in S306), the e-mail address input screen for the third user is displayed (S307). If there is a fourth user (Y in S308), the e-mail address input screen for the fourth user is displayed (S309). If there is no third person, skip from S307 to S309 (N in S306), and if there is no fourth person, skip S309 (N in S308).

After selecting the image for transmission and inputting the e-mail address, the SP movie is presented on the LCD monitor 311 and transmitted to the management server 300 (S310), and the output image is displayed while "Preparing to print" is displayed on the LCD monitor 311. Is printed on the sticker paper (S311). At this time, the photographic image group of all the captured images for the main shooting that have been edited is uploaded to the management server 300. Finally, the completion of printing is displayed on the liquid crystal monitor 311 and the process ends (S312).

FIG. 12 shows an example of a screen that allows the user to select the content of moving image shooting in the preset setting. FIG. 12A is a guidance screen explaining that the screen is about to shift to the video theme selection screen, and the preset unit 40 has thumbnails of two sample movies in two sample movie frames 640 displayed on the touch panel 41. Are played back and displayed respectively. FIG. 12B is a movie theme selection screen, and the preset unit 40 displays six movie candidate frames 641 on the touch panel 41. Thumbnails of different sample movies are played back and displayed in each of the six movie candidate frames 641 as video theme options. The sample movie was created by embedding a model video, which is a video of the model movement at the time of shooting, in a template movie that includes a background image, background music, and a frame image to be combined with the subject in the moving image to be shot later. It is a thing. The type of template video such as background video, background music, and frame image and the model operation are different for each video theme. The user simply selects one of the sample movies to his liking, and the shooting and editing are performed according to the video theme corresponding to the sample movie, and a moving image for output according to the video theme can be easily created. The movie candidate frame 641 has a role of a selection button for displaying thumbnails of sample movies created in advance based on six types of video themes. The thumbnail of the sample movie reduces the display size of the sample movie to display a part of the content, and the first few seconds of the movie may be played once or repeatedly. Further, the entire contents may be fast-forwarded and reproduced while skipping frames at predetermined intervals. The entire contents may be reproduced without skipping or the like.

The preset unit 40 simultaneously previews and reproduces thumbnails of six types of sample movies to be displayed in the six movie candidate frames 641. This saves the trouble of switching the selection while checking the preview, as compared with the case where a separate preview frame is provided and only the sample movies of the thumbnails selected by the user are played back one by one in the preview frame. In addition, the time required for presetting can be shortened, and the setting can be simplified. The preset unit 40 superimposes the selection frame 642 on the movie candidate frame 641 of the thumbnail selected by the user by touch operation, and indicates that this thumbnail is currently selected. When the user selects one of the movie candidate frames 641, the preset unit 40 outputs the background music included in the sample movie corresponding to the selected thumbnail from the speaker installed near the entrance. At this time, the preset unit 40 reduces the volume of the normal background music output from the speaker near the entrance at normal times, and outputs the background music of the sample movie at a louder volume than the normal background music. When the user switches the selection of the movie candidate frame 641, the preset unit 40 plays only the newly selected sample movie from the beginning, and switches to the output of the background music of the sample movie to play from the beginning. At the time of switching, the preset unit 40 stores the time position where the sample movie before switching was being played and the time position where the background music was being played, and when the same sample movie is selected again. Resumes playback of the sample movie and background music from the memorized time position. At the time of shifting to the selection screen of FIG. 12B, the selection frame 642 is initially superimposed on the predetermined movie candidate frame 641, for example, the movie candidate frame 641 on the upper left, and is selected by the user's touch operation. If any of the movie candidate frames 641 including the movie candidate frame 641 is touched with the OK button selected, the selection of the movie theme is confirmed. When the selection of the video theme is confirmed by touching the OK button, the preset unit 40 stops the playback of the selected sample movie and the background music, and returns the volume of the normal background music to the original volume.

FIG. 13 shows an example of a guidance screen displayed before the start of moving image shooting. The display control unit 135 plays and displays three model videos 645a to 645a to c and a sample movie 676 created using them side by side as a guidance screen, and "shoots a video like this! Imitating the movement of the model". I will briefly explain the outline of the subsequent shooting procedure with a character string such as "Try it!". The movie shooting is performed three times, and the models of the three shot movies are displayed as the first to third model movies 645a to c, and the sample of the edited movie is displayed as the sample movie 676. These three model videos and sample movies are moving images prepared in advance based on a video theme selected by the user in advance. In the actual shooting stage described later, the first moving image is shot after displaying the first model moving image 645a, the second moving image is taken after displaying the second model moving image 645b, and the second moving image is taken. After displaying the model moving image 645c of 3, the third moving image is taken. On the guidance screen of FIG. 13, three model moving images 645a to 645 corresponding to each shooting are played back in the same order as the shooting order, and finally the sample movie 676 is played back.

The model videos that serve as models for the three shooting videos are fast-forward videos of the characters performing different actions for each video, and for example, professional fashion models and talents are appointed as the characters. It is an image that behaves like a model. Here, the moving image shot after this is finally embedded in the template moving image in the form of being played back in fast forward. The reason why the fast-forward playback is performed is to make the awkwardness of the user's movement less noticeable by making the image faster than the actual movement of the person to be the subject. Since the user who is the subject is a general person and is not usually a professional model, it is easy to make an image like a professional model by making the awkwardness inconspicuous even if it behaves like a model. It can be made easier to create. In order to shorten the explanation time while reproducing such fast-forward to make the user imagine, the model moving image displayed in FIG. 13 is also reproduced in a form in which the characters are fast-forwarded. The sample of the edited movie to be displayed as the sample movie 676 is a sample of the movie created by embedding three model movies in the template movie. The illustrated edited video is a video in which a frame image of a cube's three-dimensional object is rotated against a predetermined image as a background, and the first to third model videos 645a are formed on three of the six sides of the cube. It is composed by embedding ~ c. A template moving image using a frame image of a three-dimensional object will be described later with reference to FIG.

As a modification, a model moving image synthesized using the user's face image may be displayed on the guidance screen and the model screen described later. The user's face image is a face image extracted from the photographed image for authentication, and may be combined with the face part of the person included in the model video of the face image and created as a model video dedicated to the user. .. Alternatively, in the form of a so-called avatar, a model moving image may be created in a form of reproducing the movement of the model by using an animation in which a user's face image is pasted or a doll of a three-dimensional model. Further, the model moving image of the avatar created in this way may be uploaded to the management server 300 and downloaded by the user together with other shot images and shot videos.

FIG. 14 shows the screen transition in the first half of the moving image shooting process. First, the display control unit 135 displays the start screen 650 on the front monitor 113 for 4 seconds, displays a character string explaining the start of moving image shooting and the scheduled number of times of shooting on this screen, and reads out the explanation. Is output. However, in the modified example, the screen display itself of the start screen 650 is omitted, and when the model screen 651 which is the next screen or the screen of FIG. 13 which is the previous screen is displayed, only voice is output. May be shortened.

Next, the display control unit 135 displays the model screen 651 for 3 seconds and displays the first model moving image 645a in a large size in the upper center of the screen. By outputting the audio of the explanation while displaying a character string such as "Let's imitate the model pose!" On the screen, the first model video 645a will be played before the start of playback. The user is urged to see and memorize the movement of the person shown in the first model moving image 645a in order to imitate the movement. The model video displayed here is one of three types of shot video models prepared based on the video theme selected by the user, and is in the form of a still image in which the frame at the beginning of the movie is paused. Display with. By displaying small thumbnails of the second model video 645b and the third model video 645c on the right side of the first model video 645a, it is currently a model for the first shooting, and the rest. It suggests that two shots are coming up. In this way, the three model moving images 645 are arranged in the horizontal direction, and the thumbnail of the movie currently being shot is displayed in a relatively large size. On the model screen 651, the first thumbnail from the left, which indicates the first thumbnail, is enlarged and displayed so as to be gradually reduced to the second and third thumbnails.

Next, the display control unit 135 displays the model screen 652 for 3 seconds and reproduces and displays the first model moving image 645a on the screen for 3 seconds. At this time, the background music corresponding to the moving image theme selected by the user is also played together with the image of the first model moving image 645a. The first model video 645a is the same video as the first model video 645a played on the guidance screen of FIG. 13, but is played on the guidance screen at a frame rate of less than 3 seconds in fast forward. On the model screen 652, playback is performed at a frame rate of 3 seconds, which is the same as the shooting time, according to the actual shooting time. The user sees the movement of the person included in the first model moving image 645a and temporarily stores the moving image. The first model moving image 645a played at this time is basically the same as the first model moving image 645a displayed on the guidance screen of FIG. 13, but it takes less than 3 seconds on the guidance screen. While it was played back in fast forward, it is played back at a relatively slow speed on the model screen 652. For example, on the model screen 652, the reproduction is performed so that the actual shooting time is 3 seconds, which is the same as the shooting time. Both may be displayed with different frame rates of the same movie, may be prepared as separate movies with different frame rates, and the movie in which the person in the picture moves relatively fast and the movie in which the motion is slow. May be prepared separately. The action that actually prompts the user is an action at a normal speed or a slow speed that can be easily reproduced as a human action, and the action is shot at a frame rate of, for example, 10 fps, and this is fast-forwarded. For example, what is reproduced at 24 fps is a model moving image that is fast-forwarded and reproduced on the guidance screen. The movie finally provided to the user is also a movie that is fast-forwarded and played back at a frame rate of 24 fps, and the awkwardness of the user's operation can be made inconspicuous.

After the reproduction of the first model moving image 645a is completed on the model screen 652, when the screen shifts to the shooting screen 653, the display control unit 135 first displays the display area of the first model moving image 645a that has been displayed so far. The image of the user who is the subject is displayed on the shooting screen 653 in real time by switching to the image of the live view 646a. Again, the background music corresponding to the video theme selected by the user continues to play. Further, when a moving image shot according to a moving image theme selected by the user is set as an image in which the user is fitted in a predetermined image frame, the image in the image frame is set as the foreground of the first live view 646a. Is displayed. On the other hand, if the video shot according to the video theme selected by the user is set to display decorative objects such as characters and pictures in the foreground so as to overlap the person, the live view overlaps with the user's video and the user himself / herself. Such a foreground may be hidden to avoid obscuring the image of. Here, a character string such as "Are you ready? I'll shoot!" Is displayed to inform the user that recording is about to start. Three seconds after the display of the shooting screen 653, the display control unit 135 counts down the time until the start of recording on the screen like the shooting screen 654 and displays the image of the opened clapperboard 648 to indicate that recording is imminent. Suggest.

Next, an image in which the clapperboard 648 is closed is displayed as in the shooting screen 655, and a voice "recording start" is output to notify the start of recording. Then, recording of the moving image is started while the first live view 646a is displayed on the screen. During the recording, the background music corresponding to the video theme selected by the user continues to play. The image acquisition unit 133 acquires a moving image by shooting at a frame rate of 10 fps. That is, the frame rate is lower than that of normal movie shooting, and if this is played back at a frame rate of 10 fps, the frame interval becomes wider than that of a normal speed movie, and the moving image becomes slightly closer to frame advance. .. 30 frames are shot by shooting for 3 seconds. Three seconds after the start of recording, a voice "cut" is output to notify the end of recording. Then, as shown on the shooting screen 656, the fact that the recording has been completed successfully and the fact that two more recordings remain are displayed as character strings. At this time, instead of the first live view 646a that has been displayed up to that point, the shooting screen 656 displays a still image stopped at the frame when the recording ends. Since this frame is the same as the final frame of the first shooting movie 647a that has just finished recording, the thumbnail of the first shooting movie 647a should be displayed in the form of a still image substantially stopped at the final frame. be equivalent to.

Next, the second recording is started, and the display control unit 135 switches to the model screen 657. The thumbnail of the first shooting movie 647a displayed on the previous shooting screen 656 is moved to the left while being reduced, and the second model movie 645b is moved to the center while being enlarged from the right. .. At this point, it is displayed in the form of a still image of the opening frame of the second model moving image 645b. On the model screen 658, playback of the second model moving image 645b is started in the same manner as on the model screen 652.

FIG. 15 shows the screen transition in the latter half of the moving image shooting process. After the reproduction of the second model moving image 645b is completed on the final model screen 658 of FIG. 14, the screen shifts to the shooting screen 659 of FIG. 15, and the display control unit 135 switches to the display of the second live view 646b. A countdown to the start of recording is displayed and an image of the opened clapperboard 648 is displayed to indicate that recording is imminent. On the shooting screen 660, an image in which the clapperboard 648 is closed is displayed, and a voice "recording start" is output to notify the start of recording. Then, recording of the moving image is started while displaying the second live view 646b.

After the second recording is completed, the third recording is started. On the model screen 661, the thumbnails of the first shooting movie 647a and the second shooting movie 647b are displayed small on the left side, and the third model movie 645c is displayed in the center. Make the thumbnail of the movie larger. After playing the third model video 645c on the model screen 662, the third live view 646c is displayed on the shooting screen 663 to count down to the start of recording, and the image of the opened clapperboard 648 is displayed to record soon. It suggests that. On the shooting screen 664, an image in which the clapperboard 648 is closed is displayed, and a voice "recording start" is output to notify the start of recording. Then, the moving image is recorded while the third live view 646c is displayed. After the recording is completed, the final frame of the third shooting movie 647c is displayed as a thumbnail on the shooting screen 665. As described above, the process from the reproduction of the model video to the display and recording of the live view is repeated three times. In other words, it is possible to easily shoot three ideal videos by repeating the process of recording a video one by one while showing the user a model operation and memorizing it. Can be done.
In addition, during the recording of the moving image, a red character string or a mark of "REC" indicating that the moving image is being recorded may be displayed on or near the live view 646.

FIG. 16 shows an example of a template moving image using a frame image of a three-dimensional object. The edited moving image illustrated in FIG. 13 is also a moving image in which a predetermined image corresponding to the moving image theme is reproduced as a background image 675, and a frame image of a cube object 670, which is a three-dimensional object, is rotated as a foreground. Further, an image for displaying characters and patterns may be superimposed and displayed as a foreground layer. The six surfaces that make up the cube object 670 are transparent and have a skeleton shape with only a skeleton. By embedding the first to third captured moving images in three of the six surfaces and playing them back, a moving image for output is set. As shown in the figure, captured moving images are embedded in each of the first surface 671, the second surface 672, and the third surface 673. For embedding a moving image in each surface, for example, a technique such as motion tracking or match move is used. In this case, the feature points at the four corners of the surface are specified as tracking points in order to specify the surface to be tracked, and the moving image screen is also deformed according to the shape change of the surface while tracking the tracking points. A moving image can be embedded in each surface of the cube object 670 by performing the fitting composition. However, the three moving images embedded in the first surface 671, the second surface 672, and the third surface 673 are moving images of only the subject whose background is removed by the background processing unit 141 using a chroma key mask. As a result, even if a moving image is embedded in the three surfaces of the cube object 670, the portion other than the subject is transparent on each surface so that the back can be seen. In the example of the template video shown in the figure, the part other than the subject is transparent, but depending on the video theme selected by the user, the background image corresponding to the video theme is displayed without transmitting the part other than the subject on each surface. A template moving image to be combined with a part other than the subject may be used.

In the illustrated template video, the cube object 670 is an image that rotates to the left, and when each side surface faces the front side, playback of the moving image embedded in that surface is started, and the other surfaces The embedded moving image is stopped playing with the opening frame displayed. That is, first, when the first surface 671 faces the front side, the embedded moving image of that surface starts to be reproduced, and two seconds after the reproduction of the embedded moving image ends, the speed at which the second surface 672 approaches the front side. The cube object 670 rotates to the left at. When the moving image of the first surface 671 is completed and the second surface 672 approaches the front side, the moving image of the second surface 672 is started to be reproduced. Similarly, the moving image of the third surface 673 is started to be reproduced at the timing when the reproduction of the second surface 672 is completed and the third surface 673 approaches the front side. By playing back the three moving images, each of which is fast-forwarded for 2 seconds, in a chained manner in this way, an image of about 6 seconds is obtained, and an edited moving image of about 15 seconds including the preceding and following images is created. Here, an example of embedding a moving image in the surface of a three-dimensional object such as a cube was explained, but as another video theme, the moving image is embedded in an image frame arranged two-dimensionally and is edited. Template videos for videos are also available.

FIG. 17 shows an example of a guidance screen for guiding the provision of the edited moving image. This guidance screen is displayed on the liquid crystal monitor 311 of the output housing 31 while printing the output image, and displays a character string explaining that the edited video has been completed and how to provide the edited video, and the display thereof. The voice that reads the explanation is output. In the center of the screen, a part of the edited video and the background music corresponding to that part are played as a digest version of the edited video. However, as a modification, all parts of the edited moving image and all parts of the background music thereof may be played. As a method of providing the edited video, a method of having the user download a dedicated application and playing the edited video on the management server 300 through the application may be used. Alternatively, a method of providing a URL for downloading the edited video itself from the management server 300 by displaying a character string or a two-dimensional code may be used. Further, on this screen, a button for downloading a dedicated application may be displayed. When this button is pressed, an e-mail containing the URL for downloading the dedicated application is sent to the entered e-mail address.

(Second Embodiment)
The amusement photographing device in the present embodiment is different from the first embodiment in that it has an editing function capable of performing graffiti and the like not only on a still image but also on a moving image. Hereinafter, the differences from the first embodiment will be mainly described, and the description of the common points will be omitted.

In the "main part of graffiti" (S230 in FIG. 10), the amusement shooting device of the present embodiment includes not only graffiti on the shot image as a still image but also the shot image embedded in the template moving image selected in the preset setting in FIG. It has a function that allows selection and scribbling on the captured image and captured video. The template videos illustrated below are pre-produced on the premise that users who shoot together will give each other a video or give a video to another friend, and will send a message to the recipient of the gift. Can be inserted into videos in the form of stamps and text.

In the present embodiment, the image acquisition unit acquires a moving image and a still image of the user captured by the imaging unit. The moving image processing unit embeds a moving image in the frame shape for moving images included in the template moving image held in advance and reproduces the moving image, and embeds the still image in the frame shape for still images included in the template moving image and plays it as a moving image. Set the moving image. The display control unit presents a plurality of template moving images as options as the template moving images. The display control unit presents a plurality of still images for embedding in the still image frame shape as options. The display control unit presents as an option to embed only a still image that can match the aspect ratio of the still image frame shape included in the template moving image selected by the user in the still image frame shape. The moving image processing unit embeds a still image selected by the user in each of the frame shapes for still images on a plurality of frames included in the template moving image, thereby embedding the still image in the template moving image. The display control unit displays a play button image on the screen together with a frame of a portion to be edited in the template moving image. When the play button image is operated by the user, the display control unit plays and displays a predetermined number of frames located before and after the frame of the edit target portion.

FIG. 18 shows an example of the initial state of the editing screen displayed on the editing monitor 213. The editing screen of this figure is an example of the screen displayed in the "main part of graffiti" (S230 of FIG. 10), and not only the editing process for the still image taken by the user but also the editing process of the moving image using the template moving image. It can be carried out. First, as a premise, ordinary graffiti on a captured still image will be described.

The editing screen is divided into a pair of left and right editing spaces so that two users can use them in parallel. In the editing space on the left side of the editing monitor 213, input can be accepted by the touch pen 214A on the left side. Similarly, in the editing space on the right side, input can be accepted by the touch pen 214B on the right side. In the upper center of the editing space, a rectangular writing area 810 that can display and edit the image to be edited is provided. Thumbnails 812 of five images to be edited are displayed above the draft area 810, and one image to be edited selected from the thumbnails 812 is displayed in the draft area 810. The five images to be edited here are based on three close-up images and five full-body images. More specifically, the three close-up images, the combined image of the three types of close-up background images included in the background image set selected in the preset settings, and the two full-body shot images are This corresponds to a composite image of two types of whole-body background images included in the background set. The edits made by the user in the writing area 810 are also reflected in the corresponding thumbnails 812. In the draft area 810 and the thumbnail 812, the edit target image in which the background image selected by the preset in the preset unit 40 is combined is initially displayed.

One image selected by the user from the thumbnails 812 of the five images to be edited is highlighted by being surrounded by the selection frame 812'in a thick frame, and the image to be edited corresponding to the thumbnail is displayed in the draft area 810. .. The user can change the edit target image to be displayed in the writing area 810 by selecting any one thumbnail of the thumbnail 812 of the edit target image using the touch pens 214A and 214B. The user can also change the image to be edited to be displayed in the draft area 810 by moving the selection frame 812'by a drag operation using the touch pens 214A and 214B. In this case, the image to be edited corresponding to the thumbnail 812 surrounded by the selected frame 812'after the movement is displayed in the writing area 810.

An "end" button 814 is arranged at the upper center of the edit screen. Further, in the center of the editing screen, an eraser button 816, a "forward" button 818, and a "back" button 820 are arranged from the top. The "end" button 814 is a button for terminating the editing, and when the "end" button 814 is pressed, the editing process is spontaneously terminated and the process proceeds to the output process. Four types of eraser buttons 816 are arranged, and three types of eraser buttons and an "eraser" button for erasing all written letters are arranged. For example, if one of the size eraser buttons is pressed with the touch pens 214A and 214B, the size eraser is set, and if the writing area 810 is traced with the touch pens 214A and 214B in this state, the writing of the traced portion is erased. It has become like. Further, when the "Erase all" button is pressed, all the drafts made for the image to be edited displayed in the draft area 810 are erased.

The "back" button 820 is a so-called undo button for canceling and restoring the writing process performed immediately before. On the contrary, the "forward" button 818 is a so-called redo button for restoring the writing process immediately before pressing the "back" button 820.

A timer 822 for counting down is provided between the "end" buttons 814 at the upper center of the editing screen. The timer 822 can be set in various ways. For example, a basic time limit for editing a draft (for example, 300 seconds) is set in advance, and a countdown is started at the same time as the editing of the draft is started, and the timer 822 is displayed. It can be configured to forcibly end the draft editing when it becomes zero and shift to the output processing. Alternatively, when there is no user in the shooting unit 50, that is, when there is no subsequent user waiting for the editing work, the editing work may be performed without a limit without counting down, or the maximum time limit (for example, 20 minutes). May be set so that editing work can be performed up to the maximum time limit. However, in such a state, when the shooting process by a new user is started while the other editorial unit 51 has a succeeding user, the setting is set in advance according to the start of the shooting process. It is preferable to start the countdown from the time (eg 60 seconds). The timer 822 is common to the left and right users, and the editing processing time of both users is managed by one timer 822.

Below the draft area 810, a palette area 828 for selecting an editing tool for editing the image to be edited is arranged. A tool for writing is displayed in the palette area 828. A pen pallet area 830 is arranged outside the writing area 810 and the pallet area 828. In the pen palette area 830, buttons used for selecting the line type, thickness, and color of the pen image used for handwriting input are displayed.

The user scribbles until the remaining time of the timer 822 reaches 0 (or the user voluntarily terminates). Then, when the remaining time of the timer becomes 0 or the user touches the "end" button 814 to voluntarily end the graffiti, the editing monitor 213 displays the message "end of writing". Then, at that time, the edit target image being edited on the edit screen is regarded as an output image (edited image), and this output image is stored in the data storage unit 150, and the editing process is completed. In this way, when the user voluntarily touches the "end" button 814, the editing of the image to be edited by the user is confirmed. In this sense, the above-mentioned writing area 810 and thumbnail 812 are previews of editing.

When the user on the right side selects the leftmost thumbnail of the five thumbnails 812 placed in the upper right with the touch pen 214B, the leftmost thumbnail is highlighted surrounded by a thick frame selection frame 812'and corresponds to that thumbnail. The image to be edited is enlarged and displayed in the writing area 810 on the right side. In addition, a character string such as "Enlarging" is superimposed on the thumbnail highlighted in the selection frame 812', indicating that the user on the right side is currently editing. The superimposed character strings may be combined with transparency so as to make the thumbnail transparent. At this time, of the five thumbnails 812 arranged at the top of the editing space on the left side, the leftmost thumbnail being edited by the user on the right side is superimposed with a character string such as "The person on the right is expanding". , Currently indicated by the user on the right that the editing process is in progress. When the user on the left selects the second thumbnail from the right with the touch pen 214A out of the five thumbnails 812 placed in the upper left, the thumbnail is highlighted surrounded by a thick selection frame 812'and the corresponding edit. The target image is enlarged and displayed in the writing area 810 on the left side. A character string such as "Enlarging" is superimposed on the thumbnail highlighted in the selection frame 812', indicating that the editing process is in progress. Of the five thumbnails 812 placed at the top of the editing space on the right side, the thumbnail being edited by the user on the left side is superimposed with a character string such as "The person on the left is expanding" and is currently edited by the user on the left side. It is shown that it is being processed.

At the top of the edit screen, a video editing start button 840, which is an icon image in which a character string such as "movie creation" is written between the thumbnail 812 and the "end" button 814, is displayed. When the user presses the video editing start button 840, the video editing process using the template video is started.

FIG. 19 shows an example of a state in which video editing is started on the editing screen. In the example of this figure, the left side user presses the left side moving image editing start button 840, and the state in which the moving image editing is started in the left side editing space is shown. In order to limit the video editing from being started in the other editing space at the same time while editing the video in one of the left and right editing spaces, the video editing start button 840 displayed in FIG. 18 is pressed at the start of video editing. Erase both the left and right sides. As a modification, the moving image editing start button 840 may be grayed out without being erased so that it cannot be pressed. As another modification, a specification may be provided in which only one video editing start button 840 is provided on only one of the left and right editing spaces so that video editing can be performed at the stage of FIG. As a modification, two left and right users may be able to process moving image editing at the same time in the left and right editing spaces. In that case, one moving image may be edited at the same time, or separate moving images may be created. Even when a single video is created, each of the two users' play IDs is assigned, and when separate videos are created, their own play ID is assigned to each of the videos created by them. While the user on the left side edits the moving image, the user on the right side can continue scribbling on the still image as in the state of FIG. If the right user presses the right video editing start button 840, video editing is started in the right editing space, while the left user can continue scribbling on the still image in the left editing space.

A vertically long template area 842 having the appearance of a movie film is displayed on the outside of the editing space on the left side, and characteristic frames included in the template moving image are vertically arranged and displayed. The template moving image displayed in the template area 842 is a template moving image having a design selected by the user in the preset setting of FIG. 12B. However, in the specification that allows the user to present and select multiple types of template videos as template video options, the template video with the design corresponding to the background image selected in the preset is automatically selected as the initial setting. May be good. Alternatively, only the template video of the design corresponding to the background image selected in the preset may be selected.

The movie film type template area 842 is configured to include eight frames of images arranged in the vertical direction, and a selection frame indicating that the selection is being made is displayed only for the frames to be edited in the editing space. In the example of FIG. 19, only the lower three frames (first edit target frame 843, second edit target frame 844, third edit target frame 845) of the eight frames are frames in which the selection frame can be displayed as the edit target. , The selection frame is not displayed in the upper 5 frames that are not edited. In the non-editing target frame, characteristic frames in the template moving image and frames included in the first to third shot movies that have been shot are displayed as thumbnails.

In the illustrated template video, three video parts can be edited, and in correspondence with these three editable parts, the first edit target frame 843, the second edit target frame 844, and the third edit target frame 845. Thumbnails are displayed at the bottom of the template area 842. The first edit target frame 843, the second edit target frame 844, and the third edit target frame 845 correspond to the first edit button 846, the second edit button 847, and the third edit button 848, respectively. When the user presses the first edit button 846, the first edit target frame 843 becomes the edit target and the selection frame is displayed, and when the second edit button 847 is pressed, the second edit target frame 844 becomes the edit target and the selection frame is displayed. Is displayed, and when the third edit button 848 is pressed, the third edit target frame 845 becomes the edit target and the selection frame is displayed.

Initially, the first edit target frame 843 is the edit target and the selection frame is displayed, and that the first edit target frame 843 is associated with the first edit button 846, that is, the first edit button 846. It is indicated by an arrow connected to the first edit target frame 843. Further, in the edited moving image display area 850 in the center of the editing space, the frame of the video portion corresponding to the first editing target frame 843 is displayed. Similarly, when the second edit button 847 is pressed, an arrow is displayed from the second edit button 847 to the second edit target frame 844, a selection frame is displayed in the second edit target frame 844, and the edited video display area 850 is displayed. Displays the frame of the video portion corresponding to the second edit target frame 844. When the third edit button 848 is pressed, an arrow is displayed from the third edit button 848 to the third edit target frame 845, a selection frame is displayed in the third edit target frame 845, and the third edit video display area 850 is filled with the third. The frame of the video portion corresponding to the edit target frame 845 is displayed.

In the example of FIG. 19, as a video portion corresponding to the first edit target frame 843 displayed in the edited video display area 850, a still image taken by the user may be embedded in a frame of a background video prepared as a template. The possible image frame 851 is displayed. Initially, the image frame 851 is displayed as a black-painted rectangular frame, and by displaying the camera mark icon in the rectangular frame, it is suggested that the captured image can be embedded in this portion. Below the edited moving image display area 850, thumbnails 854 for embedding a plurality of still images taken by the user are displayed as options for the still images to be embedded in the image frame 851 being displayed in the edited moving image display area 850. Here, three close-up shot images and two whole-body shot images, which are images to be edited in normal graffiti, are displayed, and when the user selects one of the images, the selected shot image is displayed. It is embedded in the image frame 851 and displayed. Depending on the template moving image, a plurality of image frames 851 may be provided in one video portion. In that case, the user selects as many captured images as the number of image frames 851 and specifies the embedding position.

Here, since the aspect ratio is different between the close-up image and the whole body image, when an image that does not match the aspect ratio of the image frame 851 is selected, the image frame 851 for each frame is selected with respect to the selected image. Processing such as deformation and trimming may be added according to the aspect ratio, shape, size, and the like. Alternatively, a process of changing the aspect ratio of the image frame 851 or changing the design of the image frame 851 itself may be executed according to the aspect ratio of the selected image. When trimming the selected image, the face may be detected from the image and the trimming position may be adjusted so that a part of the face is not deleted. As a modification, as a specification, only the thumbnail 854 for embedding the captured image that matches the aspect ratio of the image frame 851 is displayed as an option, and the thumbnail 854 for embedding the captured image that does not match the aspect ratio of the image frame 851 is not displayed. May be good. In that case, the screen configuration can be switched so that only the close-up shot image or only the whole-body shot image is displayed as an option, and the selected image can be embedded in the image frame 851 without any processing such as deformation or trimming. it can. As another modification, the selected image may be enlarged or reduced around the range of the image frame 851 and moved (panned or slid) so as to flow in a certain direction to be displayed. In this case, it is not necessary for the aspect ratio of the image frame 851 to match the aspect ratio of the image, and the entire image is displayed by displaying the selected image in a size slightly larger than the image frame 851 and shifting the image. be able to.

A moving image playback button 852 is displayed in the center directly below the edited moving image display area 850. When the video playback button 852 is pressed by the user, the video portion before and after the frame being displayed in the edited video display area 850 is played back and displayed as a preview. That is, the frame displayed in the edited moving image display area 850 is used as one editing point, and the 24 frames before and after the editing point are played back once as a preview moving image for 2 seconds at 12 fps to return to the original state. An image frame 851 is provided for each of the 24 frames, and the position, size, and shape of the image frame 851 may be slightly different depending on the frame. When the user selects any image from the embedding thumbnails 854, the image is embedded in all image frames 851 in all 24 frames. The frame displayed in the first edit target frame 843 is also reflected in real time, and the frame in which the selected captured image is embedded is displayed. If the shape, position, and size of each image frame 851 of the 24 frames are different, the selected captured image is deformed or scaled to fit each image frame 851 and embedded in each image frame 851. Is done. Since the preview video is only a part of the whole video created by the template video and contains only 24 frames in the video shorter than the whole video, it is composited while adjusting to fit the image frame 851 for each frame. Even so, the processing load is relatively small. The preview video is generated in the background in a form that reflects the edited content at that time in the video editing process, and even if the video playback button 852 is not pressed, the first edit target frame 843 and the second edit target frame 844 are displayed. It may be a specification that is played back in a loop.

Until any of the captured images is selected, the preview moving image is played back in the form in which the captured images are not combined in the image frame 851 and the initial image of the predetermined design is inserted. As a modification, once the moving image playback button 852 is pressed, the preview moving image may be repeatedly loop-played, and the moving image playback button 852 may be replaced with a stop button or a pause button during that time. In that case, when the user presses the stop button or the pause button, the loop playback is stopped and the original moving image playback button 852 is returned. As a modification, the video playback button 852 may not be displayed, and the preview video may be repeatedly loop-played in the edited video display area 850 without the user pressing such a button. Here, if the specification is such that the video playback button 852 is pressed and then replaced with the stop button, the edited video display area 850 when the stop button is pressed contains the frame at the initial editing point (the selected captured image is displayed). (Embedded frame) is displayed. On the other hand, if the specification is such that the video playback button 852 is pressed and then replaced with the pause button, the edited video display area 850 when the pause button is pressed contains the frame at the time of pressing (the selected captured image is displayed). (Embedded frame) is displayed. Regardless of whether the stop button or the pause button is displayed, the user can reselect the captured image after pressing the button. However, when reselecting the captured image at a frame other than the initial edit point, especially after pressing the pause button, the captured image is embedded with an aspect ratio, shape, position, and size different from the initial edit point. It may be displayed in the edited moving image display area 850.

On the side of the embedded thumbnail 854, a no-photo button 856 for selecting not to embed any captured image in the image frame 851 is displayed. As a setting of the template moving image, when the user does not specify the image to be embedded in the image frame 851, an initial image for embedding or replacing the image frame 851 is prepared. When the user presses the no photo button 856, or when the time limit elapses without the user selecting an image, or when the editing process is voluntarily terminated, the initial image is embedded in the image frame 851. Or it can be replaced.

The image of the embedded thumbnail 854 displayed as an option is an image obtained by synthesizing the background image and the captured image selected in advance by the user, similarly to the thumbnail 812. When a button image for erasing the background such as "Backgroundless button" is displayed as a modification and the user presses the "Backgroundless button", all of the multiple images displayed as the embedded thumbnail 854 are displayed. The background image is erased, that is, the background is replaced with the default background and recombined, or the background image and the captured image before chroma key compositing are replaced. As a modification, the background image may be deleted only for the captured image selected by the user instead of all the embedded thumbnails 854 being displayed. Also, after erasing the background image, replace the "without background button" with the "button with background" and press the button image to restore the background image, that is, the background image and captured image selected in the preset settings. May be executed to replace the image with a composite image. In this way, when the user feels that the background image does not match the design of the template video when the captured image combined with the background image is embedded in the image frame 851, the background can be erased according to the user's preference. it can.

FIG. 20 shows an example of a state in which a still image to be embedded in the template moving image is selected on the editing screen. When the user selects one of the embedding thumbnails 854, the selected image is immediately displayed as a preview in the frame of the image frame 851. By displaying a thick selection frame 854'for highlighting around the selected embedded thumbnail 854, it is clearly indicated that the image is being selected. When the user reselects another embedding thumbnail 854, the image embedded in the image frame 851 is immediately switched to the reselected image, and the position where the selection frame 854'is displayed is also the thumbnail selected by the user. Switch to. When the specification is such that the moving image is repeatedly loop-played in the edited moving image display area 850, the selection result of the embedded thumbnail 854 or the photoless button 856 may be reflected in the loop-playing video in real time. When graffiti is executed in the editing space on the opposite side, the result of the graffiti is reflected in the thumbnail 812 at the top of the editing space, but it is reflected in the thumbnail 854 for embedding and the image frame 851 below the editing video display area 850. It is not reflected in the embedded image. Embedding the process of graffiti, which is still being edited and undecided, in the video at the same time may complicate the editing process and the compositing process. However, as a modification, the content being scribbled by the other user may be immediately reflected in the embedded thumbnail 854 below the edited moving image display area 850 or the image embedded in the image frame 851.

Here, when the no-photo button 856 is pressed by the user, the selection frame 854'is erased, and the image frame 851 is embedded with an initial image for the case where the photographed image is not selected. As a modification, when the time limit elapses without the user selecting an image, or when the editing process is voluntarily completed, the captured image is randomly selected or the first captured image is automatically selected. It may be a specification to be embedded in the image frame 851, or a specification to embed the above-mentioned initial image without embedding the photographed image.

While editing a video in the edit space on the left, the user on the left does not change the screen and switch to scribbles on still images when selecting the thumbnail 812 located at the top of the edit space. .. Therefore, the thumbnail 812 placed at the top of the editing space during video editing may suggest that the thumbnails cannot be selected by reducing their brightness and saturation and displaying them as a dark image. ..

Below the embedded thumbnail 854, a save button 858 instructing the decision to retain the editing process up to that point and a cancel button 859 instructing the cancellation of the editing process up to that point are displayed. A character string such as "stop editing" is displayed on the cancel button 859. When the save button 858 is pressed by the user, the edited contents up to that point are saved. At this time, the thumbnail 812 is returned to a state in which the thumbnail 812 can be selected so that the process can be performed to scribble the captured image as a still image as shown in FIG. However, as a modification, the specification may be such that the screen does not return to the screen of FIG. 18 and remains in the state of FIG. 20, which is equivalent to pressing the second edit button 847 in order to efficiently transition to the next edit target. As a process, it may be a specification to switch to the stage of editing the video portion corresponding to the next second edit target frame 844. After the first user saves the edited content of the video and returns to the screen of FIG. 18, when the second user edits the video, the edit is based on the edited content saved by the first user. As a continuation, the second user will make further edits.

On the other hand, when the cancel button 859 is pressed by the user, the edited contents up to that point are discarded, and the edited contents are returned to the initial state or the saved state at the time when the save button 858 was pressed last time. As a modification, when the cancel button 859 is pressed by the user, a process of switching to a state similar to the editing space on the right side, that is, a state in which scribbling of a still image is performed may be further executed.

When the second edit button 847 is pressed by the user, the video portion to be edited corresponding to the second edit target frame 844 included in the template video is displayed in the edit video display area 850, and corresponds to the first edit target frame 843. Processes the selection and embedding of captured images in the same way as the video part to be edited. When the third edit button 848 is pressed by the user, the video portion to be edited corresponding to the third edit target frame 845 included in the template moving image is displayed in the edited moving image display area 850. However, in the template moving image illustrated here, the image frame 851 for embedding the captured image is not provided in the third editing target frame 845, and a video for accepting graffiti by the user is prepared.

FIG. 21 shows an example of performing graffiti on a template moving image on an editing screen. The third edit button 848 is pressed by the user and the third edit button 848 is highlighted, and the third edit button 848 is connected to the corresponding third edit target frame 845 by an arrow. In the edited moving image display area 850, a frame of the image corresponding to the third editing target frame 845 is displayed. The video edited here is a video designed on the assumption that stamp images and characters will be inserted as a message to the recipient of the edited video, or handwritten characters and pictures that the user inputs with the touch pen will be inserted. Is. For example, it is a video designed with a letter as a motif, and the user can directly apply graffiti to the video of the template video.

Since the design differs depending on the template video selected by the user, the types of stamp images, character strings, etc. that can be input are limited so as to match the design of the template video. The drafting tool corresponding to the template video selected by the user is displayed in the palette area 828, and the user can edit the template video according to the design by using the drafting tool. The user can select a stamp image from the displayed writing tool and simply touch the position where the stamp image is to be pasted in the edited video display area 850 with the touch pen, and the stamp image can be synthesized at that position. ..

Here, the position or range that can be scribbled is limited to a certain range according to the tool selected by the user. For example, while the user is selecting the pen, the range that can be input with the pen is limited to a certain range on the screen, for example, the ruled line part 861 (letter message field) in the illustration of the letter, so that the design of other parts can be designed. Can be harmonized with. On the contrary, when the user selects the stamp image, the range in which the stamp image can be combined is limited to a certain range on the screen, for example, the design part 860 other than the message field of the letter, so that it overlaps with the text of the message. Can be avoided. On the other hand, if no characters are entered in the message field of the letter, the basic message or image prepared in advance is automatically inserted as the initial setting, and the design looks bad like a blank letter. Try not to be. For example, if the design is based on the assumption that a person's name will be inserted after the letter or message field, such as "HAPPY BIRTHDAY TO", which of the two names the user has entered in advance It may be a specification to select and input. When the third edit button 848 is pressed, the pen palette area 830 may be displayed instead of the template area 842 in the area where the template area 842 has been displayed so far.

A video playback button 852 is displayed in the center directly below the edited video display area 850, and when the video playback button 852 is pressed by the user, a preview video in which the graffiti input by the user is synthesized is displayed once in the edited video display area 850. Only play is displayed and returns to the original. The preview video in which the graffiti is combined is composed of 24 frames as in the preview video when the captured image is embedded in FIGS. 19 and 20. The design included in the 24 frames may be slightly different for each frame, and the position where the graffiti such as stamp image and pen input is combined and the size, shape, color, etc. of the graffiti are set so as to match the design for each frame. You may adjust. For example, the position and shape of the graffiti may be changed for each frame so that the contents of the graffiti appear to rotate in the preview video, or the contents of the graffiti may be deformed so as to appear as a three-dimensional shape. By playing back 24 frames including the frame in which the graffiti is synthesized at 12 fps, a preview movie for 2 seconds is obtained. By adjusting and synthesizing graffiti for each frame for a plurality of frames, it is possible to generate a moving graffiti movie. Also, the preview video is only a part of the whole video created by the template video, and it is shorter than the whole video and contains only 24 frames, so it is synthesized while adjusting the graffiti for each frame. However, the processing load is relatively small. The preview video is generated in the background in a form that reflects the edited content at that time in the process of scribbling, and even if the video playback button 852 is not pressed, the preview video is played in a loop at the third edit target frame 845. Good.

If the specification is such that the video playback button 852 is pressed and then replaced with the stop button, the frame (frame reflecting graffiti) at the initial editing point is displayed in the edited video display area 850 when the stop button is pressed. It is also possible for the user to re-edit the graffiti. On the other hand, if the specification is such that the video playback button 852 is pressed and then replaced with the pause button, the edited video display area 850 when the pause button is pressed is the frame at the time of pressing (the frame in which the graffiti is reflected). ) Is displayed, and the re-editing of graffiti at the time of the pause is restricted. This is because at the time of pausing, the graffiti may be deformed and displayed at a position, size, shape, color, etc. different from the initial editing point, and it is difficult to re-edit in that state.

As described above, each video portion can be edited by selecting any of the first edit button 846, the second edit button 847, and the third edit button 848. While editing any of the video parts, select any of the first edit button 846, the second edit button 847, and the third edit button 848 at any time until the "end" button 814 is pressed to finish editing. You can fix it and switch to editing other video parts. That is, the editing screens of each video portion can be exchanged with each other, and the editing results can be checked while switching. In that respect, it is different from the step-up method in which the first edit button 846, the second edit button 847, and the third edit button 848 are edited one step at a time and confirmed in order. Further, even when the process transitions to the scribble processing on the still image as shown in FIG. 18 after pressing the save button 858, the continuation of the moving image editing can be resumed by pressing the moving image editing start button 840 again.

For the moving image embedded in the template moving image, the foreground image may be superimposed on the layer on the front side of the moving image depending on the design of the template moving image. The user may be urged to adjust the standing position by displaying such a foreground in advance on the front monitor 113 at the time of shooting or in the model moving image so that the design and the moving image are well integrated in terms of design.

When the "end" button 814 is pressed and the editing process is completed, the information indicating the template moving image, the editing content instructed by the user in the editing process, and the captured still image and moving image are transmitted to the management server 300. .. The management server 300 embeds captured still images and moving images in the specified template video based on the edited contents instructed by the user and the editing rules set for each template video, and causes the user to download the video. Create a video for. However, as a modification, a specification may be made in which a dedicated information presentation application installed on the user's mobile terminal creates a moving image instead of the management server 300 side. In that case, the dedicated information presentation application acquires the template video specification, the edited content instructed by the user, the captured still image and the moving image from the management server 300, and the still image and the video captured based on them. Create a video by embedding the image in a template video. In the case of the specification of creating a moving image on the application side, the load of the moving image creation process can be distributed to a plurality of terminals as compared with the case of creating a moving image on the server side. That is, when the management server 300 is created, the processing is concentrated on the server, and if a problem occurs in the server processing, it may affect all users. On the other hand, when it is created on the application side, even if a problem occurs in the processing in the application, it affects only that user and does not affect other users, which is advantageous. Also, if a new model of amusement photography device is released, it is enough to update the application.

The photography system 600 according to the embodiment of the present invention has been described above. The present invention is not limited to the above-described embodiment, and various modifications can be made without departing from the spirit of the present invention.

10 shooting space, 100 shooting equipment for games, 132 imaging unit, 133 image acquisition unit, 144 video processing unit, 160 output processing unit.

Claims (4)

A shooting unit that shoots the user multiple times in succession and acquires multiple shooting moving images,
A model moving image for showing the user a model of the subject movement during shooting is displayed on the screen prior to the shooting, the display of the model moving image is ended and hidden, and the image is captured by the shooting unit. A display control unit that displays the user's image in real time and notifies the shooting start timing, which is a predetermined time after the start of the real-time display, in parallel with the real-time display.
A moving image processing unit that generates output moving image data so that the obtained plurality of captured moving images are displayed in the frames of a plurality of image frames defined at predetermined positions of a plurality of frame images included in the template moving image. When,
With
In the template moving image, at least one of the position, size, and shape of the image frame is defined in at least one of a plurality of frame images included.
The photographing unit and the display control unit acquire the plurality of photographed moving images by repeating the process from the display of the model moving image to the photographing.
The model moving image is a game shooting device characterized in that a model of a different subject movement is displayed before shooting as a content each time the shooting is performed, and is hidden during shooting. A shooting unit that shoots the user multiple times in succession and acquires multiple shooting moving images,
A model moving image for showing the user a model of the subject movement during shooting is displayed on the screen prior to the shooting, the display of the model moving image is ended and hidden, and the image is captured by the shooting unit. A display control unit that displays the user's image in real time and notifies the shooting start timing, which is a predetermined time after the start of the real-time display, in parallel with the real-time display.
A moving image processing unit that generates output moving image data so that the obtained plurality of captured moving images are displayed in the frames of a plurality of image frames defined at predetermined positions of a plurality of frame images included in the template moving image. When,
With
In the template moving image, at least one of the position, size, and shape of the image frame is defined in at least one of a plurality of frame images included.
The photographing unit and the display control unit acquire the plurality of photographed moving images by repeating the process from the display of the model moving image to the photographing.
The model moving image is an image processing device characterized in that a model of a different subject movement is displayed before shooting as a content each time the shooting is performed, and is hidden during shooting. A step of displaying a model video on the screen prior to the shooting to show the user a model of the subject movement during movie shooting, and
A step of ending the display of the model moving image to hide it and displaying the user's image captured by a predetermined shooting means in real time.
A step of notifying the shooting start timing, which is a predetermined time after the start of the real-time display, in parallel with the real-time display,
The step of shooting the user and acquiring the shooting video,
Output video so that the plurality of acquired moving images are displayed in the frame of the image frame in which at least one of the positions, sizes, and shapes of the plurality of frame images included in the template video is defined. Steps to generate image data and
With
By repeating the steps from the display step to the acquisition step a plurality of times in succession, a plurality of captured moving images are acquired, and a model moving image containing a model of a different subject movement for each display step is produced. An image processing method characterized in that the model moving image is hidden in the step of displaying and acquiring. A function to shoot the user multiple times in succession and acquire multiple shot moving images,
It is a function to display an image on the screen, and a model video for showing the user a model of the subject movement during shooting is displayed on the screen prior to the shooting, and the display of the model video is finished and is not displayed. A function of displaying the user's image captured by the imaging means in real time and notifying the shooting start timing, which is a predetermined time after the start of the real-time display, in parallel with the real-time display.
A function to generate output video data so that the plurality of acquired moving images are displayed in the frames of a plurality of image frames defined at predetermined positions of a plurality of frame images included in the template moving image.
To the computer,
In the template moving image, at least one of the position, size, and shape of the image frame is defined in at least one of a plurality of frame images included.
The acquisition function and the display function acquire the plurality of shooting moving images by repeating the process from the display of the model moving image to the shooting.
The model moving image is a computer program characterized in that a model of a different subject movement is displayed before shooting as a content each time the shooting is performed, and is hidden during shooting.

Images