JP6849851B2 - Shooting system, shooting device and user shooting method - Google Patents

Description

The present invention relates to a photographing system, a photographing apparatus, and a user photographing method.

Conventionally, there is an automatic photo creation device for a game in which a subject including a user is photographed with a camera and a composite image obtained by combining the photographed image and an additional image is output as a print photograph.

However, in the above-mentioned automatic photo creation device, the device maker separately creates an image source as an additional image and stores it in the device in advance, and the user uses it for editing work. Therefore, the user has no choice but to use the additional image prepared in advance by the device maker, and cannot obtain a sufficient sense of satisfaction.

Therefore, an automatic photo creation device has been invented that can register a photographed image of a preceding subject and an edited image thereof as a new additional image for adding to a photographed image of a user as a succeeding subject (Patent Document 1). ). The preceding subject is a person having added value to whom the right of publicity or the right of commercialization can be recognized, for example, a talent. As a result, it is possible to take pictures alongside talents, etc., and it is said that a feeling of satisfaction that has never existed before can be obtained.

Japanese Unexamined Patent Publication No. 2008-005401

By the way, the additional image of the automatic photo creation device is a still image, and an object of the automatic photo creation device is to provide a photo in which a user, a talent, or the like are arranged side by side.

However, the above-mentioned automatic photo creation device does not allow the user to experience sharing the same space as a talent or the like, and merely provides a composite photo, which does not give the user a sufficient sense of satisfaction. There wasn't.

Therefore, the present invention provides a photographing system, an imaging device, and a user photographing method in which a user can obtain an experience of sharing the same space as a performer such as a talent, and the user can obtain a recorded image of the experience. To provide.

One aspect of the present invention includes a management device and at least one or more photographing devices, and the management device stores recorded images transmitted from the photographing devices, and is information indicating a storage location of a user. The imaging device includes a storage unit that transmits storage location information that can be read by a terminal to the photographing device, and the photographing device includes a camera for photographing a user, a display unit, augmented reality video data that is a moving image of a performer, and the like. A storage unit that stores the content data including the guidance by the performer of the extended reality video data and the audio data including the audio of the shooting start timing of the still image, and the moving image of the performer of the extended reality video data of the content data. , The combined image is generated by synthesizing the image taken by the user taken by the camera, the combined image is displayed on the display unit, and the still image of the combined image is displayed in accordance with the shooting start timing by the voice of the performer. The video processing unit to be recorded, the audio output unit that outputs the voice of the performer in synchronization with the composite video displayed on the display unit using the audio data of the content data, and the still image as the recorded image. An imaging system including a transmission unit that transmits data to the management device and a storage location presentation unit that receives the storage location information and presents the storage location information.

One aspect of the present invention is a camera for photographing a user, a display unit, augmented reality video data which is a moving image of a performer, guidance by the performer of the augmented reality video data, and audio of a still image shooting start timing. A composite video is generated by synthesizing a storage unit that stores content data including audio data, a video of a performer of the extended reality video data of the content data, and a video shot by a user shot by the camera. The display unit uses the video processing unit that displays the composite video on the display unit and records the still image of the composite video in accordance with the shooting start timing by the voice of the performer, and the audio data of the content data. Information indicating an audio output unit that outputs the voice of the performer in synchronization with the composite video displayed on the screen, a transmission unit that transmits the still image as a recorded image to the management device, and a storage location of the recorded image. An imaging device including a storage location presentation unit that receives storage location information that can be read by a user's terminal from the management device and presents the storage location information.

One aspect of the present invention stores content data including extended reality video data which is a moving image of a performer and audio data including audio of guidance by the performer of the extended reality video data and shooting start timing of a still image. It is a user shooting method of a computer provided with a storage unit, and the computer synthesizes a video shot by the user shot by the camera and a video of the performer of the extended reality video data to generate a composite video, and the composite video is generated. The video is displayed on the display unit, the computer uses the audio data of the content data, and outputs the voice of the performer in synchronization with the composite video displayed on the display unit, and the computer uses the voice of the performer. A still image of the composite image is recorded according to a shooting start timing, the computer transmits the still image as a recorded image to a management device, and the computer is information indicating a storage location of the recorded image, which is information of a user. This is a user photographing method in which storage location information readable by a terminal is received from the management device and the storage location information is presented.

According to the present invention, the user can obtain an experience in which the user shares the same space as the performer, and the user can obtain a recorded image of the experience.

FIG. 1 is a diagram showing an example of the overall configuration of the photographing system according to the present embodiment. FIG. 2 is a block diagram of the photographing apparatus 1. FIG. 3 is a diagram for explaining the compositing process of the composite video performed by the video processing unit 154. FIG. 4 is a diagram showing an example of a content database. FIG. 5 is a diagram showing an example of setting data. FIG. 6 is a diagram showing the appearance of the photographing apparatus 1. FIG. 7 is a block diagram of the management server 2. FIG. 8 is an example of a photographing apparatus management database. FIG. 9 is a block diagram of the payment server 3. FIG. 10 is a sequence diagram between devices. FIG. 11 is an operation flowchart of the setting process (Step 1) of the photographing device 1. FIG. 12 is an operation flowchart of the management process (Step 2) of the management server 2. FIG. 13 is an operation flowchart of the shooting process (Step 3). FIG. 14 is an example of a content selection screen displayed on the operation input unit 13. FIG. 15 is an example of a usage fee screen displayed on the operation input unit 13. FIG. 16 is an example of a screen on which a two-dimensional code indicating the address of the storage location is displayed. FIG. 17 is an example of a case where the recorded image downloaded to the terminal 4 is displayed on the display of the terminal 4. FIG. 18 is a diagram showing an example of the overall configuration of the photographing system according to the second embodiment. FIG. 19 is a block diagram of the performer photographing device 5. FIG. 20 is a block diagram of the management server 2 of the second embodiment. FIG. 21 is an example of a photographing apparatus management database according to the second embodiment.

<First Embodiment>
[overall structure]
FIG. 1 is a diagram showing an example of the overall configuration of the photographing system according to the present embodiment. As shown in FIG. 1, the photographing system includes a photographing device 1, a management server 2, a payment server 3, and a terminal 4. The photographing device 1, the management server 2, the payment server 3, and the terminal 4 can be connected to the communication line N and can communicate with each other.

The communication line N means a communication path capable of data communication. That is, the communication line N includes a telephone communication network, a cable network, a communication network such as the Internet, as well as a LAN by a dedicated line (dedicated cable) for direct connection and Ethernet (registered trademark), and the communication method is wired. / Regardless of wireless.

The photographing device 1 is installed in a facility such as an event venue or a movie theater, and is a device that generates a composite image obtained by synthesizing a photographed image of a user (subject) and a content image. The content video is an augmented reality video (for example, a video) of an entertainer (cast of drama / movie, idol, etc.), an athlete, etc. (hereinafter, these are collectively referred to as a performer). The photographing device 1 also has a digital signage function. In addition, there are basically a plurality of photographing devices 1, and they are installed in facilities such as event venues and movie theaters.

The management server 2 is a server that manages content images and advertisements used by the photographing device 1 and stores recorded images provided to the user (subject).

The payment server 3 is a server that pays the usage fee of the photographing device 1.

The terminal 4 is a terminal owned by the user (subject), and can be connected to the communication line N via a wireless communication base station or the like to perform data communication with the management server 2. The terminal 4 is, for example, a smartphone, a mobile phone, a personal computer, a tablet computer, or the like. Basically, there are a plurality of terminals 4, and each user (subject) operates them. The terminal 4 has at least a function of receiving image data and storing or displaying the image data, a shooting function, and the like.

[Configuration of each device]
The configuration of the photographing device 1 will be described. FIG. 2 is a block diagram of the photographing apparatus 1.

In FIG. 2, 10 is a camera, 11 is a display, 12 is a speaker, 13 is an operation input unit, 14 is a short-range wireless communication unit, 15 is a processing unit, and 16 is a transmission / reception unit. It is a part, and 17 is a storage part.

The camera 10 is a camera that captures a user (subject) and a background of a space in which the user (subject) exists. An image (including both a still image and a moving image) captured by the camera 10 is hereinafter referred to as a captured image. This captured image is input to the processing unit 15.

The display 11 displays a video in which the shot video shot by the camera 10 and the content video are combined.

The speaker 12 outputs the sound of the content.

The operation unit input unit 13 is for the user (subject) to input various operations related to shooting, and outputs an operation input signal corresponding to the operation input to the processing unit 15. The functions of the operation input unit 13 include not only elements operated by the user such as a keyboard, a mouse, a touch operation pad, a home button, a button switch, a joystick, and a trackball, but also an acceleration sensor, an angular speed sensor, an inclination sensor, and geomagnetism. It can also be realized by an element such as a sensor that detects movement or posture. The operation unit input unit 13 of the present embodiment is a touch operation panel including a display on which operation information is displayed.

The short-range wireless communication unit 14 is a reader / writer for a contactless IC card used for payment of usage fees.

The processing unit 15 comprehensively controls the operation of the photographing device 1 based on the program and content data stored in the storage unit 17, the operation input signal from the operation input unit 13, and the like. The function of the processing unit 15 can be realized by, for example, a microprocessor such as a CPU or GPU, or an electronic component such as an ASIC or an IC memory. The processing unit 15 presents the management unit 151, the operation control unit 152, the payment processing unit 153, the video processing unit 154, the signage control unit 155, the voice control unit 156, and the storage location as the main functional units. A unit 157 is provided.

The management unit 151 sets the photographing device 1 and manages the data. The setting of the photographing device 1 is performed by the administrator after authentication by the account and password given to the photographing device 1. As a typical setting of the photographing device 1, there is a setting of contents used in the photographing device 1. The content is downloaded from the management server 2 and stored in the storage unit 17. There are two types of content download: an automatic update mode in which the content is automatically downloaded, and a manual update mode in which the administrator selects and downloads the desired content. When the administrator sets either mode, the content is downloaded and the content is stored in the photographing device 1.

The content includes a content image synthesized with a photographed image of a user (subject) and a signage image (advertisement image) used for digital signage. The content video is a video of a performer such as an entertainer (cast of a drama / movie, an idol, etc.) or an athlete, and is video data for augmented reality. The content video is basically a video. Further, the signage video is an advertisement video, and includes both a moving image and a still image. In addition, the content data including the content video and the metadata (identification information, usage fee, etc.) related to the video is described. Similarly, the signage video and the metadata (identification information, etc.) related to the video are described as signage data.

The operation control unit 152 displays the content (for example, thumbnail video) and operation information on the operation unit input unit 13. In addition, the operation signal from the operation unit input unit 13 is received.

The payment processing unit 153 receives the payment information of the contactless IC card read by the short-range wireless communication unit 14, and transmits the payment information to the payment server 3. When the payment is completed, the payment completion information is received from the payment server 3.

The image processing unit 154 inputs the captured image of the user (subject) from the camera 10 and generates a composite image in which the content image of the content selected by the user (subject) is superimposed on the captured image.

The generation of the composite video performed by the video processing unit 154 will be described. In order to generate a composite video, three types of video signals, a captured video, a fill signal, and a key signal, are required. The Fill signal is a content image of the performer of the content selected by the user (subject). The key signal is a signal that specifies an area on which the content video, which is a fill signal, is superimposed. In the present embodiment, the fill signal and the key signal are augmented reality video data.

FIG. 3 is a diagram for explaining the compositing process of the composite video performed by the video processing unit 154. The compositing process generates a compositing image by fitting the image of the performer of the fill signal into the area designated based on the key signal in the entire area of the captured image. It should be noted that FIG. 3 is a diagram for explaining the compositing process of a predetermined frame. In reality, since the captured video and the content video are moving images, the compositing video is generated in frame units.

If the positional relationship between the user (subject) and the performer in the shooting space is known, the part where the superimposed fill signal (performer's image) and the user (subject) area overlap is the one located in front. Is preferentially synthesized. In this way, it is possible to produce a more natural image of the virtual space. As for the positional relationship between the user (subject) and the performer, conventional methods such as infrared measurement and video image analysis can be used.

The video processing unit 154 also has a function of recording a video at a certain time in the composite video (moving image) as a still image. The recorded recorded image is saved in a storage location of the management server 2 described later. The recorded image is not limited to a still image, and may be a moving image of all or a part of a composite video (moving image).

The signage control unit 155 uses the signage data to display an advertisement on the display 11 when the photographing device 1 is not used for photographing.

The voice control unit 156 uses the voice data included in the content data to output the voice of the performer on the video, the voice of the usage guide, and the like from the speaker 12 when shooting the shot video of the user (subject).

The storage location presentation unit 157 notifies the user (subject) of the storage location of the recorded image described above. As a notification method, there is a method of displaying a two-dimensional code (QR code (registered trademark) or the like) indicating the address of the storage location of the recorded image on the display 11. However, the present invention is not limited to this, and the address of the storage location may be displayed directly on the display 11, or another method may be used.

The transmission / reception unit 16 transmits / receives data to / from the management server 2 and the payment server 3 via the communication line N.

The storage unit 17 stores various data required for the photographing device 1. For example, a content database, setting data, and the like.

FIG. 4 is a diagram showing an example of a content database. The database shown in FIG. 4 stores the content data of the performer used for shooting with the user and the signage data used as an advertisement. The content data includes content identification information (ID), a content name, video data for augmented reality (including a fill signal and a key signal), audio data, and metadata. The signage data includes signage identification information (ID), an advertisement name, video data, audio data, and metadata. Note that FIG. 4 is an example, and the present invention is not limited to this.

FIG. 5 is a diagram showing an example of setting data. The setting data shown in FIG. 5 includes an account of the photographing device 1, a management password, the presence / absence of automatic update, an advertisement start time (elapsed time since the photographing device 1 was last used), and auxiliary data. Note that FIG. 5 is an example, and the present invention is not limited to this.

The appearance of the above-mentioned photographing apparatus 1 is shown in FIG.

Next, the configuration of the management server 2 will be described. FIG. 7 is a block diagram of the management server 2.

In FIG. 7, 20 is an input unit, 21 is a transmission / reception unit, 22 is a processing unit, and 23 is a storage unit.

The input unit 20 is for the administrator to input various operations, and outputs an operation input signal corresponding to the operation input to the processing unit 22. The functions of the input unit 20 include not only elements operated by the user such as a keyboard, mouse, touch operation pad, home button, button switch, joystick, and trackball, but also an acceleration sensor, an angular velocity sensor, an inclination sensor, and a geomagnetic sensor. It can also be realized by an element that detects movement or posture.

The transmission / reception unit 21 transmits / receives data to / from the photographing device 1 via the communication line N.

The processing unit 22 comprehensively controls the operation of the management server 2 based on the program and content data stored in the storage unit 23, the operation input signal from the input unit 20, and the like. The function of the processing unit 15 can be realized by, for example, a microprocessor such as a CPU or GPU, or an electronic component such as an ASIC or an IC memory. The processing unit 22 includes a content management unit 121 and a shooting data storage management unit 122 as main functional units.

The content management unit 121 manages the content data set in the photographing device 1. Specifically, the content used in the photographing device 1 is managed by using the photographing device management database stored in the storage unit 23. FIG. 8 is an example of a photographing apparatus management database. The photographing device management database shown in FIG. 8 records the identification information of the photographing device, the account, the password, the presence / absence of automatic update, and the identification information of the available contents for each photographing device 1.

The shooting data storage management unit 122 saves the recorded image shot by the shooting device 1 in the storage 23, and notifies the shooting device 1 of the address of the storage location.

Next, the configuration of the payment server 3 will be described. FIG. 9 is a block diagram of the payment server 3.

In FIG. 9, 30 is a transmission / reception unit, 31 is a payment processing unit, and 32 is a storage unit.

The transmission / reception unit 30 transmits / receives data to / from the photographing device 1 via the communication line N.

The payment processing unit 31 receives payment information from the photographing device 1, performs payment processing based on the program and payment data stored in the storage unit 32, and transmits the payment completion information to the photographing device 1. The payment processing unit 31 can be realized by, for example, a microprocessor such as a CPU or GPU, or an electronic component such as an ASIC or an IC memory. It should be noted that a conventional method can be used for payment of the contactless IC card fee.

[Operation of each device]
The operation of the first embodiment will be described. FIG. 10 is a sequence diagram between devices.

First, a setting process is performed in order to make the photographing device 1 available (Step 1). Then, the management server 2 performs management processing in response to the setting request from the photographing device 1 and transmits the content data and the signage data (Step 2).

Here, the operation of the setting process (Step 1) of the photographing apparatus 1 will be described with reference to a flowchart. FIG. 11 is an operation flowchart of the setting process (Step 1) of the photographing device 1.

First, the administrator of the photographing device 1 operates the operation input unit 13 to display the management screen. The operation control unit 152 displays the management screen on the operation input unit 13 (Step 100).

The administrator operates the operation input unit 13 to input the identification information (account) and the password (Step 101).

The management unit 151 collates the input identification information (account) and password (Step 102), and if the authentication is successful (Step 103), transitions to the selection of the update mode. On the other hand, if authentication is not possible, the setting process ends.

If the authentication is successful, the management screen displays the automatic update mode and the manual update mode. The administrator selects one of the modes.

When the administrator selects the automatic update mode (Step104), the content data and signage data available in the photographing device 1 are transmitted from the management server 2, and the management unit 151 stores the content data and the signage data. It is stored in the content database of Part 17, and the content setting is completed. Then, the setting completion notification is transmitted to the management server 2.

On the other hand, when the administrator selects the manual update mode (Step104), the management server 2 transmits a list of content data and signage data available in the photographing device 1, and the management unit 151 sends the content data and signage data. Display list information (Step 105). The administrator operates the operation input unit 13 and selects the content data and signage data to be used from the list information (Step 106). Then, the management unit 151 downloads the selected content data and the signage data from the management server 2 (Step 107), stores the selected content data and the signage data in the content database of the storage unit 17, and completes the content setting (Step 108). Then, the setting completion notification is transmitted to the management server 2.

Next, the operation of the management process (Step 2) of the management server 2 will be described with reference to a flowchart. FIG. 12 is an operation flowchart of the management process (Step 2) of the management server 2.

The management server 2 responds to the setting request from the photographing device 1 and collates the identification information (account) (Step 200). If the identification information (account) is registered (Step 201), the process corresponding to the mode request from the photographing device 1 is performed.

When the mode requested by the photographing device 1 is the manual update mode (Step202), the content described in the available content field of the account of the photographing device 1 requested to be set in the photographing device management database of the storage unit 23. A list of content data and signage data corresponding to the ID or signage ID is transmitted (Step 203). When the selection information of the content data and the signage data is received from the photographing device 1, the selected content data and the signage data are transmitted to the photographing device 1 (Step 204). Then, when the setting completion notification is received from the photographing device 1, the management process is terminated (Step 205).

On the other hand, when the mode requested by the photographing device 1 is the automatic update mode, the flag of the field of whether or not the account of the photographing device 1 requested to be set is automatically updated is set to "ON" in the photographing device management database of the storage unit 23. (Step 206). Then, among the contents registered in the content database, the content data and the signage data corresponding to the content ID or the signage ID described in the available content field of the account of the photographing device 1 for which the setting is requested are searched (Step 207). ), The available content data and signage data are transmitted (Step 208). Then, when the setting completion notification is received from the photographing device 1, the management process is terminated (Step 205).

For the shooting device 1 of the account for which automatic update is enabled, when the available content is changed, the content data or signage data corresponding to the ID described in the available content field is automatically converted. Is transmitted to the photographing device 1 to automatically update the contents of the photographing device 1.

Returning to the description of the sequence diagram of FIG.

When the setting of the photographing device 1 is completed, the user (subject) can use the photographing device 1. Then, when the user (subject) starts using the photographing device 1, the photographing device 1 executes the photographing process (Step 3).

Here, the operation of the shooting process (Step 3) will be described with reference to a flowchart. FIG. 13 is an operation flowchart of the shooting process (Step 3).

The operation control unit 152 of the photographing device 1 displays a list of contents on the operation input unit 13 (Step 300). FIG. 14 is an example of a content selection screen displayed on the operation input unit 13. In the example of FIG. 14, it is an example of a screen in which three types of contents, content AAA, content BBB, and content CCC, can be selected.

The user who uses the photographing device 1 selects a desired content from the list of contents displayed on the operation input unit 13. The selection is made by touching the desired content from the content list (Step 301).

When the content is selected, the content usage fee screen is displayed on the screen of the operation input unit 13 (Step 302). FIG. 15 is an example of a usage fee screen displayed on the operation input unit 13. In the example of FIG. 15, the usage fee of the content AAA is 600 yen, and it is an example of a screen requesting that the IC card be touched by the short-range wireless communication unit 14.

When the user touches the short-range wireless communication unit 14 with the IC card according to the instruction on the usage charge screen, the payment processing unit 153 receives the payment information of the charge (Step 303) and transmits the payment information to the payment server 3 (Step 303). Step 304).

The payment processing unit 153 of the photographing device 1 receives the payment completion information from the payment server 3 (Step 305), displays the completion of the payment on the operation input unit 13, and notifies that the shooting is started (Step 306). ..

When the camera starts shooting the user (subject), the shot image is displayed on the display 11. Subsequently, the video processing unit 154 superimposes the video data for virtual reality of the selected content on the captured video to generate a composite video (Step 307). Then, the video processing unit 154 displays the composite video (moving image) on the display 11 (Step 308). At this time, the voice of the performer is also output from the speaker according to the voice data of the content. For example, a voice such as "AAA. Thank you for participating in the photo session today." Is output. In this way, the display 11 displays an image in which the performer and the user coexist in the same space (see the composite image in FIG. 3), and the sound can be heard, so that the user shares the same space with the performer. You can get a simulated experience like that. Then, the performer announces the timing of commemorative photography, and the recorded image (still image) is photographed (Step 309).

When the shooting is completed, the storage location presentation unit 157 uploads the still image to the management server (Step 310), and displays the address of the storage location on the screen of the operation input unit 13 using the two-dimensional code (Step 311). ). FIG. 16 is an example of a screen on which a two-dimensional code indicating the address of the storage location is displayed.

This is the end of the description of the shooting process.

Returning to the description of the sequence diagram of FIG.

Step 4 is a payment process performed by the payment server 3. The payment information of the user's IC card transmitted from the photographing device 1 is received, and the payment completion information is transmitted to the photographing device 1 based on the payment information.

Step 5 is a storage process performed by the management server 2. The recorded image (still image) transmitted from the photographing device 1 is saved, and the address of the storage location is transmitted to the photographing device 1.

The user uses the photographing function of the terminal 1 to photograph the two-dimensional code displayed on the screen of the operation input unit 13 and access the address indicated by the two-dimensional code (Step 6). Then, the terminal 1 downloads the captured recorded image (still image) to the terminal 1 (Step 7). FIG. 17 is an example of a case where the recorded image downloaded to the terminal 4 is displayed on the display of the terminal 4.

On the other hand, the signage control unit 155 of the photographing device 1 selects video data and audio data from the signage data of the content database of the storage unit 17 when the user does not use the photographing device 1 for a certain period of time (for example, 3 minutes). The image is acquired, an advertisement based on video data is displayed on the display 11, and audio based on audio data is output from the speaker 12 (Step 8).

This is the end of the description of the operation of the first embodiment.

In the first embodiment, the image (video) of the performer and the image taken by the user are combined and displayed on the display so that the sound can be heard, so that the user shares the same space with the performer. You can get an experience like that. In addition, the recorded image recorded as a memorial of the experience is saved in the management server and can be downloaded to the terminal by the user. Therefore, for example, it is possible to improve the convenience of sharing by SNS or the like and physically. Costs can be reduced because no printing is performed.

Further, in the first embodiment, since the content used in the photographing device is collectively managed by the management server, the immediacy of setting the content used in the photographing device and the flexibility of registration are realized. Can be done.

Further, since the photographing apparatus of the first embodiment is also equipped with a digital signage function, it is possible to effectively utilize the time when the photographing apparatus is not used.

<Second embodiment>
A second embodiment will be described. FIG. 18 is a diagram showing an example of the overall configuration of the photographing system according to the second embodiment.

In the second embodiment, in addition to the first embodiment, the live image of the performer is photographed, and the content data (fill signal, key signal, and audio data) used in the photographing device 1 from the live image is taken. A performer photographing device 5 for generating the above is provided.

FIG. 19 is a block diagram of the performer photographing device 5. In FIG. 19, 50 cameras, 51 is a microphone, 52 is a display, 53 is a transmission / reception unit, 54 is a processing unit, and 55 is a storage unit.

The camera 50 is a camera that captures a live image of the performer. The image taken by the camera 50 is referred to as a live image. This live video is input to the processing unit 54.

The microphone 51 acquires the voice of the performer and outputs the acquired voice to the processing unit 54.

The display 52 displays a live image taken by the camera 50.

The transmission / reception unit 53 transmits / receives data to / from the management server 2 via the communication line N.

The processing unit 54 comprehensively controls the operation of the performer photographing device 5 based on the program stored in the storage unit 55. The function of the processing unit 54 can be realized by, for example, a microprocessor such as a CPU or GPU, or an electronic component such as an ASIC or an IC memory. The processing unit 54 includes a fill signal generation unit 151, a key signal generation unit 152, and a live content data generation unit 153 as main functional units.

The fill signal generation unit 151 generates a fill signal of the performer in real time from the live image captured by the camera 50. The key signal generation unit 152 generates a key signal of the performer in real time from the live image captured by the camera 50. For these processes, for example, chroma key technology can be used.

The live content data generation unit 153 generates live content data including the performer's fill signal, the key signal, and the performer's voice acquired by the microphone 51. This live content data is transmitted to the management server 2 in real time via the transmission / reception unit 53.

The management server 2 of the second embodiment will be described. FIG. 20 is a block diagram of the management server 2 of the second embodiment. The difference between the management server 2 of the second embodiment and the first embodiment is that the real-time distribution unit 123 has been added.

The real-time distribution unit 123 streams and distributes the live content data received from the performer photographing device 5 to the predetermined photographing device 1 in real time. The photographing device 1 to be distributed is determined by using the photographing device management database stored in the storage unit 23. FIG. 21 is an example of a photographing apparatus management database according to the second embodiment. The photographing apparatus management database shown in FIG. 21 is provided with a field for presence / absence of live distribution in addition to the first embodiment. The real-time distribution unit 123 transmits the received live content data to the photographing device 1 in which the live distribution field of the photographing device management database is a flag of “Yes”.

When the photographing device 1 receives the live content data from the management database 2, the photographing device 1 starts photographing the user (subject) by the camera while storing the live content data in the storage unit 17. The captured image is displayed on the display 11. Subsequently, the video processing unit 154 synthesizes the fill signal and the photographed image of the user by using the key signal of the live content data (Step 307 in FIG. 13). Then, the video processing unit 154 displays the composite video (moving image) on the display 11 (Step 308 in FIG. 13). Other processes are the same as in the first embodiment.

In the second embodiment, content data is generated from the live video of the performer, and the management device performs live distribution to each shooting device. Therefore, it is possible for a single performer to share a space with many users in different places at the same time.

Although the present invention has been described above with reference to preferred embodiments, the present invention is not necessarily limited to the above embodiments, and can be variously modified and implemented within the scope of the technical idea.

1 Shooting device 2 Management server 3 Payment server 4 Terminal 5 Performer shooting device 10 Camera 11 Display 12 Speaker 13 Operation input unit 14 Short-range wireless communication unit 15 Processing unit 16 Transmission / reception unit 17 Storage unit 20 Input unit 21 Transmission / reception unit 22 Processing unit 23 Storage unit 30 Transmission / reception unit 31 Payment processing unit 32 Storage unit

Claims (9)

It is equipped with a management device and at least one or more photographing devices.
The management device is
A storage unit that stores the recorded image transmitted from the photographing device and indicates the storage location, and transmits the storage location information that can be read by the user's terminal to the photographing device.
With
The photographing device is
The first camera that shoots the user,
Display and
A storage unit that stores content data including augmented reality video data that is a moving image of the performer and audio data including audio for guidance by the performer of the augmented reality video data and shooting start timing of a still image.
A composite video is generated by synthesizing the video of the performer of the extended reality video data of the content data and the video shot by the user taken by the first camera, and the composite video is displayed on the display unit. A video processing unit that records a still image of the composite video according to the shooting start timing by the performer's voice, and
Using the audio data of the content data, an audio output unit that outputs the audio of the performer in synchronization with the composite video displayed on the display unit, and an audio output unit.
A transmission unit that transmits the still image as the recorded image to the management device, and
A storage location presentation unit that receives the storage location information and presents the storage location information,
A shooting system equipped with. The photographing system according to claim 1, wherein the storage location presentation unit displays the storage location information on the display unit. The shooting system according to claim 1 or 2, wherein the video processing unit records the composite video from the start to the end of the shooting of the user as the recorded image. A second camera that shoots the performer,
An augmented reality video data generation unit that generates augmented reality video data of the performer from the video shot by the performer taken by the second camera.
With a microphone that acquires the voice of the performer,
A voice data generation unit that generates voice data from the voice of the performer acquired by the microphone, and
A transmission unit that transmits live content data including at least the augmented reality video data and audio data to the management device, and
Equipped with a performer shooting device
The management device includes a live distribution unit that receives the live content data and performs live distribution of the live content data to at least one or more shooting devices.
The photographing device includes a receiving unit that receives the live content data.
The video processing unit of the shooting device synthesizes the augmented reality video data of the live content data and the user's shot video shot by the first camera to generate a composite video, and displays the composite video. Display on the unit, and record at least a part of the composite video as the recorded image.
Any of claims 1 to 3, wherein the audio output unit of the photographing device uses the audio data of the live content data to output the audio of the performer in synchronization with the synthetic video displayed on the display unit. The shooting system described in Crab. A camera that shoots the user and
Display and
A storage unit that stores content data including augmented reality video data that is a moving image of the performer and audio data including audio for guidance by the performer of the augmented reality video data and shooting start timing of a still image.
The video of the performer of the extended reality video data of the content data and the video shot by the user taken by the camera are combined to generate a composite video, the composite video is displayed on the display unit, and the voice of the performer is displayed. A video processing unit that records a still image of the composite video according to the shooting start timing by
Using the audio data of the content data, an audio output unit that outputs the audio of the performer in synchronization with the composite video displayed on the display unit, and an audio output unit.
A transmission unit that transmits the still image as a recorded image to the management device,
A storage location presentation unit that receives information indicating a storage location of the recorded image and can be read by a user's terminal from the management device and presents the storage location information.
A shooting device equipped with. The imaging device according to claim 5, wherein the storage location presentation unit displays the storage location information on the display unit. The imaging device according to claim 5 or 6, wherein the image processing unit records the composite image from the start to the end of the image of the user as the recorded image. It is provided with a receiving unit that receives live content data including at least the augmented reality video data of the performer generated from the live video of the performer and the audio data generated from the audio of the performer from the management device.
The video processing unit synthesizes the augmented reality video data of the live content data and the video captured by the user taken by the camera to generate a composite video, displays the composite video on the display unit, and displays the composite video. At least a part of the composite video is recorded as the recorded image,
The method according to any one of claims 5 to 7, wherein the audio output unit uses the audio data of the live content data to output the audio of the performer in synchronization with the composite video displayed on the display unit. Shooting device. A computer equipped with a storage unit that stores content data including augmented reality video data that is a moving image of a performer and audio data including audio for guidance by the performer and audio of a still image shooting start timing. It is a user shooting method of
The computer synthesizes the video captured by the user taken by the camera and the video of the performer of the augmented reality video data to generate a composite video, and displays the composite video on the display unit.
The computer uses the audio data of the content data to output the audio of the performer in synchronization with the composite video displayed on the display unit.
The computer records a still image of the composite video at the timing of starting shooting by the voice of the performer.
The computer transmits the still image as a recorded image to the management device, and then
A user photographing method in which a computer is information indicating a storage location of the recorded image, receives storage location information that can be read by a user's terminal from the management device, and presents the storage location information.

Images