JP7466042B1 - Information processing system, information processing device, and information processing method - Google Patents

Abstract

[Problem] To suppress delays when distributing return video to the photographing device side. [Solution] An information processing device 2 has a receiving unit 231 that receives encoded data, which is video data encoded by each of the first photographing device 1, from each of the multiple first photographing device 1, a decoding unit 232 that decodes the multiple encoded data received from each of the multiple first photographing device 1 to restore the multiple video data, a selecting unit 235 that selects one video data from the multiple video data restored by the decoding unit 232, and a distributing unit 236 that distributes the encoded data corresponding to the video data selected by the selecting unit 235 and received by the receiving unit 231 to each of the first photographing device 1. [Selected Figure] Figure 3

Description

The present invention relates to an information processing system, an information processing device, and an information processing method.

Conventionally, in the production of television broadcasting programs, filming is performed using multiple camera devices, and one of the video data generated by each of the multiple camera devices is selected using a switcher as the video data to be aired (see, for example, Patent Document 1).

JP 2005-295431 A

When multiple imaging devices are FPUs (Field Pickup Units) or wireless cameras and video data is transmitted to a switcher via wireless communication, the video data is encoded using an encoder to reduce communication capacity before being transmitted to the switcher, where the encoded video data is decoded. In this case, when on-air video data is distributed to multiple imaging devices as return video, the video data is encoded to reduce communication capacity before being distributed to the imaging devices, which creates a problem of delays.

The present invention was made in consideration of these points, and aims to reduce delays when delivering return footage to the shooting device.

The information processing system according to the first aspect of the present invention is an information processing system having an information processing device and a plurality of first imaging side devices, each of which has a first imaging unit that generates video data by capturing an image, a first encoding unit that encodes the video data generated by the first imaging unit, a first imaging side transmitting unit that transmits encoded data, which is the encoded video data, to the information processing device, a first imaging side receiving unit that receives encoded data distributed from the information processing device, a first imaging side decoding unit that decodes the encoded data received by the first imaging side receiving unit to restore the video data, and a first display control unit that displays the video data restored by the first imaging side decoding unit on a display unit, and the information processing device has a receiving unit that receives encoded data from each of the plurality of first imaging side devices, a decoding unit that decodes the multiple encoded data received from each of the plurality of first imaging side devices to restore the multiple video data, a selecting unit that selects one video data from the multiple video data restored by the decoding unit, and a distributing unit that distributes the encoded data corresponding to the video data selected by the selecting unit to the multiple first imaging side devices.

The information processing device may be capable of communicating with a second photographing side device different from the first photographing side device, and the second photographing side device may have a second photographing unit that generates video data by photographing, a second photographing side transmitting unit that transmits the video data generated by the second photographing unit to the information processing device, a second photographing side receiving unit that receives encoded data distributed from the information processing device, a second photographing side decoding unit that decodes the encoded data received by the second photographing side receiving unit to restore the video data, and a second display control unit that displays the video data restored by the second photographing side decoding unit on a display unit, and the receiving unit receives the video data from the second photographing side device, and the information processing device may have an encoding unit that encodes the video data received by the receiving unit, and the selection unit may select one of the multiple video data restored by the decoding unit and the video data received from the second photographing side device, and the distribution unit may distribute the encoded data corresponding to the video data selected by the selection unit to the multiple first photographing side devices and the second photographing side device.

The distribution unit may specify a delay time, which is a time from when the encoded data is distributed to the first imaging side devices until each of the first imaging side devices starts displaying video data corresponding to the encoded data, and distribute the encoded data to the first imaging side devices so that each of the first imaging side devices starts displaying the video data corresponding to the encoded data at the same time based on the delay time specified for each of the first imaging side devices.

The information processing device may have a storage control unit that causes the multiple encoded data received by the receiving unit from each of the multiple first imaging side devices to be stored in a storage unit, and the distribution unit may, upon receiving a retransmission request for the encoded data from at least one of the multiple first imaging side devices, retransmit the encoded data stored in the storage unit to the first imaging side device that sent the retransmission request.

The storage control unit stores the encoded data from the current time to a predetermined time ago in the storage unit, and when the distribution unit receives a re-distribution request for the encoded data corresponding to the predetermined time from at least one of the plurality of first image capturing side devices, if the encoded data corresponding to the predetermined time is not stored in the storage unit, the distribution unit may obtain the encoded data corresponding to the predetermined time from the first image capturing side device that is the source of the encoded data, and re-distribute the obtained encoded data to the first image capturing side device that sent the re-transmission request.

The first imaging side transmitting unit may transmit the encoded data to the information processing device by wireless communication, and the receiving unit may receive the encoded data from each of the first imaging side devices by wireless communication.
The first image capturing side transmitting section may transmit the encoded data to the information processing device via a communication network of a mobile phone.

The first photographing side receiving unit of the first photographing side device may receive a notification signal indicating whether the video data generated by the first photographing side device that the first photographing side device has is the photographing side device corresponding to the video data selected by the selection unit, and the first display control unit of the first photographing side device may display the video data generated by the first photographing unit on the display unit when the first display control unit of the first photographing side device determines, based on the notification signal received by the first photographing side receiving unit, that the video data corresponding to the first photographing side device that the first photographing side device has has been selected by the selection unit.

The first image capturing side receiving unit may receive the image synthesis data, and the first display control unit may synthesize the acquired image synthesis data with image data decoded from the encoded data distributed from the information processing device, and display the synthesized image on the display unit.

The information processing device according to the second aspect of the present invention includes a receiving unit that receives encoded data from each of a plurality of image capturing devices, the encoded data being video data encoded by each of the image capturing devices; a decoding unit that decodes the encoded data received from each of the plurality of image capturing devices to restore the plurality of video data; a selecting unit that selects one of the plurality of video data restored by the decoding unit; and a distributing unit that distributes the encoded data received by the receiving unit, which corresponds to the video data selected by the selecting unit, to each of the image capturing devices.

The information processing method according to the third aspect of the present invention includes a computer-executed step of receiving encoded data from each of a plurality of image capturing devices, the encoded data being video data encoded by each of the image capturing devices; a step of decoding the encoded data received from each of the plurality of image capturing devices to restore the plurality of video data; a step of selecting one of the restored plurality of video data; and a distribution unit that distributes the received encoded data, which is encoded data corresponding to the selected video data, to each of the image capturing devices.

The present invention has the advantage of being able to reduce delays when delivering return video to the imaging device.

FIG. 1 is a diagram illustrating an overview of an information processing system according to a first embodiment. FIG. 2 is a diagram showing a functional configuration of a first imaging side device. FIG. 1 is a diagram illustrating a functional configuration of an information processing device according to a first embodiment. 11 is a flowchart showing an example of a processing flow in an information processing device. FIG. 2 is a diagram showing a functional configuration of a second imaging side device. FIG. 11 is a diagram illustrating a functional configuration of an information processing device according to a second embodiment.

First Embodiment
[Overview of Information Processing System S]
FIG. 1 is a diagram for explaining an overview of an information processing system S according to a first embodiment. The information processing system S has a plurality of first photographing side devices 1 for photographing and an information processing device 2. The information processing system S is a system for selecting one of the video data generated in the plurality of first photographing side devices 1 and outputting the selected video data as broadcast video data, which is video data for on-air television broadcasting. Note that, in FIG. 1, two first photographing side devices 1 are shown as the plurality of first photographing side devices 1, but this is not limiting, and three or more first photographing side devices 1 may be provided.

The information processing device 2 is communicatively connected to each of the multiple first imaging side devices 1 via a wireless communication network such as a mobile phone line. The information processing device 2 receives encoded data from each of the multiple first imaging side devices 1, which is data obtained by encoding video data generated by the first imaging side devices 1. The information processing device 2 decodes the received encoded data to restore the multiple video data.

The information processing device 2 selects, from among the multiple restored video data, video data to be output as broadcast video data to be used for television broadcasting. The information processing device 2 outputs the selected video data to the external device 3 as broadcast video data.

The information processing device 2 also displays the selected video data as return video on the first imaging device 1. In conventional information processing devices, in order to display the selected video data on the first imaging device, the selected video data is encoded to generate encoded data, and the generated encoded data is distributed to multiple imaging devices. This causes a problem of a delay in generating the encoded data.

In response to this, the information processing device 2 directly distributes the encoded data corresponding to the selected video data from among the multiple encoded data received to the multiple first imaging side devices 1 without undergoing decoding or encoding processing in the information processing device 2. In this way, the information processing system S can reduce delays in distributing the return video to the imaging device side.

[Functional configuration of the first photographing side device 1]
Next, the functional configuration of the information processing system S will be described. First, the functional configuration of the first photographing side device 1 will be described. Fig. 2 is a diagram showing the functional configuration of the first photographing side device 1. It is assumed that each of the multiple first photographing side devices 1 has the configuration shown in Fig. 2. In the example shown in Fig. 2, the first photographing side device 1 is realized by one device, but this is not limited to this and may be realized by multiple devices.

As shown in FIG. 2, the first photographing side device 1 has a first photographing unit 11, a first communication unit 12, a first notification unit 13, a first display unit 14, a first memory unit 15, and a first control unit 16.
The first image capturing unit 11 captures the surroundings to generate video data. The first image capturing unit 11 outputs the generated video data to the first control unit 16.
The first communication unit 12 is a communication interface for transmitting and receiving data to and from the information processing device 2 and the like via a wireless communication network.

The first notification unit 13 is, for example, a tally lamp, and notifies those around that the video data generated by its own device, which is the first imaging side device 1 corresponding to itself, has been selected as video data to be used for television broadcast. The first notification unit 13 notifies those around whether the video data generated by its own device has been selected as broadcast imaging data to be used for television broadcast by changing the lighting state. For example, when the tally lamp is lit red, it indicates that the video data generated by its own device has been selected as broadcast data, and when it is lit green, it indicates that the video data generated by its own device has not been selected as broadcast data.

The first display unit 14 is a display device that displays an image represented by the video data selected by the information processing device 2 as a return image.
The first storage unit 15 is a storage medium that stores various data, and includes a read only memory (ROM), a random access memory (RAM), a hard disk, a solid state drive (SSD), a flash memory, etc. The first storage unit 15 stores a program executed by the first control unit 16. The first storage unit 15 stores a program that causes the first control unit 16 to function as a first encoding unit 161, a first imaging side transmitting unit 162, a first imaging side receiving unit 163, a first notification control unit 164, a first imaging side decoding unit 165, and a first display control unit 166.

The first control unit 16 is, for example, a CPU (Central Processing Unit). The first control unit 16 executes the programs stored in the first storage unit 15 to function as a first encoding unit 161, a first image capturing side transmitting unit 162, a first image capturing side receiving unit 163, a first notification control unit 164, a first image capturing side decoding unit 165, and a first display control unit 166.

The first encoding unit 161 encodes the video data generated by the first image capture unit 11 to generate encoded data. The first encoding unit 161 encodes the video data using, for example, an encoder conforming to the video compression standard H.265 to generate encoded data.

The first image capturing side transmission unit 162 transmits the encoded data, which is the video data encoded by the first encoding unit 161, to the information processing device 2 by wireless communication via the first communication unit 12. For example, the first image capturing side transmission unit 162 transmits the encoded data to the information processing device 2 via a mobile phone communication network.

The first image capturing side receiving unit 163 receives encoded data distributed from the information processing device 2 corresponding to the video data selected by a switcher (not shown). The switcher is a device for selecting broadcast video data from among multiple video data generated by each of the multiple first image capturing side devices 1.

The first image capturing side receiving unit 163 also receives video synthesis data such as subtitles to be synthesized with the video data and computer graphic images for synthesis. The first image capturing side receiving unit 163 receives the video synthesis data from, for example, a switcher or an external device 3 that is used in a broadcasting station and generates video synthesis data. The external device 3 may be provided in the broadcasting station or on the cloud.

In response to the switcher selecting the broadcast video data, the first imaging side receiving unit 163 receives a notification signal indicating whether or not the video data generated by the first imaging side device 1 owned by the first imaging side receiving unit 163 has been selected as the broadcast video data. The notification signal is, for example, a tally signal for changing the color of the tally lamp. The first imaging side receiving unit 163 receives, as the tally signal, a red lighting instruction signal for lighting the tally lamp red, or a green lighting instruction signal for lighting the tally lamp green. Here, the first imaging side receiving unit 163 receives the tally signal from the switcher, but is not limited to this, and may receive it from the information processing device 2.

The first notification control unit 164 notifies the surroundings whether the video data captured by the first image capture unit 11 has been selected as video data for broadcast by controlling the tally lamp as the first notification unit 13 based on the tally signal received by the first image capture side receiving unit 163. For example, when the first image capture side receiving unit 163 receives a red light instruction signal as a tally signal, the first notification control unit 164 turns the tally lamp red. Also, when the first image capture side receiving unit 163 receives a green light instruction signal as a tally signal, the first notification control unit 164 turns the tally lamp green.

The first image capturing side decoding unit 165 decodes the encoded data received by the first image capturing side receiving unit 163 to restore the video data. The first image capturing side decoding unit 165 decodes the received encoded data using, for example, an H.265 decoder to restore the video data.

The first display control unit 166 causes the first display unit 14 to display the image indicated by the video data restored by the first image capturing side decoding unit 165. The first display control unit 166 generates a composite image by combining the video synthesis data acquired by the first image capturing side receiving unit 163 with the video data decoded from the encoded data, and causes the first display unit 14 to display the generated composite image. The first image capturing side device 1 generates a composite image based on the video synthesis data and the encoded data separately acquired by the first image capturing side receiving unit 163, thereby causing the first display unit 14 to display an image indicating a caption or the like corresponding to the video synthesis data. In this way, the video synthesis data is combined with the encoded data without encoding and decoding, so that an image such as a caption indicated by the video synthesis data can be displayed in high definition on the first display unit 14.

[Functional configuration of information processing device 2]
Next, a description will be given of the functional configuration of the information processing device 2. Fig. 3 is a diagram showing the functional configuration of the information processing device 2 according to the first embodiment. The information processing device 2 has a communication unit 21, a storage unit 22, and a control unit 23.

The communication unit 21 is a communication interface for transmitting and receiving data to and from each of the first photographing side devices 1 .
The storage unit 22 is a storage medium that stores various data, and includes a ROM, a RAM, a hard disk, an SSD, a flash memory, etc. The storage unit 22 stores a program executed by the control unit 23. The storage unit 22 stores a program that causes the control unit 23 to function as a receiving unit 231, a decoding unit 232, an output unit 233, a storage control unit 234, a selecting unit 235, and a distributing unit 236.

The control unit 23 is, for example, a CPU. The control unit 23 executes the programs stored in the storage unit 22, thereby functioning as a receiving unit 231, a decoding unit 232, an output unit 233, a storage control unit 234, a selecting unit 235, and a distributing unit 236.

The receiving unit 231 receives the encoded data from each of the multiple first imaging side devices 1 via wireless communication. Specifically, the receiving unit 231 receives the encoded data, which is the video data encoded by each of the multiple first imaging side devices 1, from each of the multiple first imaging side devices 1 via wireless communication via a wireless communication network.

The decoding unit 232 decodes the multiple encoded data received by the receiving unit 231 from each of the multiple first image capture side devices 1 to restore the multiple video data. The decoding unit 232 decodes the multiple encoded data received using, for example, an H.265 decoder to restore the multiple video data.

The output unit 233 outputs the multiple video data restored by the decoding unit 232. For example, the output unit 233 outputs the multiple video data restored by the decoding unit 232 to one or more display devices (not shown) via the communication unit 21, thereby causing the display device to display images represented by each of the multiple video data. Here, the display device is used by a person in charge of selecting broadcast video data to be used for television broadcasting to check images represented by each of the multiple video data. Note that the information processing device 2 may have a display unit, and the output unit 233 may cause the display unit of the information processing device 2 to display images represented by the multiple video data.

The storage control unit 234 stores in the storage unit 22 the multiple encoded data received by the receiving unit 231 from each of the multiple first imaging side devices 1. For example, the storage control unit 234 stores in the storage unit 22 the encoded data from the current time to a predetermined time ago. For example, the storage control unit 234 associates the multiple encoded data corresponding to each time successively received by the receiving unit 231 from the first imaging side device 1 with the time at which the encoded data was received, and stores them in the storage unit 22. The encoded data stored in the storage unit 22 is used to retransmit to the first imaging side device 1.

The selection unit 235 selects one piece of video data from the multiple pieces of video data restored by the decoding unit 232. For example, the information processing device 2 is communicably connected to a switcher, which is a device for selecting broadcast video data. The selection unit 235 receives video data identification information for identifying the selected video data from the switcher, and selects one piece of video data that corresponds to the received video data identification information from the multiple pieces of video data.

In response to the selection of one piece of video data, the selection unit 235 transmits a tally signal to the first imaging side device 1 as a notification signal indicating whether or not the video data captured by the selection unit 235 has been selected as broadcast data. For example, the selection unit 235 transmits a red light indication signal as a tally signal to the first imaging side device 1 corresponding to the selected video data, indicating that the video data has been selected as broadcast video data to be used for television broadcast. In addition, in response to the selection of one piece of video data, the selection unit 235 transmits a green light indication signal as a tally signal to the first imaging side device 1 corresponding to the video data selected immediately before the selection of the video data, indicating that the video data has not been selected as broadcast video data to be used for television broadcast.

When there is no response from a specific first imaging side device 1, the selection unit 235 may perform control via a switcher or display device (not shown) so as not to select video data corresponding to that first imaging side device 1. In this case, the selection unit 235 may perform control so as not to select video data corresponding to the unresponsive first imaging side device 1. In this way, it is possible to prevent video data corresponding to the unresponsive first imaging side device 1 from being selected during a television broadcast.

The distribution unit 236 distributes the encoded data corresponding to the video data selected by the selection unit 235 from among the multiple encoded data received by the reception unit 231 to the multiple first imaging side devices 1. Here, the distribution unit 236 may specify a delay time that is the time from when the encoded data corresponding to the video data selected by the selection unit 235 is distributed to the multiple first imaging side devices 1 to when the first display control unit 166 of each of the multiple first imaging side devices starts displaying the video data corresponding to the encoded data.

For example, the distribution unit 236 identifies a distribution start time, which is the time when the encoded data corresponding to the video data selected by the selection unit 235 is distributed to each of the multiple first photographing side devices 1. The distribution unit 236 also receives, via the receiving unit 231, from each of the multiple first photographing side devices 1, time information indicating a display start time, which is the time when each of the multiple first photographing side devices 1 causes the first display unit 14 to display the video indicated by the video data restored by decoding the encoded data, and identifies the display start time. Then, the distribution unit 236 identifies a delay time corresponding to each of the multiple first photographing side devices 1 based on the distribution start time and the display start time corresponding to each of the multiple first photographing side devices 1.

The distribution unit 236 adjusts the distribution time of the encoded data for each of the multiple first imaging side devices 1 based on the delay time identified for each of the multiple first imaging side devices 1. As a result, the distribution unit 236 distributes the encoded data to the multiple first imaging side devices 1 so that the time when the first display control unit 166 of each of the multiple first imaging side devices 1 starts displaying the video data corresponding to the encoded data is the same. In this way, the information processing device 2 can display the video data corresponding to the encoded data at the same timing on each of the multiple first imaging side devices 1, taking into account the delay time and the like associated with decoding the encoded data in each of the multiple first imaging side devices 1.

The distribution unit 236 also receives a retransmission request for the encoded data from at least one of the multiple first imaging side devices 1 via the receiving unit 231. The distribution request includes the shooting time corresponding to the encoded data requested to be retransmitted by the first imaging side device 1. When the distribution unit 236 receives a retransmission request for the encoded data, it redistributes the encoded data from the shooting time onwards that is stored in the storage unit 22 to the first imaging side device 1 that sent the retransmission request, based on the shooting time included in the distribution request.

When the distribution unit 236 receives a re-distribution request for encoded data corresponding to a specific shooting time from at least one of the multiple first imaging side devices 1, if the encoded data corresponding to the shooting time is not stored in the storage unit 22, the distribution unit 236 transmits an acquisition request for the encoded data corresponding to the shooting time to the first imaging side device 1 that is the sender of the encoded data. The acquisition request includes the shooting time corresponding to the encoded data requested to be resent.

When the first photographing side transmitting unit 162 of the first photographing side device 1 acquires the request to acquire the encoded data, it retransmits the encoded data stored in the first storage unit 15 after the photographing time included in the acquisition request to the information processing device 2 based on the photographing time included in the acquisition request. The distributing unit 236 acquires the encoded data corresponding to the photographing time included in the distribution request from the first photographing side device 1, and redistributes the acquired encoded data to the first photographing side device 1 that sent the retransmission request.

The first display control unit 166 of the first photographing side device 1 that sent the retransmission request causes the first display unit 14 to display the redistributed video data restored by the first photographing side decoding unit 165. In this case, the first display control unit 166 causes the first display unit 14 to display, for example, the video represented by the redistributed video data while skipping some of the frames corresponding to the video. In this way, the first photographing side device 1 can quickly eliminate delays in displaying the video data.

[Operation flow]
Next, a description will be given of a process flow related to the information processing device 2. Fig. 4 is a flowchart showing an example of the process flow in the information processing device 2. It is assumed that, before the process related to this flowchart is performed, the output unit 233 of the information processing device 2 outputs the video data selected by the selection unit 235 to the external device 3, and the distribution unit 236 distributes the encoded data corresponding to the video data to the multiple first imaging side devices 1.

First, the receiving unit 231 receives encoded data from each of the first image capturing side devices 1 via wireless communication (S1).
Next, the decoding unit 232 decodes the encoded data received from the first image capturing side devices 1, respectively, to restore the video data (S2).

Next, the output unit 233 outputs the multiple pieces of video data restored by the decoding unit 232 to a display device (S3).
Next, the selection unit 235 determines whether or not new video data has been selected (S4). If the selection unit 235 determines that new video data has been selected (YES in S4), the process proceeds to S5, and if the selection unit 235 determines that new video data has not been selected (NO in S4), the process returns to S1.

In S5, the selection unit 235 switches the video data to be output to the external device 3 to the newly selected video data.
Next, the distribution unit 236 switches the encoded data to be output to the first image capturing side devices 1 to the newly selected encoded data (S6).

Then, the distribution unit 236 determines whether or not an end instruction for the process related to the first photographing side device 1 has been received from a management device (not shown) that manages the settings of the first photographing side device 1 (S7). If the distribution unit 236 determines that an end instruction has been received (YES in S7), it ends the process related to this flowchart, and if it determines that an end instruction has not been received (NO in S7), it proceeds to S1.

[Modification 1]
In the above embodiment, the output unit 233 outputs the video data selected by the selection unit 235 to the external device 3 as broadcast video data to be used for television broadcasting, but this is not limited to the above. The output unit 233 may also output the video data selected by the selection unit 235 to the external device 3 as distribution video data to be used for data distribution.

[Modification 2]
In the above embodiment, the first display control unit 166 causes the first display unit 14 to display the video data restored by the first imaging side decoding unit 165, but this is not limited to the above. When the first display control unit 166 determines, based on the notification signal received by the first imaging side receiving unit 163, that the video data corresponding to the first imaging side device 1 owned by the first display control unit 166 has been selected by the selection unit 235, the first display control unit 166 may cause the first display unit 14 to display the video data generated by the first imaging unit 11.

The distribution unit 236 may also be configured not to distribute the encoded data corresponding to the video data selected by the selection unit 235 to the first image capturing side device 1 among the multiple first image capturing side devices 1. In this way, the information processing device 2 can reduce the communication capacity required for distributing the encoded data.

[Effects of information processing device 2]
As described above, the information processing device 2 according to the first embodiment receives encoded data from each of the multiple first imaging side devices 1, decodes the received encoded data, restores the multiple video data, selects one video data from the restored multiple video data, and distributes the received encoded data, which corresponds to the selected video data, to each first imaging side device 1. In this way, the information processing device 2 can suppress delays in distributing return video to the imaging device side.

Second Embodiment
Next, a second embodiment will be described. Among the photographing side devices, there are second photographing side devices such as wired cameras and analog cameras that transmit video data to the information processing device 2 without encoding it. In contrast, the information processing device 2 according to the second embodiment differs from the first embodiment in that the video data received from the second photographing side device is encoded. The information processing device 2 according to the second embodiment will be described below. Note that the description of the same parts as in the first embodiment will be omitted as appropriate.

[Functional configuration of second photographing side device 4]
Fig. 5 is a diagram showing a functional configuration of the second photographing side device 4. Note that, although the second photographing side device 4 is realized by one device as shown in Fig. 5, the present invention is not limited to this and may be realized by a plurality of devices.

As shown in FIG. 5, the second photographing side device 4 has a second photographing unit 41, a second communication unit 42, a second notification unit 43, a second display unit 44, a second memory unit 45, and a second control unit 46. The functions of the second photographing unit 41, the second communication unit 42, the second notification unit 43, the second display unit 44, and the second memory unit 45 are similar to the functions of the first photographing unit 11, the first communication unit 12, the first notification unit 13, the first display unit 14, and the first memory unit 15, so a description thereof will be omitted.

The second control unit 46 is, for example, a CPU. The second control unit 46 executes a program stored in the second storage unit 45 to function as a second image capturing side transmitting unit 462, a second image capturing side receiving unit 463, a second notification control unit 464, a second image capturing side decoding unit 465, and a second display control unit 466.

The second image capturing side transmission unit 462 transmits the video data generated by the second image capturing unit 41 to the information processing device 2 via the second communication unit 42. For example, the second image capturing side transmission unit 462 transmits the video data to the information processing device 2 via a wired communication network.

The functions of the second image capturing side receiving unit 463, the second notification control unit 464, the second image capturing side decoding unit 465, and the second display control unit 466 are similar to the functions of the first image capturing side receiving unit 163, the first notification control unit 164, the first image capturing side decoding unit 165, and the first display control unit 166, and therefore will not be described.

[Functional configuration of information processing device 2]
Next, a functional configuration of the information processing device 2 according to the second embodiment will be described with reference to Fig. 6, which is a diagram showing the functional configuration of the information processing device 2 according to the second embodiment.

As shown in FIG. 6, the control unit 23 of the information processing device 2 according to the second embodiment has an encoding unit 237 .
The encoding unit 237 encodes the video data received by the receiving unit 231 from the second image capturing side device 4 to generate encoded data.

The distribution unit 236 distributes the encoded data corresponding to the video data selected by the selection unit 235, i.e., the broadcast video data to be used for television broadcasting, to a plurality of first image capturing side devices 1 and second image capturing side devices 4.

Specifically, when the video data selected by the selection unit 235 is video data generated by any of the multiple first photographing side devices 1, the distribution unit 236 distributes the encoded data corresponding to the video data to the multiple first photographing side devices 1 and the second photographing side device 4. When the video data selected by the selection unit 235 is video data generated by the second photographing side device 4, the distribution unit 236 distributes the encoded data generated by the encoding unit 237 encoding the video data to the multiple first photographing side devices 1 and the second photographing side devices 4.

By doing this, even if video data received from a second image capturing side device 4 that does not have an encoding function is selected, the information processing device 2 can distribute the encoded data obtained by encoding the video data to multiple first image capturing side devices 1 and second image capturing side devices 4.

Note that the encoding unit 237 unconditionally encodes the video data received by the receiving unit 231 from the second photographing side device 4 to generate encoded data, but this is not limited to the above. The encoding unit 237 may encode the video data received from the second photographing side device 4 in response to the selection by the selection unit 235 of the video data generated by the second photographing side device 4. In this way, the information processing device 2 does not encode the video data received from the second photographing side device 4 when the selection unit 235 has not selected the video data generated by the second photographing side device 4, thereby reducing the load associated with encoding.

Furthermore, this invention will make it possible to contribute to Goal 9 of the United Nations' Sustainable Development Goals (SDGs), which is "Build resilient infrastructure, promote inclusive and sustainable industrialization, and promote innovation and infrastructure."

Although the present invention has been described above using embodiments, the technical scope of the present invention is not limited to the scope described in the above embodiments, and various modifications and changes are possible within the scope of the gist of the invention. For example, all or part of the device can be configured by distributing or integrating functionally or physically in any unit. In addition, new embodiments resulting from any combination of multiple embodiments are also included in the embodiments of the present invention. The effect of the new embodiment resulting from the combination also has the effect of the original embodiment.

1 First photographing side device 2 Information processing device 3 External device 4 Second photographing side device 11 First photographing unit 12 First communication unit 13 First notification unit 14 First display unit 15 First memory unit 16 First control unit 21 Communication unit 22 Memory unit 23 Control unit 41 Second photographing unit 42 Second communication unit 43 Second notification unit 44 Second display unit 45 Second memory unit 46 Second control unit 161 First encoding unit 162 First photographing side transmission unit 163 First photographing side receiving unit 164 First notification control unit 165 First photographing side decoding unit 166 First display control unit 231 Receiving unit 232 Decoding unit 233 Output unit 234 Memory control unit 235 Selection unit 236 Distribution unit 237 Encoding unit 462 Second photographing side transmission unit 463 Second photographing side receiving unit 464 Second notification control unit 465 Second photographing side decoding unit 466 Second display control unit S Information processing system

Claims (14)

An information processing system including an information processing device, a plurality of first photographing side devices , and a plurality of second photographing side devices different from the first photographing side devices,
Each of the plurality of first imaging side devices
a first imaging unit that captures images to generate video data;
a first encoding unit that encodes the video data generated by the first imaging unit;
a first image capturing side transmitting unit configured to transmit encoded data, which is encoded video data, to the information processing device;
a first image capturing side receiving unit for receiving encoded data distributed from the information processing device;
a first image capture side decoding unit that decodes the encoded data received by the first image capture side receiving unit to restore video data;
a first display control unit that causes a display unit to display the video data restored by the first image capture side decoding unit;
having
The second imaging side device is
A second imaging unit that captures images to generate video data;
a second image capturing side transmitting unit that transmits the image data generated by the second image capturing unit to the information processing device;
a second image capturing side receiving unit for receiving the encoded data distributed from the information processing device;
a second image capturing side decoding unit that decodes the encoded data received by the second image capturing side receiving unit to restore video data;
a second display control unit that causes a display unit to display the video data restored by the second image capturing side decoding unit;
having
The information processing device includes:
a receiving unit that receives encoded data from each of the first image capturing side devices and receives video data from the second image capturing side device ;
an encoding unit that encodes the video data received by the receiving unit from the second image capturing device;
a decoding unit that decodes the encoded data received from each of the first image capturing devices to restore a plurality of video data;
a selection unit that selects one of the plurality of video data restored by the decoding unit and the video data received from the second imaging side device ;
a distribution unit that distributes encoded data corresponding to the video data selected by the selection unit to a plurality of the first image capturing side devices and the second image capturing side device ;
having
Information processing system. An information processing system including an information processing device and a plurality of first imaging side devices,
Each of the plurality of first imaging side devices
a first imaging unit that captures images to generate video data;
a first encoding unit that encodes the video data generated by the first imaging unit;
a first image capturing side transmitting unit configured to transmit encoded data, which is encoded video data, to the information processing device;
a first image capturing side receiving unit for receiving encoded data distributed from the information processing device;
a first image capture side decoding unit that decodes the encoded data received by the first image capture side receiving unit to restore video data;
a first display control unit that causes a display unit to display the video data restored by the first image capture side decoding unit;
having
The information processing device includes:
a receiving unit that receives encoded data from each of the first image capturing side devices;
a decoding unit that decodes the encoded data received from each of the first image capturing devices to restore a plurality of video data;
a selection unit that selects one of the plurality of video data restored by the decoding unit;
a distribution unit that distributes encoded data corresponding to the video data selected by the selection unit to the first imaging side devices, specifies a delay time that is a time from when the encoded data is distributed to the first imaging side devices to when each of the first imaging side devices starts displaying the video data corresponding to the encoded data, and distributes the encoded data to the first imaging side devices so that each of the first imaging side devices starts displaying the video data corresponding to the encoded data at the same time based on the delay time specified for each of the first imaging side devices;
having
Information processing system. An information processing system including an information processing device and a plurality of first imaging side devices,
Each of the plurality of first imaging side devices
a first imaging unit that captures images to generate video data;
a first encoding unit that encodes the video data generated by the first imaging unit;
a first image capturing side transmitting unit configured to transmit encoded data, which is encoded video data, to the information processing device;
a first image capturing side receiving unit for receiving encoded data distributed from the information processing device;
a first image capture side decoding unit that decodes the encoded data received by the first image capture side receiving unit to restore video data;
a first display control unit that causes a display unit to display the video data restored by the first image capture side decoding unit;
having
The information processing device includes:
a receiving unit that receives encoded data from each of the first image capturing side devices;
a storage control unit that stores in a storage unit the plurality of encoded data received by the receiving unit from the plurality of first imaging side devices;
a decoding unit that decodes the encoded data received from each of the first image capturing devices to restore a plurality of video data;
a selection unit that selects one of the plurality of video data restored by the decoding unit;
a distribution unit that distributes encoded data corresponding to the video data selected by the selection unit and received by the receiving unit to a plurality of the first imaging side devices , and when a retransmission request for the encoded data is received from at least one of the plurality of the first imaging side devices, retransmits the encoded data stored in the storage unit to the first imaging side device that has transmitted the retransmission request ;
having
Information processing system. The storage control unit stores encoded data from a current time to a predetermined time ago in the storage unit,
when the distribution unit receives a re-distribution request for the encoded data corresponding to the predetermined time from at least one of the plurality of first imaging side devices, and if the encoded data corresponding to the predetermined time is not stored in the storage unit, the distribution unit acquires the encoded data corresponding to the predetermined time from the first imaging side device that is a transmission source of the encoded data, and re-distributes the acquired encoded data to the first imaging side device that has transmitted the re-distribution request.
The information processing system according to claim 3 . the first image capturing side transmitting unit transmits the encoded data to the information processing device by wireless communication;
the receiving unit receives encoded data from each of the first imaging side devices via wireless communication;
The information processing system according to any one of claims 1 to 3 . the first image capturing side transmitting unit transmits the encoded data to the information processing device via a communication network of a mobile phone;
The information processing system according to claim 5 . the first photographing side receiving unit of the first photographing side device receives a notification signal indicating whether the video data generated by the first photographing side device is the photographing side device corresponding to the video data selected by the selection unit;
when determining, based on the notification signal received by the first imaging side receiving unit, that the video data corresponding to the first imaging side device having the first display control unit is selected by the selection unit, the first display control unit of the first imaging side device causes the video data generated by the first imaging unit to be displayed on the display unit.
7. The information processing system according to claim 6 . The first image capturing side receiving unit receives image synthesis data,
the first display control unit combines the acquired video composition data with video data decoded from the encoded data distributed from the information processing device and displays the combined video data on a display unit;
The information processing system according to any one of claims 1 to 3 . a receiving unit that receives encoded data, which is video data encoded by each of the first photographing side devices, from each of the first photographing side devices, and receives video data photographed and generated by the second photographing side device from a second photographing side device that is different from the first photographing side devices, the second photographing side device receiving the encoded data, decoding the received encoded data to restore video data, and displaying the restored video data on a display unit ;
an encoding unit that encodes the video data received by the receiving unit from the second image capturing device;
a decoding unit that decodes the encoded data received from each of the first image capturing devices to restore a plurality of video data;
a selection unit that selects one of the plurality of video data restored by the decoding unit and the video data received from the second imaging side device ;
a distribution unit that distributes encoded data corresponding to the video data selected by the selection unit to a plurality of the first image capturing side devices and the second image capturing side device ;
An information processing device having the above configuration. a receiving unit that receives encoded data, which is video data encoded by each of the imaging side devices, from each of the imaging side devices;
a decoding unit that decodes the encoded data received from each of the image capturing devices to restore the video data;
a selection unit that selects one of the plurality of video data restored by the decoding unit;
a distribution unit that distributes encoded data corresponding to the video data selected by the selection unit to the multiple imaging side devices , specifies a delay time that is a time from when the encoded data is distributed to the multiple imaging side devices to when each of the multiple imaging side devices starts displaying the video data corresponding to the encoded data, and distributes the encoded data to the multiple imaging side devices so that each of the multiple imaging side devices starts displaying the video data corresponding to the encoded data at the same time based on the delay time specified for each of the multiple imaging side devices ;
An information processing device having the above configuration. a receiving unit that receives encoded data, which is video data encoded by each of the imaging side devices, from each of the imaging side devices;
a storage control unit that causes the plurality of encoded data received by the receiving unit from the plurality of imaging side devices to be stored in a storage unit;
a decoding unit that decodes the encoded data received from each of the image capturing devices to restore the video data;
a selection unit that selects one of the plurality of video data restored by the decoding unit;
a distribution unit that distributes encoded data corresponding to the video data selected by the selection unit and received by the receiving unit to a plurality of the imaging side devices , and when a retransmission request for the encoded data is received from at least one of the plurality of the imaging side devices, retransmits the encoded data stored in the storage unit to the imaging side device that has transmitted the retransmission request ;
An information processing device having the above configuration. The computer executes
receiving encoded data from each of the first photographing side devices, which is video data encoded by each of the first photographing side devices, and receiving video data captured and generated by the second photographing side devices from a second photographing side device different from the first photographing side devices, which receives the encoded data, decodes the received encoded data to restore video data, and displays the restored video data on a display unit;
encoding the video data received from the second image capture device;
decoding the encoded data received from the first image capture devices to restore the video data;
selecting one of the restored video data and the video data received from the second image capturing device ;
a distribution unit that distributes encoded data corresponding to the selected video data to the first image capturing side device and the second image capturing side device ;
An information processing method comprising the steps of: The computer executes
receiving encoded data from each of the plurality of image capturing devices, the encoded data being video data encoded by each of the image capturing devices;
decoding the encoded data received from each of the plurality of image capturing devices to restore a plurality of video data;
selecting one image data from the plurality of restored image data;
a distribution unit that distributes the received encoded data to each of the imaging side devices, the encoded data being encoded data corresponding to the selected video data, specifies a delay time that is a time from when the encoded data is distributed to the multiple imaging side devices to when each of the multiple imaging side devices starts displaying the video data corresponding to the encoded data, and distributes the encoded data to the multiple imaging side devices so that each of the multiple imaging side devices starts displaying the video data corresponding to the encoded data at the same time based on the delay time specified for each of the multiple imaging side devices ;
An information processing method comprising the steps of: The computer executes
receiving encoded data from each of the plurality of image capturing devices, the encoded data being video data encoded by each of the image capturing devices;
storing the encoded data received from each of the imaging side devices in a storage unit;
decoding the encoded data received from each of the plurality of image capturing devices to restore a plurality of video data;
selecting one image data from the plurality of restored image data;
a distribution unit that distributes the received encoded data corresponding to the selected video data to the plurality of imaging side devices and, upon receiving a retransmission request for the encoded data from at least one of the plurality of imaging side devices, retransmits the encoded data stored in the storage unit to the imaging side device that has transmitted the retransmission request;
An information processing method comprising the steps of:

Images