JP6473469B2 - Video distribution system - Google Patents

Description

  The present invention relates to a video distribution system including a distribution server and a viewer terminal for distributing and distributing a plurality of input videos in one video area and distributing the distribution video represented by the entire video area.

  2. Description of the Related Art Conventionally, there is known a video distribution system including a distribution server and a viewer terminal that distribute and distribute a plurality of input videos arranged in one video area and distribute the distribution video represented by the entire video area.

  For example, Non-Patent Documents 1 and 2 disclose a technique called “fabric video (fabric video)” (registered trademark) in which a plurality of videos are combined into one video at a server and distributed to clients via a network. ing. As a technique related to this technique, Patent Document 1 discloses that a server specifies a video area to be viewed by a client in a server, and transmits a single stream for transmission corresponding to the specified video area as a plurality of video bitstreams. The technique which comprises is disclosed.

  Non-Patent Documents 1 and 2 disclose that the arrangement and size of each video in the combined video received from the server can be changed. Specifically, the arrangement and size change of each video is requested from the client to the server via the network. At the server, each time the client requests to change the arrangement or size of each video, the video is rearranged or resized according to the request to regenerate a single video for transmission. Is distributed (returned) via a network.

International Publication No. 2012/039404

Gnzo Co., Ltd., “fabric video”, [online], [February 6, 2017 search], Internet <URL: https: // gnzo. com / fabricvideo >> Campus Create Co., Ltd., “Multi-view live video service such as sports video, concert video, event video, etc. by“ Fabric Video ”, [online], [Search February 6, 2017], Internet <URL: http: //Www.open-innovation-portal.com/corporate/it/post_2.html>

  However, in the above-described conventional technology, when a large number of viewers simultaneously request a change in the display area of a plurality of input videos for a distribution video with a large number of viewers such as an Olympic distribution video, a large amount of data is sent to the server. Therefore, there is a possibility that it takes a long time until the display areas of a plurality of input videos are changed in each viewer terminal that makes the request. In addition, when a heavy load is applied to the server, a viewer terminal that does not request any change in the display area of the input video requires a long time to receive the distribution video from the server, and displays the distribution video. May be interrupted.

  In addition, using the above-described conventional technology, the distribution video generated by the server is recorded on a recording medium such as a DVD or a BD (Blu-ray Disc), and the distribution medium can be distributed to the client to distribute the distribution video. Conceivable. However, in this case, when the viewer who has viewed the distribution video recorded on the recording medium desires to change the display area of the plurality of input videos included in the distribution video, it is necessary to notify the server of the change contents. . In the server, there is a burden of regenerating the distribution video in which the display areas of the plurality of input videos are changed according to the change contents, and recording the regenerated distribution video on the recording medium. Further, it takes time to provide the regenerated media to the viewer.

  The present invention has been made in view of the above circumstances, and is a video distribution capable of quickly reorganizing a display area in a viewer terminal of a plurality of input videos included in a distribution video without placing a burden on the distribution server. The purpose is to provide a system.

The video distribution system according to the present invention is a video distribution system including a distribution server and a viewer terminal that distribute and distribute a plurality of input videos in one video area and distribute the distribution video represented by the entire video area. The distribution server stores a distribution unit including distribution information that defines an arrangement area of each of the plurality of input videos in the video area, and arranges each of the plurality of input videos. A viewer configured to generate and distribute the distribution video represented by the entire video region, and a distribution unit that distributes the distribution video and the distribution information; From the display unit that displays a video in a predetermined video display area, the acquisition unit that acquires the distribution video and the distribution information distributed by the distribution unit, and the distribution unit acquired by the acquisition unit, the acquisition unit Take An extraction unit that is arranged in the arrangement area indicated by the arrangement information included in the distribution information and that extracts and divides each of the plurality of input videos individually; and the plurality in the video display area A setting unit for setting each display area of the input video, and controlling the display unit to display each of the plurality of input videos extracted by the extraction unit in the display area set by the setting unit e Bei and a display control unit for the setting unit, by the display area that has been edited by the editing operation in which the receiving unit has received, the display area set before reception of the editing operation by the receiving unit The updating and editing operation of the display area includes an operation of deleting the display area.

According to this configuration, a plurality of input videos are individually divided and extracted using the distribution video and distribution information acquired from the distribution server in the viewer terminal. Then, each of the extracted plurality of input videos is displayed in a display area set by a setting unit included in the viewer terminal. For this reason, unlike the above-described conventional technique, the display area of the plurality of input videos included in the distribution video is quickly reorganized into the display area set on the viewer terminal without imposing a burden on the distribution server. can do.
According to this configuration, when an editing operation of the display area of each of the plurality of input videos by the viewer is accepted, the display set before the editing operation is accepted by the display area edited by the editing operation. The area is updated. For this reason, the viewer can reorganize the display areas of the plurality of input videos without imposing a burden on the distribution server by editing the display areas of the plurality of input videos on the viewer terminal. Can do.
According to this configuration, the viewer performs an operation of deleting the display area when the number of input videos displayed in the video display area is too large, and the input video displayed in the display area is not displayed. Can be displayed. Thus, the viewer can easily view an input video that is displayed in the video display area and is different from the non-displayed input video.

  The distribution information further includes initial display information that defines an initial arrangement of the display area, and the setting unit includes the initial display information indicated by the initial display information included in the distribution information acquired by the acquisition unit. The display area may be set according to the arrangement.

  According to this configuration, the display unit displays each of the plurality of input videos according to the initial arrangement of the display areas of each of the plurality of input videos indicated by the initial display information included in the distribution information acquired from the distribution server by the acquisition unit. An area is set. For this reason, in a viewer terminal, the effort which sets each display area | region of several input image | video separately can be reduced.

  The editing operation for the display area preferably includes operations for enlarging and reducing the display area.

  According to this configuration, the viewer can perform an operation of enlarging or reducing the display area to enlarge or reduce the input video displayed in the display area so that it can be easily viewed.

  The display area editing operation preferably includes an operation of moving the display area.

  According to this configuration, the viewer can perform an operation of moving the display area to move the input video displayed in the display area to a position where it can be easily viewed.

In addition, a video distribution system according to the present invention includes a distribution server and a viewer terminal that distribute and distribute a plurality of input videos in one video area and distribute the distribution video represented by the entire video area. The distribution server includes a storage unit that stores distribution information including arrangement information that defines an arrangement area of each of the plurality of input videos in the video area, and each of the plurality of input videos. A distribution unit configured to generate and distribute the distribution video represented by the entire video region, and a distribution unit to distribute the distribution video and the distribution information, The person terminal includes a display unit that displays video in a predetermined video display area, an acquisition unit that acquires the distribution video and the distribution information distributed by the distribution unit, and the distribution video acquired by the acquisition unit, Get In the video display area, an extraction unit that is arranged in the arrangement area indicated by the arrangement information included in the distribution information acquired by each of the plurality of input videos and separately extracts the input video, A setting unit that sets each display region of the plurality of input videos, and the display region that the setting unit sets each of the plurality of input videos extracted by the extraction unit by controlling the display unit A display control unit to be displayed on the screen, and a posture detection unit that detects the posture of the video display area intended by the viewer, and the plurality of input videos are obtained by shooting all directions from a predetermined shooting viewpoint. contains the omnidirectional image which is, before Symbol display control unit, taking a direction perpendicular said included in the omnidirectional image, the plane of the same position as the position where the posture detecting section detects from the shooting viewpoint Virtual obtained by shooting as a direction Between images, and it controls the display unit, the may be displayed on the display area of the omnidirectional image.

  According to this configuration, the omnidirectional video display area is captured in a direction orthogonal to a plane having the same posture as the posture of the video display region intended by the viewer from a predetermined shooting viewpoint included in the omnidirectional video. A virtual space image obtained by shooting as a direction is displayed. For this reason, the viewer visually recognizes in the omnidirectional video display area the same video as the video viewed when the head posture is changed in the same manner as the video display area at the omnidirectional video shooting location. can do.

In addition, a video distribution system according to the present invention includes a distribution server and a viewer terminal that distribute and distribute a plurality of input videos in one video area and distribute the distribution video represented by the entire video area. The distribution server includes a storage unit that stores distribution information including arrangement information that defines an arrangement area of each of the plurality of input videos in the video area, and each of the plurality of input videos. A distribution unit configured to generate and distribute the distribution video represented by the entire video region, and a distribution unit to distribute the distribution video and the distribution information, The person terminal includes a display unit that displays video in a predetermined video display area, an acquisition unit that acquires the distribution video and the distribution information distributed by the distribution unit, and the distribution video acquired by the acquisition unit, Get In the video display area, an extraction unit that is arranged in the arrangement area indicated by the arrangement information included in the distribution information acquired by each of the plurality of input videos and separately extracts the input video, A setting unit that sets each display region of the plurality of input videos, and the display region that the setting unit sets each of the plurality of input videos extracted by the extraction unit by controlling the display unit a display control unit for displaying on, by the setting unit, the display area, a selection unit for receiving a selection of whether to move in accordance with a change in the orientation of the image display area, may be provided.

  According to this configuration, it is possible to appropriately select whether or not to move each display area of the plurality of input videos in accordance with a change in the attitude of the video display area.

In addition, a video distribution system according to the present invention includes a distribution server and a viewer terminal that distribute and distribute a plurality of input videos in one video area and distribute the distribution video represented by the entire video area. The distribution server includes a storage unit that stores distribution information including arrangement information that defines an arrangement area of each of the plurality of input videos in the video area, and each of the plurality of input videos. A distribution unit configured to generate and distribute the distribution video represented by the entire video region, and a distribution unit to distribute the distribution video and the distribution information, The person terminal includes a display unit that displays video in a predetermined video display area, an acquisition unit that acquires the distribution video and the distribution information distributed by the distribution unit, and the distribution video acquired by the acquisition unit, Get In the video display area, an extraction unit that is arranged in the arrangement area indicated by the arrangement information included in the distribution information acquired by each of the plurality of input videos and separately extracts the input video, A setting unit that sets each display region of the plurality of input videos, and the display region that the setting unit sets each of the plurality of input videos extracted by the extraction unit by controlling the display unit And the distribution information further includes animation information defining an animation effect to be applied when each of the plurality of input videos is selected and an input video to which the distribution is applied. The viewer terminal receives a selection of one input video included in the plurality of input videos by the viewer, and the selection of the one input video is received by the video selection unit. In this case, as indicated by the animation information included in the distribution information acquired by the acquisition unit, the animation effect applied when the one input video is selected is applied to the input video of the application destination. And an application unit to be applied.

  According to this configuration, when one input video is selected by the viewer, the animation effect is applied to the input video of a predetermined application destination as indicated by the animation information included in the distribution information. For this reason, the viewer can interactively operate the input video of the application destination only by selecting one input video.

In addition, a video distribution system according to the present invention includes a distribution server and a viewer terminal that distribute and distribute a plurality of input videos in one video area and distribute the distribution video represented by the entire video area. The distribution server includes a storage unit that stores distribution information including arrangement information that defines an arrangement area of each of the plurality of input videos in the video area, and each of the plurality of input videos. A distribution unit configured to generate and distribute the distribution video represented by the entire video region, and a distribution unit to distribute the distribution video and the distribution information, The person terminal includes a display unit that displays video in a predetermined video display area, an acquisition unit that acquires the distribution video and the distribution information distributed by the distribution unit, and the distribution video acquired by the acquisition unit, Get In the video display area, an extraction unit that is arranged in the arrangement area indicated by the arrangement information included in the distribution information acquired by each of the plurality of input videos and separately extracts the input video, A setting unit that sets each display region of the plurality of input videos, and the display region that the setting unit sets each of the plurality of input videos extracted by the extraction unit by controlling the display unit A display control unit for displaying the distribution information , wherein the distribution information includes audio recording information defining an audio recording area for recording each of one or more input sounds that can be reproduced during display of the distribution video; Reproduction information defining respective reproduction methods of the input voices, and the generation unit further records each of the one or more input voices in the voice recording area indicated by the voice recording information. And each of the one or more input voices One distribution sound having the recorded sound recording area is generated, the distribution unit further distributes the distribution sound, the acquisition unit further acquires the distribution sound, and the extraction unit includes: Further, the one or more input voices recorded in the voice recording area indicated by the voice recording information included in the delivery information acquired by the acquisition unit from the distribution voice acquired by the acquisition unit. The viewer terminal divides and extracts each of them, and the viewer terminal reproduces each of the one or more input sounds extracted by the extraction unit in the distribution information acquired by the acquisition unit A playback unit that plays back the playback method indicated by the information; and in the distribution information, each of the one or more input sounds is associated with one input video of the plurality of input videos, and the playback The method includes the one or more input voices A first reproduction method for reproducing the first input sound when a first input image associated with the first input sound is displayed in the image display area may be included .

  According to this configuration, one or more input sounds that can be reproduced during display of the distribution video are individually divided and extracted using the distribution sound and distribution information acquired from the distribution server in the viewer terminal. Then, each of the extracted one or more input voices is reproduced by the reproduction method indicated by the reproduction information included in the distribution information. For this reason, each of the one or more input voices can be reproduced by a reproduction method specific to each input voice without imposing a burden on the distribution server.

  According to this configuration, the first input video that is determined to be played by the first playback method according to the playback information is displayed in the video display area. If it is played.

  For this reason, for example, when an input video displayed as a background in the video display area is included in the distribution video, playback information that determines that the input audio associated with the input video is played back by the first playback method Is included in the distribution information, the input sound can be reproduced when the input video is displayed as a background.

  In addition, the viewer terminal includes a video selection unit that accepts selection of one input video included in the plurality of input videos by the viewer, and the playback method includes the one of the one or more input sounds. When the selection of the second input video associated with the second input audio is accepted by the video selection unit, the second input audio is reproduced at a predetermined first volume, and the one or more inputs A second playback method may be included in which, among the voices, the input voice that is determined to be played back by the first playback method in the playback information is played back at a second volume lower than the first volume.

  According to this configuration, when the second input video associated with the second input audio that is determined to be played by the second playback method according to the playback information is selected by the viewer, the second input Audio is played at the first volume. In addition, the input sound that is determined to be reproduced by the first reproduction method in the reproduction information is reproduced at a second volume that is smaller than the first volume.

  Thus, for example, when the first input video displayed as the background and the second input video displayed superimposed on the first input video are included in the distribution video, the first input video associated with the first input video is displayed. The distribution information includes reproduction information that defines that one input sound is reproduced by the first reproduction method, and that the second input sound associated with the second input video is reproduced by the second reproduction method. be able to.

  Accordingly, the viewer can hear the first input sound when the first input video is displayed as the background. After that, the viewer selects the second input video to play the second input sound at a volume higher than the first input sound, and listens to the second input sound from the first input sound. Can be made easier.

  In addition, the viewer terminal includes a video selection unit that accepts selection of one input video included in the plurality of input videos by the viewer, and the playback method includes the one of the one or more input sounds. When the selection of the third input video associated with the third input audio is accepted by the video selection unit, a third reproduction method for reproducing only the third input audio may be included.

  According to this configuration, when the third input video associated with the third input sound that is determined to be played by the third playback method according to the playback information is selected by the viewer, the third input Only audio is played.

  Thus, for example, when the first input video displayed as the background and the third input video displayed superimposed on the first input video are included in the distribution video, the first input video associated with the first input video is displayed. The distribution information includes reproduction information that defines that one input sound is reproduced by the first reproduction method, and that the third input sound associated with the third input video is reproduced by the third reproduction method. be able to.

  Accordingly, the viewer can hear the first input sound when the first input video is displayed as the background. Thereafter, the viewer can reproduce only the third input sound by selecting the third input video, and can listen to the third input sound with focus.

  The viewer terminal further includes a viewpoint detection unit that detects the viewer's viewpoint in the video display area, and the video selection unit includes the viewpoint detected by the viewpoint detection unit in the display area. In this case, the selection of the input video displayed in the display area may be accepted.

  According to this configuration, the viewer can easily select the input video only by moving the viewpoint to the display area where the input video is displayed.

  ADVANTAGE OF THE INVENTION According to this invention, the video delivery system which can reorganize rapidly the display area in the viewer terminal of the several input video contained in a delivery video, without putting a burden on a delivery server can be provided.

It is a block diagram which shows an example of a function structure of the video delivery system which concerns on 1st embodiment of this invention. It is a figure which shows an example of the delivery information which concerns on 1st embodiment of this invention. It is a figure which shows an example of a delivery image | video. It is a figure which shows an example of the initial arrangement | positioning of each display area of the some input video | video in a video display area. It is a figure which shows an example at the time of reducing a display area. It is a figure which shows an example at the time of moving, deleting, and enlarging a display area. It is a figure which shows the example of a display of the virtual space image | video according to the attitude | position of an image | video display area, and a one-sided image, (A) is a figure which shows the example of a display before the attitude | position of an image display area changes, (B) FIG. 6C is a diagram showing a display example after the change of the posture of the video display region, and FIG. 6C is a diagram showing a display example different from (B) after the change of the posture of the video display region. It is a flowchart which shows operation | movement of a delivery server. It is a flowchart which shows the operation | movement of the initial display of the delivery image | video in a viewer terminal. It is a flowchart which shows the operation | movement after the initial display of the delivery image | video in a viewer terminal. It is a block diagram which shows an example of a function structure of the video delivery system which concerns on 2nd embodiment of this invention. It is a figure which shows an example of a structure of a delivery audio | voice. It is a figure which shows an example of the delivery information which concerns on 2nd embodiment of this invention.

(First embodiment)
Hereinafter, a video distribution system according to a first embodiment of the present invention will be described with reference to the drawings. FIG. 1 is a block diagram showing an example of a functional configuration of a video distribution system 100 according to the first embodiment of the present invention.

(Configuration of video distribution system 100)
As shown in FIG. 1, the video distribution system 100 includes a distribution server 1 and a viewer terminal 2. The distribution server 1 and the viewer terminal 2 are communicably connected via a network (not shown) such as a LAN, the Internet, or a broadcast data network.

(Configuration of distribution server 1)
The distribution server 1 arranges and synthesizes a plurality of input videos in one video area, and distributes the distribution video represented by the entire video area. Specifically, the distribution server 1 includes a plurality of video output units 11, a storage unit 12, a generation unit 13, and a distribution unit 14.

  The video output unit 11 outputs video data representing a video such as a still image or a moving image to the generation unit 13. Specifically, the video output unit 11 is a camera that outputs video data representing a still image or a moving image (hereinafter referred to as a one-sided video) obtained by shooting a predetermined one direction, or a predetermined shooting viewpoint. Omnidirectional camera (360-degree camera) that outputs video data representing still images and moving images (hereinafter referred to as omnidirectional video) obtained by shooting omnidirectional images, and still images and animations such as telops and logos The video editing apparatus such as a personal computer that can output video data representing the edited still image or moving image is possible. Hereinafter, the video represented by the video data output from the video output unit 11 and input to the generation unit 13 is referred to as an input video.

  The storage unit 12 stores distribution information related to the distribution video. Specifically, the storage unit 12 includes a storage device such as an HDD (Hard Disk Drive) or an SSD (Solid State Drive).

  The distribution information includes distribution video identification information, a title, and a URL. The distribution information further includes identification information of each of the plurality of input videos included in the distribution video, video classification information, arrangement information, initial display information, animation information and the like related to each of the plurality of input videos. Area follow-up information and the like are included.

  Video segment information is information indicating whether each of a plurality of input videos is a background video or a sub video. The background video is a video displayed as a background in the entire predetermined video display area for displaying the video provided in the viewer terminal 2. The sub video is a video displayed in a part of the video display area.

  The arrangement information is information that defines each arrangement area of a plurality of input videos in the video area. The initial display information is information that defines an initial arrangement of display areas of a plurality of input videos in the video display area.

  The animation information is information that defines an animation effect to be applied when each of a plurality of input videos is selected and an input video of the application destination. The area following information is information that determines whether or not to move each display area of a plurality of input videos in accordance with a change in the attitude of the video display area.

  FIG. 2 is a diagram showing an example of the distribution information PF according to the first embodiment of the present invention. FIG. 3 is a diagram illustrating an example of the distribution video V. For example, it is assumed that the storage unit 12 stores the distribution information PF illustrated in FIG. 2 related to the distribution video V illustrated in FIG. As shown in FIG. 3, the distribution video V is a video representing the entire video area VA generated by arranging and synthesizing four input videos VB, V1, V2, and V3 in the video area VA. Shall. The input video VB is assumed to be an omnidirectional video representing a moving image obtained by shooting an omnidirectional image from a predetermined shooting viewpoint with an omnidirectional camera. However, in FIG. 3, a virtual space image that represents a point in time of a moving image obtained by shooting from a shooting viewpoint with a predetermined direction as a shooting direction included in the omnidirectional video is illustrated as an input video VB. Yes. Assume that the input videos V1, V2, and V3 are one-sided videos representing moving images obtained by photographing a predetermined direction. Note that the distribution video V shown in FIG. 3 is merely an example. For example, the input video (the input video VB in FIG. 3) as the background video may be a still image such as a landscape image or a single color image. Further, the input video as the sub video (input video V1, V2, V3 in FIG. 3) may be a still image such as a telop or a logo.

  As described above, when the distribution video V is configured as shown in FIG. 3, as shown in FIG. 2, the distribution information PF includes the identification information “V” of the distribution video V and the title “video of the cat's eye”. And the URL “http: ***”.

  The distribution information PF includes identification information “VB”, “V1”, “V2”, and “V3” for each of the four input videos VB, V1, V2, and V3 included in the distribution video V. ing. The distribution information PF includes video segment information “background video”, “sub video”, “sub video”, and “sub video” of the four input videos VB, V1, V2, and V3. That is, it is determined that the input video VB is a background video and the three input videos V1, V2, and V3 are sub-videos.

  In the distribution information PF, as shown by the broken line portion in FIG. 2, the arrangement areas PAB, PA1, PA2, and the four input videos VB, V1, V2, and V3 in the video area VA (FIG. 3) Arrangement information defining PA3 (FIG. 3) is included. Hereinafter, the arrangement areas of a plurality of input videos in the video area VA are collectively referred to as arrangement areas PA.

  For example, the arrangement area PA has the upper left corner of the video area VA (FIG. 3) as a reference point, the horizontal position in the video area VA (FIG. 3) is indicated by the X coordinate, and the vertical direction in the video area VA (FIG. 3). Is determined using a two-dimensional coordinate indicating the position of Y.

  Specifically, the placement area PA (eg, placement area PAB) of each input video (eg, input video VB) in the video area VA (FIG. 3) is the two-dimensional display indicating the placement position of the upper left corner of each input video. The ratio of the width of each input video to the coordinates “clipX, clipY” (eg, 0, 0) and the horizontal length of the video area VA (hereinafter, the horizontal length is referred to as the width) “clipW” ( (Example: 10000 (100.00%)) and the ratio of the height of each input video to the vertical length of the video area VA (hereinafter, the length in the vertical direction is referred to as height) “clipH” (example: 6667 (66.67%)).

  FIG. 4 is a diagram illustrating an example of an initial arrangement of the display areas DAB, DA1, DA2, and DA3 of the plurality of input videos VB, V1, V2, and V3 in the video display area VDA. In the distribution information PF, as shown in the two-dot chain line portion of FIG. 2, the display areas DA1 of the three input videos V1, V2, and V3 in the video display area VDA (FIG. 4) of the viewer terminal 2, Initial display information defining the initial arrangement of DA2 and DA3 (FIG. 4) is included. Similar to FIG. 3, in FIG. 4, a virtual space image, which is included in the omnidirectional video and is obtained by photographing with the omnidirectional camera as the photographing direction from the photographing viewpoint, is illustrated as the input video VB. Show.

  Further, as described above, the input video VB is defined as a background video and is displayed on the entire video display area VDA (FIG. 4). That is, it is clear that the display area DAB (FIG. 4) of the input video VB in the video display area VDA is the entire video display area VDA (FIG. 4). For this reason, in the initial display information shown in FIG. 2, the definition of the initial arrangement of the display area of the input video VB in the video display area VDA is omitted. However, in the initial display information, the video display area VDA may be defined as the initial arrangement of the display area of the input video VB, which is the background video, like the sub video. Hereinafter, the display areas of a plurality of input videos in the video display area VDA (FIG. 4) are collectively referred to as the display area DA.

  The display area DA is generally used when a virtual space image obtained by capturing a predetermined direction from a predetermined shooting viewpoint among omnidirectional images is displayed on a flat display (screen). It is determined using three-dimensional coordinates such as a viewpoint coordinate system. However, the display area DA is not limited to this, and the display area DA has the horizontal position in the video display area VDA (FIG. 4) as the reference point with the upper left corner of the video display area VDA (FIG. 4) as the reference point, similarly to the arrangement area PA. The position in the vertical direction in the video display area VDA (FIG. 4) may be determined using a two-dimensional coordinate indicated by a Y coordinate.

  The display area DA shown in FIG. 2 is defined using the three-dimensional coordinates of the viewpoint coordinate system. That is, the display area DA is based on the assumption that the virtual space video (eg, the input video VB shown in FIG. 4) is displayed on the entire video display area VDA (FIG. 4). ) As a reference point, the positions in the horizontal direction, vertical direction, and depth (line-of-sight) direction in the virtual space image are determined using three-dimensional coordinates indicated by an X coordinate, a Y coordinate, and a Z coordinate, respectively. . Note that the coordinates of the positions of both ends of the horizontal direction, the vertical direction, and the depth (line of sight) direction of the virtual space image are “1” and “−1”. The position of the Z coordinate “1” is the farthest position in the virtual space image, and the position of the Z coordinate “−1” is the furthest position in the virtual space image.

  Specifically, the initial arrangement of the display area DA (eg, DA1) of each input video (eg, input video V1) in the video display area VDA (FIG. 4) is a three-dimensional display that indicates the center display position of each input video. When displaying the coordinates “posX, posY, posZ” (for example, −0.8, 0.25, 0.7) and each input video, each coordinate axis of the three-dimensional coordinates (X axis, Y axis, Z axis) ) As a rotation axis (degrees) “rotX, rotY, rotZ” (for example, −15 (degrees), −45 (degrees), 0 (degrees)) and when displaying each input video And a ratio “scale” (for example, 1.457) for enlarging or reducing each input video.

  The distribution information PF includes animation information “animation” that defines an animation effect to be applied when each of the three input videos V1, V2, and V3 is selected and an input video of the application destination. .

  Specifically, as shown in FIG. 2, in the animation information (“animation”: “blinking”) corresponding to the input video V1, when the input video V1 is selected, an animation is displayed on the selected input video V1. It is stipulated that the effect “blinking” is applied. In addition, in the animation information (“animation”: “zoom out”) corresponding to the input video V2, when the input video V2 is selected, the animation effect “zoom out” is applied to the selected input video V2. Is stipulated. As described above, the animation information in the case where the same input video as the selected input video is used as the application destination of the animation effect is determined only by the animation effect to be applied (eg, zoom out).

  In addition, in the animation information (“animation”: “blinking: VB”) corresponding to the input video V3, when the input video V3 is selected, the animation effect “ It is stipulated to apply “blinking”. As described above, the animation information in the case where an input video different from the selected input video is used as the application destination of the animation effect includes the animation effect to be applied (eg, blinking) and a predetermined delimiter (eg, “:”). And identification information (for example, VB) of the input video to which the animation effect is applied. In the same manner, the animation information when the same input video as the selected input video is used as the application destination of the animation effect is also applied to the animation effect to be applied (eg, zoom out) and a predetermined delimiter (eg: “:”) And identification information (eg, V2) of the input video to which the animation effect is applied.

  When the animation effect “blinking” is applied to the input video, after the input video is displayed for a predetermined time, the input video is not displayed for a predetermined time. When the animation effect “zoom out” is applied to an input video, the input video is reduced at a predetermined reduction rate every predetermined time and is not displayed.

  The animation effects that can be applied to the input video are not limited to “blinking” and “zoom-out”, but “zoom-in”, “enlargement”, “reduction”, “slide-in”, “video change (video identification information after change) Other animation effects such as “)” may be used.

  For example, when the animation effect “zoom-in” is applied to the input video, the input video is reduced to a predetermined initial size and then at a predetermined magnification every predetermined time, contrary to the animation effect “zoom-out”. It is enlarged and becomes the same size as the display area of the input video. When the animation effect “enlargement (reduction)” is applied to an input video, the input video is enlarged (reduction) at a predetermined enlargement rate (reduction rate) for a predetermined time.

  When the animation effect “slide-in” is applied to an input video, the input video is linearly moved from a predetermined side of the video display area VDA (FIG. 4) to the display area DA of the input video.

  When the animation effect “video change (video identification information after change)” is applied to the input video, the input video is changed to a video identified by the identification information described in parentheses. For example, it is assumed that the animation information corresponding to the input video V2 is ““ animation ”:“ video change (V2): VB ””. In this case, when the input video V2 is selected, the input video VB is changed to the input video V2 described in parentheses. In this way, animation is performed by connecting parentheses to a character string representing an animation effect (eg, video change) and describing parameters (eg, V2) necessary for applying the animation effect in the parenthesis. In the information, parameters necessary for applying the animation effect may be determined.

  The distribution information PF shown in FIG. 2 does not define animation information corresponding to the input video VB that is the background video. However, as with the three input videos V1, V2, and V3 that are sub-videos, animation information corresponding to the input video VB that is a background video may be determined. A mode when the animation effect is applied to the input video will be described later.

  In the distribution information PF, the display areas DA1, DA2, and DA3 (FIG. 4) of the three input videos V1, V2, and V3 are moved according to the change in the posture of the video display area VDA (FIG. 4). Region tracking information “headTrackMoving” (for example, 0, 1, 0) indicating whether or not.

  When the area follow-up information of the display area DA (for example, the display area DA1) is set to “0”, the display area DA moves according to the change in the posture of the video display area VDA (FIG. 4). When the area follow-up information of the display area DA (eg, display area DA2) is set to “1”, the display area DA does not move even if the posture of the video display area VDA (FIG. 4) changes.

  Note that the distribution information PF shown in FIG. 2 includes area following information indicating whether or not the display area DAB of the input video VB, which is the background video, is moved according to the change in the attitude of the video display area VDA (FIG. 4). Is not defined. However, for example, when an input video representing a still image such as a landscape image is a background video, the input video that is a background video is displayed in the same manner as the three input videos V1, V2, and V3 that are sub-videos. You may determine the area | region tracking information of an area | region. A mode in which the display area DA is moved according to the attitude of the video display area VDA (FIG. 4) will be described later.

  Returning to FIG. The generation unit 13 includes a plurality of input videos (for example, VB, V1, V2, and V3 (FIG. 3)) included in the distribution information PF (FIG. 2) stored in the storage unit 12. Arranged in the arrangement area (example: PAB, PA1, PA2, PA3 (FIGS. 2 and 3)) indicated by the information (broken line portion in FIG. 2) and synthesized. Thereby, the production | generation part 13 produces | generates one delivery video V (FIG. 3) which the whole video area VA (FIG. 3) represents. Specifically, the generation unit 13 is configured by a switcher or the like.

  The distribution unit 14 encodes the distribution video V (FIG. 3) generated by the generation unit 13 using a predetermined method (eg, MPEG-4, HLS (HTTP Live Streaming), etc.), and generates video data representing the distribution video V. Generate. The distribution unit 14 distributes the generated video data representing the distribution video V and the distribution information PF (FIG. 2) related to the distribution video V stored in the storage unit 12 via the network. Specifically, the distribution unit 14 includes an encoder and a communication interface circuit that performs communication between the viewer terminal 2 via the network.

  Here, the distribution of the distribution video V and the distribution information PF includes two modes of so-called on-demand distribution and live distribution. That is, in the on-demand distribution, the distribution unit 14 causes the generation unit 13 to generate the distribution video V using the distribution information PF, and then to the viewer terminal 2 permitted to display the distribution video V via the network. Distribution information PF is transmitted. Thereafter, when the viewer terminal 2 accesses the URL (FIG. 2) of the distribution video V included in the distribution information PF, the distribution unit 14 transmits the video data representing the distribution video V to the viewer. Reply to terminal 2.

  On the other hand, in the live distribution, the distribution unit 14 causes the generation unit 13 to generate the distribution video V in real time using the distribution information PF generated in advance at a predetermined time, similarly to the broadcast of the video data of the TV program. The video data representing the distribution video V and the distribution information PF are broadcast using a broadcast data network included in the network.

  The plurality of video output units 11, the storage unit 12, the generation unit 13, and the distribution unit 14 may be provided in one server device, or may be appropriately distributed and provided in a plurality of server devices connected to be communicable with each other. You may do it. In addition, the distribution unit 14 includes a distribution unit 14 that distributes video data representing the distribution video V and a distribution unit 14 that distributes distribution information PF related to the distribution video V in different server devices. You may do it.

(Configuration of viewer terminal 2)
On the other hand, the viewer terminal 2 acquires from the distribution server 1 video data representing the distribution video V and distribution information PF related to the distribution video V. The viewer terminal 2 displays the distribution video V represented by the acquired video data in a predetermined video display area VDA (FIG. 4) based on the acquired distribution information PF. Specifically, the viewer terminal 2 is configured by an information communication device used by the viewer, such as a smartphone, a tablet terminal, a personal computer, or a television.

  The information communication apparatus includes a microcomputer having a CPU, a RAM, a ROM, etc., a communication interface circuit that communicates with the distribution server 1 via the network, a decoder that decodes encoded video data, a video A display device such as a liquid crystal display having a planar video display area VDA (FIG. 4), a touch panel, a remote controller, a keyboard, and a keyboard for causing the viewer to perform various operations of the viewer terminal 2 An operation device such as a mouse is provided.

  The display device may be an HMD (head mounted display) that is connected to the information communication device and displays an image under the control of the information communication device. The display device includes an angular velocity (gyro) sensor that detects the posture of the video display area VDA (FIG. 4). In addition, the display device includes a line-of-sight tracking sensor that detects the viewer's viewpoint in the video display area VDA (FIG. 4) using infrared rays or the like.

  Alternatively, the display device may not include the angular velocity (gyro) sensor, and the remote controller for operating the display device may include an angular velocity (gyro) sensor that detects the attitude of the remote controller. In addition, the visual line tracking sensor may be provided in a remote controller for operating the display device without providing the visual line tracking sensor in the display device.

  As shown in FIG. 1, the viewer terminal 2 functions as a display unit 21, an operation unit 22, an acquisition unit 23, an extraction unit 24, a setting unit 25, and a display control unit 26.

  The display unit 21 is configured by the display device, and displays an image on the image display area VDA (FIG. 4) included in the display device under the control of the microcomputer. The operation unit 22 includes the operation device, and allows the viewer to perform various operations on the viewer terminal 2.

  The acquisition unit 23 is configured by the microcomputer and the communication interface circuit, and communicates with the distribution server 1 to perform video data representing the distribution video V (FIG. 3) and distribution information PF ( 2) is acquired.

  Specifically, when the distribution unit 14 distributes the distribution video V and the distribution information PF in the above-described on-demand distribution mode, the acquisition unit 23 first distributes the distribution video V transmitted by the distribution unit 14. The information PF is received. The acquisition unit 23 stores the received distribution information PF in a RAM, a ROM, or the like included in the microcomputer. Thereafter, when the viewer performs an operation for acquiring a predetermined distribution video V using the operation unit 22, the acquisition unit 23 acquires the distribution information PF related to the distribution video V from the RAM, the ROM, or the like. The URL of the distribution video V included in the distribution information PF is accessed. Then, the acquisition unit 23 acquires video data representing the distribution video V returned from the distribution unit 14.

  On the other hand, when the distribution unit 14 distributes the distribution video V and the distribution information PF in the above-described live distribution mode, the acquisition unit 23 is broadcast by the distribution unit 14 using the broadcast data network after a predetermined time. The video data representing the distribution video V and the distribution information PF are acquired.

  The extraction unit 24 includes the decoder and the microcomputer, and generates the distribution video V (FIG. 3) by decoding the video data acquired by the acquisition unit 23. The extraction unit 24 executes a known image extraction process to thereby arrange the arrangement information (broken line part in FIG. 2) included in the distribution information PF acquired by the acquisition unit 23 from the generated distribution video V (FIG. 3). ) Of a plurality of input images (example: VB, V1, V2, V3 (FIG. 3)) arranged in the arrangement area PA (example: PAB, PA1, PA2, PA3 (FIG. 2, FIG. 3)). Each is divided and extracted individually.

  The setting unit 25 is configured by the microcomputer, and each display area DA (for example, each of the plurality of input videos (for example, VB, V1, V2, and V3 (for example, FIG. 4)) in the video display area VDA (FIG. 4). DAB, DA1, DA2, DA3 (FIG. 4)) are set.

  Specifically, the setting unit 25 represents each of a plurality of input videos indicated by the initial display information (two-dot chain line portion in FIG. 2) included in the distribution information PF (FIG. 2) acquired by the acquisition unit 23. (For example, initial arrangement of display area DA1 of input video V1 “posX: −0.8, posY: 0.25, posZ: 0.7, rotX: −15, rotY: −45, rotZ: 0, scale: 1.457 ”(FIG. 2)), each display area DA of a plurality of input videos (eg, display area DA1 of input video V1 (FIG. 4)) is set.

  If the initial arrangement of the display area DA of the input video (eg, input video VB (FIG. 4)) defined as the background video is not defined in the initial display information (two-dot chain line portion in FIG. 2), the setting is made. The unit 25 sets the entire video display area VDA (FIG. 4) as a display area of the input video (eg, display area DAB (FIG. 4)).

  The display control unit 26 is configured by the microcomputer, and controls the display unit 21 to display each of a plurality of input images (for example, VB, V1, V2, and V3 (FIG. 4)) extracted by the extraction unit 24. The display unit DA displays a plurality of input videos set by the setting unit 25 (for example, DAB, DA1, DA2, DA3 (FIG. 4)).

  The viewer terminal 2 further functions as a reception unit 27, a posture detection unit 91, a selection unit 28, a viewpoint detection unit 92, a video selection unit 20, and an application unit 29.

  The reception unit 27 is configured by the microcomputer, and each display area DA of a plurality of input images (for example, VB, V1, V2, and V3 (FIG. 4)) performed by the viewer using the operation unit 22. (Example: DAB, DA1, DA2, DA3 (FIG. 4)). In accordance with this, the setting unit 25 updates the display area DA set before accepting the editing operation by the accepting unit 27 with the display area DA edited by the editing operation accepted by the accepting unit 27.

  Specifically, the editing operation for the display area DA includes an operation for reducing the display area DA. The editing operation for the display area DA includes an operation for enlarging, moving, and deleting the display area DA.

  For example, as shown in FIG. 4, it is assumed that four input images VB, V1, V2, and V3 are displayed in the display areas DAB, DA1, DA2, and DA3, respectively.

  In this case, it is assumed that the viewer uses the operation unit 22 to perform an operation of reducing the display areas DA1, DA2, and DA3 of the three input videos V1, V2, and V3. The operation for reducing the display area DA is not particularly limited as long as it is the same as the operation for reducing a general figure using a mouse or a touch panel.

  In this case, the accepting unit 27 accepts an operation for reducing the display areas DA1, DA2, and DA3. FIG. 5 is a diagram showing an example when the display area DA is reduced. As shown in FIG. 5, the setting unit 25 updates the three display areas DA1, DA2, and DA3 with the display areas DA1a, DA2a, and DA3a that are reduced by the operation received by the receiving unit 27.

  On the other hand, it is assumed that the viewer uses the operation unit 22 to delete the display areas DA2 and DA3 of the input videos V2 and V3. Further, it is assumed that the viewer uses the operation unit 22 to move the display area DA1 of the input video V1 to the position of the display area DA2 shown in FIG. 4 and perform an operation of enlarging the display area DA1 after the movement. . The operation for deleting, moving, and enlarging the display area DA is not particularly limited, and includes operations for deleting, moving, and enlarging general figures using a mouse or a touch panel. The same operation may be used.

  In this case, the accepting unit 27 accepts an operation for deleting the display areas DA2 and DA3, and an operation for moving and enlarging the display area DA1. FIG. 6 is a diagram illustrating an example when the display area DA is moved, deleted, and enlarged. As illustrated in FIG. 6, the setting unit 25 deletes the display areas DA2 and DA3, and updates the display area DA1 with the display area DA1b that has been moved and enlarged by the operation received by the receiving unit 27.

  In this way, the viewer can perform an operation of enlarging or reducing the display area DA to enlarge or reduce the input video displayed in the display area DA so that it can be easily viewed. In addition, the viewer can perform an operation of moving the display area DA to move the input video displayed in the display area DA to a position where it can be easily viewed. In addition, the viewer performs an operation of deleting the display area DA when the number of input videos displayed in the video display area VDA is too large, and the input video displayed in the display area DA is not displayed. Can be. Thus, the viewer can easily view an input video that is displayed in the video display area VDA and is different from the non-displayed input video.

  The posture detection unit 91 is configured by the angular velocity (gyro) sensor provided in the display device or the remote controller for operating the display device, and detects the posture of the video display area VDA (FIG. 4) intended by the viewer. .

  Specifically, the posture detection unit 91 is configured to provide the angular velocity (gyro) included in the display device when the viewer actually changes the posture of the video display area VDA (FIG. 4) to the posture intended by the viewer. The posture after the change of the video display area VDA (FIG. 4) is detected by the sensor. Alternatively, the posture detection unit 91 changes the posture of the remote controller to the posture intended by the viewer instead of actually changing the posture of the video display area VDA (FIG. 4) of the television or the like, for example. In this case, the post-change posture of the remote controller is detected by the angular velocity (gyro) sensor included in the remote controller as the post-change posture of the video display area VDA (FIG. 4).

  Accordingly, when the omnidirectional video (eg, VB (FIG. 3)) is included in the plurality of input videos extracted by the extraction unit 24, the display control unit 26 is included in the omnidirectional video. Then, a virtual space image (for example, an image indicated by a symbol VB shown in FIG. 3) obtained by capturing a predetermined direction as the capturing direction from the capturing viewpoint of the omnidirectional camera that captured the omnidirectional image is acquired. The predetermined direction is a direction orthogonal to a plane having the same posture as the posture detected by the posture detection unit 91. Then, the display control unit 26 controls the display unit 21 to display the acquired virtual space video in the omnidirectional video display area DA (eg, DAB (FIG. 4)) set by the setting unit 25. .

  The display control unit 26 includes a omnidirectional video, and the method for acquiring the virtual space video obtained by shooting the predetermined direction from the shooting viewpoint as the shooting direction is not particularly limited. What is necessary is just to implement | achieve by the method similar to the method mounted in the software attached to a camera.

  FIGS. 7A and 7B are diagrams showing display examples of the virtual space images VBa and VBb and the one-sided video V4 in accordance with the posture of the video display area VDA. FIG. FIG. 7B is a diagram showing a display example before the change in the posture of the display area VDA, and FIG. 7B is a diagram showing a display example after the change in the posture of the video display region VDA.

  For example, as shown in FIG. 7A, the virtual space video VBa included in the omnidirectional video extracted by the extraction unit 24 is displayed as the omnidirectional video display area set by the setting unit 25 as a background video. It is assumed that the image is displayed on the entire video display area VDA which is DABa. The virtual space video VBa is shot by taking a direction orthogonal to a plane having the same posture as the posture of the video display area VDA detected by the posture detection unit 91 included in the omnidirectional video extracted by the extraction unit 24 as a shooting direction. It is the obtained image. Further, it is assumed that the input video V4 that is the one-sided video extracted by the extraction unit 24 is displayed in the display area DA4 set by the setting unit 25.

  In this case, the viewer rotates the video display area VDA in the right direction in FIG. 7A, and the back surface of the video display area VDA in FIG. 7A (the back surface in FIG. 7A). ) Is rotated so as to face the ground, and the posture of the video display area VDA is changed.

  In this case, the posture detection unit 91 detects the posture of the video display area VDA (FIG. 4) after the change. The display control unit 26 captures a direction orthogonal to the plane of the same posture as the posture of the changed video display area VDA detected by the posture detection unit 91 included in the omnidirectional video extracted by the extraction unit 24. As a result, a virtual space video VBb obtained by photographing is acquired. The display control unit 26 displays the acquired virtual space video VBb as a background video on the entire video display area VDA that is the display area DABa of the omnidirectional video.

  The selection unit 28 is configured by the microcomputer, and accepts selection of whether or not to move the display area DA in accordance with the change in the posture of the video display area VDA by the setting unit 25.

  For example, the area following information (length of FIG. 2) of the display area DA4 (FIG. 7A) of the input video V4 (FIG. 7A) included in the distribution information PF (FIG. 2) acquired by the acquisition unit 23. It is assumed that the broken line part) is “0”. In this case, the selection unit 28 accepts a selection by the setting unit 25 not to move the display area DA4 of the input video V4 in accordance with the change in the attitude of the video display area VDA.

  In this case, as shown in FIGS. 7A and 7B, the virtual space image VBa displayed in the image display area VDA is changed into the virtual space image VBb according to the change in the attitude of the image display area VDA. Even if the setting has been changed, the setting unit 25 does not move the display area DA4. As a result, the display area DA4 appears to move following the direction in which the viewer is facing.

  On the other hand, the area follow-up information (length of FIG. 2) of the display area DA4 (FIG. 7A) of the input video V4 (FIG. 7A) included in the distribution information PF (FIG. 2) acquired by the acquisition unit 23. It is assumed that the broken line portion is “1”. In this case, the selection unit 28 accepts a selection by the setting unit 25 to move the display area DA4 of the input video V4 in accordance with the change in the attitude of the video display area VDA.

  Note that the method by which the selection unit 28 receives the selection as to whether or not the setting unit 25 moves the display area DA according to the change in the posture of the video display area VDA is not limited to the above method. For example, when the viewer can perform the selection operation using the operation unit 22 and the viewer performs the selection operation performed using the operation unit 22, the selection unit 28 performs the selection operation. You may make it receive the result of selection by.

  FIG. 7C is a diagram showing a display example different from FIG. 7B after the posture of the video display area VDA is changed. In this case, as shown in FIGS. 7A and 7C, the setting unit 25 changes the virtual space video VBa displayed in the video display area VDA according to the change in the attitude of the video display area VDA. In accordance with the change to the virtual space video VBb, the display area DA4 is moved in a direction opposite to the direction of the change in the attitude of the video display area VDA by an amount corresponding to the change. As a result, the display area DA4 near the upper left corner of the video display area VDA moves outside the video display area VDA, and the input video V4 is not displayed.

  That is, in this case, the setting unit 25 integrates the display area DA4 with the virtual space image VBa (FIG. 7A) before changing the attitude of the image display area VDA, and the virtual space image VBa is integrated with the image display area VDA. The display area DA4 is moved so as to follow the movement in the direction opposite to the direction of the change according to the change in the posture.

  As described above, in the configuration of the first embodiment, whether or not to move each display area DA of the plurality of input videos is appropriately selected according to the change in the posture of the video display area VDA (FIG. 4). Can do.

  Returning to FIG. The viewpoint detection unit 92 includes the line-of-sight tracking sensor, and detects the viewer's viewpoint in the video display area VDA (FIG. 4).

  The video selection unit 20 is configured by the microcomputer, and accepts selection of one input video included in a plurality of input videos by the viewer. Specifically, the video selection unit 20 uses the operation unit 22 to select one input video from a plurality of input videos displayed in the video display area VDA. The selection of the input video is accepted.

  Alternatively, when the viewpoint detected by the viewpoint detection unit 92 exists in one display area DA in the video display area VDA, the video selection unit 20 displays one input displayed in the one display area DA. Accept video selection. For this reason, the viewer can easily select the input video only by moving the viewpoint to the display area DA where the input video is displayed.

  The application unit 29 is configured by the microcomputer, and when the selection of one input video is received by the video selection unit 20, the animation information included in the distribution information PF (FIG. 2) acquired by the acquisition unit 23. As indicated by the one-dot chain line portion in FIG. 2, the animation effect applied when the one input video is selected is applied to the input video of the application destination.

  For example, as shown in FIG. 4, it is assumed that input images V1, V2, and V3 are displayed in three display areas DA1, DA2, and DA3, respectively. In addition, as shown in the alternate long and short dash line in FIG. 2, when each of the three input videos V1, V2, and V3 indicated by the animation information included in the distribution information PF acquired by the acquisition unit 23 is selected. It is assumed that the animation effects applied to are “blinking”, “zoom-out”, and “blinking”. In addition, it is assumed that the input images to which the animation effect is applied when the input images V1, V2, and V3 indicated by the animation information are selected are the input images V1, V2, and VB, respectively. .

  In this case, it is assumed that selection of the input video V1 (V3) is accepted by the video selection unit 20. In this case, the application unit 29 applies the animation effect “flashing” to the input video V1 (VB). Specifically, the application unit 29 repeatedly performs the display of the input video V1 (VB) for a predetermined time after the input video V1 (VB) is displayed for a predetermined time.

  On the other hand, it is assumed that selection of the input video V2 is accepted by the video selection unit 20. In this case, the application unit 29 applies the animation effect “zoom out” to the input video V2. Specifically, the application unit 29 reduces the input video V2 at a predetermined reduction rate every predetermined time and hides the input video V2.

  Hereinafter, operations of the distribution server 1 and the viewer terminal 2 will be described. FIG. 8 is a flowchart showing the operation of the distribution server 1. FIG. 9 is a flowchart showing the initial display operation of the distribution video V in the viewer terminal 2. FIG. 10 is a flowchart showing an operation after the initial display of the distribution video V in the viewer terminal 2.

(Operation of distribution server 1)
As shown in FIG. 8, in the distribution server 1, video data representing a plurality of input videos (eg, VB, V1, V2, V3 (FIG. 3)) included in the distribution video V (FIG. 3) from the video output unit 11. Is input to the generation unit 13 (S11), the generation unit 13 acquires distribution information PF (FIG. 2) related to the distribution video V stored in the storage unit 12 (S12).

  The generation unit 13 arranges each of a plurality of input videos (eg, VB, V1, V2, V3 (FIG. 3)) included in the distribution information PF (FIG. 2) acquired in S12 (FIG. 2). The distribution video V (FIG. 3) is generated by arranging and synthesizing in the arrangement area (example: PAB, PA1, PA2, PA3 (FIGS. 2 and 3)) indicated by the broken line portion (S13).

  Next, the distribution unit 14 generates video data representing the distribution video V (FIG. 3) generated in S13, the video data representing the generated distribution video V, and the distribution video V stored in the storage unit 12. The distribution information PF (FIG. 2) related to is distributed via the network (S14).

(Operation of the viewer terminal 2)
As shown in FIG. 9, in the viewer terminal 2, the acquisition unit 23 acquires the distribution information PF (FIG. 2) related to the distribution video V (FIG. 3) distributed by the distribution unit 14 (S21), and distributes it. Video data representing the video V is acquired (S22).

  The extraction unit 24 arranges the arrangement area PA indicated by the arrangement information (broken line part in FIG. 2) included in the distribution information PF acquired in S21 from the distribution video V generated by decoding the video data acquired in S22. (Example: PAB, PA1, PA2, PA3 (Figs. 2 and 3)), each of a plurality of input videos (example: VB, V1, V2, V3 (Fig. 3)) is divided individually. To extract (S23).

  The setting unit 25 uses the initial display information (two-dot chain line portion in FIG. 2) included in the distribution information PF (FIG. 2) acquired in S21 to extract in S23 in the video display area VDA (FIG. 4). Each display area DA (eg, DAB, DA1, DA2, DA3 (FIG. 4)) of the plurality of input videos (eg: VB, V1, V2, V3 (FIG. 3)) is set (S24). For this reason, in the viewer terminal 2, the effort which sets individually the display area DA of each of the some input video extracted by S23 can be reduced.

  The display control unit 26 controls the display unit 21 to display each of the plurality of input videos extracted in S23 in each display area DA of the plurality of input videos set in S24 (S25). .

  After S25, when the receiving unit 27 receives an editing operation for each display area DA (eg, DAB, DA1, DA2, DA3 (FIG. 4)) of a plurality of input videos (S26; YES), setting The unit 25 updates the display area DA set before accepting the editing operation with the display area DA edited by the editing operation accepted in S26 (S27). For this reason, the viewer edits the display area DA of each of the plurality of input videos extracted in S23 on the viewer terminal 2, so that the plurality of input videos are not burdened on the distribution server 1. Each display area DA can be reorganized.

  The display control unit 26 controls the display unit 21 to display each of the plurality of input videos extracted in S23 in each display area DA of the plurality of input videos updated in S27 (S28). .

  As shown in FIG. 10, when the posture of the video display area VDA (FIG. 4) detected by the posture detection unit 91 is changed (S31; YES), the omnidirectional video ( Example: Assume that VBa (FIG. 7A)) is included (S32; YES).

  In this case, as described above, the display control unit 26 controls the display unit 21 to change the predetermined direction from the shooting viewpoint of the omnidirectional camera that captured the omnidirectional video included in the omnidirectional video. A virtual space video (eg, VBb (FIG. 7B, FIG. 7C)) obtained by shooting as the shooting direction is displayed in the display area DA (eg, DABa (FIG. 7B) of the omnidirectional video. 7C)) (S33).

  As a result, the viewer can view the same image as the image visually recognized when the head posture is changed in the same manner as the posture of the video display area VDA at the omnidirectional video shooting location. Can be visually recognized.

  After S28 (FIG. 9), the attitude of the video display area VDA detected by the attitude detector 91 has changed (S31; YES), but the omnidirectional video is not included in the plurality of input videos extracted in S23. In the case (S32; NO) and after S33, the selection unit 28 causes the setting unit 25 to move the display area DA of the input video extracted in S23 according to the change in the attitude of the video display area VDA. A selection of whether or not is accepted (S34).

  In S34, the selection unit 28 includes the input video (eg, V4 (FIG. 7A)) extracted in S23 included in the distribution information PF (FIG. 2) acquired in S21 (FIG. 9). Based on the area following information (long broken line part in FIG. 2) of the display area DA (eg, DA4 (FIG. 7A)), the setting unit 25 causes the display area DA to change according to the change in the posture of the video display area VDA. It is assumed that selection of movement is accepted (S34; YES). In this case, the setting unit 25 moves the display area DA according to the change in the posture of the video display area VDA (see FIG. 7C) (S35). Thereafter, the processing after S36 is performed.

  After S28 (FIG. 9), when the attitude of the video display area VDA detected by the attitude detection unit 91 has not changed (S31; NO), and in S34, the selection unit 28 causes the setting unit 25 to display the display area. When the selection not to move the DA in accordance with the change in the posture of the video display area VDA is received (S34; NO), the processes after S36 are performed.

  In S36, the video selection unit 20 starts accepting selection of one input video by the viewer (S36). When the video selection unit 20 accepts selection of one input video in S36 (S36; YES), the application unit 29 adds the one of the distribution information PF (FIG. 2) acquired in S21 (FIG. 9) to the one. It is determined whether or not animation information corresponding to the input video (dotted line portion in FIG. 2) is included (S37).

  When the application unit 29 determines in S37 that the animation information (the one-dot chain line portion in FIG. 2) corresponding to the one input video (example: V3 (FIG. 2)) is included (S37; YES), As the animation information indicates, an animation effect (eg, blinking (FIG. 2)) is applied to an input video (eg, VB (FIG. 2)) of a predetermined application destination (S38).

  Thus, when one input video is selected by the viewer, the animation effect is applied to the input video of a predetermined application destination as indicated by the animation information included in the distribution information PF. For this reason, the viewer can interactively operate the input video of the application destination simply by moving the viewpoint to one input video or selecting and operating the one input video.

  When the video selection unit 20 does not accept selection of one input video in S36 (S36; NO), the application unit 29 determines in S37 that animation information corresponding to the one input video is not included (S37; NO) and after the execution of S38, the viewer does not perform a predetermined end instruction input operation for ending viewing of the distribution video V using the operation unit 22 (S39; NO). , S26 (FIG. 9) and subsequent processes are performed. On the other hand, when the viewer uses the operation unit 22 to input the end instruction (S39; YES), the operations shown in FIGS. 9 and 10 in the viewer terminal 2 are ended.

  As described above, according to the configuration of the first embodiment, a plurality of input videos are individually divided and extracted using the distribution video V and the distribution information PF acquired from the distribution server 1 in the viewer terminal 2. Then, each of the extracted plurality of input videos is displayed in the display area DA set by the setting unit 25 provided in the viewer terminal 2. For this reason, unlike the above-described conventional technique, the display area DA of the plurality of input videos included in the distribution video V on the viewer terminal 2 is set on the viewer terminal 2 without imposing a burden on the distribution server 1. It is possible to quickly reorganize the area DA.

(Second embodiment)
Hereinafter, a video distribution system according to a second embodiment of the present invention will be described with reference to the drawings. In addition, in 2nd embodiment, the thing of the same structure as 1st embodiment attaches | subjects the same code | symbol, and abbreviate | omits description.

(Configuration of video distribution system 100a)
FIG. 11 is a block diagram showing an example of a functional configuration of the video distribution system 100a according to the second embodiment of the present invention. FIG. 12 is a diagram illustrating an example of the configuration of the distribution sound A. FIG. 13 is a diagram showing an example of distribution information PFa according to the second embodiment of the present invention. As shown in FIG. 11, the video distribution system 100a according to the second embodiment includes a distribution server 1a and a viewer terminal 2a. The distribution server 1a and the viewer terminal 2a are communicably connected via a network (not shown) such as a LAN, the Internet, or a broadcast data network.

(Configuration and operation of distribution server 1a)
The distribution server 1a includes distribution video V (FIG. 3) described in the first embodiment, distribution information PFa (FIG. 13) having a configuration different from the distribution information PF (FIG. 2) described in the first embodiment, and distribution. The distribution audio A (FIG. 12) including one or more input audios that can be reproduced during the display of the video V is distributed. Specifically, the distribution server 1a includes a plurality of video output units 11 and storage units 12, one or more audio output units 15, a generation unit 13a, and a distribution unit 14a described in the first embodiment. ing.

  The audio output unit 15 outputs audio data representing audio that can be reproduced while the distribution video V is displayed to the generation unit 13a. Specifically, the audio output unit 15 collects audio around the camera constituting the video output unit 11 and outputs a sound data representing the collected audio, and audio editing operation is possible. And a voice editing device such as a personal computer that can output voice data representing the edited voice. Hereinafter, the voice represented by the voice data output from the voice output unit 15 and input to the generation unit 13a is referred to as input voice.

  In the second embodiment, the distribution information PFa (FIG. 13) related to the distribution video V stored in the storage unit 12 further includes audio recording information (dashed line portion) and reproduction information (two-dot chain line portion). ing. The audio recording information is information that defines an audio recording area in which each of one or more input sounds that can be reproduced during display of the distribution video V is recorded. The reproduction information is information that defines a reproduction method for each of the one or more input sounds. Further, in the distribution information PFa, each of the one or more input sounds is associated with one input video among a plurality of input videos included in the distribution video V.

  For example, the storage unit 12 stores the distribution information PFa shown in FIG. 13 including the audio recording information and the reproduction information related to the four input sounds AB, A1, A2, and A3 included in the distribution audio A shown in FIG. It shall be remembered.

  As shown in FIG. 12, it is assumed that the distribution sound A includes four input sounds AB, A1, A2, and A3 that can be reproduced while the distribution video V (FIG. 3) is displayed. The input audio AB is assumed to be 6-channel audio collected around the omnidirectional camera that captures the input video VB (FIG. 3). That is, the input audio AB is assumed to be audio associated with the input video VB (FIG. 3). The three input voices A1, A2, and A3 are assumed to be two-channel (stereo) voices collected around the camera that captures the input videos V1, V2, and V3 (FIG. 3), respectively. That is, the three input voices A1, A2, and A3 are voices associated with the input videos V1, V2, and V3 (FIG. 3), respectively.

  Further, it is assumed that the distribution sound A has three sound tracks AT0, AT1, and AT2. The audio track AT0 records 6-channel input audio AB, the audio track AT1 records 2-channel input audio A1 and 2-channel input audio A2, and the audio track AT2 stores 2 channels. It is assumed that the input voice A3 is recorded.

  The delivery sound A shown in FIG. 12 is merely an example. For example, the distribution audio A has only one audio track, and records only one channel monaural input audio associated with the input image as the background image or sub-image on the audio track. Also good.

  As described above, when the distribution sound A is configured as shown in FIG. 12, the distribution information PFa includes four input sounds AB, A1, A2, A3 (see FIG. 13). 12) includes audio recording information defining audio recording areas RAB, RA1, RA2, and RA3 (FIG. 12) for recording each of the above. Hereinafter, when each voice recording area of one or more input voices is generically referred to as a voice recording area RA.

  Further, as indicated by the two-dot chain line portion in FIG. 13, the distribution information PFa includes reproduction information that defines the respective reproduction methods “audioPlayMode” of the four input voices AB, A1, A2, and A3 (FIG. 12). Is included.

  As shown in FIG. 13, in the distribution information PFa, an audio recording area RA (eg, RA2) for recording an input audio (eg, input audio A2) and a playback method “audioPlayMode” (eg, SelectedOnly) of the input audio are: By being described in parentheses following identification information (eg, V2) of one input video (eg, input video V2 (FIG. 3)) associated with the input audio, it is associated with the one input video. It has been. Thereby, in the distribution information PFa, each of one or more input sounds (for example, input sound A2 (FIG. 12)) and one input image (for example, input image V2 (FIG. 3)) of the plurality of input images. ) Is defined.

  An audio recording area RA (eg, RA2) that records input audio (eg, A2 (FIG. 12)) includes identification information “audioTrackName” (eg, AT1) of the audio track that records the input audio, and the input audio. The number of the first channel of the one or more channels (hereinafter referred to as the start channel index) “audioChannelNumber” (eg, 2) and the type of the input audio when recording on one or more channels in the audio track “AudioType” (for example, Stereo).

  As in the audio recording areas RAB and RA3 shown in FIG. 13, the input audio start channel index “audioChannelNumber” can be determined by “−1”. This indicates that the input sound is recorded on all channels in the sound track.

  The type of audio input “audioType” is determined so that the number of channels of input audio can be identified. Specifically, when the input audio type “audioType” is “Stereo”, it is determined that the input audio type is two-channel stereo audio. For this reason, it can be identified that the number of channels of the input sound is two. When the input audio type “audioType” is “Spacial-6”, it is determined that the input audio type is 6-channel special audio. In this case, since the number “6” is included in the type of the input voice, it can be identified that the number of channels of the input voice is six. However, the present invention is not limited to this, and the type “audioType” of the input voice (eg, AB) may be determined by the number of channels (eg, “6”) of the input voice (eg, “audioType”: “6”).

  The input audio playback method “audioPlayMode” is one of three predetermined playback methods “Normal” (first playback method), “SelectedMix” (second playback method), and “SelectedOnly” (third playback method). Or one playback method.

  Specifically, it is assumed that the playback method “audioPlayMode” of the input sound is defined as the playback method “Normal”. In this case, when the input video (first input video) associated with the input voice (first input voice) is displayed in the video display area VDA (FIG. 4), the input voice is reproduced. .

  For example, in the reproduction information shown in FIG. 13, the reproduction method “audioPlayMode” of the input sound AB is defined as the reproduction method “Normal”. That is, when the input video VB (FIG. 3) associated with the input audio AB (FIG. 12) is displayed in the video display area VDA (FIG. 4), it is determined that the input audio AB is reproduced.

  Assume that the playback method “audioPlayMode” of the input sound is defined as the playback method “SelectedMix”. In this case, when the selection of the input video (second input video) associated with the input audio (second input audio) is received by the video selection unit 20 (FIG. 11), the input audio is predetermined. Is played at the first volume. In addition, among the one or more input sounds, an input sound that is determined to be reproduced by the reproduction method “Normal” in the reproduction information is reproduced at a second volume smaller than the first volume.

  For example, in the reproduction information shown in FIG. 13, the reproduction method “audioPlayMode” of the input audio AB is defined as the reproduction method “Normal”, and the reproduction method “audioPlayMode” of the input audio A3 is defined as the reproduction method “SelectedMix”. That is, when selection of the input video V3 (FIG. 3) associated with the input audio A3 is accepted by the video selection unit 20 (FIG. 1), the input audio A3 is reproduced at a predetermined first volume, and the input audio It is determined that AB is reproduced at a second volume that is smaller than the first volume.

  Assume that the playback method “audioPlayMode” of the input sound is defined as the playback method “SelectedOnly”. In this case, when the selection of the input video (third input video) associated with the input audio (third input audio) is received by the video selection unit 20 (FIG. 1), only the input audio is received. Played.

  For example, in the reproduction information shown in FIG. 13, the reproduction method “audioPlayMode” of the input audio AB is defined as the reproduction method “Normal”, and the reproduction method “audioPlayMode” of the input audio A1 (A2) is defined as the reproduction method “SelectedOnly”. ing. That is, when selection of the input video V1 (V2) associated with the input audio A1 (A2) is accepted by the video selection unit 20, the volume of the input audio AB is set to 0 (muted), and the input audio is It is determined that only A1 (A2) is reproduced.

  The generation unit 13a includes a switcher and a mixer. In S13 (FIG. 8), the generation unit 13a generates the distribution video V (FIG. 3) as described above, and further, one or more input sounds (eg, AB, A1, A2, A3 (FIG. 12)). Are recorded in the audio recording area (example: RAB, RA1, RA2, RA3) indicated by the audio recording information (broken line portion in FIG. 13) included in the distribution information PFa (FIG. 12) stored in the storage unit 12. (FIG. 12)), and one distribution sound A (FIG. 12) having a sound recording area (for example, RAB, RA1, RA2, RA3 (FIG. 12)) in which each of one or more input sounds is recorded. Is generated.

  The distribution unit 14a includes an encoder and a communication interface circuit that performs communication between the viewer terminal 2a via the network. In S14 (FIG. 8), the distribution unit 14a generates video data representing the distribution video V (FIG. 3) generated in S13 as described above, and further, the distribution audio A (FIG. 12) generated in S13. Is encoded by a predetermined method (eg, MP3, AAC, etc.) to generate audio data representing the distribution audio A (FIG. 12).

  The distribution unit 14a then generates video data representing the generated distribution video V, audio data representing the generated distribution audio A, and distribution information PFa related to the distribution video V stored in the storage unit 12 (FIG. 13). Are distributed via the network.

(Configuration and operation of viewer terminal 2a)
On the other hand, the viewer terminal 2a acquires the video data representing the distribution video V and the distribution information PFa related to the distribution video V from the distribution server 1a, and further acquires the distribution audio A. As described above, the viewer terminal 2a displays the distribution video V represented by the acquired video data on the predetermined video display area VDA (FIG. 4) based on the acquired distribution information PF, and further acquires the acquired distribution information. Based on the PF, the distribution sound A is reproduced.

  Specifically, the viewer terminal 2a includes the information communication apparatus described in the first embodiment, speakers such as surround speakers and headphones, and audio of one or more channels recorded in one or more audio tracks. And a sound output device capable of simultaneously outputting to the speaker at a predetermined volume.

  As shown in FIG. 11, the viewer terminal 2a includes the display unit 21, the operation unit 22, the setting unit 25, the display control unit 26, the reception unit 27, the selection unit 28, the application unit 29, and the video described in the first embodiment. It functions as the selection unit 20, the posture detection unit 91, and the viewpoint detection unit 92. Further, the viewer terminal 2a further functions as an acquisition unit 23a, an extraction unit 24a, and a playback unit 31.

  The acquisition unit 23a is configured by the microcomputer and the communication interface circuit. The acquisition unit 23a communicates with the distribution server 1a to acquire the distribution information PFa (FIG. 12) distributed by the distribution unit 14a in S21 (FIG. 9). In S22 (FIG. 9), the distribution video V Is acquired, and further audio data representing the distribution audio A is acquired.

  The extraction unit 24a is configured by the decoder and the microcomputer. In S23 (FIG. 9), the extraction unit 24a individually divides and extracts each of the plurality of input videos included in the distribution video V as described above, and is further acquired in S22 (FIG. 9). The voice data is decoded to generate the distribution voice A (FIG. 12). Then, the extraction unit 24a performs the audio recording indicated by the audio recording information included in the distribution information PFa (FIG. 12) acquired by the acquisition unit 23a in S22 (FIG. 9) from the generated distribution audio A (FIG. 12). Each of one or more input voices (for example, AB, A1, A2, A3 (FIG. 12)) recorded in the area RA (for example, RAB, RA1, RA2, RA3 (FIG. 12, FIG. 13)) Separate and extract individually.

  The reproduction unit 31 includes the speaker and the audio output device. The reproduction unit 31 acquires, in S22 (FIG. 9), one or more input voices extracted by the extraction unit 24a in S23 (FIG. 9) after S25 (FIG. 9) and before execution of S26 (FIG. 9). Input audio (for example, AB ()) that is determined to be reproduced by the reproduction method “Normal” based on reproduction information (two-dot chain line portion in FIG. 13) included in the distribution information PFa (FIG. 13) acquired by the unit 23a. FIG. 12)) is reproduced.

  According to this configuration, for example, when an input video (for example, VB (FIGS. 2 and 3)) defined as a background video is included in the distribution video V (FIG. 3), it is associated with the input video. If the distribution information PFa includes reproduction information that determines that the input audio (eg, AB (FIG. 12)) is reproduced by the reproduction method “Normal”, the input audio is displayed as the background video. If you can play it.

  In S36 (FIG. 10), it is assumed that selection of one input video by the viewer is accepted by the video selection unit 20 (S36; YES). The one input video is reproduced by the reproduction method “SelectedMix” by the reproduction information (two-dot chain line portion in FIG. 13) included in the distribution information PFa (FIG. 13) acquired in S22 (FIG. 9). It is assumed that the input video (for example, V3 (FIG. 3)) corresponding to the input voice (for example, A3 (FIG. 12)) determined to be performed.

  In this case, the playback unit 31 plays back the input audio (eg, A3 (FIG. 12)) corresponding to the input video (eg: V3 (FIG. 3)) selected in S36 at a predetermined first volume, and S25 ( The volume of the input sound (eg, AB (FIG. 12)) that is determined to be played back by the playback method “Normal” played after S26 (FIG. 9) and after the execution of S26 (FIG. 9) is determined from the first volume. Also change to a lower second volume.

  According to this configuration, for example, a first input video (for example, VB (FIGS. 2 and 3)) defined as a background video and a second input defined as a sub video displayed superimposed thereon Assume that a video (example: V3 (FIGS. 2 and 3)) is included in the distribution video V. In this case, it is determined that the first input sound (for example, AB (FIG. 12)) associated with the first input image is reproduced by the reproduction method “Normal”, and is associated with the second input image. In addition, the distribution information PFa (FIG. 13) can include reproduction information that determines that the second input voice (eg, A3 (FIG. 12)) is reproduced by the reproduction method “SelectedMix”.

  Thereby, the viewer can hear the first input sound when the first input video is displayed as the background video. Thereafter, the viewer selects the second input video (for example, V3 (FIG. 13)), so that the second input video associated with the second input video is louder than the first input audio. The input voice (example: A3 (FIG. 12)) is reproduced, and the second input voice can be made easier to hear than the first input voice (example: AB (FIG. 12)).

  In S36 (FIG. 10), it is assumed that selection of one input video by the viewer is accepted by the video selection unit 20 (S36; YES). Then, the one input video is reproduced by the reproduction method “SelectedOnly” by the reproduction information (two-dot chain line portion in FIG. 13) included in the distribution information PFa (FIG. 13) acquired in S22 (FIG. 9). It is assumed that the input video (for example, V1 (V2) (FIG. 3)) corresponding to the input voice (for example: A1 (A2) (FIG. 12)) determined to be performed.

  In this case, the playback unit 31 plays back the input sound (eg, AB (FIG. 12) that is determined to be played back by the playback method “Normal” played after S25 (FIG. 9) and before the execution of S26 (FIG. 9). )) Is set to 0 (muted), and the input audio corresponding to the input video (eg, V1 (V2) (FIG. 3)) selected in S36 (eg: A1 (A2) (FIG. 12)) Only play at a predetermined volume.

  According to this configuration, for example, a first input video (for example, VB (FIGS. 2 and 3)) defined as a background video and a third input defined as a sub video displayed superimposed thereon Assume that videos (eg, V1 and V2 (FIGS. 2 and 3)) are included in the distribution video V. In this case, it is determined that the first input sound (eg, AB (FIG. 12)) associated with the first input image is reproduced by the reproduction method “Normal”, and is associated with the third input image. In addition, the distribution information PFa (FIG. 13) can include reproduction information that determines that the third input sound (eg, A1, A2 (FIG. 12)) is reproduced by the reproduction method “SelectedOnly”.

  Thereby, the viewer can hear the first input sound when the first input video is displayed as the background video. Thereafter, the viewer selects only the third input video (eg, V1, V2 (FIGS. 2 and 3)), thereby reproducing only the third input sound (eg, A1, A2 (FIG. 12)). And focus on the third input voice.

  Thus, according to the configuration of the second embodiment, the distribution video V (FIG. 3) is obtained using the distribution audio A (FIG. 12) and the distribution information PFa (FIG. 13) acquired from the distribution server 1a in the viewer terminal 2a. ), One or more input sounds (for example, AB, A1, A2, A3 (FIG. 12)) that can be reproduced while being displayed are individually divided and extracted. Then, each of the extracted one or more input voices is reproduced by the reproduction method indicated by the reproduction information included in the distribution information PFa. For this reason, each of the one or more input voices can be reproduced by a reproduction method specific to each input voice without placing a burden on the distribution server 1a.

(Modified embodiment)
In addition, said 1st and 2nd embodiment is only the illustration of embodiment which concerns on this invention, and is not the meaning which limits this invention to said 1st and 2nd embodiment. For example, the following modified embodiment may be used.

  (1) The viewer terminals 2 and 2a may not function as the viewpoint detection unit 92 (FIGS. 1 and 11). That is, the video selection unit 20 (FIGS. 1 and 11) uses the operation unit 22 (FIGS. 1 and 11) and the viewer selects one of the plurality of input videos displayed in the video display area VDA. The selection of the one input video may be accepted only when the input video is selected.

  (2) The reproduction method “SelectedOnly” may not be determined in the reproduction information (two-dot chain line portion in FIG. 13). Furthermore, the reproduction method “SelectedMix” may not be defined in the reproduction information.

  (3) The viewer terminals 2 and 2a may not function as the video selection unit 20 and the application unit 29 (FIGS. 1 and 11). In accordance with this, the distribution information PF (FIG. 2) and PFa (FIG. 13) may not include animation information (the chain line portion of FIG. 2), and S36 to S38 (FIG. 10) may be omitted.

  (4) The viewer terminals 2 and 2a may be configured not to function as the selection unit 28 (FIGS. 1 and 11). Accordingly, the distribution information PF (FIG. 2) and PFa (FIG. 13) may not include the area following information (the long broken line portion in FIG. 2), and S34 and S35 (FIG. 10) may be omitted. .

  (5) The viewer terminals 2 and 2a may not function as the posture detection unit 91 (FIGS. 1 and 11). In accordance with this, S31, S32 and S33 (FIG. 10) may be omitted.

  (6) The initial display information (two-dot chain line portion in FIG. 2) may not be included in the distribution information PF (FIG. 2) and PFa (FIG. 13). In accordance with this, for example, in S24 (FIG. 9), the setting unit 25 divides the video display area VDA evenly by the number of input videos, and each divided area is automatically set as the display area DA of each input video. You may make it set to. Alternatively, the arrangement area PA (eg, PA1 (FIG. 2)) indicated by the arrangement information (broken line portion in FIG. 2) included in the distribution information PF and PFa acquired in S21 (FIG. 9) is the video display area VDA. In FIG. 4, the setting unit 25 may set the display area DA as the display area DA, assuming that the display area DA of each input video is shown.

  That is, the arrangement area PA (eg, the arrangement area PA1 (clipX “0”, clipY “6668”, clipW “3333”, clipY “3333”) (FIG. 2)) is set to the upper left corner of the video display area VDA (FIG. 4). Is a reference point, the horizontal position in the video display area VDA (FIG. 4) is indicated by an X coordinate, and the vertical position in the video display area VDA (FIG. 4) is determined using a two-dimensional coordinate. The display area DA may be substituted.

  Alternatively, the setting unit 25 controls the display unit 21 in S24 (FIG. 9) to guide the viewer to perform an editing operation on each display area DA of a plurality of input videos (for example, “ Please edit the display area of the input video. ”) May be displayed in the video display area VDA (FIG. 4). And you may make it perform the same process as S26 and S27 (FIG. 9).

  (7) The distribution server 1 (1a) and the viewer terminal 2 (2a) may not be able to communicate with each other via the network. Accordingly, the distribution unit 14 (14a) is configured by an encoder and a recording device that records data and information on a recording medium such as a DVD or a BD (Blu-ray Disc), and the distribution unit 14 (14a) In FIG. 8), the distribution video V and distribution information PF (distribution video V, distribution information PFa, and distribution audio A) may be recorded on a recording medium. Then, the provider of the distribution video V may provide the recording medium to the viewer terminal 2 (2a) by mail or the like. Accordingly, the distribution video V and the distribution information PF (distribution video V, distribution information PFa, and distribution audio A) may be distributed on the recording medium.

  In this case, the acquisition unit 23 (23a) may be configured by the microcomputer and an acquisition device that acquires data and information from a recording medium such as a DVD or a BD (Blu-ray disc). Then, in S21 and S22 (FIG. 9), the acquisition unit 23 (23a) receives the distribution video V and the distribution information PF (distribution video V, distribution information PFa, and distribution audio A from the recording medium provided by the provider. ).

100, 100a Video distribution system 1, 1a Distribution server 12 Storage unit 13, 13a Generation unit 14, 14a Distribution unit 2, 2a Viewer terminal 20 Video selection unit 21 Display unit 23, 23a Acquisition unit 24, 24a Extraction unit 25 Setting unit 26 Display Control Unit 27 Reception Unit 28 Selection Unit 29 Application Unit 31 Playback Unit 91 Attitude Detection Unit 92 Viewpoint Detection Unit DA Display Area PA Arrangement Area PF, PFa Distribution Information V Distribution Video VB, V1 to V4 Input Video VA Video Area VBa, VBb Virtual space video VDA video display area A Distribution audio AB, A1-A4 Input audio RA Audio recording area

Claims (11)

A video distribution system comprising a distribution server and a viewer terminal for arranging and synthesizing a plurality of input videos in one video area and distributing the distribution video represented by the entire video area,
The distribution server
A storage unit for storing distribution information including arrangement information defining respective arrangement areas of the plurality of input videos in the video area;
Each of the plurality of input videos is arranged in the arrangement area indicated by the arrangement information and synthesized, and a generation unit that generates the distribution video represented by the entire video area;
A distribution unit for distributing the distribution video and the distribution information;
With
The viewer terminal is
A display unit for displaying video in a predetermined video display area;
An acquisition unit for acquiring the distribution video and the distribution information distributed by the distribution unit;
Each of the plurality of input videos that are arranged in the arrangement area indicated by the arrangement information included in the distribution information acquired by the acquisition unit from the distribution video acquired by the acquisition unit individually An extraction unit for dividing and extracting;
A setting unit for setting each display area of the plurality of input videos in the video display area;
A display control unit that controls the display unit to display each of the plurality of input videos extracted by the extraction unit in the display area set by the setting unit;
A reception unit that receives an editing operation of the display area by a viewer;
Equipped with a,
The setting unit updates the display area set before accepting the editing operation by the accepting unit with the display area edited by the editing operation accepted by the accepting unit,
The video distribution system includes an operation of deleting the display area in the editing operation of the display area . A video distribution system comprising a distribution server and a viewer terminal for arranging and synthesizing a plurality of input videos in one video area and distributing the distribution video represented by the entire video area,
The distribution server
A storage unit for storing distribution information including arrangement information defining respective arrangement areas of the plurality of input videos in the video area;
Each of the plurality of input videos is arranged in the arrangement area indicated by the arrangement information and synthesized, and a generation unit that generates the distribution video represented by the entire video area;
A distribution unit for distributing the distribution video and the distribution information;
With
The viewer terminal is
A display unit for displaying video in a predetermined video display area;
An acquisition unit for acquiring the distribution video and the distribution information distributed by the distribution unit;
Each of the plurality of input videos that are arranged in the arrangement area indicated by the arrangement information included in the distribution information acquired by the acquisition unit from the distribution video acquired by the acquisition unit individually An extraction unit for dividing and extracting;
A setting unit for setting each display area of the plurality of input videos in the video display area;
A display control unit that controls the display unit to display each of the plurality of input videos extracted by the extraction unit in the display area set by the setting unit;
A posture detection unit that detects a posture of the video display region intended by the viewer;
Equipped with a,
The plurality of input videos include an omnidirectional video obtained by shooting omnidirectional from a predetermined shooting viewpoint,
The display control unit is a virtual space obtained by photographing a direction perpendicular to a plane having the same posture as the posture detected by the posture detection unit from the photographing viewpoint, which is included in the omnidirectional video. A video distribution system that controls the display unit to display video in the display area of the omnidirectional video . A video distribution system comprising a distribution server and a viewer terminal for arranging and synthesizing a plurality of input videos in one video area and distributing the distribution video represented by the entire video area,
The distribution server
A storage unit for storing distribution information including arrangement information defining respective arrangement areas of the plurality of input videos in the video area;
Each of the plurality of input videos is arranged in the arrangement area indicated by the arrangement information and synthesized, and a generation unit that generates the distribution video represented by the entire video area;
A distribution unit for distributing the distribution video and the distribution information;
With
The viewer terminal is
A display unit for displaying video in a predetermined video display area;
An acquisition unit for acquiring the distribution video and the distribution information distributed by the distribution unit;
Each of the plurality of input videos that are arranged in the arrangement area indicated by the arrangement information included in the distribution information acquired by the acquisition unit from the distribution video acquired by the acquisition unit individually An extraction unit for dividing and extracting;
A setting unit for setting each display area of the plurality of input videos in the video display area;
A display control unit that controls the display unit to display each of the plurality of input videos extracted by the extraction unit in the display area set by the setting unit;
A selection unit that receives a selection as to whether or not to move the display area in accordance with a change in posture of the video display area by the setting unit;
A video distribution system. A video distribution system comprising a distribution server and a viewer terminal for arranging and synthesizing a plurality of input videos in one video area and distributing the distribution video represented by the entire video area,
The distribution server
A storage unit for storing distribution information including arrangement information defining respective arrangement areas of the plurality of input videos in the video area;
Each of the plurality of input videos is arranged in the arrangement area indicated by the arrangement information and synthesized, and a generation unit that generates the distribution video represented by the entire video area;
A distribution unit for distributing the distribution video and the distribution information;
With
The viewer terminal is
A display unit for displaying video in a predetermined video display area;
An acquisition unit for acquiring the distribution video and the distribution information distributed by the distribution unit;
Each of the plurality of input videos that are arranged in the arrangement area indicated by the arrangement information included in the distribution information acquired by the acquisition unit from the distribution video acquired by the acquisition unit individually An extraction unit for dividing and extracting;
A setting unit for setting each display area of the plurality of input videos in the video display area;
A display control unit that controls the display unit to display each of the plurality of input videos extracted by the extraction unit in the display area set by the setting unit;
Equipped with a,
The distribution information further includes animation information defining an animation effect to be applied when each of the plurality of input videos is selected and an input video of the application destination,
The viewer terminal is
A video selection unit that receives selection of one input video included in the plurality of input videos by a viewer;
When the selection of the one input video is accepted by the video selection unit, the one input video is selected as indicated by the animation information included in the distribution information acquired by the acquisition unit. An application unit that applies the animation effect applied to a case to the input video of the application destination;
A video distribution system further comprising: A video distribution system comprising a distribution server and a viewer terminal for arranging and synthesizing a plurality of input videos in one video area and distributing the distribution video represented by the entire video area,
The distribution server
A storage unit for storing distribution information including arrangement information defining respective arrangement areas of the plurality of input videos in the video area;
Each of the plurality of input videos is arranged in the arrangement area indicated by the arrangement information and synthesized, and a generation unit that generates the distribution video represented by the entire video area;
A distribution unit for distributing the distribution video and the distribution information;
With
The viewer terminal is
A display unit for displaying video in a predetermined video display area;
An acquisition unit for acquiring the distribution video and the distribution information distributed by the distribution unit;
Each of the plurality of input videos that are arranged in the arrangement area indicated by the arrangement information included in the distribution information acquired by the acquisition unit from the distribution video acquired by the acquisition unit individually An extraction unit for dividing and extracting;
A setting unit for setting each display area of the plurality of input videos in the video display area;
A display control unit that controls the display unit to display each of the plurality of input videos extracted by the extraction unit in the display area set by the setting unit;
Equipped with a,
The distribution information includes audio recording information defining an audio recording area for recording each of one or more input sounds that can be reproduced during display of the distribution video, and a method for reproducing each of the one or more input sounds. Further including reproduction information that has been defined,
The generation unit further records each of the one or more input voices in the voice recording area indicated by the voice recording information, and stores the voice recording area in which each of the one or more input voices is recorded. Has one distribution voice,
The delivery unit further delivers the delivery audio,
The acquisition unit further acquires the delivery audio,
The extraction unit is further recorded in the audio recording area indicated by the audio recording information included in the distribution information acquired by the acquisition unit from the distribution audio acquired by the acquisition unit. Each of the above input speech is divided and extracted individually,
The viewer terminal is
A reproduction unit that reproduces each of the one or more input sounds extracted by the extraction unit by the reproduction method indicated by the reproduction information included in the distribution information acquired by the acquisition unit;
In the distribution information, each of the one or more input sounds is associated with one input video of the plurality of input videos,
In the reproduction method, the first input sound is reproduced when a first input image associated with the first input sound among the one or more input sounds is displayed in the image display area. A video distribution system including a first reproduction method . The distribution information further includes initial display information that defines an initial arrangement of the display area,
The said setting part sets the said display area as the said initial arrangement | positioning which the said initial display information contained in the said delivery information acquired by the said acquisition part shows as described in any one of Claim 1 to 5 Video distribution system. Wherein the editing operation of the display area, the video delivery system of claim 1 including the operation of enlarging and reducing the display area. The video distribution system according to claim 1 or 7 , wherein the editing operation of the display area includes an operation of moving the display area. The viewer terminal is
A video selection unit that accepts selection of one input video included in the plurality of input videos by a viewer;
In the reproduction method, when the selection of a second input video associated with a second input audio among the one or more input audios is received by the video selection unit, the second input audio is received. Of the one or more input sounds, the input sound that is determined to be reproduced by the first reproduction method in the reproduction information is a second volume smaller than the first volume. The video delivery system according to claim 5 , wherein a second playback method of playing back with the method is included. The viewer terminal is
A video selection unit that accepts selection of one input video included in the plurality of input videos by a viewer;
In the reproduction method, when the selection of a third input video associated with a third input audio among the one or more input audios is received by the video selection unit, the third input audio is received. 10. The video distribution system according to claim 5 or 9 , wherein a third reproduction method for reproducing only video is included. The viewer terminal is
A viewpoint detection unit for detecting the viewpoint of the viewer in the video display area;
The image selecting unit, when the viewpoint detected by the viewpoint detecting unit is present in the display area, any claim 4, 9 and 10 receives a selection of the input image displayed on the display area The video distribution system according to claim 1.