JP2007150747A - Receiving apparatus and main line image distribution apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To seamlessly connect visual points and lines of sight at the time of switching from a main line image of a broadcast program to an image from an arbitrary visual point, when image distribution service by existing broadcast or the like and image service from arbitrary visual points are provided in cooperation with each other. <P>SOLUTION: The receiving apparatus receives a broadcast stream which includes main line image signals as images for broadcasting and visual point information with respect to the main line image signals, and uses the visual point information extracted from the broadcast stream as a initial value of the visual point of the image from arbitrary visual points when it makes a delivery request for images from arbitrary visual points which are images from arbitrary visual points generated by several images. In this way, at the time of switching from the main line image to the image from arbitrary visual points, the visual points can be seamlessly connected before and after switching between the main line image and the image from arbitrary visual points, since an image of an arbitrary visual point similar to the main line image can be used as the first image from the arbitrary visual point. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to a receiving apparatus and a main line video distribution apparatus for receiving video from an arbitrary viewpoint position desired by a viewer.

  Conventionally, a large number of cameras are used to shoot an object, such as a soccer game, from all directions, and from a large number of captured images, a viewpoint image in an arbitrary location and direction is interpolated and reconstructed. Many studies have been made on generating an arbitrary viewpoint or a free viewpoint video that can obtain a video with a viewpoint and line of sight that has not been shot (see, for example, Patent Document 1).

The technology disclosed in Patent Document 1 relates to a distribution device that provides this arbitrary viewpoint video like a video-on-demand system, and in accordance with a request from the viewer, the video from the viewpoint position desired by the viewer is displayed. Can be provided.
JP 2004-193941 A

  However, the arbitrary viewpoint video providing system disclosed in Patent Document 1 has only a function of distributing the video at the viewpoint position requested by the viewer. Therefore, the case where an arbitrary viewpoint video service is provided in cooperation with an existing broadcast program is not considered. Therefore, it is not considered that the viewpoint is seamlessly connected from the main video to the arbitrary viewpoint video when the main video of the broadcast program is switched to the arbitrary viewpoint video. For this reason, when the viewpoint changes from the main line video to the arbitrary viewpoint video, the viewer may feel uncomfortable.

  The present invention has been made in view of this point, and an object thereof is to enable seamless connection from a main video to an arbitrary viewpoint video when switching from the main video of the broadcast program to the arbitrary viewpoint video. .

  The present invention receives a mainstream video signal that is a broadcast video and a broadcast stream that includes viewpoint information for the mainline video signal, and requests distribution of an arbitrary viewpoint video that is an image at an arbitrary viewpoint generated from a plurality of videos. At this time, the viewpoint information extracted from the broadcast stream is used as the initial value of the viewpoint of the arbitrary viewpoint video.

  According to the present invention, when switching from main video to arbitrary viewpoint video, it is possible to request the arbitrary viewpoint video providing system to distribute arbitrary viewpoint video using the viewpoint information of the main video. As a result, when switching from main video to arbitrary viewpoint video, the same arbitrary viewpoint video can be used as the first arbitrary viewpoint video for the main video, so the viewpoint can be seamless before and after switching between main video and arbitrary viewpoint video. Can be connected. Therefore, it is possible to perform a favorable switching that does not cause a sense of discomfort for the viewer.

  A receiving apparatus according to a first aspect of the present invention includes a receiving unit that receives a main line video signal that is a broadcast video and a broadcast stream that includes viewpoint information for the main line video signal, and extracts the viewpoint information from the broadcast stream. A viewpoint information extraction unit, a viewpoint control unit that requests distribution of an arbitrary viewpoint video that is an image at an arbitrary viewpoint generated from a plurality of videos, with the viewpoint information as an initial value, and an arbitrary viewpoint based on the distribution request An arbitrary viewpoint video receiving unit that receives video distribution is employed.

  Thereby, it is possible to receive distribution of an arbitrary viewpoint video that is the same as the viewpoint information of the main line video. As a result, when switching from main video to arbitrary viewpoint video, the same arbitrary viewpoint video can be used as the first arbitrary viewpoint video for the main video, so the viewpoint can be seamless before and after switching between main video and arbitrary viewpoint video. Can be connected.

  According to a second aspect of the present invention, in the receiving device according to the first aspect, when the direction of the cross button for instructing the direction is pressed, the viewpoint control unit is the main line at that time. The arbitrary viewpoint video of the viewpoint information for the video is requested.

  As a result, switching to an arbitrary viewpoint video and viewpoint movement can be performed with the same button operation, so that the number of man-hours for the operator can be reduced.

  The main line video distribution apparatus according to the third aspect of the present invention is a main line video signal that is a broadcast video and the main line video that is used as an initial value of a viewpoint when accessing an arbitrary viewpoint video service by a receiving apparatus. A configuration is provided that includes an encoding unit that temporally associates viewpoint information for a signal and encodes it into a broadcast stream, and a main line video distribution unit that distributes the broadcast stream.

  As a result, the main line video and the viewpoint information for the main line video can be distributed.

  According to a fourth aspect of the present invention, in the main line video distribution apparatus according to the third aspect, the encoding unit describes the viewpoint information in a PES header of the main line video signal.

  As a result, the main line video and the viewpoint information for the main line video can be distributed using broadcasting.

  According to a fifth aspect of the present invention, in the main line video distribution apparatus according to the third aspect, the encoding unit describes the viewpoint information in a header part of a transmission line code of the main line video signal.

  According to this, it is possible to distribute the main line image and the viewpoint information for the main line image without depending on the format of the image to be actually distributed.

  According to a sixth aspect of the present invention, in the main line video distribution apparatus according to the third aspect, the viewpoint information is described and distributed in a private_data_byte part of the MPEG2 section.

  Thereby, viewpoint information can be distributed separately from a video signal. Thereby, almost no modification of the existing service is required, and it becomes easier to implement as an extension of the existing service.

  According to a seventh aspect of the present invention, in the main line video distribution apparatus according to the third aspect, a viewpoint information data generating unit that generates a viewpoint information list from the viewpoint information and a time stamp of the main line video signal, and the viewpoint information list And a data broadcast carousel generating unit that encodes the data in the carousel format of the data broadcast, and distributes the viewpoint information list encoded in the carousel format.

  Thereby, viewpoint information can be distributed using data broadcasting.

  An arbitrary viewpoint video system according to an eighth aspect of the present invention includes a receiving device according to a first aspect, a main line video distribution device according to a third aspect, and an arbitrary viewpoint video distribution device that generates and distributes the arbitrary viewpoint video. The main line video distribution device distributes the broadcast stream to the reception device, the reception device receives the distributed broadcast stream, and the arbitrary viewpoint video distribution device receives the broadcast stream. When making an arbitrary viewpoint video distribution request, the viewpoint information extracted by the viewpoint information extraction unit is used as an initial value of the viewpoint of the arbitrary viewpoint video, and the arbitrary viewpoint video distribution device distributes from the receiving device The arbitrary viewpoint video of the viewpoint requested is generated and distributed to the receiving device.

  Thereby, the receiving apparatus can receive the distribution of the arbitrary viewpoint video that is the same as the viewpoint information of the main line video. As a result, when switching from main video to arbitrary viewpoint video, the same arbitrary viewpoint video can be used as the first arbitrary viewpoint video for the main video, so the viewpoint can be seamless before and after switching between main video and arbitrary viewpoint video. Can be connected.

  According to a ninth aspect of the present invention, there is provided a recording medium that records a video signal and uses the video signal for use as an initial value of a viewpoint when a receiving device accesses an arbitrary viewpoint video service related to the video signal. And viewpoint information for the video signal recorded in association with the.

  Thereby, it is possible to acquire a video and viewpoint information for the video from the recording medium.

  A receiving / reproducing apparatus according to a tenth aspect of the present invention is a receiving / reproducing apparatus for reproducing viewpoint information for the video together with the video recorded on the recording medium of the ninth aspect, wherein the viewpoint information is reproduced from the reproduced data. A viewpoint information extraction unit that extracts a viewpoint, a viewpoint control unit that makes a distribution request of an arbitrary viewpoint video that is an image at an arbitrary viewpoint generated from a plurality of videos, using the viewpoint information as an initial value, and a distribution request based on the distribution request An arbitrary viewpoint video receiving unit that receives distribution of an arbitrary viewpoint video is employed.

  Accordingly, the reception / playback apparatus can use the cooperation service with the arbitrary viewpoint video providing system using the viewpoint information recorded on the recording medium.

  An arbitrary viewpoint video distribution apparatus according to an eleventh aspect of the present invention is a main line generated by recording a time stamp of a main video signal, which is a video for broadcasting, and viewpoint information for the main video signal in a time-related manner. A main line viewpoint information database for storing a viewpoint information list is provided, and in response to a distribution request from a receiving apparatus, an arbitrary viewpoint video with the origin of the main line video viewpoint information recorded in the main line viewpoint information is distributed.

  As a result, the main line video and the viewpoint information for the main line video can be associated with each other by the time stamp and distributed.

  A receiving apparatus according to a twelfth aspect of the present invention includes a main video signal receiving unit that is a broadcast video, a video decoder that reproduces the main video signal and extracts a video time stamp, A viewpoint control unit that makes a distribution request using the extracted time stamp at the time of distribution request to the arbitrary viewpoint video distribution apparatus of the aspect, and arbitrary viewpoint video reception that receives distribution of the arbitrary viewpoint video based on the distribution request A portion.

  Thereby, the receiving device can receive the distribution of the arbitrary viewpoint video identical to the viewpoint information of the main line video from the arbitrary viewpoint video distribution device by the distribution request using the time stamp. As a result, when switching from main video to arbitrary viewpoint video, the same arbitrary viewpoint video can be used as the first arbitrary viewpoint video for the main video, so the viewpoint can be seamless before and after switching between main video and arbitrary viewpoint video. Can be connected.

  Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. Note that the present invention is not limited to these embodiments, and can be implemented in various modes without departing from the scope of the invention.

(Embodiment 1)
An arbitrary viewpoint video system according to the first embodiment of the present invention will be described. The arbitrary viewpoint video system according to the first embodiment realizes a service that provides an arbitrary viewpoint video in cooperation with an existing broadcast-type program.

  First, an outline of the arbitrary viewpoint video system according to the first embodiment will be described with reference to FIG.

  The arbitrary viewpoint video system 100 according to the first embodiment includes a main line video providing system 101 that distributes a video of a relay broadcast program (hereinafter referred to as a main line video), an arbitrary viewpoint video providing system 102 that distributes an arbitrary viewpoint video, and a main line. A receiving apparatus 103 that receives video and an arbitrary viewpoint video is employed.

  The main line video providing system 101 provides a broadcast-type program to viewers, that is, the receiving device 103 by digital broadcasting such as BS / CS digital broadcasting, terrestrial digital broadcasting, and CATV. The arbitrary viewpoint video providing system 102 provides an arbitrary viewpoint video through a communication line such as the Internet.

  The receiving device 103 displays a screen 200 as shown in FIG. 2 when a broadcast type program, in this example, a soccer relay program is selected.

  When the viewer selects a channel in the receiving apparatus 103, the main line video 201 is reproduced. The receiving device 103 overlays and displays GUI (Graphical User Interface) 202 and 203 for displaying program information and the like on the main line video 201 for a few seconds immediately after the viewer selects a channel.

  The GUI 202 is an arbitrary viewpoint video providing icon indicating that the arbitrary viewpoint video is being provided and the arbitrary viewpoint video can be received. The GUI 203 is a panel that displays “currently providing an arbitrary viewpoint video”, and notifies the viewer of the existence of an arbitrary viewpoint video service.

  The existence of the arbitrary viewpoint video service is described and provided in the data of an electronic program guide (EPG) as part of program information. The receiving apparatus 103 can recognize the existence of the arbitrary viewpoint video service by referring to the data of the electronic program guide. In addition, the receiving apparatus 103 inquires of a server that searches for a service related to a program using the ID of the program, for example, a CRID (Content Referencing Identifier) standardized in the TV Anytime Forum as a key, for the existence of the arbitrary viewpoint video service. You can also get it.

  Also, the viewer of the receiving apparatus 103 can watch the main line video 201 and switch to the arbitrary viewpoint video at an arbitrary timing. For example, in order to see the state of a favorite player who is not shown in the main line image 201, it is possible to switch to an arbitrary viewpoint video and move the viewpoint so that the player is reflected and view it. In addition, in order to view a decisive moment such as a shoot scene at a different angle from the main line video, the viewer can also switch to an arbitrary viewpoint video and play back the shoot scene from another viewpoint.

  In addition, the viewer can switch to the arbitrary viewpoint video using a general TV remote controller for digital broadcasting as shown in FIG.

  A cross button 302 is provided in the middle of the remote control 301. The cross button 302 is assigned a function for switching to an arbitrary viewpoint video. The viewer can switch to the arbitrary viewpoint video by pressing any direction of the cross button 302. With this configuration, the operator can also move the viewpoint using the cross button 302 after switching to the arbitrary viewpoint video.

  Thereby, since the operator can switch to the arbitrary viewpoint video and move the viewpoint with the same button operation, it is possible to reduce the operation man-hours of the operator.

  Note that a switch button for arbitrary viewpoint video may be separately prepared on the remote controller 301, and switching to the arbitrary viewpoint video may be performed by pressing the button.

  Also, the arbitrary viewpoint video system 100 according to the first embodiment is configured to seamlessly connect the viewpoint and line of sight before and after switching between the main line video and the arbitrary viewpoint video in the receiving apparatus 103.

  That is, when the receiving apparatus 103 switches from the main video to the arbitrary viewpoint video, the receiving apparatus 103 switches to the arbitrary viewpoint video at the same viewpoint as the main video that has been viewed up to now. This is because the receiving apparatus 103 requires an arbitrary viewpoint video having the same viewpoint as that of the camera capturing the main line video, more specifically, the same shooting position, shooting direction, and angle of view (viewing angle) as the main line video camera. Realized by playing. That is, it is only necessary for the receiving apparatus 103 to know the viewpoint (shooting position, shooting direction, angle of view, etc.) of the camera that is shooting the main line video at an arbitrary time.

  In order to realize this, the first embodiment adopts a configuration in which main line video providing system 101 sends to main unit video data including the viewpoint information of the camera that shot the video in the main line video data. On the other hand, when switching to the arbitrary viewpoint video, the receiving device 103 includes the viewpoint information (main line video viewpoint information and main line video viewpoint information) included in the video data of the currently displayed main line video. The arbitrary viewpoint video providing system 102 is requested for the arbitrary viewpoint video of that viewpoint. The receiving apparatus 103 acquires the arbitrary viewpoint video requested from the arbitrary viewpoint video providing system 102 and then switches from the main video to the playback screen of the arbitrary viewpoint video. Thus, since the arbitrary viewpoint video starts from the same viewpoint as the main video, the main video and the arbitrary video can be seamlessly connected from the same viewpoint.

  FIG. 4 is a diagram illustrating an example of a screen while the receiving apparatus 103 is viewing an arbitrary viewpoint video. As shown in FIG. 4, an arbitrary viewpoint video 401 is displayed on the entire screen. In addition, a main line video thumbnail 402 and a virtual viewpoint window 403 are superimposed on the arbitrary viewpoint video 401.

  The virtual viewpoint window 403 displays the current main line video viewpoint (line of sight) 404 and the viewpoint (line of sight) 405 of the arbitrary viewpoint video together with the soccer court graphic in real time. Usually, the camera that shoots the main line video shoots the most noteworthy place, players, etc. of the game. Therefore, by making it possible to simultaneously operate the viewpoint of the arbitrary viewpoint video while referring to the current main video viewpoint 404, the positional relationship with the main video can be easily understood. As a result, it is expected that it becomes easy to move the viewpoint to a place to be noticed and to specify the viewpoint so that the place is viewed from another angle.

  In this way, the viewer of the receiving apparatus 103 can view a soccer game or the like while switching between the main line video and the arbitrary viewpoint video at an arbitrary timing. Furthermore, by applying this embodiment, when switching from main video to arbitrary viewpoint video, it is possible to switch to the arbitrary viewpoint video of the same viewpoint as the main video that has been viewed so far, and use the main video viewpoint. Thus, the viewpoint operation of the arbitrary viewpoint video can be easily performed.

  Here, digital broadcasting has been described as an example of main line video, but the video providing medium is not particularly limited. For example, the same applies when a relay program is provided by Internet streaming distribution or the like.

  In addition, the main line video refers to the general video that is provided by the video creator when the video is shot or edited, and can be the main video except for the arbitrary viewpoint video that allows the viewer to control the viewpoint. . For example, a “multi-angle video” in which videos taken from a plurality of viewpoints are provided simultaneously can be a main video because the viewer can only switch between a plurality of viewpoints determined by the producer.

  The main line video viewpoint information is not only used when switching from the main line video to the arbitrary viewpoint video, but can be used for various purposes in a role of assisting the viewpoint operation of the arbitrary viewpoint video.

  For example, the viewer can operate any parameter in the viewpoint information indicated by the shooting position, shooting direction, and angle of view of the viewpoint of the arbitrary viewpoint video by relative designation with respect to the main line video viewpoint. The case where the angle of view is operated will be described as an example. When the viewer wants to zoom in on the main line image 402, the value of the angle of view is specified in the minus direction (zoom up direction) with respect to the main line image viewpoint. . More specifically, when the angle of view of the current mainline viewpoint video is 60 degrees, 30%, which is 50%, is designated as the viewpoint of the arbitrary viewpoint video. Since this operation is relative designation, if the main line video viewpoint changes, the viewpoint of the arbitrary viewpoint video also changes. When the angle of view of the main-line viewpoint video changes from 60 degrees to 120 degrees, the viewpoint of the arbitrary viewpoint video also follows and changes from 30 degrees to 60 degrees.

  This feature makes it easier to grasp the flow of the game by looking over a wider range than the main video when watching on a large screen such as a plasma TV, high-definition television, When viewing on a small liquid crystal screen of a portable terminal, the main line image is too small to discriminate the player, which is particularly effective when zooming in and viewing.

  In addition, a function to synchronize the viewpoint with the main line video viewpoint can be assigned to the button on the remote control, and the main video viewpoint at the time of pressing can be set as the viewpoint of the arbitrary viewpoint video by pressing the button. . In other words, the viewer can return to the viewpoint of the main line video at any time by simply pressing this button. This is effective when the main line image 402 shows that, for example, the close battle before the goal has started, and if the user wants to quickly move the viewpoint before the goal, or if the positional relationship becomes unclear while moving the viewpoint. is there.

  Also, the timing at which the main line video is provided is not particularly limited. For example, the main line video may be recorded or provided on a video-on-demand basis, or may be provided on a package medium such as a DVD. In addition, it may be provided as a part of another program, for example, a part of sports news.

  Next, the overall configuration of the arbitrary viewpoint video system 100 according to the first embodiment will be described with reference to FIG. FIG. 5 is a block diagram of the arbitrary viewpoint video system 100 according to the first embodiment.

  The main line video providing system 101 includes a plurality of broadcast video shooting cameras (broadcast cameras) 501, a main line video switcher 502, an encoding device 503, and a main line video distribution device 504.

  The broadcast camera 501 is a broadcast camera that photographs a soccer game. Broadcast camera 501 outputs viewpoint information parameter information together with video (main video). The plurality of broadcast cameras 501 input a plurality of photographed main line images and viewpoint information parameters to the main line image switcher 502.

  The main line video switcher 502 switches the video that is actually broadcast, that is, the camera video that becomes the main line video, according to an instruction from the director of the relay program. The main line video switcher 502 inputs the main line video of the switched camera to the encoding device 503. The main line video switcher 502 also inputs the viewpoint information parameter of the switched camera together with the main line video to the encoding device 503.

  The encoding device 503 is a device that compresses the input main line video and converts it into a format necessary for broadcasting. The encoding apparatus 503 sends the viewpoint information parameter to the main line video distribution apparatus 504 together with the converted main line video.

  The main line video distribution apparatus 504 sends the received main line video and viewpoint information parameters to the receiving apparatus 103.

  The arbitrary viewpoint video providing system 102 includes a number of arbitrary viewpoint video shooting cameras (hereinafter referred to as arbitrary viewpoint cameras) 505, an arbitrary viewpoint video generation device 506, an arbitrary viewpoint video DB 507, and an arbitrary viewpoint video distribution device 508.

  The arbitrary viewpoint camera 505 is a camera that captures the same object as the main line image, here, a soccer game for the arbitrary viewpoint image. The arbitrary viewpoint camera 505 shoots an object from all directions and shoots video data sufficient for generating an arbitrary viewpoint video. A number of arbitrary viewpoint cameras 505 simultaneously input captured images to the arbitrary viewpoint video generation device 506. The arbitrary viewpoint camera 505 may be shared with the broadcasting camera 501.

  The arbitrary viewpoint video generation device 506 generates an arbitrary viewpoint video with respect to a large number of input videos by using an arbitrary viewpoint video generation algorithm as described in Patent Document 1, for example. The arbitrary viewpoint video generation device 506 sends the generated arbitrary viewpoint video to the arbitrary viewpoint video DB 507.

  The arbitrary viewpoint video DB 507 inputs an arbitrary viewpoint video and temporarily stores it. Also, the arbitrary viewpoint video DB 507 manages the arbitrary viewpoint video using time and viewpoint as keys.

  The arbitrary viewpoint video distribution device 508 takes out a corresponding arbitrary viewpoint video (with the same viewpoint) from the arbitrary viewpoint video DB 507 and distributes it to the reception device 103 in accordance with a request from the reception device 103. In this embodiment, the arbitrary viewpoint video is generated in advance by the arbitrary viewpoint video generation device and stored in the arbitrary viewpoint video DB in the state of the arbitrary viewpoint video. The video may be stored as it is captured by the camera 505, and the arbitrary viewpoint video of the viewpoint requested by the arbitrary viewpoint video generation device may be generated each time according to the request from the reception device 103. . In addition, although the present embodiment is described on the assumption that an arbitrary viewpoint video in which the main line video and the viewpoint completely match can be generated, the viewpoint actually matches actually due to restrictions on the number of cameras for arbitrary viewpoints and installation locations. There may be cases where video cannot be obtained (cannot be provided). In such a case, the viewpoint change at the time of switching to the arbitrary viewpoint video is minimized by providing the arbitrary viewpoint video of the closest viewpoint within the range that can be provided.

  The reception device 103 is a reception device that supports reception and reproduction of both the broadcast service and the arbitrary viewpoint video service. Specifically, the receiving apparatus 103 receives a main line video from the main line video providing system 101. Further, when receiving the distribution of the arbitrary viewpoint video, the receiving device 103 transmits the main line video viewpoint information transmitted from the main line video providing system 101 to the arbitrary viewpoint video providing system 102. Next, an arbitrary viewpoint video of the same viewpoint as the main line video sent from the arbitrary viewpoint video providing system 102 is received. Then, the receiving apparatus 103 seamlessly reproduces from the main video to the arbitrary viewpoint video having the same viewpoint information as the main video.

  As described above, the receiving apparatus 103 reproduces from the arbitrary viewpoint video having the same viewpoint information as the main line video, thereby smoothly switching from the main line video to the arbitrary viewpoint video.

  As described above, the arbitrary viewpoint video system 100 is configured.

  Next, viewpoint information parameters will be described with reference to FIG. The viewpoint information parameter is a parameter for specifying the viewpoint and line of sight of the camera, that is, the shooting position, shooting direction, and angle of view (viewing angle) of the camera.

  The camera position is a three-dimensional position of the broadcast camera 501 and is represented by three parameters x, y, and z. The shooting direction is represented by two parameters of a horizontal component and a vertical component of the shooting direction, and is represented by a pan angle θ and a tilt angle φ, respectively. The angle of view (viewing angle) is a range of video captured by the camera. The angle of view is generally represented by two parameters, a horizontal angle of view and a vertical angle of view. In this example, it is assumed that the aspect ratio of the captured image is fixed in advance to a value such as 16: 9. If one of them is known, the other is also obtained, so that only the horizontal angle of view is used.

  Note that the viewpoint information parameters do not necessarily have to be expressed by the seven parameters shown in this example, and may be anything as long as the shooting position, shooting direction, and angle of view can be specified.

  Next, the configuration of the broadcast camera 501 will be described with reference to FIG. FIG. 7 shows a configuration for acquiring viewpoint information parameters, and a general configuration of the camera is omitted.

  The broadcast camera 501 is provided with a GPS (Global Positioning System) 603. The GPS 603 measures the camera position. By using a high-accuracy position measurement technique generally used in GPS surveying, it is possible to measure a three-dimensional position with an accuracy of several centimeters at most.

  In addition, the broadcast camera 501 is provided with a focal length sensor 604 that measures the focal length from the position of the zoom lens. A focal length sensor 604 obtains a focal field angle from the focal length and the size of the imaging surface (CCD).

  The broadcast camera 501 includes a camera base 602 that rotatably holds the camera, a rotation angle sensor 606 that is installed on the camera base 602 and measures the pan angle of the camera, and an inclination angle sensor that measures the tilt angle of the camera. 605 is provided. The shooting direction is obtained using the camera pan angle measured by the rotation angle sensor 606 and the camera tilt angle measured by the tilt angle sensor 605.

  With the configuration as described above, the broadcast camera 501 captures a main line video, acquires a viewpoint information parameter, and outputs it.

  Next, the configuration of encoding apparatus 503 will be described using FIG. FIG. 8 is a block diagram of an internal configuration of encoding apparatus 503 in the first embodiment.

  The encoding device 503 receives the main line video signal 701 and the viewpoint information parameter 703 output from the main line video switcher 502, and outputs MPEG2 transport stream format (TS: Transport Stream) stream data 707 including viewpoint information. To do.

  Specifically, the encoding device 503 is provided with a video compression unit 702 that compresses the video signal 701 of the main video output from the main video switcher 502. In this example, the video compression unit 702 performs compression processing based on a video encoding method defined by MPEG2 Video (ISO / IEC 13818-2). The video compression unit 702 outputs compressed video data in the format of an MPEG2 elementary stream (ES: Elementary Stream).

  In addition, the encoding apparatus 503 is provided with a viewpoint information conversion unit 704. The viewpoint information conversion unit 704 inputs the viewpoint information parameter 703 of the broadcast camera 501 that has captured the main line video. The viewpoint information conversion unit 704 converts viewpoint information parameters to viewpoint information.

  Here, in order to clarify this conversion process, the difference between the viewpoint information parameter and the viewpoint information will be described. The viewpoint information parameter is data measured and output by the broadcast camera 501. On the other hand, the viewpoint information is an optional viewpoint video service provided separately and parameter units, etc., and are arranged in a common parameter system that can be used mutually.

  The viewpoint information will be described more specifically. The camera position of the viewpoint information is described in 24 bits in units of 0.1 mm in each direction of xyz, for example, with the center of the soccer court as the origin. The angle of view of the viewpoint information is described in 24 bits from 0 degree to 360 degrees in the horizontal view angle. The photographing direction of the viewpoint information is described by 16 bits each of a pan angle and a tilt angle of −180 degrees to 180 degrees. In this way, it is converted into a predetermined format and described.

  Also, the encoding apparatus 503 is provided with a PES packetizing unit 705 that inputs the video data output from the video compression unit 702 and the viewpoint information output from the viewpoint information conversion unit 704. The PES packetizing unit 705 packetizes ES format video data (main line video) output from the video compression unit 702. The PES packetizing unit 705 also describes viewpoint information by including it in a part of a packet of video data.

  Specifically, as shown in FIG. 9, the PES packetization unit 705 converts the ES format video data 801 output from the video compression unit 702 into MPEG2 Systems (ISO / IEC13818-1) as shown in 802. Packetize into the specified PES format.

  Further, the PES packetizing unit 705 describes the viewpoint information 804 by including it as a part of the PES format packet 802. Specifically, the PES packetization unit 705 describes the format in a PES_extension_field 805 that is an extension area of the PES header area 803.

  As described above, the PES packetization unit 705 outputs the packetized PES data to the TS packetization unit 706.

  The TS packetizing unit 706 packetizes the PES data packetized by the PES packetizing unit 705 in a Transport Stream format defined by MPEG2 Systems (ISO / IEC13818-1), and forms a transport stream (TS) 707. Output.

  In this way, the encoding device 503 superimposes the viewpoint information on the video signal 701 and outputs it as a transport stream 707.

  Thereafter, the transport stream 707 is broadcast-encoded by the main line video distribution apparatus 504 and broadcasted to the reception apparatus 103.

  Next, the configuration of receiving apparatus 103 will be described with reference to FIG. FIG. 10 is a block diagram of the internal configuration of the receiving apparatus 103.

  The receiving apparatus 103 inputs a main line video system block that receives the broadcast signal 901 and outputs the main line video 905, a block that extracts viewpoint information corresponding to the main line video, and a viewpoint operation 913 by the user of the receiving apparatus 103. The viewpoint operation information 908 is output to the arbitrary viewpoint video distribution apparatus 508, the arbitrary viewpoint video 909 corresponding to the viewpoint is received and input from the arbitrary viewpoint video distribution apparatus 508, and the arbitrary viewpoint video system 912 is output. Block.

  The main line video block will be described. The main line video block is provided with a broadcast receiving unit 902 that receives a broadcast signal 901 broadcast from the main line video distribution apparatus 504. The broadcast receiving unit 902 decodes the broadcast signal 901, extracts the transport stream, and outputs it to the TS decoder 903.

  The TS decoder 903 receives the transport stream, decodes it into a PES data stream, and outputs it to the video decoder 904.

  The video decoder 904 decodes the input PES data, further decompresses the compressed video, and outputs the main video 905. The video decoder 904 also outputs the PES packet of the video output as the main video 905 to a block for extracting viewpoint information corresponding to the main video.

  A viewpoint information extraction unit 906 is provided in a block for extracting viewpoint information corresponding to the main line video. The viewpoint information extraction unit 906 receives the PES packet and extracts the time stamp and viewpoint information of the video described in the PES header area. This viewpoint information is referred to as “main line video viewpoint information”. The “main line video viewpoint information” and time stamp extracted by the viewpoint information extraction unit 906 are input to blocks of an arbitrary viewpoint video system as needed.

  An arbitrary viewpoint video block is provided with a viewpoint control unit 907. The viewpoint control unit 907 always inputs the latest main line video viewpoint information and time stamp output from the viewpoint information extraction unit 906. When the user switches from the main line video to the arbitrary viewpoint video, the viewpoint control unit 907 generates viewpoint operation information 908 using the latest main line video viewpoint information and a time stamp. Then, the viewpoint control unit 907 requests the video with the same viewpoint as the main video 905 by sending this viewpoint operation information 908 to the arbitrary viewpoint video distribution apparatus 508.

  The arbitrary viewpoint video distribution device 508 generates a video of the requested viewpoint, and distributes it as an arbitrary viewpoint video 909 to the arbitrary viewpoint video system block of the reception device 103. The arbitrary viewpoint video receiver 910 of the arbitrary viewpoint video block receives and inputs the arbitrary viewpoint video 909 and outputs it to the arbitrary viewpoint video decoder 911.

  The arbitrary viewpoint video decoder 911 decodes the arbitrary viewpoint video 909 and outputs it as an arbitrary viewpoint video 912.

  As described above, the main line video viewpoint information transmitted in association with the main line video in the broadcast signal 901 is extracted by the viewpoint information extraction unit 906, and the latest main line video viewpoint information is always set in the viewpoint control unit 907. . Thereby, when switching from the main line video to the arbitrary viewpoint video, it is possible to request an arbitrary viewpoint video of the same viewpoint as the main line video by using the main line video viewpoint information at that time. As a result, when switching from the main line video to the arbitrary viewpoint video, the switching can be performed without changing the viewpoint.

  The viewpoint control unit 907 also sends viewpoint operation information 908 to the arbitrary viewpoint video distribution device 508 based on the viewpoint operation 913 from the user, and receives and displays the corresponding arbitrary viewpoint video 909 as needed.

  An example of the viewpoint operation information is shown in FIG. In the example of FIG. 11, each parameter of the position corresponding to the viewpoint information, the viewing angle, and the line-of-sight direction is absolutely specified. That is, an arbitrary viewpoint video of the parameter specified by this viewpoint operation information is obtained.

  Note that the viewpoint operation information may be specified relative to the previously specified viewpoint except for the first viewpoint operation after switching to an arbitrary viewpoint video. In this case, the viewpoint for each receiving apparatus 103 is held and managed on the arbitrary viewpoint video distribution apparatus 508 side, and the arbitrary viewpoint video of the specified viewpoint is obtained from the parameters specified relative thereto.

  Next, the processing sequence of the arbitrary viewpoint video system 100 will be described with reference to FIG. FIG. 12 is a diagram illustrating a protocol sequence for acquiring an arbitrary viewpoint video of the arbitrary viewpoint video system 100.

  First, the broadcast camera 501 of the main line video providing system 101 performs main line video shooting, and sends viewpoint information parameters to the main line video switcher 502 together with main line video data. The main video switcher 502 selects main video data and viewpoint information parameters desired by the director from a plurality of input main video data and viewpoint information parameters in response to an instruction from the director of the relay program, and the like. send. In this way, main line video providing system 101 determines main line video data and viewpoint information parameters (ST1101).

  Next, the encoding device 503 compresses the main line video data output from the main line video switcher 502. Also, encoding apparatus 503 converts the viewpoint information parameter into viewpoint information, superimposes it on the compressed main line video data, and sends it to main line video distribution apparatus 504 (ST1102).

  Next, main line video distribution apparatus 504 sends main line video data on which viewpoint information is superimposed to receiving apparatus 103 using digital broadcasting (ST1103).

  In this way, the main line video providing system 101 sends not only the main line video data but also the viewpoint information for the main line video to the receiving apparatus 103, so that the receiving apparatus 103 makes a request for arbitrary viewpoint video data corresponding to the main line video. can do.

  Next, the receiving apparatus 103 performs processing to switch from reception of main line video to reception of arbitrary viewpoint video (ST1104), and proceeds to reception processing of arbitrary viewpoint video.

  In this embodiment, the protocol used by the receiving apparatus 103 and the arbitrary viewpoint video providing system 102 is based on RTSP (Real Time Streaming Protocol) recommended by RFC as RFC 2326, and is optionally added with commands and parameters. It has been expanded for viewpoint video distribution.

  First, the viewpoint control unit 907 of the receiving apparatus 103 sends a distribution preparation request (SETUP) to the arbitrary viewpoint video distribution apparatus 508 of the arbitrary viewpoint video providing system 102 (ST1105). On the other hand, the arbitrary viewpoint video providing system 102 prepares for distribution of the arbitrary viewpoint video, and when the preparation is completed, returns OK indicating that the preparation is completed to the viewpoint control unit 907 of the receiving apparatus 103 (ST1106).

  Next, the viewpoint control unit 907 throws a playback (PLAY) command to the arbitrary viewpoint video distribution apparatus 508 (ST1107). At this time, the viewpoint control unit 907 can add viewpoint operation information to the playback command. The viewpoint control unit 907 designates the main line video viewpoint in the viewpoint operation information at this time, as described above, except for a special case.

  In response to this, the arbitrary viewpoint video distribution apparatus 508 returns OK indicating the success of the command to the reception apparatus 103 in response to the received playback command (ST1108), and arbitrarily selects the requested viewpoint (here, the main line video viewpoint). The viewpoint video data is extracted from the arbitrary viewpoint video DB 507 and distributed to the receiving apparatus 103 (ST1109).

  On the other hand, receiving apparatus 103 receives arbitrary viewpoint video data of the main video viewpoint at arbitrary viewpoint video receiving section 910, decodes it by arbitrary viewpoint video decoder 911, and outputs it as a reproducible arbitrary viewpoint video (ST1110). ).

  As described above, when the receiving apparatus 103 requests an arbitrary viewpoint video, the arbitrary viewpoint video distribution apparatus 508 returns an arbitrary viewpoint video of the main line video viewpoint. Since the arbitrary viewpoint video of the main line video viewpoint is the video of the same viewpoint as the main line video, even if the viewer switches from the main line video to the arbitrary viewpoint video, the viewer does not feel a sense of incongruity due to a large change in viewpoint.

  Next, the viewpoint control unit 907 of the receiving apparatus 103 receives a viewpoint operation from the user, puts the viewpoint operation information in a viewpoint movement (MOVE) command, and sends it to the arbitrary viewpoint video distribution apparatus 508 (ST1111).

  On the other hand, the arbitrary viewpoint video distribution apparatus 508 returns OK to the receiving apparatus 103 (ST1112) and distributes arbitrary viewpoint video data of the designated viewpoint (ST1113).

  On the other hand, the receiving device 103 receives the arbitrary viewpoint video data of the viewpoint designated by the viewer in the arbitrary viewpoint video receiving unit 910, decodes it by the arbitrary viewpoint video decoder 911, and outputs it as a reproducible arbitrary viewpoint video. (ST1114).

  If the viewpoint designated by the receiving apparatus 103 exceeds the valid range, the arbitrary viewpoint video distribution apparatus 508 returns NG to the receiving apparatus 103, and the arbitrary viewpoint video at the current viewpoint (previously designated viewpoint). Continue to deliver.

  It is assumed that the arbitrary viewpoint video distribution device 508 maintains the current viewpoint until a new viewpoint operation is performed from the receiving device 103. That is, the arbitrary viewpoint video distribution device 508 continues streaming distribution of the video of the current viewpoint in time series unless a command from the reception device 103 is sent.

  In addition, a viewpoint movement (MOVE) command is sent from the receiving apparatus 103 to the arbitrary viewpoint video distribution apparatus 508 in order to perform a viewpoint operation and a viewpoint movement by the user, and the arbitrary viewpoint video distribution apparatus 508 sends the corresponding arbitrary viewpoint video data. Is distributed to the receiving apparatus 103.

  Finally, when stopping viewing arbitrary viewpoint video or returning to main line video, the viewpoint control unit 907 of the receiving apparatus 103 sends a playback end (TEARDOWN) command to the arbitrary viewpoint video distribution apparatus 508, Distribution of the viewpoint video data ends (ST1115).

  In this way, the main line video providing system 101 describes the main line video viewpoint information in the header portion of the corresponding main line video data packet and sends it to the receiving apparatus 103 together with the main line video. The receiving apparatus 103 receives the main line video and the main line video viewpoint information, and uses the main line video viewpoint information as an initial value when switching to the arbitrary viewpoint video. As a result, when switching to the arbitrary viewpoint video, the receiving apparatus 103 starts reproduction from the arbitrary viewpoint video having the same viewpoint as the main video. As a result, there is no change in the viewpoint when the main line video viewpoint information is switched to the arbitrary viewpoint video, and the viewer does not feel uncomfortable.

  Although the above-described configuration is mainly intended for digital broadcasting, it can also be applied to a case where main line video is distributed through a communication path such as the Internet using a similar configuration. In the above description, it has been described that the main line video distribution apparatus 504 performs “broadcast encoding”, but instead of performing “broadcast encoding”, a protocol generally used for video distribution over the Internet or the like. It is also possible to further encode the TS packet with RTP (Real Time Protocol) and distribute it through a communication channel such as the Internet.

  When performing distribution by RTP, the encoding device 503 can be configured as shown in FIG. In this configuration, the elementary stream output from the video compression unit 702 is directly placed on the RTP packet. In addition, the same code | symbol is provided to the already demonstrated part.

  In the configuration shown in FIG. 13, the RTP packetizing unit 1205 generates the RTP stream 1206 in which the main line video viewpoint information output from the viewpoint information converting unit 704 is described in the extension header area of the RTP packet as shown in FIG. Output.

  In this way, by describing the main line video viewpoint information in the extension header area, the main line video viewpoint information can be combined with the video even in a video format that does not have a PES structure, for example, without depending on the actual video format to be distributed. It is possible to send.

  Instead of using a live-action video like the arbitrary viewpoint video described in the first embodiment, the game may be reproduced by three-dimensional computer graphics. This embodiment can be applied particularly to a case where an arbitrary viewpoint video is provided by computer graphics in cooperation with a game machine.

(Embodiment 2)
Next, a second embodiment of the present invention will be described. In the first embodiment, the main line video viewpoint information is described in the header area of the main line video packet. However, in the second embodiment, the main line video viewpoint information is transmitted in a packet different from the video packet. By adopting the configuration as in the second embodiment, it is expected that the modification of the existing device is almost unnecessary, and it is easier to realize as an extension of the existing service.

  First, the configuration of the arbitrary viewpoint video system according to the second embodiment will be described with reference to FIG. In addition, the same code | symbol is provided to the part same as the already demonstrated part, and detailed description is abbreviate | omitted.

  The arbitrary viewpoint video system 1400 according to the second embodiment is provided with a main line video providing system 1401 having a viewpoint information encoding device 1404 and a multiplexing device 1405.

  Further, the output destination of the main video switcher 502 is separated by the video data and the viewpoint information parameter, and input to the video encoding device 1403 and the viewpoint information encoding device 1404, respectively. The outputs of the video encoding device 1403 and the viewpoint information encoding device 1404 are input to the multiplexing device 1405.

  The operation of receiving apparatus 1411 is also partly different from receiving apparatus 103 of Embodiment 1. Details of the reception device 1411 will be described later.

  Next, the internal configurations of the video encoding device 1403 and the viewpoint information encoding device 1404 will be described with reference to FIG.

  The video encoding device 1403 receives the video signal 701 and outputs a video transport stream 1505.

  Further, the viewpoint information encoding device 1404 receives the viewpoint information parameter 703 and outputs a main video viewpoint information transport stream 1510 obtained by transporting a data packet in which the main video viewpoint information is described.

  The video encoding device 1403 is provided with a video compression unit 1502. The video compression unit 1502 receives the video signal 701, performs video compression processing, for example, video compression encoding processing based on MPEG2 Video, and generates an elementary stream. The video compression unit 1502 outputs the elementary stream to the PES packetization unit 1503.

  Next, the PES packetization unit 1503 converts the elementary stream into a PES packet and outputs the PES packet to the TS packetization unit 1504. The TS packetizing unit 1504 converts the PES packetized elementary stream into a TS packet and outputs it as a video transport stream 1505.

  Here, it should be noted that the time stamp information included in the PES packet is passed from the PES packetization unit 1503 to the MPEG2 sectioning unit 1508 of the viewpoint information encoding device 1404. Details will be described in the description of the MPEG2 sectioning unit 1508.

  On the other hand, the viewpoint information encoding device 1404 receives the viewpoint information parameter 703 from the main line video switcher 502, and the viewpoint information conversion unit 704 converts it into main line video viewpoint information. This process is the same as that described in the first embodiment. Then, the viewpoint information conversion unit 704 passes the converted main line video viewpoint information to the MPEG2 sectioning unit 1508.

  Next, the MPEG2 sectioning unit 1508 encodes the main line video viewpoint information in the form of an MPEG2 Private section defined by MPEG2 Systems (ISO / IEC13818-1).

  FIG. 17 is a diagram illustrating an example of main line video viewpoint information encoded by the MPEG2 sectioning unit 1508. The MPEG2 sectioning unit 1508 combines the main line video viewpoint information given from the viewpoint information converting unit 704 and the time stamp information given from the PES packetizing unit 1503 of the video encoding device 1403, as shown in FIG. 17, MPEG2 Private. It is described in the private_data_byte area 1601 of the Section structure. Here, as the time stamp information, PTS (PresentationTimeStamp) described by 32 bits of the main line video is used.

  In this way, the MPEG2 sectioning unit 1508 encodes the main line video viewpoint information and outputs it to the TS packetizing unit 1509. Then, the TS packetizing unit 1509 converts the encoded main video viewpoint information into a TS packet, and outputs it as a main video viewpoint information transport stream 1510.

  The video transport stream 1505 and the main line video viewpoint information transport stream 1510 generated in this way are input to the multiplexing device 1405. The multiplexing device 1405 is a device having a function of converting (multiplexing) a plurality of input transport streams 1505 and 1510 into one transport stream. As described above, the multiplexing apparatus 1405 multiplexes the video transport stream and the main line video viewpoint information transport stream into one transport stream, and outputs the result to the main line video distribution apparatus 1406.

  The main line video distribution apparatus 1406 performs broadcast encoding and the like, and broadcasts to the reception apparatus 1411.

  Next, the configuration of receiving apparatus 1411 will be described using FIG. FIG. 18 shows the internal configuration of the receiving apparatus 1411.

  The difference between the configuration of the reception device 1411 and the reception device 103 in the first embodiment is that the viewpoint information extraction unit 906 is connected to the video decoder 904 in the first embodiment, whereas in the second embodiment, It is connected to the TS decoder 1703.

  A broadcast signal 1701 broadcast from the main line video distribution apparatus 1406 is input to the broadcast receiving unit 1702. The broadcast receiving unit 1702 decodes the broadcast signal 1701 and extracts a transport stream. Then, the broadcast receiving unit 1702 inputs this transport stream to the TS decoder 1703. As described above, this transport stream is obtained by multiplexing the video transport stream and the main video viewpoint information transport stream into one.

  Next, the TS decoder 1703 operates to separate these two types of transport streams, output the video to the video decoder 1704, and output the main video video viewpoint information to the viewpoint information extraction unit 906. Since the subsequent processing of the receiving apparatus 1411 is the same as that described in the receiving apparatus 103 of Embodiment 1, detailed description thereof is omitted.

  Thus, according to Embodiment 2, the modification to the existing main line video providing system can be minimized. In addition, the receiving apparatus 1411 is also expected to be easily realized by software processing as compared with the configuration of the first embodiment. In other words, it is possible to achieve the object with fewer changes to the existing video distribution service.

  In addition, as shown in FIG. 19, you may make it send viewpoint information using data broadcasting. In the viewpoint information encoding apparatus 1800 illustrated in FIG. 19, the viewpoint information list generation unit 1808 associates the time stamp of the video signal output from the PES packetization unit 1503 with the viewpoint information output from the viewpoint information conversion unit 704. Generate a viewpoint information list. Next, the viewpoint information list generation unit 1808 sends the generated viewpoint information list to the data broadcast carousel generation unit 1809. The data broadcast carousel generation unit 1809 encodes the viewpoint information list in the data broadcast carousel format, and sends the encoded information to the TS packetization unit 1810. Then, the TS packetizing unit 1810 converts the viewpoint information list encoded in the carousel format into a TS stream 1811 and outputs it.

  With such a configuration, viewpoint information can be sent using data broadcasting.

(Embodiment 3)
The third embodiment assumes that the main line video is separately provided on a package medium such as a DVD. FIG. 20 is a diagram for explaining the outline of the arbitrary viewpoint video system according to the third embodiment.

  In the third embodiment, the main line video and main line video viewpoint information provided online via a communication path such as a broadcast wave or the Internet in the first and second embodiments is stored in a package medium such as a DVD 1901 and distributed. It is what you do. The DVD 1901 stores an access method 1903 to an arbitrary viewpoint video providing system such as a URL.

  The viewer can view the main line video by playing the DVD 1901 on the reception / playback apparatus 1902 compatible with the arbitrary viewpoint service. Further, the viewer can acquire the access method to the arbitrary viewpoint providing system and the main line viewpoint information by reproducing the DVD 1901. The viewer can use the cooperation service with the arbitrary viewpoint video providing system as described in the first embodiment using the access method 1903 to the arbitrary viewpoint providing system and the main line viewpoint information. .

  Note that the package medium can be similarly applied to not only a DVD but also a next-generation package medium such as Blu-Ray disc (registered trademark) or a distribution type provided as storage-type content in server-type broadcasting.

  Next, the configuration of the arbitrary viewpoint video system according to the third embodiment will be described with reference to FIGS. 21 and 22. FIG. 21 is a block diagram of a production system for a package medium, and FIG. 22 is a diagram illustrating a playback device for the package medium, a receiving device for an arbitrary viewpoint video service, and an arbitrary viewpoint video providing system. In addition, the same code | symbol is attached | subjected to the part same as the already demonstrated part, and detailed description is abbreviate | omitted.

  The encoding device 2003 of the control system 2001 removes the TS packetizing unit 706 from the encoding device 503 of the first embodiment, and outputs a superimposition of the main line video signal and the viewpoint information parameter in the PES packet state. It is a thing.

  The DVD production apparatus 2004 receives the PES packet stream containing the main line video and main line video viewpoint information from the encoding apparatus 2003, and converts the PES packet stream into the DVD format. At this time, the DVD production apparatus 2004 describes the access method to the corresponding arbitrary viewpoint video service as data to be written to the DVD. This access method is, for example, a URL (Uniform Resource Locator) of a server that provides an arbitrary viewpoint video service.

  The DVD production apparatus 2004 generates a DVD disk image from the data thus created, and creates a target DVD (software) 1901 based on the generated disk image.

  The DVD software 1901 created in this way is generally distributed and sold, and is reproduced by the receiving / reproducing apparatus 1902 of the viewer who purchased the software.

  The reception / playback device 1902 is a device having a playback function of the DVD software 1901 and a reception / playback function of an arbitrary viewpoint video provided from the arbitrary viewpoint video distribution device 2105 via the network. Unlike the receiving apparatus 103 of the first embodiment, the receiving / playback apparatus 1902 has a DVD 1901 input, so that the part that handles the main line video is replaced with the DVD 1901 part from the broadcast part.

  The configuration of reception / playback apparatus 1902 will be described with reference to FIG. In the reception / playback device 1902, the input is a disc signal 2201, and the broadcast reception unit 902 and the TS decoder 903 of the reception device 103 according to the first embodiment are an analog front end 2202 and a PS decoder 2203, respectively. It is.

  The disc signal 2201 read by the DVD laser pickup is input to the analog front end 2202. The analog front end 2202 performs analog demodulation, error correction, and other processing on the input disc signal 2201 and outputs it in the MPEG2 program stream (PS) format. Then, the analog front end 2202 outputs this program stream to the PS decoder 2203.

  The PS decoder 2203 receives the program stream output from the analog front end 2202, decodes it into a PES data stream, and outputs it to the video decoder 2204.

  The video decoder 2204 outputs the main video 2205 from the PES data stream, extracts a video time stamp and main video video viewpoint information from the PES data stream, and outputs them to the viewpoint information extraction unit 906.

  Since the other processes of the reception / playback apparatus 1902 are the same as those of the reception apparatus 103 of Embodiment 1, detailed description thereof is omitted.

  According to the third embodiment, it is possible to provide a linkage service between video (main video) recorded on a package medium such as DVD software and arbitrary viewpoint video.

(Embodiment 4)
Next, a fourth embodiment will be described. In Embodiments 1 to 3, the main video viewpoint information is multiplexed with the main video stream and provided to the receiving apparatus. On the other hand, in the fourth embodiment, the main line video viewpoint information generated on the broadcast video providing system side is sent to the arbitrary viewpoint video providing system side together with the time stamp and managed. That is, the viewpoint of the main video at that time can be obtained from the time stamp of the main video on the arbitrary viewpoint video providing system side.

  By adopting such a configuration, it is not necessary to upgrade the broadcasting system such as an existing video encoding device or multiplexing device, so that there is a merit that it is easy for a broadcasting station to introduce.

  Next, an arbitrary viewpoint video system according to Embodiment 4 will be described with reference to FIG. FIG. 24 is a block diagram of an entire configuration of the arbitrary viewpoint video system according to the fourth embodiment.

  Compared with the second embodiment, the arbitrary viewpoint video system 2300 according to the fourth embodiment has a configuration in which the main line video viewpoint information generated by the viewpoint information encoding device 1404 is multiplexed in the configuration illustrated in FIG. In FIG. 24 showing the fourth embodiment, the main line video viewpoint information generated by the viewpoint information encoding device 2304 is displayed together with the main line video time stamp corresponding to the main line video viewpoint information. The main difference is that it is input to the main line video viewpoint information DB 2310 on the arbitrary viewpoint video providing system 2302 side.

  First, the viewpoint information encoding apparatus 2304 of the broadcast video providing system 2301 generates main line video viewpoint information and a time stamp, and registers them in the main line video viewpoint information DB 2310 of the arbitrary viewpoint video providing system 2302. Here, the viewpoint information encoding device 2304 and the main line video viewpoint information DB 2310 are connected by a communication line such as a TCP / IP network, and the main line video viewpoint information can be registered online.

  FIG. 25 is a diagram illustrating an example of a management structure of main line video viewpoint information in the main line video viewpoint information DB 2310.

  In this example, it consists of a main line video viewpoint information list 2402 in which a broadcast program ID 2401 for identifying a program that broadcasts a main line video and main line video viewpoint information are listed in time series according to a time stamp. The broadcast program ID setting method may be set from the viewpoint information encoding device 2304 or the like together with the main line video viewpoint information, or may be set manually by an operator or the like.

  Next, the receiving device 2311 will be described. FIG. 26 is a diagram illustrating a configuration of the reception device 2311 according to the fourth embodiment.

  The receiving device 2311 receives the broadcast signal 2501 output from the main line video distribution device 2305 in the broadcast receiving unit 902, decodes it by the TS decoder 903, decodes it by the video decoder 2504, and outputs it as a main line video 905. Along with this, the video decoder 2504 of the receiving device 2311 extracts the time stamp of the video at that time and supplies it to the viewpoint control unit 2506.

  When the user performs a viewpoint operation 913 for switching from the main line video to the arbitrary viewpoint video, the viewpoint control unit 2506 generates viewpoint operation information 2507. FIG. 27 is an example of the generated viewpoint operation information 2507. Unlike the viewpoint operation information in the first to third embodiments, since the main line video viewpoint information is not provided on the receiving apparatus side, the viewpoint operation information is designated relative to the current viewpoint.

  When switching to an arbitrary viewpoint video with the same viewpoint as the main video, the viewpoint operation with 0 specified for each of the time stamp and position (relative), viewing angle (relative), and viewing direction (relative) of the main video at the time of switching Use information.

  Using the viewpoint operation information 2507, the viewpoint control unit 2506 requests the arbitrary viewpoint video distribution device 2309 to start distribution (playback) of the arbitrary viewpoint video. Here, the viewpoint control unit 2506 sets the initial value of the viewpoint of the arbitrary viewpoint video at the start of distribution (playback) to the main video viewpoint at the specified time.

  On the other hand, the arbitrary viewpoint video distribution apparatus 2309 inquires and acquires the main line video viewpoint information DB 2310 for the main line video viewpoint corresponding to the time stamp specified by the viewpoint operation information 2507. The arbitrary viewpoint video distribution apparatus 2309 acquires an arbitrary viewpoint video corresponding to the acquired main line video viewpoint from the arbitrary viewpoint video DB 2308 and distributes it to the reception apparatus 2311.

  As described above, according to the fourth embodiment, particularly in a system for broadcasting a main line video, it can be realized by adding a viewpoint information encoding device 2304 to a new level without significantly affecting existing facilities. The merit that it is easy to introduce is obtained.

  The present invention is a distribution device and receiving terminal that distributes the same target as an arbitrary viewpoint video in cooperation with conventional video distribution (mainline video) such as broadcast distribution or live distribution on the Internet, distribution of arbitrary viewpoint video It can be applied to devices.

1 is a diagram for explaining an overview of an arbitrary viewpoint video system according to a first embodiment of the present invention; The figure which shows the screen at the time of the receiving apparatus concerning Embodiment 1 receiving the broadcast type program, The figure which shows the remote control concerning Embodiment 1. The figure which shows the screen during arbitrary viewpoint video viewing of the receiver concerning Embodiment 1. FIG. Block diagram of an arbitrary viewpoint video system according to the first embodiment The figure which shows the viewpoint information parameter concerning Embodiment 1. The figure which shows the camera for broadcast concerning Embodiment 1. FIG. Block diagram of internal configuration of encoding apparatus according to Embodiment 1 The figure which shows the video data packetized by the PES format concerning Embodiment 1 Block diagram of internal configuration of receiving apparatus according to Embodiment 1 The figure which shows the viewpoint operation information concerning Embodiment 1. The figure which showed the protocol sequence of the arbitrary viewpoint video acquisition of the arbitrary viewpoint video system concerning Embodiment 1 The figure which shows the other example of the encoding apparatus concerning Embodiment 1. FIG. The figure which shows the main-line video | video viewpoint information made into the RTP stream concerning Embodiment 1. FIG. FIG. 2 is a block diagram showing a configuration of an arbitrary viewpoint video system according to a second embodiment of the present invention. FIG. 3 is a block diagram showing an internal configuration of a video encoding device and a viewpoint information encoding device according to the second embodiment. The figure which shows the main line image | video viewpoint information which the MPEG2 sectioning part concerning Embodiment 2 encoded. Block diagram of receiving apparatus according to second embodiment The figure which shows the other example of the viewpoint information encoding apparatus concerning Embodiment 2. FIG. The figure explaining the outline | summary of the arbitrary viewpoint video system concerning Embodiment 3 of this invention. Block diagram of the production system for package media according to the third embodiment The figure which shows the reproduction | regeneration of the package media concerning Embodiment 3, the receiving apparatus of arbitrary viewpoint video services, and the arbitrary viewpoint video provision system Block diagram of a receiving and reproducing apparatus according to the third embodiment FIG. 6 is a block diagram showing an overall configuration of an arbitrary viewpoint video system according to a fourth embodiment of the present invention. The figure which showed an example of the management structure of the main image | video viewpoint information in the main image | video viewpoint information DB concerning Embodiment 4. FIG. FIG. 4 is a block diagram showing a configuration of a receiving apparatus according to the fourth embodiment. The figure which shows an example of the viewpoint operation information concerning Embodiment 4.

Explanation of symbols

100, 1400, 2300 Arbitrary viewpoint video system 101, 1401, 2301 Main line video providing system 102, 1402, 2302 Arbitrary viewpoint video providing system 103, 1411, 2311 Receiving device 501 Broadcast video shooting camera 503, 2003 Encoding device 504, 1406 2305 Main line video distribution device 505 Arbitrary viewpoint video shooting camera 506 Arbitrary viewpoint video generation device 507 Arbitrary viewpoint video DB
508, 2105, 2309 Arbitrary viewpoint video distribution device 1403 Video encoding device 1404, 2304 View point information encoding device 1405 Multiplexing device 1901 DVD
1902 Reception / Reproduction Device 2004 DVD Production Device 2310 Main Line Video Viewpoint Information DB

Claims (12)

  A reception unit that receives a main line video signal that is a broadcast video and a broadcast stream that includes viewpoint information for the main line video signal, a viewpoint information extraction unit that extracts the viewpoint information from the broadcast stream, and the viewpoint information as an initial value A viewpoint control unit that makes a distribution request for an arbitrary viewpoint video that is an image at an arbitrary viewpoint generated from a plurality of videos, and an arbitrary viewpoint video reception unit that receives distribution of an arbitrary viewpoint video based on the distribution request, A receiving device comprising the receiving device.   The viewpoint control unit requests the arbitrary viewpoint video of the viewpoint information for the main line video at that time when any direction of a cross button for instructing a direction is pressed. Item 4. The receiving device according to Item 1.   The main line video signal, which is a video for broadcasting, and the viewpoint information for the main line video signal to be used as the initial value of the viewpoint when accessing the arbitrary viewpoint video service at the receiving device are temporally related and encoded into a broadcast stream. A main line video distribution apparatus comprising: an encoding unit that performs distribution; and a main line video distribution unit that distributes the broadcast stream.   4. The main line video distribution apparatus according to claim 3, wherein the encoding unit describes the viewpoint information in a PES header of the main line video signal.   4. The main line video distribution apparatus according to claim 3, wherein the encoding unit describes the viewpoint information in a header part of a transmission line code of the main line video signal.   4. The main line video distribution apparatus according to claim 3, wherein the viewpoint information is described and distributed in a private_data_byte part of an MPEG2 section. A viewpoint information data generation unit that generates a viewpoint information list from the viewpoint information and a time stamp of the main line video signal, and a data broadcast carousel generation unit that encodes the viewpoint information list in a data broadcast carousel format,
4. The main line video distribution apparatus according to claim 3, wherein the viewpoint information list encoded in the carousel format is distributed. A receiving device according to claim 1, a main line video distribution device according to claim 3, and an arbitrary viewpoint video distribution device that generates and distributes the arbitrary viewpoint video,
The main line video delivery device delivers the broadcast stream to the receiving device,
When the receiving device receives the broadcast stream distributed and requests the arbitrary viewpoint video distribution device to distribute the arbitrary viewpoint video, the viewpoint information is used as an initial value of the viewpoint of the arbitrary viewpoint video. Using the viewpoint information extracted by the extraction unit,
The arbitrary viewpoint video distribution device generates the arbitrary viewpoint video of the viewpoint requested to be distributed from the receiving device and distributes the arbitrary viewpoint video to the receiving device.   The video signal is recorded, and the viewpoint information for the video signal recorded in association with the video signal is used as an initial value of the viewpoint when accessing the arbitrary viewpoint video service related to the video signal in the receiving device. A recording medium comprising the recording medium.   A reception / playback apparatus for playing back the viewpoint information for the video together with the video recorded on the recording medium according to claim 9, wherein the viewpoint information extraction unit extracts the viewpoint information from the reproduced data, and the viewpoint information As an initial value, a viewpoint control unit that performs distribution request of an arbitrary viewpoint video that is an image at an arbitrary viewpoint generated from a plurality of videos, and an arbitrary viewpoint video reception unit that receives distribution of an arbitrary viewpoint video based on the distribution request, A receiving and reproducing apparatus comprising:   A main line viewpoint information database for storing a main line viewpoint information list generated by temporally associating and recording time stamps of main line video signals, which are broadcast videos, and viewpoint information for the main line video signals, from a receiving device. An arbitrary-viewpoint video distribution apparatus that distributes an arbitrary-viewpoint video with the origin of the main-line video viewpoint information recorded in the main-line viewpoint information in response to the distribution request.   12. A distribution request for a main video signal that is a broadcast video, a video decoder that reproduces the main video signal and extracts a video time stamp, and a video distribution device according to claim 11. A receiving apparatus comprising: a viewpoint control unit that makes a distribution request using the extracted time stamp at a time point; and an arbitrary viewpoint video receiving unit that receives distribution of an arbitrary viewpoint video based on the distribution request .

Images