JP2009171294A - Video distribution system, video relay apparatus, and video relay method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To distribute video streams from which video can be reproduced continuously. <P>SOLUTION: The video distribution system comprises: a video distribution device for transmitting video streams and a video relay apparatus for receiving the plurality of transmitted video streams and transferring at least one of the received video streams to a video receiving device. The video distribution device adds to a video stream a first identifier for identifying whether data of an independently decodable unit included in the video stream corresponds in time or not, and transmits the video stream to which the first identifier has been added, and the vide relay apparatus specifies, based on first identifiers added to the received video streams, the data of the independently decodable unit of the video streams transferred after switching in a case where the video stream is switched to another video stream in the independently decodable unit, and transfers the video stream transferred after switching from the specified data of the decodable unit. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to a video distribution system for distributing video streams, and more particularly to a method for switching a plurality of video streams.

  With the spread of network technology, a technology for transmitting a video stream using an Internet protocol (IP: Internet Protocol) has been developed. Such video stream transmission techniques include a method of distributing a video stream in real time using IP multicast and a method of downloading a video stream and reproducing the stored video data. As a method for encoding video data, a method using MPEG2 (Moving Picture Experts Group 2) is known. Further, as a method for distributing a video stream, a method for distributing a stream in real time by IP multicast using RTP / UDP (Real-time Transport Protocol / User Datagram Protocol) has been studied.

  Furthermore, video communication systems have been proposed that can distribute video from a plurality of viewpoints and select the video according to the user's wishes as the communication line becomes wider (see, for example, Patent Document 1). The video distribution device disclosed in Patent Document 1 generates additional information indicating the content of a video for each video frame, and distributes a video stream based on the generated additional information.

  There is also known a video presentation device that automatically selects a video stream according to a user's preference from video streams shot from a plurality of viewpoints (see, for example, Patent Document 2). The video presentation apparatus disclosed in Patent Document 2 automatically selects a video stream according to the user's preference using metadata including the subject identifier of the video stream for each video frame.

  Furthermore, when switching video channels, a system is known that switches from the I frame of the video channel to be switched to (see, for example, Patent Document 3). The channel converter disclosed in Patent Document 3 switches the video channel from the I frame stored in the switching destination buffer and distributes the video stream.

On the other hand, in a video encoding method that predicts a frame based on the similarity between frames as in MPEG2, there are frames called I frame, P frame, and B frame.
An I frame is a frame generated without using prediction between frames, and can be reproduced independently. The P frame is a frame reproduced by forward prediction (forward prediction) from other I frames and P frames. Specifically, a P frame is a frame that is compression-coded using an I frame or a P frame that exists in time. Therefore, in order to decode the P frame, the I frame or the P frame used for the compression encoding is necessary.

The B frame is a frame that is compression-coded using data of an I frame and a P frame that exist before and after in time. Therefore, in order to decode the B frame, the I frame or the P frame used for the compression encoding is necessary.
JP 2003-179908 A JP 2004-312208 A Special table 2001-516184 gazette

  When switching between multiple viewpoint videos and transmitting them to the video receiving terminal, the video coding method using inter-frame prediction, such as MPEG2, uses the information of the previous and subsequent frames for video playback. If the stream is not switched, the terminal that receives the switched stream cannot correctly reproduce the video. In MPEG2, a plurality of frames are managed as one group, and this group is called GOP (Group of Pictures). The GOP includes one or more I frames, a plurality of P frames, and a plurality of B frames, and is a video playback unit.

  When editing video without re-encoding, it is necessary to perform it in units of video playback. Therefore, for example, when decoding from video data of P frame or B frame without decoding from video data of I frame, the video to be reproduced is disturbed.

  In addition, in order to switch the video so that the video receiving terminal can correctly reproduce the stream, it is necessary to switch the video stream in consideration of the playback unit of the video stream.

  However, since the above-described conventional techniques (Patent Document 1 and Patent Document 2) switch the video in units of video frames, there is a first problem that the video cannot be switched in units of playback of the video stream.

  In addition, when video is distributed from a plurality of video distribution servers using an IP network, since each video stream is not distributed synchronously, a new transfer is performed immediately after the end point of the stream (end stream) to end the transfer. The starting point of the starting stream (starting stream) does not always arrive. Also, since the arrival order of IP packets is not guaranteed in the IP network, the stream switching point notification may arrive at the video relay server before or after the stream packet to be switched. If the stream is switched when the notification arrives, the start stream is transmitted from the start point, and the transmission of the end stream cannot be ended at the end point.

  Further, when the channel converter disclosed in Patent Document 3 receives an instruction to switch channels, the channel converter distributes the video stream from the I frame that is the switching destination without depending on the position of the playback frame before switching the channel. Therefore, when the channel is switched in the middle of the video frame before switching, only a part of the data constituting the video frame is transmitted, so that there is a second problem that the reproduced video is disturbed.

  FIG. 20 shows the relationship between the transmission order of video frames being transmitted and the playback order of video frames being played. As shown in FIG. 20, the I-frame or P-frame data is reproduced after the video of the subsequent B frame is decoded. Therefore, for example, when the video stream is switched immediately before the B (3) frame as indicated by 450 in FIG. 20, on the playback side, although the B (3) and B (4) frames are missing, , P (5) frame is reproduced. As described above, there is a third problem that the reproduced video is discontinuous.

  Also, as the timing for switching the video, for example, when the video stream is switched immediately before the P (11) frame indicated by 451 in FIG. 20, on the playback side, the B (6) frame, B (7) frame, and P ( 8) Since the frame is reproduced, the reproduced video is not discontinuous. However, among the 15 frames from B (0) to P (14) as a GOP unit, since 6 video frames are missing, the amount of video data to be reproduced as a whole is reduced. There are challenges.

  Furthermore, when switching video streams, in order to switch video data so that video frames are not lost, a method of switching video data in units of GOPs before and after switching the video data is conceivable. However, in the IP network, since transmission of video frames is not performed synchronously, there is a fifth problem that switching of GOP units synchronized between video streams cannot be performed.

  In order to solve such a problem, the present invention prevents the video receiving terminal from reproducing video when switching from a plurality of stream-distributed videos to a video desired by the user and delivering it to the video receiving terminal. An object is to distribute a video stream that can be continuously played back from a video relay server.

  It is another object of the present invention to provide a video stream distribution system that distributes a video stream by switching to a GOP unit.

  A typical example of the present invention is as follows. Specifically, a video distribution system comprising: a video distribution device that transmits a video stream; and a video relay device that receives the plurality of transmitted video streams and transfers at least one of the received video streams to the video reception device. The video distribution apparatus may include a first identifier for identifying whether or not data in a unit that can be independently decoded included in the video stream corresponds to data in time. The video relay apparatus transmits the video stream to which the first identifier is added, and the video relay apparatus switches the video stream to the received video stream when switching the video stream in units that can be independently decoded. Based on the added first identifier, the data of the decodable unit of the video stream transferred after switching is specified, and the switching is performed. The video stream transferred later to transfer the data of the identified decodable units.

  According to an embodiment of the present invention, video can be switched so that playback is possible without interruption in consideration of a playback unit of a video stream.

<First Embodiment>
FIG. 1 is a system configuration diagram showing the configuration of the video distribution system according to the first embodiment of the present invention.

  The video distribution system according to the first embodiment includes a video distribution device 1, a metadata distribution device 2, a video relay device 3, a video reception device 4A, a video reception device 4B, a network A5, and a network B6. Hereinafter, the video receiving devices 4A and 4B may be collectively referred to as the video receiving device 4.

  The network A5 connects the video distribution device 1, the metadata distribution device 2, and the video relay device 3. The network B6 connects the video relay device 3, the video receiving device 4A, and the video receiving device 4B.

  The video distribution device 1 distributes a multi-view video stream (video data) to the video relay device 3. The metadata distribution device 2 distributes metadata indicating the content of the video stream distributed from the video distribution device 1 to the video relay device 3.

  The video relay device 3 receives the multi-view video based on the multi-view video stream received from the video delivery device 1, the metadata received from the metadata delivery device 2, and the user request received from the video receiving devices 4A and 4B. Outputs personalized video stream data from the stream. Then, the output video stream data is distributed to the video receivers 4A and 4B. Note that the data of the output video stream may be reproduced by, for example, a display device connected to the video relay device 3.

  The video receiving devices 4A and 4B transmit a user request to the video relay device 3 and reproduce the data of the video stream received from the video relay device 3.

  FIG. 2 is a sequence diagram of video stream data transmission / reception processing of the video distribution system according to the first embodiment of this invention.

  First, after the apparatus is activated, the video distribution apparatus 1 performs 0 value initial setting of the GOP number in order to distribute video data (step 501). The 0 value initial setting of the GOP number means that the GOP number given when the video stream of each viewpoint is packetized is initialized to zero.

  Next, in the metadata distribution apparatus 2, after the apparatus is activated, metadata information distributed to the video relay apparatus 3 is registered by the administrator (step 502).

  Next, after the apparatus is activated, the video receiving apparatus 4A transmits participation request data for the video distribution service to the video relay apparatus 3 (step 503).

  Next, the video relay device 3 receives the participation request data for the video distribution service transmitted from the video reception device 4A, and the identifier (for example, the video reception device) of the video reception device 4A that requested the participation in the video distribution service. 4A IP address) is registered (step 504).

  Next, since the video receiving device 4 participating in the video distribution service has occurred, the video relay device 3 requests the metadata distribution device 2 for metadata of the video data to be distributed (step 505).

  Next, when receiving the metadata request from the video relay apparatus 3, the metadata distribution apparatus 2 distributes the metadata information registered in step 502 to the video relay apparatus 3 (step 506).

  Next, the video relay device 3 registers the metadata information received from the metadata distribution device 2 (step 507). Then, the video relay apparatus 3 transmits a multi-view video data distribution request for providing the service to the video distribution apparatus 1 (step 508).

  Next, the video distribution apparatus 1 receives a video data distribution request from the video relay apparatus 3 and distributes multi-viewpoint video data to the video relay apparatus 3 using a multicast method (step 509).

  Next, the video relay apparatus 3 starts to receive multi-viewpoint video data distributed from the video distribution apparatus 1 and buffers the received video data until the maximum delay video data for 1 GOP is received. (Step 510). Then, the personalized video data is distributed to the video receiver 4A according to the initial setting (for example, the setting of the video data of the viewpoint received with the smallest delay as the initial distribution data) (step 511).

  Next, the video reception device 4A reproduces the received personalized video data (step 512).

  Next, the video reception device 4A receives request data (for example, “A player”) input by the user (step 513). Then, the input user request data is transmitted to the video relay device 3 (step 514).

  Next, the video relay device 3 receives the user request data transmitted from the video receiving device 4A, and registers the received user request data (step 515).

  Next, the video distribution device 1 continuously distributes multi-viewpoint video data (step 516).

  Next, the video relay apparatus 3 identifies the video of the viewpoint that matches the user request data received in Step 515 from the metadata registered in Step 507, and switches the viewpoint in GOP units. The switched viewpoint video data is distributed to the video receiver 4A as personalized video data (step 517).

  Next, the video receiving device 4A receives the personalized video data distributed from the video relay device 3, and reproduces the received personalized video data (step 518).

  Next, after the apparatus is activated, the video receiving apparatus 4B transmits participation request data for the video distribution service to the video relay apparatus 3 (step 519).

  Next, the video relay device 3 receives the participation request data for the video distribution service transmitted from the video reception device 4B, and registers the identifier (IP address) of the video reception device 4B that requested the participation in the video distribution service. (Step 520).

  Next, the video distribution apparatus 1 continuously distributes multi-viewpoint video data (step 521).

  Next, the video relay device 3 continuously distributes the video data of the viewpoint specified in step 517 as personalized video data to the video receiving device 4A (step 522).

  Next, the video receiving device 4A receives the personalized video data distributed from the video relay device 3, and reproduces the received personalized video data (step 523).

  Next, the video relay device 3 then distributes the personalized video data to the video receiving device 4B according to the initial setting (for example, setting the video data of the viewpoint received with the smallest delay as the initial distribution data) (step 524). ).

  Next, the video reception device 4B receives the personalized video data distributed from the video relay device 3, and reproduces the received personalized video data (step 525).

  FIG. 3 is a block diagram illustrating a configuration of the metadata distribution device 2 according to the first embodiment of this invention.

  The metadata distribution device 2 includes an input interface 21, a CPU 23, a main memory 24, a program storage unit 25, and a transmission unit 26. The input interface 21, CPU 23, main memory 24, program storage unit 25, and transmission unit 26 are connected by a bus 29.

  The input interface 21 is an interface through which a service administrator inputs metadata indicating the content of video to be distributed. The input interface 21 may be configured with a keyboard or the like, for example.

  The CPU 23 executes an OS (Operating System) and various application programs. The main memory 24 temporarily stores data necessary for the CPU 23 to execute various application programs. Further, at least a part of the program stored in the program storage unit 25 is copied to the main memory 24 as necessary. The program storage unit 25 stores various application programs. The transmission unit 26 is an interface that distributes metadata input by the administrator via the network A5.

  Next, the operation of the metadata distribution device 2 will be described.

  FIG. 4 is a flowchart of the metadata distribution processing according to the first embodiment of this invention.

  The metadata distribution device 2 reads out the program stored in the program storage unit 25 to the CPU 23, and the CPU 23 executes the read program to start the metadata distribution process (step 200).

  First, the metadata distribution device 2 determines whether or not the administrator has input metadata (step 201).

  If the administrator inputs metadata in the process of step 201, the process proceeds to step 202. On the other hand, if it is determined in step 201 that the administrator has not input metadata, the processing in step 201 is repeated to determine whether metadata has been input from the administrator.

  Next, the metadata distribution device 2 determines whether or not a metadata request is received from the video relay device 3 (step 202).

  If a metadata request is received from the video relay apparatus 3 in the process of step 202, the process proceeds to step 203. On the other hand, when the metadata request is not received from the video relay apparatus 3 in the process of step 202, the process of step 202 is repeated to determine whether the metadata request is received from the video relay apparatus 3.

  Next, the metadata distribution device 2 distributes the metadata information input from the administrator to the video relay device 3 (step 203).

  FIG. 5 is an explanatory diagram showing a configuration of a metadata transmission packet distributed from the metadata distribution device 2 to the video relay device 3. The distributed metadata is converted into a UDP packet and an IP packet, and distributed from the metadata distribution device 2 to the video relay device 3. The metadata may include, for example, information of “view point 1: A player”.

  Next, the metadata distribution device 2 determines whether or not the administrator has input metadata (step 204).

  If the administrator inputs metadata in the process of step 204, the process proceeds to step 205. On the other hand, if it is determined in step 204 that the administrator has not input metadata, the processing in step 204 is repeated to determine whether metadata has been input from the administrator.

  Next, the metadata distribution device 2 distributes the metadata information input from the administrator to the video relay device 3 (step 205). Then, the process returns to step 204 to determine whether or not the administrator has input metadata to be distributed.

  FIG. 6 is a block diagram illustrating a configuration of the video distribution device 1 according to the first embodiment of this invention.

  The video distribution device 1 includes video cameras 11A to 11N, encoders 12A to 12N, a CPU 13, a main memory 14, a program storage unit 15, and a transmission unit 16. The video cameras 11A to 11N, the encoders 12A to 12N, the CPU 13, the main memory 14, the program storage unit 15, and the transmission unit 16 are connected by a bus 19, respectively. Hereinafter, the video cameras 11 </ b> A to 11 </ b> N may be collectively referred to as the video camera 11. The encoders 12A to 12N may be collectively referred to as the encoder 12 when collectively referred to.

  The video camera 11 captures images from a plurality of viewpoints. The encoder 12 encodes the video data of each viewpoint photographed by the video camera 11 (for example, MPEG2 compression encoding). The CPU 13 executes an OS (Operating System) and various application programs. The main memory 14 temporarily stores data necessary for the CPU 13 to execute various application programs. Also, at least a part of the program stored in the program storage unit 15 is copied to the main memory 14 as necessary. The program storage unit 15 stores various application programs. The transmission unit 16 is an interface that distributes the encoded video data via the network A5.

  Next, the operation of the video distribution device 1 will be described.

  FIG. 7 is a flowchart of video distribution processing according to the first embodiment of this invention.

  The video distribution apparatus 1 reads out the program stored in the program storage unit 15 to the CPU 13 and starts the video distribution process by the CPU 13 executing the read program (step 250).

  First, the video distribution apparatus 1 performs 0 value initial setting of the GOP number in order to distribute video data (step 251).

  Next, the video distribution apparatus 1 determines whether a video distribution request has been received from the video relay apparatus 3 (step 252).

  If a video distribution request is received from the video relay device 3 in step 252, the process proceeds to step 253. On the other hand, if the video delivery request is not received from the video relay device 3 in the process of step 252, the process of step 252 is repeated to determine whether a video delivery request is received from the video relay device 3.

  Next, the video distribution apparatus 1 encodes video data of a plurality of viewpoints in synchronization (step 253). Specifically, the video distribution device 1 compresses and encodes the video data by aligning the GOP numbers given to the video data of each viewpoint.

  Next, the video distribution device 1 determines whether or not video frame data has been received from all the encoders 12 (step 254).

  If the video frame data is received from all the encoders 12 in the process of step 254, the process proceeds to step 255. On the other hand, if the video frame data has not been received from all the encoders 12 in the process of step 254, the process of step 254 is repeated to determine whether video frame data has been received from all the encoders 12.

  Next, the video distribution device 1 determines whether or not the received video data includes GOP head data (GOP head frame data) (step 255).

  If it is determined in step 255 that the received video data includes the GOP head data, the process proceeds to step 256. On the other hand, if the received video data does not include the top data of the GOP in the process of step 255, the process proceeds to step 257.

  In step 256, the video distribution apparatus 1 adds a GOP head identifier to the head RTP packet that transmits the video data, and distributes the video data as an RTP packet (step 256). Specifically, the video distribution apparatus 1 increments the GOP number by 1 and adds it to the RTP packet.

  In step 257, the video distribution apparatus 1 assigns the head identifier of the video frame to the head RTP packet that transmits the video data, and distributes the video data as an RTP packet (step 257). Specifically, the video distribution device 1 assigns the GOP number assigned to the top data of the same GOP to the RTP packet.

  Note that the process of assigning the start identifier of the video frame to the RTP packet and the process of assigning the start identifier of the GOP will be described later with reference to FIGS. In the present embodiment, the process is performed by identifying the GOP head data. However, when a sequence is configured for each GOP, the process of step 255 may be performed for each sequence.

  FIG. 8 is an explanatory diagram illustrating a configuration of the RTP header according to the first embodiment of this invention.

  The P bit 270 of the RTP header is an unused padding area. The M bit 271 is a bit indicating the boundary of the video frame data. In the first embodiment, the P bit 270 and the M bit 271 are used to notify the video relay apparatus 3 that the RTP packet includes the top data of the GOP or the top data of the frame. A specific method using the P bit 270 and the M bit 271 will be described later with reference to FIG.

  FIG. 9 is an explanatory diagram illustrating a relationship between the P bit and M bit, the GOP, and the head of the video frame data in the RTP header according to the first embodiment of this invention.

  As shown in FIG. 9, in the case of an RTP packet that transmits the head data of a GOP, P bit = 1 and M bit = 1 are set. In the case of an RTP packet that transmits the head data of a video frame, but not the head of the GOP, P bit = 0 and M bit = 1 are set. In the case of an RTP packet that transmits data that is not the head of a GOP or the head of a video frame, P bit = 0 and M bit = 0 are set. Therefore, it is possible to detect whether the received video data transmitted by the RTP packet includes the top data of the GOP or whether the top of the video frame data is included.

  FIG. 10 is an explanatory diagram illustrating a configuration of the extension header of the RTP packet according to the first embodiment of this invention.

  The extension header of the RTP packet shown in FIG. 10 is used to indicate the GOP sequence number.

  The extension header of the RTP packet includes the type of extension header, the data length, and the GOP number. By setting the X bit 272 indicating the presence or absence of the extension header of the RTP header shown in FIG. 8 (for example, X = 1 is set), the extension header is validated and used.

  FIG. 11 is an explanatory diagram illustrating a configuration of a packet in which the MPEG2-TS packet according to the first embodiment of this invention is converted into an RTP packet, a UDP packet, and an IP packet.

  A packet 280 shown in FIG. 11 shows the configuration of an IP packet used when transmitting the head data of a new GOP. A packet 281 indicates the configuration of an IP packet used when transmitting the head data of a video frame, not the head of the GOP. A packet 282 indicates the configuration of an IP packet used when transmitting video data that is not the head of a GOP or the head of a video frame.

  The packet 280 includes an IP header (IP_H), a UDP header (UDP_H), an RTP header (RTP_H), an RTP extension header (extension_H), and MPEG2-TS-1 to 7. P bit = 1 and M bit = 1 are set in the RTP header 285, and a GOP sequence number (GOP number) is given to the extension header. On the other hand, the packet 281 has the same configuration as the packet 280, but P bit = 0 and M bit = 1 are set in the RTP header 286, and the GOP sequence number is assigned to the extension header. In the case of GOP video data shot and encoded at the same time, the same GOP number is given. The packet 282 has the same configuration as that of the packet 280, but P bit = 0 and M bit = 0 are set in the RTP header 287, and the GOP sequence number is assigned to the extension header.

  Note that the same GOP number is set in the extension headers of RTP packets that transmit data belonging to the same GOP. Further, the example shown in FIG. 11 shows a configuration in which each packet includes seven MPEG2-TS packets, but may have a configuration including a number other than seven MPEG2-TS packets.

  FIG. 12 is a block diagram illustrating a configuration of the video relay device 3 according to the first embodiment of this invention. The video relay device 3 includes a reception unit 32, a CPU 33, a main memory 34, a program storage unit 35, and a transmission unit 36. The reception unit 32, the CPU 33, the main memory 34, the program storage unit 35, and the transmission unit 36 are connected by a bus 39, respectively.

  The receiving unit 32 is an interface that receives video data, metadata, and the like. The CPU 33 executes an OS (Operating System) and various application programs. The main memory 34 temporarily stores data necessary for the CPU 33 to execute various application programs. In the main memory 34, at least a part of the program stored in the program storage unit 35 is copied as necessary. The program storage unit 35 stores various application programs. The transmission unit 36 is an interface that transmits video data, metadata requests, and the like.

  Next, the operation of the video relay device 3 will be described.

  FIG. 13 is a flowchart of video relay processing according to the first embodiment of this invention.

  The video relay device 3 reads out the program stored in the program storage unit 35 to the CPU 33, and starts the video relay processing by the CPU 33 executing the read program (step 300).

  First, the video relay device 3 determines whether or not a request for participation in the video distribution service has been received from the video receiving device 4 (step 301).

  If a request for participation in the video distribution service is received from the video receiving device 4 in the process of step 301, the process proceeds to step 302. On the other hand, if the request for participation in the video distribution service has not been received from the video reception device 4 in the process of step 301, has the process of step 301 been repeated to determine whether the request for participation in the video distribution service has been received from the video reception device 4? Determine whether or not.

  Next, the video relay apparatus 3 registers the identifier (for example, IP address) of the terminal (video receiving apparatus 4) that has transmitted the request for participation in the video distribution service in the main memory 34 (step 302).

  Next, the video relay device 3 transmits a metadata request to the metadata distribution device 2 (step 303).

  Next, the video relay device 3 determines whether or not metadata has been received from the metadata distribution device 2 (step 304).

  If the metadata is received from the metadata distribution device 2 in the process of step 304, the process proceeds to step 305. On the other hand, if the metadata is not received from the metadata delivery apparatus 2 in the process of step 304, the process of step 304 is repeated to determine whether the metadata is received from the metadata delivery apparatus 2.

  Next, the video relay device 3 registers the received metadata in the main memory 34 (step 305).

  Next, the video relay device 3 transmits a video data distribution request to the video distribution device 1 (step 306).

  Next, the video relay device 3 receives the video data transmitted from the video distribution device 1, and determines whether or not 1 GOP data of the video data of each viewpoint has been buffered (step 307). Note that the number of GOP data for determining whether the buffering is performed may be changed according to the load on the network (network A5) between the video distribution apparatus 1 and the video relay apparatus 3. For example, it may be determined whether or not 3 GOPs of video data of each viewpoint can be buffered.

  If the data of 1 GOP can be buffered from each viewpoint in the process of step 307, the process proceeds to step 308. On the other hand, if the data of 1 GOP cannot be buffered from each viewpoint in the process of step 307, the process of step 307 is repeated to determine whether the data of 1 GOP at each viewpoint has been buffered.

  Next, the video relay device 3 performs setting for distributing video data personalized from the received video data of a plurality of viewpoints (step 308). Specifically, the video data of the viewpoint to be distributed is selected based on the metadata registered in the process of step 305 and the user's request for viewing (user request registered in the process of step 313). If no user request is registered, video data of an initial viewpoint (for example, video data of a viewpoint with the smallest viewpoint number set for each viewpoint or video data received with a minimum delay) is obtained. select. Then, a setting is made to distribute the video data of the selected viewpoint to the video receiving device 4 registered in the process of step 302.

  Next, the video relay apparatus 3 transmits the video data for one frame of the video data selected as the personalized video in step 308 to the set delivery destination (step 309). If a plurality of video receiving apparatuses 4 that distribute the same video data are registered, the selected personalized video data is distributed to the plurality of registered video receiving apparatuses 4.

  Next, the video relay device 3 determines whether video data has been received from the video distribution device 1 (step 310).

  If the video data is received from the video distribution device 1 in the process of step 310, the process proceeds to step 311. On the other hand, if the video data is not received from the video distribution device 1 in the process of step 310, the process proceeds to step 312.

  Next, the video relay device 3 stores the received video data in the buffer of the main memory 34 (step 311).

  Next, the video relay device 3 determines whether a user request has been received from the video reception device 4 (step 312).

  If the user request is received from the video reception device 4 in the process of step 312, the process proceeds to step 313. On the other hand, when the user request is not received from the video receiving device 4 in the process of step 312, the process proceeds to step 314.

  Next, the video relay apparatus 3 registers the received user request in the main memory 34 (step 313).

  Next, the video relay device 3 determines whether or not a service participation request has been received from a new user (step 314).

  If a new service participation request is received in step 314, the process proceeds to step 315. On the other hand, if a new service participation request has not been received in the process of step 314, the process proceeds to step 316.

  Next, the video relay apparatus 3 registers the identifier (for example, IP address) of the terminal (video receiving apparatus 4) that has transmitted the request for participation in the video distribution service in the main memory 34 (step 315).

  Next, the video relay device 3 determines whether or not new metadata has been received from the metadata distribution device 2 (step 316).

  If new metadata is received from the metadata distribution device 2 in the process of step 316, the process proceeds to step 317. On the other hand, if new metadata has not been received from the metadata distribution device 2 in the process of step 316, the process proceeds to step 318.

  Next, the video relay apparatus 3 registers the received metadata in the main memory 34 (step 317).

  Next, the video relay device 3 determines whether or not video data for 1 GOP has been distributed (step 318).

  If video data for 1 GOP has been distributed in the process of step 318, the process proceeds to step 319. On the other hand, if the video data for 1 GOP is not distributed in the process of step 318, the process returns to step 309.

  Next, the video relay apparatus 3 discards video data of the same GOP number as the GOP number of the video data of the transmitted viewpoint among video data of the viewpoint that has not been transmitted (that is, video data for 1 GOP is discarded). (Step 319). Then, the process returns to step 308 to continue the personalized video distribution process.

  Through the above processing, the personalized video data can be switched in units of GOP data. Also, since the video data is switched to the next sequence in units of GOP data based on the sequence number assigned to each GOP data, it is possible to reproduce a personalized video stream without any disturbance.

  Further, even if GOP data is partially lost in the video data transmitted from the video distribution device 1, the video data is switched based on the sequence number of the GOP data, so that synchronization is continuously performed. Can do.

  In addition, since synchronization is performed for each GOP data, even if the video stream is switched, it is possible to prevent the switching back and forth in time.

  FIG. 14 is a block diagram illustrating a configuration of the video reception device 4 according to the first embodiment of this invention.

  The video reception device 4 includes a reception unit 42, a transmission unit 46, a display unit 48, a CPU 43, a main memory 44, a program storage unit 45, and an input interface 41. The reception unit 42, the transmission unit 46, the display unit 48, the CPU 43, the main memory 44, the program storage unit 45, and the input interface 41 are connected by a bus 49, respectively.

  The receiving unit 42 is an interface that receives video data. The transmission unit 46 is an interface that transmits a request for participation in the video distribution service and user request data. The CPU 43 executes an OS (Operating System) and various application programs. The main memory 44 temporarily stores data necessary for the CPU 43 to execute various application programs. In the main memory 44, at least a part of a program or the like stored in the program storage unit 45 is copied as necessary. The program storage unit 45 stores various application programs. The input interface 41 is an interface for inputting a user request. The input interface 41 may be configured with a keyboard or the like, for example. The display unit 48 displays the received video data.

  Next, the operation of the video reception device 4 will be described.

  FIG. 15 is a flowchart of video reception processing according to the first embodiment of this invention.

  The video receiving device 4 reads out the program stored in the program storage unit 45 to the CPU 43, and starts the video receiving process when the CPU 43 executes the read program (step 350).

  First, the video receiving device 4 transmits a request for participation in the video transmission service to the video relay device 3 in order to receive video data (step 351).

  Next, the video receiving device 4 determines whether or not personalized video data has been received from the video relay device 3 (step 352).

  If personalized video data is received from the video relay device 3 in the process of step 352, the process proceeds to step 353. On the other hand, if the personalized video data is not received from the video relay device 3 in the processing of step 352, the processing proceeds to step 354.

  Next, the video relay device 3 reproduces the received personalized video data (step 353).

  Next, the video relay apparatus 3 determines whether or not a user request is input from the input interface 41 (step 354). For example, it is determined whether or not the data desired by the user (“A player”) has been input.

  If a user request is input in step 354, the process proceeds to step 355. On the other hand, if it is determined in step 354 that no user request has been input, the process returns to step 352 and the video reception process is continued.

  Next, the video relay device 3 transmits the input user request data to the video relay device 3 (step 355). Then, the process returns to step 352 to continue the video reception process.

  FIG. 16 is an explanatory diagram of the display screen 50 displayed on the display unit 48 of the video reception device 4 according to the first embodiment of this invention. As shown in FIG. 16, the display screen 50 includes a user request input interface 55 for inputting a user's desire to view and a display screen 51 for reproducing personalized video stream data received from the video relay device 3.

  As shown in FIG. 21, the first embodiment of the present invention provides a system for switching and delivering video streams (viewpoints) in units of GOPs in video streams from a plurality of viewpoints received asynchronously. Specifically, when the viewpoint is switched, as shown in the reproduction flow 460, after the first GOP data (GOP-A1) of the viewpoint A is distributed, the second GOP data (GOP-B2) of the viewpoint B is distributed. To deliver. Similarly, a system is provided in which viewpoints are switched sequentially in units of GOPs, and video streams are switched and distributed as in GOP-C3 and GOP-D4.

  As described above, according to the first embodiment of the present invention, the video stream is switched in synchronism with each unit in which the video stream can be reproduced, so that the video stream can be switched without any disturbance.

  In addition, since a plurality of video streams are received, synchronized in units of GOPs, and video data is switched in units of synchronized GOPs, when switching the video data, before and after the time of the switched video data does not occur. It is possible to create a personalized video that can be viewed along the flow of the video.

  In addition, video data that matches the user's wishes is cut out from a plurality of video data in GOP units to create one personalized video stream, so there is no need to change the video stream (the application executed on the client is again The video data can be switched and played back at high speed.

  In addition, since GOP delimiter information is assigned to the RTP header using the P bit and the video is switched in units of GOPs, it is possible to create personalized video at high speed and create multiple personalized videos with less processing power. Can do.

  Further, since a sequence identifier for each GOP is added to the RTP extension header, even if a packet loss of GOP data occurs, synchronization can be continued.

<Second Embodiment>
In the first embodiment, the video relay device 3 distributes only the personalized video data to the video receiving device 4. In the second embodiment, the video relay device 3 distributes the original multi-view video data received from the video distribution device 1 to the video reception device 4 in addition to the personalized video data.

  FIG. 17 is a system configuration diagram illustrating a configuration of a video distribution system according to the second embodiment of this invention. The basic system configuration is the same as that of the first embodiment. The difference from the first embodiment is that the video relay device 3 transmits multi-viewpoint video data to the video reception device 4.

  FIG. 18 is a video stream data transmission / reception processing sequence diagram of the video distribution system according to the second embodiment of this invention.

  In the process shown in FIG. 18, the same processes as those in FIG. 2 are given the same numbers. In the following, a description will be given focusing on differences from the processing shown in FIG.

  The processing from step 501 to step 511 is the same as the processing from step 501 to step 511 shown in FIG.

  Next, the video relay device 3 distributes the multi-viewpoint video data received from the video distribution device 1 to the video reception device 4A (step 550).

  Next, the video receiving device 4A reproduces the received personalized video data and multi-view video data (step 551).

  The processing from step 513 to step 517 is the same as the processing from step 513 to step 517 shown in FIG.

  Next, the video relay device 3 distributes the multi-viewpoint video data received from the video distribution device 1 to the video reception device 4A (step 552).

  Next, the video receiving device 4A reproduces the received personalized video data and multi-view video data (step 553).

  The processing from step 519 to step 522 is the same as the processing from step 519 to step 522 shown in FIG.

  Next, the video relay device 3 distributes the multi-viewpoint video data received from the video distribution device 1 to the video reception device 4A (step 554).

  Next, the video receiving device 4A reproduces the received personalized video data and multi-view video data (step 555).

  The processing in step 524 is the same as the processing in step 524 shown in FIG.

  Next, the video relay apparatus 3 distributes the multi-viewpoint video data received from the video distribution apparatus 1 to the video reception apparatus 4B (step 556). In the example shown in FIG. 18, the multi-view video data is described as being redistributed as video data different from the received multi-view video, but the multi-view video data is converted using the multicast method. In the case of distribution, the same data as the received multi-view video is multicast distributed.

  Next, the video receiving device 4B reproduces the received personalized video data and multi-view video data (step 557).

  FIG. 19 is a flowchart of video relay processing according to the second embodiment of this invention. In the video relay process shown in FIG. 13, the video relay apparatus 3 did not distribute the video data of the individual viewpoints. However, in the video relay process shown in FIG. 19, the video relay apparatus 3 receives the video data of the individual viewpoints. To deliver.

  In the process of FIG. 19, the same processes as those of FIG. 13 are given the same numbers. In the following, a description will be given focusing on differences from the processing shown in FIG.

  The processing from step 300 to step 317 is the same as the processing from step 300 to step 317 shown in FIG.

  Next, the video relay device 3 distributes video data (data for one frame) of individual viewpoints to the video receiving device 4 (step 401). Note that, when video data of individual viewpoints is distributed using the unicast method, video data of individual viewpoints is distributed to each video receiving device 4. On the other hand, when video data of individual viewpoints is distributed using the multicast method, multicast distribution is performed to the video receiving device 4.

  Next, the video relay device 3 determines whether or not the video data (data for one frame) of each viewpoint has been distributed to all the video reception devices 4 (step 402).

  When the video data of each viewpoint (data for one frame) is distributed to all the video receivers 4 in the process of step 402, the process proceeds to step 403. On the other hand, when the video data of each viewpoint (data for one frame) is not distributed to all the video receivers 4 in the process of step 402, the process returns to step 310.

  Next, the video relay apparatus 3 determines whether or not video data for 1 GOP has been distributed among the video data to be distributed (step 403).

  In the process of step 403, when video data for 1 GOP is distributed among the video data to be distributed, the process returns to step 308. On the other hand, if it is determined in step 403 that video data for 1 GOP is not distributed among the video data to be distributed, the process returns to step 309.

  FIG. 22 is an explanatory diagram of a display screen 56 displayed on the display unit 48 of the video reception device 4 according to the second embodiment of this invention.

  As shown in FIG. 22, the display screen 56 includes a user request input interface 55 for inputting the user's desire to view, a display screen 51 for playing back personalized video stream data received from the video relay device 3, and individual viewpoints. It includes display screens 52 to 54 for playing back video.

  As described above, according to the second embodiment of the present invention, it is possible to reproduce not only a personalized video without disturbance but also individual video streams simultaneously.

<Third Embodiment>
FIG. 23 is a system configuration diagram showing the configuration of the video distribution system according to the third embodiment of the present invention.

  The video distribution system according to the third embodiment includes a video distribution device 1, a metadata distribution device 2, and a video reception device 4. The network C7 connects the video distribution device 1, the metadata distribution device 2, and the video reception device 4.

  The video distribution device 1 receives a video distribution request from the video reception device 4 and distributes a multi-viewpoint video stream to the video reception device 4. The metadata distribution device 2 receives a metadata request from the video reception device 4 and distributes metadata indicating the content of the video stream to be distributed to the video reception device 4.

  The video reception device 4 transmits a video distribution request to the video distribution device 1 and receives the video data transmitted from the video distribution device 1. Further, the video reception device 4 transmits a metadata request to the metadata distribution device 2 and receives the metadata transmitted from the metadata distribution device 2. The video receiving device 4 receives a request from the user, selects video data that matches the received metadata and the request received from the user, and outputs the selected video data. Then, the output video data is reproduced by the display unit 48.

  FIG. 24 is a flowchart of video reception processing according to the third embodiment of this invention.

  The video reception device 4 reads out the program stored in the program storage unit 45 to the CPU 43, and starts the video reception process when the CPU 43 executes the read program (step 1330).

  First, the video reception device 4 transmits a metadata request to the metadata distribution device 2 (step 1303).

  Next, the video reception device 4 determines whether or not metadata has been received from the metadata distribution device 2 (step 1304).

  If the metadata is received from the metadata distribution device 2 in the process of step 1304, the process proceeds to step 1305. On the other hand, when the metadata is not received from the metadata delivery apparatus 2 in the process of step 1304, the process of step 1304 is repeated to determine whether the metadata is received from the metadata delivery apparatus 2.

  Next, the video reception device 4 registers the received metadata in the main memory 44 (step 1305).

  Next, the video reception device 4 transmits a video data distribution request to the video distribution device 1 (step 1306).

  Next, the video reception device 4 receives the video data transmitted from the video distribution device 1 and determines whether or not 1 GOP data of the video data of each viewpoint has been buffered (step 1307).

  If it is possible to buffer 1 GOP of video data of each viewpoint in the processing of step 1307, the processing proceeds to step 1331. On the other hand, if it is not possible to buffer 1 GOP of the video data of each viewpoint in the processing of step 1307, the processing of step 1307 is repeated to check whether 1 GOP of the video data of each viewpoint has been buffered. Determine.

  Next, the video reception device 4 performs setting for creating personalized video data from the received video data of a plurality of viewpoints (step 1331). Specifically, the video data of the viewpoint to be selected is set based on the metadata registered in the process of step 1305 and the user's request for viewing (request registered in the process of step 1313). If the user request is not registered, the video data of the initial viewpoint (for example, video data of the viewpoint with the smallest viewpoint number set for each viewpoint) is selected.

  Next, the video receiver 4 reproduces video data for one frame of the video data of the personalized video selected in step 1331 (step 1332).

  Next, the video reception device 4 determines whether or not video data has been received from the video distribution device 1 (step 1310).

  If the video data is received from the video distribution device 1 in the process of step 1310, the process proceeds to step 1311. On the other hand, if the video data is not received from the video distribution device 1 in the process of step 1310, the process proceeds to step 1333.

  Next, the video reception device 4 stores the received video data in the buffer of the main memory 44 (step 1311).

  Next, the video reception device 4 determines whether or not a user request has been input (step 1333).

  If a user request is input in the process of step 1333, the process proceeds to step 1313. On the other hand, if no user request is input in step 1333, the process proceeds to step 1316.

Next, the video receiving device 4 registers the user request input to the main memory 44 (step 1313).
Subsequently, the video reception device 4 next determines whether or not new metadata has been received from the metadata distribution device 2 (step 1316).

  If new metadata is received from the metadata distribution device 2 in the process of step 1316, the process proceeds to step 1317. On the other hand, if new metadata has not been received from the metadata distribution device 2 in the process of step 1316, the process proceeds to step 1334.

  Next, the video receiving device 4 registers the received metadata in the main memory 44 (step 1317).

  Next, the video receiver 4 determines whether or not video data for 1 GOP has been reproduced (step 1334).

  When the video data for 1 GOP is reproduced in the process of step 1334, the process proceeds to step 1335. On the other hand, if the video data for 1 GOP has not been reproduced in the process of step 1334, the process returns to step 1332.

  Next, the video receiving device 4 discards video data of the same GOP number as the GOP number of the video data of the viewpoint that has been reproduced out of the video data of the viewpoint that has not been reproduced (step 1335). Then, the process returns to step 1331 to continue the personalized video creation and playback process.

  As described above, according to the third embodiment of the present invention, the video reception device 4 can create personalized video data and reproduce the personalized video data.

<Fourth Embodiment>
FIG. 25 is a system configuration diagram showing the configuration of the video distribution system according to the fourth embodiment of the present invention.

  The operation of the video distribution system shown in FIG. 25 is the same as the operation of the video distribution system shown in FIG. However, as shown in FIG. 25, the video distribution device 1, the metadata distribution device 2, and the video relay device 3 are directly connected without a network, or the video distribution device 1, the metadata distribution device 2, and By installing the video relay device 3 at one point, it is possible to collectively manage video distribution equipment, and the maintenance load is reduced. For example, the video distribution device 1, the metadata distribution device 2, and the video relay device 3 may implement a system using one shared memory. Further, they may be connected by a cable such as a USB cable. Further, the video distribution device 1, the metadata distribution device 2, and the video relay device 3 may be configured by the same hardware and connected by a bus.

<Fifth Embodiment>
FIG. 26 is a system configuration diagram showing the configuration of the video distribution system according to the fifth embodiment of the present invention.

  The operation of the video distribution system shown in FIG. 26 is the same as the operation of the video distribution system shown in FIG. However, as shown in FIG. 26, by connecting the video relay device 3 and the video receiving device 4 without going through a network, the response to the request from the video receiving device 4 can be speeded up. The video relay device 3 and the video reception device 4 may be connected by the same connection method (for example, IEEE 1394). Further, the system may be configured such that the video relay device 3 is installed at the entrance of the home and the video receiving device 4 is installed in each room.

<Sixth Embodiment>
FIG. 27 is a block diagram illustrating a configuration of the video distribution device 8 according to the sixth embodiment of this invention. A video distribution apparatus 8 according to the sixth embodiment is a modification of the video distribution apparatus 1 according to the first embodiment. The video distribution device 8 includes a storage 88, a CPU 83, a main memory 84, a program storage unit 85, and a transmission unit 86. The storage 88, the CPU 83, the main memory 84, the program storage unit 85, and the transmission unit 86 are connected by a bus 89, respectively.

  The storage 88 stores a plurality of captured video data. The CPU 83 executes an OS (Operating System) and various application programs. The main memory 84 temporarily stores data necessary for the CPU 83 to execute various application programs. In the main memory 84, at least a part of the program stored in the program storage unit 25 is copied as necessary. The program storage unit 85 stores various application programs. The transmission unit 86 is an interface for distributing the encoded video data over the network.

  Next, the operation of the video distribution device 8 will be described.

  FIG. 28 is a flowchart of video distribution processing according to the sixth embodiment of this invention.

  The video distribution device 8 reads out the program stored in the program storage unit 85 to the CPU 83, and starts the video distribution process by the CPU 83 executing the read program (step 600).

  First, the video distribution device 8 performs 0 value initial setting of the GOP number in order to distribute video data (step 601).

  Next, the video distribution device 8 determines whether a video distribution request is received from the video relay device 3 (step 602).

  If a video distribution request is received from the video relay device 3 in the process of step 602, the process proceeds to step 603. On the other hand, if the video distribution request is not received from the video relay apparatus 3 in the process of step 602, the process of step 602 is repeated to determine whether the video distribution request from the video relay apparatus 3 has been received. .

  Next, the video distribution device 8 reads video data for one frame of the video data of each viewpoint from the video data stored in the storage 88 (step 603).

  Next, the video distribution device 8 encodes video data of a plurality of viewpoints in synchronization (step 604). Specifically, the video distribution device 8 compresses and encodes the video data of each viewpoint by aligning the GOP numbers given to the video data of each viewpoint. If the video data stored in the storage 88 and the video data distributed from the video distribution device 8 are in the same format, the process proceeds to step 605 without executing the process of step 604.

  For example, MPEG2 format video data is stored in the storage 88, and when video data is distributed in the MPEG format, there is no need to re-encode. move on. In the case of video data in which a frame is predicted across a plurality of GOPs, it is necessary to re-encode in step 604 so that the frame is predicted in units of GOPs so that the frame can be applied to the present invention.

  Next, the video distribution device 8 determines whether or not the video data encoded in step 604 includes the top data of the GOP (step 605).

  If it is determined in step 605 that the encoded video data includes the GOP head data, the process proceeds to step 606. On the other hand, if the encoded video data does not include the GOP head data in the process of step 605, the process proceeds to step 607.

  In step 606, the video distribution device 8 adds the GOP head identifier to the head RTP packet that transmits the video data, and distributes the video data as an RTP packet (step 606). Specifically, the video distribution device 8 increments the GOP number by 1 and adds it to the RTP packet.

  In step 607, the video distribution device 8 assigns the head identifier of the video frame to the head RTP packet that transmits the video data, and distributes the video data as an RTP packet (step 607). Specifically, the video distribution device 8 assigns the GOP number assigned to the top data of the same GOP to the RTP packet.

<Seventh Embodiment>
In the first embodiment of the present invention, video data personalized by the video relay device 3 is distributed to the video receiving device 4 based on user request data. In the seventh embodiment of the present invention, the video relay device 3 creates a video stream that can be selected in advance, and transmits a keyword for selecting the created video stream to the video reception device 4. The user selects a requested keyword from the transmitted keywords, and transmits the selected keyword to the video relay device 3.

  For example, in the case of distributing baseball broadcast video data using the system of the seventh embodiment, the video relay device 3 creates a video stream for a user who prefers a first-come team in advance and a second-team A video stream is created for a user who likes the video, and a video service is provided to allow the user to select which of the first team and the second team is preferred.

  FIG. 29 is a sequence diagram of video stream data transmission / reception processing of the video distribution system according to the seventh embodiment of this invention.

  In the process shown in FIG. 29, the same processes as those in FIG. 2 are assigned the same numbers. In the following, a description will be given focusing on differences from the processing shown in FIG.

  The processing from step 501 to step 507 is the same as the processing from step 501 to step 507 shown in FIG.

  Next, the video relay device 3 transmits a keyword for selecting a video stream created in advance to the video receiving device 4 (step 650).

  Next, the video receiver 4 registers the received keyword so that it can be selected from the user request interface shown in FIG. 16, for example (step 651).

  The processing from step 508 to step 510 is the same as the processing from step 508 to step 510 shown in FIG.

  Next, the video relay device 3 uses a preset initial setting (for example, “neutral video”: a video created by switching the video of the first and second teams to the same extent in the case of a baseball broadcast). ), The initial setting selection video data is distributed to the video receiver 4 (step 652).

  The process of step 512 is the same as the process of step 512 shown in FIG.

  Next, the video receiver 4 detects the keyword selected by the user (step 653).

  Next, the video reception device 4 transmits the keyword detected in step 653 to the video relay device 3 (step 654).

  Next, the video relay device 3 receives the keyword transmitted from the video receiving device 4 in step 654, and registers the received keyword (step 655).

  The process of step 516 is the same as the process of step 516 shown in FIG.

  Next, the video relay device 3 selects a video stream created corresponding to the keyword based on the keyword registered in step 655 (for example, “first team”), and the selected video stream is received by the video receiving device 4. (Step 656).

  The process of step 518 is the same as the process of step 518 shown in FIG.

  FIG. 30 is a flowchart of video relay processing according to the seventh embodiment of this invention.

  In the process shown in FIG. 30, the same number is assigned to the same process as the process of FIG. In the following, a description will be given focusing on differences from the processing shown in FIG.

  The processing from step 300 to step 305 is the same as the processing from step 300 to step shown in FIG.

  Next, the video relay device 3 notifies the video reception device 4 of keywords corresponding to video streams that can be selected in advance (step 701).

  Steps 306 and 307 are the same as steps 306 and 307 shown in FIG.

  Next, the video relay apparatus 3 creates a pre-selectable video stream from the received video data of a plurality of viewpoints (for example, “neutral video”, “video for the first-team candidate”, and “desired team hope” A video such as “personal video” is created), and a terminal (video receiving terminal 4) that receives a selectable video is specified (step 702).

  Next, the video relay apparatus 3 distributes video data for one frame of video data corresponding to the keyword to the distribution destination set in step 702 based on the keyword selected by the user (step 703).

  The processing of step 310 and step 311 is the same as the processing of step 310 and step 311 shown in FIG.

  Next, the video relay device 3 determines whether or not the keyword selected by the user has been received from the video receiving device 4 (step 704).

  If the keyword selected by the user is received from the video reception device 4 in the process of step 704, the process proceeds to step 705. On the other hand, if the keyword selected by the user is not received from the video receiving device 4 in the process of step 704, the process proceeds to step 316.

  Next, the video relay apparatus 3 associates the received keyword with the terminal (video reception apparatus 4) that has requested distribution of the video and registers it in the main memory 34 (step 705).

  The processing from step 316 to step 319 is the same as the processing from step 316 to step 319 shown in FIG.

  FIG. 31 is a flowchart of video reception processing according to the seventh embodiment of this invention.

  The video receiver 4 reads out the program stored in the program storage unit 45 to the CPU 43, and starts the video reception process when the CPU 43 executes the read program (step 750).

  First, the video reception device 4 transmits a request for participation in the video transmission service to the video relay device 3 in order to receive video data (step 751).

  Next, it is determined whether or not selected video data has been received from the video relay device 3 (step 752).

  If the selected video data is received from the video relay device 3 in the process of step 752, the process proceeds to step 753. On the other hand, if the selected video data is not received from the video relay device 3 in the process of step 752, the process proceeds to step 754.

  Next, the video receiver 4 reproduces the received video data (step 753).

  Next, the video reception device 4 determines whether or not a keyword for selecting video data has been received from the video relay device 3 (step 754).

  If the keyword is received from the video relay device 3 in step 754, the process proceeds to step 755. On the other hand, if the keyword is not received from the video relay device 3 in the process of step 754, the process proceeds to step 756.

  Next, the video receiver 4 registers the received keyword in the main memory 44 (step 755).

  Next, the video reception device 4 determines whether or not a keyword desired by the user is selected from the input interface 41 (step 756). That is, it is determined whether or not an image desired by the user (an image for a first-aid team applicant) has been input.

  If the keyword desired by the user is selected from the input interface 41 in the process of step 756, the process proceeds to step 757. On the other hand, if the keyword desired by the user is not selected from the input interface 41, the process returns to step 752 to continue the video reception process.

  Next, the video reception device 4 transmits the selected keyword to the video relay device 3 (step 757).

<Eighth Embodiment>
FIG. 32 is a system configuration diagram showing the configuration of the video distribution system according to the eighth embodiment of the present invention.

  The video distribution system of the eighth embodiment includes a plurality of video distribution apparatuses 1 and metadata distribution apparatuses 2 (video distribution apparatus 1A and video distribution apparatus 1B) of the video distribution system of the first embodiment shown in FIG. , The metadata distribution device 2A and the metadata distribution device 2B). Hereinafter, the video distribution apparatuses 1A and 1B may be collectively referred to as the video distribution apparatus 1. Further, when the metadata distribution devices 2A and 2B are collectively referred to, the metadata distribution device 2 may be described.

  The video distribution system according to the first embodiment can be used for relaying competitions held at one place such as baseball and soccer. On the other hand, the video distribution system according to the eighth embodiment is composed of a plurality of video distribution apparatuses 1 and metadata distribution apparatuses 2, and therefore can be used for a large-scale video distribution system extending over a plurality of points. For example, by installing a plurality of video distribution apparatuses 1 and metadata distribution apparatuses 2 at each business office such as a large-scale company having a plurality of business offices, the status of each business office can be changed to a remote video receiver 4. Can be confirmed. In addition, a system for confirming the image of an employee whose situation is to be confirmed can be provided by distributing the image of the meeting of a specific office using metadata.

  The following are typical examples of aspects of the present invention other than those described in the claims.

  (1) A video comprising: a video distribution device that transmits a video stream; and a video relay device that accumulates the video stream transmitted from the video distribution device in a buffer and transfers the selected video stream to a video reception terminal. In the distribution system, the video distribution device transmits the video stream by adding delimiter information indicating a reproducible unit of the video stream, and the video relay device transmits the transferred video stream to the delimiter information. A video distribution system characterized by switching only at a location.

  (2) The distribution system further includes a metadata distribution device that transmits information about the video stream transmitted from the video distribution device, and the metadata distribution device includes information on a subject included in the video stream, and The video relay device transmits metadata including the location information of the video stream corresponding to the subject information, and the video relay device transmits the metadata transmitted from the metadata distribution device and the subject included in the video stream. Including the information, receiving the video transmission request transmitted from the video receiving device, and referring to the received metadata to determine whether or not a video stream matching the received video transmission request is distributed. If a video stream that matches the received video transmission request is distributed, the first video stream currently being transferred is sent. The video distribution system according to (1), wherein the video is transferred to a video specified by the division information, and the second video stream of the switching destination is transferred from the video specified by the division information. .

  (3) When the video relay apparatus stores the data of the second video stream in the buffer and receives the delimiter information of the second video stream, the second video stream stored in the buffer The video distribution system according to (2), wherein the data is deleted.

  (4) The video stream includes at least a first frame that can be reproduced independently without using prediction between frames, and a second frame that can be reproduced only by prediction in the forward direction from other frames. The video relay device transfers the first video stream immediately before the first frame or immediately before the second frame, and transfers the second video stream from the first frame. The video distribution system according to (2), which is characterized.

  (5) The video distribution system according to (2), wherein the information on the location of the video stream corresponding to the subject information includes an RTP sequence number of the video at which the subject starts to appear.

  (6) The video distribution system according to (1), wherein the video distribution device transmits the video stream by an RTP packet, and the delimiter information is included in a header portion of the RTP packet.

  (7) The video distribution apparatus adds the delimiter information to a location where a frame that can be generated independently without using inter-frame prediction is detected in a video stream to be distributed ( The video distribution system according to 1).

  (8) A video relay apparatus that accumulates a video stream transmitted from a video distribution apparatus in a buffer and transfers the selected video stream to a video reception terminal, wherein the video distribution apparatus can reproduce the video stream The video stream is transmitted by adding delimiter information indicating a unit, and the video relay apparatus includes information on a subject included in the video stream and information on a location of the video stream corresponding to the subject information. A video transmission request transmitted from the video reception device, which is connected to a metadata distribution device for transmitting metadata and includes metadata transmitted from the metadata distribution device and information on a subject included in the video stream. The video stream matching the received video transmission request is distributed with reference to the received metadata. If the video stream that matches the received video transmission request is distributed, the first video stream currently being transferred is transferred to the video specified by the delimiter information, The video relay device, wherein the second video stream of the switching destination is transferred from the video specified by the delimiter information.

  (9) storing the data of the second video stream in the buffer, and erasing the data of the second video stream stored in the buffer when the delimiter information of the second video stream is received. (8) The video relay device according to (8).

  (10) The video stream includes at least a first frame that can be played independently without using prediction between frames, and a second frame that can be played back only by forward prediction from other frames. The video relay device transfers the first video stream immediately before the first frame or immediately before the second frame, and transfers the second video stream from the first frame. The video relay device according to (8), which is characterized.

  (11) The video relay device according to (8), wherein the video stream location information corresponding to the subject information includes an RTP sequence number of the video at which the subject starts to appear.

  (12) The video relay device according to (8), wherein the video stream transmitted by the RTP packet is received from the video distribution device, and the delimiter information is extracted from a header portion of the RTP packet.

  (13) A video distribution device for transmitting a video stream, a video relay device for storing the video stream transmitted from the video distribution device in a buffer, and transferring the selected video stream to a video receiving terminal, and the video distribution And a metadata distribution device that transmits information about the video stream transmitted from the device, wherein the video distribution device includes delimiter information indicating a reproducible unit of the video stream. In addition, the video stream is transmitted, and the metadata distribution device transmits metadata including information on a subject included in the video stream and information on a location of the video stream corresponding to the information on the subject, The video relay device includes metadata transmitted from the metadata distribution device, and the video stream. Receiving the video transmission request transmitted from the video receiving device, referring to the received metadata, and delivering a video stream that matches the received video transmission request. If the video stream that matches the received video transmission request is distributed, the first video stream currently being transferred is transferred to the video specified by the delimiter information, A video relay method, wherein the second video stream of the switching destination is transferred from the video specified by the delimiter information.

  (14) When the video relay device stores the data of the second video stream in the buffer and receives the delimiter information of the second video stream, the second video stream stored in the buffer The video relay method according to (13), wherein the data is deleted.

  (15) The video stream includes at least a first frame that can be reproduced independently without using prediction between frames, and a second frame that can be reproduced only by forward prediction from other frames. The video relay device transfers the first video stream immediately before the first frame or immediately before the second frame, and transfers the second video stream from the first frame. The video relay method according to (13), which is characterized.

  (16) The video relay method according to (13), wherein the video stream location information corresponding to the subject information includes an RTP sequence number of the video at which the subject starts to appear.

  (17) The video according to (13), wherein the video distribution device transmits the video stream by an RTP packet, and the video relay device extracts the delimiter information from a header portion of the RTP packet. Relay method.

1 is a system configuration diagram illustrating a configuration of a video distribution system according to a first embodiment of this invention. It is a sequence diagram of the video stream data transmission / reception process of the video delivery system of the 1st Embodiment of this invention. It is a block diagram which shows the structure of the metadata delivery apparatus of the 1st Embodiment of this invention. It is a flowchart of the metadata delivery process of the 1st Embodiment of this invention. It is explanatory drawing which shows the structure of the transmission packet of the metadata transmitted to a video relay apparatus from a metadata delivery apparatus. It is a block diagram which shows the structure of the video delivery apparatus of the 1st Embodiment of this invention. It is a flowchart of the video delivery process of the 1st Embodiment of this invention. It is explanatory drawing which shows the structure of the RTP header of the 1st Embodiment of this invention. It is explanatory drawing which shows the relationship between the P bit and M bit in the RTP header of the 1st Embodiment of this invention, the GOP, and the head of video frame data. It is explanatory drawing which shows the structure of the extension header of the RTP packet of the 1st Embodiment of this invention. It is explanatory drawing which shows the structure of the packet into which the MPEG2-TS packet of the 1st Embodiment of this invention was RTP packetization, UDP packetization, and IP packetization. It is a block diagram which shows the structure of the video relay apparatus of the 1st Embodiment of this invention. It is a flowchart of the video relay process of the 1st Embodiment of this invention. It is a block diagram which shows the structure of the video receiver of the 1st Embodiment of this invention. It is a flowchart of the video reception process of the 1st Embodiment of this invention. It is explanatory drawing of the display screen displayed on the display part of the video receiver of the 1st Embodiment of this invention. It is a system block diagram which shows the structure of the video delivery system of the 2nd Embodiment of this invention. It is a video stream data transmission / reception process sequence diagram of the video delivery system of the 2nd Embodiment of this invention. It is a flowchart of the video relay process of the 2nd Embodiment of this invention. It is explanatory drawing which shows the relationship between the transmission order of the video frame in transmission, and the reproduction | regeneration order of the video frame in reproduction | regeneration. It is explanatory drawing which shows the reproduction | regeneration order of GOP video data of the 1st Embodiment of this invention. It is explanatory drawing of the display screen displayed on the display part of the video receiver of the 2nd Embodiment of this invention. It is a system block diagram which shows the structure of the video delivery system of the 3rd Embodiment of this invention. It is a flowchart of the image | video reception process of the 3rd Embodiment of this invention. It is a system block diagram which shows the structure of the video delivery system of the 4th Embodiment of this invention. It is a system block diagram which shows the structure of the video delivery system of the 5th Embodiment of this invention. It is a block diagram which shows the structure of the video delivery apparatus of the 6th Embodiment of this invention. It is a flowchart of the video delivery process of the 6th Embodiment of this invention. It is a video stream data transmission / reception process sequence diagram of the video delivery system of the 7th Embodiment of this invention. It is a flowchart of the video relay process of the 7th Embodiment of this invention. It is a flowchart of the image | video reception process of the 7th Embodiment of this invention. It is a system block diagram which shows the structure of the video delivery system of the 8th Embodiment of this invention.

Explanation of symbols

1, 8 Video distribution device 2 Metadata distribution device 3 Video relay device 4 Video reception device 5, 6, 7 Network 11 Video camera 12 Encoder 13, 23, 33, 43, 83 CPU
14, 24, 34, 44, 84 Main memory 15, 25, 35, 45, 85 Program storage unit 16, 26, 36, 46, 86 Transmission unit 19, 29, 39, 49, 89 Bus 21, 41 Input interface 32 42 Receiver 48 Display 88 Storage

Claims (20)

A video distribution device for transmitting a video stream;
A video distribution system comprising: a video relay device that receives the plurality of transmitted video streams and transfers at least one of the received video streams to a video reception device;
The video distribution device includes:
Adding a first identifier to the video stream for identifying whether or not the unit of data that can be independently decoded included in the video stream corresponds in time,
Transmitting a video stream to which the first identifier is added;
The video relay device is:
When switching the video stream in units that can be independently decoded,
Based on the first identifier added to the received video stream, identify the data of the decodable unit of the video stream transferred after switching,
A video distribution system, wherein the video stream transferred after the switching is transferred from the specified unit of decodable data. The video relay device is:
Of the received video stream, store at least one unit of video stream that can be independently decoded in a buffer;
Receiving metadata indicating the content of the received video stream;
Receiving request information of the video stream from the video receiver;
The video distribution system according to claim 1, wherein a video stream to be distributed is selected based on the received metadata and request information.   When transferring the selected video stream to the video receiving device, the video relay device identifies a video stream that has not been selected and identifies the same time as the transferred unit data that can be decoded. 3. The video distribution system according to claim 2, wherein the data in a unit that can be decoded with the identifier of 1 is deleted from the buffer in a unit that can be decoded independently.   The video distribution system according to claim 2, wherein the video relay apparatus transfers an unselected video stream to the video reception apparatus.   The video distribution system according to claim 1, wherein the unit of the video stream that can be decoded independently is GOP data.   2. The video distribution system according to claim 1, wherein the video distribution apparatus adds the first identifier to a header of an RTP packet of a video stream divided into units that can be independently decoded. .   The video distribution system according to claim 1, wherein the video distribution system includes the video relay device and the video reception device in a single casing. The video distribution device includes:
Store the video stream,
Read the stored video stream;
Adding a first identifier to the read video stream for identifying whether or not the unit data that can be independently decoded included in the video stream corresponds to the time.
The video distribution system according to claim 1, wherein the video stream to which the first identifier is added is transmitted. The video relay device is:
Creating a plurality of video streams to be distributed to the video receiver;
Notifying the video receiver of a second identifier for selecting one video stream from the created video streams,
The video distribution system according to claim 1, wherein the video stream specified by the second identifier selected by the video receiving device is selected. The video receiver is
Receiving a second identifier for selecting the video stream from the video relay device;
Transmitting a second identifier selected from the received second identifier to the video relay device;
Receiving a video stream selected by the video relay device;
The video distribution system according to claim 9, wherein the received video stream is reproduced. A video relay device that receives the plurality of transmitted video streams and transfers at least one of the received video streams to a video receiving device;
The video distribution device includes:
Adding a first identifier to the video stream for identifying whether or not the unit of data that can be independently decoded included in the video stream corresponds in time,
Transmitting a video stream to which the first identifier is added;
The video relay device is:
When switching the video stream in units that can be independently decoded,
Based on the first identifier added to the received video stream, identify the data of the decodable unit of the video stream transferred after switching,
The video relay apparatus, wherein the video stream transferred after the switching is transferred from the data of the specified decodable unit. The video relay device is:
Of the received video stream, store at least one unit of video stream that can be independently decoded in a buffer;
Receiving metadata indicating the content of the received video stream;
Receiving request information of the video stream from the video receiver;
12. The video relay apparatus according to claim 11, wherein a video stream to be distributed is selected based on the received metadata and request information.   When transferring the selected video stream to the video receiving device, the video relay device identifies a video stream that has not been selected and identifies the same time as the transferred unit data that can be decoded. 13. The video relay apparatus according to claim 12, wherein the data of a decodable unit to which the identifier of 1 is added is deleted from the buffer in a unit that can be decoded independently.   13. The video relay apparatus according to claim 12, wherein the video relay apparatus transfers an unselected video stream to the video reception apparatus. The video relay device is:
Creating a plurality of video streams to be distributed to the video receiver;
Notifying the video receiver of a second identifier for selecting one video stream from the created video streams,
12. The video relay apparatus according to claim 11, wherein the video stream specified by the second identifier selected by the video receiving apparatus is selected. A video distribution device for transmitting a video stream;
A video relay method in a video distribution system comprising: a video relay device that receives the plurality of transmitted video streams and transfers at least one of the received video streams to a video receiver device,
The video relay method is:
Adding a first identifier to the video stream for identifying whether or not the unit of data that can be independently decoded included in the video stream corresponds in time,
Transmitting a video stream to which the first identifier is added;
When switching the video stream in units that can be independently decoded,
Based on the first identifier added to the received video stream, identify the data of the decodable unit of the video stream transferred after switching,
A video relay method, wherein the video stream transferred after the switching is transferred from the specified unit of data that can be decoded. The video relay method is:
Of the received video stream, store at least one unit of video stream that can be independently decoded in a buffer;
Receiving metadata indicating the content of the received video stream;
Receiving request information of the video stream from the video receiver;
The video relay method according to claim 16, wherein a video stream to be distributed is selected based on the received metadata and request information.   When transferring the selected video stream to the video receiving device, the video relay device identifies a video stream that has not been selected and identifies the same time as the transferred unit data that can be decoded. 18. The video relay method according to claim 17, wherein the decodable unit data to which the identifier of 1 is added is deleted from the buffer in an independently decodable unit.   The video relay method according to claim 17, wherein the video relay apparatus transfers the unselected video stream to the video reception apparatus. The video relay method is:
Creating a plurality of video streams to be distributed to the video receiver;
Notifying the video receiver of a second identifier for selecting one video stream from the created video streams,
The video relay method according to claim 16, wherein the video stream specified by the second identifier selected by the video receiving device is selected.

Images