JP2004056819A - Video data transmitting apparatus, video data transmitting-receiving system, and method thereof - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a transmitting-receiving system in which video data are correctly decoded and displayed on a screen on a receiving side even from a portion just after transmission restart (in the middle of GOP) when interrupting and then restarting transmission-reception of compression-coded video data for any reason. <P>SOLUTION: When on-demand video data are transmitted in the manner of interrupt while broadcast video data are being transmitted from a distribution server 17 to a user terminal 18, before restarting the transmission of the broadcast video data after the transmission of the on-demand video data is completed, the distribution server 17 transmits I frame data for interpolation to the user terminal 18. The user terminal 18 first decodes and displays the I frame for interpolation, and in the broadcast video data, several frames just after transmission restart are decoded and displayed by using the I frame for interpolation as a reference frame. <P>COPYRIGHT: (C)2004,JPO

Description

The present invention is a video data transmission / reception system including a transmission side and a reception side, wherein the transmission side transmits video data compressed and encoded using motion compensation inter-frame prediction to the reception side, and the reception side The present invention relates to a video data transmission / reception system that decodes the video data and displays it on a screen.

In recent years, with the spread of broadband networks such as ADSL and FTTH, services for providing video data to user terminals via the Internet have been spreading. The provided video data is compression-encoded in order to reduce the amount of data. As a compression-encoding method, a method defined by a standard such as MPEG-4 is generally used. In this method, the data amount is compressed by performing coding by combining intra-frame coding / inter-frame coding and motion compensation prediction.

As one form of such a video data providing service, there is a broadcast type service (a transmission side device that distributes the same video data (broadcast video data) to a plurality of user terminals in a same time zone according to a predetermined time table). . Data distribution in a broadcast-type service is performed by broadcast or multicast as referred to in one-to-many communication. In this method, since there is no need to perform individual processing for each user, it is possible to increase the number of destination user terminals without increasing the load on the transmitting device.

However, as one of the service diversifications, not only broadcast video data is unilaterally distributed from a transmission side device to a user terminal, but another video data is distributed according to a unique request from each user terminal. Services (on-demand data distribution) are also provided.

For example, a service is conceivable in which, while a sports program is being transmitted as broadcast video data, past data on a specific player is appropriately transmitted in response to a request from a user terminal.

As described above, when transmitting on-demand video data to a specific user terminal during transmission of broadcast video data, there are several methods for handling the broadcast video data.

One is that the broadcast video data is continuously transmitted to the user terminal while the on-demand video data is being transmitted. In this method, the user terminal needs a communication band capable of receiving two types of video data at the same time, and additional processing such as selective decoding and reproduction of the received video data occurs. The load becomes heavy. Therefore, a method is often adopted in which transmission of broadcast video data to a user terminal is interrupted during transmission of on-demand video data, and only on-demand video data is transmitted.
JP-A-09-018881

However, for broadcast video data whose transmission has been temporarily interrupted due to the above-mentioned on-demand video data transmission or the like, the following problems may occur immediately after transmission is resumed. That is, the user terminal cannot decode the broadcast video data immediately after the restart, and a time period occurs in which the video is not displayed on the screen. This is due to the video data compression coding method (motion-compensated inter-frame prediction). Hereinafter, how such a problem occurs will be described with reference to the drawings.

FIG. 12 is a schematic diagram showing how the video data is received by the user terminal when on-demand video data is transmitted after the broadcast video data transmission by the Internet broadcast is interrupted. Here, reception of broadcast video data 111 is interrupted at time t0, and on-demand video data 112 is received instead until time t1. Then, reception of the broadcast video data is restarted from time t1. The broadcast video data 111 is obtained by compressing and encoding a video of 30 frames / second by providing one I frame (intra-frame coded frame) in 5 seconds. Therefore, one unit (GOP: Group @ Of \ Picture) is composed of 150 frames around one I frame. In Internet broadcasting, the ratio of I frames is smaller than that of digital broadcasting (two I frames per second at 30 frames / second).

GOP 1110 in FIG. 12 is a GOP of broadcast video data 111 that is received across time t1. The GOP 1110 is composed of one I frame and a plurality of P and B frames, like other GOPs. Each P frame is inter-frame coded using the immediately preceding I or P frame as a reference frame, and each B frame using the preceding or succeeding I or P frame as a reference frame.

The first three frames including the I frame in the GOP 1110 are not received by the user terminal because the transmission timing is overlapped with the on-demand video data. Therefore, the fourth P frame encoded using the first I frame as a reference frame cannot be decoded. Further, the P and B frames that are referred to after the P frame of the fourth frame cannot be decoded. As a result, the video of the portion corresponding to the P and B frames after the fourth frame is not correctly displayed on the screen. This state lasts for a maximum of 149 frames (slightly less than 5 seconds) in the case of broadcast video data encoded by a method of providing one I frame every 5 seconds at 30 frames / sec (2 per second). In the case of video data encoded with one I-frame, this is only 14 frames (less than 0.5 seconds).

To prevent this, (1) broadcast video data is continuously transmitted during on-demand video data transmission, and the user terminal decodes both video data in parallel, or (2) the transmitting device performs: The transmission timing of the broadcast video data is shifted only to the user terminal concerned, or the GOP is rearranged and re-encoded in such a manner that the frame immediately after the restart is changed to the I frame, so that the transmission of the I frame data is always performed when the transmission is restarted. Although it is conceivable to receive the video data in (1), the bandwidth is required and the processing load on the user terminal is doubled. In (2), the transmitting device receives the on-demand video data. It becomes necessary to distinguish the processed user terminal from the other user terminals and perform another processing, and the load on the transmitting apparatus becomes heavy. This increase in load becomes heavier as the number of user terminals that simultaneously receive on-demand video data increases, which may lead to a situation in which the number of users who can simultaneously broadcast is suppressed.

The present invention has been made in view of the above-described problems, and when a situation arises in which video data encoded using motion compensation prediction must be decoded in the middle of a GOP due to the above-described causes, the transmission side or the reception side A video data transmission / reception system that allows a receiving user terminal to correctly decode and display broadcast video data even from a portion immediately after transmission restart (in the middle of a GOP) without significantly increasing the processing load. The purpose is to provide.

In order to achieve the above object, a video data transmission / reception system according to the present invention includes a transmission device that transmits video data compressed and encoded using motion compensation inter-frame prediction, and a plurality of reception devices that receive and decode the video data. A video data transmission / reception system including a terminal, wherein the transmission-side device generates the video data by performing an intra-frame encoding process or an inter-frame encoding process on each of a plurality of frames constituting moving image data. (1) encoding means, and second encoding means for performing an intra-frame encoding process on frames constituting the moving image data to generate interpolation I-frame data in parallel with the encoding process by the first encoding device; Transmitting means for transmitting the video data to the plurality of receiving terminals, the transmitting means, the transmitting means for any of the plurality of receiving terminals When the transmission of the image data is interrupted and then resumed, the interpolation I-frame data for at least one frame is transmitted to the receiving terminal before the transmission of the video data is resumed, and the receiving terminal transmits the image data. When the transmission of the video data once interrupted by the means is resumed, first, the interpolation I frame data transmitted from the transmitting means is received and decoded, and the decoded result data is used as reference frame data to resume. Decoding the video data later.

In such a video data transmission / reception system, when the transmission of video data whose transmission has been suspended is restarted, if the frame data immediately after the restart is that of an inter-coded frame (P frame or B frame), the frame Instead of the data, I-frame data for interpolation, which is generated by separately intra-coding the same frame, is transmitted to the user terminal. Then, the user terminal first decodes the interpolation I-frame data, and decodes the subsequent frame data by referring to the decoding result of the interpolation I-frame data as reference frame data. Therefore, decoding can be performed correctly immediately after restart (in the middle of a GOP), and video data display is not interrupted. Further, the processing load on the user terminal device does not become heavy, and the transmission side device only generates a load for generating the interpolation I frame data and storing it for several frames. Even in a system in which video data is frequently interrupted and restarted, the processing load does not lead to a situation in which the number of users who can broadcast simultaneously is suppressed.

Note that the above effects can also be realized by the transmitting device in such a system, that is, the video data transmitting device alone. Further, the above effects can also be realized by a video data transmission method executed in the system having the above configuration, or a program for causing a computer to execute an operation as an apparatus having such a configuration.

As is apparent from the above description, the video data transmission / reception system of the present invention includes a transmission device that transmits video data compressed and encoded using motion compensation inter-frame prediction, and a plurality of reception devices that receive and decode the video data. This is a video data transmission / reception system including a terminal. A first encoding unit configured to generate the video data by performing an intra-frame encoding process or an inter-frame encoding process on each of a plurality of frames included in the moving image data; and a first encoding unit. In parallel with the encoding process, a second encoding unit that performs intra-frame encoding of the frames constituting the moving image data to generate I-frame data for interpolation, and transmits the video data to the plurality of receiving terminals. Transmitting means for transmitting the video data to any one of the plurality of receiving terminals, after interrupting the video data once and then resuming the transmission of the video data. At least one frame data is transmitted to the receiving terminal, and the receiving terminal transmits the video data temporarily interrupted by the transmitting unit. When the transmission of the data is resumed, first, the interpolation I-frame data transmitted from the transmitting means is received and decoded, and the decoded data is used as reference frame data to decode the resumed video data. It is characterized by the following.

In such a video data transmission / reception system, when the transmission of video data whose transmission has been suspended is restarted, if the frame data immediately after the restart is that of an inter-coded frame (P frame or B frame), the frame Instead of the data, I-frame data for interpolation, which is generated by separately intra-coding the same frame, is transmitted to the user terminal. Then, the user terminal first decodes the interpolation I-frame data, and decodes the subsequent frame data by referring to the decoding result of the interpolation I-frame data as reference frame data. Therefore, decoding can be performed correctly immediately after restart (in the middle of a GOP), and video data display is not interrupted. Further, the processing load on the user terminal device does not become heavy, and the transmission side device only generates a load for generating the interpolation I frame data and storing it for several frames. Even in a system in which video data transmission is interrupted and restarted frequently, the processing load does not reduce the number of users who can broadcast simultaneously.

Note that the above effects can also be realized by the transmitting device in such a system, that is, the video data transmitting device alone.

Such a transmission device further includes option data transmission means for transmitting optional video data to any of the plurality of receiving terminals in parallel with the transmission of the video data, and further comprising: The transmission of the video data to any one of the terminals may be interrupted by the optional data transmission by the optional data transmission means to the one of the receiving terminals being executed in an interrupted manner.

In the case of this configuration, a service is provided in which the same video data is simultaneously transmitted to a large number of receiving terminals in a broadcast image while another video data (optional video data) is transmitted to a specific receiving terminal in an interrupt manner. The above effect can also be realized in a video data transmission / reception system. Note that examples of the optional video data include on-demand video data and CM video data.

Further, the option data transmitting unit has an information collecting unit that collects information on user's preference in advance from at least a part of the plurality of receiving terminals,
The content of the optional video data to be transmitted may be selected based on the information collected by the information collecting means.

According to this configuration, the above effect can be realized even in a video data transmission / reception system that provides a service of transmitting optional video data led by a configuration on the side that provides video data.

Further, the transmission means has a broadcast transmission means for broadcasting the same data to a plurality of transmission destinations, and an individual transmission means for transmitting individual data to the individual transmission destinations. May be configured such that the broadcast transmitting unit is used, the individual transmitting unit is used for the interpolation I frame data, and the optional data transmitting unit individually transmits the optional video data. In this case, since the video data is broadcast, even if the number of receiving terminals increases, the processing load on the transmitting apparatus does not increase. Further, the transmitting means has a switching means for excluding the receiving terminal to which the interpolation I frame data or the optional video data is being transmitted from the transmission of the video data by the broadcast transmitting means, Then, since the receiving terminal does not receive the broadcast video data and the optional video data or the interpolation I-frame data at the same time, it is possible to suppress an increase in the network load and the processing load of the receiving terminal. Further, the option data transmission means includes an insertion means for transmitting the second option data in a form of being inserted during the transmission of the first option data, and a first option data after the transmission of the second option data is completed. Before the transmission of the first option data is resumed, for at least one frame from the transmission restart position of the first option data, a third encoding unit for generating corresponding I-frame data for option data interpolation, When the transmission of the first option data is resumed after the transmission of the second option data is completed, the option data transmitting means transmits the I / O for the option data interpolation before resuming the transmission of the first option data. If the frame data is transmitted to the receiving terminal, optional video data Not only the pattern to insert the data, there is also the pattern to be inserted another option video data in optional video data, displaying the first option image data will not be interrupted immediately after transmission resumption.

The first encoding means and the second encoding means may be realized by separate encoders.

According to this configuration, there is no need to provide a special encoder that performs the encoding process based on normal motion-compensated inter-frame prediction and the intra-frame encoding process for all frames in parallel. It is only necessary to provide any one of the above-described processes, and the above-described effect can be realized using a conventional encoder.

In order to determine the number of frames to be transmitted specifically for the interpolation I-frame in order to obtain the above-mentioned effect, the transmission unit determines whether the interpolation I-frame to be transmitted to the receiving terminal before resuming the transmission of the video data. The number of frames of data may be determined based on the GOP configuration of the video data, in particular, the frequency of appearance of I- or P-attribute frames.

A video data transmitting apparatus for transmitting video data compressed and coded using motion-compensated inter-frame prediction to a plurality of receiving terminals, wherein the video data transmitting apparatus performs intra-frame coding processing on a frame constituting moving image data. A first encoding unit that generates inner encoded video data, a second encoding unit that generates interframe encoded video data by performing an interframe encoding process on a frame that constitutes moving image data, Video data generating means for generating the video data from the intra-coded video data generated by the coding means and the inter-frame coded video data generated by the second coding means; Transmitting means for transmitting to the terminal, the transmitting means temporarily interrupts transmission of the video data to any of the plurality of receiving terminals. When restarting after transmission of the video data, at least one frame of the intra-frame encoded video data generated by the first encoding means is transmitted to the receiving terminal as I-frame data for interpolation before resuming the transmission of the video data. The same effect as described above can also be obtained by the video data transmission device characterized in that the video data transmission device is set in advance. In this case, the intra-frame encoded data generated by the first encoding unit is used as the interpolation I-frame data.

Also, the video data transmission / reception system of the present invention includes a plurality of video data providing apparatuses for transmitting video data compressed and encoded using motion compensated inter-frame prediction, and receiving video data from any of the plurality of video data providing apparatuses. A video data transmission / reception system comprising: a plurality of receiving terminals for decoding and decoding; and a distribution server for relaying video data between the plurality of video data providing devices and the plurality of receiving terminals. The video data providing apparatus includes: a first encoding unit configured to generate video data by performing an intra-frame encoding process or an inter-frame encoding process on each of a plurality of frames included in the moving image data; In parallel with the encoding process by the encoding unit, a second encoding unit that generates an interpolation I frame data by performing an intra-frame encoding process on each of the plurality of frames constituting the moving image data, The distribution server includes: a switching request receiving unit that receives a switching request for switching video data to be received from any of the plurality of receiving terminals; and a receiving unit that receives a switching request when the switching request receiving unit receives the switching request. The transmission of the video data before switching to the terminal is stopped, and then the I-frame for interpolation obtained from the video data providing device that is the source of the video data after switching is provided. The data after transmitted to the switch requesting receiving terminal, having a switching transmitting means for transmitting the video data after the switch to the switch requesting the receiving terminal can also be realized by a configuration, wherein.

According to such a video data transmission / reception system, in a video data transmission / reception system performing a service of transmitting a plurality of types of broadcast video data in a broadcast image in a selectable manner to a large number of reception terminals, the reception terminal side includes: Broadcast video data can be correctly decoded even immediately after switching of the broadcast video data (during the middle of a GOP), and the processing load on neither the transmitting device nor the receiving terminal is significantly increased.

An encoder for compressing and encoding moving image data using motion-compensated inter-frame prediction, wherein an encoded image is obtained by performing an intra-frame encoding process or an inter-frame encoding process on each of a plurality of frames constituting the moving image data. A first encoding unit for generating data; and an intra-frame encoding process for each of the plurality of frames constituting the moving image data to generate interpolation I frame data in parallel with the encoding process by the first encoding unit. According to the encoder, the encoded data of the I frame corresponding to the reference frame is not correctly transmitted to the device on the decoding side. By using the interpolation I-frame data instead of the I-frame data, the I-frame and the reference frame are used. Frame can be correctly decoded. If the encoder is applied to a system for transmitting and receiving video data in a broadcast image, the above effects can be realized.

Further, the above effect is an encoder that compresses and encodes moving image data using motion compensation inter-frame prediction, and performs intra-frame encoding processing on frames constituting the moving image data to generate intra-frame encoded video data. A first encoding unit that performs an inter-frame encoding process on a frame that constitutes moving image data to generate inter-frame encoded video data; and a frame that is generated by the first encoding unit. Coded video data generating means for generating coded video data from inner coded video data and inter-frame coded video data generated by the second coding means, and intra-frame code generated by the first coding means And an interpolation data generating means for generating interpolation I frame data from the coded video data.

The effects described above can also be realized by a video data transmission / reception method executed by the system having the above configuration, or a program that causes a computer to execute an operation as an apparatus having such a configuration.

Hereinafter, embodiments of the video data transmission / reception system of the present invention will be described with reference to the drawings.

(Overview)
FIG. 1 is a diagram showing a configuration of a video data transmission / reception system 1 according to the present embodiment. The video data transmission / reception system 1 transmits video data in a broadcast format from a video data transmission source via the Internet N to a user terminal 18 that wants to receive video data. The transmission in the broadcast format is to transmit the same video data simultaneously to all the user terminals that desire to receive the data according to a predetermined time table notified to the user. The video data transmitted in the broadcast format is hereinafter referred to as “broadcast video data”. That is, “broadcast” in the broadcast video data here does not mean that the broadcast video data is transmitted and received.

Further, some video data transmitted / received by the video data transmission / reception system 1 are transmitted only to the specific user terminal that issued the request when the request is issued, separately from the broadcast video data. Video data. " During transmission of on-demand video data, transmission of broadcast video data to the user terminal is interrupted (broadcast video data is continuously transmitted to other user terminals). Then, when the transmission and reception of the on-demand video data is completed, the transmission of the broadcast video data to the user terminal is restarted.

The video data transmitting / receiving system 1 enables the user terminal to correctly decode and reproduce the broadcast video data immediately after the restart when the transmission of the broadcast video data is restarted, as described above. The frame data of the B frame is characterized in that the same frame is replaced with I-frame data (interpolation I-frame) which has been separately intra-coded and transmitted to the user terminal.

Hereinafter, a specific example of transmission and reception of the video data including the I frame data for interpolation will be described with reference to the drawings.

In FIG. 2, the user terminal receives the on-demand video data 23 at a timing that extends over two GOPs 21 and 22 that constitute the broadcast video data 20. The last frame of the on-demand video data 23 overlaps the second frame 222 in the GOP 22. Frames 223 and 224 of the broadcast video data 20 at the position immediately after the restart are provided. The frame 223 is a B frame, and reference frames at the time of decoding are the first I frame 221 of the GOP 22 and the subsequent P frame 224. The frame 224 is a P frame, and the reference frame at the time of decoding is the I frame 221.

In order to decode the frames 223 and 224, it is necessary to refer to the I frame 221. If the frame 224 cannot be decoded correctly, the subsequent frame 225 and subsequent frames cannot be decoded, but the user terminal 18 receives the I frame 221. I haven't. Therefore, when restarting the transmission of the broadcast video data 20 to the user terminal 18, the material selection device 15 does not transmit the first two frames 223, 224 that cannot be correctly decoded, but instead transmits the corresponding interpolation I frame data 231, 232. Send

The user terminal 18 decodes the interpolation I frame data 231 and 232 corresponding to the frames 223 and 224, and displays them on the screen. Then, since the frame 225 and subsequent frames can be decoded with reference to the decoding result of the frame 232, it is not necessary to replace the frame with the interpolation I frame.
(Configuration of each part)
Next, the configuration of the video data transmission / reception system 1 that performs the above-described processing will be described.

As shown in FIG. 1, the video data transmission / reception system 1 has a configuration in which a plurality of user terminals 18, which are components on the reception side, and a configuration on the transmission side are connected via the Internet N. It has a video camera 11, an editing / broadcasting material server 12, a switcher 13, an encoder 14, a material selecting device 15, an on-demand material server 16, and a distribution server 17.

The video camera 11 provides relay video data, which is a type of video data. This is a device that converts an optical image incident on an optical lens into an electric signal. Video data is uncompressed or compressed in a format such as DVCPRO or consumer DV and transmitted through signal lines such as SDI, SDTI, and IEEE1394. Output to the switcher 13.

The editing / broadcasting material server 12 provides video data of a “program” edited by a broadcasting station as a kind of broadcast video data. Holding. These data are stored on a hard disk, a magnetic tape, or the like, and output to the switcher 13 as needed.

The switcher 13 is located in the broadcasting center and edits video data obtained from the video camera 11 and the editing / broadcasting material server 12 to be broadcast data. The switcher 13 broadcasts the broadcast data at a timing according to a predetermined time table. To the encoder 14.

The encoder 14 compresses and encodes the broadcast data output from the switcher 13 in a format (here, “MPEG-4”) suitable for transmission over the Internet to generate broadcast video data, and outputs the broadcast video data to the material selection device 15. . If the broadcast data has already been encoded at the time of output from the switcher 13 and cannot be input to the encoder 14 as it is, the broadcast data is once decoded by a decoder (not shown) and then input to the encoder 14. Further, the encoder 14 separately encodes each frame of the broadcast data within a frame, and outputs the result to the material selection device 15 as interpolation I frame data.

That is, the encoder 14 performs two different encoding processes on the input broadcast data, and outputs the results as broadcast video data and interpolation I-frame data.

FIG. 3 shows the configuration of the encoder 14. The difference from a general encoder is that it has two systems of a combination of a DCT processing unit and a quantization processing unit. The DCT unit 141 and the quantization unit 142 of the first system encode all the frames of the input broadcast data in a frame and output the I-frame data for interpolation. The DCT unit 143 and the quantization unit 144 of the second system are for encoding a frame of the input broadcast data into intra-frame or inter-frame and outputting the same as broadcast video data. Performs the same processing as the conversion unit. The encoded data output from each of the first system and the second system is output to the material selection device 15 while synchronizing in frame units.

The on-demand material server 16 holds the on-demand video data and provides it to the material selecting device 15 in response to a user request. The on-demand video data is compressed and encoded in the MPEG-4 format and stored like the broadcast video data.

The material selection device 15 receives the broadcast video data and the interpolation I-frame data from the encoder 14 and the on-demand video data from the on-demand material server 16 and sends them to the distribution server 17. Of these, as for the interpolation I-frame data, only the necessary frame data is output at the required timing. The processing related to the interpolation I-frame data by the material selecting device 15 will be described in more detail later.

The distribution server 17 transmits the three types of video data (broadcast video data, on-demand video data, and interpolation I-frame data) output from the material selection device 15 to the user terminal 18 via the Internet N.

In the case of the broadcast video data, the distribution server 17 transmits the broadcast video data simultaneously to all the user terminals to be broadcast at the timing according to the time table. The user terminal to be broadcast is a user terminal that has issued a request for transmission of broadcast video data to the distribution server 17 before that.

In the case of the on-demand video data, upon receiving the on-demand video data from the material selection device 15, the distribution server 17 interrupts the transmission of the broadcast video data to the requesting user terminal and transmits the on-demand video data.

The interpolation I-frame data is transmitted to the user terminal requesting the on-demand video data when the transmission of the above-described on-demand video data is completed. Then, after the transmission of the interpolation I-frame data is completed, the transmission of the broadcast video data to the user terminal is restarted.

The user terminal 18 is a device that transmits a user's request to the distribution server 17, decodes video data transmitted from the distribution server 17 in response to the request, and displays the screen. In particular, when resuming reception of broadcast video data after receiving on-demand video data, it receives interpolation I-frame data, decodes it, displays it on a screen, and then uses the decoded interpolation I-frame data as reference frame data. Then, the frames of the broadcast video data received thereafter are decoded.
(Description of main components)
Hereinafter, the main components (the material selection device 15, the distribution server 17, and the user terminal 18) of the above-described video data transmission / reception system 1 will be described in more detail.
(Detailed configuration)
Hereinafter, the three main components in the video data transmission / reception system 1 according to the present embodiment will be described in more detail with reference to FIG. In particular, the processing related to the interpolation I-frame data will be described.

FIG. 4 is a block diagram showing the configuration of the three main components.

(Distribution server 17)
The distribution server 17 includes a request receiving unit 171, a user managing unit 172, a video requesting unit 173, a video receiving unit 174, and a video transmitting unit 175. The processing content of each component will be described.

The request receiving unit 171 receives a request related to video data transmission (a broadcast video data transmission start / end request, an on-demand video data transmission request) from the user terminal 18 via the Internet N, and outputs the request to the user management unit 172. . The request includes identification information of the requesting user terminal (such as an IP address) and identification information of the video data to be requested.

(4) The user management unit 172 manages information on the user terminal to which the video data is to be transmitted. The identification information of the user terminals to which the data is to be transmitted and the management information indicating the type and identification information (in the case of on-demand video data) of the data to be transmitted to each of the user terminals are held, and are stored in the request receiving unit 171. Update according to the content of the request from the user sent via the In addition, when receiving a request for transmitting on-demand video data, it outputs the content to the video request unit 173.

Upon receiving the transmission request for the on-demand video data from the user management unit 172, the video request unit 173 sends the request to the material selection device 15 to instruct reading of the on-demand video data.

The video receiving unit 174 receives various video data from the material selection device 15 and sends the video data to the video transmitting unit 175. The on-demand video data and the interpolation I-frame data among the video data sent from the material selection device 15 have the identification information of the requesting user terminal included in the above request added.

The video transmission unit 175 transmits various video data transmitted from the video reception unit 174 to the user terminal 18 according to management information held by the user management unit 172. When the transmission of the on-demand video data and the interpolation I-frame data is completed, the identification information of the destination user terminal is sent to the user management unit 172, and the content of the management information for the user terminal is set to “on-demand video transmission”. The content is updated from that shown to “broadcast video data transmission”.

(Material selection device 15)
The material selection device 15 includes a transmission / reception unit 151, a material determination unit 152, an I frame buffer for interpolation 153, a frame determination unit 154, and an on-demand data reading unit 155.

The transmission / reception unit 151 receives the transmission request for the on-demand video data from the distribution server 17 and outputs the content (the identification information of the video data and the identification information of the requesting user terminal) to the material determination unit 152. Further, it transmits various video data output via the material determination unit 152 to the distribution server 17.

The on-demand data reading unit 155 reads the requested on-demand video data from the on-demand material server 16 and outputs the data to the material determining unit 152. When outputting the last frame of the on-demand video data, information indicating the end is added.

The frame determination unit 154 receives the broadcast video data output from the encoder 14 and outputs the broadcast video data to the material determination unit 152. At this time, for each of the frame data constituting the video data, the attribute information is analyzed or the like, and the frame type (I, P, B) and the serial number of the frame (the order in the entire video data or the order in the GOP to which it belongs) Is output to the material determination unit 152 together with the frame data. These pieces of information are referred to for determining the necessity of outputting the interpolation I frame data described later and for specifying the interpolation I frame data to be output. Note that the frame determination unit 154 may count the frame serial number by itself.

The I frame buffer for interpolation 153 holds the I frame data for interpolation output from the encoder 14 for several frames. Also, the counter holds the counter and counts the serial number of each I-frame data for interpolation, and adds it to each data (counting is performed by a method corresponding to the serial number obtained by the frame determination unit 154). Then, when an output request for the interpolation I-frame data is received from the material determination unit 152, the interpolation I-frame data corresponding to the serial number added to the request is output to the material determination unit 152. The interpolation I-frame data exceeding the predetermined number is deleted from the oldest one.

The material determination unit 152 acquires broadcast video data, on-demand video data, and I-frame data for interpolation, outputs the data to the transmission / reception unit 151, and transmits the data to the distribution server 17 as appropriate. The processing of the material determining unit 152 when transmitting the on-demand video data and when resuming the transmission of the broadcast video data after the transmission of the on-demand video data is completed will be described below with reference to the drawings.

FIG. 5 is a flowchart showing the flow of processing by the material determination unit 152.

At the time of transmission of the on-demand video data, when the requested on-demand video data is output from the on-demand data reading unit 155, the material determination unit 152 outputs this to the distribution server 17 in parallel with the broadcast video data. (S501).

Then, when the end frame of the on-demand video data is output from the on-demand data reading unit 155 (S502: Yes), the material determination unit 152 determines the last frame of the on-demand video data among the frames of the broadcast video data. Subsequently, for a predetermined number of frames to be transmitted, a serial number, attribute information, and the like are obtained (S503), and based on these, it is determined whether or not replacement with interpolation I frame data is necessary. The number of frames of data to be determined depends on the frame configuration at the time of encoding the broadcast video data. As shown in FIG. 2, the configuration of the broadcast video data handled by the present system 1 is that “I or P frame appears at a rate of one out of three”, so it is determined whether or not replacement is necessary for three frames. . Alternatively, a frame up to the appearance of the next I frame data (start of the next GOP) is set as a determination target. If the frame structure (reference relationship) is irregular, it is preferable to repeat the process until the I frame of the next GOP appears.

If the type of the frame to be determined is “I” (S504: Yes), the frame and the subsequent frame can be correctly decoded even at the user terminal 18, so that it is not necessary to replace the frame with interpolation I frame data. Then, the distribution server 17 is notified of the fact and the broadcast video data is transmitted to the user terminal 18 (S508). Further, the determination of the necessity of replacement with the interpolation I frame data is terminated.

If the frames to be determined are P and B frames (S504: No), the material determination unit 152 checks whether or not a reference frame of the frame has been transmitted to the user terminal 18 (in the case of a B frame, 2). (The preceding reference frame of the two reference frames) (S505). Specifically, the serial number of the reference frame of the frame is compared with the serial number of the frame data of the broadcast video data corresponding to the last frame in the on-demand video data, and if the former value is larger than the latter, The reference frame has reached the user terminal 18. If the reference frame has been received by the user terminal 18 (S505: Yes), the broadcast video data frame can be decoded. Therefore, the frame data is not replaced with the interpolation I frame data, and the frame data is directly transmitted to the user terminal 18. The distribution server 17 is instructed to transmit (S507).

The serial number of the reference frame can also be obtained from the serial number of the frame to be determined itself and the frame structure of the broadcast video data. In the case of the broadcast video data of the present example having the frame structure shown in FIG. 2, the reference frame of the P frame is the I or P frame that is three numbers earlier than the P frame. In the case of the B frame whose serial number is a multiple of 3 (frame 223 in FIG. 2), the reference frame is the serial number and the serial number of the I or P frame (frame 221 in FIG. 2) two immediately before. The next P frame (frame 224 in FIG. 2) and its own serial number is “multiple of 3−1” (frame 222 in FIG. 2) is the reference frame of the immediately preceding I frame. Alternatively, the P frame (the frame 221 in FIG. 2) and the P frame (the frame 224 in FIG. 2) that is two numbers later in sequence.

On the other hand, if the reference frame has not been transmitted (S505: No), decoding is impossible even if the data of the broadcast video data frame is transmitted to the user terminal 18, so that the corresponding (having the same serial number) The I frame data is read from the interpolation I frame buffer 153, sent to the distribution server 17, and transmitted to the user terminal 18 (S506). Note that while the interpolation I-frame data is being transmitted to the user terminal 18, the broadcast video data must be transmitted to the other user terminals. Broadcast video data is also transmitted in parallel with the frame data.

The processes of steps S503 to S507 are repeated until a determination is made for a predetermined number of frames after resumption or until I-frame data of the next GOP appears (S509). Then, after the processing shown in this figure ends, broadcast video data after the check target frame is transmitted unconditionally.

In the above description, the material determination unit 152 assumes that the broadcast video data and the on-demand video data are output to the distribution server 17 in parallel one frame at a time, and the broadcast video data is converted into the interpolation I frame one frame at a time. Although it is determined whether or not to perform replacement, based on information such as a frame configuration at the time of encoding and the number of frames of on-demand video data, at the stage of starting transmission of on-demand video data, “replacement necessity is determined” , "The number and serial number of frames to be replaced" can be determined in advance. Specifically, at the start of the transmission of the on-demand video data, the serial number of the broadcast video data frame corresponding to the first frame of the on-demand video data is obtained, and the number of frames of the on-demand video data is added thereto. The frame serial number of the broadcast video data at the time can be calculated. The frame structure of the broadcast video data is determined in advance as shown in FIG. 2, and as described above, a frame whose serial number is 3n + 1 (n is an integer of 0 or more) is an I or P frame. (The determination of I or P is made based on the number of frames of one GOP), and the frames having other serial numbers are B frames. Can be determined.

When the serial number of the frame to be replaced with the interpolation I-frame data can be predicted in advance, the interpolation I-frame data may be generated only for the replacement-predicted frame (in the above description, the interpolation I-frame data may be generated for all the frames). Was generated). Then, the processing load for generating the interpolation I-frame data can be reduced. This is particularly effective in a system in which on-demand video data transmission requests do not frequently occur.

Also, fixed replacement may be performed in a fixed manner in such a manner that “n frames immediately after restarting are unconditionally replaced (without determination processing) by interpolation I frames (n is the frequency of appearance of I or P frames). Good. In such a case, the load of the necessity of replacement is eliminated. As shown in FIG. 2, the frame configuration of the broadcast video data in the present embodiment is such that “I or P type frame data appears every three frames (I or P frame appearance frequency = 3), The immediately preceding I or P frame is used, and the B frame uses the preceding or succeeding I or P frame as a reference frame. " That is, no matter what kind of frame of the broadcast video data immediately after the restart, if at most three frames (= frequency of occurrence of I or P frames) are replaced with interpolation I frames, all subsequent frames can be decoded without any problem. .

Also, regardless of the appearance frequency of I or P frames, if the system is replaced with the interpolation I frame data up to the first P frame data after resuming, the necessity of replacement can be determined only by checking the frame type. As a result, the load of the determination process can be reduced. That is, if the frame immediately after the restart is the P type, only the P type frame is replaced with the interpolation I frame. If the frame immediately after the restart is the B frame, the interpolation I frame including the B frame is interpolated until the first subsequent P frame. Replace with a frame.

(User terminal 18)
The user terminal 18 receives various video data from the distribution server 17, decodes the video data, and displays the video data on a screen. The user terminal 18 includes a request receiving unit 181 that receives a request from the user, a request transmitting unit 182 that transmits the request to the distribution server 17, a video receiving unit 183 that receives video data transmitted from the distribution server 17 in response to the request, It has a decoder 184 for decoding video data, and a video display unit 185 for displaying the video of the decoded result on the screen.
(Flow of video data transmission / reception processing)
Next, regarding the transmission / reception processing of the video data in the video data transmission / reception system 1, the exchange of various data between the above-described main components and the flow of the processing of each section will be described below.

FIG. 6 shows a process in which a certain user terminal 18 interrupts reception of broadcast video data, receives on-demand video data, and then resumes reception of broadcast video data in the video data transmission / reception system 1 of the present embodiment. FIG. 9 is a sequence diagram showing a processing flow and data exchange of the user terminal 18, the distribution server 17, and the material selection device 15 when the data is exchanged.

First, an on-demand video data transmission request is sent from the user terminal 18 receiving broadcast video data to the distribution server 17 to start processing (S601). Although not shown in the figure, at this point, the broadcast video data has been transmitted from the distribution server 17 to other user terminals.

The request is further sent from the distribution server 17 to the material selection device 15.

(4) The material selection device 15 that has received the request via the distribution server 17 reads out the requested on-demand video data from the on-demand material server 16 and sends it to the distribution server 17. The distribution server 17 interrupts the transmission of the broadcast video data to the user terminal 18 and transmits the on-demand video data (S602). However, even during this time, the distribution server 17 continues to transmit broadcast video data to other user terminals.

Upon detecting the end of the on-demand video data (S603), the material selection device 15 transmits the broadcast video data to the user terminal 18 immediately after the completion of the transmission of the on-demand video data among the broadcast video data being transmitted to the other user terminals. The attribute and the reference frame are checked for several frames, and it is determined whether the transmission of the interpolation I-frame data is necessary (S604). If the type of the frame is P or B, and the reference frame of the frame is not received by the user terminal 18 overlapping with the on-demand video data, the material selection device 15 sets the The one corresponding to the frame is selected and sent to the distribution server 17, and the distribution server 17 transmits the interpolation I-frame data to the user terminal 18 following the end data of the on-demand video data (S605).

(4) The user terminal 18 decodes the interpolation I-frame data and displays it on the screen (S606).

After the transmission of the interpolation I frame (or when the transmission of the interpolation I frame is unnecessary), the transmission of the broadcast type video data from the distribution server 17 to the user terminal 18 is restarted (S607). The decoding of the broadcast video data after restart is started using the decoding result of the previously received interpolation I-frame data as reference frame data (S608).
(Summary)
As is apparent from the above description, in the video data transmission / reception system 1 according to the present embodiment, when the transmission of the on-demand video data is interrupted while the broadcast video data is being transmitted from the transmitting device to the user terminal. Before the transmission of the broadcast video data is resumed after the transmission of the on-demand video data is completed, the I-frame data for interpolation is transmitted to the user terminal. The user terminal first decodes and displays the interpolation I frame, and decodes several frames of the broadcast video data immediately after the restart using the interpolation I frame as a reference frame and displays the decoded image on the screen.

This allows the user terminal to correctly decode the broadcast video data and display it on the screen immediately after the end of the on-demand video data.

In the present embodiment, the material selecting device 15 and the distribution server 17 are described as separate devices, but the present invention can be implemented as a single device by combining these two devices.

Also, in the present embodiment, as shown in FIG. 2, “the appearance frequency of the I or P frame is 3, the P frame refers to the immediately preceding I or P frame, and the B frame refers to the preceding or succeeding I or P frame. Although the encoded data having the structure of “frame” and the reference rule are described as an example, the structure and reference rule of the encoded data to which the present invention can be applied are not limited thereto. Regardless of encoded data having any structure and reference rules, it is possible to determine whether or not replacement at the time of resuming transmission is possible as long as a reference frame can be specified for each frame of broadcast video data. Is effective.

Further, the encoder 14 in the present embodiment has two systems of “DCT to quantization processing mechanism”, and one performs conventional processing (mixing of intra-frame coding and inter-frame coding). The other is a special encoder that performs only intra-frame encoding. However, two conventional encoders are prepared, one for generating I-frame data for interpolation, and the other for generating broadcast video data. The system may be configured to perform each of them. With this configuration, two encoders are required and the configuration becomes large, but instead, the present invention can be implemented using only conventional encoders.

The encoder according to the present embodiment is divided into a first system, a second system DCT unit, and a quantization unit. The first system generates I frame data for interpolation, and the second system generates broadcast video data. However, data generation is not limited to such a form. For example, a part of the I frame data generated by the first system may be diverted to broadcast video data, and the second system may generate only P and B frame data. That is, for the first frame of the GOP, the first system performs intra-frame encoding, while the second system passes the encoding process, and the intra-frame encoded data output by the first system is interpolated for the frame. The data is output to the material selection device as part of the broadcast video data as well as the I-frame data for use, and is also output to the inverse quantization unit to be used as reference frame data of the subsequent frame. Other frames are coded in respective systems as in the present embodiment. In this case, a new process is required in the SW to output coded data output from the first system as broadcast video data only for I frame data. In this method, the duplication of processing in which both systems perform intra-frame encoding for the first frame of the GOP is eliminated, and the load is reduced accordingly.

Also, in the system described in the embodiment, data transmission from the distribution server to the user terminal is simply performed via the Internet, but various forms of data transmission are conceivable.

FIG. 7 is a diagram illustrating, as an example, a video data transmission / reception system in which video data transmission from a distribution server 717 to user terminals 718 (collectively, terminals 718a, 718b, 718c, and 718d) is performed via a router. . Here, the routers 701 to 703 are multicast routers.

FIG. 7A shows a situation in which broadcast video data (indicated by black arrows) is transmitted from the distribution server 717 to the user terminal 718. In FIG. 7B, the terminal 718a among the user terminals issues a transmission request of the on-demand video data, and the on-demand video data is transmitted from the distribution server 717 to the user terminal 718a via the routers 701 and 702 in response to the request. Indicates the situation. After that, when the transmission of the on-demand video data is completed, the interpolation I-frame data (indicated by the white arrow) is transmitted to the user terminal 718a through the same path as the transmission of the on-demand video data in FIG. 7B. Then, as shown in FIG. 7A, the broadcast video data is transmitted again.

In this system, the contents of processing performed by the material selection device 15, the distribution server 17, and the like in connection with the transmission of the interpolation I frame are the same as those of the system 1 in the embodiment.

Further, the present embodiment is a video data transmission / reception system in which on-demand video data is transmitted in an interrupted manner during transmission of broadcast video data. However, the system configuration and video data Are not limited to those shown in the present embodiment. An object of the present invention is a system in which main video data compressed and encoded using inter-frame motion compensation prediction is transmitted from a transmission source device to a reception side device, and the main video data is transmitted to the reception side device. This is a system in which reception of a message may be interrupted. However, the interruption is not merely an interruption, but is an interruption in which the content of the reference frame remaining in the decoding unit of the receiving apparatus does not correspond to the frame of the main video data at the time of resuming the transmission.

As such a pattern of interruption, besides the one described in the embodiment that “other data is transmitted in a form of being replaced with a part of the main video data”, “other data is transmitted in the middle of the main video data Or a case where part of the main video data (particularly, I-frame data) is not received due to a communication failure.

Also, if an interruption occurs in a system that transmits video data in a broadcast image according to the time table, when resuming transmission, transmission is resumed from the point where the frame for the interruption time was skipped instead of resuming from the interrupted point However, in this case, even if the receiving apparatus does not decode other data during the suspension period, the content of the reference frame at the time of resumption does not correspond to the frame of the main video data. In this case, the cause of the interruption may be communication failure, transmission / reception of data (character data or still image data) other than video data, and the like.

Further, the system according to the present embodiment is such that on-demand video data is transmitted from the distribution server to the requesting user terminal in response to a request from the user terminal, thereby interrupting the broadcast video data. There is also a video data transmission / reception system in which the transmission side plays a main role and inserts other video data in the middle of broadcast video data.

シ ス テ ム The system shown in FIG. 8 is a video data transmission / reception system 8 in which the transmission side mainly plays a role of inserting other video data in the middle of broadcast video data. During the transmission of the broadcast video data to the user terminal 18, the distribution server 817 transmits other video data (here, video data of a CM that matches the taste and taste of each user) to each user terminal as appropriate. Therefore, the system 8 includes a distribution material selection device 801 that selects CM video data according to taste for each user. The distribution material selection device 801 obtains information on the hobbies and preferences of each user as a selection criterion (user's age, gender, products of interest, etc.) from the user terminal 18 via the Internet N using a means such as a questionnaire. I do.

The distribution material selection device 801 outputs, to the material selection device 815, for example, commercial video data based on a contract concluded between the provider and the sponsor of the pay broadcast video data transmitted in a broadcast format. Prompt for transmission to the destination user terminal. On the other hand, the material selection device 815 outputs the CM video data for the CM destination user terminal to the distribution server 817, and causes the user terminal 18 to transmit. Then, when the end of the CM video data is detected, the material selection device 815 transmits the necessary interpolation I-frame data to the user terminal 18 and broadcasts it in the same procedure as described in the first embodiment. The transmission of the video data is restarted.

As described above, the same effects as those of the system 1 of the embodiment can be obtained in a system in which the transmitting apparatus takes the initiative to suspend and resume broadcast video data. Note that the type of video data transmitted by the transmitting device is not limited to CM video data, and may be other video materials.
(Modification)
The system 1 according to the above-described embodiment enables the broadcast video data to be correctly decoded when the transmission / reception of the broadcast video data is interrupted / resumed due to the interruption of the on-demand video data. In a modification, when a user terminal capable of receiving a plurality of pieces of broadcast video data switches broadcast video data, depending on the type of a frame immediately after the switching in the broadcast video data, the broadcast after switching until the next GOP start is performed. This is to solve the problem that video data cannot be decoded correctly.

シ ス テ ム Hereinafter, the system of the modified example will be described. Since there are many configurations common to the system in the embodiment, particularly different portions will be described.

In the system 9 shown in FIG. 9, a plurality of broadcast video data providing devices 91a and 91b (camera and / or material servers, switchers, encoders, material selection devices, etc.) are transmitted to a distribution server 917 for transmitting video data to a user terminal. ) Provides broadcast video data. From each of the two providing devices, different broadcast video data is provided to the distribution server 917 according to a predetermined time table. The distribution server 917 transmits any broadcast video data in response to a request from the user terminal. This means that each of the plurality of broadcast video data providing devices corresponds to one channel, and the user can switch the broadcast video data so as to switch the channel. Then, when switching the channel (broadcast video data), the interpolation I-frame data is transmitted to the user terminal prior to the start of the transmission of the broadcast video data after the switch. Thus, even at the time of switching, the broadcast video data after switching is immediately reproduced and displayed correctly by the user terminal.

(Constitution)
In the system 9, it is the distribution server 917 and the material selection devices 915a and 915b in the first and second video data providing devices 91a and 91b that perform the processing relating to the transmission of the interpolation I frame data when the broadcast video data is switched. . Hereinafter, the processing of these components will be described. The description of the processing common to the system 1 is omitted.

The distribution server 917 receives the broadcast video data from each of the first and second video data providing devices 91a and 91b, and transmits each to the desired user terminal. When a broadcast video data switching request is received from the user terminal 918 that has received the video data provided by the first video data providing device 91a, the second video data providing device 91b that provides the broadcast video data after switching is received. The material selection device 915b on the side is notified. Then, when the interpolation I frame data is output from the material selection device 915b, the transmission of the first video data to the user terminal 918 is stopped, and then the interpolation I frame data is transmitted. Transmission of broadcast video data is started. On the other hand, if the notification of “I frame data for interpolation is unnecessary” is sent from the material selection device 915b, the transmission of the first video data to the user terminal 918 is stopped, and the transmission of the second broadcast video data is stopped. Start.

When the material selection device 915b receives the notification that the switching has been requested, if the switching is performed at this time, the material selecting device 915b specifies the frame to be received by the user terminal 918 immediately after the switching. The processing after detecting this frame is the same as the processing in the video data transmission / reception system 1. That is, it is determined whether or not replacement of the detected frame and, if necessary, several subsequent frames with the interpolation I-frame data is necessary. If necessary, the interpolation I-frame data is read from the I-frame buffer and the frame serial number is read. Is output to the distribution server 917. Then, when a frame that does not need to be replaced with the interpolation I frame is detected, a notification that “interpolation I frame is unnecessary” is sent to the distribution server 917. In this case, the serial number of the frame determined not to be replaced is also sent to the distribution server 917, and transmission of the second broadcast video data to the user terminal from the data of the frame corresponding to the serial number in the second broadcast video data is started. Will be prompted. In addition, the distribution server 917 determines the frame serial number in the same manner as the material selection devices 915a and 915b.

(Processing flow)
Hereinafter, the flow of transmission switching processing of broadcast video data in the video data transmission / reception system 4 will be described with reference to a sequence diagram showing a transition of control between main components and a data flow.

FIG. 10 is a sequence diagram showing a control transition and data flow between the two broadcast video data providing devices 91a and 91b, the distribution server 917, and the user terminal 918 in the broadcast video data transmission switching process. The sequence shown here is a case where the video data to be transmitted and received is switched from the first broadcast video data to the second broadcast video data.

First, in a state where the first broadcast video data is transmitted via the distribution server 917 from the first providing device 915a (S1001), the user terminal 918 receiving the user's instruction is A request to switch to the second broadcast video data is transmitted to the distribution server 917 (S1002).

に 対 し In response, the distribution server 917 notifies the second broadcast video data providing device 91b, which is the source of the second broadcast video data, of the occurrence of the switching request. On the other hand, the second broadcast video data providing device 91b determines whether or not the frame to be received by the user terminal 918 immediately after the switching is to be replaced with the interpolation I frame data (S1003), and determines that the frame is necessary. In this case, the I frame data for interpolation is transmitted to the distribution server 917 (S1004). The distribution server 917 stops transmitting the first broadcast video data to the user terminal 918, and transmits the received interpolation I frame data. The user terminal 918 receives the I frame data for interpolation, decodes it, and displays it on the screen (S1005). After that, the distribution server 917 transmits the second broadcast video data to the user terminal 918 from a portion that does not need to be replaced with the interpolation I frame, and the user terminal 918 uses the interpolation I frame data decoded in step S1005 as a reference frame. , The decoding of the second broadcast video data is started (S1006).

As described above, in the present system 9, a plurality of video data providing devices 91a and 91b are connected to the distribution server 917, and different broadcast video data is provided from each. Then, when there is a video data switching request from the user terminal 918, an I frame for interpolation is provided to the user terminal 918 for the frame immediately after switching in the video data after switching. Thus, even when the broadcast video data received by the user terminal is switched, the broadcast video data after the switch can be decoded and displayed on the screen without interruption.

Although this modification shown in FIG. 9 does not include a configuration for transmitting and receiving on-demand video data, the system in the embodiment shown in FIG. It is also possible to realize a system that performs transmission and reception of interpolation I-frame data at any time of switching of broadcast video data after data transmission and reception.
(Modification 2)
Also, as a form of service, another on-demand data (second on-demand data) is transmitted in an interrupting manner during transmission of certain on-demand data (first on-demand data), and then transmission of the first on-demand data is performed. May be restarted. In that case, the same problem as in the above embodiment may occur immediately after switching from the second on-demand data to the first on-demand data. Therefore, the present modified example shows a system corresponding to such a form switching.

FIG. 11 shows a material selection device 15A in the video data transmission / reception system according to the present modification. The characteristic part in this modification is the on-demand data reading part 155A in the material selection device 15A. The on-demand data reading unit 155A switches on-demand data to be output to a certain user terminal, and performs re-switching from second on-demand data after switching to first on-demand data before switching. , And outputs an interpolation I-frame before transmitting the first on-demand data.

Regarding the I-frame data for interpolation, it is conceivable to generate the I-frame data for interpolation for all of the on-demand data in advance and store it in the on-demand material server 16, but this is not realistic because the data amount becomes large. Therefore, before restarting the output of the first on-demand data, interpolation I-frame data is generated from several frames immediately after the restart, and is output.

The on-demand data reading unit 155A includes a switching unit 1551, an auxiliary decoder 1552, and an auxiliary encoder 1553 as a configuration for that purpose. Further, on-demand data reading section 155A also performs the same processing as on-demand data reading section 155 in the embodiment, but a description thereof will be omitted.

The switching unit 1551 determines the serial number and the frame type of the frame of the first on-demand data transmitted immediately after resuming the transmission from the frame serial number of the first on-demand data and the total number of frames of the second on-demand data at the time of switching. judge. Then, first, it is determined whether or not to generate the interpolation I-frame data based on the frame type (not necessary for an I-frame, otherwise required). If necessary, the number of frames to be replaced with the I frame for interpolation is further obtained, and the identification information of the first on-demand data, the serial number of the frame immediately after the restart of transmission, and the information of the number of frames are sent to the auxiliary decoder 1552. Instructs execution of processing. The determination of the necessity of the interpolation I frame and the determination of the frame requiring replacement may be performed in the same procedure as that performed by the material determination unit 152 in the embodiment.

Then, the switching unit 1551 replaces the interpolation I frame data obtained as a result of the processing performed by the auxiliary decoder 1552 and the auxiliary encoder 1553 with the frame of the first on-demand data immediately after restarting the transmission, and outputs the frame. I do.

Upon receiving the instruction from the switching unit 1551, the auxiliary decoder 1552 determines the GOP to be decoded (the GOP including the frame immediately after the restart of transmission) in the first on-demand data based on the frame serial number information. Then, only the GOP is read from the on-demand material server 16 (at this time, refer to the identification information). As a determination method, for example, the value of the serial number is divided by the number of frames per GOP, and a quotient is obtained by rounding up a decimal. Assuming that the quotient is n, the n-th GOP from the head of the first on-demand data is obtained.

Next, when the auxiliary decoder 1552 decodes the read GOP from the beginning to several frames after the restart of transmission (a frame to be replaced with an interpolation I frame), the auxiliary decoder 1552 adds the respective serial numbers to the decoding result data of the replacement target frame. Then, an output is made to the auxiliary encoder 1553, and an instruction is made to generate I frame data for interpolation from these frames.

The auxiliary encoder 1553 performs intra-frame encoding on all three frames received from the auxiliary decoder 1552, and outputs the result to the switching unit 1551 together with the respective frame serial numbers.
(Other)
In the above-described embodiment and its modified example, the Internet is cited as a communication path when video data is encoded by an encoder and distributed via a distribution server such as an ISP (Internet Service Provider). The applicable transmission / reception path is not limited to this, and a broadcast wave or CATV may be used as other communication means. For example, it is also conceivable that main broadcast program data is transmitted and received by broadcast radio waves, and optional on-demand data is transmitted and received via a communication line such as the Internet.

In the above-described embodiment and its modifications, the present invention is a video data transmission / reception system. However, the present invention can also be realized by executing a program on a computer. May be recorded in the file.

Further, in the above-described embodiment and its modified example, the standard of the compression encoding of the video data is described as MPEG-4. However, the present invention performs the encoding using the inter-frame encoding and the motion compensation inter-frame prediction. This is effective for the entire data to be obtained. For example, video data compressed by another compression encoding method such as MPEG-2 is also an object of the present invention.

Further, in the above-described embodiment and its modification, the on-demand video data is always transmitted to the end, but the transmission of the on-demand video data may be interrupted halfway by a request from the user. sell. The present invention can cope with such a case of discontinuation on the way. Specifically, in the above-described embodiment, the processing that has been performed when “the end of on-demand video data is detected” is changed to “when the request for discontinuing the on-demand video data is issued from the receiving device side”. "Can also be performed.

The video data transmission / reception system, the video data transmission device, and the method according to the present invention provide a video data compression-encoded using motion-compensated inter-frame prediction, even if the video data is reproduced from the middle of a GOP due to interruption or the like. This has the effect that video data can be correctly decoded and displayed on the screen, and is useful as a video transmission / reception system or the like in which the above-described video data is transmitted by the transmitting device and the receiving device decodes and displays the screen.

1 is a configuration diagram illustrating an embodiment of a video data transmission / reception system according to the present invention. FIG. 3 is a schematic diagram showing a reception state of various video data according to the embodiment. FIG. 2 is a block diagram illustrating a configuration of an encoder according to the embodiment. FIG. 2 is a block diagram illustrating a detailed configuration of a material selection device, a distribution server, and a user terminal according to the embodiment. It is a flowchart which shows the flow of the replacement necessity determination processing in the embodiment. FIG. 3 is a sequence diagram showing a flow of a video data transmission / reception process in the embodiment. FIG. 3 is a diagram illustrating a configuration when a router is used for transmission and reception according to the present embodiment. FIG. 3 is a diagram illustrating a configuration in a case where transmission and reception of on-demand video data is mainly performed by a transmission side according to the present embodiment. FIG. 14 is a block diagram showing a configuration of a modification example of the same embodiment. It is a sequence diagram which shows the flow of the video data transmission / reception processing in the modification. It is a block diagram which shows the structure of the material selection device among the video data transmission / reception systems in another modification of the embodiment. FIG. 9 is a schematic diagram illustrating a reception state of various video data in the related art.

Explanation of reference numerals

1, 8, 9 Video data transmission / reception system 141 First DCT section 142 First quantization section 143 Second DCT section 144 Second quantization section 15, 15A Material selection device 152 Material determination section 153 Interpolation I-frame buffer 154 Frame determination section 155 , 155A On-demand data reading unit 16 On-demand material server 17 Distribution server 18 User terminal

Claims (17)

A video data transmitting and receiving system comprising a transmitting device that transmits video data compressed and encoded using motion compensation inter-frame prediction and a plurality of receiving terminals that receive and decode the video data,
The transmitting device,
A first encoding unit that generates the video data by performing an intra-frame encoding process or an inter-frame encoding process on each of a plurality of frames configuring moving image data;
In parallel with the encoding process by the first encoding unit, a second encoding unit that generates an interpolation I-frame data by performing an intra-frame encoding process on a frame configuring the moving image data;
Transmitting means for transmitting the video data to the plurality of receiving terminals,
The transmission means, when temporarily interrupting the transmission of the video data to any of the plurality of receiving terminals and then restarting, the interpolation I frame data for at least one frame prior to restarting the transmission of the video data, Transmitted to the receiving terminal,
When the transmission of the video data once interrupted by the transmission unit is resumed, the reception terminal first receives and decodes the interpolation I-frame data transmitted from the transmission unit, and decodes the decoded result data. Decoding the video data after resumption as reference frame data,
A video data transmitting and receiving system characterized by the following. A video data transmitting device that transmits video data that has been compression-encoded using motion compensation inter-frame prediction to a plurality of receiving terminals,
A first encoding unit that generates the video data by performing an intra-frame encoding process or an inter-frame encoding process on each of a plurality of frames configuring moving image data;
In parallel with the encoding process by the first encoding unit, a second encoding unit that generates an interpolation I-frame data by performing an intra-frame encoding process on a frame configuring the moving image data;
When the video data is transmitted to the plurality of receiving terminals and transmission of the video data to any one of the plurality of receiving terminals is temporarily interrupted and then resumed, at least one frame is transmitted before resuming the transmission of the video data. Transmitting means for transmitting the interpolation I-frame data to the receiving terminal. In parallel with the transmission of the video data, further comprising option data transmission means for transmitting optional video data to any of the plurality of receiving terminals,
The transmission of the video data from the transmitting unit to any one of the plurality of receiving terminals is interrupted by the optional video data transmission by the optional data transmitting unit to any of the receiving terminals being interrupted. thing,
The video data transmission device according to claim 2, wherein: The option data transmission unit has an information collection unit that collects information about user preferences in advance from at least a part of the plurality of reception terminals,
Selecting the content of the optional video data to be transmitted based on the information collected by the information collecting means,
The video data transmission device according to claim 3, wherein: The transmission means has a broadcast transmission means for broadcasting the same data to a plurality of transmission destinations, and an individual transmission means for transmitting individual data to individual transmission destinations. The broadcast transmission means, using the individual transmission means for the interpolation I frame data,
The option data transmitting means individually transmits the option video data,
The video data transmission device according to claim 3 or 4, wherein: The transmitting means has a switching means for excluding the receiving terminal to which the interpolation I frame data or the optional video data is being transmitted from the transmission of the video data by the broadcast transmitting means,
The video data transmitting apparatus according to claim 5, wherein: The option data transmitting means,
Insertion means for transmitting the second option data in the form of inserting the second option data during transmission of the first option data;
Before the transmission of the second option data is completed and before the transmission of the first option data is resumed, at least one frame from the position where the transmission of the first option data is resumed is a corresponding I-frame for the option data interpolation. And third encoding means for generating data.
When the transmission of the first option data is resumed after the transmission of the second option data is completed, the option data transmitting means may perform the interpolation of the option data before the transmission of the first option data. Transmitting I frame data to the receiving terminal;
The video data transmitting apparatus according to claim 3, wherein: The first encoding unit and the second encoding unit are realized by separate encoders;
The video data transmitting device according to claim 2, wherein: The transmitting means determines the number of frames of the interpolation I frame data to be transmitted to the receiving terminal before resuming the transmission of the video data based on the GOP configuration of the video data, in particular, the appearance frequency of the I attribute or P attribute frame. To decide,
The video data transmitting device according to claim 2, wherein: A video data transmitting device that transmits video data that has been compression-encoded using motion compensation inter-frame prediction to a plurality of receiving terminals,
First encoding means for performing an intra-frame encoding process on a frame constituting moving image data to generate intra-frame encoded video data;
A second encoding unit that generates inter-frame encoded video data by performing an inter-frame encoding process on a frame configuring the moving image data;
Video data generating means for generating the video data from intra-coded video data generated by the first coding means and inter-coded video data generated by the second coding means;
Transmitting means for transmitting the video data to the plurality of receiving terminals,
When the transmission unit temporarily suspends the transmission of the video data to any of the plurality of receiving terminals and then resumes the transmission, the transmission unit transmits the video data in a frame generated by the first encoding unit before resuming the transmission of the video data. Transmitting at least one frame of the encoded video data to the receiving terminal as I-frame data for interpolation;
A video data transmission device characterized by the above-mentioned. A plurality of video data providing devices for transmitting video data compressed and encoded using motion compensated inter-frame prediction, a plurality of receiving terminals for receiving and decoding video data from any of the plurality of video data providing devices, A video data transmitting and receiving system comprising a video data providing device and a distribution server that relays video data between the plurality of receiving terminals,
The video data providing devices each include:
A first encoding unit that generates video data by performing an intra-frame encoding process or an inter-frame encoding process on each of a plurality of frames forming the moving image data;
In parallel with the encoding process by the first encoding unit, an intra-frame encoding process for each of the plurality of frames constituting the moving image data to generate interpolation I-frame data; ,
The distribution server,
A switch request receiving unit that receives a switch request for switching video data to be received from any of the plurality of receiving terminals,
When the switching request receiving unit receives the switching request, the transmission of the video data before switching to the receiving terminal of the switching request source is stopped, and then the interpolation obtained from the video data providing apparatus that is the provider of the video data after switching is received. Switching transmission means for transmitting the I frame data for use to the switching request source receiving terminal, and then transmitting the switched video data to the switching request source receiving terminal,
A video data transmitting and receiving system characterized by the following. A plurality of video data providing devices for transmitting video data compressed and encoded using motion compensated inter-frame prediction, a plurality of receiving terminals for receiving and decoding video data from any of the plurality of video data providing devices, The distribution server in a video data transmitting and receiving system comprising a distribution server that relays video data between the video data providing device and the plurality of receiving terminals,
A switch request receiving unit that receives a switch request for switching video data to be received from any of the plurality of receiving terminals,
When the switching request receiving means receives the switching request, the transmission of the video data before switching to the receiving terminal of the switching request is stopped, and the interpolation I obtained from the video data providing apparatus which is the provider of the video data after switching is received. Having transmitted frame data to the receiving terminal of the switching request source, and having a switching transmitting unit transmitting the video data after the switching to the receiving terminal of the switching request source,
A distribution server characterized by the above-mentioned. A plurality of video data providing devices for transmitting video data compressed and encoded using motion compensated inter-frame prediction, a plurality of receiving terminals for receiving and decoding video data from any of the plurality of video data providing devices, The video data providing device in a video data transmitting and receiving system comprising a video data providing device and a distribution server that relays video data between the plurality of receiving terminals,
A first encoding unit that generates video data by performing an intra-frame encoding process or an inter-frame encoding process on each of a plurality of frames forming the moving image data;
In parallel with the encoding process by the first encoding unit, a second encoding unit that generates an interpolation I frame data by performing an intra-frame encoding process on each of the plurality of frames constituting the moving image data;
Transmitting the video data to the distribution server, and when any of the plurality of receiving terminals requests to switch the video data to be received to the video data, the interpolation I frame data for at least one frame; Before switching, transmitting means for transmitting to the receiving device of the previous request via the distribution server,
A video data providing device characterized by the following. An encoder for compressing and encoding moving image data using motion compensation inter-frame prediction,
First encoding means for generating encoded video data by performing an intra-frame encoding process or an inter-frame encoding process on each of a plurality of frames constituting the moving image data;
In parallel with the encoding process by the first encoding unit, an intra-frame encoding process for each of the plurality of frames constituting the moving image data to generate interpolation I-frame data; ,
An encoder characterized by the above. An encoder for compressing and encoding moving image data using motion compensation inter-frame prediction,
First encoding means for performing an intra-frame encoding process on a frame constituting moving image data to generate intra-frame encoded video data;
A second encoding unit that generates inter-frame encoded video data by performing an inter-frame encoding process on a frame configuring the moving image data;
Coded video data generation means for generating coded video data from intra-frame coded video data generated by the first coding means and inter-frame coded video data generated by the second coding means;
Interpolating data generating means for generating I frame data for interpolation from the intra-coded video data generated by the first coding means;
An encoder characterized by the above. In a video data transmitting and receiving system including a transmitting device that transmits video data compressed and encoded using motion compensation inter-frame prediction and a plurality of receiving terminals that receive and decode the video data, the transmitting device and the plurality of A method of transmitting and receiving video data with one of the receiving terminals,
In the transmitting device, a first encoding step of generating the video data by performing an intra-frame encoding process or an inter-frame encoding process for each of a plurality of frames constituting moving image data;
A second encoding step of performing in-frame encoding processing on each of the plurality of frames constituting the moving image data to generate interpolation I frame data in parallel with the first encoding step in the transmitting device;
In the transmitting device, a video data transmitting step of transmitting the video data to a receiving terminal,
In the transmitting device, a transmission interruption step of interrupting transmission of the video data to the reception terminal,
In the transmitting device, an interpolation data transmission step of transmitting the interpolation I frame data for at least one frame to the reception terminal;
In the receiving terminal, an interpolation data decoding step of decoding the correction I frame data;
In the transmitting device, a video data retransmission step of restarting transmission of the video data to the receiving terminal,
In the receiving terminal, having data obtained as a result of executing the interpolation data decoding step as reference frame data, a video data decoding step of decoding the video data after restarting,
A video data transmitting and receiving method characterized by the above-mentioned. Video data transmission method of the transmission side apparatus in a video data transmission / reception system including a transmission side apparatus transmitting video data compressed and encoded using motion compensation inter-frame prediction and a plurality of receiving terminals for receiving and decoding the video data Is a program for causing a computer to execute, the video data transmission method,
A first encoding step of generating the video data by performing an intra-frame encoding process or an inter-frame encoding process on each of a plurality of frames constituting moving image data;
In parallel with the first encoding step, a second encoding step of performing intra-frame encoding processing on each of a plurality of frames constituting the moving image data to generate interpolation I-frame data;
A video data transmitting step of transmitting the video data to a receiving terminal,
A transmission suspending step of suspending transmission of the video data to the receiving terminal,
An interpolation data transmitting step of transmitting at least one frame of the interpolation I frame data to the receiving terminal;
A video data retransmission step of restarting transmission of the video data to the receiving terminal.

Images