JP4373730B2 - Video data transmission apparatus, video data transmission / reception system, and method thereof - Google Patents

Description

  The present invention is a video data transmission / reception system composed of a transmission side and a reception side, wherein the transmission side transmits video data compressed and encoded using motion compensation interframe prediction to the reception side. The present invention relates to a video data transmission / reception system in which the video data is decoded and displayed on a screen.

  In recent years, with the spread of broadband networks such as ADSL and FTTH, services that provide video data to user terminals via the Internet are becoming widespread. The provided video data is compressed and encoded in order to reduce the amount of data. As a compression encoding method, a method defined by a standard such as MPEG-4 is generally used. In this method, the amount of data is compressed by combining intra-frame coding / inter-frame coding and motion compensation prediction.

  One type of such video data providing service is a broadcast-type service (where the transmission side device delivers the same video data (broadcast video data) to a plurality of user terminals in the same time zone according to a predetermined time table). . Data distribution in the broadcast type service is performed by broadcast or multicast as in one-to-many communication. In this method, since it is not necessary to perform individual processing for each user, the number of transmission destination user terminals can be increased without increasing the load on the transmission side device.

  However, as one of service diversification, broadcast video data is not only unilaterally distributed from the transmission side device to the user terminal, but also other video data is distributed in response to a specific request from each user terminal. Services (on-demand data distribution) are also available.

  For example, a service may be considered in which past data relating to a specific player is appropriately transmitted in response to a request from a user terminal while a sports program is being transmitted as broadcast video data.

  As described above, when on-demand video data is transmitted to a specific user terminal while broadcast video data is being transmitted, there are several methods for handling the broadcast video data.

One is to continue to transmit broadcast video data to user terminals that are transmitting on-demand video data. In the case of this method, the user terminal needs a communication band that can receive two types of video data at the same time, and the decoding process of the received video data also involves additional processing such as selective decoding and reproduction. The load becomes heavy. Therefore, during transmission of on-demand video data, a method of interrupting transmission of broadcast video data to the user terminal and transmitting only on-demand video data is often employed.
JP 09-018881 A

  However, for broadcast video data whose transmission is temporarily interrupted for reasons such as the above-mentioned on-demand video data transmission, the following problems may occur immediately after the transmission is resumed. That is, the user terminal cannot decode the broadcast video data immediately after the restart, and a time zone in which the video is not displayed on the screen occurs. This is due to the compression coding method (motion compensation interframe prediction) of video data. Hereinafter, how these problems occur will be described with reference to the drawings.

  FIG. 12 is a schematic diagram showing how the video data is received by the user terminal when broadcast video data transmission by Internet broadcasting is interrupted and on-demand video data is transmitted. Here, reception of broadcast video data 111 is interrupted at time t0, and on-demand video data 112 is received instead until time t1. Then, reception of broadcast video data is resumed from time t1. Broadcast video data 111 is obtained by compression-coding 30 frames / second video by a method of providing one I frame (intra-frame encoded frame) every 5 seconds. Therefore, one unit (GOP: Group Of Picture) is composed of 150 frames centered on one I frame. Internet broadcasts have a smaller I frame rate than digital broadcasts (30 frames / second and two I frames per second).

  A GOP 1110 in FIG. 12 is a GOP received from the broadcast video data 111 across the time t1. The GOP 1110 is composed of one I frame and a plurality of P and B frames, like other GOPs. Each P frame is inter-frame encoded using the immediately preceding I or P frame as a reference frame and each B frame is encoded using the preceding or following I or P frame as a reference frame.

  Among the GOP 1110, the first three frames including the I frame are not received by the user terminal because the transmission timing overlaps with the on-demand video data. For this reason, the fourth P frame encoded using the first I frame as a reference frame cannot be decoded. Further, the P and B frames that are in and after the fourth P frame cannot be decoded. As a result, the image of the portion corresponding to the P and B frames after the fourth frame is not correctly displayed on the screen. This state continues for a maximum of 149 frames (a little less than 5 seconds) in the case of broadcast video data encoded by a method in which one I frame is provided every 5 seconds at 30 frames / second (2 per second). In video data encoded with one I frame, it is only 14 frames (less than 0.5 seconds)).

  In order to prevent this, (1) the transmission of broadcast video data is continued even during on-demand video data transmission, and the user terminal decodes both video data in parallel, or (2) on the transmission side device side, The transmission timing of the broadcast video data is shifted only for the user terminal, or the GOP is reorganized and re-encoded in the form of the I-frame as the frame immediately after the restart, so that the I-frame data is always transmitted when transmission is restarted. Although it is conceivable to make it receive, in (1) the bandwidth needs to be expanded and the processing load on the user terminal is doubled. In (2), the transmitting device receives the on-demand video data. Therefore, it is necessary to distinguish the user terminal from other user terminals and perform separate processing, which increases the load on the transmission side device. This increase in load becomes heavier as the number of user terminals that simultaneously receive on-demand video data increases, which may lead to a situation where the number of users who can simultaneously broadcast is suppressed.

  In view of the above-described problems, the present invention can be applied to either the transmission side or the reception side when a situation occurs in which video data encoded using motion compensated prediction has to be decoded from the middle of a GOP. Therefore, a video data transmission / reception system that allows the receiving user terminal to correctly decode and display the broadcast video data from the portion immediately after the resumption of transmission (in the middle of the GOP) without significantly increasing the processing load. The purpose is to provide.

  In order to achieve the above object, a video data transmission / reception system according to the present invention includes a transmission-side device that transmits video data that has been compression-encoded using motion-compensated interframe prediction, and a plurality of receptions that receive and decode the video data. A video data transmission / reception system comprising a terminal, wherein the transmission side device generates the video data by performing intra-frame coding processing or inter-frame coding processing on each of a plurality of frames constituting moving image data. In parallel with the encoding process by the first encoding means and the first encoding means, the second encoding means for generating the I-frame data for interpolation by intra-frame encoding processing the frames constituting the moving image data; Transmitting means for transmitting the video data to the plurality of receiving terminals, and the transmitting means transmits the video to any one of the plurality of receiving terminals. When resuming transmission of data once interrupted, transmission of the I-frame data for interpolation at least one frame to the receiving terminal prior to resuming transmission of the video data, the receiving terminal transmits the transmitting means When transmission of the video data once interrupted by is resumed, the interpolating I frame data transmitted from the transmitting means is first received and decoded, and the decoding result data is used as reference frame data after the resumption. The video data is decoded.

  In such a video data transmission / reception system, when the transmission of video data that has been once stopped is resumed, if the frame data immediately after the restart is an inter-frame encoded frame (P frame or B frame), Instead of data, I-frame data for interpolation, which has been generated by separately intra-coding the same frame, is transmitted to the user terminal side. The user terminal first decodes the interpolation I frame data, and further decodes the subsequent frame data by referring to the decoding result of the interpolation I frame data as reference frame data. Therefore, since decoding can be performed correctly immediately after resumption (in the middle of GOP), video data display is not interrupted. Furthermore, the processing load on the user terminal device does not increase, and the transmission side device also only has a load for generating I frame data for interpolation and storing several frames. Even in a system in which video data is frequently interrupted and resumed, the processing load for this does not lead to a situation where the number of users who can simultaneously broadcast is suppressed.

  Note that the above-described effect can be realized by a transmission-side device in such a system, that is, a video data transmission device alone. The above effect can also be realized by a video data transmission method executed in the system having the above configuration or a program that causes a computer to execute an operation as a device having such a configuration.

  As is apparent from the above description, the video data transmission / reception system of the present invention includes a transmission-side apparatus that transmits video data that has been compression-encoded using motion-compensated interframe prediction, and a plurality of receptions that receive and decode the video data. A video data transmission / reception system comprising a terminal. The transmission side device includes a first encoding unit that generates the video data by performing an intra-frame encoding process or an inter-frame encoding process for each of a plurality of frames constituting the moving image data, and a first encoding unit. In parallel with the encoding process, a second encoding unit that generates an I-frame data for interpolation by intra-frame encoding a frame constituting the moving image data, and transmitting the video data to the plurality of receiving terminals. And transmitting means, and when the transmission of the video data to any one of the plurality of receiving terminals is interrupted and then restarted, prior to restarting the transmission of the video data, the transmission means Frame data is transmitted to the receiving terminal for at least one frame, and the receiving terminal transmits the video data that has been temporarily interrupted by the transmitting means. When the transmission of the data is resumed, first, the I frame data for interpolation transmitted from the transmission means is received and decoded, and the video data after the restart is decoded using the decoded result data as reference frame data. It is characterized by this.

  In such a video data transmission / reception system, when the transmission of video data that has been once stopped is resumed, if the frame data immediately after the restart is an inter-frame encoded frame (P frame or B frame), Instead of data, I-frame data for interpolation, which has been generated by separately intra-coding the same frame, is transmitted to the user terminal side. The user terminal first decodes the interpolation I frame data, and further decodes the subsequent frame data by referring to the decoding result of the interpolation I frame data as reference frame data. Therefore, since decoding can be performed correctly immediately after resumption (in the middle of GOP), video data display is not interrupted. Furthermore, the processing load on the user terminal device does not increase, and the transmission side device also only has a load for generating I frame data for interpolation and storing several frames. Even in a system in which video data transmission is frequently interrupted and resumed, the processing load does not cause a situation where the number of users who can simultaneously broadcast is suppressed.

  Note that the above-described effect can be realized by a transmission-side device in such a system, that is, a video data transmission device alone.

  Such a transmission apparatus further includes option data transmission means for transmitting option video data to any of the plurality of receiving terminals in parallel with the transmission of the video data, and the plurality of receptions from the transmission means. The interruption of transmission of the video data to any one of the terminals may be caused by interrupted execution of the optional video data transmission by the option data transmission means for any one of the receiving terminals.

  In the case of this configuration, a service is provided in which the same video data is simultaneously transmitted to a large number of receiving terminals in a broadcast image, while another video data (optional video data) is interruptedly transmitted to a specific receiving terminal. The above-described effects can also be realized in the existing video data transmission / reception system. Examples of option video data include on-demand video data and CM video data.

In addition, the option data transmission unit includes an information collection unit that collects information about user preferences in advance from at least some of the plurality of receiving terminals,
The content of the optional video data to be transmitted may be selected based on the information collected by the information collecting means.

  According to this configuration, the above-described effect can be realized even in a video data transmission / reception system that provides a service in which optional video data is transmitted under the initiative of the configuration on the video data providing side.

  The transmission means includes broadcast transmission means for broadcasting the same data to a plurality of transmission destinations, and individual transmission means for transmitting individual data to individual transmission destinations. The broadcast transmission unit may be used, the individual transmission unit may be used for the interpolation I frame data, and the option data transmission unit may individually transmit the option video data. In this case, since video data is broadcast, the processing load on the transmission side apparatus does not increase even if the number of receiving terminals increases. Furthermore, the transmission means includes a switching means for excluding the receiving terminal to which the interpolation I frame data or the option video data is transmitted from the video data transmission target by the broadcast transmission means; In this case, since the receiving terminal does not receive broadcast video data and option video data or I-frame data for interpolation at the same time, it is possible to suppress an increase in network load and processing load on the receiving terminal. Further, the option data transmitting means includes insertion means for transmitting the second option data in the middle of transmission of the first option data, and first option data after the transmission of the second option data is completed. And a third encoding means for generating corresponding option data interpolation I-frame data for at least one frame from the transmission restart position of the first option data before transmission of the first option data is resumed, When the transmission of the first option data is resumed after the transmission of the second option data is completed, the option data transmission means prior to restarting the transmission of the first option data, the option data interpolation I If the configuration is such that the frame data is transmitted to the receiving terminal, an optional video is displayed in the video data. Not only the pattern to insert the data, there is also the pattern to be inserted another option video data in optional video data, displaying the first option image data will not be interrupted immediately after transmission resumption.

  The first encoding unit and the second encoding unit may be realized by separate encoders.

  According to this configuration, there is no need to provide a special encoder that performs encoding processing based on normal motion compensation interframe prediction and intraframe encoding processing for all frames in parallel, and two conventional encoders are provided. The above-described effects can be realized using a conventional encoder.

  Note that in order to obtain the above effect, regarding the determination of how many frames of the interpolating I frame are to be transmitted, the transmitting means transmits the interpolating I frame to the receiving terminal before resuming the transmission of the video data. The number of data frames may be determined based on the GOP configuration of the video data, in particular, the appearance frequency of I attribute or P attribute frames.

  A video data transmitting apparatus that transmits video data compressed and encoded using motion compensation inter-frame prediction to a plurality of receiving terminals, and performs intra-frame coding processing on frames constituting moving image data First encoding means for generating inner encoded video data; second encoding means for generating inter-frame encoded video data by performing inter-frame encoding processing on frames constituting moving image data; Video data generating means for generating the video data from intra-frame encoded video data generated by the encoding means and inter-frame encoded video data generated by the second encoding means; and the plurality of receptions of the video data Transmitting means for transmitting to a terminal, wherein the transmitting means temporarily interrupts transmission of the video data to any one of the plurality of receiving terminals. When restarting later, prior to restarting transmission of the video data, at least one frame of the intra-frame encoded video data generated by the first encoding means is transmitted to the receiving terminal as I-frame data for interpolation. The same effect as described above can also be obtained by the video data transmitting apparatus characterized by the above. In that case, the intra-frame encoded data generated by the first encoding means is used as the I-frame data for interpolation.

  Also, the video data transmission / reception system of the present invention receives video data from any of a plurality of video data providing devices that transmit video data compressed and encoded using motion compensated interframe prediction and the plurality of video data providing devices. A video data transmission / reception system comprising a plurality of receiving terminals for decoding and decoding, and a distribution server for relaying video data between the plurality of video data providing devices and the plurality of receiving terminals. The video data providing device includes first encoding means for generating video data by performing intra-frame encoding processing or inter-frame encoding processing for each of a plurality of frames constituting the moving image data, In parallel with the encoding process by the encoding means, a second encoding means for generating an I-frame data for interpolation by intra-frame encoding each of a plurality of frames constituting the moving image data, The distribution server receives a switching request receiving unit that receives a switching request for switching the video data to be received from any of the plurality of receiving terminals, and receives the switching request when the switching request receiving unit receives the switching request. The transmission of video data before switching to the terminal is stopped, and then the I-frame for interpolation obtained from the video data providing apparatus that is the source of the video data after switching The data after transmitted to the switch requesting receiving terminal, having a switching transmitting means for transmitting the video data after the switch to the switch requesting the receiving terminal can also be realized by a configuration, wherein.

  According to such a video data transmission / reception system, in a video data transmission / reception system that provides a service of transmitting a plurality of types of broadcast video data in a selectable form to a large number of reception terminals in a broadcast image, the reception terminal side Broadcast video data can be correctly decoded even immediately after the switching of the broadcast video data (in the middle of the GOP), and the processing load on both the transmission side device and the reception terminal does not increase significantly.

  An encoder that compresses and encodes moving image data using motion-compensated inter-frame prediction, and encodes video by performing intra-frame encoding processing or inter-frame encoding processing on each of a plurality of frames constituting the moving image data. In parallel with the first encoding means for generating data and the encoding processing by the first encoding means, intra-frame encoding processing is performed on each of the plurality of frames constituting the moving image data to generate I-frame data for interpolation. According to the encoder having the second encoding means, the encoded data of the I frame corresponding to the reference frame is not correctly transmitted to the apparatus on the decoding side in general. By using I-frame data for interpolation instead of the I-frame data, the subsequent I frame and this as a reference frame Frame can be correctly decoded. If this encoder is applied to a system that transmits and receives video data in a broadcast image, the above effect can be realized.

  In addition, the above effect is an encoder that compresses and encodes moving image data using motion compensation inter-frame prediction, and generates intra-frame encoded video data by performing intra-frame encoding processing on the frames that make up the moving image data. First encoding means, second encoding means for generating inter-frame encoded video data by performing inter-frame encoding processing on frames constituting moving image data, and a frame generated by the first encoding means Encoded video data generating means for generating encoded video data from inner encoded video data and interframe encoded video data generated by the second encoding means, and intra-frame code generated by the first encoding means It can also be realized by an encoder having interpolation data generation means for generating I frame data for interpolation from the converted video data.

  The effects described so far can also be realized by a video data transmission / reception method executed by the system having the above configuration, or a program that causes a computer to execute an operation as a device having such a configuration.

  Hereinafter, embodiments of the video data transmission / reception system of the present invention will be described with reference to the drawings.

(Overview)
FIG. 1 is a diagram showing a configuration of a video data transmission / reception system 1 according to the present embodiment. The video data transmission / reception system 1 transmits video data in broadcast format from the video data transmission source to the user terminal 18 who desires to receive video data via the Internet N. The transmission in the broadcast format means that the same video data is simultaneously transmitted to all user terminals that wish to receive according to a time table that is predetermined and notified to the user. The video data transmitted in this broadcast format is hereinafter referred to as “broadcast video data”. That is, the “broadcast” in the broadcast video data here does not mean that transmission / reception is performed by a broadcast wave.

  Further, video data transmitted / received by the video data transmission / reception system 1 may be transmitted only when a request is issued to a specific user terminal that has issued a request, separately from broadcast video data. Say “video data”. During transmission of on-demand video data, transmission of broadcast video data to the user terminal is interrupted (broadcast video data continues to be transmitted to other user terminals). Then, when transmission / reception of on-demand video data is completed, transmission of broadcast video data to the user terminal is resumed.

  The video data transmission / reception system 1 enables the user terminal to correctly decode and reproduce the broadcast video data immediately after the restart when the transmission of the broadcast video data is restarted as described above. The frame data of the B frame is characterized in that the frame is replaced with I frame data (interpolation I frame) that has been separately intra-coded and transmitted to the user terminal.

  Hereinafter, a specific example of transmission / reception of video data including this I-frame data for interpolation will be described with reference to the drawings.

  In FIG. 2, the user terminal receives on-demand video data 23 at a timing across two GOPs 21 and 22 constituting the broadcast video data 20. The last frame of the on-demand video data 23 overlaps with the second frame 222 in the GOP 22. Of the broadcast video data 20, frames 223 and 224 are located immediately after the restart. A frame 223 is a B frame, and a reference frame at the time of decoding is a leading I frame 221 of the GOP 22 and a subsequent P frame 224. The frame 224 is a P frame, and the reference frame at the time of decoding is an I frame 221.

  In order to decode the frames 223 and 224, it is necessary to refer to the I frame 221. If the frame 224 cannot be decoded correctly, the subsequent frame 225 and the subsequent frames cannot be decoded, but the user terminal 18 receives the I frame 221. Not done. Therefore, when resuming transmission of the broadcast video data 20 to the user terminal 18, the material selection device 15 does not transmit the first two frames 223 and 224 that cannot be correctly decoded, but instead, the corresponding I-frame data for interpolation 231 and 232 are transmitted. Send.

The user terminal 18 decodes the interpolated I frame data 231 and 232 corresponding to the frames 223 and 224 and displays them on the screen. Since the frame 225 and subsequent frames can be decoded with reference to the decoding result of the frame 232 and the like, it is not necessary to replace the I frame for interpolation.
(Configuration of each part)
Next, the configuration of the video data transmission / reception system 1 that performs the above processing will be described.

  As shown in FIG. 1, the video data transmission / reception system 1 has a configuration in which a plurality of user terminals 18 that are components on the reception side and a configuration on the transmission side are connected via the Internet N. It has a video camera 11, an editing / broadcasting material server 12, a switcher 13, an encoder 14, a material selection device 15, an on-demand material server 16, and a distribution server 17.

  The video camera 11 provides relay video data that is a kind of video data. This is a device that converts an optical image incident on an optical lens into an electrical signal. Video data is uncompressed or compressed in a format such as DVCPRO and consumer DV, and then passed through signal lines such as SDI, SDTI, IEEE 1394, etc. Output to the switcher 13.

  The editing / broadcasting material server 12 provides video data of a “program” edited by a broadcasting station as a kind of broadcast video data. In addition to the edited program, video data and audio data used as material Holding. These data are stored in a hard disk, a magnetic tape, etc., and are output to the switcher 13 as needed.

  The switcher 13 is located in the broadcasting center, edits video data acquired from the video camera 11 and the editing / broadcasting material server 12 into broadcasting data, and broadcasts the broadcasting data at a timing according to a predetermined time table. For output to the encoder 14.

  The encoder 14 compresses and encodes broadcast data output from the switcher 13 in a format suitable for transmission over the Internet (here, “MPEG-4”) to form broadcast video data, and outputs the broadcast video data to the material selection device 15. . If the broadcast data has already been encoded at the time of output from the switcher 13 and cannot be input to the encoder 14 as it is, it is once decoded by a decoder (not shown) and then input to the encoder 14. Further, the encoder 14 separately encodes each frame of the broadcast data and outputs the result to the material selection device 15 as interpolation I frame data.

  That is, the encoder 14 performs two types of different encoding processes on the input broadcast data, and outputs the results as broadcast video data and interpolation I frame data.

  FIG. 3 shows the configuration of the encoder 14. A difference from a general encoder is that two combinations of a DCT processing unit and a quantization processing unit are provided. The DCT unit 141 and the quantization unit 142 of the first system encode all the frames of the input broadcast data by intra-frame encoding and output as I-frame data for interpolation. The DCT unit 143 and the quantization unit 144 of the second system output the broadcast data as intra-frame coding or inter-frame coding as broadcast video data. The same processing as that of the conversion unit is performed. The encoded data output from each of the first system and the second system is output to the material selection device 15 while synchronizing in units of frames.

  The on-demand material server 16 holds on-demand video data and provides it to the material selection device 15 in response to a user request. On-demand video data is compressed and encoded in the MPEG-4 format and stored in the same manner as broadcast video data.

  The material selection device 15 receives broadcast video data and interpolation I frame data from the encoder 14, and on-demand video data from the on-demand material server 16, and sends them to the distribution server 17. Of these, the I-frame data for interpolation is output only for the necessary frames at the necessary timing. The processing related to the interpolation I frame data by the material selection device 15 will be described in more detail later.

  The distribution server 17 transmits the three types of video data (broadcast video data, on-demand video data, and interpolation I frame data) output from the material selection device 15 to the user terminal 18 via the Internet N, respectively.

  In the case of broadcast video data, the distribution server 17 transmits to all user terminals that are “broadcast targets” at the same time in accordance with the time table. The broadcast target user terminal is a user terminal that has previously issued a request for transmission of broadcast video data to the distribution server 17.

  In the case of on-demand video data, when receiving the on-demand video data from the material selection device 15, the distribution server 17 interrupts the transmission of the broadcast video data to the requesting user terminal and transmits the on-demand video data.

  Interpolation I frame data is transmitted to the user terminal requesting the on-demand video data when the transmission of the on-demand video data is completed. Then, after the transmission of the interpolation I frame data is completed, the transmission of the broadcast video data to the user terminal is resumed.

The user terminal 18 is a device that transmits a user request to the distribution server 17, decodes video data sent from the distribution server 17 in response to the request, and displays the screen. In particular, when receiving broadcast video data again after receiving on-demand video data, the interpolated I frame data is received, decoded and displayed on the screen, and then the decoded interpolated I frame data is referred to as reference frame data. Then, the frame of broadcast video data received thereafter is decoded.
(Description of main components)
Hereinafter, the main components (the material selection device 15, the distribution server 17, and the user terminal 18) of the video data transmission / reception system 1 described above will be described in more detail.
(Detailed configuration)
Hereinafter, the three main components in the video data transmission / reception system 1 according to the present embodiment will be described in more detail with reference to FIG. In particular, processing related to I frame data for interpolation will be described.

  FIG. 4 is a block diagram showing the configuration of three main components.

(Distribution server 17)
The distribution server 17 includes a request reception unit 171, a user management unit 172, a video request unit 173, a video reception unit 174, and a video transmission unit 175. The processing contents of each component will be described.

  The request receiving unit 171 receives a request regarding video data transmission (broadcast video data transmission start / end request, on-demand video data transmission request) from the user terminal 18 via the Internet N, and outputs the request to the user management unit 172. . The request includes identification information (such as an IP address) of the requesting user terminal and identification information of the requested video data.

  The user management unit 172 manages information related to the user terminal that is the video data transmission destination. Management information indicating the identification information of user terminals to which data is to be transmitted, the type of data to be transmitted to each of the user terminals, and identification information (in the case of on-demand video data) is held, and this is received as the request reception unit 171. It is updated according to the content of the request from the user that is sent via. When a transmission request for on-demand video data is received, the content is output to the video request unit 173.

  When the video request unit 173 receives a transmission request regarding on-demand video data from the user management unit 172, the video request unit 173 sends the request to the material selection device 15 and instructs to read the on-demand video data.

  The video receiving unit 174 receives various types of video data from the material selection device 15 and sends them to the video transmission unit 175. Of the video data sent from the material selection device 15, on-demand video data and interpolation I frame data are added with the identification information of the requesting user terminal included in the above request.

  The video transmission unit 175 transmits various video data transmitted from the video reception unit 174 to the user terminal 18 in accordance with management information held by the user management unit 172. When the transmission of the on-demand video data and the I-frame data for interpolation is completed, the identification information of the destination user terminal is sent to the user management unit 172, and the content of the management information for the user terminal is set to “on-demand video transmission”. The information shown is updated to the one indicating “broadcast video data transmission”.

(Material selection device 15)
The material selection device 15 includes a transmission / reception unit 151, a material determination unit 152, an interpolation I frame buffer 153, a frame determination unit 154, and an on-demand data reading unit 155.

  The transmission / reception unit 151 receives a transmission request regarding on-demand video data from the distribution server 17, and outputs the contents (video data identification information, requesting user terminal identification information) to the material determination unit 152. In addition, various video data output via the material determination unit 152 is transmitted to the distribution server 17.

  The on-demand data reading unit 155 reads the requested on-demand video data from the on-demand material server 16 and outputs it to the material determination unit 152. Further, when outputting the final frame of the on-demand video data, information indicating the end is added.

  The frame determination unit 154 receives the broadcast video data output from the encoder 14 and outputs it to the material determination unit 152. At that time, for each piece of frame data constituting the video data, the attribute information is analyzed and the frame type (I, P, B) and the serial number of the frame (the order in the entire video data or the order in the GOP to which it belongs) Is output to the material determination unit 152 together with the frame data. These pieces of information are referred to for determining whether or not to output I-frame data for interpolation, which will be described later, and for specifying I-frame data for interpolation to be output. It should be noted that the frame serial number may be counted by the frame determination unit 154 itself.

  The interpolation I frame buffer 153 holds several frames of interpolation I frame data output from the encoder 14. Further, a counter is held to count the serial number of each I-frame data for interpolation, and the serial number is added to each data (counted by a method corresponding to the serial number obtained by the frame determination unit 154). When an output request for interpolation I frame data is received from the material determination unit 152, the interpolation I frame data corresponding to the serial number added to the request is output to the material determination unit 152. Note that the I frame data for interpolation exceeding the predetermined number is deleted from the oldest.

  The material determination unit 152 acquires broadcast video data, on-demand video data, and I-frame data for interpolation, and appropriately outputs them to the transmission / reception unit 151 for transmission to the distribution server 17. The processing of the material determination unit 152 at the time of transmission of on-demand video data and when the transmission of the on-demand video data is completed and the broadcast video data transmission is resumed will be described below with reference to the drawings.

  FIG. 5 is a flowchart showing the flow of processing by the material determination unit 152.

  When the on-demand video data is transmitted, the material determination unit 152 outputs the requested on-demand video data from the on-demand data reading unit 155 to the distribution server 17 in parallel with the broadcast video data. (S501).

  When the end frame of the on-demand video data is output from the on-demand data reading unit 155 (S502: Yes), the material determination unit 152 sets the final frame of the on-demand video data among the frames of the broadcast video data. Subsequently, serial numbers, attribute information, and the like are acquired for a predetermined number of frames to be transmitted (S503), and whether or not replacement with I-frame data for interpolation is necessary is determined based on these. The number of frames of data to be determined depends on the frame configuration at the time of broadcasting video data encoding. As shown in FIG. 2, the configuration of the broadcast video data handled by the system 1 is “I or P frames appear at a rate of 1 in 3”, so it is determined whether or not replacement is required for 3 frames. . Alternatively, a frame until the appearance of the next I frame data (start of the next GOP) is set as a determination target. If the frame structure (reference relationship) is irregular, it is preferable to repeat until the I frame of the next GOP appears.

  If the type of the determination target frame is “I” (S504: Yes), the frame and the subsequent frame can be correctly decoded by the user terminal 18; therefore, the replacement with the interpolation I frame data is unnecessary. The distribution server 17 is notified of this, and the broadcast video data is transmitted to the user terminal 18 (S508). Further, the determination of necessity of replacement with the interpolation I frame data is terminated.

  When the determination target frame is a P or B frame (S504: No), the material determination unit 152 checks whether the reference frame of the frame is transmitted to the user terminal 18 (in the case of the B frame, 2). Among the two reference frames, the preceding reference frame) (S505). Specifically, the serial number of the reference frame of the frame is compared with the serial number of the frame data of the broadcast video data corresponding to the last frame in the on-demand video data, and if the former value is greater than the latter value, The reference frame has reached the user terminal 18. If the reference frame is received by the user terminal 18 (S505: Yes), since the broadcast video data frame can be decoded, the frame data is not transferred to the user terminal 18 without being replaced with the interpolation I frame data. The distribution server 17 is instructed to transmit (S507).

  The serial number of the reference frame can also be obtained from the serial number of the determination target frame itself and the frame structure of the broadcast video data. In the case of the broadcast video data of this example having the frame structure shown in FIG. 2, the reference frame of the P frame is the I or P frame that is three consecutive numbers before itself. In the case of the B frame whose own serial number is a multiple of 3 (frame 223 in FIG. 2), the reference frame is the serial number and is the serial number of the previous I or P frame (frame 221 in FIG. 2). The reference frame of the next P frame (frame 224 in FIG. 2) whose serial number is “a multiple of 3 minus 1” (frame 222 in FIG. 2) is the previous I frame with the serial number. Or it becomes the P frame (frame 221 in FIG. 2) and the P frame (frame 224 in FIG. 2) that is the next P frame by the serial number.

  On the other hand, if the reference frame has not been transmitted (S505: No), it cannot be decoded even if the data of the broadcast video data frame is sent to the user terminal 18, and therefore corresponding interpolation (having the same serial number) is performed. The I frame data is read from the interpolation I frame buffer 153, sent to the distribution server 17, and sent to the user terminal 18 (S506). Note that while the interpolating I frame data is being transmitted to the user terminal 18, it is necessary to transmit the broadcast video data to the other user terminals, so that the material selecting device 15 sends the interpolating I frame data to the distribution server 17. Broadcast video data is also transmitted in parallel with the frame data.

  The processes in steps S503 to S507 are repeated until a predetermined number of frames after the restart are determined or until I frame data of the next GOP appears (S509). And after the process shown in this figure is complete | finished, the broadcast video data after a check object frame is transmitted unconditionally.

  In the above description, it is assumed that the material determination unit 152 outputs the broadcast video data and the on-demand video data one frame at a time in parallel to the distribution server 17 so that the broadcast video data is converted into an I frame for interpolation one frame at a time. Although it was decided whether or not to replace, based on information such as the frame structure at the time of encoding and the number of frames of on-demand video data, at the stage of starting transmission of on-demand video data, The “number of frames to be replaced and the serial number” can be determined in advance. Specifically, when transmission of on-demand video data is started, transmission is resumed by obtaining the serial number of the broadcast video data frame corresponding to the first frame of on-demand video data and adding the number of frames of on-demand video data to this. The frame sequence number of the broadcast video data at that time can be calculated. The frame structure of the broadcast video data is determined in advance as shown in FIG. 2. As described above, a frame whose serial number is 3n + 1 (n is an integer of 0 or more) is an I or P frame. (I or P is determined by the number of frames of 1 GOP), and frames having other serial numbers are B frames. Therefore, if the serial number of the broadcast video data frame at the time of restarting transmission is known, the type of the frame is easy. Can be determined.

  If the serial number of the frame to be replaced with the interpolation I frame data can be predicted in advance as described above, the interpolation I frame data may be generated only for the predicted replacement frame (in the above description, all the frames are described). Generated). By doing so, the processing load for generating the interpolation I frame data can be reduced. This is particularly effective in a system in which there are not many on-demand video data transmission requests.

  In addition, it is also possible to perform a fixed replacement in the form of “unconditionally replace n frames immediately after restart with an I frame for interpolation (without determination processing) (n is the frequency of occurrence of I or P frames)”. Good. In such a case, the burden of determining whether replacement is necessary is eliminated. As shown in FIG. 2, the frame structure of the broadcast video data in this embodiment is “I or P type frame data appears every 3 frames (frequency of appearance of I or P frame = 3), and P frame is The immediately preceding I or P frame is the reference frame, and the B frame is the preceding or following I or P frame. In other words, whatever type of frame the broadcast video data immediately after resumption is, if a maximum of 3 frames (= appearance frequency of I or P frame) is replaced with an I frame for interpolation, all subsequent frames can be decoded without any problem. .

  In addition, regardless of the frequency of occurrence of I or P frames, if the method is to replace with I frame data for interpolation until the first P frame data after restarting, it is only necessary to check the frame type to determine whether or not to replace. Since it is good, the load of determination processing can be reduced. That is, if the frame immediately after resumption is the P type, only this P type frame is replaced with the I frame for interpolation. If the frame immediately after resumption is the B frame, the I I for interpolation is included up to the first subsequent P frame including the B frame. Replace with a frame.

(User terminal 18)
The user terminal 18 receives various video data from the distribution server 17 and decodes and displays the screen. The user terminal 18 includes a request reception unit 181 that receives a request from a user, a request transmission unit 182 that transmits this request to the distribution server 17, a video reception unit 183 that receives video data transmitted from the distribution server 17 in response to the request, It has a decoder 184 that decodes video data and a video display unit 185 that displays the video of the decoding result on the screen.
(Flow of video data transmission / reception processing)
Next, regarding the video data transmission / reception processing in the video data transmission / reception system 1, the exchange of various data between the main components described above and the flow of processing of each unit will be described below.

  FIG. 6 shows a process in the video data transmission / reception system 1 of the present embodiment in which a certain user terminal 18 interrupts reception of broadcast video data, receives on-demand video data, and then resumes reception of broadcast video data. 7 is a sequence diagram showing the flow of processing and data exchange of the user terminal 18, the distribution server 17, and the material selection device 15 when the process is performed.

  First, an on-demand video data transmission request is sent from the user terminal 18 receiving broadcast video data to the distribution server 17 to start processing (S601). Although not shown in the figure, broadcast video data is also transmitted from the distribution server 17 to other user terminals at this time.

  The request is further sent from the distribution server 17 to the material selection device 15.

  The material selection device 15 that has received the request via the distribution server 17 reads the requested on-demand video data from the on-demand material server 16 and sends it to the distribution server 17. The distribution server 17 interrupts transmission of broadcast video data to the user terminal 18 and transmits on-demand video data (S602). However, during this time, the distribution server 17 continues to transmit broadcast video data to other user terminals.

  When the end of the on-demand video data is detected (S603), the material selection device 15 is transmitted to the user terminal 18 immediately after the completion of the on-demand video data transmission among the broadcast video data being transmitted to the other user terminals. The attribute and reference frame of several frames are checked to determine whether or not it is necessary to transmit I-frame data for interpolation (S604). When the frame type is P or B, and the reference frame of the frame is not received by the user terminal 18 because it overlaps with the on-demand video data, the material selection device 15 stores the interpolation I frame that is held. The one corresponding to the frame is selected and sent to the distribution server 17, and the distribution server 17 transmits the interpolation I frame data to the user terminal 18 following the end data of the on-demand video data (S605).

  The user terminal 18 decodes the I frame data for interpolation and displays it on the screen (S606).

After transmission of the I frame for interpolation (or when transmission of the I frame for interpolation is not necessary), transmission of broadcast-type video data from the distribution server 17 to the user terminal 18 is resumed (S607), and the user terminal 18 Then, the decoding of the resumed broadcast video data is started using the decoding result of the previously received I-frame data for interpolation as the reference frame data (S608).
(Summary)
As is apparent from the above description, in the video data transmission / reception system 1 according to the present embodiment, when transmission of on-demand video data is performed in an interrupted manner while broadcast video data is being transmitted from the transmission side device to the user terminal. Then, before the transmission of the broadcast video data is resumed after the transmission of the on-demand video data is completed, the I-frame data for interpolation is transmitted to the user terminal. The user terminal first decodes and displays the I frame for interpolation, and decodes and displays the I frame for interpolation as a reference frame for several frames of broadcast video data immediately after restart.

  As a result, the user terminal can correctly decode the broadcast video data and display it on the screen immediately after the end of the on-demand video data.

  In the present embodiment, the material selection device 15 and the distribution server 17 are described as separate devices, but it is also possible to implement the present invention by combining these two devices as a single device.

  In addition, as shown in FIG. 2, the present embodiment shows that the appearance frequency of the I or P frame is 3, the P frame refers to the immediately preceding I or P frame, and the B frame refers to the preceding or following I or P frame. The encoded data having the structure “referred to as a frame” and the reference rule is described as an example, but the structure of the encoded data and the reference rule to which the present invention can be applied are not limited to this. Even if the encoded data has any structure and reference rule, it is possible to determine whether or not replacement is necessary when transmission is resumed as long as the reference frame can be specified for each frame of the broadcast video data. Is effective.

  Further, the encoder 14 in the present embodiment has two systems of “DCT to quantization processing mechanism”, one of which performs conventional processing (mixing of intraframe coding and interframe coding), The other is a special encoder that only performs intra-frame coding, but two conventional encoders are prepared, one for generating I-frame data for interpolation and the other for generating broadcast video data. The system may be configured in such a way that each is performed. With this configuration, two encoders are required and the configuration becomes large. Instead, the present invention can be implemented using only a conventional encoder.

  The encoder in the present embodiment is divided into a first system, a second system DCT unit, and a quantization unit. The first system generates I frame data for interpolation, and the second system generates broadcast video data. However, data generation is not limited to this form. For example, a part of I frame data generated by the first system may be used for broadcast video data, and only the P and B frame data may be generated by the second system. That is, for the first frame of the GOP, the first system performs intra-frame encoding, while the second system passes the encoding process, and the intra-frame encoded data output by the first system is interpolated for the frame. Not only as I-frame data but also as part of broadcast video data, it is output to the material selection device, and further output to the inverse quantization unit, and used as reference frame data for subsequent frames. Other frames are encoded in each system as in the present embodiment. In this case, only for I frame data, a process of outputting encoded data output from the first system as broadcast video data is newly required in the SW. This method eliminates the duplication of processing that both systems perform intra-frame coding for the first frame of the GOP, and the load is reduced.

  Further, in the system shown in the embodiment, data transmission from the distribution server to the user terminal is simply performed via the Internet, but various forms of data transmission are conceivable.

  FIG. 7 is a diagram showing a video data transmission / reception system in which video data transmission from a distribution server 717 to a user terminal 718 (generic name for terminals 718a, 718b, 718c, 718d) is performed via a router as an example. . Here, the routers 701 to 703 are multicast routers.

  FIG. 7A shows a situation in which broadcast video data (indicated by a black arrow) is transmitted from the distribution server 717 to the user terminal 718. In FIG. 7B, the terminal 718a of the user terminals issues a transmission request for on-demand video data, and the on-demand video data is transmitted from the distribution server 717 to the user terminal 718a via the routers 701 and 702 accordingly. Indicates the situation. Thereafter, when the on-demand video data transmission is completed, the I-frame data for interpolation (indicated by a white arrow) is transmitted to the user terminal 718a through the same route as the on-demand video data is transmitted in FIG. 7B. Thereafter, as shown in FIG. 7A, the broadcast video data is transmitted again.

  In this system, the contents of processing performed by the material selection device 15 and the distribution server 17 in connection with the transmission of the I-frame for interpolation are the same as those in the system 1 in the embodiment.

  In addition, the present embodiment is a video data transmission / reception system in which on-demand video data is transmitted in an interrupted manner during the transmission of broadcast video data. However, the system configuration and video data to which the present invention is applied are described. The types are not limited to those shown in this embodiment. An object of the present invention is a system in which main video data compressed and encoded using inter-frame motion compensation prediction is transmitted from a transmission source device to a reception side device. It is a system that the reception of the may be interrupted. However, this is not just an interruption, but an interruption in the form that the content of the reference frame remaining in the decoding unit of the receiving-side apparatus does not correspond to the frame of the main video data when transmission is resumed.

  Such interruption patterns include “other data is transmitted in the form of being replaced with part of the main video data” as described in the embodiment, and “other data is in the middle of the main video data. It is also conceivable that “a part of main video data (particularly I frame data) is not received due to communication failure”.

  In addition, if a system that transmits video data as a broadcast image is interrupted according to the timetable, when transmission is resumed, transmission is resumed from the point where the frame for the interruption time was skipped instead of resuming from the suspended point. However, in this case, the content of the reference frame at the time of restart does not correspond to the frame of the main video data even if the receiving side apparatus does not decode other data during the interruption period. In this case, the cause of interruption may be communication failure, transmission / reception of data other than video data (character data or still image data), and the like.

  In addition, the system according to the present embodiment is such that on-demand video data is transmitted from the distribution server to the requesting user terminal in response to a request from the user terminal, whereby the broadcast video data is interrupted. There can also be a video data transmission / reception system in which other video data is inserted in the middle of broadcast video data mainly by the transmission side.

  The system shown in FIG. 8 is a video data transmission / reception system 8 in which other video data is inserted in the middle of broadcast video data with the transmission side as the main body. During transmission of broadcast video data to the user terminal 18, the distribution server 817 appropriately transmits other video data (here, CM video data according to each user's hobbies and preferences) to each user terminal. Therefore, the system 8 includes a distribution material selection device 801 that selects CM video data according to the preference for each user. The material selection device 801 for distribution obtains information on hobbies and preferences for each user, which is a reference for selection (user age, gender, products of interest, etc.) from the user terminal 18 via the Internet N using means such as a questionnaire. To do.

  For example, the distribution material selection device 801 outputs CM video data based on a contract between a provider and a sponsor of pay broadcast video data transmitted in a broadcast format to the material selection device 815, Prompt to send to the destination user terminal. On the other hand, the material selection device 815 outputs CM video data for the CM transmission destination user terminal to the distribution server 817 and transmits it to the user terminal 18. When the end of the CM video data is detected, the material selection device 815 transmits the necessary interpolation I frame data to the user terminal 18 and broadcasts in the same procedure as described in the first embodiment. Resume transmission of video data.

As described above, the same effect as that of the system 1 according to the embodiment can be obtained even in a system in which the transmission side apparatus takes the lead in suspending and resuming broadcast video data. Note that the type of video data transmitted by the transmission side apparatus is not limited to CM video data, and may be other video material.
(Modification)
The system 1 in the above embodiment enables broadcast video data to be correctly decoded when transmission / discontinuation of transmission of broadcast video data occurs due to interruption of on-demand video data. In the modified example, when the broadcast video data is switched in a user terminal capable of receiving a plurality of broadcast video data, depending on the type of the frame immediately after the switching in the broadcast video data, the broadcast after switching until the start of the next GOP is performed. This solves the problem that video data cannot be decoded correctly.

  Hereinafter, the system of the modified example will be described. Since there are many configurations in common with the system in the embodiment, different parts will be described in particular.

  In the system 9 shown in FIG. 9, a plurality of broadcast video data providing devices 91a and 91b (cameras and / or material servers, switchers, encoders, material selection devices) are distributed to a distribution server 917 that transmits video data to a user terminal. Broadcast video data is provided from the set. Separate broadcast video data is provided from each of the two providing devices to the distribution server 917 according to a predetermined time table. The distribution server 917 transmits any broadcast video data in response to a request from the user terminal. This means that each of the plurality of broadcast video data providing apparatuses corresponds to one channel, and the user can switch the broadcast video data so as to switch the channel. When the channel (broadcast video data) is switched, the I-frame data for interpolation is transmitted to the user terminal prior to the start of transmission of the broadcast video data after the switching. Thereby, even at the time of switching, the post-switching broadcast video data is immediately reproduced and displayed correctly by the user terminal.

(Constitution)
In the system 9, it is the distribution server 917 and the material selection devices 915a and 915b in the first and second video data providing devices 91a and 91b that perform processing related to transmission of I-frame data for interpolation when switching broadcast video data. . Hereinafter, processing of these components will be described. A description of processes common to the system 1 is omitted.

  The distribution server 917 receives broadcast video data from each of the first and second video data providing devices 91a and 91b, and transmits the broadcast video data to the desired user terminal. When the broadcast video data switching request is received from the user terminal 918 that has received the video data provided by the first video data providing device 91a, the second video data providing device 91b that provides the broadcast video data after switching is received. The material selection device 915b on the side is notified. When the I frame data for interpolation is output from the material selection device 915b, the transmission of the first video data to the user terminal 918 is stopped, the I frame data for interpolation is transmitted, and then the second I frame data is transmitted. Start transmission of broadcast video data. On the other hand, when a notification “interpolation I frame data unnecessary” is sent from the material selection device 915b, the transmission of the first video data to the user terminal 918 is stopped as it is, and the transmission of the second broadcast video data is stopped. Start.

  When the material selection device 915b receives a notification that there is a switching request, the material selection device 915b specifies a frame to be received by the user terminal 918 immediately after switching when switching is performed at this time. The processing after the detection of this frame is the same as the processing in the video data transmission / reception system 1. That is, it is determined whether it is necessary to replace the detected frame and, if necessary, several subsequent frames with the I-frame data for interpolation. If necessary, the I-frame data for interpolation is read from the I-frame buffer and the frame sequence number is read. At the same time, it is output to the distribution server 917. If a frame that does not need to be replaced with the interpolation I frame is detected, a notification “no interpolation I frame is required” is sent to the distribution server 917. In this case, the serial number of the frame determined not to be replaced is also sent to the distribution server 917, and transmission of the second broadcast video data from the data of the frame corresponding to the serial number of the second broadcast video data to the user terminal is started. We will encourage you to do so. In addition, the distribution server 917 grasps the frame serial number in the same manner as the material selection devices 915a and 915b.

(Process flow)
Hereinafter, the flow of the broadcast video data transmission switching process in the video data transmission / reception system 4 will be described with reference to a sequence diagram showing a transition of control between main components and a data flow.

  FIG. 10 is a sequence diagram illustrating control transition and data flow between the two broadcast video data providing apparatuses 91a and 91b, the distribution server 917, and the user terminal 918 in the broadcast video data transmission switching process. Shown here is a sequence when the video data to be transmitted and received is switched from the first broadcast video data to the second broadcast video data.

  First, the user terminal 918 receives a user instruction in a state where the first broadcast video data is transmitted from the first providing device 915a as a supply source via the distribution server 917 (S1001). A request for switching to the second broadcast video data is transmitted to the distribution server 917 (S1002).

  In response to this, the distribution server 917 notifies the second broadcast video data providing apparatus 91b, which is the supply source of the second broadcast video data, of the occurrence of the switching request. On the other hand, the second broadcast video data providing apparatus 91b determines whether or not it is necessary to replace the I-frame data for interpolation with respect to a frame that the user terminal 918 receives immediately after switching (S1003). In this case, I frame data for interpolation is transmitted to the distribution server 917 (S1004). The distribution server 917 stops the transmission of the first broadcast video data to the user terminal 918 and transmits the received I frame data for interpolation. The user terminal 918 receives the I frame data for interpolation, decodes it, and displays it on the screen (S1005). Thereafter, the distribution server 917 transmits the second broadcast video data to the user terminal 918 from a portion that does not require replacement with the interpolation I frame, and the user terminal 918 uses the I frame data for interpolation decoded in step S1005 as a reference frame. Then, the decoding of the second broadcast video data is started (S1006).

  As described above, in the present system 9, a plurality of video data providing devices 91a and 91b are connected to the distribution server 917, and different broadcast video data is provided from each. When there is a video data switching request from the user terminal 918, an interpolation I frame is provided to the user terminal 918 for the frame immediately after switching in the video data after switching. Thereby, even when the broadcast video data received by the user terminal is switched, the broadcast video data after the switching can be decoded and displayed on the screen without interruption.

9 does not include a configuration for on-demand video data transmission / reception, but the system in the embodiment shown in FIG. 1 is combined with this modification to generate an on-demand video. It is also possible to realize a system that transmits / receives interpolating I-frame data at any time of switching of broadcast video data after data transmission / reception.
(Modification 2)
As a service form, another on-demand data (second on-demand data) is transmitted in an interrupted manner during the transmission of certain on-demand data (first on-demand data), and then the first on-demand data is transmitted. It is also possible to resume. In that case, immediately after switching from the second on-demand data to the first on-demand data, the same problem as in the above embodiment may occur. Therefore, the present modification shows a system corresponding to such a change of shape.

  FIG. 11 shows a material selection device 15A in the video data transmission / reception system according to this modification. A component that is a feature of this modification is an on-demand data reading unit 155A in the material selection device 15A. The on-demand data reading unit 155A switches on-demand data to be output to a certain user terminal, and at the time of switching from the second on-demand data after switching to the first on-demand data before switching. Prior to the transmission of the first on-demand data, an interpolation I frame is output.

  As for the I-frame data for interpolation, it is conceivable that the I-frame data for interpolation is generated in advance for all the on-demand data and stored in the on-demand material server 16, but the data amount becomes large and is not realistic. Therefore, before the output of the first on-demand data is restarted, I-frame data for interpolation is generated from several frames immediately after the restart, and this is output.

  The on-demand data reading unit 155A includes a switching unit 1551, an auxiliary decoder 1552, and an auxiliary encoder 1553 as a configuration for that purpose. The on-demand data reading unit 155A performs the same processing as the on-demand data reading unit 155 in the embodiment, but the description thereof is omitted.

  Based on the frame number of the first on-demand data at the time of switching and the total number of frames of the second on-demand data, the switching unit 1551 determines the serial number and frame type for the first on-demand data frame transmitted immediately after the transmission is resumed. judge. First, it is determined whether or not it is necessary to generate I-frame data for interpolation based on the frame type (unnecessary for I-frame, necessary otherwise). If necessary, the number of frames to be replaced with the I-frame for interpolation is further obtained, and the identification information of the first on-demand data, the serial number of the frame immediately after resuming transmission, and the information on the number of frames are sent to the auxiliary decoder 1552 and decoded. Instructs execution of processing. The necessity determination of the I frame for interpolation and the determination of the frame that needs to be replaced may be performed by the same procedure as that performed by the material determination unit 152 in the embodiment.

  Then, the switching unit 1551 outputs the I-frame data for interpolation obtained as a result of the processing performed by the auxiliary decoder 1552 and the auxiliary encoder 1553 in accordance with the above instruction by replacing the frame with the first on-demand data immediately after the transmission is resumed. To do.

  When the auxiliary decoder 1552 receives an instruction from the switching unit 1551, the auxiliary decoder 1552 determines a GOP (GOP including a frame immediately after the restart of transmission) to be decoded from the first on-demand data based on the frame sequence number information. Then, only the GOP is read from the on-demand material server 16 (refer to the identification information at this time). As a determination method, for example, the serial number value is divided by the number of frames per 1 GOP, and the quotient is obtained by rounding up the number. If the quotient is n, the nth GOP from the top of the first on-demand data is obtained.

  Next, when the auxiliary decoder 1552 decodes the read GOP from the beginning to the several frames after the transmission is resumed (a frame that can be replaced with the I frame for interpolation), the decoding result data of the replacement target frame is added with the respective serial numbers. Then, it outputs to the auxiliary encoder 1553 and instructs to generate I-frame data for interpolation from these frames.

The auxiliary encoder 1553 performs intra-frame encoding processing on all three frames received from the auxiliary decoder 1552 and outputs the result together with the frame sequence numbers to the switching unit 1551.
(Other)
In the above embodiment and its modifications, the Internet is cited as a communication path when video data is encoded by an encoder and distributed via a distribution server such as an ISP (Internet Service Provider). Applicable transmission / reception paths are not limited to this, and broadcast radio waves and CATV may be used as other communication means. For example, it is conceivable that main broadcast program data is transmitted / received by broadcast radio waves, and optional on-demand data is transmitted / received via a communication line such as the Internet.

  In the above-described embodiment and its modifications, the present invention is a video data transmission / reception system. However, the present invention can also be realized by executing a program on a computer. In this case, the program is stored in a recording medium. It is good also as being recorded on.

  In the above-described embodiment and its modification, the video data compression coding standard has been described as MPEG-4. However, the present invention performs coding using inter-frame coding and motion-compensated inter-frame prediction. It is effective for the data to be processed in general. For example, video data compressed by another compression encoding method such as MPEG-2 is also an object of the present invention.

  In the above-described embodiment and its modification, it has been described that on-demand video data is always transmitted to the end. However, transmission of on-demand video data may be interrupted halfway due to a request from the user. sell. The present invention can also cope with such a case of an abortion. Specifically, in the above embodiment, the process executed when “the end of on-demand video data is detected” is changed to “when a request for aborting the on-demand video data is issued from the receiving device side” As well.

  The video data transmission / reception system, the video data transmission device and the method according to the present invention can reproduce video data compressed and encoded using motion compensated inter-frame prediction from the middle of a GOP due to an interruption or the like. This has the effect that video data can be correctly decoded and displayed on the screen, and is useful as a video transmission / reception system or the like in which the transmission side device transmits the above video data and the reception side device decodes and displays the screen.

It is a block diagram which shows embodiment of the video data transmission / reception system concerning this invention. It is a schematic diagram which shows the receiving state of the various video data in the embodiment. It is a block diagram which shows the structure of the encoder in the embodiment. It is a block diagram which shows the detailed structure of the raw material selection apparatus in the same embodiment, a delivery server, and a user terminal. It is a flowchart which shows the flow of the necessity determination process for replacement | exchange in the embodiment. It is a sequence diagram which shows the flow of the video data transmission / reception process in the embodiment. It is a figure which shows the structure at the time of using a router for transmission / reception regarding this Embodiment. It is a figure which shows a structure in case transmission side performs transmission / reception mainly by a transmission side regarding this Embodiment. It is a block diagram which shows the structure of the modification regarding the same embodiment. It is a sequence diagram which shows the flow of the video data transmission / reception process in the modification. It is a block diagram which shows the structure of the raw material selection apparatus among the video data transmission / reception systems in another modification regarding the same embodiment. It is a schematic diagram which shows the receiving state of the various video data in a prior art.

Explanation of symbols

1,8,9 Video data transmission / reception system 141 First DCT unit 142 First quantization unit 143 Second DCT unit 144 Second quantization unit 15, 15A Material selection unit 152 Material determination unit 153 Interpolation I frame buffer 154 Frame determination unit 155 , 155A On-demand data reading unit 16 On-demand material server 17 Distribution server 18 User terminal

Claims (13)

A video data transmission / reception system comprising a transmission-side apparatus that transmits video data compressed and encoded using motion-compensated interframe prediction, and a plurality of reception terminals that receive and decode the video data,
The transmitting device is:
A first encoding unit that includes a first DCT unit and a first quantization unit, and generates inter-frame encoded video data of the video data by performing inter-frame encoding processing on a plurality of frames constituting moving image data ; ,
A second DCT unit and a second quantization unit are provided, and in parallel with the encoding processing by the first encoding means, intra-frame encoding processing is performed on the frames constituting the moving image data to generate I-frame data for interpolation. Second encoding means for:
Video data generating means connected to the first encoding means and the second encoding means for forming the video data by combining the interpolated I-frame data and the inter-frame encoded video data; ,
Transmitting means for transmitting the video data and the I-frame data for interpolation to the plurality of receiving terminals,
When the transmission means resumes transmission of the video data to any one of the plurality of receiving terminals once, is a reference frame for a frame transmitted after the transmission resumed among the video data transmitted? If it is determined that the reference frame is not transmitted before the resumption of transmission of the video data, the interpolation I corresponding to the frame for which it is determined that the reference frame is not transmitted. Sending frame data to the receiving terminal;
When the transmission of the video data once interrupted by the transmitting means is resumed, the receiving terminal first receives and decodes the interpolation I frame data transmitted from the transmitting means, and the decoding result data Decoding the video data after resumption with reference frame data,
A video data transmission / reception system. A video data transmitting apparatus that transmits video data compressed and encoded using motion compensated inter-frame prediction to a plurality of receiving terminals,
A first encoding unit that includes a first DCT unit and a first quantization unit, and generates inter-frame encoded video data of the video data by performing inter-frame encoding processing on a plurality of frames constituting moving image data ; ,
A second DCT unit and a second quantization unit are provided, and in parallel with the encoding processing by the first encoding means, intra-frame encoding processing is performed on the frames constituting the moving image data to generate I-frame data for interpolation. Second encoding means for:
Video data generating means connected to the first encoding means and the second encoding means for forming the video data by combining the interpolated I-frame data and the inter-frame encoded video data; ,
Transmitting means for transmitting the video data and the I-frame data for interpolation to the plurality of receiving terminals,
When the transmission means resumes transmission of the video data to any one of the plurality of receiving terminals once, is a reference frame for a frame transmitted after the transmission resumed among the video data transmitted? If it is determined that the reference frame is not transmitted before the resumption of transmission of the video data, the interpolation I corresponding to the frame for which it is determined that the reference frame is not transmitted. Transmitting frame data to the receiving terminal;
A video data transmitting apparatus characterized by the above. In parallel with the transmission of the video data, further comprising optional data transmission means for transmitting optional video data to any of the plurality of receiving terminals,
The interruption of transmission of the video data from the transmission means to any one of the plurality of receiving terminals occurs when option video data transmission by the option data transmission means to any one of the receiving terminals is executed in an interrupted manner. thing,
The video data transmitting apparatus according to claim 2. The option data transmission means includes information collection means for collecting information on user preferences in advance from at least some of the plurality of receiving terminals,
Selecting the content of the optional video data to be transmitted based on the information collected by the information collecting means;
The video data transmitting apparatus according to claim 3. The transmission means includes broadcast transmission means for broadcasting the same data to a plurality of transmission destinations, and individual transmission means for transmitting individual data to individual transmission destinations. Using the broadcast transmission means, the individual transmission means for the interpolation I frame data,
The option data transmitting means individually transmits the option video data;
The video data transmitting apparatus according to claim 3 or 4, characterized in that: The transmission means includes switching means for excluding the receiving terminal from which the interpolating I frame data or the option video data is transmitted from the video data transmission by the broadcast transmission means;
The video data transmitting apparatus according to claim 5. The option data transmission means includes:
Insertion means for transmitting the second option data in the form of inserting the first option data;
Before the transmission of the first option data is resumed after the transmission of the second option data is completed, the corresponding option data interpolation I frame for at least one frame from the transmission resume position of the first option data. Third encoding means for generating data,
When transmission of the first option data is resumed after transmission of the second option data is completed, the option data transmission means is configured to interpolate the option data prior to resuming transmission of the first option data. Transmitting I frame data to the receiving terminal;
The video data transmission apparatus according to claim 3, wherein the video data transmission apparatus is a video data transmission apparatus. The transmission means determines the number of I-frame data for interpolation to be transmitted to the receiving terminal before resuming transmission of the video data based on the GOP configuration of the video data, in particular, the appearance frequency of I attribute or P attribute frames. To decide,
The video data transmitting apparatus according to claim 2 , wherein the video data transmitting apparatus is a video data transmitting apparatus. A video data transmitting apparatus that transmits video data compressed and encoded using motion compensated inter-frame prediction to a plurality of receiving terminals,
A first encoding unit that includes a first DCT unit and a first quantization unit, and performs intra-frame encoding processing on frames constituting moving image data to generate intra-frame encoded video data;
A second encoding unit that includes a second DCT unit and a second quantization unit, and generates inter-frame encoded video data by performing inter-frame encoding processing on frames constituting moving image data;
Video data generating means for generating the video data from intra-frame encoded video data generated by the first encoding means and inter-frame encoded video data generated by the second encoding means;
Transmitting means for transmitting the video data to the plurality of receiving terminals,
When the transmission means resumes transmission of the video data to any one of the plurality of receiving terminals once, is a reference frame for a frame transmitted after the transmission resumed among the video data transmitted? If it is determined that the reference frame is not transmitted before the resumption of transmission of the video data, the intra-frame code corresponding to the frame for which it is determined that the reference frame is not transmitted is determined. Transmitting the converted data to the receiving terminal as I-frame data for interpolation,
A video data transmitting apparatus characterized by the above. A plurality of video data providing apparatuses that transmit video data compressed and encoded using motion-compensated interframe prediction; a plurality of receiving terminals that receive and decode video data from any of the plurality of video data providing apparatuses; A video data transmission / reception system comprising a video data providing apparatus and a distribution server that relays video data between the plurality of receiving terminals,
The video data providing devices are respectively
1st encoding which has a 1st DCT unit and a 1st quantization unit, and produces | generates the interframe encoding video data of the said video data by performing an interframe encoding process with respect to each of several frames which comprise moving image data Means,
An I-frame data for interpolation by performing intra-frame coding processing on each of the frames constituting the moving image data in parallel with the coding processing by the first coding means , having a second DCT unit and a second quantization unit. Second encoding means for generating
Video data generating means connected to the first encoding means and the second encoding means for forming the video data by combining the interpolated I-frame data and the inter-frame encoded video data; Have
The distribution server
A switching request receiving means for receiving a switching request for switching the video data to be received from any of the plurality of receiving terminals;
When the switching request accepting unit accepts the switching request, transmission of the video data before switching to the receiving terminal that is the switching request source is stopped, and a reference frame for a frame transmitted at the start of transmission of the video data after switching is transmitted. If it is not transmitted , the I-frame data for interpolation obtained from the video data providing apparatus that is the video data providing source after switching is transmitted to the receiving terminal that is the switching request source. Then, the switching transmission means for transmitting the video data after the switching to the receiving terminal of the switching request source,
A video data transmission / reception system. A plurality of video data providing apparatuses that transmit video data compressed and encoded using motion-compensated interframe prediction; a plurality of receiving terminals that receive and decode video data from any of the plurality of video data providing apparatuses; The video data providing apparatus in a video data transmitting / receiving system comprising a distribution server that relays video data between the video data providing apparatus and the plurality of receiving terminals,
1st encoding which has a 1st DCT unit and a 1st quantization unit, and produces | generates the interframe encoding video data of the said video data by performing an interframe encoding process with respect to each of several frames which comprise moving image data Means,
An I-frame data for interpolation by performing intra-frame coding processing on each of the frames constituting the moving image data in parallel with the coding processing by the first coding means , having a second DCT unit and a second quantization unit. Second encoding means for generating
Video data generating means connected to the first encoding means and the second encoding means for forming the video data by combining the interpolated I-frame data and the inter-frame encoded video data; Have
When the video data is transmitted to the distribution server and one of the plurality of receiving terminals requests to switch the video data to be received to the video data, a reference frame for a frame transmitted after switching is transmitted. is and whether the determined and, if not sent, the I-frame data for the interpolation, before the switching, and a transmission means for transmitting to said requesting receiver device via the distribution server thing,
An apparatus for providing video data. In the video data transmission / reception system comprising a transmission side device that transmits video data that has been compression-encoded using motion compensated interframe prediction and a plurality of reception terminals that receive and decode the video data, the transmission side device and the plurality of video data A video data transmission / reception method with one of the receiving terminals,
In transmitting side apparatus, a first encoding step of generating an inter-frame coded video data of the video data by performing frame between encoding a plurality of frames constituting the moving image data,
In the transmitting apparatus, in parallel with the first encoding step, a second encoding step for generating I-frame data for interpolation by intra-frame encoding each of a plurality of frames constituting the moving image data;
A combining step of generating the video data by combining the I-frame data for interpolation and the inter-frame encoded video data;
In the transmission side device, a video data transmission step of transmitting the video data to one of the plurality of receiving terminals ;
At the transmitting side device, interrupting transmission interruption step of transmission to one of said plurality of receiving terminals of the video data,
A reference frame determination step of determining whether or not a reference frame for a frame transmitted after resuming transmission is transmitted to one of the plurality of receiving terminals in the transmission side device;
Corresponding to a frame that is determined not to transmit a reference frame in the I-frame data for interpolation prior to resuming transmission of the video data to one of the plurality of receiving terminals in the transmission side device. Interpolation data transmission step of transmitting interpolation I frame data to one of the plurality of receiving terminals ;
In one of the plurality of receiving terminals, an interpolation data decoding step for decoding the correction I frame data;
In one of the plurality of receiving terminals, and resumes the video data retransmission step of transmission to one of said plurality of receiving terminals of the video data,
A video data decoding step of decoding the video data after resuming transmission using, as reference frame data, data obtained as a result of execution of the interpolation data decoding step in one of the plurality of receiving terminals ;
A method for transmitting and receiving video data. Video data transmission method of the transmission side apparatus in a video data transmission / reception system comprising a transmission side apparatus that transmits video data compressed and encoded using motion compensated inter-frame prediction and a plurality of reception terminals that receive and decode the video data Is a program for causing a computer to execute the video data transmission method,
A first encoding step of generating an inter-frame coded video data of the video data into a plurality of frames constituting the moving image data by performing frame interframe coding processing,
In parallel with the first encoding step, a second encoding step for generating I-frame data for interpolation by intra-frame encoding each of a plurality of frames constituting the moving image data;
A combining step of generating the video data by combining the I-frame data for interpolation and the inter-frame encoded video data;
A video data transmission step of transmitting the video data to one of the plurality of receiving terminals ;
Interrupting transmission interruption step of transmission to one of said plurality of receiving terminals of the video data,
A reference frame determining step for determining whether a reference frame for a frame transmitted after resuming transmission is transmitted to one of the plurality of receiving terminals of the video data;
Prior to resuming transmission of the video data to one of the plurality of receiving terminals, I-frame data for interpolation corresponding to a frame for which it is determined that a reference frame is not transmitted among the I-frame data for interpolation. Interpolating data transmission step for transmitting to one of the plurality of receiving terminals ;
A program characterized by having a video data retransmission resuming transmission to one of said plurality of receiving terminals of the video data.

Images