JP2006319956A - Mpeg coded stream decoding device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an MPEG coded stream decoding device having excellent characteristics to be used by a Video On Demand (VOD) system for solving the problem that the delay of a response or the disturbance of an output video voice is generated, and for realizing a high speed response to the thriving business request of a user and the establishment of the synchronization of high speed video and voice during status transition between a normal reproduction status and a specific reproduction status. <P>SOLUTION: This MPEG coded stream encoding device is provided with a clock memory part 114 for storing an STC value to be outputted by an STC generating part 113 in a designated timing, and when a temporary stop control request is made from a user, an STC value is immediately stored, and the output to an output monitor part 108 and an output speaker part 11 is stopped. Afterwards, when a reproduction request is made, an elapsed time is calculated, and PTS and DTS values stored in a video ES buffer 105 and an audio data buffer 109 are updated. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to an MPEG encoded stream decoding apparatus that decodes an MPEG multiplexed stream (MPEG transport stream) obtained by multiplexing MPEG encoded image data and MPEG encoded audio data to generate decoded image data and decoded audio data. Is.

  In recent years, video / audio services via a network are spreading along with the development of video / audio compression techniques represented by MPEG and the spread of high-speed communication networks. One of the features of the video / audio service using such a communication network is that it is different from the conventional broadcast type service, such as selection of program type and start time, pause, fast forward, and fast reverse according to the user's request. On-demand performance that enables special playback operations and the like can be mentioned.

  A conventional MPEG encoded stream decoding apparatus will be described below.

  Conventionally, an MPEG coded stream decoding device described in Patent Document 1 is known. The MPEG encoded stream decoding apparatus is shown in FIG.

  FIG. 12 is a functional block diagram of a conventional MPEG encoded stream decoding apparatus. In FIG. 12, reference numeral 1 denotes a stream buffer memory for temporarily storing and storing a bit stream of MPEG encoded multiplexed data. Reference numeral 2 denotes a buffer memory control unit that reads the Pack stream data temporarily stored in the stream buffer memory 1 and reads and controls the video PES stream data temporarily stored in the video buffer memory 6. A Pack header processing unit 3 detects a stream ID in the Pack header, extracts a system time reference, and separates the Pack header. A PES packet processing unit 4 performs PES packet header separation and PTS extraction in the PES packet header and separates video packets and audio packets for each pack. A GOP detection unit 5 detects a GOP flag indicating the start of a GOP video stream from a video packet for each pack. Reference numeral 6 denotes a video buffer memory that temporarily stores a video stream and reads it in units of one picture frame. 7 is a buffer for outputting video data for one picture frame from the video buffer memory 6 in a picture frame cycle and outputting a control signal read for each picture frame stored in the video display processing unit 10. It is a memory control unit. A video MPEG decoding unit 8 decodes video data read from the video buffer memory 6 for each picture frame. A decode data storage unit 9 stores the video data decoded by the video MPEG decoding unit 8 for one picture frame at one hour, and detects the presence or absence of a data error such as missing. A video display processing unit 10 outputs video data of one picture frame in accordance with video display timing. A display timing generation unit 11 outputs a video display timing signal. Reference numeral 12 denotes a video D / A converter which converts display video data from digital / analog (D / A) to an image monitor. A frame check detection unit 13 outputs a data error detection signal together with error information when there is an error in the video data stored in the decoded data storage unit 9. A video error information processing unit 14 outputs a retransmission request for a picture frame having a data error based on the data error detection signal output from the frame check detection unit 13 and error information. Reference numeral 15 denotes a clock generation unit that generates an in-device clock signal for MPEG decoding based on the SCR extracted by the Pack header processing unit 3. Reference numeral 16 denotes a time information generation unit that generates in-device time information for picture frame data read timing and audio / video synchronization timing decoded in MPEG based on the PTS information extracted by the PES packet processing unit. Reference numeral 17 denotes a CPU bus. Reference numeral 18 denotes a host CPU. Reference numeral 19 denotes an audio buffer memory for temporarily storing an audio stream input from the PES packet processing unit 4. Reference numeral 20 denotes an AV synchronization timing generation unit that generates an AV synchronization timing signal for synchronizing MPEG-decoded video and audio based on the in-device time information output from the time information generation unit 16. An audio MPEG decoding unit 21 performs MPEG decoding on the audio stream read from the audio buffer memory 19. Reference numeral 22 denotes an audio D / A converter for digital / analog conversion of decoded audio data output from the audio MPEG decoder. 25 is inputted with the in-device clock signal from the clock generation unit 15, the in-device time information from the time information generation unit 16, the GOP detection signal from the GOP detection unit 5, and the error information signal from the error information processing unit 14. The host CPU 18 is controlled via the CPU bus 17 and the host memory 23 to control and respond to the data among the video MPEG decoding unit 8, the audio MPEG decoding unit 21, the buffer memory control unit 7, etc., and the line interface unit. 24 is a CPU control unit that controls the transmission of error information such as a request signal for video transmission stop or retransmission of picture frame video data having an error such as data loss to the MPEG encoding multiplexing side.

  The operation of the conventional MPEG encoded stream decoding apparatus configured as described above will be described below.

  The MPEG multiplexed bit stream is stored and accumulated in the stream buffer memory 1 through the transmission line via the line interface unit 26.

  The buffer memory control unit 2 is controlled by the GOP detection signal from the GOP detection unit 5 to control the stream buffer memory 1 and reads the stream data of the Pack packet every time the GOP flag is detected. repeat.

  The Pack header processing unit 3 separates the Pack header by the header detection / separation unit 30 based on the Pack start code and the stream ID code in the header, and SCR information serving as a system clock reference arranged in the separated header. Is extracted by the SCR extraction unit 31 and supplied to the clock generation unit 15 that generates the in-device clock signal used on the decoding side.

  In the PES packet processing unit 4, the PES packet header is separated from the PES packet stream from which the Pack header is separated by the packet separation unit 1042 and the header separation unit 1040 based on the packet start and the stream ID code in the PES packet header. Then, the PTS information serving as a reference for the time information arranged in the separated header is extracted by the PTS extraction unit, and time information is generated as timing information for obtaining AV synchronization used on the decoding side. In addition to being supplied to the unit 16, it is separated into PES packets for each medium and output.

  The picture buffer memory 6 stores and accumulates the picture stream of the video PES packet input from the PES packet processing unit 4, and the audio buffer memory 19 stores the audio stream of the input audio PES packet. Store and store streams.

  The GOP detection unit 5 generates a timing for reading each stream data in a GOP cycle from each of the stream buffer memory 1 and the picture buffer memory 6, and therefore from the GOP output port of the PES packet processing unit 4. A GOP flag arranged at the head of the video PES packet from which the input PES packet header is separated is detected, output as a GOP detection signal, and supplied to the buffer memory control unit 2.

  As a result, the buffer memory control unit 2 performs reading control for the picture buffer memory 6 at the same time as reading to the stream buffer memory 1.

  The video MPEG decoding unit 8 MPEG-decodes the picture stream data read from the picture buffer memory 6 in units of one picture frame, and the 1-frame memory (picture decoding data frame memory 1090) of the decoding data storage unit 9 ).

  In the memory management unit 1091 of the decode data storage unit 9, when the decode data for one picture frame input from the video MPEG decoding unit 8 is aligned in the picture decode data frame memory 1090 (that is, stored normally), the 1 Picture decode data for the picture frame is output and stored in the video display frame memory 1100 of the video display processing unit 10. Here, if all of the decoded data for one picture frame is not prepared in the picture decode data frame memory 1090, that is, if there is a missing picture decode data, the picture position information of the picture frame having the missing data The error information is output together with the error detection signal. When all of the picture decode data is stored normally, the storage normal signal is output upon completion of the storage, and at the same time, the stored picture decode data is output to the video display frame memory 1100. .

  The video display frame memory 1100 of the video display processing unit 10 stores normal picture decode data output from the picture decode data frame memory 1090 for several picture frames. That is, a data missing error detection signal is output from the memory management unit 1091, and a picture retransmission request signal from the error information processing unit 14 is sent to the MPEG encoding multiplexing side via the CPU control unit 25, The picture decode data for several picture frames stored in the video display frame memory 1100 until the time is retransmitted, MPEG decoded and stored in the picture decode data frame memory 1090 again. If it is read out for display, the display screen will be in a pause state while displaying the previous picture frame screen at the next screen display of the picture frame with missing data. Since the picture and frame images are lost, the resend pict Picture decoding data corresponding to the number of required picture frame to make the time until the storage completion of the decoded data is to accumulate the video display frame memory 1100 in the.

  Then, the display control unit 1101 of the video display processing unit 10 controls each one picture frame from the video display frame memory 1100 that is read out for screen display output in a picture frame cycle under the control of the buffer memory control unit 7. The picture decode data is controlled by the video display timing signal supplied from the display timing generation unit 11 and output to the video D / A conversion unit 12.

  Here, the video display timing signal output from the display timing generation unit 11 is a frame signal having a period of 1/30 second, which is a video vertical synchronization signal of the NTSC system, for example.

  The frame check detection unit 13 monitors the output of the memory management unit 1091 of the decoded data frame memory 1090 at the video display timing signal period supplied from the display timing generation unit 11 and outputs a normal storage signal at each video display timing. If the error detection signal is output at the video display timing, the error information is output to the error information processing unit 14 together with the error detection signal.

  The error information processing unit 14 generates the error to the MPEG encoding multiplexing side via the CPU control unit 25 based on error information such as picture position information supplied from the memory management unit 1091 through the frame check detection unit 13. Then, a video retransmission request signal for requesting retransmission of video data for one GOP including the picture frame or the picture frame is generated and supplied to the CPU control unit 25.

The CPU control unit 25 controls the host memory 23 and the host CPU 18 based on the in-device clock signal from the clock generation unit 15, the in-device time information from the time information generation unit 16, the GOP detection signal from the GOP detection unit 5, and the like. The video MPEG decoding unit 8, the audio MPEG decoding unit 21, and the buffer memory control unit 7 are controlled through the control of the MPEG decoding process, the reading process from the buffer memory, and the like, and the video from the error information processing unit 14 Based on the retransmission request signal, a retransmission request for video data in units of picture frames or GOPs with missing data is made to the MPEG encoding and multiplexing side through the line interface unit 24.
Japanese Patent No. 3087826 (paragraph numbers [0018] to [0032])

  However, in the above-described conventional configuration, in order to manage in units of received pictures, a user control request is transmitted from the receiving terminal side to the server side, and then a request result is received until the receiving terminal receives a processing result corresponding to the request. It was not obtained, and there was a problem that the response was delayed.

  For example, FIG. 13 shows a timing chart in pause control of a conventional MPEG stream encoding / decoding apparatus.

  First, from time T-2f to time T + 2f, a state during normal reproduction is shown. During normal playback, picture data 1301a, 1302a, 1303a, 1304a are received in order. The picture data 1301a is input to the video MPEG decoding unit 8 at time Tf, and the decoded image data is stored in the video display processing unit 10. Next, at time T, the picture data 1302 a is input to the video MPEG decoding unit 8, and the decoded image data 1302 b is output to the video D / A conversion unit 12. Subsequently, at time T + f, picture data 1303 a is input to the video MPEG decoding unit 8, and decoded image data 1303 b is output to the video D / A conversion unit 12.

  When the user requests suspension control at time T + 2f, the CPU control unit requests suspension control to the MPEG multiplexing side through the line interface unit 24.

  When receiving the pause control request, the MPEG multiplexing side stops sending the MPEG encoded multiplexed stream. Until the MPEG multiplexing side stops the stream (= d1), the received picture data 1305a and 1306a are decoded and output, and the output image is actually stopped after the user requests the pause control. It takes a response delay time (= d3) until this is done.

  When the user requests reproduction control again at time T ′ + 2f (where T ′ = T + p), the CPU control unit requests reproduction control to the MPEG multiplexing side through the line interface unit 24. When the MPEG multiplexing side receives the pause control request, it starts sending an MPEG encoded multiplexed stream. Then, after a certain control delay time (= d2) has elapsed, the reception of the stream is resumed, and the response delay time (= d4) from when the user requests playback control until the playback image is actually output. ).

  As described above, in the pause control in the conventional MPEG stream encoding / decoding apparatus, a delay time corresponding to the decode delay + control processing delay is required from the time T + 2f at which the pause is requested until the output image is actually stopped. .

  In order to speed up the response time, a method of erasing the data in the buffer at the time of a pause request is also conceivable. In this case, more delay time is required when normal playback is resumed, and the output video and audio The problem of being disturbed occurs.

  The present invention solves the above-described conventional problems, and an object thereof is to provide an MPEG stream encoding / decoding device that responds at high speed to control requests such as temporary stop, fast forward, and fast reverse.

  According to the first aspect of the present invention, there is provided an MPEG decoding method for decoding an MPEG transport stream including MPEG compressed and encoded image data and audio data and a program time reference value indicating an absolute time reference value in the stream. A stream decoding device comprising at least a payload information analysis unit, a video ES buffer unit, an audio ES buffer unit, a video decoder unit, an audio decoder unit, an STC generation unit, a system control unit, a clock A memory unit, a video time stamp rewriting interface unit, and an audio time stamp rewriting interface unit are provided. During normal playback, they are extracted and separated by the payload information analysis unit, and decoding corresponding to each is performed. Video E with time information and presentation time information The video ES data and audio ES data temporarily stored in the buffer unit and the audio ES buffer unit are input to the video decoder and the audio decoder at a predetermined timing, and a pause control signal is input to the system control unit at the time of pause. At the same time, the STC value output from the STC generation unit is held in the clock memory unit, and at the same time, the input to the video decoder unit and the input to the audio decoder unit are stopped, and when normal playback is requested again. The decoding time information and display time information stored in the video ES buffer unit are corrected by the video time stamp rewriting interface, and the decoding time information stored in the audio ES buffer unit by the audio time stamp rewriting interface and Correct the presentation time information It has the effect of restarting the inputs to the input and the audio decoder section to the video decoder section.

  According to a second aspect of the present invention, there is provided an MPEG stream for decoding an MPEG transport stream that includes MPEG compression encoded image data and audio data and a fixed program time reference value indicating a relative time reference value in the stream. A decoding device comprising at least a payload information analysis unit, a video ES buffer unit, an audio ES buffer unit, a video decoder unit, an audio decoder unit, an STC generation unit, a system control unit, and a clock memory And a STC correction interface, and during normal playback, the video ES buffer unit and the audio ES buffer unit are extracted and separated by the payload information analysis unit, together with the corresponding decoding time information and presentation time information. Video ES data and audio E stored temporarily Data is input to the video decoder and the audio decoder at a predetermined timing, and the STC value output from the STC generation unit is input to the clock memory unit at the timing when the suspension control signal is input to the system control unit at the time of suspension. At the same time, the input to the video decoder unit and the input to the audio decoder unit are stopped, and when normal reproduction is requested again, the STC value output from the STC generation unit by the STC correction interface is again stored in the clock memory. The STC value held in the unit is corrected and the input to the video decoder unit and the input to the audio decoder unit are resumed.

  According to a third aspect of the present invention, there is provided an MPEG stream for decoding an MPEG transport stream including MPEG compression encoded image data and audio data and a fixed program time reference value indicating a relative time reference value in the stream. A decoding device comprising at least a payload information analysis unit, a video ES buffer unit, an audio ES buffer unit, a video decoder unit, an audio decoder unit, an STC generation unit, a system control unit, and a clock memory Unit, an STC correction interface, and a time stamp memory unit, and output of the STC generation unit at the timing when a fast-forward or fast-reverse control request is input to the system control unit during fast-forward or fast-reverse control The STC value to be stored is held in the clock memory unit, and the time The value of the decoding time information of the video ES data received last in the video ES buffer unit is held in the memory unit, and the decoding time information of the video ES data received first when normal playback is requested again And the STC value generated by the STC generation unit from the difference value between the time value held in the time stamp memory unit and the STC value stored in the clock memory unit.

  According to a fourth aspect of the present invention, there is provided an MPEG stream for decoding an MPEG transport stream including MPEG compressed and encoded image data and audio data and a fixed program time reference value indicating a relative time reference value in the stream. A decoding device comprising at least a payload information analysis unit, a video ES buffer unit, an audio ES buffer unit, a video decoder unit, an audio decoder unit, an STC generation unit, a system control unit, and a clock memory Unit, an STC correction interface, and a video reordering buffer unit. During normal playback, the video information is extracted and separated by the payload information analysis unit, and the video ES buffer unit together with the corresponding decoding time information and presentation time information, Video temporarily stored in the audio ES buffer S data and audio ES data are input to a video decoder and an audio decoder at a predetermined timing, and simultaneously input to the video decoder unit and an output of the video reordering buffer unit at the time of pause, an audio decoder unit When the received data is interrupted, the STC value output from the STC generator is held in the clock memory unit. When normal reproduction is requested again, STC is generated by the STC correction interface. The STC value output from the unit is corrected to the STC value held in the clock memory unit, and the decoding time information and presentation of each data stored in the video ES buffer unit, audio ES buffer unit, and video reordering buffer unit are presented. The time information is corrected and input to the video decoder and video Resume the inputs to the output and the audio decoder section of Daringu buffer unit, has the effect of

  As described above, the present invention provides an excellent effect that a quick and seamless response output can be obtained in response to a user's control request.

  Hereinafter, embodiments of the present invention will be described with reference to FIGS.

(Embodiment 1)
FIG. 1 is a functional block diagram of an MPEG encoded stream decoding apparatus according to the present invention.

  In FIG. 1, 101 is a stream receiver that receives an MPEG transport stream including PCR, PTS, and DTS generated in real time from the transmission side, and 102 is an MPEG transport stream received by the stream receiver 101. A header information analysis unit for analyzing header information of individual transport stream packets to be performed; 103, an adaptation field information analysis unit for analyzing adaptation field information of transport stream packets; and 104, a transport stream packet Payload information analysis unit for analyzing packet payload information; 105, video ES buffer unit; 106, video decoder unit; 107, video reordering buffer unit 108: output monitor unit; Dio ES buffer unit, 110 audio decoder, 111 an output speaker, 112 system control unit, 113 STC generating unit, 114 is a clock memory unit.

  Reference numeral 115 denotes a video time stamp rewriting interface. Reference numeral 116 denotes an audio time stamp rewriting interface.

  FIG. 2 is a schematic diagram for explaining the frame order during encoding and decoding in MPEG image encoding.

  In FIG. 2, 201a to 215a indicate the arrangement order of the frames of the image data before compression encoding. In addition, the arrangement of pictures after 201b to 215b compression coding is shown. Reference numerals 201c to 215c denote states in which the decoded image data are rearranged in the display order.

  In MPEG encoding, one moving picture screen (frame) data is called a picture, and I (Intra-coded) picture, P (Predictive-coded) picture, B (Bidirectionally-predictive-), depending on the encoding type. Three types of pictures called “coded” pictures are used. At this time, the I picture is a frame that independently encodes the input image frame data. A P picture is a frame that uses motion compensation prediction in the forward direction. A B picture is a frame that uses bidirectional prediction. Then, inter-frame prediction is independently processed in a unit called GOP (Group Of Pictures) in which continuous picture data is divided every predetermined number.

  Each GOP first encodes an I picture as its reference picture. For this reason, after the 203a frame that is an I picture in the GOP structure is encoded independently (becomes 203b), the 201a frame and the 202a frame that are B pictures are sequentially encoded with reference to the 203a frame, and a compressed encoded stream (Elementary stream) is formed (respectively 201b and 202b).

  Similarly, in 204a, 205a and 206a, 206a is encoded with reference to 203a (becomes 206b), and 204a and 205a are sequentially encoded with reference to 203a and 206a (becomes 204b and 205b). Thereafter, the same processing is repeated. The compressed encoded stream is configured in the order of 203b, 201b, 202b, 206b, 204b, 205b,... According to the order of the encoding process.

  On the other hand, at the time of decoding, encoded data is received in the order of 203b, 201b, 202b, 206b, 204b, 205b,... From the compressed encoded stream, and decoding is started in that order.

  However, since the decoding order is different from the original input image, it is necessary to rearrange the decoded images in the original image frame order for display.

  FIG. 3 is a schematic diagram showing a hierarchical structure of an MPEG transport stream.

  Each elementary stream (ES) of video and audio has a decoding / playback unit called an access unit (one picture for video and one audio frame for audio). A packetized elementary stream (PES) is obtained by dividing an elementary stream in units of access units and adding a header. The PES packet header is decoded at any reference time for each unit of access units. Time stamps indicating whether or not playback should be performed are described as a presentation time stamp (PTS) and a decoding time stamp (DTS), respectively. When the STC of the decoder matches the PTS of each access unit, the decoding result of each access unit is output from the decoder. In addition, since a stream encoded with MPEG-2 video may have a different decoding order and reproduction order, a DTS indicating a decoding time is added to the PTS in this case. With these time stamps, video and audio can be played back synchronously.

  Further, a code (stream ID) for identifying an individual stream type, a packet size, other control information (scrambled presence / absence of copyright, CRC addition, etc.) are described in the PES packet header.

  A PES packet containing individual streams such as a video elementary stream and an audio elementary stream is divided into a plurality of transport stream (TS) packet payloads and transmitted. The TS packet is a 188-byte fixed-length packet, and includes a 4-byte fixed-length packet header, a variable-length adaptation field (adaptation field), and a payload (payload). PID (packet identifier) and various flags are defined in the packet header. Each elementary stream is identified by this PID. There is a case where only one of adaptation_field and payload is present or both, and the presence / absence thereof is indicated by a flag (adaption_field_control) in the packet header.

  The adaptation_field has a stuffing function in the TS packet for transmitting information such as PCR (Program_Clock_Reference) and making the TS packet a fixed length of 188 bytes. The PCR is a 27 MHz time stamp, and the value of the PCR is referred to reproduce the reference time when encoded by the STC of the decoder.

  In FIG. 3, reference numerals 301 to 303 denote continuous video access units, and reference numerals 304 to 307 denote continuous audio access units. Reference numerals 308 to 310 denote continuous video PES packets, and reference numerals 311 to 314 denote continuous audio PES packets. Reference numerals 315 to 325 denote consecutive TS packets.

  FIG. 4 is a schematic diagram showing the structure of a PES packet.

  4, 401 is a packet start code prefix which is a start code of a PES packet, 402 is a stream ID indicating the type of data stream in the PES packet, 403 is a PES packet length indicating the length of the PES packet, and 404 is a PES header. 405 is a stuffing byte inserted to adjust the packet length, and 406 is a PES packet data byte in which ES data corresponding to the data portion of the PES packet is arranged. . In the PES header option, 407 is a fixed bit which is always a fixed value “10” of 2 bits, 408 is a flag 1 indicating the presence or absence of PES priority or copyright, and 409 is a PTS_DTS indicating the presence or absence of a PTS or DTS field in the packet. Flag, 410 indicates the presence or absence of various data fields in the PES header data, 411 indicates a PES header data length indicating the length of the PES header, 412 indicates an option field, and 413 starts in this PES packet PTS and DTS 414 indicating values obtained by counting the output time and decoding time of the access unit with a clock of 90 kHz are data fields in which data indicated by the flag 2 is arranged.

  FIG. 5A is a schematic diagram showing the structure of a TS packet. As described above, the TS packet is a 188-byte fixed-length packet, and includes a 4-byte packet header, a variable-length adaptation field, and a payload as shown in FIG.

  In FIG. 5A, 501 is a synchronization byte indicating the head of a TS packet, 502 is a transport error indicator indicating the presence or absence of an error in the TS packet, and 503 is a head byte such as PES or section in the TS packet. Payload unit start indicator indicating whether or not it is included, 504 is a transport priority indicating the priority of this TS packet, 505 is a PID identifying a stream number included in this TS packet, 506 is a scramble of the payload part Transport scramble control indicating presence / absence, 507 is an adaptation field and adaptation field control indicating presence / absence of payload, 508 is a continuity index indicating continuity of packets for each PID, and 509 is adaptation field control 5 The adaptation field or payload, or both depending on the value of 7.

  FIG. 5B is a schematic diagram showing the structure of the adaptation field in the TS packet.

  In FIG. 5B, 510 is an adaptation field length indicating the length of the adaptation field, 511 is a discontinuity indicator indicating packet discontinuity, and 512 is a point when this TS packet is randomly accessed. 513 is an elementary stream priority indicator indicating the priority of the payload of this TS packet, 514 is a flag indicating the presence / absence of each data in the adaptation field, 515 is present in the flag 514 An option field 516 in which the processed data is arranged is a stuffing field for adjusting the length of the packet.

  FIG. 5C is a schematic diagram showing the structure of the option field in the adaptation field.

  In FIG. 5C, 517 is a PCR (program time reference value) indicating the current time of the stream counted at 27 MHz, and 518 is an OPCR (original PCR) for leaving the original value when the value of the PCR 517 is changed. 519 is a splice countdown indicating the number of TS packets with the same PID up to the editing point, 520 is transport private data indicating user-defined data, 521 is an adaptation field indicating expanded adaptation field data It is a field extension.

  6A and 6B are schematic diagrams showing the configuration of data stored in the video ES buffer 105. FIG. As shown in FIG. 6, the video ES buffer 105 stores the corresponding PTS and DTS values together with the picture data. When B picture data without DTS is received, the same value as PTS is stored as the value of DTS.

  The operation of the MPEG encoded stream decoding apparatus configured as described above will be described with reference to FIGS.

  During normal playback, the header information of the MPEG transport stream received by the stream receiving unit 101 is analyzed by the header information analysis unit 102 for each TS packet, and is identified as image data or audio data according to the PID. The image data is stored in the video ES buffer unit 105, and the audio data is stored in the audio ES buffer unit 109.

  Data accumulated in units of pictures in the video ES buffer unit 105 is input to the video decoder unit 106 every time the STC value output from the STC generation unit 113 matches the DTS value. The decoded image data is output and displayed as it is through the output monitor unit 108 when the corresponding PTS value is equal to the DTS value. When the decoded image data is delayed as compared with the DTS, the video reordering buffer 107 is temporarily displayed. And the output timing is adjusted.

  Data stored in the audio ES buffer unit 109 in units of audio frames is also decoded in the same manner as video.

  Then, by synchronizing the decoding timing indicated by the DTS and the output timing indicated by the PTS of the video data and the audio data with the STC, the output video and audio can be synchronized.

  When the suspension control is requested by the user, the suspension control information is input to the system control unit 112. At this time, the system control unit stores the STC value output from the STC generation unit 113 at the timing when the suspension control is requested in the clock memory unit 114. At the same time, the data input of the video decoder unit 106 and the audio decoder unit 110 and the data input of the output monitor unit 108 and the output speaker unit 111 are stopped.

  Further, the suspension control request from the user is also notified to the transmission server side through a communication line not shown in the figure, and after a certain time, the transmission of the stream from the transmission server is stopped.

  As a result, for example, data as shown in FIG. 6A is accumulated in the video ES buffer. FIG. 6A shows a state in which three picture data of P6 picture data, B4 picture data, and B5 picture data and corresponding PTS and DTS values are stored.

  Thereafter, when the user requests reproduction control again, reproduction control information is input to the system control unit 112. At this time, the system control unit reads the STC value output from the STC generation unit 113 at the timing when the reproduction control is requested, and the time during which the system control unit has been in a suspended state from the difference value from the STC value stored in the clock memory unit 114 (= P) is calculated. Then, the video time stamp rewriting interface 115 and the audio time stamp rewriting interface 116 are updated as a value obtained by adding the pause time corresponding to each picture data stored in the video ES buffer unit. FIG. 6B shows the result of updating the PTS and DTS from the state of FIG.

  The PTS and DTS of each audio frame data stored in the audio ES buffer unit are similarly updated. Further, the data input of the video decoder unit 106 and the audio decoder unit 110 and the data input of the output monitor unit 108 and the output speaker unit 111 are resumed.

  As a result, the PTS and DTS of the data stored in the buffer can be matched with the STC generated by the STC generation unit 113 when playback is resumed, and playback can be resumed from the data stored in the buffer.

  FIG. 7 is a timing chart showing the state of the video data at the time of the state transition from “reproduction to pause to reproduction” according to the present invention.

  First, from time T-2f to time T + 2f, a state during normal reproduction is shown. During normal playback, picture data 701a, 702a, 703a, and 704a are received in order. These are stored in the buffer together with the corresponding PTS and DTS values of each picture. At time Tf, picture data 701 a is input to the video decoder unit 106, and image data obtained as a result of decoding is stored in the video reordering buffer 107. Next, at time T, picture data 702 a is input to the video decoder unit 106, and image data 702 b as a decoding result is output to the output monitor unit 108. Subsequently, at time T + f, picture data 703 a is input to the video decoder unit 106, and decoded image data 703 b is output to the output monitor unit 108.

  When the user requests pause control at time T + 2f, the input to the video decoder unit 106 and the input to the output monitor unit 108 are stopped immediately. Thereafter, the received picture data 705 a and 706 a is stored in the video ES buffer unit 105.

  When the user again requests playback control at time T ′ + 2f (where T ′ = T + p), the video time stamp rewriting interface 115 updates the PTS and DTS of the data stored in the video ES buffer unit 105, respectively. Then, the input to the video decoder unit 106 and the input to the output monitor unit 108 are resumed.

  As a result, first, the picture data 704 a is input to the video decoder unit 106, and the decoded image data is stored in the video reordering buffer 107. At the same time, the image data 701 b is read from the video reordering buffer 107 and output from the output monitor unit 108. Subsequently, at time T ′ + 3f, the picture data 705 a is input to the video decoder unit 106, and the decoded image data 705 b is output to the output monitor unit 108. By performing the same processing thereafter, it is possible to output image data seamlessly from the paused state. Also, by processing audio data in the same way as video data, video and audio can be synchronized.

  As described above, according to the present embodiment, by providing the time reference value holding means for holding the STC value from the normal reproduction to the pause control, the response time to the control request is increased, and the time from the pause state is increased. By updating the PTS and DTS in the buffer during playback control, playback can be resumed quickly and seamlessly.

(Embodiment 2)
FIG. 8 is a block diagram showing the configuration of the MPEG encoded stream decoding apparatus according to the present invention.

  8, each component having the same reference numeral as in FIG. 1 has the same meaning as in FIG. 1 is different from the configuration of FIG. 1 in that the system control unit 112 does not update the data of the video ES buffer unit 105 and the video ES buffer unit 109 by the video time stamp rewriting interface 115 and the audio time stamp rewriting interface 116. The STC correction interface 801 changes the STC value generated by the STC generator 113.

  In addition, the MPEG transport stream received in the present invention includes PCR, PTS, and DTS values as fixed values in the stream, as in the case where the file is recorded in advance on the transmission side. To do.

  The operation of the MPEG encoded stream decoding apparatus configured as described above will be described with reference to FIG.

  First, when the user requests the suspension control, the suspension control information is input to the system control unit 112. At this time, the system control unit 112 stores the STC value output from the STC generation unit 113 at the timing when the suspension control is requested in the clock memory unit 114. At the same time, the data input of the video decoder unit 106 and the audio decoder unit 110 and the data input of the output monitor unit 108 and the output speaker unit 111 are stopped.

  Thereafter, when the user requests reproduction control again, reproduction control information is input to the system control unit 112. At this time, the system control unit 112 reads the STC value stored in the clock memory unit 114 at the timing when the reproduction control is requested, and rewrites the STC value generated by the STC generation unit 113 by the STC correction interface 801. Then, data input to the video decoder unit 106 and the audio decoder unit 110 and data input to the output monitor unit 108 and the output speaker unit 111 are resumed.

  As a result, the PTS and DTS of the data stored in the buffer can be matched with the STC generated by the STC generation unit 113 when the reproduction is resumed, and the reproduction of the data accumulated in the buffer can be resumed.

  FIG. 9 is a timing chart showing the state of video data at the time of transition from “playback to pause to playback”.

  First, the normal playback state is shown from time 0 to time 4f. During normal playback, picture data 901a, 902a, 903a, and 904a are received in order. These are stored in the buffer together with the corresponding PTS and DTS values of each picture. Since the STC value is f at time f, the picture data 901 a is input to the video decoder unit 106, and the decoded image data is stored in the video reordering buffer 107. Next, when the STC value is 2f, the picture data 902a is input to the video decoder unit 106, and the decoded image data 902b is output to the output monitor unit 108. Subsequently, when the STC value is 3f, the picture data 903a is input to the video decoder unit 106, and the decoded image data 903b is output to the output monitor unit 108.

  When the user requests pause control at time 4f, the input to the video decoder unit 106 and the input to the output monitor unit 108 are stopped immediately. Thereafter, the received picture data 905 a and 906 a are stored in the video ES buffer unit 105.

  When the user requests playback control again at time p + 4f, the STC value is returned to 4f which was the STC value at the time when the pause was requested, and the input to the video decoder unit 106 and the output monitor unit 108 are performed. Resume input.

  As a result, first, picture data 904 a is input to the video decoder unit 106, and image data as a decoding result is stored in the video reordering buffer 107. At the same time, image data 901 b is read from the video reordering buffer 107 and output from the output monitor unit 108. Subsequently, when the STC is 5f, the picture data 905a is input to the video decoder unit 106, and the decoded image data 905b is output to the output monitor unit 108. By performing the same processing thereafter, it is possible to output image data seamlessly from the paused state. Also, by processing audio data in the same way as video data, video and audio can be synchronized.

  As described above, according to the present embodiment, by providing the time reference value holding means for holding the STC value from the normal reproduction to the pause control, the response time to the control request is increased, and the time from the pause state is increased. By updating the STC value during playback control, playback can be resumed quickly and seamlessly.

(Embodiment 3)
FIG. 10 is a block diagram showing the configuration of an MPEG encoded stream decoding apparatus according to the present invention.

  In FIG. 10, each component is the same as in FIG. 8 differs from the configuration of FIG. 8 in that a time stamp memory unit for storing DTS of picture data accumulated in the video ES buffer unit 105 is provided in 1001.

  It is assumed that the MPEG transport stream received in the present invention includes PCR, PTS, and DTS values as fixed values in the stream, as in the case where the MPEG transport stream is recorded in advance on the transmission side as a file.

  The operation of the MPEG encoded stream decoding apparatus configured as described above will be described with reference to FIG.

  First, when fast-forward control is requested by the user, fast-forward control information is input to the system control unit 112. At this time, the system control unit stores the STC value output from the STC generation unit 113 at the timing when the fast-forward control is requested in the clock memory unit 114, and the picture data stored last in the video ES buffer unit 105. Are stored in the time stamp memory unit 1001. At the same time, the data input to the audio decoder unit 110 is stopped. Then, the picture data stored in the video ES buffer unit 105 is once cleared.

  Further, the fast-forward control request from the user is also notified to the transmission server through a communication line not shown in the figure, and during the fast-forward playback, the stream for fast-forward playback is intermittently sent from the transmission server.

  The MPEG encoded stream decoding device side decodes and outputs only the I picture that can be independently decoded from the received stream regardless of DTS and PTS.

  Thereafter, when the user requests reproduction control again, reproduction control information is input to the system control unit 112. At this time, the system control unit 112 clears all picture data stored in the video ES buffer 105 at the timing when the reproduction control is requested. First, the DTS value of the picture data stored in the video ES buffer unit 105 is extracted, and the time increment on the stream advanced by the fast-forward control from the difference value from the DTS value stored in the time stamp memory unit 1001 Calculate the value. Then, the STC value generated by the STC generation unit 113 is updated using the calculated time increment value and the STC value stored in the clock memory unit 114, and the normal reproduction operation is resumed.

  As a result, the PCR, PTS, and DTS on the stream advanced by the fast-forward operation can be matched with the STC generated by the STC generation unit 113, and the reproduction can be resumed normally.

  FIG. 11 is a timing chart showing a state of video data at the time of transition from “playback to fast forward to playback”.

  First, the normal playback state is shown from time 0 to time 4f. During normal playback, picture data 1101a, 1102a, 1103a, and 1104a are received in order. These are stored in the video ES buffer unit 105 together with the corresponding PTS and DTS values of each picture. Since the STC value is f at time f, the picture data 1101 a is input to the video decoder unit 106, and the decoded image data is stored in the video reordering buffer 107. Next, when the STC value is 2f, the picture data 1102 a is input to the video decoder unit 106, and the decoded image data 1102 b is output to the output monitor unit 108. Subsequently, when the STC value is 3f, the picture data 1103a is input to the video decoder unit 106, and the decoded image data 1103b is output to the output monitor unit 108.

  When the user requests fast-forward playback control at time 4f, all picture data in the video ES buffer unit 105 is once discarded. In fast-forward playback control, independently decoded I-picture data 1105a and 1106a are input to the video decoder unit 106 each time they are received regardless of DTS and PTS, and the decoded image data is 1105b, As 1106b, an output is displayed from the output monitor unit.

  When the user requests reproduction control again at time w + 4f, the STC value is STC1 (= 4f), which is the STC value at the time when fast-forwarding control is requested, and the DTS of the last received picture data at that time From the value D1 (= 4f) and the DTS value D2 (= 46f) of the first picture data received after the fast-forward playback control request, the new STC value is STC2 = STC1 + (D2-D1) = 4f + (46f-4f ) = 46f, and the normal reproduction operation is resumed.

  As a result, at time w + 6f (STC = 46f), the picture data 1107a is input to the video decoder unit 106, and the decoded image data is stored in the video reordering buffer 107. Subsequently, when the STC is 47 f, the picture data 1108 a is input to the video decoder unit 106, and the decoded image data 1108 b is output to the output monitor unit 108. By performing the same processing thereafter, it is possible to immediately shift from the fast-forward playback state to the normal playback state. Also, by processing audio data in the same way as video data, video and audio can be synchronized.

  As described above, according to the present embodiment, the decoding time information holding unit that holds the DTS value of the received picture data and the time reference value holding unit that holds the STC value during the fast-forward control from the normal reproduction time. By providing it, the response time to the control request can be increased, and the STC value can be updated at the time of transition to the normal playback from the fast-forward playback state so that the video and audio can be synchronized at high speed and playback can be resumed. Become.

  In the above description, the operation during fast-forward playback has been described. However, the same control can be performed in fast-rewind playback, slow playback, and reverse playback.

(Embodiment 4)
FIG. 14 is a timing chart showing the state of STC and video data at the time of transition from “playback to pause to playback”.

  The configuration of the MPEG encoded stream decoding apparatus in the present invention is the same as that in FIG. The operation of the MPEG encoded stream decoding apparatus of the present invention will be described with reference to FIG.

  First, when the user requests the suspension control, the suspension control information is input to the system control unit 112. At this time, the data input of the video decoder unit 106 and the audio decoder unit 110 and the data input of the output monitor unit 108 and the output speaker unit 111 are stopped.

After that, the STC value at the time when the stream is actually interrupted is stored in the clock memory unit 114. Then, when the reproduction control is requested again from the user, the reproduction control information is input to the system control unit 112. At this time, the system control unit 112 reads the STC value stored in the clock memory unit 114 at the timing when the reproduction control is requested, and rewrites the STC value generated by the STC generation unit 113 by the STC correction interface 801. Then, the data input of the video decoder unit 106 and the audio decoder unit 110 and the data input of the output monitor unit 108 and the output speaker unit 111 are resumed.

  At this time, the data stored in the video ES buffer unit 105 and the audio ES buffer unit 109 is corrected by adding the time corresponding to the actual stream stop period to the PTS value and the DTS value. Also, the time correction value is added to the data received after the restart and is accumulated in the buffer.

  As a result, the PTS and DTS of the data stored in the buffer can be matched with the STC generated by the STC generation unit 113 when the reproduction is resumed, and the reproduction of the data accumulated in the buffer can be resumed.

  In FIG. 14, (a) shows STC values from time T0 to T0 + 11f. Circles in the figure indicate that PCR information is received at that time. (B) shows frame data received between time T0 and T0 + 11f. (C) shows frame data that is displayed and output between time T0 and T0 + 11f.

  First, from time T0 to time T0 + 4f, a state during normal reproduction is shown. During this time, frame data 1401b, 1402b, 1403b, and 1404b are received as shown in FIG. These are stored in the buffer together with the corresponding PTS and DTS values of each frame. At the same time, the frames 1401c, 1402c, 1403c, and 1404c are output and displayed in order.

  When the user requests pause control at time T0 + 4f, the input to the video decoder unit 106 and the input to the output monitor unit 108 are stopped immediately. Thereafter, the received frame data 1405 b and 1406 b are stored in the video ES buffer unit 105.

  When the user again requests playback control at time T0 + 7f, the STC value is returned to the STC value C0 + 5f at time T0 + 6f at which reception was interrupted. Also, the interval + f between the time T0 + 6f at which reception is interrupted and the time T0 + 7f at which playback is resumed is set as a time correction amount, and is added to the PTS and DTS values of the frame data stored in each buffer. The input to the unit 106 and the input to the output monitor unit 108 are resumed. Further, the frame data received thereafter is added to the PTS and DTS values by + f and stored in the buffer. As a result, the STC value at the time of resuming reproduction becomes C0 + 5f, and reproduction is resumed.

  FIG. 15 is a table showing the PTS value and DTS value of each received frame data. FIG. 16 shows the DTS and PTS values of the frame data stored in each buffer. (A) in the figure shows data accumulated in (1) video ES buffer and (2) video reordering buffer when reception is interrupted, and (b) data in a state where DTS value and PTS value are corrected after resuming reproduction. Show.

  As shown in FIGS. 16B and 16A, the DTS value and the PTS value of the frame B5 after the resumption of reproduction are corrected to C0 + 5f and coincide with the STC value C0 + 5f corrected at the time of resumption of reproduction. Is done. The DTS value and the PTS value of the frame data received thereafter are also corrected by + f, so that the received PCR value and the STC value can be matched. That is, it becomes possible to output image data seamlessly from the paused state. Also, by processing audio data in the same way as video data, video and audio can be synchronized.

  As described above, according to the present embodiment, by providing the time reference value holding means for holding the STC value from the normal reproduction to the pause control, the response time to the control request is increased, and the pause control request is made. Even when the delay time from the time to the reception interruption and the delay time from the playback resumption control request and the reception resumption are different, the STC value, the PTS value, and the DTS value are updated at high speed and seamlessly during the reproduction control from the paused state. Playback can be resumed.

  The MPEG encoded stream decoding apparatus according to the present invention has an effect of enabling a high-speed and seamless response in pause from normal reproduction, fast-forward control, etc., and video on demand using an MPEG encoded stream ( VOD) · Useful as a special playback control system for the system.

Functional block diagram of an MPEG encoded stream decoding apparatus in Embodiment 1 of the present invention Schematic diagram showing the frame order during encoding and decoding in MPEG image encoding Schematic diagram showing the hierarchical structure of an MPEG transport stream Schematic diagram showing the structure of a PES packet Schematic diagram showing the structure of TS packet FIG. 2 is a schematic diagram showing a configuration of data stored in a video ES buffer according to Embodiment 1 of the present invention. Timing chart showing a state of video data at the time of transition from “playback to pause to playback” in the first embodiment of the present invention Functional block diagram of an MPEG encoded stream decoding apparatus in Embodiment 2 of the present invention Timing chart showing the state of video data at the time of transition from “playback to pause to playback” in Embodiment 2 of the present invention Functional block diagram of an MPEG encoded stream decoding apparatus in Embodiment 3 of the present invention Timing chart showing the state of video data at the time of transition from “playback to fast forward to playback” in Embodiment 3 of the present invention Functional block diagram of a conventional MPEG encoded stream decoding device Timing chart showing the state of video data at the time of transition from “playback to pause to playback” in a conventional MPEG encoded stream decoding apparatus Timing chart showing the state of video data at the time of transition from “playback to pause to playback” in Embodiment 4 of the present invention The figure which shows DTS value and PTS value of each received frame data The figure which shows the DTS value and PTS value of the frame data accumulate | stored in each buffer

Explanation of symbols

DESCRIPTION OF SYMBOLS 101 Stream receiving part 102 Header information analysis part 103 Adaptation field information analysis part 104 Payload information analysis part 105 Video ES buffer part 106 Video decoder part 107 Video reordering buffer part 108 Output monitor part 109 Audio ES buffer part 110 Audio Decoder unit 111 Output speaker unit 112 System control unit 113 STC generation unit 114 Clock memory unit 115 Video time stamp rewriting interface 116 Audio time stamp rewriting interface

Claims (4)

An MPEG stream decoding apparatus for decoding an MPEG transport stream including MPEG compression-encoded image data and audio data and a program time reference value indicating an absolute time reference value in the stream, at least a payload information analysis unit A video ES buffer unit, an audio ES buffer unit, a video decoder unit, an audio decoder unit, an STC generation unit, a system control unit, a clock memory unit, and a video time stamp rewriting interface unit And an audio time stamp rewriting interface unit, which are extracted and separated by the payload information analysis unit during normal reproduction, and together with the corresponding decoding time information and presentation time information, a video ES buffer unit and an audio ES buffer unit Temporarily The stored video ES data and audio ES data are input to the video decoder and the audio decoder at a predetermined timing, and when paused, the STC generation unit outputs at the timing when the pause control signal is input to the system controller. The STC value is held in the clock memory unit, and at the same time, the input to the video decoder unit and the input to the audio decoder unit are stopped. When normal playback is requested again, the video time stamp rewriting interface Correction of decoding time information and display time information stored in the video ES buffer unit, correction of decoding time information and presentation time information stored in the audio ES buffer unit by an audio time stamp rewriting interface, Input to the decoder section and audio data MPEG encoded stream decoding apparatus characterized by resuming the input to chromatography reader unit. An MPEG stream decoding apparatus for decoding an MPEG transport stream including MPEG compression encoded image data and audio data and a fixed program time reference value indicating a relative time reference value in the stream, and at least payload information An analysis unit, a video ES buffer unit, an audio ES buffer unit, a video decoder unit, an audio decoder unit, an STC generation unit, a system control unit, a clock memory unit, and an STC correction interface are provided. In normal playback, the video ES data and the audio ES data that are extracted and separated by the payload information analysis unit and temporarily stored in the video ES buffer unit and the audio ES buffer unit together with the corresponding decoding time information and presentation time information, Video at a given timing The STC value output from the STC generation unit is held in the clock memory unit at the timing when the pause control signal is input to the system control unit and is input to the decoder and audio decoder. The input and the input to the audio decoder unit are stopped, and when the normal reproduction is requested again, the STC value output from the STC generation unit is corrected to the STC value held in the clock memory unit by the STC correction interface. And an MPEG encoded stream decoding apparatus, wherein input to the video decoder unit and input to the audio decoder unit are resumed. An MPEG stream decoding apparatus for decoding an MPEG transport stream including MPEG compression encoded image data and audio data and a fixed program time reference value indicating a relative time reference value in the stream, and at least payload information Analysis unit, video ES buffer unit, audio ES buffer unit, video decoder unit, audio decoder unit, STC generation unit, system control unit, clock memory unit, STC correction interface, time A stamp memory unit that holds the STC value output from the STC generation unit in the clock memory unit at the time when the fast-forward or fast-reverse control request is input to the system control unit during fast-forward or fast-reverse control; , Time stamp memory part, video ES buffer part The value of the decoding time information of the video ES data received last is held, and when the normal playback is requested again, it is held in the decoding time information and time stamp memory unit of the video ES data received first. An MPEG encoded stream decoding apparatus, wherein an STC value generated by an STC generation unit is corrected based on a difference value between the time value and an STC value stored in a clock memory unit. An MPEG stream decoding apparatus for decoding an MPEG transport stream including MPEG compression encoded image data and audio data and a fixed program time reference value indicating a relative time reference value in the stream, and at least payload information Analysis unit, video ES buffer unit, audio ES buffer unit, video decoder unit, audio decoder unit, STC generation unit, system control unit, clock memory unit, STC correction interface, video A reordering buffer unit, which is extracted and separated by the payload information analysis unit during normal playback, and temporarily stored in the video ES buffer unit and the audio ES buffer unit together with the corresponding decoding time information and presentation time information ES data and audio E Data is input to the video decoder and audio decoder at a predetermined timing. At the time of pause, the input to the video decoder unit, the output of the video reordering buffer unit, and the input to the audio decoder unit are performed simultaneously. The STC value output from the STC generation unit is held in the clock memory unit at the timing when the received data is interrupted, and the STC output from the STC generation unit is output by the STC correction interface when normal reproduction is requested again. The value is corrected to the STC value held in the clock memory unit, and the decoding time information and presentation time information of each data stored in the video ES buffer unit, audio ES buffer unit, and video reordering buffer unit are corrected. Video decoder unit and video reordering buffer unit MPEG encoded stream decoding apparatus characterized by resuming the input to the output and the audio decoder unit.