JP2007195064A - Device and method for transmitting, image information, program, and storage medium - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To output digital data for special reproduction through a digital interface, in the form of being decodable by the existing decoder. <P>SOLUTION: A device for transmitting image information includes: an instructing means for instructing a special reproduction mode; a means for changing timing control data for regulating display timing of each frame in an encoded image stream reproduced by a reproducing means in response to an instruction by the instructing means; a means for generating copied picture data being data for instructing the display of image data being the same as an immediately-before image data; a means for generating stuffing data for controlling a storage amount of a virtual input buffer of a decoder for decoding the encoded image stream; and an outputting means for inserting the copied picture data and the stuffing data into the encoded image stream output from a changing means in response to the instruction by the instructing means. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to an image information transmission apparatus, an image information transmission system, an image information transmission method, a program, and an image information transmission apparatus that are applied when the MPEG (Moving Picture Experts Group) video signal recorded on a recording medium is read out and recorded again. The present invention relates to a storage medium.

  In recent years, an MPEG stream recorded on a recording medium is read and output via a digital interface such as IEEE 1394, and is recorded on an external recording device or decoded and displayed by a playback device.

  When the MPEG stream is transmitted with special reproduction such as slow, still, frame advance, etc., it can be realized by sending the header information of the ISO / IEC13818-1 system stream. Specifically, information related to special playback is described in a DSM_trick_mode field (DSM: Digital Storage Media) in header information of a PES (Packetized Elementary Stream) packet.

However, there are many decoders that do not support DSM_trick_mode in the header information of PES packets. For this reason, a method for outputting special reproduction of an MPEG stream that can be supported by an existing decoder to a digital interface has been proposed. For example, a method (Patent Document 1 and Patent Document 2) is proposed in which a copy picture is generated and output, and data indicating the storage amount of the virtual input buffer of the decoder is invalidated. Furthermore, a method (Patent Document 3) has also been proposed in which encoded data is once decoded and then re-encoded and transmitted.
JP 2002-77815 A JP 2003-101961 A JP 2001-28748 A

  However, the devices disclosed in Patent Document 1 and Patent Document 2 invalidate the data indicating the storage amount of the virtual input buffer of the decoder, which may cause the output destination decoder to fail. When the encoded data to be transmitted is data in which image data and audio data are multiplexed, audio data is interrupted on the receiving device side due to the insertion of a copy picture.

  Further, in the reproduction transmission apparatus disclosed in Patent Document 3, another reproduction encoder is required for the reproduction transmission apparatus, which increases cost and power consumption.

  Furthermore, since the B transmission and the P picture replacement occur in the reproduction transmission apparatus disclosed in Patent Document 3, the frame transmission frequency is significantly different between the reproduction transmission apparatus and the reception apparatus, and a good special reproduction image is obtained in the reception apparatus. Can't get. In addition, when the encoded data to be transmitted is a combination of image data and audio data, the audio data is unduly interrupted in the receiving device, as in the devices disclosed in Patent Literature 1 and Patent Literature 2. To do.

  Furthermore, in the reproduction transmission device disclosed in Patent Document 3, there is no clear definition in the decoding operation when receiving dummy packet information or DIT information and the recording operation on the recording medium, and the operation differs depending on the receiving device. Therefore, there is no guarantee that the special reproduction image can be obtained by the receiving device. Further, when the encoded data to be transmitted is obtained by multiplexing the image data and the audio data, the audio data is interrupted in the receiving device as in the device of Patent Document 1.

  Accordingly, an object of the present invention is to output special reproduction data in a form that can be decoded by an existing decoder.

In order to achieve the above object, the image information transmission apparatus according to the embodiment of the present invention reproduces an encoded image stream encoded by intraframe encoding or interframe predictive encoding from a recording medium for each frame. Means,
Instruction means for instructing a special reproduction mode for reproducing the encoded image stream at a speed slower than a normal reproduction speed;
Changing means for changing timing control data defining display timing of each frame in the encoded image stream reproduced by the reproducing means in response to an instruction from the instruction means;
Copy picture generation means for generating copy picture data which is data instructing display of the same image data as the previous image data;
Stuffing generation means for generating stuffing data for controlling the storage amount of a virtual input buffer of a decoder for decoding the encoded image stream;
Output control means for inserting and outputting the copy picture data and the stuffing data to the encoded image stream output from the change means in response to an instruction from the instruction means, To do.

In order to achieve the above object, an image information transmission method according to another embodiment of the present invention includes:
For each frame, a reproduction step of reproducing an encoded image stream encoded by intraframe encoding or interframe predictive encoding from a recording medium;
An instruction step for instructing a special reproduction mode for reproducing the encoded image stream at a speed slower than a normal reproduction speed;
A change step of changing timing control data defining a display timing of each frame in the encoded image stream reproduced in the reproduction step according to an instruction by the instruction step;
A copy picture generation step of generating copy picture data which is data instructing display of the same image data as the previous image data;
A stuffing generation step of generating stuffing data for controlling a storage amount of a virtual input buffer of a decoder for decoding the encoded image stream;
An output control step of inserting and outputting the copy picture data and the stuffing data to the encoded image stream output in the change step in response to an instruction in the instruction step, To do.

In order to achieve the above object, an image information transmission method according to still another embodiment of the present invention includes an intra-frame encoded picture, and at least one of a forward prediction encoded picture and a bidirectional predictive encoded picture. An accumulation step of accumulating the encoded bitstream;
An output control step for controlling to output the encoded bitstream in an output mode corresponding to a designated special reproduction operation;
A first rewriting step of rewriting control data defining a display timing of a picture of the encoded video bitstream of the encoded bitstream;
A copy picture generation step of generating a copy picture that is data instructing display of the same image data as the previous image data;
A second rewriting step of rewriting control data defining the output sound of the encoded audio bitstream of the encoded bitstream based on the number of output copies of the copy picture,
An encoded bit stream and a copy picture in which the respective control data are rewritten are output according to the control of the output control step.

In order to achieve the above object, an image information transmission method according to still another embodiment of the present invention includes an I picture that is an intra-frame coded picture, a P picture that is a forward predictive coded picture, and a bidirectional predictive coded picture. An accumulation step of accumulating an encoded bitstream including at least any one of the B pictures that are:
An output control step for controlling to output the encoded bitstream in an output mode corresponding to a designated special reproduction operation;
A rewriting step of rewriting control data defining the display timing of a picture with respect to the encoded bitstream;
A copy picture generation step of generating a copy picture that is data instructing to display the same image data as the previous image data,
Outputting the picture of the encoded bitstream with the control data rewritten and the copy picture according to the control of the output control step;
If the encoded bit stream is a multiplexed audio data and image data, the audio data and control information related to the audio data are discarded and output as an encoded bit stream of only image data. It is characterized by that.

  In order to achieve the above object, a program according to still another embodiment of the present invention causes a computer to execute the procedure of the image information transmission method described in the above embodiment.

  In order to achieve the above object, a storage medium according to still another embodiment of the present invention stores a program for causing a computer to execute the procedure of the image information transmission method described in the above embodiment.

  According to the apparatus of the present invention, it is possible to output special reproduction data in a form that can be decoded by an existing decoder. Specifically, a bit stream for special reproduction such as still reproduction, slow reproduction, and frame advance reproduction can be output.

<Embodiment 1>
Embodiment 1 of the present invention will be described below. In the first embodiment of the present invention, image data encoded according to the MPEG standard is reproduced from a recording medium. Then, the reproduced MPEG stream is converted into a form that can be transmitted by a digital interface such as IEEE 1394 and output. The MPEG stream format on IEEE 1394 is assumed to be a transport stream (hereinafter referred to as TS). Also, there are two special playback operation modes implemented in this embodiment: still playback and slow playback. Also, it is assumed that the MPEG stream is in a 60i format in which Repeat_Frist_Field (hereinafter referred to as RF) / Top_Field_First (hereinafter referred to as TF) is common to all frames. The 60i format means a frame rate: 30 / 1.001 Hz and an interlace format.

  FIG. 1 shows the overall configuration of an image information transmission apparatus according to the first embodiment. The transmission device 120 and the display device 121 are connected via an IEEE 1394 interface. Then, the MPEG data transmitted from the transmission device 120 in the TS format is received by the display device 121, reproduced, and displayed.

  The transmission apparatus 120 includes a storage medium 100 that stores MPEG data to be reproduced and a memory 101 that temporarily stores MPEG data acquired from the storage medium. In addition, when the acquired MPEG data is multiplexed, it has a demultiplexer 102 for converting it into an elementary stream (hereinafter referred to as ES). Furthermore, it has a video decoder 103 that decodes the video ES and a monitor 104 that displays the decoded video signal.

  The audio decoder 105 that decodes the audio ES and the speaker 106 that outputs the decoded audio signal are provided. Furthermore, it has a video analysis / rewriting unit 107 for performing header analysis and rewriting of the video ES, and an audio analysis / rewriting unit 108 for performing header analysis and rewriting of the audio ES. In addition, it has a copy picture generation unit 109 that generates a copy picture that is data instructing to display the same image data as the previous image data, and a stuffing generation unit 110 that generates stuffing that is null data. Furthermore, it has a mute generator 111 that generates silent audio data, and a TS multiplexer 112 that multiplexes the video ES and audio ES to generate TS packets. It also includes an IEEE 1394 interface control unit 113, a program memory (not shown), a processor, a user interface, and the like.

  On the other hand, the display device 121 includes an IEEE 1394 interface control unit 114, a TS demultiplexer 115 that generates video and audio ES from TS packets, and a video decoder 116 that decodes the video ES. Furthermore, it has a monitor 117 for displaying the decoded video signal, an audio decoder 118 for decoding the audio ES, and a speaker 119 for outputting the decoded audio signal. Further, a program memory, a processor, a user interface and the like (not shown) are also included.

  When a transmission operation is requested from the transmission apparatus 120, the video signal and audio signal encoded in the MPEG format output from the storage medium 100 are temporarily stored in the memory 101. The MPEG video signal stored in the memory 101 may be in TS format, PS format, or ES format that is not multiplexed.

  When the encoded data stored in the memory 101 is extracted, if the encoded data is multiplexed, the encoded data is separated into video ES and audio ES by the demultiplexer 102 and converted. Then, it is decoded by the video decoder 103 and the audio decoder 105 and output to the monitor 104 and the speaker 106.

  Since the stream format on IEEE 1394 is the TS format, when the MPEG stream recorded in the memory 101 is output by IEEE 1394, the demultiplexer 102 rewrites the information of the video ES and audio ES that have been converted to the ES format.

  That is, if necessary, the video header analysis / rewriting unit 107 and the audio header analysis / rewrite unit 108 are used for decoding time stamp (hereinafter referred to as DTS), presentation time stamp (hereinafter referred to as PTS), TF, Rewrite information such as RF.

  If the special reproduction operation is being performed, the copy picture and the stuffing generated by the copy picture generation unit 109 and the stuffing generation unit 110 are inserted between the pictures of the video ES. Next, the data is multiplexed in the TS format by the TS multiplexer 112, and the result of the TS multiplexer 112 is output via the external interface 113.

  When the display device 121 receives a TS packet from the transmission device 120 via the interface control unit 114, the display device 121 separates and converts it into a video ES and an audio ES by the TS demultiplexer 115. The obtained video ES and audio are decoded by the ES video decoder 116 and the audio decoder 118 and output to the monitor 117 and the speaker 119. The above is the normal operation.

  Next, a copy picture used for the special reproduction MPEG stream will be described. The following two are given as examples of copy pictures.

  The first example is a P picture having the structure shown in FIG. One block in FIG. 2 represents a macro block (hereinafter referred to as MB). Macroblocks at both ends of the slice cannot be omitted according to MPEG standards. Therefore, macroblock_type is MC, NotCoded (MB address information and (0, 0) motion vector is transmitted, and DCT coefficient is not transmitted MB), and all others are skipped macroblocks (hereinafter referred to as SMB). ). The decoder that has received this P picture that is a copy picture outputs a video signal that is the same picture as the previous picture. The P picture, which is a copy picture, is mostly fixed data. For this reason, data stored in advance in a ROM or the like may be read and used, or a sequence header in the bit stream is analyzed to examine the image frame, and a P picture suitable for the image frame is generated. May be used.

  In the second example, when the immediately preceding picture is a B picture, it is assumed that the immediately preceding B picture is duplicated. By sending the previous B picture a plurality of times, the same video signal can be displayed many times on the external display device 121.

  In actual copy picture generation, it is necessary to use the first example for the I picture and P picture, but for the B picture, either the first or second example may be used. In the present embodiment, a case where the copy picture of the second example is used for the B picture will be described.

  Next, still reproduction, which is one of special reproduction operations, will be described with reference to FIG. In the case of normal reproduction, one picture of MPEG data read from the storage medium 100 is read onto the buffer in step S301. If still reproduction is not selected in step S302, the process proceeds to step S303. In step S303, the DTS and PTS, which are system information of pictures on the buffer, are changed so as to be consistent with the picture sent last, and are sent to the display device 121 in step S304. After sending, the process returns to step S301, and by repeating this, normal reproduction can be obtained on the display device 121.

When still playback is requested in response to an instruction from the user, the process proceeds from step S302 to step S305. In step S305, it is determined whether the picture sent last is I, B or P. If it is a B picture, the process proceeds to step S306. However, if it is not a B picture, the process advances to step S307 to prepare a copy picture in the B picture or P picture format described in FIG. In step S308, the header information such as DTS and PTS is changed so that the copy picture prepared in step S306 and step S307 is consistent with the last picture sent out, and is sent to the display device 121 in step S309. In step S310, the stuffing size for the transmitted copy picture is calculated. The size of the stuffing is a size that adjusts the deviation of the accumulation amount of the virtual input buffer (VBV buffer) caused by the copy picture insertion until the stuffing is inserted. This is obtained by the following equation.
(Stuffing data size) =
(Video ES bit rate) x (frame rate) x
(Number of inserted copy pictures)-
(Total data size of the inserted copy picture)
FIG. 4 is a diagram showing the size of stuffing. 4A shows normal playback, FIG. 4B shows a case where the copy picture and stuffing are 1: 1, FIG. 4C shows a case where the copy picture and stuffing are 2: 1, and FIG. 4D shows a copy picture. The case where the stuffing is 4: 1 is shown respectively.

  Therefore, in step S311, stuffing based on the size calculated in step S310 is generated and sent to the display device 121 in step S312. In step S313, it is determined whether the request for still playback has been canceled. If released, the process returns to step S301, and the normal reproduction operation is started. However, if not released, the process returns to step S305 and the still reproduction operation is continued.

  In the flowchart shown in FIG. 3, stuffing data is inserted every time a copy picture of one frame is inserted. However, the stuffing data may be inserted every time an n-frame copy picture is inserted. In this case, generally, as the value of n increases, the size of the stuffing data inserted at one time increases. If the value of n becomes too large, the difference between the target buffer and the amount of data in the virtual buffer due to the accumulation of copy pictures increases. Therefore, the value of n is appropriately determined according to the amount of data stored in the additional buffer at the start of still playback. Is preferred.

  When shifting from still reproduction to normal reproduction, it is necessary to adjust the storage amount of the virtual input buffer of the decoder by stuffing so that the reproduced image is not disturbed. For this reason, it is assumed that transition from still playback to normal playback is performed in units of n frames. Therefore, even if the user requests cancellation of still playback in a period shorter than n frames, the cancellation time is internally delayed and is not canceled until n frames where stuffing enters. In the case of n-to-one, step S305 to step S309 are looped n times, and then the process proceeds to step S310. By repeating the above, the display device 121 can obtain a still reproduction image.

  Next, as an example of slow playback, 1 / 3-speed slow playback (sending one copy of normal picture and sending two copy pictures) will be described with reference to the flowchart of FIG. In the case of normal reproduction, one picture of the MPEG data read from the storage medium 100 in step S501 is read onto the buffer. In step S502, DTS and PTS, which are header information of pictures on the buffer, are changed so as to be consistent with the last picture sent out, and are sent to the display device 121 in step S503. If the slow playback is not selected in step S504, the process returns to step S501 after the transmission, and normal playback can be obtained on the display device 121 by repeating this.

  When slow playback is requested by an instruction from the user, the process proceeds from step S504 to step S505. First, in step S505, a counter m indicating how many copy pictures have been sent is initialized. In step S506, it is determined whether or not two copy pictures have been sent for sending one picture of MPEG actual video data. If it has been sent, the process returns to step S501. If it has not been sent, the process advances to step S507.

  Step S507 is the same as steps S305 to S312 in the still reproduction flowchart described with reference to FIG. Since the copy picture / stuffing insertion ratio n is a natural number less than m (= 3), it is 1 or 2 when m is 3. In step S508, the copy picture insertion number counter is updated by adding the copy picture / stuffing insertion ratio n to the copy picture insertion number counter m, and the process returns to step S506. By repeating the above, slow playback can be obtained on the display device 121.

  In addition, the method for calculating the size of the stuffing data is the same as that in the above-described still playback during slow playback. During slow playback, the number m of copy picture insertion frames is determined by the playback speed during slow playback. Therefore, the number n of frames into which the stuffing data is inserted is m or less.

  Next, regarding frame-by-frame playback, frame-by-frame playback is internally realized by repeating still playback every time one frame is sent in normal playback. Details are the same as in still playback, and will be omitted.

  Audio muting may or may not be performed. A method for muting when audio is muted will be described later (from the fourth embodiment).

<Embodiment 2>
In the first embodiment, the case where the frames having the same RF / TF such as MPEG data are continuous has been described, but in the second embodiment, the case where the RF / TF is pulled down 2-3 is different for each frame.

  FIG. 6 shows an example of a video stream pulled down 2-3. 2-3 pull-down video streams have four types of RF / TF: {RF, TF} = {1,1}, {0,0}, {0,1}, {1,0} in display order It is the structure which repeats a frame. Hereinafter, for the sake of explanation, the above four types of frames are described in order as Frame 1, Frame 2, Frame 3, and Frame 4.

  In order to protect the sequence of these four types of frames during special playback operation, copy pictures are inserted in units of 4 frames, and the transition from special playback operation to normal playback operation is internally performed until the output of these 4 frames is completed. Delay.

A specific copy picture is generated and inserted by the last picture output in the display order.
If it was Frame 0, Frame 1 → Frame 2 → Frame 3 → Frame 0
If it was Frame 1, Frame 2 → Frame 3 → Frame 0 → Frame 1
If it was Frame 2, Frame 3 → Frame 0 → Frame 1 → Frame 2
If it was Frame 3, Frame 0 → Frame 1 → Frame 2 → Frame 3
In this order, copy pictures are inserted in units of four.

Since the transition from the special reproduction to the normal reproduction is a transition of every (4 × x), the insertion relationship n between the copy picture and the stuffing may be any divisor of 4 × x. (When x = 1 [copy picture insertion unit = every 4], stuffing is inserted every copy picture 1, 2, 4)
FIG. 7 shows an example of copy picture insertion when x = 1.

  Although the specific operation during the special playback operation is the same as that in the first embodiment, the description is omitted, but the playback speed in the case of the slow playback is 1 / (4 × x + 1) times faster.

<Embodiment 3>
Although the second embodiment has been described with reference to 2-3 pulldown, the third embodiment will be described with reference to 2-3-3-2 pulldown.

  FIG. 8 shows an example of a video stream pulled down 2-3-3-2. The 2-3-3-2 pulled-down video stream has an RF / TF in the display order {RF, TF} = {0,1}, {1,1}, {1,0}, {0,1} The four types of frames are repeated. Hereinafter, for the sake of explanation, the above four types of frames are described in order as Frame 0, Frame 1, Frame 2, and Frame 3.

  During the special playback operation, in order to protect the sequence of these four types of frames, copy pictures are inserted in units of 4 frames, and the transition from the special playback operation to the normal playback operation is internally performed until the output of 4 units is completed. Delay.

The specific copy picture generation / insertion process is the last picture output in the display order.
If it was Frame 0, Frame 1 → Frame 2 → Frame 3 → Frame 0
If it was Frame 1, Frame 2 → Frame 3 → Frame 0 → Frame 1
If it was Frame 2, Frame 3 → Frame 0 → Frame 1 → Frame 2
If it was Frame 3, Frame 0 → Frame 1 → Frame 2 → Frame 3
In this order, copy pictures are inserted in units of four.

Since the transition from the special reproduction to the normal reproduction is a transition of every (4 × x), the insertion relationship n between the copy picture and the stuffing may be any divisor of 4 × x. (When x = 1 [copy picture insertion unit = every 4], stuffing is inserted every copy picture 1, 2, 4)
FIG. 9 shows an example of copy picture insertion when x = 1. The specific operation during the special playback operation is the same as in the first and second embodiments and is omitted, but the playback speed in the case of the slow playback is 1 / (4 × x + 1) times faster.

<Embodiment 4>
In the first, second and third embodiments, the video stream has been described. In the fourth embodiment, audio muting will be described. Note that the processing for the video stream is the same as in the first, second, and third embodiments, and is therefore omitted.

The audio mute operation will be described with reference to the process flowchart of FIG. 10 and the output stream image of FIG. In the case of normal reproduction (A in FIG. 11), header information such as PTS is changed in step S1001. The PTS is changed by adding the difference value between the PTS of the actual stream and the output stream obtained by changing the PTS of the video stream to the PTS of the audio stream. ((B) in FIG. 11) Next, in step S1002, it is determined whether or not the audio data is specially reproduced. The specific determination method is to acquire the PTS of the video stream being specially played,
(1) Audio data having PTS after the PTS of the special playback start video stream (2) Audio data having PTS before the PTS of the special playback end video stream is determined as audio data during special playback. (B of FIG. 11). In the case of normal reproduction, since there is no corresponding audio data, the process proceeds to step S1004. In step S1004, audio is output. By repeating this, audio is output during normal playback.

  Next, operations during special playback such as still playback, frame advance playback, and slow playback will be described. If it is determined in step S1002 that the audio data is during special reproduction, it is replaced with mute data in step S1003 (C in FIG. 11). The mute data is obtained by sampling silence at a sampling frequency and encoding it in a predetermined format. If the sampling frequency is indefinite, it may be generated each time. However, if the sampling frequency can be specified, the mute data is fixed data, so that data prepared in advance in a ROM or the like may be used. . Since the operation after the mute data replacement is the same as that during normal reproduction from step S1004, the description thereof is omitted. By repeating the above, muting is realized.

<Embodiment 5>
Although the video stream has been described in the fourth embodiment, simple insertion of audio mute data will be described in the fifth embodiment. Since the processing for the video stream is the same as in the first to third embodiments, the description thereof is omitted. Since audio muting is the same as in the fourth embodiment, a description thereof will be omitted.

  The audio mute data insertion operation will be described with reference to the process flowchart of FIG. 12 and the output stream image of FIG. In step S1201, the audio PTS is changed in the difference value based on the copy picture insertion described in the fourth embodiment. In step S1202, the lack of audio is confirmed. A specific method for confirming the lack of audio is as follows. The PTS interval of an audio access unit (hereinafter referred to as AAU) that is the audio coding unit is obtained from the audio ES. If there is no AAU having a PTS within a range separated from the AAU by the aforementioned PTS interval, it is determined that there is a shortage, and the process proceeds to step S1203. Otherwise, the process proceeds to step S1205. In step S1203, mute data is inserted (B in FIG. 13). Similar to the fourth embodiment, the mute data is obtained by sampling silence at a sampling frequency and encoding it in a predetermined format. If the sampling frequency is indefinite, it may be generated each time, but if the sampling frequency can be specified, it may be prepared in advance in a ROM or the like. When the mute data is inserted, the process proceeds to step S1204. Then, the PTS of the audio AAU is changed from the PTS of the audio AAU by adding the AAU amount PTS (C in FIG. 13), and the process proceeds to step S1205. In step S1205, audio is output. By repeating the above, mute audio including discontinuities less than 1 AAU is generated.

<Embodiment 6>
In the fifth embodiment, simple mute data insertion has been described. In the sixth embodiment, continuous mute audio generation by mute data insertion and system information update will be described. The processing for the video stream and the audio muting are the same as those in the fifth embodiment, and are therefore omitted.

  The audio mute insertion operation will be described with reference to the process flowchart of FIG. 14 and the output stream image of FIG. First, in step S1401, the audio PTS is changed in the difference value based on the copy picture insertion as in the fourth and fifth embodiments (A in FIG. 15). In step S1402, the continuity of the audio is confirmed. A specific method for confirming the continuity of audio is as follows. The PTS interval of AAU that is the audio encoding unit is obtained from the audio ES. If there is no AAU having a PTS at a position separated from the AAU by the aforementioned PTS interval, it is determined that there is no continuity, and the process advances to step S1403. If it is determined that there is continuity, the process advances to step S1408 to output audio data.

  In step S1402, it is determined that there is no continuity, and in step S1403, mute data is inserted. The insertion of mute data is the same as in the fifth embodiment, and is therefore omitted. Next, in step S1404, it is determined whether or not there is an AAU having a PTS within 1 AAU from the inserted PTS. If not, the process returns to step S1403 to insert a mute again. However, if there is, the insertion of mute is stopped, and the process proceeds to step S1405 (B in FIG. 15). Next, in step S1405, the PTS is changed by inserting mute data. In step S1406, if the range of change exceeds the range of 1 AAU, an audio and video synchronization shift occurs due to the mute data insertion being over. Therefore, the previous mute data is deleted in step S1407, and the PTS is changed again in S1406 (C in FIG. 15). By repeating the above, continuous mute audio is generated (D in FIG. 15).

<Embodiment 7>
In the fourth to sixth embodiments, the audio muting has been described. In the seventh embodiment, the audio discard in the special reproduction operation will be described. Since the processing for the video stream is the same as in the first to third embodiments, the description thereof is omitted.

  Audio Discarding Next, the audio mute insertion operation will be described with reference to the processing flowchart of FIG. 16 and the output stream image of FIG. If special playback is not selected in step S1601, the process advances to step S1603. In step S1603, header information such as PTS is changed. The PTS is changed by adding the difference value between the PTS of the actual stream and the output stream obtained by changing the PTS of the video stream to the PTS of the audio stream. In step S1604, audio is output. By repeating this, audio is output during normal playback.

  Next, special playback operations such as still playback, frame advance playback, and slow playback will be described. If it is determined in step S1601 that special playback is to be performed, the process proceeds to step S1602. In step S1602, the audio during the special playback operation is discarded (B in FIG. 17). A specific method for determining the audio data during the special operation is as follows. Check the frame output during special playback operation for the video stream in the buffer. Next, an AAU having a PTS in the section corresponding to the PTS of the frame is selected, and this is targeted for discard (C in FIG. 17). Audio discard is realized by repeating the above.

  The object of the present invention can also be achieved by supplying a system or apparatus with a storage medium storing software program codes for realizing the functions of the above-described embodiments. That is, it goes without saying that this can also be achieved by the computer (or CPU or MPU) of the system or apparatus reading and executing the program code stored in the storage medium. In this case, the program code itself read from the storage medium realizes the functions of the above-described embodiments, and the storage medium storing the program code constitutes the present invention.

  As a storage medium for supplying the program code, for example, a flexible disk, hard disk, optical disk, magneto-optical disk, CD-ROM, CD-R, magnetic tape, nonvolatile semiconductor memory card, ROM, or the like can be used. . Further, the functions of the above-described embodiments may be realized by executing the program code read by the computer.

  However, when the OS (operating system) operating on the computer performs part or all of the actual processing based on the instruction of the program code, and the functions of the above-described embodiments are realized by the processing. Needless to say, is also included.

  Furthermore, the program code read from the storage medium may be written in a memory provided in a function expansion board inserted into the computer or a function expansion unit connected to the computer. After that, based on the instruction of the program code, the CPU of the function expansion board or function expansion unit performs part or all of the actual processing, and the processing of the above-described embodiment is realized by the processing. Needless to say.

It is a block diagram which shows one Embodiment of this invention. It is a figure explaining the copy picture in embodiment of this invention. It is a flowchart which shows the operation | movement of still reproduction in embodiment of this invention. It is a figure which shows the size of the stuffing in embodiment of this invention. It is a flowchart which shows the operation | movement of slow reproduction in embodiment of this invention. It is a figure which shows the bit stream of 2-3 pulldown in embodiment of this invention. Similarly, it is a figure which shows copy picture insertion to the bit stream of 2-3 pulldown. Similarly, it is a figure which shows the bit stream of 2-3-3-2 pulldown. Similarly, it is a figure which shows copy picture insertion to the bit stream of 2-3-3-2 pulldown. It is a flowchart which shows the operation | movement of the mute data replacement in embodiment of this invention. Similarly, it is a figure which shows the output image of mute data replacement. Similarly, it is a flowchart showing an operation of inserting mute data. Similarly, it is a figure which shows the output image of mute data replacement and mute data insertion. Similarly, it is a flowchart showing an operation of inserting mute data. Similarly, it is a figure which shows the output image of mute data replacement and mute data insertion. It is a flowchart which shows the operation | movement of audio discard in embodiment of this invention. Similarly, it is a figure which shows the output image of audio discard.

Claims (21)

Reproducing means for reproducing, from the recording medium, an encoded image stream encoded by intraframe encoding or interframe predictive encoding for each frame;
Instruction means for instructing a special reproduction mode for reproducing the encoded image stream at a speed slower than a normal reproduction speed;
Changing means for changing timing control data defining display timing of each frame in the encoded image stream reproduced by the reproducing means in response to an instruction from the instruction means;
Copy picture generation means for generating copy picture data which is data instructing display of the same image data as the previous image data;
Stuffing generation means for generating stuffing data for controlling the storage amount of a virtual input buffer of a decoder for decoding the encoded image stream;
Output control means for inserting and outputting the copy picture data and the stuffing data to the encoded image stream output from the change means in response to an instruction from the instruction means, Image information transmission device. The output means outputs the stuffing data every time the copy picture data is inserted for n (n is an integer of 1 or more) frames,
The stuffing generation means determines the size of the stuffing data according to a data rate of the encoded image stream, a value of the n, and a data amount of copy picture data for the n frames. The image information transmission apparatus according to claim 1.   The image information transmission apparatus according to claim 1, wherein the special reproduction mode includes at least one of a still reproduction mode and a slow reproduction mode.   The image information transmission apparatus according to claim 2, wherein the special reproduction mode is a slow reproduction mode, and the value of n is determined according to a reproduction speed in the slow reproduction mode. The encoded image stream is in a 30 frame / second interlaced format with 2-3 pulldown or 2-3-3-2 pulldown,
2. The image information transmission apparatus according to claim 1, wherein the output control means inserts the stuffing data every time two frames of the copy picture data are inserted. The encoded image stream is in a 30 frame / second interlaced format with 2-3 pulldown or 2-3-3-2 pulldown,
2. The image information transmission apparatus according to claim 1, wherein the output control means inserts the stuffing data every time four frames of the copy picture are inserted. The encoded image stream is in an interlace format of 30 frames / second pulled down 2-3.
The copy picture generation means generates the Top_Field_First and Repeat_First_Field values of the copy picture so as to maintain a 2-3 pull-down relationship with respect to the Top_Field_First and Repeat_First_Field values of the frame output immediately before. The image information transmission apparatus according to claim 1. The encoded image stream is an interlace format of 30-frame / second pulled down 2-3-3-2.
The copy picture generation means maintains the 2-3-3-2 pull-down relationship between the Top_Field_First and Repeat_First_Field values of the copy picture and the Top_Field_First and Repeat_First_Field values of the frame output immediately before. The image information transmission device according to claim 1, wherein the image information transmission device generates and outputs the image information. The encoded image stream is in a 30 frame / second interlaced format with 2-3 pulldown or 2-3-3-2 pulldown,
The image information transmission apparatus according to claim 1, wherein the n is a multiple of four. The encoded image stream includes encoded audio data;
2. The image information according to claim 1, wherein the rewriting unit further rewrites audio control data defining a sound output of the encoded audio data in the encoded image stream in accordance with an instruction from the instruction unit. Transmission equipment.   The image information transmission apparatus according to claim 10, wherein the encoded audio data in the encoded bitstream is replaced with encoded silence audio data and output.   11. The image information transmission apparatus according to claim 10, wherein the encoded silent audio data is inserted into a discontinuous portion of the encoded audio bitstream and output.   The control data defining the output sound of the encoded audio bitstream is rewritten in correspondence with the insertion of the encoded silent audio data into discontinuous portions of the encoded audio bitstream. 12. The image information transmission device according to 12. The encoded image stream includes encoded audio data;
The output control means discards the encoded audio data and the control information related to the encoded audio data in the encoded image stream output from the changing means in response to an instruction from the instruction means, and 2. The encoded picture data and the stuffing data are inserted and output to an encoded image stream in which encoded audio data and control information related to the encoded audio data are discarded. The image information transmission apparatus described. For each frame, a reproduction step of reproducing an encoded image stream encoded by intraframe encoding or interframe predictive encoding from a recording medium;
An instruction step for instructing a special reproduction mode for reproducing the encoded image stream at a speed slower than a normal reproduction speed;
A change step of changing timing control data defining a display timing of each frame in the encoded image stream reproduced in the reproduction step according to an instruction by the instruction step;
A copy picture generation step of generating copy picture data which is data instructing display of the same image data as the previous image data;
A stuffing generation step of generating stuffing data for controlling a storage amount of a virtual input buffer of a decoder for decoding the encoded image stream;
An output control step of inserting and outputting the copy picture data and the stuffing data to the encoded image stream output in the change step in response to an instruction in the instruction step, Image information transmission method. An accumulation step of accumulating an encoded bitstream including at least one of an intra-frame encoded picture, a forward prediction encoded picture, and a bidirectional predictive encoded picture;
An output control step for controlling to output the encoded bitstream in an output mode corresponding to a designated special reproduction operation;
A rewriting step of rewriting control data defining a display timing of a picture of the encoded bitstream;
A copy picture generation step of generating a copy picture that is data instructing display of the same image data as the previous image data;
A stuffing generation step for generating stuffing that is null data for controlling the storage amount of the virtual input buffer of the decoder, and
A picture information transmission method, comprising: outputting a picture of the encoded bitstream, the copy picture, and the stuffing, in which the control data is rewritten, according to control of the output control step. In the output mode corresponding to the specified special reproduction operation,
The stuffing is output every n copies of the copy picture,
The size of the stuffing is a size that reduces an increase in the amount of storage in the virtual input buffer generated by inserting n (n = 1, 2,...) Copy pictures. The image information transmission method according to claim 16. An accumulation step of accumulating an encoded bitstream including an intra-frame encoded picture and at least one of a forward prediction encoded picture and a bidirectional predictive encoded picture;
An output control step for controlling to output the encoded bitstream in an output mode corresponding to a designated special reproduction operation;
A first rewriting step of rewriting control data defining a display timing of a picture of the encoded video bitstream of the encoded bitstream;
A copy picture generation step of generating a copy picture that is data instructing display of the same image data as the previous image data;
A second rewriting step of rewriting control data defining the output sound of the encoded audio bitstream of the encoded bitstream based on the number of output copies of the copy picture,
An image information transmission method comprising: outputting an encoded bit stream and a copy picture in which each of the control data is rewritten according to control of the output control step. An accumulation step of accumulating an encoded bitstream including at least one of an I picture that is an intra-frame encoded picture, a P picture that is a forward predictive encoded picture, and a B picture that is a bidirectional predictive encoded picture;
An output control step for controlling to output the encoded bitstream in an output mode corresponding to a designated special reproduction operation;
A rewriting step of rewriting control data defining the display timing of a picture with respect to the encoded bitstream;
A copy picture generation step of generating a copy picture that is data instructing to display the same image data as the previous image data,
Outputting the picture of the encoded bitstream with the control data rewritten and the copy picture according to the control of the output control step;
If the encoded bit stream is a multiplexed audio data and image data, the audio data and control information related to the audio data are discarded and output as an encoded bit stream of only image data. An image information transmission method characterized by the above.   A program for causing a computer to execute the procedure of the image information transmission method according to any one of claims 15 to 19.   A computer-readable storage medium storing a program for causing a computer to execute the procedure of the image information transmission method according to any one of claims 15 to 19.

Images