JPH06275053A - Method for synchronous reproduction of compressed image data containing display time information of images as data and acoustic information data compressed at constant compression ratio without containing data of time information - Google Patents

Abstract

PURPOSE:To synchronously reproduce an image and a sound appended to the image even on the way of reproduction by shifting the reproduction start time of image information and acoustic information from the time difference generated by a decoder for image information and acoustic information. CONSTITUTION:A regenerative signal source 1 sends the regenerative object data string of both the compressed image data, which also the time information to be displayed as one part of data, and the acoustic data compressed at a certain rate without having the time information to be outputted as data although the start is coincident with the start of the image, to a transmission line 9. In the case of reproduction, when the time difference is defined to be already known between the time to first simultaneously to be outputted and the time when the sound appended to the image is outputted, the time output the acoustic data at the time point of reproduction being started on the way of time series is calculated. Based on this calculated value, the preferential order of sending the image data and the acoustic data is decided, and those data are sent while fixing the shift time of reproduction starting time so that both of data can be synchronized.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION The present invention relates to compressed image data that also includes time information to be displayed as part of the data, and the start of the compressed image data matches the start of the image, but the output time information is used as data. Even if a bitstream in which audio data compressed at a constant rate that is not possessed is interleaved in time series is reproduced from the middle, it is possible to easily reproduce the image and the sound accompanying the image in synchronization with each other. Synchronous playback method to do so.

[0002]

2. Description of the Related Art Researches for highly efficient compression of image signals for transmission, recording and reproduction have been actively conducted. For example, image data highly efficient encoded on a small disc,
A bit stream obtained by time-sequentially interleaving the image data and the acoustic data obtained by highly-efficiently encoding the acoustic signal associated with the corresponding image is recorded as data in a state illustrated in FIG. 6, for example. , Transmission was started. In FIG. 6, a section marked with V is a block (sector) of image data compressed with high efficiency, and a section marked with A is a block of acoustic data compressed with high efficiency. (Sector). By the way, regarding compression of data amount by high-efficiency encoding of image signals, with the aim of creating an international standard for compressing image data by high-efficiency encoding of image signals of moving images, MPEG (Moving Picture)
Coding Group) successively proposes various data formats relating to highly efficient compressed image data, and highly efficiently compresses image data in accordance with the data format proposed by MPEG (compressed image signal highly efficient compressed by the MPEG method. ) Is being researched and developed for a practical device for transmitting, recording and reproducing.

A moving picture information coding system (MPEG system) intended for a recording medium for recording digital data such as a CD-ROM employs a predictive coding method. An image in which image data is compressed by applying the inner prediction method {I picture (I
The image data is compressed by applying an inter-frame prediction method in which inter-frame prediction is performed based on image data of a frame (hereinafter, also referred to as an I frame) and image data of a past frame. Image {P picture (Predicted Pi
frames) (hereinafter also referred to as P-frames) and inter-frame prediction based on image data of both past frame image data and future frame image data Image whose image data has been compressed by applying the method {B picture (Bi-directional Pred
motion pictures) frame (hereinafter referred to as B
(Although it may be called a frame), each frame in three types of image modes is arranged in a predetermined arrangement on the time axis, and a predetermined header is added to the digital data to obtain image encoded data. .

In the MPEG system, the relationship between the image data compression rate in the I frame, the image data compression rate in the P frame, and the image data compression rate in the B frame is as follows. Compression rate) <(compression rate of image data in P frame) <(compression rate of image data in B frame)
The relationship is such that the reproduction is performed from the sequence header of the entry point at the time of reproduction, and the prediction is performed using the past image information and the image information of the future frame. In order to reproduce the image information of the B frame, the image information of the future P frame used for predicting the image information of the B frame needs to be recorded before the B frame.

FIG. 2 is a view for explaining the arrangement of data relating to the highly efficient compressed moving image information when the highly efficient compressed moving image information is recorded on an optical disc conforming to the CD (Compact Disc) standard. 2 (b) is an arrangement of recording data in a sequential sector portion in which data relating to moving image information compressed with high efficiency is recorded on an optical disc conforming to the CD (Compact Disc) standard. It shows the state. First, in FIG. 2A, following the MPEG system header portion in each sector shown in FIG.
The contents of the data to be recorded sequentially are recorded in one GOP.
(Group of Pictures) is illustrated. The GOP has a sequence header placed at the beginning of the GOP, a GOP header placed after the sequence header,
A sequential image frame is arranged subsequent to the GOP header.

2C to 2F show the concrete contents of the MPEG system header shown in FIG. 2B. The PACK header in each of the above figures is C
SCR (system) recorded for each sequential sector of D
value of em clock reference {constant number 1200 (MPEG) for each sector of the compact disc
1 / used as a unit of time measurement in the method
The value is increased by 90 KHz / 75 Hz = 1200) depending on the cycle of 90 KHz and the number of sectors per second of CD is 75}, and other information is also included. DTS is a time stamp, and the two types of time stamps P described above.
One of the time stamps PTS (pres) of TS and DTS
(entation time stamp) is information indicating the time when the image is actually displayed, and the other time stamp DTS.
(Decoding time stamp) is time information indicating the time at which data is sent to the MPEG video decoder.

Incidentally, there are many kinds of concrete contents of the MPEG system header as shown in (c) to (f) of FIG. 2 that the time contained in the MPEG system header is included. Depending on the presence or absence of a stamp and the type of existing time stamp, the MP
This is because the contents of the image information recorded in the sector in which the EG system header exists and the recording mode can be indicated. The concrete contents of the MPEG system header are as shown in (c) and (d) of FIG.
In the system header, two types of time stamp PTS,
If both of the DTS's are present, it means that the encoded I frame or P frame starts within the sector in which the MPEG system header is placed. The MPEG system header having the contents as shown in FIG.
It shows that a plurality of GOPs as exemplified in (a) of the above is placed in the first sector in each video sequence.

Further, when only the time stamp PTS exists in the MPEG system header as shown in FIG. 2 (e), the encoded B frame is included in the sector in which the MPEG system header is placed. 2 has started, and as shown in (f) of FIG.
If neither of the time stamps PTS and DTS exists in the system header, the I frame, P, and P are stored in the sector in which the MPEG system header is placed.
This means that it does not include the boundary of the beginning of any image frame of the frame and the B frame. In addition to the sequence header code, the sequence header is
Information about horizontal and vertical size of the image, aspect ratio,
The GOP header is composed of various other information, and the GOP header includes a group start code, a time code, and information indicating whether or not it is a closed GOP, which is arranged prior to the beginning of a GOP (Group of Pictures).
Link (when it is set to 1, for a B frame existing between an I frame and a P frame forming a GOP to which the GOP header is attached,
It has a role of preventing the MPEG video decoder from performing a decoding operation), and other various information. The GOP is composed of I frame image data, P frame image data, and B frame image data group.
Immediately after the above-mentioned GOP header, the I-frame image data is always located.

[0009]

Now, high-efficiency-coded image data and time-interleaved high-efficiency-coded acoustic data associated with the image data corresponding to the image data are interleaved in time series. The bitstream
For example, when recording and transmitting as data in the state as illustrated in FIG. 6, the high-efficiency-encoded image data is like the high-efficiency-encoded image data according to the MPEG system. When the data amount of the image data corresponding to each image is different, the interval on the time axis between the image data corresponding to the successive images changes corresponding to the difference in the data amount of each image described above. There is. Therefore, for example, when the moving image information highly compressed by the MPEG system is recorded on the optical disc conforming to the CD (Compact Disc) standard, it is added to the data of the highly efficient compressed moving image information. The numerical value indicated by the time information data (for example, the time information data PTS to be displayed already described as the time stamp) is the arrangement of the P frame and B frame described above based on the moving image information data on the time axis. Since it may be reversed, it does not monotonically increase, which is illustrated by the curve V in FIG.

However, when the image information is recorded on the CD, the average value of the time stamp PTS values for a long time is set to change linearly. A straight line Va shown by a dotted line in FIG. 5 shows how the value of the time stamp PTS increases over a long time. The straight line Va above the straight line Va shown by the dotted line and the time stamp on the time axis. The area of the region surrounded by the curve V indicating the value of PTS and the area of the region surrounded by the straight line Va below the straight line Va are equal to each other, and will be described later. It is parallel to the straight line A. That is, if a long time is taken, the average value of the time progress of the image information is made to be substantially linear. on the other hand,
Since the time position of the acoustic data obtained by high-efficiency encoding the acoustic signal accompanying the image information at a constant compression rate is linearly arranged on the time axis, the time stamp PTS is added to the acoustic data. If so, the time stamp P
The value of TS is in the state of being located on the straight line A in FIG. The straight line shown as SCR = PTS in FIG. 5 indicates a change in the numerical value of the SCR in which the numerical value increases by 1200 in the sequential sectors of the CD. Then, the straight line SCR, the straight line A, and the straight line V described above.
a is parallel to each other. Further, the interval between a and v shown on the vertical axis in FIG. 5 indicates the time difference between the decoding time required by the audio decoder and the decoding time required by the video decoder.

By the way, for example, an optical disc conforming to the CD (Compact Disc) standard is used for moving image information sand M.
A sound obtained by high-efficiency-encoding the variable compression rate data of the moving image information obtained by high-efficiency encoding by the PEG system and the acoustic information accompanying the moving image information at a constant compression rate by, for example, ADPCM. Compressed data of information, a bit stream interleaved in time series,
For example, when the data is recorded as the data illustrated in FIG. 6, that is, the compressed image data including the time information (time stamp PTS) to be displayed as a part of the data and the start thereof are the start of the image. However, if the audio data compressed at a constant rate that does not have the time information to be output as data is recorded as a bit stream that is interleaved in time series, the CD is played halfway. When this happens, there may be a discrepancy in the reproduction timing between the image and the sound accompanying the image.

The above points will be specifically described as follows. As shown in FIG. 6, for example, the image information data and the audio information data recorded on the CD are indicated by the reference numeral A, and the audio information data is recorded in the sector A. And the sector V in which the image information data indicated by the reference symbol V is recorded are V → V →
If it is V → A → V → V → V → A →, a code is added between the following sectors A in which the audio information data indicated by the code A in FIG. 6 is recorded. Since there are three sectors V in which the image information data indicated by V are recorded, the SCR values in the sectors A appearing one after another on the time axis are as shown in FIG. Sector contents correspond to A in the contents column, and SCR in FIG.
As indicated by the numerical value in the column, the SCR value of each sector A is increased by (4800). Therefore, the acoustic information data indicated by the reference symbol A in FIG. The SCR value of each sector A in which
It will be located on the straight line SCR = PTS inside. That is, since the acoustic data recorded in each sector A is data in a state of being compressed at a constant compression rate, the time of the acoustic information corresponding to the acoustic data sequentially recorded in each sector A The position on the axis corresponds to the SCR value in the state of linearly increasing on the time axis.

However, the image information recorded in the sector V in which the data of the image information indicated by the symbol V in FIG. 6 is recorded is the moving image information encoded with high efficiency by the MPEG system. The obtained data of the variable compression ratio of the moving image information, that is, the relationship between the compression ratio of the image data in the I frame, the compression ratio of the image data in the P frame, and the compression ratio of the image data in the B frame is (image in the I frame Data compression ratio) <(compression ratio of image data in P frame) <(compression ratio of image data in B frame) is added to the image data described above. The value of the time stamp PTS placed in the MPEG header corresponding to the compression rate of the image data corresponds to the value on the time axis as illustrated by the curve V in FIG. It becomes what is changing in a linear state. In the column of time stamp PTS in FIG. 4,
As illustrated in FIG. 6, the sector A in which the image information data and the audio information data recorded on the CD are indicated by the reference character A is recorded.
And a sector V in which the data of the image information indicated by the reference symbol V is recorded are V → V → V → A → V → V → V.
The value of the time stamp PTS recorded in the sequential sectors in the case of → A → is illustrated.

As described above, the compressed image data including the time information (time stamp PTS) to be displayed as a part of the data and the start of the compressed image data coincides with the start of the image, but the time information to be output. In the case where a bit stream in which time-sequentially interleaved sound data that is compressed at a constant rate and does not have is recorded, the sound information indicated by reference numeral A in FIG. Since the sector A in which the data is recorded is arranged every four sectors, the time information data (time stamp PTS) is not added to the audio information data sequentially recorded in the sector A. Even so, since the acoustic information data is compressed at a constant compression rate as described above, the sector in which the acoustic data is recorded starts at the start described above. What is the ordinal number of the sector from the sector, if you bought the one of the information determine numerical value of the SCR is the number of the beginning of the sector, it is possible to give a virtual time stamp PTS to the data of the acoustic information.

Now, for example, a sector A in which acoustic information data indicated by reference numeral A is recorded and a sector V in which image information data indicated by reference numeral V is recorded. However, as illustrated in FIG. 6, for example, V →
It is assumed that it is like V → V → A → V → V → V → A →, the first sector 0 in the sector V in which the image information data is recorded (in the sector number column in FIG. Timestamp PT of sector numbered 0)
If the value of S is 0 + 10000 (the number 10000 is an offset value), the sequential sector A
The value of the virtual time stamp PTS given to the acoustic data in is as shown by the following expression (1). Value of virtual time stamp PTS = 1200 ×
(Sector number -3) +10000 (1) However, 10000 in the equation (1) is the offset value. In addition, since the above-mentioned sector number and SCR numerical value have a primary relationship, the sector number is (2)
Will be shown by the formula. Sector number = SCR / 1200 (2) Then, the value of the virtual time stamp PTS is as described above.
From equations (1) and (2), the following equation (3) is obtained. Virtual time stamp PTS = SCR-3600 + 10000 (3) However, 10000 in the equation (3) is the offset value described above. A straight line A in FIG. 5 shows the relationship between the SCR value and the virtual time stamp PTS value.

On the other hand, when the time stamp PTS is added to the image information recorded in the sector V in which the data of the image information indicated by the reference symbol V is recorded, that sector PTS is added. Although the value of the time stamp PTS is recorded, an example of the value of the time stamp PTS for each sector is described in the time stamp PTS column in FIG. It should be noted that, in the column of the time stamp PTS in FIG. 4, a sector described as “not present” is an MPEG added to the image information recorded in the sector.
This is the case where the header has a structure as shown in FIG. As described above, the image information data recorded in the sector V in which the image information data indicated by the reference symbol V is recorded is obtained by high-efficiency encoding according to the MPEG system. If the compression ratios of the image frames are different, as in the case of frames, P frames, and B frames, MP added to the image data described above.
The value of the time stamp PTS placed in the EG header is
As illustrated by the curve V in FIG. 5, the image data changes to a non-linear state on the time axis in correspondence with the compression rate of the image data. Therefore, the SCR value and the time stamp P
The relation with the value of TS is not a relation of linear function.

Therefore, the compressed image data including the time information (time stamp PTS) to be displayed as described above as a part of the data and the start of the compressed image data match the start of the image, but the output time When a bitstream in which acoustic data that does not have information as data and is compressed at a constant rate is interleaved in time series is recorded, the image and the image are reproduced when the bitstream is reproduced from the middle. There will be a gap in the timing of the reproduction of the sound accompanying the. Therefore, a solution was sought to prevent the above problems from occurring.

[0018]

DISCLOSURE OF THE INVENTION According to the present invention, compressed image data including time information to be displayed as part of the data and time information to be output whose start coincides with the start of the image, are output. The audio data compressed at a constant rate that does not have as data is interleaved in a time series, and the time at which the images to be output at the same time should be displayed first and the accompanying time of the images When the time difference from the time when the sound should be output is known, when the bit stream is played back from the middle, the time when the audio data should be output at the time when the halfway playback is started on the time series is set. , The display time of the image calculated from the above-mentioned known time difference, and the reproduction start time of the sound and the image are shifted based on the calculated value so as to synchronize the both. The compressed image data including the even data broadcast, there is provided a synchronous reproduction method of the audio information data compressed at a fixed compression ratio that does not contain the data of the time information.

[0019]

With the compressed image data including the time information to be displayed as a part of the data and the start of the compressed image data coincides with the start of the image, the compressed image data is compressed at a constant rate without having the output time information as data. The time difference between the time when the first image to be simultaneously output should be displayed and the time when the sound accompanying the image should be output in the bit stream in which the audio data is interleaved in time series. In the case where the bit stream is reproduced from the middle when the above is known, first, as in step (1) in the flowchart shown in FIG. 3, considering the delay time generated in the audio decoder and the video decoder, The value of the virtual time stamp PTS obtained by the above equation (3) for the acoustic data and the compressed image data are added. With the value of the existing time stamp PTS, that is, {(the value of the virtual time stamp PTS of the audio data to be output first)-(the time required for the audio decoder to decode the compressed audio data)}. And {the value of the time stamp PTS of the image data to be displayed first) − (the time required for the video decoder to decode the compressed image data)}, and the above audio decoder and video decoder Which decoder to send the data to first. The sector of the image to be displayed first is
Naturally, it is the first sector of the GOP.

As a result of the calculation in the above step (1), when it is determined that the data should be sent to the audio decoder first, the process proceeds to step (2).
Further, as a result of the calculation in the above step (1), when it is found that the data should be sent to the video decoder first, the process proceeds to step (4). In step (2), the clock of the video decoder is set to the time at which compressed audio data should be transmitted from the audio decoder, that is,
Adjust the time to {(time when audio data should be sent from audio decoder)-(time required for audio data to be decoded by audio decoder)}, and use the audio decoder from buffer memory via buffer manager. So that compressed audio data is supplied to. Next, in step (3), the compressed image data is appropriately sent from the buffer memory to the video decoder via the buffer manager using the output adjustment function of the video decoder.

As a result of the calculation in the above step (1), it is found that the data should be sent to the video decoder first, and when the process proceeds to step (4), the clock of the video decoder is read in step (4). Is set to a time sufficiently before the time of the time stamp PTS accompanying the compressed image information so that the compressed image data is supplied from the buffer memory to the video decoder via the buffer manager. Next, in step (5), the clock of the video decoder displays {(the time when the decoded audio data is output from the audio decoder)-(the time required for the audio decoder to decode the compressed audio data). }, Compressed audio data is supplied from the buffer memory to the audio decoder via the buffer manager. By doing so, the image and the sound accompanying the image can be played back in synchronization even during the midway playback.

Hereinafter, with reference to the accompanying drawings, compressed image data including image display time information of the present invention as data,
The specific content of the synchronous reproduction method with the audio data compressed at a constant compression ratio that does not include time information as data will be described in detail. FIG. 1 is a reproduction apparatus to which a synchronous reproduction method of compressed image data including image display time information of the present invention as data and acoustic data compressed at a constant compression ratio not including time information as data is applied. 2 is a block diagram showing a schematic configuration of the above, and FIG. 2 shows compressed image data including image display time information of the present invention as data,
FIG. 3 is a data arrangement diagram for explaining a synchronous reproduction method with audio data compressed at a constant compression ratio that does not include time information as data, and FIG. 3 is a compressed image that also includes display time information of an image of the present invention as data. FIG. 4 is a flowchart showing an example of a procedure for enabling synchronous reproduction even when data and audio data compressed at a constant compression ratio that does not include time information as data are reproduced midway. Information type and time stamp PTS and SC
FIG. 5 is a diagram showing an example of a correspondence relation with R, FIG. 5 is a diagram showing a relation between SCR and a time stamp PTS, and FIG. 6 is attached to high-efficiency coded image data and an image corresponding to the image data. It is a figure which shows the sector arrangement | sequence which recorded the bit stream which interleaved the audio data in which the audio signal was encoded with the high efficiency in time series.

In the reproducing apparatus shown in FIG. 1, 1 is a reproduction signal source, 2 is an interface, 3 is a buffer manager, 4 is a central processing unit, 5 is a buffer memory, 6 is an MPEG video decoder, 7 is an MPEG audio decoder, 13 Is memory. In the reproducing apparatus shown in FIG. 1, the reproducing signal source 1 has time information (time stamp PTS) to be displayed in moving image information encoded with high efficiency.
The compressed image data in a state in which a predetermined header is added, which is configured to include at least SCR and SCR, and the start coincides with the start of the image, but does not have time information indicating the output time, and is always Compressed sound data compressed at a fixed compression rate, in a time series data string (bit stream) that is interleaved in time series, and the information of the type of each of the above information is video information or audio information and , A structure capable of transmitting a data string read from an information recording medium on which a data string to be reproduced for each information is recorded, for example, an optical disk, a magneto-optical disk, or another recording medium Is used.

The compressed image data output from the reproduction signal source 1 is, for example, an image frame in which the image data is compressed by applying at least the intra-frame prediction method.
(I frame) and an image frame (P frame, B frame) in which image data is compressed by applying inter-frame prediction.
A predetermined header composed of at least time information (time stamp PTS) and SCR to be displayed is added to the moving image information which is highly efficient coded by the MPEG system in which the frames are mixed. The compressed image data may be compressed image data in a state, and the compressed acoustic data arranged in time series with respect to the compressed image data output from the reproduction signal source 1 described above includes time information ( AD without time stamp PTS)
It may be compressed audio data whose data amount has been compressed by the PCM system. In the following description, the reproduction signal source 1 is a highly efficient compressed audio information and a moving image highly efficient compressed by the MPEG system. It is said that it is configured so that reproduction data from an optical disc conforming to the CD (Compact Disc) standard in which image information is recorded can be output.

The above-mentioned I frame, P frame, and B frame are mixed, and a predetermined header configured to include at least time information (time stamp PTS) and SCR to be displayed is added. A time-series bit stream including compressed image data in a state and compressed acoustic data in which the amount of data is compressed by the ADPCM method without the time information (time stamp PTS) indicating the output time is streamed. The reproduction signal source 1 to be sent out is, for example, the bit stream (data to be reproduced) reproduced from an optical disc under the control of a control signal given from the central processing unit 4 via the transmission line 8 and the interface 2. Column) to the transmission line 9.

The data string to be reproduced transmitted to the transmission line 9 as described above is stored in the buffer memory 5 via the interface 2 and the buffer manager 3. The buffer manager 3 is a reproduction data string transmitted from the reproduction signal source 1 via the transmission line 9 and the interface 2 under the control of the control signal given from the central processing unit 4 via the bus 10. Are sequentially written in the buffer memory 5, or the compressed acoustic data in the reproduction data sequence stored in the buffer memory 5 are read out and supplied to the MPEG audio decoder 7 via the bus 11, It has a function of reading compressed image data in the reproduced data string stored in the memory 5 and supplying it to the MPEG video decoder 7 via the bus 12 in substantially real time. There is.

When the reproducing apparatus is operating in the normal reproducing mode, the central processing unit 4 operates according to the program stored in the memory 13,
Information of the type of information included in the header part of the reproduction data string stored in the buffer memory 5 via the buffer manager 3 and time information of each information, that is, whether the data is acoustic information data or image information. The reproduction data stored in the above-mentioned buffer memory 5 in response to a request from each MPEG decoder (6, 7) to which each data corresponds, by going to the information such as distinction of data and information such as time information and SCR. The data for each type of information is transferred via the buffer manager 3. Each MPEG mentioned above
Requests from the decoders (6, 7) are issued at timings such that the reproduction state is continuous, and the data is transferred accordingly, so that the continuity of the reproduction signal on the time axis is in each MPEG. It will be guaranteed by the decoder (6, 7). It should be noted that the MPEG decoder (6, 7) is provided with a means for starting the reproduction at a predetermined time at this time because the reproduction is discontinuous with the reproduction from the non-reproduction at the start.

As described above, when the audio information data is transferred from the buffer memory 5 to the MPEG audio decoder 7 through the buffer manager 3 and the bus 11 under the control of the buffer manager 3, the MPEG audio decoder 7 The reproduced sound information signal obtained by expanding the compressed sound data supplied thereto is output, and the compressed image data is MP-converted from the buffer memory 5 via the buffer manager 3 and the bus 12 under the control of the buffer manager 3.
When transferred to the EG video decoder 7, the MPEG video decoder 7 outputs a reproduced image information signal obtained by decoding the compressed image data supplied thereto. Therefore, the reproduced audio information signal is continuously output on the time axis from the MPEG audio decoder 7, and the image information signal is continuously output on the time axis from the MPEG video decoder 7. Output

The bit stream output from the reproduction signal source 1 in the above-described reproduction apparatus is the compressed image data including the time information to be displayed as a part of the data, and its start coincides with the start of the image. The time at which the image to be output first at the same time is displayed in the bit stream that is interleaved in time series with the audio data that is compressed at a constant rate and does not have the time information to be output as data. And the time difference from the time at which the sound accompanying the image should be output is known, and therefore, when the bit stream is reproduced from the middle by the reproducing apparatus, it is shown in the flowchart shown in FIG. The central processing unit 4 controls the operation of each component of the reproducing apparatus so that the reproducing operation is performed according to each of the steps. Te, a sound reproduction associated with image reproduction and the reproduced image in the middle of playback operation is to be performed in sync.

That is, the reproduction signal source 1 in the reproduction device.
When reproducing from the middle of the bit stream output from the above, as in step (1) in the flowchart of FIG. 3, the delay time generated in the audio decoder and the video decoder is taken into consideration, and the expression (3) already described regarding the acoustic data Virtual time stamp PT obtained by
Calculation of the value of S and the value PTS of the time stamp PTS added to the compressed image data, that is, {(value of virtual time stamp PTS of audio data to be output first)-(compressed audio by audio decoder). (Time required to decode data)} and {value of time stamp PTS of image data to be displayed first)-(time required to decode compressed image data by video decoder)} A calculation is performed to determine which of the above-mentioned audio decoder or video decoder the data should be sent first, and as a result of the calculation in the above step (1), the audio decoder is preceded by the data. If it is found that the message should be sent, go to step (2), and as a result of the calculation in step (1) described above, bidet When it did not determine to send the data earlier towards the decoder, the process proceeds to step (4). The sector of the image to be displayed first is
Naturally, it is the first sector of the GOP.

In step (2), the clock of the video decoder is set to the time at which compressed audio data should be transmitted from the audio decoder, that is, {(time at which compressed audio data should be transmitted from the audio decoder)-(compressed audio at the audio decoder). The time required to decrypt the data)}
And the compressed audio data is supplied from the buffer memory to the audio decoder via the buffer manager. Then, in step (3), the buffer memory supplies video data to the video decoder via the buffer manager to the video decoder. The compressed image data is appropriately transmitted using the output adjustment function of.

As a result of the calculation in the above step (1), it is found that the data should be sent to the video decoder first, and when the process proceeds to step (4), the clock of the video decoder is read in step (4). Is set to a time sufficiently before the time of the time stamp PTS attached to the compressed image information so that the compressed image data is supplied from the buffer memory to the video decoder through the buffer manager, and then, In step (5), the time of the video decoder clock is {(the time when the decoded audio data is output from the audio decoder)-(the time required for the audio decoder to decode the compressed audio data)}. Then, compressed audio data is supplied from the buffer memory to the audio decoder via the buffer manager. That. By doing so, the image and the sound accompanying the image can be played back in synchronization even during the midway playback.

[0033]

As is apparent from the above detailed description, the compressed image data including the display time information of the image of the present invention as data and the compression at the constant compression ratio not including the time information as data. The synchronized playback method with the recorded acoustic data is compressed image data that also includes time information to be displayed as part of the data, and the start of the compressed image data matches the start of the image, but the output time information is The sound data compressed at a constant rate that does not have, as the time to display the images to be output at the same time in the bit stream interleaved in time series, and the sound accompanying the images are When the time difference from the time to be output is known, the bit stream is played back from the middle, and the audio decoder and video decoder In consideration of the delay time caused by the above, the value of the virtual time stamp PTS related to the audio data and the value of the time stamp PTS added to the compressed image data are calculated to determine which of the audio decoder and the video decoder described above. It is determined whether the data should be sent first to the decoder of, and as a result of the above calculation, when the data should be sent to the audio decoder first, the clock of the video decoder and the compressed acoustic data from the audio decoder are sent. The time to be sent, that is, {(time to send compressed audio data from audio decoder)-(time required to decode compressed audio data from audio decoder)}, and from buffer memory Compressed audio data is supplied to the audio decoder via the buffer manager Then, the compressed image data is appropriately sent from the buffer memory to the video decoder via the buffer manager by using the output adjustment function of the video decoder, and the virtual time stamp PTS value regarding the audio data and the compression As a result of calculating the value of the time stamp PTS added to the image data, when the data should be sent to the video decoder first, the clock of the video decoder is set to the time stamp PTS attached to the compressed image information. The time is set to a time sufficiently before the time of, so that the compressed image data is supplied from the buffer memory to the video decoder via the buffer manager, and then the clock of the video decoder sets {(decoding from audio decoder Time when the compressed audio data is output)-(decode the compressed audio data with the audio decoder Required time)}), compressed audio data is supplied from the buffer memory to the audio decoder via the buffer manager, so that the image and Since the sound can be reproduced in synchronization with the sound accompanying the image, the reproducing method of the present invention can satisfactorily solve the above-mentioned problems.

[Brief description of drawings]

FIG. 1 is a reproduction in which a synchronous reproduction method of compressed image data including image display time information of the present invention as data and audio data compressed at a constant compression ratio that does not include time information as data is applied. It is a block diagram showing a schematic structure of a device.

FIG. 2 is a view for explaining a synchronous reproduction method of compressed image data that also includes image display time information as data according to the present invention and acoustic data that is compressed at a constant compression ratio and does not include time information as data. It is a data layout diagram.

FIG. 3 is capable of synchronous reproduction even in the case of intermediate reproduction of compressed image data including display time information of an image of the present invention as data and acoustic data compressed at a constant compression ratio that does not include time information as data. 6 is a flowchart showing an example of a procedure for doing so.

FIG. 4 is a diagram showing an example of a correspondence relationship between information types, time stamps PTS, and SCR in sequential sectors.

FIG. 5 is a diagram showing a relationship between an SCR and a time stamp PTS.

FIG. 6 shows a bitstream in which high-efficiency-encoded image data and time-sequentially interleaved high-efficiency-encoded acoustic data associated with an image signal corresponding to the image data are recorded. It is a figure which shows the sector arrangement.

[Explanation of symbols]

1 ... Playback signal source, 2 ... Interface, 3 ... Buffer manager, 4 ... Central processing unit, 5 ... Buffer memory, 6 ... MPEG video decoder, 7 ... MPEG audio decoder, 13 ... Memory,

─────────────────────────────────────────────────── ───

[Procedure amendment]

[Submission date] October 8, 1993

[Procedure Amendment 1]

[Document name to be amended] Statement

[Correction target item name] 0010

[Correction method] Change

[Correction content]

However, when the image information is recorded on the CD, the average value of the time stamp PTS values for a long time is set to change linearly. On the other hand, since the time position of the acoustic data obtained by high-efficiency encoding the acoustic signal accompanying the image information at a constant compression rate is linearly arranged on the time axis, the time stamp PTS is added to the acoustic data. If added, the value of the time stamp PTS is in the state of being located on the straight line A in FIG.

[Procedure Amendment 2]

[Document name to be amended] Statement

[Correction target item name] 0011

[Correction method] Change

[Correction content]

By the way, moving image information is recorded on an optical disc conforming to the CD (Compact Disc) standard as M
A sound obtained by high-efficiency-encoding the variable compression rate data of the moving image information obtained by high-efficiency encoding by the PEG system and the acoustic information accompanying the moving image information at a constant compression rate by, for example, ADPCM. Compressed data of information, a bit stream interleaved in time series,
For example, when the data is recorded as the data illustrated in FIG. 6, that is, the compressed image data including the time information (time stamp PTS) to be displayed as a part of the data and the start thereof are the start of the image. However, if the audio data compressed at a constant rate that does not have the time information to be output as data is recorded as a bit stream that is interleaved in time series, the CD is played halfway. When this happens, there may be a discrepancy in the reproduction timing between the image and the sound accompanying the image.

[Procedure 3]

[Document name to be amended] Statement

[Correction target item name] 0012

[Correction method] Change

[Correction content]

The above points will be specifically described as follows. As shown in FIG. 6, for example, the image information data and the audio information data recorded on the CD are indicated by the reference numeral A, and the audio information data is recorded in the sector A. And the sector V in which the image information data indicated by the reference symbol V is recorded are V → V →
If it is V → A → V → V → V → A →, a code is added between the following sectors A in which the audio information data indicated by the code A in FIG. 6 is recorded. Since there are three sectors V in which the image information data indicated by V are recorded, the SCR values in the sectors A appearing one after another on the time axis are as shown in FIG. Sector contents correspond to A in the contents column, and SCR in FIG.
As indicated by the numerical value in the column, the SCR value of each sector A is increased by (4800). That is, since the acoustic data recorded in each sector A is data in a state of being compressed at a constant compression rate, the time of the acoustic information corresponding to the acoustic data sequentially recorded in each sector A The position on the axis corresponds to the SCR value in the state of linearly increasing on the time axis.

[Procedure amendment 4]

[Document name to be amended] Statement

[Correction target item name] 0016

[Correction method] Change

[Correction content]

On the other hand, when the time stamp PTS is added to the image information recorded in the sector V in which the data of the image information indicated by the reference symbol V is recorded, that sector PTS is added. Although the value of the time stamp PTS is recorded, an example of the value of the time stamp PTS for each sector is described in the time stamp PTS column in FIG. It should be noted that, in the column of the time stamp PTS in FIG. 4, a sector described as “not present” is an MPEG added to the image information recorded in the sector.
This is the case where the header has a structure as shown in FIG. As described above, the I-frame obtained by high-efficiency encoding the image information data recorded in the sector V in which the image information data indicated by the reference symbol V is recorded by the MPEG system, When the compression ratios of the image frames are different, as in the case of P frame and B frame, MPE added to the image data described above.
The value of the time stamp PTS placed in the G header changes to a non-linear state on the time axis, as illustrated by the curve V in FIG. 5, corresponding to the compression rate of the image data. Value, the SCR value and the time stamp PT
The relation with the value of S is not a relation of linear function. ─────────────────────────────────────────────────── ───

[Procedure amendment]

[Submission date] October 16, 1993

[Procedure Amendment 1]

[Document name to be amended] Statement

[Name of item to be corrected] 0023

[Correction method] Change

[Correction content]

In the reproducing apparatus shown in FIG. 1, 1 is a reproduction signal source, 2 is an interface, 3 is a buffer manager, 4 is a central processing unit, 5 is a buffer memory, 6 is an MPEG video decoder, 7 is an audio decoder, and 1 is an audio decoder.
3 is a memory. In the reproducing apparatus shown in FIG. 1, the reproducing signal source 1 has time information (time stamp PTS) and SCR to be displayed in the moving image information encoded with high efficiency.
And the compressed image data in a state in which a predetermined header is added, and the start of the image matches the start of the image, but does not have time information indicating the time of output and is always constant. Compressed sound data compressed at a compression rate is a time-series data string (bit stream) that is interleaved in a time-series manner. An information recording medium on which a data sequence to be reproduced for each information is recorded, for example, an optical disc, a magneto-optical disc, or a structure capable of transmitting a data sequence read from another recording medium is used. Has been.

[Procedure Amendment 2]

[Document name to be amended] Statement

[Correction target item name] 0026

[Correction method] Change

[Correction content]

The data string to be reproduced transmitted to the transmission line 9 as described above is stored in the buffer memory 5 via the interface 2 and the buffer manager 3. The buffer manager 3 is a reproduction data string transmitted from the reproduction signal source 1 via the transmission line 9 and the interface 2 under the control of the control signal given from the central processing unit 4 via the bus 10. Are sequentially written in the buffer memory 5, or the compressed acoustic data in the reproduction data string stored in the buffer memory 5 are read out and supplied to the audio decoder 7 via the bus 11, or 5 has a function of reading out compressed image data in the reproduction data sequence stored in 5 and supplying it to the MPEG video decoder 6 via the bus 12 in substantially real time. .

[Procedure 3]

[Document name to be amended] Statement

[Name of item to be corrected] 0027

[Correction method] Change

[Correction content]

When the reproducing apparatus is operating in the normal reproducing mode, the central processing unit 4 operates according to the program stored in the memory 13,
Information of the type of information included in the header part of the reproduction data string stored in the buffer memory 5 via the buffer manager 3 and time information of each information, that is, whether the data is acoustic information data or image information. The data is stored in the buffer memory 5 in response to a request from each decoder (MPEG video decoder, audio decoder 7) to which the data is distinguished and information such as time information and SCR is checked. The data for each type of information in the reproduced data that is present is transferred via the buffer manager 3. Requests from the above-mentioned decoders (MPEG video decoder, audio decoder 7) are issued at timings such that the reproduction state is continuous, and the data is transferred in accordance with the requests, whereby the reproduction signal on the time axis is Will be guaranteed by each decoder (MPEG video decoder, audio decoder 7). It should be noted that the MPEG video decoder 6 is provided with means for starting the reproduction at a predetermined time at this time since the reproduction is discontinuous with the reproduction from the non-reproduction at the start.

[Procedure amendment 4]

[Document name to be amended] Statement

[Correction target item name] 0028

[Correction method] Change

[Correction content]

As described above, when the audio information data is transferred from the buffer memory 5 to the audio decoder 7 via the buffer manager 3 and the bus 11 under the control of the buffer manager 3, the audio decoder 7 supplies it to the audio information data. A reproduced sound information signal obtained by expanding the compressed compressed sound data is output, and under the control of the buffer manager 3, the buffer memory 5 to the buffer manager 3 and the bus 1
2, the compressed image data is transferred to the MPEG video decoder 6, and the MPEG video decoder 6 outputs a reproduced image information signal obtained by decoding the compressed image data supplied thereto. Therefore, the reproduced audio information signal is output from the audio decoder 7 in a continuous state on the time axis, and the image information signal is output from the MPEG video decoder 6 on the time axis. The status is output.

[Procedure Amendment 5]

[Document name to be amended] Statement

[Name of item to be corrected] Brief description of the drawing

[Correction method] Change

[Correction content]

[Brief description of drawings]

FIG. 1 is a reproduction in which a synchronous reproduction method of compressed image data including image display time information of the present invention as data and audio data compressed at a constant compression ratio that does not include time information as data is applied. It is a block diagram showing a schematic structure of a device.

FIG. 2 is a view for explaining a synchronous reproduction method of compressed image data that also includes image display time information as data according to the present invention and acoustic data that is compressed at a constant compression ratio and does not include time information as data. It is a data layout diagram.

FIG. 3 is capable of synchronous reproduction even in the case of intermediate reproduction of compressed image data including display time information of an image of the present invention as data and acoustic data compressed at a constant compression ratio that does not include time information as data. 6 is a flowchart showing an example of a procedure for doing so.

FIG. 4 is a diagram showing an example of a correspondence relationship between information types, time stamps PTS, and SCR in sequential sectors.

FIG. 5 is a diagram showing a relationship between an SCR and a time stamp PTS.

FIG. 6 shows a bitstream in which high-efficiency-encoded image data and time-sequentially interleaved high-efficiency-encoded acoustic data associated with an image signal corresponding to the image data are recorded. It is a figure which shows the sector arrangement.

[Description of Codes] 1 ... Playback signal source, 2 ... Interface, 3 ... Buffer manager, 4 ... Central processing unit, 5 ... Buffer memory, 6 ... MPEG video decoder, 7 ... Audio decoder, 13 ... Memory,

[Procedure correction 6]

[Document name to be corrected] Drawing

[Name of item to be corrected] Figure 1

[Correction method] Change

[Correction content]

[Figure 1]

─────────────────────────────────────────────────── ─── Continuation of the front page (51) Int.Cl. ⁵ Identification code Office reference number FI technical display location H04N 7/08 Z 6942-5C

Claims (1)

[Claims] 1. Compressed image data that also includes time information to be displayed as part of the data, and at a constant rate whose start coincides with the start of the image but does not have output time information as data. Compressed acoustic data,
When the time difference between the time at which the images to be output simultaneously at the beginning and the time at which the sound associated with the images is to be output is known in the time-sequentially interleaved bitstream is known. , When the bitstream is played back from the middle, the time at which the audio data should be output at the time when the halfway playback is started on the time series is calculated from the above-mentioned known time difference, and the calculated value is obtained. Based on the compressed image data that also includes the display time information of the image that is synchronized with the reproduction start time of the sound and the image based on the image, and the constant compression ratio that does not include the time information data. Synchronous reproduction method with compressed audio information data.