JPH10199140A - Code recording device and code multiplexing method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To enable reduce sufficiently its capacity without burdening a buffer of a reproducing device by multiplexing codes of information of plural kinds such as video, audio and he like encoded with real time with a ratio of data quantity being approximately equal to a ratio of bit rates of each code in each instantaneous time. SOLUTION: Each compressed data transferred to a MPEG system encoder 41 is accumulated in a RAM 41. The MPEG system encoder 41 performs MPEG system encoding for compressed video data and compressed audio data. That is, making pack/packet is performed. Also, making format is performed so as to be adapted to a sector of a recording medium 90. Thereby, compressed video data and compressed audio data are multiplexed. A formatter 43 accumulates an inputted multiplexed AV bit stream in the RAM 42 for each sector unit.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to encoding of a plurality of pieces of information (= a plurality of kinds of pieces of information having the same time) in reproduction output, such as moving images and sounds. The present invention relates to a code recording apparatus and a code multiplexing method for multiplexing and recording data on a medium.

[0002]

2. Description of the Related Art As a method for compressing and encoding digital moving images and audio, for example, MPEG standardized by ISO
And H.264 recommended by CCITT. 261. In these methods, moving pictures are encoded at a high compression rate while maintaining high image quality by interweaving intra-image compression and inter-image compression. Here, the compression within an image unit is a compression mode that can be decoded using only the compressed data, and the compression between image units is a compression mode that performs encoding using a correlation with other image data.

In intra-picture compression, all macroblocks in the picture are subjected to two-dimensional discrete cosine transform (DCT) as they are to obtain a spatial frequency component, which is subjected to matrix division called quantization. In this method, a frequency component is divided by a matrix having a predetermined value. The frequency component can be obtained by multiplying the matrix by the decoder and the quantized data. In addition, it becomes possible to reduce the precision below the matrix value. In addition, utilizing the fact that human visual characteristics are insensitive on the high frequency side,
More codes are assigned to the low frequency side than to the high frequency side.

In the inter-image compression, a motion vector indicating a displacement from a reference image is obtained for each macroblock.
The area closest to the macroblock is detected. Further, two-dimensional discrete cosine transform (DCT) and quantization are performed on the difference between the region and the macroblock data, and the difference data and the motion vector are encoded.

Therefore, the amount of bits generated in a moving picture coding system such as MPEG greatly varies depending on the complexity of the picture, the size of the motion, and the like. For this reason, a method of adjusting the amount of generated bits by adjusting a multiplier for a quantization matrix called a quantization width at the time of quantization is performed.
However, this method has a problem that the image quality is degraded in a complex image or a scene with a large motion, and conversely, the image quality is unnecessarily high in a simple image or a scene with a small motion.

For this reason, in the moving picture coding method, a variable bit rate method is being adopted in which a change in the amount of generated bits is allowed to suppress a change in image quality. In this method, a buffer memory is provided in a stage preceding the decoder on the reproducing apparatus side, thereby absorbing the variation in the amount of generated bits.
For example, Japanese Patent Application Laid-Open No. 4-326687 discloses MPEG
Video is encoded at a variable bit rate according to the standard,
An apparatus for limiting the maximum bit rate is disclosed. The fact that a variable bit rate is more desirable in order to obtain a high compression rate while ensuring the quality of the reproduction output also holds true when compressing and encoding audio information.

[0007]

When a moving image is accompanied by sound, compressed data of the moving image (video) and compressed data of sound (audio) are multiplexed. For example, MPE
In the G system, as shown in FIG. 2, each of compressed video data and compressed audio data is divided into a number of packets of an appropriate length, and information indicating whether the data is video or audio is included in a packet header. Added and multiplexed.

[0008] Each of the moving image and the audio has a reproduction unit (one video frame for the moving image and one audio frame for the audio) called an access unit. The packet header of the packet including the head of the reproduction unit includes: As shown in the figure, a time stamp DT indicating a decoding time
S and a time stamp PTS indicating a reproduction time are added. Note that no time stamp is added to the packet header of a packet that does not include the head of the playback unit. Further, when the beginning of two or more playback units is included in the same packet, only the time stamp corresponding to the preceding playback unit is added to the packet header of the packet.

[0009] The DTS is an order of the video frames in the bit stream and the order of the video frames in the reproduction output, because the MPEG standard allows forward predicted frames, backward predicted frames, and bidirectional predicted frames. Has been added in consideration of the fact that That is, in the MPEG bit stream, the I picture and the P picture are added in consideration of being arranged so as to precede the B picture that refers to the I picture and the P picture.

On the decoder side, each packet is stored in a buffer memory, and the DTS is used as a system synchronization signal STC (Sy).
(Stem Time Clock), the data in the buffer memory is read and decoded, and the PTS is set to ST.
By performing playback output when the value matches C, synchronized playback of video and audio is realized.

FIG. 5 shows an example of a DVD decoder. Data read from the recording medium (DVD) 90 at a predetermined rate is decoded by a DVD decoder 101 and stored in a disc rate control RAM 103. When the disc rate control RAM 103 is full, DVD90
Reading from is suspended. In response to a request from the MPEG system decoder 105, data is transferred from the disc rate control RAM 103 to the MPEG system decoder 105, and the disc rate control RAM 10
Reading from the DVD 90 is resumed when the data can be stored in the DVD3.

As described above, the disc rate control RAM 103 functions as a buffer for adjusting the difference between the reading speed from the DVD 90 and the decoding speed of the video decoder 111 and the audio decoder 113 at the subsequent stage.
This difference is due to video decoder 111 and audio decoder
It fluctuates with the fluctuation of the decoding speed in 113. This variation amount becomes particularly large when the compressed video data has a variable bit rate as shown in FIG. 9 (the data amount varies in GOP units). FIG. 8 shows a case where the data amount in GOP unit is a fixed bit rate. Even in this case, the data amount in a micro level such as a frame unit fluctuates. Note that the disc rate control R
AM103 is a work RAM 101a of the DVD decoder 101,
And / or can be shared with the following AV rate control RAM 105a.

The MPEG transferred from the disc rate control RAM 103 to the MPEG system decoder 105
The system bit stream is stored in the AV rate control RAM 105a. This data is analyzed by the MPEG system decoder 105, separated into compressed video data and compressed audio data, and stored in the AV rate control RAM 105a.

The compressed video data and compressed audio data stored in the AV rate control RAM 105a are read out at the timing indicated by each DTS, sent to the video decoder 111 or the audio decoder 113, and sent to the video decoder 111 or the audio decoder 113. Decoded to data. The decoded video data is stored in the RAM 111a, and the audio data is stored in the RAM 113a. Video data stored in RAM 111a and RAM 113a
Are read and output at the timing indicated by each PTS. Thus, synchronous reproduction of video data and audio data is realized. Note that D which gives the above read timing
The TS and PTS are extracted and stored by analysis of the MPEG system decoder 105, and are stored in, for example, the AV rate control RAM 105a, a dedicated register, a register of the controller 131, or the like.

As described above, since the processing in the decoder is performed, the compressed video data and the compressed audio data data whose DTS time positions are close to each other (preferably at the same time) are
When the data is stored in the rate control RAM 105a at substantially the same time, the reading from the AV rate control RAM 105a is performed at substantially the same time,
The time during which unprocessed compressed data stays in 5a is also shortened, and the AV rate control RAM 105a requires a relatively small-capacity memory. FIG. 5 shows the case of the DVD decoder, but FIG. 6 shows a video-CD decoder (a decoder for reading and reproducing a video-CD in which data of the MPEG standard is recorded in the video-CD format) and FIG. DVD decoder (DVD in which data of the MPEG2 standard is recorded in the DVD format)
In the case of a decoder that reads out and reproduces the same, decoding is performed in a similar manner, and the same situation holds.

An object of the present invention is to reduce the capacity of a memory having the function of an AV rate control RAM (memory for storing compressed video data and compressed audio data) in the decoder. Also, for this purpose,
It is intended to be achievable with a simple configuration. It is another object of the present invention to be able to construct a bit stream that meets the above-mentioned purpose in real time.

In view of the fact that various high-density recording media capable of recording / reproducing moving images and audio have been proposed, the present invention compresses and encodes moving images and audio in real time as described above. An object of the present invention is to enable each code to be multiplexed and recorded on a recording medium so as to be substantially equal to a ratio of an instantaneous bit rate.

[0018]

A first aspect of the present invention is a code recording apparatus that encodes a plurality of types of information having the same time, multiplexes the information, and records the multiplexed information on a recording medium. A plurality of encoders, each of which encodes a plurality of encoders; a time adjusting unit provided at a stage subsequent to at least one of the encoders to absorb a difference in processing time between the plurality of encoders; A code recording apparatus comprising: a clock unit that outputs data; and a multiplexing unit that multiplexes a plurality of code strings in which a difference in processing time between the plurality of encoders has been absorbed in accordance with the timing data.

According to a second aspect of the present invention, there is provided a code recording apparatus for encoding a plurality of types of information having the same time, multiplexing the information, and recording the multiplexed information on a recording medium, wherein each of the plurality of types of information is encoded. An encoder, a time adjusting means provided at a stage subsequent to at least one of the encoders to absorb a difference in processing time between the plurality of encoders, and a plurality of codes absorbing the difference in processing time between the plurality of encoders A buffer memory for temporarily storing the columns, a clock means for outputting timing data to that effect every time a predetermined time elapses, and reading and multiplexing the plurality of code strings from the buffer memory according to the timing data And a multiplexing means.

According to a third aspect of the present invention, in the first or second aspect of the present invention, at least one of the encoders outputs a code sequence of a variable bit rate, and at least one of the encoders outputs a code sequence of a fixed bit rate. Is a code recording device. According to a fourth aspect of the present invention, in the third aspect, one of the variable bit rate code strings is a video data code string, and one of the fixed bit rate code strings is an audio data code string. , A code recording device.

According to a fifth aspect of the present invention, there is provided a code multiplexing method for coding and multiplexing a plurality of types of information having the same time, wherein the plurality of types of information are coded in parallel, and This is a code multiplexing method that absorbs a difference in processing time in encoding processing and multiplexes a plurality of code strings in which the difference in processing time has been absorbed.

According to a sixth aspect of the present invention, there is provided a code multiplexing method for coding and multiplexing a plurality of types of information having the same time, wherein the plurality of types of information are coded in parallel, and the coding processing time is increased. Is a code multiplexing method in which the difference is absorbed in a storage means, and each code string stored in the storage means is read and multiplexed every time a predetermined time elapses.

According to a seventh aspect of the present invention, the video data
M that performs compression encoding for each video frame according to the G standard
PEG video encoder and MPE for audio data
An MPEG audio encoder that compresses and encodes each audio frame in accordance with the G standard; and a delay circuit provided at a subsequent stage of the MPEG audio encoder so as to absorb a difference in processing time between the MPEG video encoder and the MPEG audio encoder. , The MP
A buffer memory for storing compressed video data output from the EG video encoder and compressed audio data output from the delay circuit; a timing circuit for outputting timing data at predetermined time intervals; and the buffer memory according to the timing data An MPEG system encoder for reading out the compressed video data and compressed audio data stored in the MPEG system and multiplexing them so as to form an MPEG system code sequence; and formatting and recording the MPEG system code sequence into a recording format suitable for a recording medium. Recording means for recording on a medium.

According to an eighth aspect of the present invention, in the configuration of the seventh aspect, the MPEG system encoder includes
A code recording apparatus that encodes a code string of a system so as to have a logical format that conforms to the standard of the recording medium.

According to a ninth aspect of the present invention, in the configuration of the seventh or eighth aspect, the recording means comprises: a memory for storing the code string of the MPEG system; And a writing circuit for writing data output from the formatter to a recording medium and reading the data from the formatter into a recording format of a recording medium.

The invention of claim 10 is the invention of claims 7 to 9
In any one of the above structures, the recording medium is a DVD-standard optical disk. The invention according to claim 11 is the invention according to any one of claims 7 to 9, wherein
The recording medium is a video-CD standard optical disc;
It is a code recording device.

[0027]

FIG. 1 is a block diagram of a code recording apparatus according to an embodiment of the present invention, FIG. 2 is an explanatory view showing an example of a data structure after MPEG system encoding, and FIG. 3 is compression-coded. FIG. 4 is an explanatory diagram showing the transition of the data amount of each compressed data of video and audio. FIG.
FIG. 4B is an explanatory diagram of video and audio compressed data stored in M41a, and FIG. 4C is an explanatory diagram of a multiplexed stream after MPEG system encoding.

Video data input in real time to the code recording apparatus of FIG.
After compression-encoding according to a known video compression method using 11a, it is input to the MPEG system encoder 41. The audio data input to the code recording apparatus of FIG. 1 in real time is compression-encoded by an audio encoder 12 according to a known audio compression method using a RAM 12a, and is delayed for a predetermined time by a delay circuit 13 which is a buffer memory. After that, the MPEG system encoder 41
Is input to The delay processing by the delay circuit 13 is performed by the processing time in the video encoder 11 and the audio encoder 12.
, The compressed video data and the compressed audio data, which are temporally associated (each of which is substantially associated with the same time),
This is for the purpose of accumulating at a close position in the AM 41a (for example, in FIG. 4 (b), they are adjacent to each other, preferably close to each other). That is, the delay circuit 13
Has a function as time adjusting means described in the claims. In a device to which data having a configuration in which the processing time in the video encoder 11 is shorter than the processing time in the audio encoder 12 is input, a delay circuit may be provided at a stage subsequent to the video encoder 11.

Each of the video encoder 11 and the audio encoder 12 performs compression encoding / transfer processing at predetermined fixed time intervals. In other words, the compression encoding / transfer processing (block coding) is performed for each unique frame unit.

For example, in the video encoder 11, the compression encoding process is performed intermittently according to the order of recording on the recording medium 90 for each video frame (1/30 second), and the encoded compressed video data is The data is transferred to the MPEG system encoder 41 in frame units. Since the amount of compressed data of each video frame is different, the amount of data transferred every 1/30 second also changes.

The audio encoder 12 intermittently performs compression encoding processing for each audio frame, and the encoded compressed audio data is delayed by a predetermined time by a delay circuit 13 composed of a buffer memory. Are transferred to the MPEG system encoder 41 in audio frame units. The amount of compressed data of each audio frame is substantially constant depending on the encoding format (sampling frequency and average transfer rate) (however, it changes when the encoding format is changed), so that it is transferred every audio frame period. The amount of data to be performed is also substantially constant.

Each compressed data transferred to the MPEG system encoder 41 is stored in the RAM 41a. As described above, since the amount of compressed video data transferred per unit time varies and the amount of compressed audio data transferred per unit time is substantially constant, the compressed video data and compressed audio data stored in the RAM 41a are kept constant. Are arranged as shown in FIG. 4B.

The unit of the compression encoding / transfer processing time in the audio encoder 12 is the same as the time of one video frame (1/30 second) which is the unit of the compression encoding / transfer processing time in the video encoder 11. However, different times are generally used.

The MPEG system encoder 41 performs MPEG system encoding on the compressed video data and the compressed audio data stored in the RAM 41a during a certain time τ. That is, it is packed and packetized. Also,
The recording medium 90 is formatted so as to conform to the sector. As a result, the compressed video data and the compressed audio data are multiplexed as shown in FIG. In other words, M
The PEG system encoder 41 has a function as a multiplexing unit described in the claims. By multiplexing in this manner, a multiplexed AV bit stream having the data configuration shown in FIG. 2 is obtained. That is, a multiplexed AV bit stream in which the parameters of the packet header such as PTS and DTS are determined is obtained. Here, the PTS and DTS are added to the packet header of a packet in which the head of an audio or video access unit (= reproduction unit: video frame / audio frame) exists. In FIG. 2,
Although the case where one pack is composed of a plurality of packets is shown, one pack may be composed of one packet as shown in FIG. FIG. 1 shows a circuit for measuring a certain time τ.
Although the configuration for measuring the time at 32 is shown, the configuration may be such that the timer within the controller 30 measures the time.

As described above, the standards of the recording medium 90 are taken into consideration when encoding the MPEG system. That is,
The recording medium 90 is divided into sectors in consideration of file access. One sector is usually composed of 2 Kbytes of data including an error correction code and additional data such as address data as shown in FIG. Become. The encoding process is performed by the MPEG system encoder 41 so as to conform to the sector structure of the recording medium 90. In other words, the encoding process is performed by the MPEG system encoder 41 so that the pack packet length of the multiplexed AV bit stream conforms to the above-mentioned sector length. For example,
2 packs after processing by the formatter 43 below
MPEG system encoding is performed such that one pack is recorded in one sector as KBytes.

The multiplexed AV bit stream encoded at regular time intervals τ by the MPEG system encoder 41 is sent to the formatter 43.

The formatter 43 stores the multiplexed AV bit stream input from the MPEG system encoder 41 in the RAM 42 for each sector. The storage amount is a recording block unit amount (eg, 32 sectors / 64 sectors / 12
Each time the data reaches 8 sectors), the multiplexed AV bit stream is read from the RAM 42, added with an error correction code and the like, and sent to the write circuit 50. As a result, the multiplexed AV bit stream to which the error correction code has been added is recorded on the recording medium 90. The specific configuration of the writing circuit 50 is a recording medium
Determined according to 90 types. For example, if the recording medium 90 is a high-density magneto-optical disk, it includes a head capable of recording data on the high-density magneto-optical disk and a drive circuit for the head. In the above description, an example is described in which the data is read out from the RAM 42 and is formatted into a recording format by the formatter 43. However, before the data is stored in the RAM 42, an error correction code or the like is added by the formatter 43 and the error correction code or the like is added. The subsequent bit stream is RA
It may be stored in M42. In that case, it is necessary to increase the capacity of the RAM 42.

As described above, in the decoder of FIG.
Therefore, even if the encoding speed (the average value of the compression encoding amount of video or audio data per unit time) and the writing speed (the recording amount per unit time) are significantly different, Can be absorbed.

FIG. 3 shows a case where the bit rate of the compressed video data which is the data input to the decoder of FIG. 1 fluctuates and the bit rate of the compressed audio data is substantially constant. The same argument holds true even if the relationship is reversed. Further, the same argument holds when both of them fluctuate or when both are substantially constant. The substantially constant case is a fixed bit rate shown in FIG. 8 for compressed video data. In this case, the data amount in each GOP does not fluctuate, but fluctuates in each frame therein.

In the decoder of FIG. 1, the case where the type of input data is compressed video data and compressed audio data is shown, but the type of data is not limited to these and may be character information such as subtitles. Is also good. That is, any information that requires simultaneous reproduction of the corresponding access units, in other words, information having the same time characteristic may be used.

[0041]

According to the present invention, codes of a plurality of types of information such as video and audio encoded in real time are multiplexed at a data amount ratio substantially equal to the bit rate ratio of each code at each instant. Therefore, no load is imposed on the buffer of the playback device, and the capacity thereof can be sufficiently reduced.

[Brief description of the drawings]

FIG. 1 is a block diagram showing a configuration of a code recording apparatus according to an embodiment.

FIG. 2 is an explanatory diagram showing a data structure after MPEG system encoding.

FIG. 3 is an explanatory diagram showing a change in the data amount of each of compression-encoded video and audio compressed data.

FIG. 4A is an explanatory diagram of video and audio compressed data amounts for each fixed time τ, FIG. 4B is an explanatory diagram of video and audio compressed data stored in a RAM 41a, and FIG.
Explanatory drawing (c) of a multiplexed stream after PEG system encoding.

FIG. 5 is a block diagram showing a configuration example of a DVD decoder.

FIG. 6 is a block diagram showing a configuration example of a decoder for decoding a video-CD in which a bit stream of the MPEG standard is recorded.

FIG. 7 is a block diagram showing a configuration example of a decoder for decoding a DVD on which a bit stream of the MPEG2 standard is recorded.

FIG. 8 is an explanatory diagram showing a configuration example of compressed video data at a fixed bit rate.

FIG. 9 is an explanatory diagram showing a configuration example of compressed video data of a variable bit rate.

FIG. 10 is an explanatory diagram showing a relationship between a sector, a pack, and a packet.

[Explanation of symbols]

 11 Video Encoder 12 Audio Encoder 13 Delay Circuit 41 MPEG System Encoder

────────────────────────────────────────────────── ─── Continued on the front page (51) Int.Cl. ⁶ Identification symbol FI H04N 7/24 (72) Inventor Mineki Taoka 2-5-5 Keihanhondori, Moriguchi-shi, Osaka Sanyo Electric Co., Ltd.

Claims (11)

[Claims] 1. A code recording device that encodes a plurality of types of information having the same time, multiplexes them, and records them on a recording medium, comprising: a plurality of encoders each encoding the plurality of types of information; A time adjusting means provided at a stage subsequent to one of the encoders to absorb a difference in processing time between the plurality of encoders; a timing means for outputting timing data to that effect every time a predetermined time elapses; Multiplexing means for multiplexing a plurality of code strings in which a difference in processing time between encoders has been absorbed in accordance with the timing data. 2. A code recording apparatus for encoding a plurality of types of information having the same time, multiplexing the encoded information, and recording the multiplexed information on a recording medium, comprising: a plurality of encoders each encoding the plurality of types of information; A time adjusting unit provided at a subsequent stage of one of the encoders to absorb a difference in processing time between the plurality of encoders; and temporarily storing a plurality of code strings absorbing the difference in processing time between the plurality of encoders. Buffer memory for storing, timing means for outputting timing data to that effect every time a predetermined time elapses, and multiplexing means for reading and multiplexing the plurality of code strings from the buffer memory according to the timing data. Code recording device. 3. The code according to claim 1, wherein at least one of the encoders outputs a code sequence of a variable bit rate, and at least one of the encoders outputs a code sequence of a fixed bit rate. Recording device. 4. The code recording according to claim 3, wherein one of the variable bit rate code strings is a video data code string, and one of the fixed bit rate code strings is an audio data code string. apparatus. 5. A code multiplexing method for coding and multiplexing a plurality of types of information having the same time characteristic, wherein said plurality of types of information are respectively coded in parallel, and a coding process of each code string is performed. A code multiplexing method that absorbs a difference in processing time between the plurality of code streams and multiplexes a plurality of code strings in which the difference in processing time is absorbed. 6. A code multiplexing method for coding and multiplexing a plurality of types of information having the same time, wherein the plurality of types of information are respectively coded in parallel and a difference in coding processing time is absorbed. A code multiplexing method which reads out and multiplexes each code string stored in the storage means every time a predetermined time elapses. 7. The video data is encoded according to the MPEG standard.
An MPEG video encoder for compressing and encoding each video frame; an MPEG audio encoder for compressing and encoding audio data for each audio frame in accordance with the MPEG standard; absorbing a difference in processing time between the MPEG video encoder and the MPEG audio encoder A delay circuit provided after the MPEG audio encoder, a buffer memory for storing compressed video data output from the MPEG video encoder and compressed audio data output from the delay circuit, A timing circuit for outputting timing data; reading out the compressed video data and compressed audio data stored in the buffer memory in accordance with the timing data and multiplexing them so as to form a code string of an MPEG system And MPEG system encoder that, the code recording apparatus having a recording means for recording on a recording medium by formatting the code sequence conforming the recording format on the recording medium of the MPEG system. 8. The code recording apparatus according to claim 7, wherein the MPEG system encoder encodes a code string of the MPEG system into a logical format conforming to the standard of the recording medium. 9. The recording device according to claim 7, wherein the recording unit stores a code string of the MPEG system, and a storage amount of the memory is a unit amount of a recording block of a recording medium. A code recording device, comprising: a formatter that reads data every time and formats it into a recording format of a recording medium; and a writing circuit that writes data output from the formatter to a recording medium. 10. The code recording device according to claim 7, wherein the recording medium is a DVD-standard optical disk. 11. The recording medium according to claim 7, wherein the recording medium is a video-CD standard optical disk.
Code recording device.