JP3523493B2 - Method and apparatus for multiplexing highly efficient encoded data - Google Patents

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to multiplexing for facilitating random access and synchronization of audio and video after random access by adding a header to high efficiency coded video coded data and audio coded data. The present invention relates to a multiplexing method and a multiplexing device for generating data.

[0002]

2. Description of the Related Art In recent years, MPE has been used as a moving image coding system.
G1 (ISO / IEC 11172), MPEG2 (I
The SO / IEC 13818) system is used in the fields of storage, communication and broadcasting. In the MPEG system, intraframe coding, interframe predictive coding in the forward direction, and interframe predictive coding in both directions are switched for each frame. In the following, a frame for intra-frame coding is an I frame, a frame for inter-frame predictive coding in the forward direction is a P frame,
A frame to be subjected to interframe predictive coding in both directions is called a B frame. FIG. 12 shows an example of the arrangement of I frames, P frames, and B frames. In the MPEG system, as shown in FIG. 12, a plurality of frame groups including one or more I-frames are grouped into Group of Pictures (hereinafter, referred to as GOP).
Abbreviated).

When the encoded data compressed by the MPEG system is reproduced from the middle of the video sequence, it is necessary to decode from the I frame. This is because P frames and B frames are interframe predictive coded, so it is necessary to first decode the image used for prediction. By tracing this dependency relationship, all P frames and B frames are I frames. It depends on.

By the way, in the MPEG system, since variable length coding is performed, time and code amount are not proportional. That is, when the video sequence is played back in the middle of random access or the like, even if the playback start time is known, it is not possible to know where the data at that time is recorded. Therefore, GO
The start position of P and the start position of the encoded data of the I frame are managed in advance to make random access efficient.

An apparatus for efficiently performing random access in the MPEG system is described in Japanese Utility Model Registration No. 3004104. In Japanese Utility Model Registration No. 3004104, a plurality of GOPs are recorded as data units (DUTs) on a disk medium to facilitate random access in units of DUTs.

FIG. 13 shows the configuration of the DUT described in Utility Model Registration No. 3004104. DUT10
1 includes extension data 102, audio data 103, and video data 104. The extension data 102 includes a header 105.
And sub-picture data 106 are included. Voice data 10
3 is audio data corresponding to the time of the video data 104,
Video data compressed by the MPEG system is recorded in the video data 104. The DUT header 105 includes a video data start position, audio data start position, and audio synchronization information.

The sector number at which the head of the DUT is recorded is separately recorded in the management file of the disc, and at the time of random access or high-speed reproduction, the start position of the target DUT is searched and jumped from the information of the management file to jump to the head of the DUT. Or from the head of the video data using the head position information of the video data recorded in the DUT header.

[0008]

However, the method described in Japanese Utility Model Registration No. 3004104 (hereinafter referred to as a conventional example) has the following two problems.

First, the conventional example has a problem in that encoded data cannot be accessed in a unit smaller than the DUT.

When video data is encoded by the MPEG system, an I picture is included in each GOP, and decoding is possible from this I picture. However, in the conventional example, only the head position of the DUT is managed. Therefore, a GOP in the middle of the DUT
In order to reproduce the data, it is necessary to first randomly access the head of the DUT and skip unnecessary data up to the GOP to be reproduced, and it takes time from the random access instruction until the data reproduction. Skipping unnecessary data is necessary not only for video data but also for audio data.

Secondly, even if random access can be made in GOP units, in the conventional example, since the information for synchronizing the audio data and the video data is only at the first position of the DUT, the audio can be accessed in random access in GOP units. And there is a problem that can not ensure the synchronization of the video.

The period of the video frame is 30,000 / 10
The frequency is 01 Hz, and encoding is also performed every cycle. On the other hand, the sampling frequency of audio data is 4
1 when encoding at 4.1 KHz with MPEG1 system
When 152 data is used as an audio frame and is encoded by the ATRAC system and the ATRAC2 system, 1024 data is often encoded as an audio frame.
The video frame is about 33 msec, and the audio frame is about 26 msec or about 23 msec. In the conventional example, calculation is required to obtain the start time at the head of the second and subsequent GOPs in the DUT. In addition, in terms of accuracy, the calculation has to be an approximate calculation, and it is difficult to achieve accurate synchronization.

Furthermore, in the case of a disk medium, it is possible to perform program reproduction or program editing in which the reproduction order is indicated by a pointer without changing the recorded data by utilizing the fact that any data can be accessed at high speed. However,
Since accurate synchronization cannot be achieved in the conventional example, there is a possibility that synchronization errors may accumulate if the program is reproduced for a short time and random access is repeated.

The present invention has been made in view of the above problems, and it is a highly efficient encoded data that can be randomly accessed or edited in units of GOP and that can ensure synchronization between audio data and video data. An object of the present invention is to provide a multiplexing method and a multiplexing device.

[0015]

According to the present invention, by providing position information and reproduction start time information for each GOP in the header portion of multiplexed data, random access in GOP units and audio / video synchronization are realized. To do.

Multiplexing of highly efficient coded data according to the present invention
The method is to set multiple video frames as GOPs,
High-efficiency encoded GOP video data and corresponding GOP
GOP audio with high-efficiency encoding of the corresponding audio frame
A method of multiplexing audio data
And one or more of the GOP video data and one or more
Is a packet having a plurality of the GOP audio data
And the packet header contains at least each GOP
Same as position information of video data and each GOP audio data
Period information, and the position information is GOP video data.
Within the packet at the beginning of the GOP audio data
Relative position of the GOP video.
Playback of the first video frame of the data to be played
Start time and audio frame corresponding to the video frame
Playback start time, and the audio frame
Of the playback start time of the
It is characterized by being a difference value.

Here, the position information is converted into the GOP video data.
Before the intra-coded video frame of the data
The start position in the packet and the intra-frame coded
The said part of the audio frame corresponding to the video frame
The start position in the packet and the synchronization information
The playback start time of the intra-coded video frame,
Playback of audio frames corresponding to video frames
At the start time and when the playback of the audio frame starts
Is the difference between the start time and the reproduction start time of the video frame.
Also good.

The high-efficiency coded data according to the present invention
The multiplexer may include the one or more GOP video data
Data, the one or more audio data and the packet.
Memory for recording the packet header and the packet header
A header generation circuit for generating,
One or more GOP video data and the one or more GOP video data;
Before counting the code amount of several GOP audio data
The packet header is generated and the packet header is generated.
The packet header is recorded in the memory every time
Header, the one or more GOP audio data
The order of the one or more GOP video data, or the
Packet header, the one or more GOP video data
In the order of the GOP audio data and the one or more GOP audio data.
According to the present invention.
It realizes a multiplexing method.

[0019]

[0020]

[0021]

BEST MODE FOR CARRYING OUT THE INVENTION Embodiments of the present invention will be described in detail below with reference to the drawings.

[First Embodiment] In the following, video is MPE with 15 frames as 1 GOP.
High-efficiency encoding with G2 system, audio is ATRAC2
And high-efficiency encoding by a method, multiplexes the encoded audio data and encoded video data, described with reference to the drawings an embodiment of the present invention when recording the multiplexed data to the disk medium.

First, the format of multiplexed data according to the present embodiment will be described with reference to FIGS. In the present embodiment, multiplexed data for a plurality of GOPs is used as a packet, the packet information is managed separately from the multiplexed data, and the information in the packet is recorded in the header portion of the packet.

FIG. 1 shows an example of multiplexed data generated by the highly efficient coded data multiplexing method of the present invention. In the present invention, GOP is the minimum unit of management, video coded data of GOP is GOP video data, and audio coded data corresponding to GOP is GOP audio data.
Packet audio data 3 is composed of two GOP audio data 5 and 6, and two GOP video data 7
8 forms packet video data 4, and further packet header 2, packet audio data 3, and packet video data 4 form packet 1. Packet header 2
The position information and the synchronization information in the packet are recorded in, but the details will be described later. The position information of the packet is recorded in a management area different from the area for recording the multiplexed data on the disk medium.

FIG. 2 shows the relationship between the video frames forming the GOP and the audio frames corresponding to the GOP.
FIG. 2 shows an example in which the start of the period of the audio frame corresponding to the GOP is later than the start of the period of the video frames forming the GOP.
The OP period is 11. When the periods of one video frame and one audio frame are different, even if a plurality of audio frames are collected, they do not match the GOP period 11. Therefore, a period in which the deviation from the beginning and the deviation from the rear end of the GOP period 11 are less than one audio frame period is set as the audio period 12.

FIG. 3 shows details of the packet header 2 shown in FIG. In the packet header 2, "packet size",
"Packet header size", "Audio size",
"Video size", "Audio start position", "Audio playback time" for each GOP included in the packet,
"Video start position" and "video playback time" are recorded.

FIG. 4 shows details of the packet header 2 shown in FIG. "Packet size" is the total packet data amount, "Packet header size" is the packet header data amount, "Audio size" is the packet audio data code amount, "Video size" is the packet video data code amount, and "Audio start""Position" is the start position of the GOP audio data and is a relative position where the beginning of the packet audio data is 0. "Audio playback time" is G
It is the reproduction start time of the OP audio data and has a precision value of 90 KHz. The "video start position" is the start position of the GOP video data and is a relative position where the head of the packet video data is 0, and the "video reproduction time" is the reproduction start time of the GOP video data and has a precision value of 90 KHz. .

FIGS. 3 and 4 show an example in which a packet is composed of 2 GOPs. When the number of GOPs forming a packet increases, the packet header increases by the amount of information for each GOP.

As described above, according to the present embodiment, the packet header includes "audio start position", "audio play time", "video start position", and "video play time" for each GOP.
In the random access, the position information of the packet recorded in the management area is used first to jump to the beginning of the packet, and then the packet header is read to jump to the beginning of the GOP.
The unit jump is performed efficiently. Since the reproduction time information for each GOP is recorded in the packet header, accurate audio and video synchronization can be realized.

Next, the operation of forming the packet 1 shown in FIG. 1 will be described with reference to the flow charts of FIGS. Figure 5
2 is an operation for determining the GOP period 11 and the audio period 12 shown in FIG. 2 and an operation for outputting the “audio reproduction time” and the “video reproduction time” in the packet header shown in FIG. This is an operation of generating other elements of the header and actually outputting one packet.

In the flowchart of FIG. 5, a variable An indicating the number of audio frames and a variable Vn indicating the number of video frames are initialized to 0 in step S11, and the variable N is set to G.
Set to the number of video frames that make up the OP. And
In steps S12 and S13, a video frame start signal and an audio frame start signal are detected. The start signal of the video frame is a signal input from the portion processing the video signal, and the start signal of the audio frame is a signal input from the portion processing the audio signal. When the audio frame start signal is input, An is incremented (step S14), and the process returns to step S12. When the video frame start signal is input, Vn is incremented (step S15), and it is determined in step S16 whether Vn is 1, that is, the first video frame in the GOP. If it is not the first video frame, it is determined in step S17 whether Vn is N, that is, the last video frame in the GOP. G
In the case of the last frame in the OP, the number of video frames of GOP is output, the variable Vn indicating the number of video frames is reset to 0 (step S18), and the process returns to step S12. The number of output GOP video frames is shown in FIG.
Used in the flowchart of If it is determined in step S17 that the frame is not the last frame of GOP, nothing is performed and the process returns to step S12. When it is determined in step S16 that this is the first video frame of the GOP, this time is output as the "video reproduction time" (step S19). The time is provided with a counter that operates at 90 KHz, and the value of the counter is output. Next, in step S20, the value of An indicating the number of audio frames is output, and An is reset to 0. An is incremented each time an audio frame start signal is input, and the value is output at the head position of the GOP period and reset to 0. Therefore, in step S20, the number of audio frames in the audio period is output. It will be. Then, the process waits until an audio frame is detected in step S21. When the audio frame is detected, the time is output as "audio reproduction time" (step S22), and An is incremented (step S23). And step S24
It is determined whether or not the coding has ended, and if not, the process returns to step S12 to continue the operation.

At step S19, the reproduction start time information of the first video frame of the GOP period is output, and step S
At 22, the reproduction start time information of the first audio frame of the GOP period is output. In step S18, GO
The number of video frames forming P is output, and step S
At 20, the number of audio frames forming the audio period is output. These output values are used in the operation of forming a packet shown in FIG.

FIG. 6 is a flowchart showing the operation of forming the packet 1 shown in FIG. In the flowchart of FIG. 6, in steps S32 to S39, information on a portion of the packet header relating to audio data is generated, in steps S40 to S46, information on a portion of the packet header relating to video data is generated, and steps S47 to S49.
To output the data that makes up the packet.

First, in step S31, the variable used in the subsequent processing is initialized to 0, and in step S32, the variable n indicating the GOP number is initialized to 1. In step S33, the amount of encoded audio data is counted, and in step S34, it is determined whether the audio frame has ended. Whether or not the audio frame has ended is determined by detecting the next audio frame start code or by counting the code amount because the audio frame has a fixed length.
If it is not the end of the audio frame, step S3
Repeat 3. In step S33, the operation of adding the "count amount" to the "audio frame code amount" is also performed. If the audio frame has ended, the "audio frame code amount" is added to the "audio start position" and the "audio frame code amount" is reset to 0 (step S35). The read code amount of the code of the nth GOP is the nth +
Since it is related to the start position of 1 GOP, the audio frame code amount is added to "audio start position (n + 1)" in step S35. Then, in step S36, GO
It is determined whether the P audio data has ended. The determination is made based on the number of audio frames in the audio period output in step S20 of the flowchart shown in FIG. 5 and the number of audio frames of the read audio encoded data. If it is determined in step S36 that the GOP audio data has not ended, steps S33 to S
Repeat 35. Since the "audio frame code amount" is added to the "audio start position" in step S35, when it is determined in step S36 that the GOP audio data has ended, the start position of the next GOP, that is, the GOP audio data of the (n + 1) th GOP. Is set at the audio start position (n + 1). In step S37, the variable n indicating the GOP number is incremented, and the “audio start position (n + 1)” indicating the audio start position of 2 GOP ahead is already set. n) ”. Then, in step S38, it is determined whether or not the packet audio data ends. The determination is made by comparing the preset number of GOPs forming a packet with the number of read GOP audio data. If it is determined in step S38 that the processing has not ended,
Steps S33 to S37 are repeated.

If it is determined in step S38 that the packet audio data has ended, in step S39 the "audio size" is set to "audio start position (n)" and the variable n indicating the GOP number is reset to 1. .
Since step S39 is performed after reading the packet audio data, "audio start position (n)"
Indicates the amount of packet audio data.

Similarly, in steps S40 to S45, G
The OP video data and the packet video data are read out.

First, in step S40, the amount of encoded video data is counted, and in step S41, it is determined whether the video frame ends. The determination is made by detecting the start code of the next video frame or the end code of the video sequence. If it is determined that the video frame has not ended, step S40 is repeated. Step S40
Then, the operation of adding the "count amount" to the "video frame code amount" is also performed. In the case of the end of the video frame, "video frame code amount" is set in "video start position (n + 1)"
Is added to reset the "video frame code amount" to 0 (step S42). Then, in step S43, GO
It is determined whether the P video data has ended. The determination is made by comparing the number of video frames of the read video coded data with the number of frames constituting a preset GOP. If it is determined that the GOP video data has not ended, steps S40 to S42 are repeated. If it is determined that the GOP video data has ended, then in step S44 G
The variable n indicating the OP number is incremented, and “video start position (n + 1)” indicating the video start position of 2 GOP ahead.
Is set to the video start position “video start position (n)” of the next GOP. Then, in step S45, it is determined whether or not the packet video data ends. The determination is made by comparing the number of GOPs forming a preset packet with the number of read GOP video data. Step S
If it is determined that the processing is not finished at 45, step S40
~ S44 is repeated.

If it is determined in step S45 that the packet video data has ended, the "video size" is set to "video start position (n)" in step S46. Since step S46 is performed after reading the packet video data, "video start position (n)" indicates the data amount of the packet video data.

Next, in step S47, the packet header is output. The "packet size" in the packet header is the sum of "packet header size", "audio size", and "video size", and the "packet header size" is a value determined by the number of GOPs forming a packet.
"Audio size" and "Video size" are set in step S
39 and the value obtained in step S46. The "audio start position" is a value obtained by sequentially updating in step S35, and the "video start position" is a value obtained by sequentially updating in step S42. Further, the "audio reproduction time" is the value obtained in step S22 of FIG.
The "video reproduction time" is the value obtained in step S19 of FIG.

Next, the audio packet data is output in step S48 and the video packet data is output in step S49, and the output of the packet is completed.

With the above operation, the packet 1 shown in FIG. 1 and the packet header shown in FIG. 3 are output. Since the packet header includes the audio start position and video start position for each GOP, and the audio playback time and the video playback time, random access can be efficiently performed in GOP units, and accurate audio and video Synchronization can be secured.

In the operation of forming the packet shown in FIG. 6, the packet audio data is first read to construct a portion of the packet header information relating to the audio data, and then the packet video data is read to read the packet header information. Although the portion related to the video data is constructed, the audio processing and the video processing may be performed in parallel.

In this embodiment, the "audio start position" is a relative value with the head of GOP audio data set to 0, and the "video start position" is a relative value with the start of GOP video data set to 0. The relative value may be any relative value within the range, for example, a relative value in which the head of the packet is 0 or a relative value in which the trailing end of the packet header is 0.

Further, in the present embodiment, the packets are multiplexed in the order of the packet header, the packet audio data, and the packet video data, but the order is not fixed to this, and for example, the packet header, the packet video data, Packet audio data may be in order,
Control data and user data may be added in addition to audio data and video data.

Further, the present embodiment is characterized in that the position information and the reproduction time information for each GOP are recorded in the packet header, and the structure of the packet header is not fixed to this embodiment. .

[Second Embodiment] In the first embodiment, the "audio start position" and "video start position" of each GOP are the GOP start position, "audio reproduction time", and "video reproduction time". Was the reproduction start time at the beginning of the GOP, but in the present embodiment, the "video start position" indicates the beginning position of the intra-frame coded frame (I frame) within the GOP, and "audio start" The "position" indicates the start position of the audio frame corresponding to the I frame, and the "audio reproduction time" and the "video reproduction time" are different in that they indicate the reproduction start time of the frame specified by the start position, respectively. "packet size", "packet header size", "audio size", "video size" Ru same der in the first embodiment.

As described above, when reproduction is started from random access, it is necessary to reproduce from I frame. As in the example shown in FIG. 12, since the head of a GOP may not be an I frame, by setting the information for each GOP to the I frame instead of the head of the GOP, the data jumps to data that can be reproduced more efficiently. be able to.

Even if the head of the GOP is not an I frame,
It is necessary to code the video from the I frame, and the I frame is the head of the video GOP data in the arrangement of the coded data. Therefore, even in the present embodiment, the "video start position" has the same value as in the first embodiment, but since the audio frames are encoded in the input order, the position of the audio frame corresponding to the I frame is GOP. The position is different from the beginning of the audio data.

First, the operation for obtaining the "audio reproduction time" and the "video reproduction time" is performed in the flowchart of FIG.
Although the flowchart of FIG. 7 and the flowchart of FIG. 5 operate in a similar manner, the timing for outputting the “audio playback time” and the “video playback time” and the variables for counting the audio frames in the flowchart of FIG. 7 are An1 and An2. 2 is used, An1 outputs the number of audio frames in the audio period 11 shown in FIG. 2 similarly to the first embodiment, and An2 outputs the number of audio frames in the period based on the I picture. different. The description of the same operations as those in the flowchart of FIG. 5 will be omitted below, and operations of different points will be described.

First, the number of frames indicated by An2 will be illustrated with reference to FIG. FIG. 8 shows a video frame forming a GOP and which coding type each frame is coded. An2 indicates the number of audio frames in the audio period 21 of FIG. In the audio period 21, the start time is the start time of the audio frame immediately after the I frame, and the end time is the start time of the audio frame immediately after the next I frame.

In the flowchart of FIG. 7, a variable An2 indicating the number of audio frames in the audio period 21 is initialized to 0 in step S51. When an audio frame is detected, An2 is incremented in step S54. When a video frame is detected in step S52, it is determined in step S58 whether the detected video frame is an I frame. The determination is made from the GOP structure set in advance. If it is determined in step S58 that the frame is an I frame, this time is output as the "video playback time" (step S63), and the value of An2 indicating the number of audio frames up to the I frame is output as the number of audio frames 2. (Step S6
4). Then, the start time of the audio frame immediately after the I frame is output as the "audio reproduction time" (step S66), and An2 is incremented (step S67). In step S59, it is determined whether or not the GOP data has ended. If it is determined that the GOP data has ended, step S61.
Determines whether the packet has ended. If it is determined that the packet has ended, An2 is reset to 0 in step S62, so that the value of the audio frame number 2 output in step S64 is the first output value in the packet from the beginning of the packet. Audio period 2 shown in
The number of audio frames is up to 1, and after that, the number of audio frames in the audio period 21 is output.

As described above, in this embodiment, the number of audio frames in the audio period 12 shown in FIG. 2 is output in step S57, and the audio period 21 shown in FIG.
The number of audio frames is output in step S64.

Next, FIG. 9 is a flowchart showing the operation of forming a packet. The operation of the flowchart in FIG.
Although it is similar to the operation of the flowchart of FIG. 6 of the first embodiment, the first audio start position is always 0 in the first embodiment, so it was calculated from the audio start position (2). The present embodiment is different in that it is obtained from the audio start position (1). In FIG.
The parts performing the same operations as those in the flowchart of FIG.

In FIG. 9, when the end of the audio frame is detected in step S34, the audio frame code amount is added to the audio start position (n) in step S71. Then, in step S72, it is determined whether the audio frame corresponds to the I frame. The determination is made by comparing the number of audio frames 2 output in step S64 of FIG. 7 with the number of audio frames of the read audio encoded data. The audio frame number 2 indicates the number of audio frames from the beginning of the packet to the start of the audio period 21 shown in FIG. 8 or the number of audio frames in the audio period 21. When it is determined in step S72 that the audio frame corresponds to the I frame, the audio start position (n + 1) is set to the audio start position (n) in step S73, and the variable n indicating the number of GOPs is incremented. When it is determined in step S72 that the audio frame corresponds to the I frame, the audio start position (n) is set to the data amount from the head of the GOP audio data to the head of the audio frame corresponding to the I frame. . Next, in step S74, it is determined whether or not the packet audio data has ended. The determination is made by comparing the cumulative value of the audio frame number 1 output in step S57 of FIG. 7 with the read audio frame number. If it is determined in step S74 that the packet audio data has ended, the audio size is set to the audio start position (n) in step S75. The audio start position (n) indicates the audio code amount with the head of the packet audio data as 0.

In step S72, the number of audio frames based on the I frame is compared, and the audio start position is set as the start position of the audio frame corresponding to the I frame. In step S74, the number of audio frames based on the GOP is compared. The audio size to GOP
Is set to the integrated code amount of the audio period corresponding to, that is, the code amount of the audio packet data.

Steps S40 to S49 perform exactly the same operation as in the first embodiment. In step S47, the "video reproduction time" output in step S63 of FIG. 7 and the "audio reproduction time" output in step S66 are included. Then, the packet header is output, the packet audio data is output in step S49, and the packet video data is output in step 50 to complete the output of the packet.

As described above, in this embodiment, the start position and the reproduction start time of the I frame and the start position and the reproduction start time of the audio frame corresponding to the I frame are recorded in the packet header, so that the G frame is used for random access.
Even when reproducing from an I frame that is not the beginning of OP, it is possible to more efficiently jump to the data required for synchronous reproduction without reading extra audio data.

[Third Embodiment] In the first and second embodiments, the reproduction time information for synchronization is the reproduction start time of the audio frame and the reproduction start time of the video frame. that the differential value of the playback start time of the reproduction start time and a video frame of the audio frame that Do different in the embodiment.

FIG. 10 shows the structure of the packet header in this embodiment. Of the information for each GOP, the reproduction time information is the same as in the first and second embodiments, except that it is the "start time difference".

The operation of this embodiment is also almost the same as that of the first and second embodiments. However, in the first embodiment, when outputting a packet, "audio reproduction time" and "video reproduction time" are set. Although the "time" was output, in the present embodiment,
(“Audio playback time” − “Video playback time”) is obtained, and this value is output as the “start time difference”.

According to the present embodiment, there is an advantage that the size of the packet header can be made smaller than that in the first and second embodiments.

[Fourth Embodiment] FIG. 11 shows a fourth embodiment of the present invention. Fourth embodiment, the high-efficiency encoded audio data and video data according to the operation shown in the first to third embodiments multiplexed, Ru multiplexer der constituting a packet.

The multiplexer shown in FIG. 11 comprises an audio encoder 32, a video encoder 34, a memory 35, a header generation circuit 36, and a synchronization information generation circuit 37.

In FIG. 11, the audio input terminal 31
The audio data input from is highly efficient encoded by the audio encoder 32, and audio encoded data is output. Further, the video data input from the video input terminal 33 is highly efficiently encoded by the video encoder 34, and the encoded video data is output. The audio coded data and the video coded data are temporarily stored in the memory 35, separated for each audio packet data and video packet data, and output to the recording medium as a packet.

The synchronization information generating circuit 37 has a counter operating at 90 KHz, inputs an audio frame pulse or a video frame pulse, and generates reproduction time information for each GOP according to the flowcharts shown in FIGS.

The header generation circuit 36 controls the memory 35, counts the audio encoded data amount and the video encoded data amount written in the memory, and uses the reproduction time information input from the synchronization information generating circuit. The packet header shown in FIGS. 3 and 10 is generated. Each time the header generation circuit 36 completes the generation of the packet header,
The packet header is recorded in the memory 35. Then, when a recording request is generated from the recording medium, the memory 35 is controlled so that the packet header, the audio packet data, and the video packet data are sequentially output to the recording medium.

In the present embodiment, the packet shown in FIG. 1 can be easily constructed only by adding the header generation circuit and the synchronization information generation circuit to the audio encoder and the video encoder.

[0068]

According to the present invention, the position information and the synchronization information of the audio coded data and the video coded data for each GOP are recorded in the packet header, so that the GOP can be stored without increasing the capacity of the management area of the recording medium. Random access of units can be easily performed, and accurate synchronization can be achieved after random access. Further, even when random access is performed in a unit smaller than the GOP, the encoded data is decoded in the unit of GOP, so that the synchronization can be ensured.

Further, since the data in the management area of the disk must be held in the memory of the reproducing device at the time of reproduction,
Not increasing the capacity of the management area also has the effect of reducing the memory capacity of the playback device.

Further, according to the present invention, since the position information for each GOP is a relative position within a packet, it is sufficient that a small value can be expressed and a small amount of data can be obtained. Also,
There is also an effect that the calculation for obtaining the position information can be performed by a small arithmetic circuit.

Further, according to the present invention, since the position information and the reproduction time information of the I frame and the audio frame corresponding to the I frame are recorded for each GOP, even when the GOP does not start from the I frame, the random information is generated. No extra audio data is read when accessing.

Further, according to the present invention, since the reproduction time information is one difference value between the audio reproduction time and the video reproduction time, the size of the packet header can be reduced and the data amount allocated to the encoded data can be reduced. Can be increased, and as a result, the image quality and sound quality are improved.

Further, according to the present invention, a multiplexer which outputs a packet effective for random access for each GOP can be constructed by only adding a small additional circuit to the audio encoder or the video encoder. Also,
Many of the additional circuits are also required in the conventional circuits such as memory control and generation of synchronization information, and can be realized only by changing these operations, and many effects can be obtained at a low cost.

[Brief description of drawings]

FIG. 1 is a diagram showing a configuration of a packet configured in the present invention.

FIG. 2 is a diagram illustrating a GOP period and an audio period.

FIG. 3 is a diagram showing a structure of a packet header.

FIG. 4 is a diagram illustrating the contents of a packet header.

FIG. 5 is a flowchart showing a configuration of a GOP and an operation of generating start time information according to the first embodiment of the present invention.

FIG. 6 is a flowchart showing operations of packet header generation and packet generation in the first exemplary embodiment of the present invention.

FIG. 7 is a flowchart showing a configuration of a GOP and an operation of generating start time information according to the second embodiment of the present invention.

FIG. 8 is a diagram illustrating a GOP period and an audio period based on an I frame.

FIG. 9 is a flow chart showing an operation of generating a packet header and a packet in the second embodiment of the present invention.

FIG. 10 is a diagram showing a structure of a packet header according to a third embodiment of the present invention.

FIG. 11 is a block diagram showing a configuration of a fourth exemplary embodiment of the present invention.

FIG. 12 is a diagram illustrating a GOP.

FIG. 13 is a diagram illustrating a DUT in a conventional example.

[Explanation of symbols]

1 packet 2 Packet header 3 audio packet data 4 Video packet data 5 Audio GOP data 6 Audio GOP data 7 Video GOP data 8 video GOP data 11 GOP period 12 audio periods 21 audio period 31 Audio input terminal 32 audio encoder 33 Video input terminal 34 Video Encoder 35 memory 36 Header generation circuit 37 Synchronous information generation circuit 101 DUT 202 Extended data 203 voice data 204 video data 205 header 206 Sub video data

─────────────────────────────────────────────────── --- Continuation of the front page (56) Reference JP-A-7-334938 (JP, A) JP-A-8-223531 (JP, A) JP-A-8-45249 (JP, A) JP-A-8- 116512 (JP, A) JP 7-87443 (JP, A) JP 7-284060 (JP, A) JP 10-200854 (JP, A) JP 11-355722 (JP, A) (58) Fields investigated (Int.Cl. ⁷ , DB name) H04N 5/91-5/956 H04N 7/24

Claims (3)

(57) [Claims] 1. A plurality of video frames are GOPs, and G
GOP video data in which OP is highly efficiently encoded, and the GO
G which is a high-efficiency coded audio frame corresponding to P
A method of multiplexing OP audio data, comprising: forming a packet having a packet header, one or more of the GOP video data and one or more of the GOP audio data, wherein the packet header includes at least each GOP video. Data and position information and synchronization information of each GOP audio data are provided.
The position information is GOP video data and GOP audio.
The relative position in the packet of the start position of the data
The synchronization information is the first in the GOP video data.
The playback start time of the video frame to be played, and the video
The playback start time of the audio frame corresponding to the frame
And the playback start time and the previous of the audio frame
A high-efficiency coded data multiplexing method, which is a difference value between the reproduction start times of the video frames . 2. A plurality of video frames are GOPs, and G
GOP video data in which OP is highly efficiently encoded, and the GO
G which is a high-efficiency coded audio frame corresponding to P
A method for multiplexing OP audio data, comprising a packet header and one or more GOP video data.
And one or more of the GOP audio data
Packet, and the packet header contains at least each GOP video data.
And position information and synchronization information of each GOP audio data
The position information is in the frame of the GOP video data.
The beginning of the encoded video frame within the packet
Corresponds to position and intra-frame encoded video frame
Position of the audio frame to be played in the packet
And the synchronization information is the intra-coded video frame.
Playback start time and the audio corresponding to the video frame.
The playback start time of the audio frame,
Playback start time of the video frame and
High-efficiency code characterized by the difference value of the start time
Data multiplexing method. 3. The one or more GOP video days.
Data, the one or more audio data and the packet.
Memory for recording the packet header and the packet header
A header generation circuit for generating,
One or more GOP video data and the one or more GOP video data;
Before counting the code amount of several GOP audio data
The packet header is generated and the packet header is generated.
The packet header is recorded in the memory every time
Header, the one or more GOP audio data
The order of the one or more GOP video data, or the
Packet header, the one or more GOP video data
In the order of the GOP audio data and the one or more GOP audio data.
The packet is output from the memory, and the packet is output.
A high-efficiency coded data that realizes the multiplexing method of Requirement 2
Duplication device.