JPH08289255A - Encoding device, decoding device and coding/decoding device - Google Patents

Abstract

PURPOSE: To start the reproduction of each raw material at designated time and to reproduce plural raw materials successively by multiplexing compression data by attaching a specific code which supplies operating timings of start and completion according to a multiplex format depending on only the time length of the raw material and whose leading packet size is larger than that of the following packet by first buffet quantity. CONSTITUTION: When an encoder part 11 performs coding and multiplexing, a mark is attached at a 'position equivalent to a time when a demultiplexer 16 issues an operation instruction to decoders 17, 18' in a data file. Also, when the demultiplexer 16 detects the mark from the data file, a controller 20 sends out control signals S4, S5 to the video decoder 17 and the audio decoder 18 corresponding to the content of the mark based on a mark detection signal. In this way, the controller 20 drives the video decoder 17 and the audio decoder 18 at required timings, respectively in this coding/secoding device 10.

Description

Detailed Description of the Invention

[0001]

[Table of Contents] The present invention will be described in the following order. Field of Industrial Application Conventional Techniques Problems to be Solved by the Invention Means for Solving the Problems Action Example Effect of Invention

[0002]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an encoding device, a decoding device and an encoding / decoding device, and is suitable for application to, for example, an AV server system.

[0003]

2. Description of the Related Art Conventionally, there have been various methods for digital compression of video signals and audio signals. Among them, one of the methods that is currently receiving the most attention is MPEG (Moving).
Pictur Experts Group) standard. AV (Audio Visual: Audio / Video) compressed according to this MPEG standard
When reproducing (decoding) data, it is necessary to synchronize AVs, and there are roughly two types of methods, an implicit synchronization method and a time stamp method.

In the implicit synchronization method, AV data corresponding to almost the same time and the same time length are multiplexed in a certain unit, and the AV head is set at the head of one AV material (hereinafter referred to as clip). This is a method for achieving AV synchronization by adjusting the respective decoding start times so that On the other hand, the time stamp method is a time stamp indicating the reproduction time for each unit of AV data when the AV is multiplexed.
Stamps (DTS: Decoding Time Stamp and PTS: Presenta
option time stamp), and a time stamp (SCR: Sy
stem Clock Refernece or PCR: Program Clock Refern
ece) is added to each unit and each decoder executes reproduction in accordance with this time stamp to establish AV synchronization. The MPEG system standard adopts this method.

Generally, the time stamp method has higher accuracy than the implicit synchronization method, but has a feature that the implementation cost becomes higher because the mechanism becomes complicated. A little more about how to use time stamps. As shown in FIG. 17, the demultiplexer 1 extracts the reference time information PCR from the supplied AV multiplexed data D _Av and adjusts the system clock CL1 so as to synchronize with this value. The system clock CL1 is given to the audio decoder 2 and the video decoder 3, respectively.

The demultiplexer 1 also supplies the audio data D _Ad extracted from the AV multiplexed data D _Av to the audio decoder 2 together with the reproduction time information DTS, and the extracted video data D _Vi together with the reproduction time information DTS and PTS. It is supplied to the video decoder 3. As a result, the audio decoder 2 and the video decoder 3 have the system clock CL1 and the reproduction time information DTS and PTS, respectively.
It is arranged to judge when to start the reproducing operation by comparing these with each other.

[0007]

Consider a case where a plurality of clips are continuously reproduced. In the case of the time stamp method, as shown in FIG. 18, the data of the next clip (clip 2) has already begun to be input to the demultiplexer 1 at the time near the clip connection, and the system clock CL1 is also clipped. Although it is synchronized with the reference time information PCR of 2, the audio decoder 2 and the video decoder 3 may have a time zone T _{0 in} which the previous clip (clip 1) is still being processed. In this case, since the reference time information PCR of each clip 1 and 2 is independent, if the audio decoder 2 and the video decoder 3 try to decode according to the system clock CL1 during this time zone T ₀ , they may not be reproduced correctly. There is.

As described above, the conventional time stamp system has a problem that it is difficult to continuously reproduce a plurality of clips. In order to solve such a problem, conventionally, as a method of realizing continuous reproduction of a plurality of clips, two sets of audio decoders 2 and video decoders 3 are prepared, and these sets are alternately used to output the output of the demultiplexer 1. The method of switching is widely used. According to this method, it is also possible to adopt the time stamp method as the AV synchronization method.

However, with such a method, it is necessary to prepare two sets of the audio decoder 2 and the video decoder 3, and a device for performing switching is also required, which causes a problem that the system scale becomes large. The present invention has been made in consideration of the above points, and an encoding device, a decoding device, and an encoding / decoding device that can start reproduction of each clip at a designated time and can continuously reproduce a plurality of clips. It is intended to propose a digitalization device.

[0010]

In order to solve such a problem, according to the present invention, on the encoding side, it is obtained by compressing and encoding the video signal and the audio signal of a material consisting of a series of the video signal and the audio signal. Start and operation of the compressed video data and compressed audio data, which are dependent only on the time length of the material, and in accordance with a predetermined multiplexing format which is larger by a first buffer amount than a packet in which the size of the first packet is continuous. The special code that gives the stop timing is put everywhere, and when the amount of compressed video data is less than the maximum value that can be generated as the compressed video data of the material for that time length, the dummy data is used for the shortage. Is added to compressed video data and multiplexed with compressed audio data, while decoding The decoding of the compressed audio data based on the special code is on the side, only the reproduction time difference between the compressed video data and compressed audio data to be stopped.

Further, according to the present invention, when decoding the multiplexed compressed video data and compressed audio data forming the first material supplied from the encoding unit via the data supply unit, the first material is decoded. The decoding operation for the compressed audio data by the second decoding means is stopped by the reproduction time difference between the compressed video data and the compressed audio data according to the length of time.

[0012]

The operation of the compressed video data and the compressed audio data is started according to a predetermined multiplexing format which depends only on the time length of the material and which is larger than the succeeding packet by the first buffer amount by the first packet size. A special code that gives the timing of operation stop is put everywhere while being multiplexed, and when the amount of compressed video data is less than the maximum value that can be generated as the compressed video data of the material for that time length, a dummy amount is added. While adding data to compressed video data and multiplexing with compressed audio data, the decoding side performs decoding processing for compressed audio data based on this special code.
By stopping the playback time difference between the compressed video data and compressed audio data, the playback of the first material of a series of each material is started at the specified time, and then these multiple materials are played continuously. You can

Further, when decoding the multiplexed compressed video data and compressed audio data forming the first material supplied from the encoding unit through the data supply unit, respectively, depending on the time length of the first material, By stopping the decoding operation for the compressed audio data by the second decoding means by the difference in the reproduction time between the compressed video data and the compressed audio data, it is possible to continuously reproduce the plurality of first materials. .

[0014]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of the present invention will be described in detail below with reference to the drawings.

[1] First Embodiment (1) Utilization Technology (1-1) Video Compression According to MPEG Standard (1-1-1) Virtual Buffer When compressing video data according to the MPEG standard, each frame (image) is compressed. The bit amount generated due to the change. If this bit amount is too large or too small, the buffer on the decoder side may overflow or underflow. For this reason, generally, the buffer of the decoder is assumed to be a certain amount, and the supplied video data is encoded while driving and controlling the encoder so that the buffer does not fail. Hereinafter, such a buffer will be referred to as a virtual buffer.

(1-1-2) Start-up delay The image is not immediately reproduced when data is input to the decoder buffer at the time of decoding, but the reproduction is started after a certain time has elapsed. This is to prevent the decoder buffer from failing. Hereinafter, this time is referred to as a start up day. The start-up delay is not the same no matter how the video material is compressed, and generally differs depending on the video material. Figure 1 shows the virtual buffer 4 and the start-up delay. In FIG. 1, T ₁ is one frame time, T ₂ is a start-up delay, and V _b is the capacity of the virtual buffer 4. Further, here, since it is considered to supply the data to the video decoder 5 at a constant speed, the line showing the accumulated amount of the input data is a straight line.

In such a virtual buffer 4, as shown in FIG. 2A, for example, if the decoding start is too late with respect to a predetermined start-up delay T ₂ , there is a risk of overflow. On the other hand, as shown in FIG. 2 (B), if the decoding starts earlier than the predetermined start-up delay T ₂ , there is a risk of underflow.

(1-1-3) Data amount of compressed video data file Hereinafter, a data group obtained by encoding one video material will be referred to as a "compressed video data file". Usually, the data amount of the compressed video data file generated from different video materials having the same time length is not the same, and as is clear from FIGS. 3A and 3B,
Assuming that the data amount of the compressed video data file is V _V , the video compression bit rate is R _V , the capacity of the virtual buffer is V _b , and the time length of the video material is T ₄ , the following equation is obtained.

[Equation 1] As described above, ± capacity of the virtual buffer varies within the range of V _b .

(1-2) Continuous reproduction of compressed video data file 1. The MPEG method is adopted as the video data compression method. 2. The compression bit rate of video data is constant. 3. Start up of each compressed video data file
Daylays are not necessarily equal. 4. The data amount of each compressed video data file is not necessarily equal to the product of the compression bit rate and the reproduction time (time length of video material). Under the condition, in order to continuously reproduce (decode) the compressed video data file, 1. Prepare at least three times (capacity or number) of the virtual buffer as the video decoder file. 2. The leading file of the compressed video data file group to be continuously reproduced starts decoding after the amount of data of the virtual buffer has accumulated in the video decoder buffer. 3. Data is transferred to the decoder at a transfer rate that allows transfer of the maximum video data file amount within the playback time. That method is effective.

The method will be described below. Consider a case where two compressed video data files F ₁ and F ₂ are continuously reproduced in this order. At this time, the compression bit rate is both R _v, and the size of the virtual buffer is V _b . The reproduction times are T _F1 and T _F2 , and the start-up delay is Td ₁ and Td ₂ , respectively.

(1-2-1) Continuous reproduction of compressed video data file For simplicity, the data amount of each compressed video data file is equal to the product of the compression bit rate and the reproduction time (the data amount can be specified). I assume. If Td ₁ = T
If d ₂ , these two compressed video data files F
_{If 1} and F ₂ are continuously supplied to the decoder at the transfer rate of the compression bit rate R _v , these two compressed video data files F ₁ and F ₂ are continuously reproduced. However, when Td ₁ <Td ₂ , decoding of the compressed video data file F ₂ starts earlier than Td ₂ , and there is a risk that the virtual buffer will underflow.

On the contrary, when Td ₁ > Td ₂ , the decoding start of the compressed video data file F ₂ becomes later than Td ₂ , and there is a risk that the virtual buffer overflows. The maximum start-up that can occur in this case
The delay is equal to "time (T _b = V _b / R _v )" required to transfer data corresponding to the capacity of the virtual buffer at a transfer rate equal to the compression bit rate.

[0023] In the first file of the slave go-between continuous playback to be compressed video data file group, regardless of its start-up-& Deirei, when to start decoding from the wait only this time only T _b, buffer Never underflow. That is, the decoding may be started after the data of V _{b has} accumulated in the video decoder buffer. The minimum start-up-& Deirei that can occur is "0", and delaying the start of decoding such files only T _b, worst V _b
The buffer overflows by the amount. Therefore, if the video decoder buffer is prepared twice as much as the virtual buffer, it will not overflow.

(1-2-2) Consideration of Data Insufficiency For simplicity, start each compressed video data file.
Assuming that the up delays are the same (the start up delay can be specified), FIGS. 4 (A) and 4 (B)
Will be described with reference to. In general, the size V _V of the compressed video data file does not match the product of the compression bit rate Rv and its reproduction time T (V _V ≠ R _v T). Therefore, when data is supplied to the decoder at a transfer rate equal to the compression bit rate, the input Time required (V _V / R _v ) and output (playback)
Does not match the time T required for.

In the compressed video data file F ₁ ,
Decoder input required time is longer than output required time (V _V1
In the case of / R _v > T _F1 ), the decoding start of the compressed video data file F ₂ is later than the start-up delay Td ₂ and there is a risk of buffer overflow. On the contrary, in the compressed video file F ₁ , the time required for input is shorter than the time required for output (V _V1 / R _v <T _F1 ).
In this case, the decoding of the compressed video data file F ₂ will start earlier than the start up delay T _F2 .
There is a risk of buffer underflow.

The maximum amount of compressed video data that can be generated is
The following formula

[Equation 2] Is. Therefore, when the actually obtained compressed video data amount is less than this value, the "dust" data D _d is added to the end of the file to set the size V _V of any file to V _vmax . To do so. According to the MPEG standard, “0” can be added as “dust” data (hereinafter, “dust” data D _d added for file size adjustment in this way is referred to as “pad”). A transfer rate (R _v ′ = R _v), which is sufficient to transfer all the data of the file thus adjusted in size in time T
If input to the decoder at + V _b / T)
As shown in (B), since both the input required time and the output required time are T, the buffer failure as described above does not occur.

However, since this transfer rate R _v ′ is higher than the original transfer rate R _v, there is a risk that the buffer will overflow regardless of whether continuous reproduction is performed. In the worst case, the buffer overflows by V _b, so in order to prevent such a situation, the video decoding buffer should be prepared twice as much as the virtual buffer. From the above, considering that the start-up delay is uncertain and the data amount is uncertain after all, three times (capacity or number) of the virtual buffer should be prepared as the video decoder buffer. I understand.

(1-3) Audio compression according to MPEG standard Unlike audio data with audio data, when compressed according to the MPEG standard, there is no equivalent to the start up delay of the video data. However, since the amount of compressed data matches the product of the compressed bit rate and the time length of the material, it can be played back continuously if multiple compressed audio data files are continuously supplied to the decoder without any special measures. It

(1-4) Minimum Unit of Compressed Data A video signal is handled in a unit called frame, and MP
When compressed based on the EG standard, GOP (Gr
oup Of Picture) will be treated as a unit. For example, in the case of NTSC system, the frame is 29.97 [Hz]
That is, 1 GOP is normally composed of 15 frames. In the case of the PAL system, the frame is 25 [Hz],
GOP consisting of 12 frames and GOP consisting of 13 frames
The structure is repeated alternately.

On the other hand, when audio is compressed based on the MPEG standard, for example, the sampling frequency is 48 [KHz].
And the number of samples is 1152, the minimum unit is
It becomes 24 [ms]. As described above, since there is a minimum unit that can be handled in compressed data, it is not possible to strictly realize a clip with an arbitrary time length.
For the sake of simplicity, the minimum time length unit of the clip is 1 second. Even if such a restriction is imposed, there is almost no problem in actual operation. At this time, the time length of the clip is T, and the closest feasible video time equal to or longer than this time is T _v . In addition, the audio time that is the closest to this time and is less than the time T _v is T _a .

In the case of the PAL system, T _v = T, but in the case of the NTSC system, T _v ≠ T.
Specific numerical examples are shown in FIG. In this example, the video is based on the NTSC system, and the audio is based on the minimum unit of 24 [ms]. In the case of this example, there are 24 kinds of error T _VX −T _c between video and audio, from 0 to 23.

(2) Configuration of Encoding / Decoding Device According to Embodiment In FIG. 6, reference numeral 10 indicates an encoding / decoding device as a whole, and an encoder unit 11 supplies a video signal and an audio signal based on the supplied video audio signal S1. After being encoded and multiplexed, the compressed AV multiplexed data D1 obtained is supplied to the data supply unit 12. Data supply unit 12
Records the supplied compressed AV multiplexed data D1 in the recording medium 14 under the control of the controller 13. In addition, the data supply unit 12 reads the compressed AV multiplexed data D2 recorded in the recording medium 14 in file units under the control of the controller 13, and supplies this to the demultiplexer 16 of the decoder 15 at a constant speed.
The demultiplexer 16 divides the supplied compressed AV multiplexed data D2 into compressed video data D3 which is a compression-encoded video signal and compressed audio data D4 which is a compression-encoded audio signal, and these are respectively divided. It is sent to the video decoder 17 and the audio decoder 18.

At this time, the controller 13 of the data supply unit 12 sends the control information signal S3 to the controller 20 through the shared memory 19, and the controller 20 sends the control signals S4 and S5 based on the control information signal S3 to the video decoder 17 respectively. , And is supplied to the audio decoder 18. Thus the video decoder 17
In accordance with the control signal S4, the compressed video data D3 is sequentially stored in the buffer 21, and while being read, the compressed video data D3 is sequentially decoded and output. Similarly, the audio decoder 18 sequentially stores the compressed audio data D4 in the buffer 22 on the basis of the control signal S5, and while decoding the data, sequentially decodes and outputs the data.

In the case of this embodiment, as the buffer 21 corresponding to the video decoder 17, a buffer having a capacity three times as large as the virtual buffer is used. As a result, it is possible to prevent the start-up delay of the compressed video data D3 supplied from being indefinite and the buffer 21 from failing (overflow or underflow) due to an indefinite amount of data. Further, in the case of this embodiment, FIG.
As shown in (B), the demultiplexer 16 divides the compressed video data D3 and the compressed audio data D4.
Is transmitted as it is to the video decoder 17 or the audio decoder 18 in a burst manner without speed conversion.

In this case, originally, the data compressed at a predetermined compression bit rate should be supplied to each of the decoders 17 and 18 at a constant speed as shown in FIG. 7A. A circuit for speed conversion is required in the demultiplexer 25. Therefore, in the encoding / decoding device 10 of the embodiment, even if the data is burst-input to the video decoder 17 and the audio decoder 18, the buffers 21 and 22 can be reproduced without being damaged. The format is defined as follows.

(3) Realization of continuous reproduction and scheduled reproduction In the encoding / decoding device 10 of this embodiment, an implicit synchronization method which does not use a time stamp is adopted as a method for synchronizing AV. In this case, in order to continuously reproduce the clip while maintaining AV synchronization in such a system, Even if the clips are consecutive, the heads of video data and audio data should be aligned at the head of each clip. It suffices to satisfy two points that each decoder buffer of video data and audio data does not fail during reproduction of each clip even if the clips are consecutive. A method for satisfying each of these conditions will be described below.

(3-1) AV Head-Aligning Method for Realizing Scheduled Playback and Continuous Playback Currently, there are various types of decoders, but in the encoding / decoding device 10 of the embodiment, As the video decoder 17, "1.
"Playback that starts after 5 hours (0.05 seconds)" is applied, and as an audio decoder, "Playback starts immediately after receiving a start command (normal decoder specification)" is applied, and The scheduled playback is realized by the sequence.

1. Before the time T _S seconds (the time required to transfer the compressed video data for the virtual buffer to the buffer 22 of the video decoder 18) at the time when the reproduction is desired to be started, the controller 13 (FIG. 1) of the data supply unit reads the recording medium 14 from the recording medium 14. Start data transfer. This time T _S seconds is the same for any clip as described later. 2. When the demultiplexer 16 receives the leading data of the first clip, it notifies the controller 20 of the decoder 15 that the data transfer has started. 3. After (T _S -0.05) seconds have elapsed, the controller 20 issues a reproduction operation start command to the video decoder 17. 4. After a further 0.05 second has elapsed, the controller 20 issues a reproduction start command to the audio decoder 18.

As a result, the reproduction of the video data and the audio data is started at a predetermined time at the same time, but in order to realize such a method, it is necessary to measure the elapsed time. However, the encoding / decoding device 10 of this embodiment is
Since the compressed AV multiplexed data D1 is transferred to the demultiplexer 16 at a constant speed, the elapsed time can be measured if the number of data passing through the demultiplexer 16 is counted.

Therefore, in the case of this embodiment, when the encoder section 11 encodes and multiplexes the video signal and the audio signal without using a counter or the like, "the multiplexer 16 is assigned to each of the decoders 17 and 18" in the data file. On the other hand, a special code (hereinafter referred to as a mark) is inserted as a mark at a position "corresponding to the time when the operation command should be issued". Further, when the demultiplexer 16 detects this mark from the data file, the demultiplexer 16 transmits this to the controller 20 as a mark detection signal S10 (FIG. 6), while the controller 20 responds to the contents based on the mark detection signal S10 in accordance with the contents of the video decoder. The control signals S4 and S5 are sent to the audio decoder 17 and the audio decoder 18. As a result, in the encoding / decoding device 10, the controller 20 causes the video decoder 17 and the audio decoder 18 to operate at the timing as described above.
Can be driven respectively.

As this mark, MP is used so that it can be identified from the reproduction compression multiplexed data D2.
According to the EG system standard, the mark uses a 4-byte code which is guaranteed not to appear in the reproduction compression multiplexed data D2. In the case of this embodiment, the codes "000001BD" and "000001BF" in hexadecimal notation are used as the video decoding start mark that causes the video decoder 17 to start decoding and the audio decoding start mark that causes the audio decoder 18 to start decoding. are doing.

Further, the transfer rate of the compressed AV multiplexed data D2 is set to 5 [Mbps], and the compressed reproduction multiplexed data D2 is transferred to the buffer 21 of the video decoder 17 to transfer the compressed video data D4 corresponding to the capacity of the virtual buffer. [Mbit] As shown in FIG. 8, it is necessary to supply the following formula from the beginning.

(Equation 3) Place the video decoding start mark M ₁ “000001BD” at the position of

[Equation 4] The audio decoding start mark M ₂ "000001BF" is placed later. Next, an AV head alignment method for realizing continuous reproduction will be described.

As described above in "(1-4) Minimum unit of compressed data", the reproduction times of the video data and the audio data do not match. For example, time is 6
For clips that are seconds, the video will take 0.006 seconds longer to play. Therefore, when two 6-second clips are continuously reproduced, even if the first clip is completely AV-synchronized, the audio reproduction of the second clip is delayed by 0.006 seconds. With this difference, you don't actually feel any strangeness,
This difference accumulates as the number of consecutive clips increases, so if you continue 10 clips, for example, the difference between audio and video will be 0.06 seconds, and you will feel something strange. Therefore, in the encoding / decoding device 10 of this embodiment, the head of each clip is made to match the head of AV by the following method. 1. When the audio reproduction time T _c has elapsed since the reproduction was started, the audio decoder 18 is stopped. 2. Further, when the time elapses by the reproduction time difference ( _Tv - _Tc ), the audio decoder 18 is operated again. 3. This operation is repeated for each clip. In this case, in order to realize this method, it is necessary to measure the elapsed time as in the case of the regular playback.

Therefore, in this embodiment, as in the case described above, the encoder section 11 also adds a predetermined mark to a predetermined position of each data file. For example, if the 5 of the clip of T = 6 [s], since the reproduction time difference is 0.006 seconds, audio decoder prior to the position of the audio decoding start mark M ₁ 1
In FIG. 8, audio decoding stop marks M _{n1 to} M _n23 (FIG. 8) for stopping decoding are inserted. In this case, when the minimum unit of compressed audio data is 24 [ms], there are 24 reproduction time differences from 0 to 23 [ms], but the difference is 0 [m
s], it is not necessary to stop the audio decoder 18, so audio decoding stop marks M _{n1 to} M
_{As n23} , 23 pieces should be prepared.

For this reason, in the encoder section 11, as the audio decoding stop marks M _{n1 to} M _n23 , for example, 23 codes "000001C1" to "000001D7" are associated with reproduction time differences of 1 to 23 [ms]. ing. Therefore, these codes are 5 [Mbps] x 0.001 [s] = 625
It will appear every [byte]. Also this encoder 5
Is added to the end of each clip with a mark (hereinafter, referred to as a stop mark designation mark M ₃ , for example, “000001D8”) indicating which of these 23 codes should be used for stopping the decoding. After that, the effective audio decoding stop mark M ₄ is continuously added. For example, in the case of a clip of T = 6 [s], since the reproduction time difference is 0.006 [s], it is necessary to validate the audio decoding stop mark M _n6 . Therefore, the last 8 [byte] of this clip is " 000001D8000001C6 ”.

Clip 1 (T = 6 [s]) and clip 2
When continuously reproducing (T = 7 [s]), it is known from the stop mark designation mark M _{3 of} the clip 1 that the audio decoding stop mark M _n6 is valid, and therefore the audio decoding stop mark M of the clip 2 is reproduced. _{When n6} is detected by the demultiplexer 16, a stop command is issued to the audio decoder 18, and when the audio decode start mark M _{2 of the} clip ₂ is detected, a start command is issued to the audio decoder 18. There is. It should be _noted that in order to clear each valid stop mark M _{n1 to} M _n23 before starting the continuous clip reproduction, "0000000"
By setting it to "0", the stop mark M of the first clip
_A stop command is issued at _{n1 to Mn23} .

(3-2) Method of Using Buffer for Realizing Continuous Playback (3-2-1) Basic Structure of Compressed AV Multiplexing Format Hereinafter, a block of the same type of data will be referred to as a “packet”. Here, the type means three types of video, audio and packet. Attach a head to the head of each packet to indicate its type. A group of several packets will be called a "pack". Unlike a packet, a packet does not have any identifying header. Here, in the encoding / decoding device 10 of this embodiment,
The following three types of packs are used. 1. Standard packet: As a general rule, it consists of one video packet, one audio packet, and one packet packet. 2. First pack ... The amount of data in the video packet is larger than the standard pack by the virtual buffer. 3. Residual pack: It consists of the remaining data that is not included in the above-mentioned two types of packs.

In the case of this embodiment, one compressed AV multiplexed data file obtained by encoding one clip has one "head pack" at its head as shown in FIG. 9 (A). After that, a plurality of "standard packs" are arranged in series, and finally one "remaining pack" comes. Since all the data control marks are located in the video packet of the head packet, the marks are not inserted in the contents of the packet header and the packet header cannot be recognized.

(3-2-2) Packet / Header As a header for identifying the type of each packet, it is necessary to adopt a pattern that never appears in the compressed data. Therefore, in this encoding / decoding device 10, MP
The pattern reserved for Hedda in the EG system standard is adopted. In practice, for example, the following pattern is used. Each header is displayed in hexadecimal and its size is 4 bytes. Video packet header ... 000001E3 Audio packet header ... 000001C0 Pad packet header ... 000001BE Thus, the demultiplexer 16 checks any continuous 4 bytes of the input compressed AV multiplexed data D2. If it is a video packet header, the following data is distributed to the video decoder 17, and if it is an audio packet header, the data that follows is distributed to the audio decoder 18, whether it is a packet packet data. For example, the following data is discarded.

(3-2-3) Limitation on compression bit rate value, etc. The maximum compressed video that can occur when a clip with a time length T is compressed with a virtual buffer amount V _b and a video compression bit rate R _v.・ The data volume is

(Equation 5) Given in. Therefore, in the encoding / decoding device 10 of this embodiment, if the actually obtained compressed video data amount is less than this value, the pad data is added to the end of the compressed video data amount so that the size of an arbitrary file is reduced. _{Is set} to V _vmax . When all the data after the size adjustment is transferred in the time T, the video data transfer rate R _v ′ is expressed by the following equation.

(Equation 6) R _v ′ becomes maximum R _{v max} ′ when the clip time length is minimum T _min .

(Equation 7) Compressed AV multiplexed data transfer rate is R _m, it must be greater than the sum of the least compressed video data rate compressed audio data rate. For audio, a transfer rate equal to its compression bit rate is sufficient. Therefore, these parameters must satisfy the following equation. (To be precise, it is necessary to consider the head and mark, but here it is omitted for simplicity.)

(Equation 8)

[Equation 9]

In practice, in this encoding / decoding device 10, R
Parameters such as _v , R _m , R _a , T _min , and V _b cannot be set to arbitrary values, and are limited depending on their use and hardware convenience. Therefore, in the encoding / decoding device 10, the parameter having the strongest limitation is determined among these parameters, and the parameters are sequentially selected so as to finally satisfy the equation (9).

(3-2-4) Calculating Packet Size When a parameter value such as a compression bit rate value is given, an equation for calculating the size of each packet should be calculated. It is shown below. First, the variables used in the following calculations are listed. Video compression bit rate …… R _v [Mbps] Video data transfer rate …… R _v ′ [Mbps] Audio compression bit rate …… R _a [kbps] Compressed AV multiplexed data transfer rate …… R _m [Mbps ] Clip time length (1 second unit) ...... T [s] Minimum clip time length (1 second unit) ...... T _min [s] Video reproduction time that is closest to T and is closest to T above T ……
T _v [s] T _v, which is the closest to T and is the most realistic audio playback time. T _a [s] Time required to transfer the clip ... T _t [s] Compressed video data for a virtual buffer Time required for data transfer ... T _S [s] Virtual buffer amount ... V _b [Mbit] 1 clip compressed video data amount ... V _v [byte] 1 clip compressed audio data amount ... V _a [byt
e] 1 clip pad data amount ・ ・ ・ V _p [byte] 1 clip total data amount ・ ・ ・ V _m [byte] All mark data amount ・ ・ ・ V _k [byte]
(= 4 x 27 [byte]] Packet / header size ...... L _h [byte]
(= 4 [byte]) Standard video packet size (heads not included) ...
… L _v [byte] Standard audio packet size (heads not included)… L _a [byte] Standard pad packet size (heads not included)…
… L _p [byte] Standard pack size (including header)…
... L _m [byte] Number of packs (excluding residual packs) ...
... P * Standard packet means a packet within a standard packet. Residual video packet size (heads not included) ...
… L _v ′ [byte] Residual audio packet size (head not included) …… L _a ′ [byte] Residual pad packet size (head not included)…
... L _p ′ [byte] * The residual packet means a packet in the residual pack. Units conversion factor _{(Mbit → byte) ...... C m} = 125000 = 1000 × 1000/8 units conversion factor _{(kbit → byte) ...... C k} = 125 = 1000/8

When T is set in units of 1 second, for the sake of simplicity, the size L of the standard packet in a clip with an arbitrary time length is used.
_In order to make _v and L _a equal to each other, R _v , R _a , L _v , and L _a may be selected so as to satisfy the following equation.

[Equation 10]

[Equation 11]

(Equation 12) Normally, R _v is at most 0.1 [Mbps] unit, and R _a is
One of 64, 128, 192, 256, and 384 [kbps]. Therefore, there exist integers L _v and L _a that satisfy the above equation. Fractional data generated by the limitation of the minimum unit of compressed data, that is, R _v (T _v − in video)
T) and R _a (T _a −T) worth of data in the audio form a residual pack, and the remaining data, that is, R _v T + V _b in the video and R _a T in the audio are used as the initial pack. Try to configure.

In this way, the following equation holds for R _a and R _v .

(Equation 13)

[Equation 14]

(Equation 15)

[Equation 16] (14) and (16) from R _v, R _a, be given a L _a, it can be calculated L _v. The amount of audio data V _a is

[Equation 17] Video data volume V _v

(Equation 18) The amount of multiplexed data V _m is

[Formula 19] Therefore, the pad data amount V _p is

(Equation 20)

[Equation 21] Yotsute

[Equation 22]

(Equation 23)

From equation (24), L _p and L _p ′ are obtained. When T _a <T, there are integers K and L _a ″ that satisfy the following equation.

[Equation 24] At this time, the data size of the audio packet is set to L _a − for the K + 1th pack from the end of the standard pack.
L _a ″, and immediately after this packet, the size is L _a ″ −L
Make sure to put the _h -packet / packet. Then, for the K packs from the end of the standard pack, put the same size of pad pack in place of the audio pack. In addition, the following equation holds for the residual pack.

(Equation 25)

(Equation 26)

(3-2-5) Conditions for realizing continuous reproduction As described in "(1-2) Continuous reproduction of compressed video data file", even if only compressed video data is continuous, In order to reproduce, it is necessary to prepare three times as many video decoder buffers as virtual buffers. Even in the case of multiplexing, the conditions for continuous reproduction are shown if the video decoder buffer and the audio decoder buffer are prepared three times as much as the virtual buffer. First, as shown in “(1-2-1) Consideration of indefinite start up / day delay”, the start up / up
In order to deal with the indefinite delay, twice the capacity of the virtual buffer is required. This double buffer covers the "transmissible area of the data output broken line" in FIG. Therefore, if the data input broken line can be changed within the remaining 1 × buffer, continuous reproduction can be performed with the 3 × buffer.

The video decoder buffer 17 has 1
There are times when only the data of one clip is entered and times when the data of two clips are entered. The time that two clips are populated is only the first T _S time of each clip if T _t = T _v . This time is the time when the top packet is transferred for an arbitrary clip, and since the audio packet and the pad packet are not included other than the beginning, as is clear from FIG. 10, the input polygonal line is always a straight line. Is also below. That is, the data input polygonal line does not appear in the 1 × buffer. Regarding the time when only one clip of data is contained (the time after the elapse of T _S ), it may be arranged so as to fall between the straight lines in FIG. For that purpose, the following condition may be satisfied. Condition 1: Make sure that the straight lines do not intersect at points other than the origin.

That is, for any t> 0, v ₃ (t)>
It suffices that v ₁ (t) holds.

[Equation 29]

[Equation 30] Condition 2: The straight line and the straight line do not intersect until the time t _e when the transfer of the video data ends. That is, v ₄ (t
_{e) <v 2 (t e} ) it is sufficient to hold.

[Equation 31]

[Equation 32]

[Expression 33]

The equations for each straight line are as follows. Strictly speaking, it is necessary to consider the amount of headers and marks, but since it is very small compared to the amount of compressed data, it is ignored.

(Equation 34)

[Equation 35]

[Equation 36]

(37) Therefore, in order to realize continuous reproduction at three times the virtual buffer, it is sufficient to set T _t = T _v and select each parameter so as to satisfy the equations (30) and (33). Also in the encoding / decoding device 10 of the embodiment, each parameter is selected in this way.

(3-2-6) Example of compressed AV multiplexing format capable of continuous reproduction R _m = 5 [Mbps], R _v = 4 [Mbps], _Ra = 256 [kb
ps], T _min = 6 [s], and V _b = 4 [Mbps] are shown in the format. However, the video is NT
The SC method is used. In this case, the right side of equation (9) is 5
[Mbps], the left side is the following formula

(38) Therefore, the expression (9) is satisfied, and it is understood that these values can be selected. For example, clip time length T
= 6 [s], from FIG. 5, T _t = T _v = 6.006 [s], T _a = 6 [s] Appropriately L _a = 1000 [byte] (If too large, the audio buffer fails. On the contrary, if it is too small, the number of standard packs will increase and the over head due to the header will increase).

[Formula 39] Calculate the size of the standard video packet from equation (16).

(Equation 40) Calculate the pad / packet size from equation (23).

[Formula 41] Therefore, L _p = 293 [byte] and L _p ′ = 74 [byte].

The size of the residual video packet is obtained from the equation (27).

[Equation 42] Since T _a = T, the size of the residual audio packet is zero. The size of the standard pack is calculated from the above values.

[Equation 43] At this time, it is understood that the equation (30) is satisfied. 0.92> 0.8 At this time, it can be seen that the equation (33) is also satisfied. 23.2 <24 The resulting format is shown in Fig. 12. Similarly, the calculation was performed for a clip time length of 7 seconds, and the result is shown in FIG. In this case, since T _a ≠ T, the last standard pack is different from the other standard packs, and there are pad packets corresponding to T _a -T. Once the value of each parameter is determined, regardless of the clip contents, for any clip, for each clip time length,
Since the format table as shown in FIGS. 12 and 13 can be created, when the compressed AV data is actually multiplexed, the compressed data obtained by each encoder may be multiplexed according to this table.

Therefore, the encoding / decoding device 1 of this embodiment is
At 0, the encoder unit 11
A table is stored in advance, and the encoder unit 11 multiplexes the coded video signal and the coded audio signal based on the format table.

(4) Operation of the First Embodiment With the above configuration, the encoding / decoding device 10
Video audio signal S1 supplied to the encoder unit 11
A video signal and an audio signal based on the above are encoded in this encoder unit 11, and these are multiplexed according to the above-mentioned first multiplexing format to obtain a compressed AV.
To form a multiplexed data D1, this time for each clip, the video decoding start mark M _1, audio decoder start mark M _2, audio stop mark M _S1 ~M
_S23, stop mark designated mark M ₃ and audio decoding stop mark M ₄ (hereinafter, collectively decoder control mark M ₁ ~M ₄ and, M _S1 ~M _S23 and referred) was added to each predetermined position, which data The data is supplied to the supply unit 12 and recorded on the recording medium 14. The compressed AV multiplexed data D1 is read out in data file units under the control of the controller 13 and supplied to the demultiplexer 16 as compressed AV multiplexed data D2, and the demultiplexer 16 compresses the compressed video data D3 and the compressed audio data. It is divided into D4 and sent to the video decoder 17 or the audio decoder 18, respectively. In this case the demultiplexer 16 detects this decoder control mark from the data file _{M 1 ~M 4, M S1 ~M} S23, and sends the mark detection signal S10 based on the detection result to the controller 20. Based on the mark detection signal S10, the controller 20 outputs control signals S4 and S5 to the video decoder 17 and the audio decoder 18, respectively, to drive and control the video decoder 17, so that the video decoder 17 and the audio decoder. The O-decoder 18 is started or stopped as needed. At this time, the controller 20 stops the reproduction operation of the audio decoder 18 for a predetermined time according to the clip time length.

As described above, in the encoding / decoding device 10, the encoder control side decoder control marks M ₁ to M for control according to the clip time length for each clip.
M ₄ and M _{S1 to} M _S23 are inserted, and at the decoder side, the video decoder 17 and the audio decoder 18 are based on these decoder control marks M _{1 to} M ₄ and M _{S1 to} M _S23.
Since it is driven and controlled, the head of the AV can be adjusted by completing each clip. Also, this encoding / decoding device 1
At 0, the buffer 21 of the video decoder 17 is set to three times (capacity or number) of the virtual buffer, and furthermore, it depends only on the time length of the clip so as not to break the buffer 21 of three times, and The encoder 11 encodes the encoded video signal and the encoded audio signal according to the above-described first multiplexing format, which has the feature that the size of the first video packet is larger than the other video packets by the capacity of the virtual buffer. To multiplex
The buffer 21 of the video decoder 17 will not fail. Therefore, since the two conditions presented in “(3) Realization of continuous reproduction” are satisfied, continuous reproduction can be performed while achieving AV synchronization.

Accordingly, for example, a television broadcasting CM is applied as a clip material, this is MPEG-compressed, AV-multiplexed according to the above-described multiplexing format, and then H
By recording in a large-capacity, random-accessible recording medium such as a DD, and supplying a designated compressed CM data group to a decoder at a designated time according to a time table of a CM broadcast, the designated time can be set. Moreover, CM can be continuously broadcast.

(5) Operation of the First Embodiment According to the above configuration, on the encoder side, the decoder control marks M _{1 to} M ₄ , M _{S1 to} M for control according to the clip time length are provided for each clip. _S23 is inserted, and at the decoder side, these decoder control marks M _{1 to} M ₄ , M _S1
The video decoder 17 and the audio decoder 18 are drive-controlled based on M _S23 , while the encoder unit 5
A coded video signal and a coded video signal according to a first multiplexing format which has a characteristic that it depends only on the time length of the clip and the size of the leading video packet is larger than the other video packets by the capacity of the virtual buffer. By multiplexing with the audio signal
AV heads of each clip can be matched and buffer 21
Can be prevented, and thus an encoding / decoding device capable of continuously reproducing a plurality of clips at a designated time can be realized.

[2] Second Embodiment FIG. 14 in which parts corresponding to those in FIG. 6 are assigned the same reference numerals shows an encoding / decoding device 40 according to the second embodiment, in which the controller 42 of the decoder 41 The encoding / decoding device 10 of the first embodiment except that the driving of the video decoder 17 and the audio decoder 18 is controlled based on the reproduction time difference data between the video and the audio at each clip length stored in the internal memory in advance. The configuration is almost the same as (FIG. 6). That is, in the case of this encoding / decoding device 40, the encoder 43 is provided with the supplied audio / video signal S1.
The video signal and the audio signal based on the above are encoded respectively, these are multiplexed according to the above-mentioned first multiplexing format, and the compressed AV multiplexed data D30 is sent to the data supply unit 44. The data supply unit 44 records the supplied compressed AV multiplexed data D30 in the recording medium 14 under the control of the controller 45. The data supply unit 44 also controls the compressed AV multiplexed data D31 recorded in the recording medium 14 under the control of the controller 45.
Are reproduced in clip units and sent to the demultiplexer 46 of the decoder 41. The demultiplexer 46 is
The supplied compressed AV multiplexed data D31 is sequentially divided into compressed video data D4 and compressed audio data D5, which are sent to a video decoder 17 and an audio decoder 18, respectively. At this time, the controller 45 supplies the time length of this clip as the time information signal S30 to the controller 42 of the decoder 41 via the shared memory 19.

This controller 42 stores in advance the reproduction time difference data between the video and audio in each clip time length, and the supplied time information signal S3.
The control signals S4 and S5 are generated based on the time length of this clip based on 0 and the stored corresponding reproduction time difference data, and these are sent to the video decoder 17 and the audio decoder 18, respectively. As a result, the controller 42 stops the audio decoder 18 after the audio reproduction time Tc has elapsed since the controller 45 of the data supply section 44 started reproducing the clip from the recording medium 14, and then the reproduction time difference (Tv-Tc). )
After a lapse of time, the audio decoder 18 is driven and controlled so that the audio decoder 18 is operated again for each clip in sequence so that the video and audio heads can be matched at the head of each clip. Has been done. With the above configuration, in the encoding / decoding device 40, the data supply unit 44
When the controller 45 reproduces the compressed AV multiplexed data D31 recorded in the recording medium 14 in clip units, reproduction time information of this clip is supplied to the controller 42 of the decoder 41.

On the other hand, the controller 42 is the controller 4
Playback time information of this clip supplied from 5,
Based on the reproduction time difference data held by the controller 42 itself, the video decoder 17 and the audio decoder 18 are drive-controlled so that the head of each clip and the head of the audio match. As described above, in the encoding / decoding device 40, since the drive control of the audio decoder 18 is performed for each clip unit, the reproduction time difference between video and audio can be corrected completely within each clip. Thus, AV synchronization can be achieved. Also, the buffer 21 of the video decoder 17 is selected to have a capacity three times the capacity of the virtual buffer, and the encoder 43 multiplexes the supplied video signal and audio signal according to the first multiplexing format. Therefore, as in the case of the first embodiment, each clip can be continuously reproduced while AV synchronization is achieved.

According to the above configuration, the video decoder 17
The capacity of buffer 21 of the virtual buffer is three times as large as the virtual buffer, and encoder 43 is set to multiplex the supplied encoded video and audio signals according to the first multiplexing format. Supply unit 44
When reproducing the compressed AV multiplexed data D31 recorded in the recording medium 14 by the controller 45, the reproduction time information of this clip is supplied to the controller 42 of the decoder 41, and the controller 41 also reproduces this reproduction time information. And the playback time difference data held by the controller 41 itself, the video decoder 17 and the audio decoder 18 are driven and controlled so that the head of each clip and the head of the audio match. Similar to the encoding / decoding apparatus 10 of the embodiment, an encoding / decoding apparatus capable of continuously reproducing a plurality of clips at a designated time can be realized.

[0071]

As described above, according to the present invention, on the encoding side, compressed video data obtained by compression-encoding the video signal and audio signal of a material consisting of a series of video signals and audio signals. And the compressed audio data that depends on only the time length of the material and gives the timing of starting and stopping the operation in accordance with a predetermined multiplex format that is larger than the packet with the size of the leading packet by the first buffer amount. While putting the code everywhere, when the amount of compressed video data is less than the maximum value that can be generated as the compressed video data of the material for that time length, dummy data is added to the compressed video data by the shortage. While it is multiplexed with compressed audio data, the encoding side is based on this special code. By which is adapted to stop the encoding process only reproduction time difference between the compressed video data and compressed audio data to compression audio data,
It is possible to realize an encoding device, a decoding device, and a decoding encoding device that can start reproduction of each material at a specified time and can continuously reproduce a plurality of materials. When decoding the multiplexed compressed video data and compressed audio data forming the first material supplied from the encoding unit via the data supply unit, the compressed video data and the compressed video data according to the time length of the material are decoded. Since the decoding process for compressed audio data is stopped by the reproduction time difference between audio data, an encoding device, a decoding device and a decoding encoding device capable of continuously reproducing a plurality of materials are realized. it can.

[Brief description of drawings]

FIG. 1 is a graph and block diagram for explaining a virtual buffer and a start up delay.

FIG. 2 is a graph used to explain the failure of Start Up Daylay and Buffer.

FIG. 3 is a graph for explaining a data amount of a compressed video data file.

FIG. 4 is a graph for explaining adjustment of a data amount and a data transfer rate.

FIG. 5 is a chart showing feasible times of video and audio.

FIG. 6 is a block diagram showing the configuration of the encoding / decoding device in the first embodiment.

FIG. 7 is a block diagram showing a data transfer mode after demultiplexing.

FIG. 8 is a schematic diagram for explaining a decoder control mark.

FIG. 9 is a schematic diagram showing a format of a compressed AV multiplexed data file.

FIG. 10 is a graph showing a transition of an input to a video decoder.

FIG. 11 is a graph for explaining the usage status of the video decoder buffer during continuous reproduction.

FIG. 12 is a chart showing an example of a compressed AV multiplexing format.

FIG. 13 is a chart showing an example of a compressed AV multiplexing format.

FIG. 14 is a block diagram showing the configuration of an encoding / decoding device according to the first embodiment.

FIG. 15 is a chart showing an example of a compressed AV multiplexing format.

FIG. 16 is a chart showing an example of a compressed AV multiplexing format.

FIG. 17 is a block diagram for explaining a time stamp method.

FIG. 18 is a schematic diagram for explaining continuous reproduction in the time stamp method.

[Explanation of symbols]

10, 40 ... Encoding / decoding device, 11, 43 ... Encoder, 12, 44 ... Data supply unit, 13, 20, 4
2, 45 ... Controller, 14 ... Recording medium, 1
6, 46 ... Demultiplexer, 17 ... Video decoder, 18 ... Audio decoder, 21, 22 ... Buffer, S4, S5 ... Control signal, S10 ... Mark detection signal, D1, D2 ... Compressed AV multiplex Data, M ₁ ……
Video decoding start mark, M ₂ ... Audio decoding start mark, M ₃ ... Stop mark designation mark, M ₄ ...
… Audio decoder stop mark, M _{S1 to} M _S23 …… Audio decode stop mark.

[Equation 27]

[Equation 28]

─────────────────────────────────────────────────── ───

[Procedure amendment]

[Submission date] April 19, 1995

[Procedure Amendment 1]

[Document name to be amended] Statement

[Correction target item name] 0001

[Correction method] Change

[Correction content]

[0001]

[Table of Contents] The present invention will be described in the following order. Industrial Application Conventional Technology (FIG. 17) Problems to be Solved by the Invention (FIGS. 17 and 18) Means for Solving the Problems (FIGS. 1 to 16) Actions (FIGS. 1 to 16) Embodiments (FIGS. 1-16) Effect of the invention

[Procedure Amendment 2]

[Document name to be amended] Statement

[Correction target item name] 0018

[Correction method] Change

[Correction content]

(1-1-3) Data amount of compressed video data file Hereinafter, a data group obtained by encoding one video material will be referred to as a "compressed video data file". Usually, the data amount of the compressed video data file generated from different video materials having the same time length is not the same, and as is clear from FIGS. 3A and 3B,
Assuming that the data amount of the compressed video data file is V _v , the video compression bit rate is R _v , the capacity of the virtual buffer is V _b , and the time length of the video material is T ₄ ,

[Equation 1] As described above, the capacitance fluctuates within the range of ± the virtual buffer capacity V _b .

[Procedure 3]

[Document name to be amended] Statement

[Name of item to be corrected] 0024

[Correction method] Change

[Correction content]

(1-2-2) Consideration of Data Insufficiency For simplicity, start each compressed video data file.
Assuming that the up delays are the same (the start up delay can be specified), FIGS. 4 (A) and 4 (B)
Will be described with reference to. In general, the size V _v of the compressed video data file does not match the product of the compression bit rate R _v and its playback time T (V _v , ≠ R _v , T), so the data is supplied to the decoder at a transfer rate equal to the compression bit rate. In this case, the time required for input (V _v , / R _v ) does not match the time T required for output (reproduction).

[Procedure amendment 4]

[Document name to be amended] Statement

[Correction target item name] 0026

[Correction method] Change

[Correction content]

Maximum amount of compressed video data V that can occur
_vmax is the following formula

[Equation 2] Is. Therefore, when the actually obtained compressed video data amount is less than this value, the "dust" data D _d is added to the end of the file to set the size V _v of any file to V _vmax . To do so. MPEG
According to the standard, “0” can be added as “dust” data (hereinafter, “dust” data D _d added for file size adjustment in this way is referred to as “pad”). A transfer speed (R _v ′ = R _v), which is sufficient to transfer all the data of the file thus adjusted in size in the time T.
If input to the decoder at + V _b / T)
As shown in (B), since both the input required time and the output required time are T, the buffer failure as described above does not occur.

[Procedure Amendment 5]

[Document name to be amended] Statement

[Correction target item name] 0042

[Correction method] Change

[Correction content]

Further, the transfer rate of the compressed AV multiplexed data D2 is set to 5 [Mbps], and the compressed reproduction multiplexed data D2 is transferred to the buffer 21 of the video decoder 17 to transfer the compressed video data D4 corresponding to the capacity of the virtual buffer. [Mbi
t] As shown in FIG. 8, it is necessary to supply the following equation from the beginning.

(Equation 3) Video decoding start mark M ₁ “000001” at the position
BD ”, and the next formula

[Equation 4] Later, audio decoding start mark M ₂ "000001"
BF ". Next, an AV head alignment method for realizing continuous reproduction will be described.

[Procedure correction 6]

[Document name to be amended] Statement

[Correction target item name] 0044

[Correction method] Change

[Correction content]

Therefore, in this embodiment, as in the case described above, the encoder section 11 also adds a predetermined mark to a predetermined position of each data file. For example, in the case of a clip of T = 6 [s] as shown in FIG. 5, since the reproduction time difference is 0.006 seconds, the audio decoding stop mark M that causes the audio decoder 18 to stop decoding at a position before the audio decoding start mark M _{1. Insert s1 to Ms23} (Fig. 8). In this case, when the minimum unit of compressed audio data is 24 [ms], there are 24 reproduction time differences from 0 to 23 [ms], but when the difference is 0 [ms], the audio decoder 18 is stopped. Since it is not necessary, ₂₃ audio decoding stop marks M _{s1 to} M _s23 may be prepared.

[Procedure Amendment 7]

[Document name to be amended] Statement

[Name of item to be corrected] 0045

[Correction method] Change

[Correction content]

[0045] Therefore in the encoder unit 11, to correspond to the reproduction time difference audio as a decode stop mark _M s1 _{~M s23,} for example "000001C1" - "00001D7" respectively 1-23 [ms] 23 pieces of code to ing. Therefore, these codes are 5 [M
It appears every bps] × 0.001 [s] = 625 [byte]. Also, this encoder 5 is
Three mark indicating which of whether the Supporting connexion decode it by stopping of the code (hereinafter, this is referred to as stop mark specified mark M _3, for example, "000001D8") was added at the end of each clip, this The audio decoding stop mark M _4, which becomes effective later, is continuously added. For example, in the case of a clip of T = 6 [s], since the reproduction time difference is 0.006 [s], it is necessary to validate the audio decoding stop mark M _s6, and therefore the last 8 [byte] of this clip. Is "000001
D8000001C6 ".

[Procedure Amendment 8]

[Document name to be amended] Statement

[Correction target item name] 0046

[Correction method] Change

[Correction content]

Clip 1 (T = 6 [s]) and clip 2
In the case of continuously reproducing (T = 7 [s]), it is known from the stop mark designation mark M _{3 of} the clip 1 that the audio decoding stop mark M _s6 is valid. Therefore, the audio decoding stop mark M of the clip 2 is known. _{When s6} is detected by the demultiplexer 16, a stop command is issued to the audio decoder 18, and when the audio decoding start mark M2 of the clip ₂ is detected, a start command is issued to the audio decoder 18. There is. In addition, in order to clear the effective stop marks M _{s1 to} M _s23 before starting the continuous reproduction of the clip,
It has been made so as to prevent the stop command is issued at the beginning clip of the stop mark _M s1 _{~M s23} by previously to "00000000".

[Procedure Amendment 9]

[Document name to be amended] Statement

[Correction target item name] 0050

[Correction method] Change

[Correction content]

(3-2-3) Limitation on compression bit rate value, etc. The maximum compressed video that can occur when a clip with a time length T is compressed with a virtual buffer amount V _b and a video compression bit rate R _v.・ The data volume is

(Equation 5) Given in. Therefore, in the encoding / decoding device 10 of this embodiment, if the actually obtained compressed video data amount is less than this value, the pad data is added to the end of the compressed video data amount so that the size of an arbitrary file is reduced. To V _vmax
I am trying to be. When all the data thus adjusted in size are transferred at time T, the video data transfer rate R _v ′ is expressed by the following equation.

(Equation 6) The maximum R _v ′ is the maximum R _{v max} ′ when the clip time length is the minimum T _min .

(Equation 7) Compressed AV multiplexed data transfer rate R _m must be greater than the sum of the least compressed video data rate compressed audio data rate. For audio, a transfer rate equal to its compression bit rate is sufficient. Therefore, these parameters must satisfy the following equation. (To be precise, it is necessary to consider the head and mark, but here it is omitted for simplicity.)

(Equation 8)

[Equation 9]

[Procedure Amendment 10]

[Document name to be amended] Statement

[Correction target item name] 0053

[Correction method] Change

[Correction content]

When T is set in units of 1 second, for the sake of simplicity, the size L of the standard packet in a clip with an arbitrary time length is used.
_In order to make _v and L _a equal to each other, R _v , R _a , L _v , and L _a may be selected so as to satisfy the following equation.

[Equation 10]

[Equation 11]

(Equation 12) Usually, R _v is at most 0.1 [Mbps] unit,
_Ra is 64, 128, 192, 256, 384 [kbp
s]. Therefore, there are integers L _v and L _a that satisfy the above equation. Fractional data generated by the limitation of the minimum unit of compressed data, that is, R _v , × (T _v , −T) in video and R _a × (T _a −T) data in audio constitutes a residual pack. And the rest of the data, that is R in the video
Let _v , T + V _b and R _a × T in audio form the standard and initial packs.

[Procedure Amendment 11]

[Document name to be amended] Statement

[Correction target item name] 0054

[Correction method] Change

[Correction content]

In this way, the following equations hold for R _a and R _v .

(Equation 13)

[Equation 14]

(Equation 15)

[Equation 16] (14) and (16) from _{R v, R a,} be given a _{L a,} it can be calculated _{L v.} The amount of audio data V _a is

[Equation 17] The amount of video data V _v is

(Equation 18) The amount of multiplexed data V _m is

[Formula 19] Therefore, the pad data amount V _p is

(Equation 20) Yotsute

[Equation 21]

[Procedure Amendment 12]

[Document name to be amended] Statement

[Correction target item name] 0055

[Correction method] Change

[Correction content]

From equation (21), L _p and L _p ′ are obtained. When T _a <T, there are integers K and L _a ″ that satisfy the following equation.

[Equation 22] For the last from (K + 1) -th the packed at this time standard the packed, the data size of the audio-packet L _a -
"And that, size immediately after this packet is _{L a" L} a -L
Make sure to put the _h -packet / packet. Then, for the K packs from the end of the standard pack, put the same size of pad pack in place of the audio pack. In addition, the following equation holds for the residual pack.

(Equation 23)

[Equation 24]

[Procedure Amendment 13]

[Document name to be amended] Statement

[Name of item to be corrected] 0058

[Correction method] Change

[Correction content]

That is, for any t> 0, v ₃ (t)>
It suffices that v ₁ (t) holds.

(Equation 25)

(Equation 26) Condition 2: The straight line and the straight line do not intersect until the time t _e at which the video data transfer ends. That is, v ₄ (t
_{e) <v 2 (t e} ) it is sufficient to hold. In this case,

[Equation 27] So

[Equation 28]

[Equation 29]

[Procedure Amendment 14]

[Document name to be amended] Statement

[Correction target item name] 0059

[Correction method] Change

[Correction content]

The equations for each straight line are as follows. Strictly speaking, it is necessary to consider the amount of headers and marks, but since it is very small compared to the amount of compressed data, it is ignored.

[Equation 30]

[Equation 31]

[Equation 32]

[Expression 33] Therefore, in order to realize continuous reproduction at 3 times the virtual buffer, T _t = T _v , and equations (26) and (29) are used.
It suffices to select each parameter so as to satisfy the equation, and in actuality, each parameter is also selected in the encoding / decoding device 10 of this embodiment.

[Procedure Amendment 15]

[Document name to be amended] Statement

[Correction target item name] 0060

[Correction method] Change

[Correction content]

(3-2-6) Example of compressed AV multiplexing format capable of continuous reproduction R _m = 5 [Mbps], R _v = 4 [Mbps], _Ra =
256 [kbps], T _min = 6 [s], V _b = 4
A format for realizing the requirement of [Mbps] is shown. However, the video shall be in NTSC format. In this case, the right side of the equation (9) is 5 [Mbps] and the left side is the following equation.

(Equation 34) Therefore, the expression (9) is satisfied, and it is understood that these values can be selected. For example, clip time length T
= 6 [s], from FIG. 5, T _t , = T _v , = 6.006 [s], T _a = 6 [s] Appropriately L _a = 1000 [byte] (if too large, audio. On the contrary, if it is too small, the number of standard packs will increase and the number of over heads due to headers will increase).

[Equation 35] The size of the standard video packet is obtained from the equation (16).

[Equation 36] The size of the pad / packet is obtained from the equation (21).

(37) Therefore, L _p = 293 [byte], L _p ′ = 74 [b]
yte].

[Procedure Amendment 16]

[Document name to be amended] Statement

[Correction target item name] 0061

[Correction method] Change

[Correction content]

The size of the residual video packet is obtained from the equation (23).

(38) Since T _a = T, the size of the residual audio packet is 0. Calculate the size of the standard pack from the above values.

[Formula 39] At this time, it can be seen that the expression (26) is satisfied. 0.92> 0.8 At this time, it is understood that the equation (29) is also satisfied. 23.2 <24 The format obtained as a result of the above is shown in FIG. Similarly, the calculation was performed for a clip time length of 7 seconds, and the result is shown in FIG. In this case, since T _a ≠ T, the last standard pack is different from other standard packs, and there are pad packets corresponding to T _a −T. Once the value of each parameter is determined, regardless of the clip contents, for any clip, for each clip time length,
Since the format table as shown in FIGS. 12 and 13 can be created, when the compressed AV data is actually multiplexed, the compressed data obtained by each encoder may be multiplexed according to this table.

[Procedure Amendment 17]

[Document name to be amended] Statement

[Correction target item name] 0071

[Correction method] Change

[Correction content]

[0071]

As described above, according to the present invention, on the encoding side, compressed video data obtained by compression-encoding the video signal and audio signal of a material consisting of a series of video signals and audio signals. And the compressed audio data that depends on only the time length of the material and gives the timing of starting and stopping the operation in accordance with a predetermined multiplex format that is larger than the packet with the size of the leading packet by the first buffer amount. While putting the code everywhere, when the amount of compressed video data is less than the maximum value that can be generated as the compressed video data of the material for that time length, dummy data is added to the compressed video data by the shortage. While it is multiplexed with compressed audio data, the encoding side is based on this special code. By which is adapted to stop the encoding process only reproduction time difference between the compressed video data and compressed audio data to compression audio data,
It is possible to realize an encoding device, a decoding device, and an encoding / decoding device that can start reproduction of each material at a specified time and can continuously reproduce a plurality of materials. When decoding the multiplexed compressed video data and compressed audio data forming the first material supplied from the encoding unit via the data supply unit, the compressed video data and the compressed video data according to the time length of the material are decoded. By stopping the decoding process for the compressed audio data by the reproduction time difference between the audio data, it is possible to realize a decoding device and a coding / decoding device capable of continuously reproducing a plurality of materials.

Claims (14)

[Claims] 1. A decoding device for decoding multiplexed compressed video data and compressed audio data forming a first material, which are supplied from a data supply source, respectively, wherein: Dividing means for dividing and outputting compressed audio data, first decoding means for decoding the compressed video data output from the dividing means, and second decoding means for decoding the compressed audio data output from the dividing means. Decoding means for decoding the compressed audio data by the second decoding means on the basis of the time length information of the first material supplied from the data supplying means, the compressed video data and the compressed audio data. And a control unit for stopping the reproduction time difference between data. 2. The control means stops the second decoding operation by the second decoding means for the reproduction time difference based on the time length information and the reproduction time difference stored in advance. Decoding device according to claim 1, characterized in that 3. Multiplexed compressed video data and compressed audio data, each of which is composed of an encoding unit and a decoding unit and forms the first material supplied from the encoding unit via a data supply unit. In the encoding / decoding device for decoding, the decoding unit divides the supplied multiplexed compressed video data and compressed audio data and outputs the divided data, and the compression unit outputs the divided data. First decoding means for decoding video data, second decoding means for decoding the compressed video data output from the dividing means, and time length information of the first material supplied from the data supply means The decoding operation for the compressed audio data by the second decoding means is performed on the basis of the reproduction time difference between the compressed video data and the compressed audio data. Coding and decoding apparatus characterized by comprising a control means for only stopping. 4. The second decoding operation by the second decoding means is stopped by the reproduction time difference based on the time length information and the reproduction time difference stored in advance. Item 3. The encoding / decoding device according to Item 3. 5. A decoding method for decoding respectively multiplexed compressed video data and compressed audio data which form a first material and which are supplied from a data source, wherein the compressed video data and the multiplexed data are multiplexed. A first step of dividing the compressed audio data, a second step of decoding the divided compressed video data and the divided compressed audio data, the compressed video data according to the time length of the first material, and And a third step of stopping the decoding process of the compressed audio data by the reproduction time difference between the compressed audio data. 6. A decoding device for respectively decoding the multiplexed compressed video data and compressed audio data forming the first material, which are supplied from the encoding unit via the data supply unit, are multiplexed. According to a first step for dividing the compressed video data and the compressed audio data, a second step for decoding the divided compressed video data and the compressed audio data, and a time length of the first material. And a third step of stopping the decoding process of the compressed audio data by the difference in reproduction time between the compressed video data and the compressed audio data. 7. Encoding means for compressing and encoding the video signal and the audio signal of a material consisting of a series of video signals and audio signals, and outputting the compressed video data and compressed audio data thus obtained, and the material. Of the compressed video data and the compressed audio data to the decoding side according to a predetermined multiplexing format which depends only on the time length of the packet and the size of the leading packet is larger than the packet by the first buffer amount. The special code that gives the timing of starting and stopping the operation is put in various places and multiplexed, and at this time, the data amount of the compressed video data satisfies the maximum value that can be generated as the compressed video data of the material for the time length. If there is not, add dummy data to the above compressed video data by the shortfall. An encoding apparatus further comprising a multiplexing means for multiplexing the compressed audio data. 8. A first signal corresponding to a first buffer amount for the video signal on the decoding side which is assumed in advance.
9. The encoding device according to claim 8, wherein the audio signal is encoded and compressed at the compression bit rate of 1 above, and the audio signal is compression encoded at the first compression bit rate. 9. First compressed video data and compressed audio data are separated from compressed multiplexed data supplied from a data supply source and containing special codes for giving start and stop timings everywhere, respectively. In a decoding device for decoding, the compressed video data and the compressed audio data are divided from the compressed multiplexed data and output, and code detection division means for outputting a detection signal when the special code is detected, First decoding means for decoding the compressed video data output from the code detecting / dividing means, second decoding means for decoding the compressed audio data output from the code detecting / dividing means, and the code detecting / dividing means To the compressed audio data of the second decoding means based on the detection signal supplied from A decoding device, comprising: a control means for stopping the decoding operation for the compressed video data and the compressed audio data by a reproduction time difference between the compressed video data and the compressed audio data. 10. A coding unit for compressing and coding the video signal and the audio signal of a material consisting of a series of video signals and the audio signal, respectively, and a compression coding output from the coding unit. An encoding / decoding device comprising a decoding unit for separating and decoding the multiplexed video signal and audio signal, wherein the encoding unit compression-codes the video signal and the audio signal, and thus obtains According to a coding means for outputting compressed video data and compressed audio data, and a predetermined multiplexing format which is dependent only on the time length of the material and which is larger by the first buffer amount than a packet in which the size of the first packet is continuous. Then, the compressed video data and the compressed audio data are started and stopped for the decoding side. When the special code for giving the aiming is put in everywhere and is multiplexed, when the data amount of the compressed video data is less than the maximum value that can be generated as the compressed video data of the material for the time length, the dummy amount is added. Multiplexing means for forming compressed multiplexed data by adding data to the compressed video data and multiplexing with the compressed audio data, and the decoding section is supplied from the encoding section. The compressed video data and the compressed audio data are divided and output from the compressed and multiplexed data, and a code detection division means for outputting a detection signal when the special code is detected, and output from the code detection division means The first decoding means for decoding the compressed video data, and the code detecting and dividing means. A second decoding means for decoding the compressed audio data inputted, and a decoding operation for the compressed audio data by the second decoding means on the basis of the detection signal supplied from the code detecting / dividing means. An encoding / decoding apparatus comprising: a control unit for stopping the reproduction time difference between the video data and the compressed audio data. 11. A first step for compressing and encoding the video signal and the audio signal of a material consisting of a series of video signals and audio signals, and the compressed video data and compressed audio data thus obtained, Of the compressed video data and the compressed audio data according to a predetermined multiplexing format, which depends only on the time length of the packet and the size of the leading packet is larger by the first buffer amount than the succeeding packet.
Special code that gives operation start and operation stop timing to the decoding side is added and multiplexed everywhere,
At this time, when the data amount of the compressed video data is less than the maximum value that can be generated as the compressed video data of the material with respect to the time length, dummy data is added to the compressed video data by a shortage amount to obtain the compressed audio data. And a second step of multiplexing. 12. In the first step, the video signal is encoded and compressed at a first compression bit rate corresponding to a first buffer amount for the video signal on the decoding side which is assumed in advance, and the audio signal is obtained. The encoding method according to claim 11, wherein the encoding is performed at the first compression bit rate. 13. First compressed video data and compressed audio data are separated from compressed multiplexed data supplied from a data supply source and containing special codes for giving start and stop timings everywhere, respectively. In the decoding method for decoding, the compressed video data and the compressed audio data are divided from the compressed multiplexed data and output, and a first step for searching for the special code, the compressed video data and the compressed audio data are output. A second step of decoding the data, and when the special code is detected, a decoding process for the compressed audio data is performed based on the detected special code, and a reproduction time between the compressed video data and the compressed audio data is set. And a third step for stopping only the difference. Decoding method that. 14. A video signal and an audio signal of a material consisting of a series of video signals and the audio signal are respectively compression-encoded and multiplexed, and then the compression-encoded and multiplexed video signal and the audio signal are separated. In the encoding / decoding method for decoding, the first step of compressing and encoding the video signal and the audio signal, and the compressed video data and the compressed audio data thus obtained depend only on the time length of the material. And, according to a predetermined multiplexing format which is larger by the first buffer amount than the packet in which the size of the leading packet continues, while performing the multiplexing while inserting the special code for giving the operation start and operation stop timing everywhere, At this time, the data amount of the compressed video data depends on the pressure of the material for the time length. When it is less than the maximum value that can be generated as video data, a second step of adding dummy data to the compressed video data and multiplexing the compressed data with the compressed audio data, and the multiplexed compressed video data and the compressed video data When the compressed audio data is divided and outputted, a third step for searching for the special code, a fourth step for decoding the compressed video data and the compressed audio data, and when the special code is detected, A fifth step of stopping the decoding process for the compressed audio data based on the detected special code by a reproduction time difference between the compressed video data and the compressed audio data. Decryption method.