JP2003298544A - Program stream multiplexing method and apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To multiplex a plurality of elementary streams ES independently of the capacity of a main buffer (MB) of the Program System Target Decoder (P-STD). <P>SOLUTION: The program stream multiplexing apparatus is provided with a pack selection circuit 71 for managing the state of the memory occupied amount of a VMB 204 for storing a video ESVS of the P-STD and AMB 207, 209 for storing audio ESAS1, AS2, discriminating multiplex precedence and multiplexing discrimination to multiplex the video ESVS and audio ESA1, AS2 in the order of the VMB 204 and the AMB 207, 209 which get free so as to control the multiplexing. The apparatus manages the state of the MB and multiplexes the ES in the order the MB getting free when multiplexing a plurality of ES to attain multiplexing without causing underflow to the MB even when the ES with the same MB capacity is multiplexed. <P>COPYRIGHT: (C)2004,JPO

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a program stream multiplexing method and apparatus, and more particularly, to a program stream multiplexing method and apparatus for multiplexing a plurality of program streams which are compression-encoded video and audio data.

[0002]

2. Description of the Related Art In recent years, when transmitting high-quality image (video) audio (audio) signals and various data signals in a limited transmission band, for example, MPEG2 (Moving P).
image Expert Group Phase
2) Encode and multiplex video and audio signals using the information compression encoding technology and multiplexing technology according to 1)
A video / audio signal transmission system for transmitting as a bit stream of a book is becoming widespread.

Here, MPEG is an expert group ISO / IEC of ISO and CCITT, JTC1 / SC2 / W.
G8 (Moving Picture Expert)
Group) recommended in 1993 ISO / IEC-11
172-2, first, the MPEG1 standard (currently the latest version is ISO / IEC-111) for recording on a recording medium.
72-3) was established, and thereafter ISO / IEC-1381
MPEG-2 etc. are established in 8-2. Hereinafter, for convenience of description, these standards are collectively referred to as the MPEG standard.

In such a transmitting device in the video / audio signal transmission system, a video audio signal, various data signals and the like (hereinafter referred to as a video audio signal for the sake of convenience of description) are coded and coded by a compression coding technique according to the MPEG standard. The encoded data is generated and the encoded data is multiplexed by a multiplexing technique to form a bit stream. Here, this bit stream is called a program stream (hereinafter, PS). In addition, each piece of encoded data corresponding to each of the video and audio signals is called an elementary stream (ES). That is,
A video ES corresponding to a video signal and an audio ES corresponding to an audio signal are multiplexed to form one PS.

Conventionally, it was general to generate one PS by multiplexing one video ES and one audio ES, but in the future, two or more video ESs and two or more audio ESs will be multiplexed. However, there is already a market demand for multiplexing one video ES and two audio ESs. The MPEG2 standard also defines that 32 audio ESs and 16 video ESs can be multiplexed.

In the conventional program stream multiplexing method, the capacity is small because the difference between the main buffer capacity of the audio ES program system target decoder (hereinafter, P-STD) and the P-STD main buffer capacity of the video ES is large. The audio ES was prioritized for multiplexing.

One video ES and one audio ES
Referring to FIG. 18 which shows a block diagram of a first conventional program stream multiplexing device for multiplexing, the first conventional program stream multiplexing device compresses and encodes an input digital video signal VD in accordance with the MPEG standard. A video encoder 1 that outputs a video ESVS, a video buffer 4 that temporarily holds a video ESVS, an audio encoder 2 that inputs a digital audio signal AD1 and performs compression encoding according to a compression encoding protocol of the MPEG standard and outputs an audio ESAS1. , Audio buffer 5 for temporarily holding audio ESAS1
And a multiplexer 107 that multiplexes the video ESVS and the audio ESAS1 held in the video buffer (VB) 4 and the audio buffer (AB) 5 and outputs them as a program stream PS.

The multiplexer 107 is a video buffer (VB).
VB occupancy confirmation circuit 73 for confirming and retaining the occupancy of the video ES in 4 and V for simulating a video main buffer (hereinafter, VMB) 204 of a program system target decoder (hereinafter, P-STD) described later.
MB simulation circuit 74 and A for confirming and holding the occupied amount of audio ESAS1 in audio buffer 5
A B occupancy confirmation circuit 75, an AMB simulation circuit 76 for simulating an audio main buffer (hereinafter referred to as AMB) 207 of P-STD described later, and a system clock reference for MB simulation based on data from a pack selection circuit 171 described later. SCR generation circuit 79 for generating (hereinafter, SCR) and a video ESVS that is a pack (basic unit of PS) to be multiplexed.
And a pack selection circuit 1 for controlling the multiplexing of the audio ESAS 1 by performing the multiplexing ranking and the multiplexing judgment described later.
71 and a multiplexing circuit 17 for multiplexing the selected pack
2 and.

P output from the conventional first PS multiplexer
Referring to FIG. 19 which shows a block diagram of a configuration of a P-STD, which is a virtual decoder for S, the first conventional P-STD shown in this figure converts the input PS into a video ESVS,
A demultiplexing unit 301 that decomposes audio ESAS1, SCR, PTS, and DTS, a system clock generating unit 202 that receives the SCR and generates a system time clock (STC), and a VM that holds a video ES.
B204 and AMB2 holding audio ESAS1
07, STC and PTS or DTS are compared, and in accordance with the comparison result, the VMB 204 or AMB 207/209 is instructed to output the ES corresponding to the PTS or DTS to the video decoder 205 or the audio decoder 208/210, and will be described later. Selectors 206 and 212
A time stamp comparison unit 203 which outputs a control signal of
A video decoder 205 for decoding the input video ES and outputting a video signal V, and an input audio ESAS
An audio decoder 208 that decodes 1 and outputs an audio signal A1, and selectors 206 and 2 that switch to supply the video signal V to the rearrangement buffer 211 in response to the supply of the control signal from the time stamp comparison unit 203.
12 and a rearrangement buffer 211 that rearranges to change the arrangement order of the pictures of the video signal.

Next, the operation of the first conventional program stream multiplexing apparatus will be described with reference to FIGS. 18 and 19 and FIGS. Conventionally, audio ES
First, the audio ES will be multiplexed.
The determination of multiplexing and multiplexing will be described.

Referring to FIG. 20, in the multiplexing determination step L1, the pack selection circuit 171 determines whether or not an ES having a data amount for multiplexing (hereinafter, ES to be multiplexed) exists in the audio buffer 5. A capacity confirmation request is output to the AB occupation amount confirmation circuit 75 to confirm the capacity. That is, it is confirmed whether the audio buffer occupancy ≧ the audio ES multiplexing amount.

At step L1, if the ES to be multiplexed does not exist in the audio buffer 5, the determination of the multiplexing of the audio ES is ended, and the determination of the multiplexing of the video ES after the branch M is made.

When the ES to be multiplexed is present in the audio buffer 5 in step L1, the process proceeds to steps L2 to L6, and it is determined whether the AMB 207 receives the audio ES and neither overflow nor underflow occurs.

First, in the MB free space confirmation step L2,
The pack selection circuit 171 outputs a free space confirmation request to the AMB simulation circuit 76, inputs a free space as a response to this free space confirmation request, and sets this value as the free space 1. In the free space 2 calculation step L3, the free space 2 is calculated by subtracting the multiplexing ES amount of the multiplexing order 1ES from the free space 1.

In the deletion AU amount calling step L4, the pack selection circuit 171 outputs the SCR added when multiplexing is performed to the AMB simulation circuit 76. The AMB simulation circuit 76 uses the input SCR for AMB2
Call the AU amount deleted from 07, and pack selection circuit 1
To 71.

At step L5 of calculating the free space 3, the AU amount is added to the free space 2. This value is used as the free space 3. Free space 3 is AMB2 when audio ES is multiplexed
The free space of 07 is shown.

In the MB capacity judgment step L6, the pack selection circuit 171 judges whether the free capacity 3 is within the capacity of the AMB 207.

At step L6, the free space 3 becomes AMB20.
If it is out of the capacity of 7, the audio ES multiplexing determination is ended, and the process proceeds to the branch M to make a video ES multiplexing determination.

At step L6, the free space 3 becomes AMB20.
If it is within the capacity of 7, the audio ES is multiplexed in steps L7 to L11, assuming that the AMB 207 does not overflow or underflow even if the audio ES is multiplexed.

First, in the output ES amount instruction step L7, the pack selection circuit 171 instructs the audio buffer 5 to output the ES amount to the multiplexing circuit 172. In response to the instruction, the audio buffer 5 receives the designated amount of E
The S is output to the multiplexing circuit 172.

SCR and ES parameter output step L8
Then, the pack selection circuit 171 outputs the SCR added to PS to the multiplexing circuit 172. In addition, the ES for multiplexing
When the head of U is included, the PTS and AU amount are also output to the multiplexing circuit 172.

Simulation SCR generation step L
At 9, the pack selection circuit 171 adds the SCR to the PS.
Is output to the SCR generation circuit 79.

At step L10, the pack selection circuit 171
Outputs the ES amount to be multiplexed to the AMB simulation circuit 76. If the ES to be multiplexed includes the head of AU, the PTS and AU amount are also output to the AMB simulation circuit 76.

In the addition step L11 of the SCR added value, the pack selection circuit 171 adds and holds the SCR added value to the SCR.

The multiplexing circuit 172 receives the input ES, S
Multiplexing is performed from CR, PTS, and AU amounts, and PS is output.

The AMB simulation circuit 76 uses the AM
The input ES amount is subtracted from the free capacity of B207. AMB
The simulation circuit 76 holds the input AU amount and the PTS in association with each other. VMB simulation circuit 74
Is the SCR generation circuit 79 and the VMB simulation circuit 7
When the DTS held in 4 is compared and the DTS is smaller than the simulation SCRSC, the VMB20
Add the AU amount related to DTS to the free space of 4
When S is larger, delete the DTS and the AU amount associated with the DTS.

The AMB simulation circuit 76 is an SC
R and P held in the AMB simulation circuit 76
When TS is compared and PTS is smaller, AMB207
Add the AU amount related to PTS to the free space of
, The PTS and the AU amount associated with that PTS are deleted.

Thereafter, the operation of changing and deleting the values held in the VMB simulation circuit 74 and the AMB simulation circuit 76, respectively, in association with the change of the SCR set value of the SCR generation circuit 79 described above is performed for each MB simulation. The circuit is described as changing the hold value according to the changed SCR value.

Next, with reference to FIG. 21, with respect to the multiplexing judgment and multiplexing of the video ES, which is the processing of the branch M, the difference from the above-described audio ES multiplexing judgment and multiplexing will be mainly described. Then, first, the multiplexing determination step M1
Then, the pack selection circuit 171 determines that the video ES to be multiplexed is
, VB to determine whether it exists in the video buffer 4,
A capacity confirmation request is output to the occupancy confirmation circuit 73 to confirm the capacity. That is, video buffer occupancy ≧ video E
It is confirmed whether or not it is the multiplexing amount of S.

If the video ES to be multiplexed does not exist in the video buffer 4 in step M1, the determination of the multiplexing of the video ES is finished, and the process proceeds to the SCR adjustment processing after the branch N.

If the video buffer 4 contains the video ES to be multiplexed in step M1, MB free space confirmation step M2, free space calculation step M3, deleted AU amount recall step M4, free space calculation step M5, MB capacity determination The processing of step M6 is performed to determine whether overflow or underflow occurs even if the VMB 204 inputs the video ES. The processing of these steps M2 to M6 is performed by converting the audio ES into the video ES, the audio buffer 5 into the video buffer 4, and the AMB 207 into the VMB 20.
4, the AMB simulation circuit 76 is read as the VMB simulation circuit 74, and the free capacities 1, 2, 3 are read as free capacities 7, 8, 9 respectively, and the processing of the audio ES multiplexing determination and the multiplexing steps L2 to L6 is performed. Since it is the same as, the detailed description will be omitted.

At step M6, the free capacity 9 becomes VMB20.
If it is out of the capacity of 4, the multiplexing judgment of the video ES is ended, and the process proceeds to the SCR adjustment processing of the branch N.

At step M6, the free capacity 9 becomes VMB20.
If it is within the capacity of 4, even if video ES is multiplexed, VMB
Assuming that neither an overflow nor an underflow occurs in 204, an output ES amount instruction step M7, an SCR and ES parameter output step M8, a simulation SCR generation step M9, a multiplexing target ES amount / parameter output step M10, and an SCR addition value addition step Each process of M11 is performed to multiplex the video ES. These steps M7 to M11 are also the same as the processing of the audio ES multiplexing determination and multiplexing steps L7 to L11 except for the above-mentioned replacement, and therefore detailed description thereof will be omitted.

The multiplexing circuit 172 receives the input ES, S
Multiplexing is performed based on CR, PTS, DTS, AU amount, and picture type, and PS is output.

The VMB simulation circuit 74 uses the VM
The input ES amount is subtracted from the free capacity of B204, and DTS,
The AU amount is associated and held. Each MB simulation circuit changes the held value according to the changed SCR value.

Next, the SCR adjustment processing, which is the processing of the branch N, will be described with reference to FIG. 22. The pack selection circuit 171 adjusts the SCR when the audio ES and the video ES cannot be multiplexed.

First, in the multiplexing determination step N1, the pack selection circuit 171 confirms the capacity of the AB occupancy confirmation circuit 75 to determine whether the multiplexing target ES amount exists in the audio buffer 5, and if it does not exist. , SCR adjustment processing and multiplexing processing ends.

When the amount of ES to be multiplexed is present in the audio buffer 5 in step N1, the VB occupation amount confirmation circuit is used to determine whether or not the amount of ES to be multiplexed is present in the video buffer 4 in the multiplexing determination step N2. The capacity of 73 is confirmed, and if it does not exist, the SCR adjustment processing and the multiplexing processing are terminated.

In step N2, if the multiplexing target ES amount is present in the video buffer 4, the PT
The SCR adjustment is performed by performing each processing of the S minimum value calling step N3, the DTS minimum value calling step N4, the SCR adjustment value output step N5, and the SCR changing step N6.

First, in the PTS minimum value calling step N3,
The pack selection circuit 171 outputs the minimum value confirmation to the AMB simulation circuit 76, and the AMB simulation circuit 76 outputs the held minimum value of PTS to the pack selection circuit 171. This value is set as the temporary SCR3.

At the DTS minimum value calling step N4, the pack selection circuit 171 outputs the minimum value confirmation to the VMB simulation circuit 74, and the VMB simulation circuit 74.
Is the minimum value of the DTS held by the pack selection circuit 171.
Output to. This value is set as the temporary SCR4.

At the SCR adjustment value output step N5, the pack selection circuit 171 compares the temporary SCR3 and the temporary SCR4,
The smaller value is output to the SCR generation circuit 79.

At the SCR change step N6, the pack selection circuit 171 compares the temporary SCR3 and the temporary SCR4 and sets / holds the smaller value as the SCR to be output next.

Next, a second conventional program stream multiplexer for multiplexing one video ES and two audio ESs is the same as in FIG. 18 with common reference characters / numerals attached to the same components. Referring to FIG. 23, which is shown in a block form, the difference between this second conventional program stream multiplexer and the above-mentioned first conventional program stream multiplexer is that the second audio to be multiplexed is increased. Corresponding to ES, the second digital audio signal AD is added to the component of the first program stream multiplexer.
An audio encoder 3 that inputs 2 and performs compression encoding according to the compression encoding protocol of the MPEG standard to output an audio ESAS 2, an audio buffer 6 that temporarily holds the audio ESAS, and a video buffer (VB) instead of the multiplexer 107. 4 and a multiplexer 107 which multiplexes the video ESVS and the audio ESAS1 and ESAS2 held in the audio buffers (AB) 5 and 6 and outputs them as a program stream PS.

The multiplexer 107A includes the audio ES in the audio buffer 6 in addition to the components of the multiplexer 107.
AB occupation amount confirmation circuit 77 for confirming and retaining the occupation amount of AS2
And an AMB simulation circuit 78 for simulating the AMB 209, and a video ESVS and audio ESA to be multiplexed instead of the pack selection circuit 171.
A pack selection circuit 171A that performs S1 and ESAS2 multiplexing ordering and determination, and controls multiplexing, and a multiplexing circuit 72 that multiplexes a selected pack including the audio ESAS2 instead of the multiplexing circuit 172. Is to prepare.

P corresponding to the conventional second PS multiplexer
Referring to FIG. 24, which shows the block diagram of the STD configuration,
The conventional first P of the conventional second P-STD shown in FIG.
-The difference from STD is that the PS input instead of the demultiplexing unit 301 is converted into a video ESVS and an audio ES.
AS1, ESAS2, system clock (SCR), P
Demultiplexing unit 201 for decomposing into TS and DTS
And AMB209 holding audio ESAS2,
And an audio decoder 210 for decoding the audio ESAS and outputting the audio signal A2.

Next, with reference to FIGS. 23 and 24 and FIGS. 25 to 28 which are flowcharts showing the processes of the respective units, the operation of the second conventional program stream multiplexer will be described with reference to the operation of the first program stream multiplexer. The difference from the operation of
P-STD main buffers are processed in ascending order of capacity, and when there are two audio ESs, one of them is given priority for multiplexing. In this example, for convenience of explanation, it is assumed that the audio ESAS1, the audio ESAS2, and the video ESVS are multiplexed in this order.

Referring to FIG. 25, in the multiplexing judgment and multiplexing of the audio ESAS1 shown in this figure, in the multiplexing judgment step L1A, if the ES to be multiplexed does not exist in the audio buffer 5, it is judged whether the audio ESAS1 is multiplexed. Except that the procedure is the same as that of the audio ESAS1 of the conventional first program stream multiplexer described above, except that the processing is terminated and the audio ESAS2 after the branch P is determined. Omit it.

Next, referring to FIG. 26, the multiplexing determination and multiplexing of the audio ESAS2 shown in FIG. End the multiplexing determination, and proceed to the multiplexing determination of the video ESVS after the branch M, step L2 to L11 to step P
2 to P11, free space 1, 2, 3 free space 4, 5,
Audio ES described above, except for each
Since this is the same as the AS1 multiplexing determination and multiplexing, the description thereof is omitted.

Next, referring to FIG. 27, in the multiplexing determination and multiplexing of the video ESVS shown in this figure, the multiplexing target ES is the video buffer 4 in the multiplexing determination step M1A.
If it does not exist, the video ESVS multiplexing determination and the multiplexing of the video ESVS of the above-described first conventional program stream multiplexer are performed, except that the video ESVS multiplexing determination is ended and the SCR adjustment processing after the branch Q is performed. Since it is the same as, the description will be omitted.

Next, referring to FIG. 28, the SCR of the second conventional program stream multiplexer shown in FIG.
The adjustment processing is the multiplexing determination step N of the SCR adjustment processing of the above-described first conventional program stream multiplexer.
1, N2, a multiplexing determination step Q1 of the audio ESAS2 and a PTS minimum value calling step Q2 of the ABM simulation circuit 78 which is a temporary SCR5 are added, and accordingly, instead of the SCR adjustment value output step N5,
Instead of the SCR adjustment value output step Q3 of outputting the minimum value of the comparison result of the temporary SCR3 and the temporary SCR4 to the SCR generation circuit 79 and the SCR change step N6, the pack selection circuit 171A is SCR
4 and provisional SCR5 are output next to the minimum value of the comparison result S
SCR change step Q4 of setting / holding as CR. Therefore, this SCR adjustment processing is the same as the SCR adjustment processing of the conventional first program stream multiplexing apparatus except for the above-mentioned differences, and therefore description thereof will be omitted.

Next, the second conventional method will be described by using a specific numerical example.
9 is an explanatory diagram showing the states of the multiplexer 7 before and after the operation of the program stream multiplexer of FIG.
First, the multiplexing conditions of this example are shown below.

[0053] The capacity of the VMB 204 = 229376 bytes.

AMB207 capacity = 4096 bytes AMB209 capacity = 4096 bytes Video ES multiplexing amount = 2015 bytes Audio ESAS1 multiplexing amount = 2015 bytes Audio ESAS2 multiplexing amount = 2015 bytes Referring to FIG. In step L1A, it is determined whether the occupied amount of the audio buffer 5 is larger than the multiplexed ES amount of the audio ESAS1, and the multiplexed ES amount is 2350.
Since it is a byte and the multiplexed ES amount is 2015 bytes compared to the above condition, the occupied amount is larger.

Next, in steps L2 to L6, it is determined whether the AMB 207 causes neither overflow nor underflow even when the audio ESAS1 is multiplexed.

First, in step L2, the free capacity (free capacity 1) of the AMB 207, which is 2040 bytes, is called. Ask for. In step L4, hold SCR = 6
0559 is notified to the AMB simulation circuit 76. The AMB simulation circuit 76 determines whether the AU amount and PTS to be deleted by the notification SCR are SCR.
And PTS. Since there is no PTS smaller than the notification SCR = 60559, the pack selection circuit 171A is notified of AU amount = 0 bytes.

At step L5, the notified AU amount is added to the free space 2 (25 bytes), and the audio ESAS1 is added.
25 bytes of the free space 3 indicating the free space of the AMB 207 at the time of multiplexing are obtained.

At step L6, the free space 3 becomes AMB20.
In the capacity of 7, it is determined whether it is 0 bytes or more and 4096 bytes or less. Since 25 bytes are in the capacity of the AMB 207, the pack selection circuit 171A determines that the audio ESAS1.
Are multiplexed in steps L7 to L11.

In step L7, the ES amount output to the audio buffer 5 = 2015 bytes is notified, and the audio buffer 5 outputs the notified ES amount 2015 bytes to the multiplexing circuit 72. At step L8, SCR = 60559 added at the time of multiplexing and audio ESAS1 includes the head of AU, so PTS = 6 added at the time of multiplexing.
8040 and AU amount = 2016 bytes are also output to the multiplexing circuit 72. In step L9, SCR = 60559 is output to the SCR generation circuit 79 for simulation. In step L10, the multiplexed ES amount of the audio ESAS1 =
The 2015 bytes are output to the AMB simulation circuit 76. Since the audio ESAS1 includes the head of the AU, the additional PTS = 68040 and the AU amount = 2016 bytes are also output to the AMB simulation circuit 76. In step L11, the SCR addition value = 159 is added to the held SCR. The multiplexing circuit 72 includes an audio ESAS1,
Input PTS, AU amount, SCR, multiplex, PS
Is output.

The AMB simulation circuit 76 inputs the multiplexed ES amount = 2015 of the audio ESAS1,
The multiplexed ES amount is subtracted from the free capacity of the AMB 207 to obtain a new free capacity of 25 bytes. AU amount = 2016 bytes and PTS = 68040 are input and held, and SCR = 60559 is compared with the held PTS. S
Since there is no PTS smaller than CR = 60559,
The AU amount and PTS are not deleted.

The AMB simulation circuit 78 uses the SC
R = 60559 is compared with the retained PTS, and the retained PTS =
Since SCR = 60559 is larger than 60480,
The AU amount and PTS are deleted, and the deleted AU amount is set to AMB20.
Add to the free space of 9. The addition result is 5152 bytes, which exceeds the capacity of AMB209 = 4096 bytes, so that an underflow occurs in AMB209.

As described above, in the conventional second program stream multiplexing circuit, when one video ES and two audio ES are multiplexed, an underflow occurs in the main buffer of P-STD. There is.

[0063]

In the conventional method and apparatus for program stream multiplexing described above, the multiplexing of a plurality of ESs involves the size of the ES to be multiplexed, the multiplexing condition, and the program stream target decoder (P-STD). The size of the main buffer depends on the capacity of the main buffer of the
If there is a small margin in comparison with the size and the multiplexing condition, for example, if one video ES and two audio ES are multiplexed, underflow may occur in the P-STD main buffer. There was a flaw.

It is an object of the present invention to provide a program stream multiplexing method and apparatus capable of multiplexing a plurality of ESs without depending on the capacity of the main buffer of P-STD.

[0065]

According to a first aspect of the present invention, there is provided a program stream multiplexing method, wherein a digital video signal and a first digital audio signal and a second digital audio signal are input to a compression code according to a predetermined compression coding protocol. Video elementary stream (hereinafter referred to as video E
S) and first and second audio elementary streams (hereinafter, audio ES) are generated, and the video ES and the first and second audio ES are multiplexed to generate a program stream. In the method, a program system target decoder (hereinafter, P-), which is a virtual decoder used in encoding to create the program stream.
Memory occupancy of each of a video main buffer (hereinafter, VMB) holding the video ES of STD) and first and second audio main buffers (hereinafter, AMB) holding the first and second audio ESs Of the video ES and the first and second audio ESs in the order in which the VMB and the first and second AMBs become empty.
It is characterized in that the multiplexing is ranked and the multiplexing is judged so as to be multiplexed, and the multiplexing is controlled.

The invention according to claim 2 is the program stream multiplexing method according to claim 1, wherein the V
A VMB simulation circuit that holds the states of the memory occupancy of the MB and the first and second AMBs, and the first and second
The AMB simulation circuit of the above, and from the contents of holding amount of each of the VMB simulation circuit and the first and second AMB simulation circuits, the PS
The empty order, which is the order in which the main buffer in the TD becomes empty, is grasped, and the empty order is used as the multiplexing order of the video ES and the first and second audio ESs, and the multiplexing order is performed. A first multiplexing determination is made as to whether or not multiplexing is possible, multiplexing is performed if multiplexing is possible, and if all of the video ES and the first and second audio ESs cannot be multiplexed, the multiplexing order is determined. According to the non-empty buffer state in which there is no space in the main buffer of the ES concerned, the VMB is determined as the second multiplexing decision, and the VMB is determined as the second multiplexing decision. Simulation circuit and the first,
First and second presentation time stamps (hereinafter, PTS) or decoding time stamps (hereinafter, DTS) held in each of the second AMB simulation circuits
Of the main buffer by creating the empty space in the main buffer that has no free space by setting the minimum value of the system clock reference (hereinafter referred to as SCR). No
It is characterized by performing multiplexing.

According to a third aspect of the present invention, in the program stream multiplexing method according to the second aspect, the multiplexing ranking is called from each of the VMB simulation circuit and the first and second AMB simulation circuits. The maximum values of the DTS and the first and second PTSs are compared with each other, and the maximum value is ordered from the smaller maximum value to the larger one, and the first multiplexing judgment is performed. A multiplexing decision without adjustment of the SCR for the simulation of each of the VMB and the first and second AMB, wherein the second multiplexing decision is the SC
It is characterized in that it is a multiplexing decision involving adjustment of R.

The invention according to claim 4 is the program stream multiplexing method according to claim 2, wherein:
MB simulation circuit and the first and second AMBs
Each of the simulation circuits includes the VMB or the first or second AMB corresponding to the P-STD.
(Hereinafter, MB) free space is held, and in response to the supply of the data amount of the output video or the first or second audio ES (hereinafter, output ES amount), the output ES is changed from the free space of the MB to the output ES. Amount is subtracted to obtain usable capacity, and the AU amount and the DTS in response to the supply of the AU amount which is the number of access units and the DTS or the first or second PTS (hereinafter, DTS / PTS). The first SCR output from the SCR generation circuit, which holds / PTS in association with each other.
Value of the DTS / PTS is compared with the value of the DTS / PTS.
When the / PTS value is smaller, the DT is added to the free space.
Add the AU amount associated with S / PTS to obtain the DTS
When the value of / PTS is larger, the above DT held
S / PTS and the AU associated with the DTS / PTS
A quantity, and in response to supplying the second SCR from the pack selection circuit, the value of the second SCR and the DTS /
The PTS values are compared, and when the DTS / PTS value is equal to or smaller than the second SCR value, the DTS / PT
The AU amount related to S is output to the pack selection circuit, the AU amount related to the DTS / PTS is added to the free space of the MB, and the addition value as the value of the addition result and the capacity of the MB are added. When the added value is larger, the underflow of the MB is output, and in response to the supply of the maximum value confirmation request, the maximum value of the DTS / PTS is output to supply the free space confirmation request. In response, the free space of the MB is output, and in response to the supply of the minimum value confirmation request, the minimum value of the DTS / PTS is output.

According to a fifth aspect of the program stream multiplexing method of the present invention, each of the digital video signal and the first and second digital audio signals is input and compression encoded according to a predetermined compression encoding protocol. An elementary stream (hereinafter, referred to as a video ES) and first and second audio elementary streams (hereinafter, referred to as an audio ES) are generated, and these video E are generated.
In a program stream multiplexing method for multiplexing S and first and second audio ESs to generate a program stream, a program system target that is a virtual decoder used in encoding to create the program stream Decoder (hereinafter P-STD)
Of the video main buffer (hereinafter, VMB) for holding the video ES and first and second audio main buffers (hereinafter, for holding the first and second audio ES).
VMB that holds the state of each memory occupancy of AMB)
The maximum value of each of the decoding time stamp (hereinafter, DTS) or the first and second presentation time stamp (hereinafter, PTS) held by each of the simulation circuit and the first and second AMB simulation circuits is compared. , The DTS having the smallest maximum value or the first and second Ps
The video ES or the first or second audio ES (hereinafter, ES) corresponding to the TS is set as the multiplexing order 1, and the multiplexing is performed in the order of increasing the maximum value. And the P- of the ES that is the multiplexing rank 1 (hereinafter, the multiplexing rank 1 ES).
A first multiplexing determination and multiplexing process, which is a multiplexing determination without adjustment of a system clock reference (hereinafter, SCR) for STD simulation, and an ES having a multiplexing priority of 2 (hereinafter, a multiplexing priority of 2ES). ), The first multiplexing determination and multiplexing processing, and E with the multiplexing order of 3
S (hereinafter, multiplexing order 2ES), the first multiplexing determination and multiplexing process, and the SCR having the multiplexing order 1ES
Second multiplexing determination and multiplexing processing which is multiplexing determination with adjustment of the second multiplexing determination and multiplexing processing of the multiplexing order 2ES, and the second multiplexing determination and multiplexing processing of the multiplexing order 3ES. It is characterized by performing a multiplexing determination and a multiplexing process.

The invention according to claim 6 is the program stream multiplexing method according to claim 5, wherein the first multiplexing judgment and multiplexing processing of the multiplexing order 1ES is data for multiplexing. In order to determine whether a certain amount of ESs (hereinafter referred to as a multiplexing target ES) exists in the buffer of the multiplexing order 1ES, the buffer occupancy of the multiplexing order 1ES is confirmed, and the multiplexing target ESs perform the multiplexing. Rank 1
If it does not exist in the ES buffer, this multiplexing order 1E
The first multiplexing determination of S is completed, and the multiplexing order 2ES
When the multiplexing target ES exists in the buffer of the multiplexing order 1ES in the multiplexing determining step of the multiplexing order 1ES and the multiplexing determining step of the multiplexing order 1ES, Multiplexing possibility determination processing of multiplexing order 1ES, which determines whether the main buffer of the multiplexing order 1ES inputs the ES without causing overflow or underflow, and the multiplexing possibility When the multiplexing is possible in the multiplexing possibility determination processing of the rank 1ES, the multiplexing processing of the multiplexing rank 1ES for performing the multiplexing of the multiplexing rank 1ES is included.

According to a seventh aspect of the present invention, in the program stream multiplexing method according to the sixth aspect, the multiplexing possibility judgment processing of the multiplexing order 1ES is such that the MB simulation circuit of the multiplexing order 1ES is free. A first free space confirmation step of calling a first free space, which is a free space, and a second free space calculated by subtracting the multiplexing target ES amount of the multiplexing order 1ES from the first free space. The free space calculation step of 2 and the SCR added when performing the multiplexing are output to the MB simulation circuit having the multiplexing order of 1ES, and the MB simulation circuit having the multiplexing order of 1ES performs the multiplexing at the SCR. A deletion AU amount calling step for calling an AU amount, which is the number of access units deleted from the MB having a conversion order of 1 ES; The AU amount is added to the can volume, third
A third free space calculation step of calculating the free space of
It is determined whether the third free space is within the capacity of the main buffer with the multiplexing order of 1ES. If it is within the capacity of the main buffer, the process proceeds to the multiplexing process with the multiplexing order of 1ES, If the capacity is exceeded, the multiplexing order 1 of the multiplexing order 1ES is terminated, and the process proceeds to the first multiplexing judgment of the multiplexing order 2ES.
And a step of determining the MB capacity of the ES, wherein the multiplexing process of the multiplexing order 1ES is instructed by the buffer of the multiplexing order 1ES in response to the instruction of the output ES data amount (hereinafter, ES amount). If the output ES amount instruction step of outputting the ES amount and the SCR to be added to the program stream PS are output, and the ES to be multiplexed includes the head of the AU, and if the multiplexing order 1ES is an audio ES, SCR and ES parameter output step of outputting the PTS and the AU amount, and outputting the PTS, the DTS, the AU amount and the picture type in the case of a video ES, and an SCR generating circuit for adding the SCR to the program stream. To the simulation SCR generation step, and the ES amount of the ES to be multiplexed is set to the multiplexing order 1ES. When the multiplexing target ES includes the head of the AU, the PTS and the AU amount are output when the multiplexing target ES includes the head of the AU, and the DTS is output when the multiplexing ES is the video ES. And the ES amount for multiplexing to output the AU amount and
Parameter output step and S preset in the SCR
SCR addition value addition step of adding and holding CR addition value, and performing multiplexing based on the ES, the SCR, the PTS, the DTS, the AU amount, and the picture type, and outputting the program stream. It is characterized by doing.

A program stream multiplexer according to an eighth aspect of the present invention is a video encoder for inputting a digital video signal, performing compression encoding according to a predetermined compression encoding protocol, and outputting a video elementary stream (hereinafter referred to as video ES). And a video buffer holding the video ES, and first and second digital audio signals, respectively, are input and compression-encoded according to the compression-encoding protocol, and first and second audio elementary streams (hereinafter , Audio ES) for outputting respective first and second audio encoders, first and second audio buffers for holding each of the first and second audio ES, the video buffer and the first, second The video ES held in each of the second audio buffers And a multiplexer for multiplexing the first and second audio ESs and outputting as a program stream, the multiplexer confirms and holds the occupation amount of the video ES in the video buffer. VB occupancy confirmation circuit and a video main buffer (hereinafter, VMB) of a program system target decoder (hereinafter, P-STD), which is a virtual decoder used in encoding to create the program stream. A VMB simulation circuit for performing simulation, and first and second AB occupation amount confirmation for confirming and retaining the respective occupation amounts of the first and second audio ES in the first and second audio buffers, respectively. Circuit and the first and second audio main buffers of the P-STD (hereinafter referred to as AM The simulate each) 1, and a second AMB simulation circuit, the video ES and the first and second audio E
The state of the memory occupancy of each of the VMB of the P-STD and the first and second AMBs is managed with respect to S, and the video ES is arranged in the empty order of the VMB and the first and second AMBs. And a pack selection circuit for performing multiplexing ordering and multiplexing judgment so as to multiplex the first and second audio ESs and controlling multiplexing, and the P-STD based on the data from the pack selection circuit. An SCR generation circuit that generates a system clock reference (hereinafter, SCR) for simulating the VMB and the first and second AMBs, and a multiplexing circuit that performs multiplexing under the control of the pack selection circuit are configured. There is.

The invention according to claim 9 is the program stream multiplexing method according to claim 8, wherein the multiplexer selects the pack selection circuit, the SCR generation circuit, the VMB simulation circuit, and the VB occupation amount confirmation. The circuit and the functions of the first and second AB occupation amount confirmation circuits are replaced by a software-configured microcontroller.

The invention according to claim 10 is the same as that of claim 8
In the program stream multiplexing method described above, the P-STD converts the input program stream into the video ES, the first and second audio ESs, the SCR, and a presentation time stamp (hereinafter, referred to as a presentation time stamp).
PTS) and a decoding time stamp (hereinafter referred to as DTS), a demultiplexing unit, a system clock generation unit that receives the SCR and generates a system time clock (STC), and the VM that holds the video ES.
B, the AMB that holds each of the first and second audio ES, the video decoder that decodes the video ES and outputs the video signal V, and each of the first and second audio ES. The STC and the PTS or the DTS are compared with the first and second audio decoders that decode and output the first and the second audio signals, respectively, and the VMB or the first audio decoder is selected according to the comparison result. Or second
The AMB of the PTS or DTS to output the ES corresponding to the PTS or DTS to the video decoder or the first or second audio decoder.
, A time stamp comparison unit for outputting a control signal of the selector, a rearrangement buffer for rearranging the video signal in order to change the arrangement order of the pictures of the video signal, and the video signal for re-arranging in response to the supply of the control signal. It is configured to include first and second selectors for switching to supply to the array buffer.

[0075]

BEST MODE FOR CARRYING OUT THE INVENTION Next, embodiments of the present invention will be described in detail with reference to the drawings.

The program stream multiplexing method and apparatus according to the present embodiment is an expert group I of ISO and CCITT as a compression encoding technology for information including moving images and audio.
SO / IEC, JTC1 / SC2 / WG8 (Movin
MP specified in ISO / IEC-11172-3 recommended by g Picture Expert Group)
The target is multiplexing of a program stream (hereinafter, PS) of MPEG2 (hereinafter, MPEG standard) defined by EG1 standard or ISO / IEC-13818.

When multiplexing one video elementary stream (hereinafter, ES) and two audio ESs in PS multiplexing, a program which is a virtual decoder used in encoding for PS creation. The state of the memory occupancy of the main buffer in the system target decoder (hereinafter, P-STD) is shown by a P-STD video main buffer (hereinafter, VMB) simulation circuit and the first and second P-STD audio main buffers (hereinafter, AM
B) Hold in the simulation circuit, and the pack selection circuit is the VMB simulation circuit, the first and second AMB.
From the contents of holding the occupancy of each of the simulation circuits, the vacant rank which is the order in which the main buffer in the P-STD becomes empty is grasped, and the vacant rank is set as the multiplexing rank of each ES, It is determined whether or not multiplexing is possible, and if multiplexing is possible, multiplexing is performed. When all ESs cannot be multiplexed, the pack selection circuit confirms a non-multiplexable factor that is a factor that cannot be multiplexed according to the multiplexing order, and the non-multiplexable factor is available in the main buffer of the corresponding ES. When there is no empty buffer state, the VMB simulation circuit and the first and second A
By calling the minimum value of the presentation time stamp (hereinafter referred to as PTS) or the decoding time stamp (hereinafter referred to as DTS) held in the MB simulation circuit and setting the value as the system clock reference (hereinafter referred to as SCR) It is characterized in that a vacant space is created in the main buffer that does not exist and multiplexing is performed, so that multiplexing is performed without depending on the size of the main buffer existing in the P-STD.

Next, referring to FIG. 1 which is a block diagram showing a program stream multiplexer according to a first embodiment of the present invention, the program stream multiplexer according to the present embodiment shown in this figure is a digital video signal. VD is input and compression-encoded according to the compression-encoding protocol defined by the MPEG standard, and the video elementary stream (E
S) Video encoder 1 that outputs VS and video ES
Video buffer 4 for temporarily holding VS, and first and second
Each of the audio encoders 2 and 3 for inputting each of the digital audio signals AD1 and AD2 of FIG. Audio buffers 5 and 6 held, and video E held in each of the video buffer 4 and audio buffers (AB) 5 and 6.
And a multiplexer 7 that multiplexes SVS and audio ESAS and outputs as a program stream PS.

The multiplexer 7 includes a VB occupation amount confirmation circuit 73 for confirming and retaining the occupation amount of the video ES in the video buffer (VB) 4, and a program system target decoder (hereinafter, P-STD) as described later. VMB for simulating video main buffer (VMB) 204
The simulation circuit 74, the audio buffer 5,
AB occupancy confirmation circuit 75 for confirming and retaining the occupancy of each of audio ESAS1 and AS2 in each of 6
77, AMB simulation circuits 76 and 78 for simulating each of the first and second audio main buffers (AMBs) 207 and 209 of the P-STD as described later, and data from the pack selection circuit 71 described later. An SCR generation circuit 79 for generating a system clock reference (hereinafter, SCR) for P-STD main buffer simulation based on the video ES to be multiplexed.
P-STD VMB 204 and AMB 207, 20 for video ESVS and audio ESAS1, AS2 to generate a pack (basic unit of stream) consisting of packets from VS and audio ESAS1, AS2
9 manages the state of each memory occupancy of each
And a pack selection circuit 71 for performing multiplexing ordering and determination of multiplexing in an empty order of the AMBs 207 and 209 to control selection of multiplexing, and multiplexing for multiplexing selected packs controlled by the pack selection circuit 71. And a circuit 72.

Referring to FIG. 19 showing in block form the configuration of the P-STD which is a virtual decoder for the PS output by the PS multiplexer of this embodiment, the P-STD of this embodiment shown in this figure is referred to. STD, input PS to video ESV
S, audio ESAS1, AS2, SCR, PTS, and demultiplexing unit 201 for decomposing into DTS, and SC
A system clock generation unit 202 that receives a supply of R to generate a system time clock (STC), a VMB 204 that holds a video ES, and audio ESAS1 and AS
Audio main buffers (AMBs) 207 and 209 for holding each of the two, STC and PTS or DT
SMB is compared with VMB204 or A according to the comparison result.
An ES corresponding to PTS or DTS for the MB207 / 209 is provided as a video decoder 205 or an audio decoder 2.
08/210 is instructed to output, and a time stamp comparison unit 203 which outputs control signals of selectors 206 and 212 described later, and a video decoder 205 which decodes the input video ES and outputs a video signal V are input. The audio decoders 208 and 210 for decoding the audio ESA1 and A2 and outputting the audio signals A1 and A2 respectively, and the rearrangement buffer 211 for the video signal V in response to the supply of the control signal from the time stamp comparing unit 203. Selectors 206 and 2 for switching to supply to
12 and a rearrangement buffer 211 that rearranges to change the arrangement order of the pictures of the video signal.

Next, an outline of the operation of this embodiment will be described with reference to FIGS.

First, before describing the operation of each unit, general PS generation according to the MPEG standard will be described. The encoder encodes video and audio digital data to be encoded according to the MPEG standard. The encoded data is expressed as ES. Next, the ES is packetized for each access unit (AU) or for each arbitrary size, and converted into a packetized elementary stream (PES) packet. In the case of audio, one or several audio frames, and in the case of video, all encoded data for one image is expressed as AU.

The PES packet has a PES header,
If the PES packet has an AU head, the AU
Decoding time stamp (DTS) as the decoding time of
The presentation time stamp (P
TS) is added. A PS (program stream) is created by adding a pack header to a PES packet to form (pack) a basic unit of a stream. The pack header has a system clock reference (SC) for calibrating the system clock of the decoder.
R) is added. Here, video ES and audio E
Converting S to PS is referred to as multiplexing. Therefore,
A pack contains a pack header and packets from one or more ESs.

The multiplexer adds SCR, PTS and DTS to PS so that the sound and the video can be synchronized when the PS including the audio ES and the video ES is reproduced by the decoder. In addition, the multiplexer defines a virtual program system target decoder called P-STD, and guarantees that PS does not cause underflow or overflow in all main buffers (MB) in P-STD and synchronously reproduces. Must.

Regarding each MB size of P-STD, the P-STD audio main buffer (hereinafter, AMD) size is 4096 bytes in the MPEG1 standard.
The P-STD video main buffer (hereinafter referred to as VMB) size is defined as 47104 bytes, and the VMB size is 2293 in the MPEG2 standard.
It is defined as 76 bytes.

Next, referring to FIG. 2, P of the present embodiment will be described.
To describe the operation of the STD, the demultiplexing unit 201 uses the input PS as the video ESVS, the audio ESAS1 and AS2, the SCR, the PTS, and the DT.
Decompose into S and video ESVS into VMB204 and audio ESAS1 and AS2 respectively into AMB207 and 20.
9 for each SCR is a system clock generation unit 202.
Then, the PTS and DTS are respectively supplied to the time stamp comparison unit 203. The system clock generator 202
An STC is generated based on the supplied SCR and is supplied to each unit.

The time stamp comparing section 203 receives the supplied STC and PTS or DTS (hereinafter “or” /,
(For example, PTS / DTS etc.)
V when DTS matches STC or STC is larger
P for MB204 / AMB207 / AMB209
ES corresponding to TS / DTS is converted to video decoder 205 /
Instruct the audio decoder 208/210 to output. The video decoder 205 decodes the input video ES and outputs a video signal. Audio decoder 2
08 and 210 are input audio ESAS
1 and AS2 respectively to decode audio signals A1 and A2
Output each of.

Here, when the video ES includes B pictures, the P-STD needs to change the order of outputting the video signals V. That is, in the video ES including the B picture in the PS, the I picture and the P picture (hereinafter, I and P pictures) are coded before the B picture.
Decoding is also performed first for I and P pictures. However, since the reproduction is performed after the B picture, the PTS and DTS for the I and P pictures do not match. Therefore, in the case of I and P pictures, the time stamp comparison unit 203 determines that the DTS and S
VMC when TC match or STC becomes larger
From 204, the video decoder 205 is instructed to output the video ES. On the other hand, the time stamp comparison unit 203
Instructs the selector 206 to temporarily store the decoded I and P pictures in the rearrangement buffer 211, and the PTS and STS
When the TCs match, the selector 212 is instructed to output the I and P pictures held in the rearrangement buffer 211, so that the I and P pictures are once put on standby and output. In the case of a B picture, all comparison with STC is performed by PTS, and it is not held in the rearrangement buffer 211 and is directly output as the video signal V. In this way, the output order of the video signals is changed so that the B, I, and P pictures are output in a predetermined order.

The multiplexer of the present embodiment has the above-mentioned P-ST.
The operation of D is virtually realized, and multiplexing is performed so as not to cause underflow or overflow in all main buffers in the P-STD.

A constant value is added to the SCR every time multiplexing is performed. This added value is expressed by the following equation.

Added value of SCR (theoretical value) = pack length / system bit rate (1) Here, the system bit rate is determined based on the generation amount of each of the video ES and the audio ES.

The amount of audio ES generated is determined by the audio bit rate, and its value is constant. Video ES
The generation amount of is determined by the video bit rate. However, in the case of a variable bit rate, this value is not constant and the generation amount of the video ES may fluctuate, which may cause VMB underflow and overflow. A method is used to prevent this.

Regarding the VMB underflow, the system bit rate is calculated with a value higher than the video bit rate instructed to the video encoder in advance, and the SCR is calculated.
VMB by setting the addition value of
Prevent underflow of. Regarding VMB overflow, when the VMB becomes full, the value of SCR is advanced to output ES from VMB to prevent VMB overflow.

Next, the operation of each part of this embodiment will be described with reference to FIGS. 1 and 2. First, the video encoder 1 inputs the digital video signal VD and receives MPE.
Video elementary stream (ES) VS is subjected to compression encoding according to the compression encoding protocol defined by the G standard.
Is output and supplied to the video buffer 4. The video buffer 4 holds the input video ES, outputs the video ES of the data amount instructed to the pack selection circuit 71, and supplies the video ES to the multiplexing circuit 72. Audio encoders 2 and 3
Inputs the digital audio signals AD1 and AD2, respectively, and performs compression encoding according to the compression encoding protocol of the MPEG standard, and audio ESAS1 and AS2.
Are output and supplied to the audio buffers 5 and 6, respectively. Each of the audio buffers 5 and 6 holds each of the input audio ESAS1 and AS2, outputs each of the audio ESAS1 and AS2 of the data amount instructed to the pack selection circuit 71 of the multiplexer 7, and outputs the multiplexed circuit. Supply to 72.

The multiplexing circuit 72, the details of which will be described later, receives the input video ESVS, audio ESAS1, AS.
2, SCR, PTS, DTS, AU amount of each ES, multiplexing is performed based on the picture type, and PS is output.

The VB occupation amount confirmation circuit 73 of the multiplexer 7 is
The VB occupation amount, which is the data amount of the video ESVS held in the video buffer 4, is confirmed and held. When the capacity confirmation request QCV is supplied from the pack selection circuit 71 through the video buffer 4, the VB occupying amount held according to the supply of the capacity confirmation request QCV is output to the pack selection circuit 71. Further, each of the AB occupation amount confirmation circuits 75 and 77 is the data amount of each of the audio ESAS1 and AS2 held by each of the audio buffers 5 and A.
Each of the S1 occupancy and AS2 occupancy is confirmed and held. Capacity confirmation request from the pack selection circuit 71 QCAS1, QCA
When each of S2 is supplied through each of the audio buffers 5 and 6, the capacity confirmation requests QCAS1 and QCA are received.
AS1 occupancy held for each supply of S2,
Each of the AS2 occupation amounts is output to the pack selection circuit 71.

Next, the operation of the pack selection circuit 71 when the video ES is multiplexed will be described. The pack selection circuit 71 has the multiplexing circuit 72 for the video buffer 4.
The data amount of the output video ESVO is output to the VMB simulation circuit 74, and the SCR added to the PS is output to the multiplexing circuit 72 and the SCR generation circuit 79. When the output video ESVO includes the head of AU, the PTS, DTS, AU amount, and picture type to be added to PS are output to the multiplexing circuit 72. At the same time, the AU amount and DTS are calculated by the VMB simulation circuit
Output to. Furthermore, the SC held as described later
The SCR addition value is added to R. Below, output video ESV
When distinguishing the AU amount of O, it is called the AUV amount.

Next, audio ESAS1, ESAS2
The operation of the pack selection circuit 71 when multiplexing audio data will be described. The pack selection circuit 71 instructs the audio buffers 5 and 6 to specify the data amount of each of the output audio ESAS1O and ESAS2O to the multiplexing circuit 72. This output audio ESAS1O, E
The data amount of each SAS2O is output to each of the AMB simulation circuits 76 and 78, and the SCR to be added to PS is output to the multiplexing circuit 72 and the SCR generation circuit 79. When the head of the access unit (AU) is included in each of the output audio ESAS1O and ESAS2O, the PTS and AU amount to be added to PS are output to the multiplexing circuit 72. At the same time, the number of AUs (hereinafter AU amount)
And PTS are output to each of the AMB simulation circuits 76 and 78. In addition, the SCR
Add the added value. Below, audio ESAS1O, E
AU1 amount, AU2 amount, PT for each AU amount of SAS2O
S is called PTS1 and PTS2. When the two are not distinguished, they are collectively referred to as the AU amount and PTS.

The multiplexing circuit 72 uses the audio ESAS1.
O, ESAS2O, SCR and PTS are input, multiplexed, and PS is output.

Next, the operation of the VMB simulation circuit 74 will be described. The VMB simulation circuit 74 holds the free capacity of the VMB 204 of P-STD and responds to the supply of the data amount of the output video ESVO from the pack selection circuit 71. Then, the usable capacity is obtained by subtracting the data amount of the output video ESVO from the free capacity of the VMB 204. Further, in response to the supply of the AUV amount and the DTS, the AUV amount and the DTS are held in association with each other. Further, the value of the SCR for simulation (hereinafter, SCR value) output from the SCR generation circuit 79 is compared with the value of the DTS held (hereinafter, DTS value), and the DTS is calculated from the SCR value.
When the value is smaller, the free space in the VMB 204 is set to the DTS.
Add the AUV amount associated with. When the DTS value is larger than the SCR value, the DTS held and its DTS
Delete the AU amount associated with. Also, the pack selection circuit 7
In response to the supply of the SCR output by 1, the SCR value is compared with the held DTS value, and when the DTS value is equal to or smaller than the SCR value, the AU amount associated with the DTS is selected by the pack selection circuit. 71 and VM
The AU amount related to the DTS is added to the free capacity of B204. In addition, the addition value that is the value of the addition result and the VMB 20
4 and the capacity is larger, VMB2
The underflow 04 is output to the pack selection circuit 71.
Further, in response to the supply of the maximum value confirmation request from the pack selection circuit 71, the maximum value of the held DTS is output to the pack selection circuit 71. In response to the supply of the free space confirmation request from the pack selection circuit 71, the free space of the VMB 204 is output to the pack selection circuit 71. Furthermore, in response to the supply of the minimum value confirmation request from the pack selection circuit 71,
The minimum value of the held DTS is output to the pack selection circuit 71.

Next, the AMB simulation circuit 76,
The operation of the AMB simulation circuit 76 will be described on behalf of the AMB simulation circuit 76.
Holds the free capacity of the AMB 207 of the P-STD and, in response to the supply of the data amount of the output audio ESAS1O from the pack selection circuit 71, subtracts the data amount of the output audio ESAS1O from the free capacity of the AMB 207 and decreases the usable capacity. Ask for. Further, in response to the supply of the AU1 amount and PTS1, the AU1 amount and PTS1 are held in association with each other. Further, the SCR value for simulation output by the SCR generation circuit 79 is compared with the value of PTS1 held (hereinafter referred to as PTS1 value), and when the PTS1 value is smaller than the SCR value, the free capacity of the AMB 207 is related to PTS1. Add the AU1 amount. When the PTS1 value is larger than the SCR value, the retained PTS1 and its PT
Delete the AU1 quantity associated with S1. In addition, in response to the supply of the SCR output by the pack selection circuit 71, the SC
The R value is compared with the held PTS1 value, and when the PTS1 value is equal to or smaller than the SCR value, the PTS
The AU1 amount related to 1 is output to the pack selection circuit 71, and the AU1 amount related to PTS1 is added to the free capacity of the AMB 207. Further, the added value which is the value of the addition result is compared with the capacity of the AMB 207, and when the added value is larger, the underflow of the AMB 207 is output to the pack selection circuit 71. Further, in response to the supply of the maximum value confirmation request from the pack selection circuit 71, the PTS held therein is held.
The maximum value of 1 is output to the pack selection circuit 71. In response to the supply of the free space confirmation request from the pack selection circuit 71, A
The empty capacity of the MB 207 is output to the pack selection circuit 71. Furthermore, in response to the supply of the minimum value confirmation request from the pack selection circuit 71, the held minimum value of PTS1 is output to the pack selection circuit 71.

The operation of the AMB simulation circuit 78 is the same as the operation of the AMB simulation circuit 76 described above, except that AMB207 is output to AMB209, output audio ESAS1O is output audio ESAS2O, and AU1.
Is the same as AU2, and PTS1 is PTS2.

Next, the method of determining the multiplexing order of the pack selection circuit 71 will be described.
The maximum value confirmation request is output to the VMB simulation circuit 74 and the AMB simulation circuits 76 and 78. The VMB simulation circuit 74 outputs the maximum value of the DTS held therein to the pack selection circuit 71 in response to the reception of the maximum value confirmation request. Each of the AMB simulation circuits 76 and 78 outputs the maximum value of each of PTS1 and PTS2 held therein in response to the reception of the maximum value confirmation request to the pack selection circuit 71, respectively. Pack selection circuit 71
Compares the maximum value of the input DTS and the maximum value of each of PTS1 and PTS2 in total, and sets the ES corresponding to DTS / PTS1 / PTS2, which has the smallest maximum value, as the multiplexing order 1. , And multiplex ordering is performed in which the multiplexes are performed in order of increasing maximum value. Here, the order of this multiplexing order is called a multiplexing order.

Explaining the multiplexing judgment, the pack selection circuit 71 roughly divides each ES into two kinds of multiplexing judgments depending on whether or not the SCR for P-STD main buffer simulation is adjusted. The first is a multiplexing decision without SCR adjustment, and is hereinafter referred to as a multiplexing decision 1.
The second is a multiplexing decision involving the adjustment of SCRCS.
This is called multiplexing judgment 2.

The pack selection circuit 71 makes a multiplexing decision 1 for each ES according to the determined multiplexing order, and when it is confirmed that either ES can be multiplexed, the pack selection circuit 71 performs the multiplexing process. To finish. When all ESs cannot be multiplexed in the multiplexing judgment 1, the multiplexing judgment 2 is performed for each ES according to the determined multiplexing order, and when it is confirmed that any ES can be multiplexed, After multiplexing, the multiplexing process is completed. If all ESs cannot be multiplexed even in the multiplexing determination 2, the multiplexing process ends.

Next, the operation of the multiplexing judgment 1 for the video ESVS will be described. The pack selection circuit 71
Video ESVS with sufficient amount of data for multiplexing
In order to determine whether (hereinafter, the video ES to be multiplexed) exists in the video buffer 4, a capacity confirmation request QCV is output to the VB occupation amount confirmation circuit 73 to perform capacity confirmation. When it is confirmed that the video ES to be multiplexed exists in the video buffer 4, the free space of the VMB 204 of the P-STD is confirmed with respect to the VMB simulation circuit 74. V
When it is confirmed that the free space of the MB 204 is larger than the data amount of the multiplexing target video ESVS, the video ESV
S is multiplexed and the multiplexing process is completed.

The pack selection circuit 71 includes a video buffer 4
When it is confirmed that there is no target video ES for multiplexing, and the free capacity of the VMB 204 is equal to the target video E for multiplexing.
When it is confirmed that the data amount is smaller than the SVS data amount, the multiplexing determination 1 for the video ESVS is ended.

At this time, if an ES having a lower rank than the multiplexing rank of the video ES exists, the multiplexing judgment 1 is executed for the ES. If there is no ES having a lower rank than the multiplexing rank of the video ES, the multiplexing judgment 2 is executed for the ES having the multiplexing rank 1.

Next, the operation of the multiplexing judgment 1 for the audio ESAS1 will be described. The pack selection circuit 7
1 is an audio ESAS1 satisfying the amount of data for multiplexing (hereinafter, the audio ESAS1 to be multiplexed)
Is present in the audio buffer 5,
A capacity confirmation request QCV is output to the AB occupation amount confirmation circuit 75 to confirm the capacity. When it is confirmed that the audio ESAS1 to be multiplexed exists in the audio buffer 5, P-
The free space of the STD AMB 207 is confirmed. AMB2
When it is confirmed that the free space of 07 is larger than the data amount of the multiplexing target audio ESAS1, the audio ES
The AS1 is multiplexed, and the multiplexing process ends.

When the pack selection circuit 71 confirms that the multiplexing target audio ESAS1 does not exist in the audio buffer 5 and confirms that the free capacity of the AMB 207 is smaller than the data amount of the multiplexing target audio ESAS1, the pack selection circuit 71 The multiplexing determination 1 for ESAS1 ends.

At this time, if there is an ES having a lower rank than the multiplexing rank of the audio ESAS1, the multiplexing judgment 1 is executed for the ES. If there is no ES having a lower rank than the multiplexing rank of the audio ESAS1, the multiplexing judgment 2 is executed for the ES having the multiplexing rank 1.

The operation of the multiplexing judgment 1 for the audio ESAS2 is performed by changing the audio ESAS1 from the audio ESA1.
In S2, the audio buffer 5 is replaced with the audio buffer 6
The AB occupation amount confirmation circuit 75 is replaced with the AB occupation amount confirmation circuit 77.
Further, the operation is the same as the operation of the multiplexing judgment 1 for the audio ESAS1 described above except that the AMB simulation circuit 76 is replaced with the AMB simulation circuit 78 and the AMB 207 is replaced with the AMB 209.

Next, the operation of the multiplexing judgment 2 for each ES will be described. Since the operation of the multiplexing determination 2 for each ES differs depending on the multiplexing order of each ES, the following description will be given for each multiplexing order.

First, the operation of the multiplexing determination 2 for the video ES having the multiplexing priority of 1 will be described. The pack selection circuit 71 determines whether the multiplexing multiplexing target video ES exists in the video buffer 4 or not. A capacity confirmation request QCV is output to the occupation amount confirmation circuit 73 to confirm the capacity. When it is confirmed that the video ES to be multiplexed does not exist in the video buffer 4, the multiplexing determination 2 for the video ES is ended, and the multiplexing determination 2 for the ES having the multiplexing priority 2
To execute.

The pack selection circuit 71 includes a video buffer 4
By confirming that the video ES to be multiplexed exists, it can be seen that there is no free space to input the video ES in the VMB 204 of the P-STD. At this time, the pack selection circuit 71 calls the minimum value of DTS held in the VMB simulation circuit 74, and sets the value to SC.
SCR for output to the R generation circuit 79 for simulation
Change the value of. The VMB simulation circuit 74
DTS and A held according to the changed SCR value
The UV amount and the free space of the VMB 204 are changed and deleted.

After the above operation, the pack selection circuit 71 causes the VM
A free space confirmation request is output to the B simulation circuit 74, and the free space of the VMB 204 is confirmed.
When it is confirmed that the free space of 4 is larger than the data amount of the video ES to be multiplexed, the video ES is multiplexed and the multiplexing process ends. Further, when it is confirmed that the free capacity of the VMB 204 is smaller than the amount of the video ES to be multiplexed, the pack selection circuit 71 changes the SCR, V
Change and delete the value held in the MB simulation circuit 74, A
Change and delete the value held in the MB simulation circuit 76, A
The operations of changing and deleting the held value of the MB simulation circuit 78 and checking the free capacity of the VMB 204 by the pack selection circuit 71 are repeated until the video ES can be multiplexed.

Next, the operation of the multiplexing judgment 2 for the video ES having the multiplexing order 2 will be described. The pack selection circuit 71 judges whether the multiplexing ES to be multiplexed exists in the video buffer 4. A capacity confirmation request QCV is output to the VB occupation amount confirmation circuit 73 to confirm the capacity. When it is confirmed that the video ES to be multiplexed does not exist in the video buffer 4, the multiplexing determination 2 is executed for the ES having the multiplexing order 3. When it is confirmed that the video ES to be multiplexed exists in the video buffer 4, V
It can be seen that there is no free space to input the video ES into the MB 204. At this time, the pack selection circuit 71 is
The minimum value of the DTS held in the simulation circuit 74 is called, the minimum value of the DTS is output as the SCR to the main buffer (MB) simulation circuit corresponding to the ES of the multiplexing order 1, and the P- of the multiplexing order 1 is output. STD
It is determined whether or not the main buffer of No. 1 underflows, and when it is confirmed that it underflows, the multiplexing judgment 2 is executed for the ES of the multiplexing order 3. When it is confirmed that the underflow does not occur, the minimum value of the called DTS is output to the SCR.

The VMB simulation circuit 74 adjusts the value of the changed SCR and holds the DTS and AU.
The amount of V and the free capacity of the VMB 204 are changed and deleted. AM
The B simulation circuit 76 adjusts the PTS, AUV amount, and AMB20 that are held according to the changed SCR value.
Change and delete the free space of 7. The AMB simulation circuit 78 changes and deletes the held PTS, AUV amount, and free space of the AMB 209 according to the changed SCR value.

After the above operation, the pack selection circuit 71 causes the VM
A free space confirmation request is output to the B simulation circuit 74, and the free space of the VMB 204 is confirmed.
When it is confirmed that the free space of 4 is larger than the data amount of the video ES to be multiplexed, the video ES is multiplexed and the multiplexing process ends. When it is confirmed that the free capacity of the VMB 204 is smaller than the data amount of the video ES to be multiplexed, it is determined whether or not there is an underflow of the P-STD main buffer of the multiplexing order 1 due to the SCR change by the pack selection circuit 71 described above. Confirmation and change of SCR, change and deletion of held value of VMB simulation circuit 74, change and deletion of held value of AMB simulation circuit 76, change and deletion of held value of AMB simulation circuit 78, check of free capacity of VMB 204 by pack selection circuit 71 This operation is repeated until the video ES can be multiplexed.

Next, the operation of the multiplexing judgment 2 for the video ES having the multiplexing order 3 will be described. The pack selection circuit 71 judges whether or not the multiplexing ES to be multiplexed exists in the video buffer 4. A capacity confirmation request QCV is output to the VB occupation amount confirmation circuit 73 to confirm the capacity. When it is confirmed that the video ES to be multiplexed does not exist in the video buffer 4, the multiplexing determination 2 ends. Also, when it is confirmed that the video ES to be multiplexed exists, it can be seen that there is no free space for inputting the video ES in the VMB 204.

The pack selection circuit 71 calls the minimum value of the DTS held in the VMB simulation circuit 74, outputs the minimum value of the DTS as an SCR to the MB simulation circuits of the multiplexing order 1 and the multiplexing order 2, and performs the multiplexing. It is determined whether or not the main buffer of the P-STD having the conversion order 1 and the multiplexing order 2 underflows. When it is confirmed that either or both of the main buffers of the multiplexing order 1 and the multiplexing order 2 underflow, the multiplexing judgment 2 is ended.

P-ST of multiplexing order 1 and multiplexing order 2
When it is confirmed that the D main buffer does not underflow, the SCR generation circuit 7 outputs the minimum value of the called DTS.
9, and the SCR generation circuit 79 changes the SCR based on this DTS. VMB simulation circuit 74
Is the DT that is held according to the changed SCR value.
The S and AUV amounts and the free space of the VMB 204 are changed and deleted. Each of the AMB simulation circuits 76 and 78 has
The PTS held according to the changed SCR value
1, PTS2, AU1 amount, AU2 amount, AMB207, 2
Change and delete each free space of 09.

After the above operation, the pack selection circuit 71 causes the VM
A free space confirmation request is output to the B simulation circuit 74 to confirm the free space of the MB 204. VMB204
When it is confirmed that the free space of is larger than the data amount of the video ES to be multiplexed, the video ES is multiplexed and the multiplexing process ends. Further, when it is confirmed that the free capacity of the VMB 204 is smaller than the data amount of the video ES to be multiplexed, the P-STD main buffers of the multiplexing order 1 and the multiplexing order 2 due to the SCR change by the pack selection circuit 71 described above. Confirmation of presence / absence of underflow, change of SCR, change and deletion of held value of VMB simulation circuit 74, change and deletion of held value of AMB simulation circuit 76, change and deletion of held value of AMB simulation circuit 78, VMB 204 by pack selection circuit 71 The operation of checking the free space of is repeated until the video ES can be multiplexed.

Next, the audio ESAS having the multiplexing order 1
The operation of the multiplexing determination 2 for 1 will be described. The pack selection circuit 71 determines that the multiplexing target audio ESAS1.
A exists in the audio buffer 5 to determine whether
A capacity confirmation request QCAS1 is output to the B occupation amount confirmation circuit 75 to confirm the capacity. When it is confirmed that the audio ESAS1 to be multiplexed does not exist in the audio buffer 5, the multiplexing judgment 2 for the audio ESAS1 is ended, and the multiplexing judgment 2 is executed for the ES having the multiplexing rank 2. Audio ES to be multiplexed in audio buffer 5
By confirming that AS1 exists, AMB2
It can be seen that there is no free space for inputting the audio ESAS1 in 07. At this time, the pack selection circuit 71
The minimum value of PTS1 held in the simulation circuit 76 is called and the value is output to the SCR generation circuit 79. The SCR generation circuit 79 changes the SCR based on the minimum value of PTS1. VMB simulation circuit 74
Is the DT that is held according to the changed SCR value.
The S and AUV amounts and the free space of the VMB 204 are changed and deleted. The AMB simulation circuit 76 has a modified S
According to the CR value, the amount of PTS1 and AU1 held,
The free capacity of the AMB 207 is changed and deleted. The AMB simulation circuit 78 changes and deletes the held PTS2, AU2 amount, and free space of the AMB 209 according to the changed SCR value.

After the above operation, the pack selection circuit 71 turns the AM
A free space confirmation request is output to the B simulation circuit 76 to confirm the free space of the AMB 207, and the AMB 20
When it is confirmed that the free space of 7 is larger than the data amount of the multiplexing target audio ESAS1, the audio ESA
S1 is multiplexed and the multiplexing process is completed. Also AM
Free space of B207 is audio ESAS1 to be multiplexed
When it is confirmed that the data amount is smaller than the data amount, the pack selection circuit 71 changes the SCR, the VMB simulation circuit 74 retains and deletes the retained value, the AMB simulation circuit 76 retains the changed value, and the AMB simulation circuit 78 retains. Value change and deletion, pack selection circuit 7
The operation of checking the free space of the AMB 207 by 1 is repeated until the audio ESAS1 can be multiplexed.

Next, an audio ESAS having a multiplexing rank of 2
The operation of the multiplexing determination 2 for 1 will be described. The pack selection circuit 71 uses the multiplexing target audio ES.
To determine whether AS1 exists in the audio buffer 5, a capacity confirmation request QC is issued to the AB occupancy confirmation circuit 75.
AS1 is output to confirm the capacity. Audio buffer 5
When it is confirmed that the multiplexing target audio ESAS1 does not exist, the multiplexing judgment 2 is executed for the ES having the multiplexing order 3. When it is confirmed that the audio ESAS1 to be multiplexed exists in the audio buffer 5, it can be seen that there is no free space to input the audio ESAS1 to the AMB 207. At this time, the pack selection circuit 71 calls the minimum value of PTS1 held in the AMB simulation circuit 76, and outputs this minimum value of PTS1 as an SCR to the main buffer (MB) simulation circuit corresponding to the ES of the multiplexing order 1. Then, it is determined whether or not the main buffer of the P-STD of the multiplexing order 1 underflows, and when it is confirmed that the main buffer underflows, the multiplexing determination 2 is executed for the ES of the multiplexing order 3. When it is confirmed that the underflow does not occur, the minimum value of the called PTS1 is output to the SCR.

The VMB simulation circuit 74 adjusts the value of the changed SCR and holds the DTS and AU held therein.
The amount of V and the free capacity of the VMB 204 are changed and deleted. AM
The B simulation circuit 76 matches the changed SCR value and holds the PTS1, AU1 amount, and AMB2.
The free space of 07 is changed and deleted. The AMB simulation circuit 78 changes and deletes the held PTS2, AU2 amount, and free space of the AMB 209 according to the changed SCR value.

After the above operation, the pack selection circuit 71 is
A free space confirmation request is output to the B simulation circuit 76 to confirm the free space of the AMB 207, and the AMB 20
When it is confirmed that the free space of 7 is larger than the data amount of the multiplexing target audio ESAS1, the audio ESA
S1 is multiplexed and the multiplexing process is completed. AMB20
When it is confirmed that the free space of 7 is smaller than the data amount of the audio ESAS1 to be multiplexed, it is determined whether or not there is an underflow of the P-STD main buffer of multiplexing order 1 due to the SCR change by the pack selection circuit 71 described above. Confirmation and change of SCR, change and deletion of held value of AMB simulation circuit 76, change and deletion of held value of AMB simulation circuit 76, change and deletion of held value of AMB simulation circuit 78, AMB2 by pack selection circuit 71
The operation of checking the free space of 07 is audio ESAS
Repeat until 1 can be multiplexed.

Next, an audio ESAS having a multiplexing rank of 3
The operation of the multiplexing determination 2 for 1 will be described. The pack selection circuit 71 uses the multiplexing target audio ES.
To determine whether AS1 exists in the audio buffer 5, a capacity confirmation request QC is issued to the AB occupancy confirmation circuit 75.
AS1 is output to confirm the capacity. Audio buffer 5
When it is confirmed that the multiplexing target audio ESAS1 does not exist, the multiplexing determination 2 is ended. Further, when it is confirmed that the audio ESAS1 to be multiplexed exists, it can be seen that the AMB 207 has no free space for inputting the audio ESAS1.

The pack selection circuit 71 calls the minimum value of PTS1 held in the AMB simulation circuit 76, and outputs the minimum value of PTS1 as an SCR to the MB simulation circuits of multiplexing order 1 and multiplexing order 2,
It is determined whether the P-STD main buffers of the multiplexing order 1 and the multiplexing order 2 underflow. Multiplexing order 1
When it is confirmed that either or both of the main buffers of the multiplexing order 2 underflow, the multiplexing determination 2 is ended.

P-ST of multiplexing order 1 and multiplexing order 2
When it is confirmed that the main buffer of D does not underflow, the minimum value of the called PTS1 is output to the SCR generation circuit 79, and the SCR generation circuit 79 changes the SCR based on this PTS1. VMB simulation circuit 7
4 is the D that is held according to the changed SCR value
Change and delete TS, AUV amount, and free space of VMB 204. Each of the AMB simulation circuits 76 and 78 matches the changed SCR value and holds the PT.
S1, PTS2, AU1 amount, AU2 amount, AMB207,
The respective free capacity of 209 is changed and deleted.

After the above operation, the pack selection circuit 71 is
A free space confirmation request is output to the B simulation circuit 76 to confirm the free space of the MB 204. AMB207
When it is confirmed that the free space of is larger than the data amount of the audio ESAS1 to be multiplexed, the audio ESAS
1 is multiplexed and the multiplexing process is completed. Also, AMB
When it is confirmed that the free capacity of 207 is smaller than the data amount of the multiplexing target audio ESAS1, the underflow of the P-STD main buffer of the multiplexing order 1 and the multiplexing order 2 due to the SCR change by the pack selection circuit 71 described above. Presence / absence confirmation and SCR change, AMB simulation circuit 76 holding value change deletion, AMB simulation circuit 76 holding value change deletion, AMB simulation circuit 78 holding value change deletion, pack selection circuit 71
The operation of confirming the free capacity of the AMB 207 is repeated until the audio ESAS 1 can be multiplexed.

Audio ES with multiplexing order 1, 2 and 3
The operation of the multiplexing decision 2 for the AS2 is the audio ES.
AS1 for audio ESAS2, audio buffer 5 for audio buffer 6, and AB occupied amount confirmation circuit 75
To the AB occupancy confirmation circuit 77, the AMB simulation circuit 76 to the AMB simulation circuit 78, and the AMB
The operation is the same as the operation of the multiplexing determination 2 for the audio ESAS1 having the multiplexing order 1, 2, and 3 described above, except that 207 is replaced with the AMB 209 and the capacity confirmation request QCAS1 is replaced with the capacity confirmation request QCAS2.

Next, referring to FIGS. 1 and 2 and FIGS. 3 to 8 which show the processing of each unit in a flow chart, the operation of the program stream multiplexing apparatus of this embodiment, which is the operation of the program stream multiplexing of this embodiment, will be described. The method of conversion will be described in detail.

Referring to FIG. 3, in the multiplexing ordering step A1, the pack selection circuit 71, as described above,
DTS held by the VMB simulation circuit 74
And the maximum value of each of PTS1 and PTS2 held by each of the AMB simulation circuits 76 and 78, a total of three maximum values are compared, and DTS / P with the smallest maximum value is compared.
Multiplexing prioritization is performed from ES corresponding to each of TS1 / PTS2.

The multiplexing judgment 1 and the multiplexing without the adjustment of the SCR for the P-STD simulation of the ES having the multiplexing order of 1 (hereinafter, the multiplexing order 1ES) will be described. In the following description, the multiplexing order will be described. When 1ES is a video ES, the buffer is the video buffer 4, the buffer occupancy confirmation circuit is the VB occupancy confirmation circuit 73, the P-STD main buffer (hereinafter MB) simulation circuit is the VMB simulation circuit 74, and the P-STD main buffer. Becomes VMB204. When the multiplexing order 1ES is audio ESAS1 and ESAS2, the buffers are audio buffers 5 and 6, and the buffer occupation confirmation circuit is AB.
Occupancy confirmation circuits 75 and 77, MB simulation circuits are AMB simulation circuits 76 and 78, P-STD
The main buffers of AMB are 207 and 209.

Multiplexing judgment step A of multiplexing order 1 ES
2, the pack selection circuit 71 determines that the ES of the data amount for multiplexing (hereinafter, the ES to be multiplexed) has the multiplexing order of 1 ES.
1 to determine whether or not it exists in the buffer
A capacity confirmation request is output to the ES buffer occupancy confirmation circuit to confirm the capacity. That is, it is confirmed whether or not the buffer occupation amount of the multiplexing order 1ES ≧ the multiplexing amount of the multiplexing order 1ES.

If it is confirmed in step A2 that the target ES to be multiplexed does not exist in the buffer of the multiplexing order 1ES, the multiplexing judgment 1 of the multiplexing order 1ES is terminated, and the branch B is executed.
Then, the process proceeds to the multiplexing determination 1 of the multiplexing order 2ES.

In step A2, when it is confirmed that the ES to be multiplexed exists in the buffer of the multiplexing order 1ES,
Proceeding to the multiplexing possibility judgment processing of steps A3 to A7, the main buffer of the P-STD with the multiplexing order of 1ES is the E
Even if S is input, it is judged that neither overflow nor underflow can occur and multiplexing is possible.

First, in the MB free space confirmation step A3,
The pack selection circuit 71 outputs a free space confirmation request to the MB simulation circuit of the multiplexing order 1ES, inputs a free space as a response to this free space confirmation request, and sets this value as the free space 1. In the free space 2 calculation step A4, the multiplexing ES from the free space 1 to the multiplexing order 1 ES
The amount is subtracted, and the free space 2 is calculated.

At the deletion AU amount calling step A5, the pack selection circuit 71 outputs the SCR added at the time of multiplexing to the MB simulation circuit of the multiplexing order 1ES. The MB simulation circuit with the multiplexing order of 1 ES is
The AU amount deleted from the P-STD main buffer having the multiplexing order of 1ES by the input SCR is output to the call pack selection circuit 71.

In the free space 3 calculation step A6, the pack selection circuit 71 adds the AU amount to the free space 2. This value is used as the free space 3. The free capacity 3 indicates the free capacity of the P-STD main buffer of this ES when the multiplexing order 1 ES is multiplexed.

In the MB capacity determination step A7 of the multiplexing order 1ES, the pack selection circuit 71 determines that the free capacity 3 is the P-STD main buffer capacity (MB capacity) of the multiplexing order 1ES.
Determine if it is inside.

At step A7, when the free capacity 3 exceeds the MB capacity of the multiplexing order 1ES, the multiplexing order 1ES
End the multiplexing judgment 1 of the
Proceed to S multiplexing determination 1.

If the empty capacity 3 is within the MB capacity of the multiplexing order 1ES in step A7, it is assumed that the MB of the multiplexing order 1ES does not overflow or underflow even if the multiplexing order 1ES is multiplexed. ~ A12
Then, the multiplexing of the multiplexing order 1ES is performed.

In the output ES amount instruction step A8, the pack selection circuit 71 outputs the ES data amount to the multiplexing circuit 72 in the buffer of the multiplexing order 1ES (hereinafter, ES amount).
Instruct. The buffer of the multiplexing order 1ES receives the instruction and outputs the instructed ES amount to the multiplexing circuit 72.

SCR and ES parameter output step A9
Then, the pack selection circuit 71 outputs the SCR added to PS to the multiplexing circuit 72. When the ES to be multiplexed includes the head of AU, the following parameters are output to the multiplexing circuit 72 depending on the type of the multiplexing order 1ES. If the multiplexing order 1ES is audio ESAS1 or ESAS2 (hereinafter, audio ESAS1 / ESAS2), PT
The S and AU amounts are output to the multiplexing circuit 72. Multiplexing order 1
When the ES is a video ES, the PTS, DTS, AU amount, and picture type are output to the multiplexing circuit 72.

Simulation SCR Generation Step A
At 10, the pack selection circuit 71 adds the SCR to the PS.
Is output to the SCR generation circuit 79.

Amount of ES to be multiplexed / parameter output In step A11, the pack selection circuit 71 determines the ES to be multiplexed.
The quantity is output to the MB simulation circuit with the multiplexing order of 1 ES. When the ES to be multiplexed includes the head of AU, the following parameters are output to the MB simulation circuit having the multiplexing order of 1ES depending on the type of the multiplexing order of 1ES. That is, the multiplexing order 1ES is audio 1ES
In the case of / ESAS2, the PTS and AU amount are output, and in the case of the video ES, the DTS and AU amount are output.

At step S12 of adding the SCR added value, the pack selection circuit 71 adds and holds the SCR added value to the SCR.

The multiplexing circuit 72 receives the input ES, SC
Multiplexing is performed based on R, PTS, DTS, AU amount, and picture type, and PS is output.

The MB simulation circuit having the multiplexing order of 1 ES subtracts the input ES amount from the free space of the main buffer of the P-STD having the multiplexing order of 1 ES. When the multiplexing order 1ES is a video ES, the MB simulation circuit holds the input DTS and AU amount in association with each other.
When the multiplexing order 1ES is audio 1ES / ESAS2, the MB simulation circuit receives the input PTS and AU.
Associate and hold quantities.

As described above, the VMB simulation circuit 74 uses the SCR and VMB simulation circuit 74.
When the DTS is smaller, the AU amount related to the DTS is added to the free capacity of the VMB 204, and when the DTS is larger, the DTS and the AU amount related to the DTS are compared. delete.

AMB simulation circuit 76/78
Compares the SCR with PTS1 / PTS2 held in each of the AMB simulation circuits 76 and 78,
AMB207 / 20 when S1 / PTS2 is smaller
9 free space, AU1 associated with PTS1 / PTS2
Amount / AU2 amount is added, and when PTS1 / PTS2 is larger, PTS1 / PTS2 and PTS1 / PTS2
Delete the AU amount associated with.

In the following description, the operation of changing and deleting the parameter value held in each of the VMB simulation circuit 74 and the AMB simulation circuits 76 and 78 in accordance with the above-mentioned change of the SCR set value is performed by each MB simulation circuit. It is described that the held value is changed according to the changed SCR value.

Next, the multiplexing order 2E which is the processing of the branch B
The multiplexing judgment 1 and multiplexing of the multiplexing order 2ES will be described with reference to FIG. 4 which is a flowchart showing the multiplexing judgment 1 and multiplexing of S. First, in the multiplexing judgment step B1 of the multiplexing order 2ES, the pack is determined. The selection circuit 71 determines that the ES to be multiplexed is an ES whose multiplexing order is 2 (multiplexing order 2E).
In order to determine whether or not it exists in the buffer of S), a capacity confirmation request is output to the buffer occupancy confirmation circuit of the multiplexing order 2ES to confirm the capacity. That is, it is confirmed whether or not the buffer occupation amount of the multiplexing order 2ES ≧ the multiplexing amount of the multiplexing order 1ES.

At step B1, the pack selection circuit 71
If it is confirmed that the amount of ES to be multiplexed does not exist in the buffer of the multiplexing order 2ES, the multiplexing judgment 1 of the multiplexing order 2ES is ended, and the process proceeds to the multiplexing judgment 1 of the multiplexing order 3ES after the branch C.

At step B1, the pack selection circuit 71
When it is confirmed that the amount of ES to be multiplexed exists in the buffer of the multiplexing order 2ES, the process proceeds to steps B2 to B6,
P-STD main buffer (M
Even if B) inputs the ES, it judges whether neither overflow nor underflow will occur.

First, in the MB free space confirmation step B2,
The pack selection circuit 71 outputs a free space confirmation request to the MB simulation circuit of the multiplexing order 2ES, inputs a free space as a response to this free space confirmation request, and sets this value as the free space 4. In the free space 5 calculation step B3, the multiplexing E from the free space 4 to the multiplexing order 2ES is performed.
The free space 5 is calculated by subtracting the S amount.

In the deletion AU amount calling step B4, the pack selection circuit 71 outputs the SCR added when the multiplexing is performed to the MB simulation circuit of the multiplexing order 2ES. The MB simulation circuit of multiplexing order 2ES is
The AU amount deleted from the MB of the multiplexing order 2ES by the input SCR is output to the call pack selection circuit 71.

In the free space 6 calculation step B5, the pack selection circuit 71 adds the AU amount to the free space 5 to calculate the free space 6
To calculate. The free space 6 indicates the free space of the MB of this ES when the multiplexing order 2 ES is multiplexed.

In the MB capacity judgment step B6 of the multiplexing order 2ES, the pack selection circuit 71 judges whether the empty capacity 6 is within the MB capacity of the multiplexing order 2ES.

At step B6, when the free space 6 exceeds the MB capacity of the multiplexing order 2ES, the multiplexing order 2ES
End the multiplexing determination 1 of the
Proceed to S multiplexing determination 1.

If the free capacity 6 is within the MB capacity of the multiplexing order 2ES in step B6, it is assumed that neither overflow nor underflow occurs in the MB of the multiplexing order 2ES even if the multiplexing order 2ES is multiplexed. ~ B11
Then, the multiplexing of the multiplexing order 2ES is performed.

In the output ES amount instructing step B7, the pack selection circuit 71 instructs the buffer of the multiplexing order 2 ES to output the ES amount to the multiplexing circuit 72. The buffer of the multiplexing order 2ES receives the instruction, and receives the instructed ES.
The quantity is output to the multiplexing circuit 72.

SCR and ES parameter output step B8
Then, the pack selection circuit 71 outputs the SCR added to PS to the multiplexing circuit 72. When the multiplexing target ES includes the head of the AU, the parameters based on the type of the multiplexing order 2ES similar to the case of the multiplexing order 1ES described above, that is, PTS, DTS, AU amount, and picture type are set. Output to the multiplexing circuit 72.

Simulation SCR Generation Step B
At 9, the pack selection circuit 71 outputs the SCR added to PS to the SCR generation circuit 79.

In the step B10 of outputting the amount / parameter of the multiplexing target ES, the pack selection circuit 71 determines the ES of the multiplexing target.
The quantity is output to the MB simulation circuit of multiplexing order 2ES. When the ES to be multiplexed includes the head of AU, each parameter based on the type of multiplexing order 2ES similar to the case of multiplexing order 1ES described above, that is,
The PTS, DTS, AU amount, and picture type are output.

In step S11 of adding the SCR added value, the pack selection circuit 71 adds and holds the SCR added value to the SCR.

The multiplexing circuit 72 receives inputs ES, SCR, P
Multiplexing is performed based on TS, DTS, AU amount, and picture type, and PS is output.

The MB simulation circuit of the multiplexing order 2ES inputs the input E from the free space of the MB of the multiplexing order 2ES.
Subtract the S amount. When the multiplexing order 2ES is the video ES, the MB simulation circuit holds the input AU amount and the DTS in association with each other. Multiplexing order 2 ES is audio 1
In the case of ES / ESAS2, the MB simulation circuit holds the input AU amount and PTS in association with each other. Each MB
The simulation circuit changes the held value according to the changed SCR value.

Next, the multiplexing order 3E which is the processing of the branch C
5 is a flow chart showing the multiplexing decision 1 and the multiplexing of S. With reference to FIG. 5, the multiplexing decision 1 and the multiplexing of the multiplexing order 3 ES are different from the multiplexing determination 1 and the multiplexing of the above-mentioned multiplexing order 2 ES. Except for the following description, the multiplexing judgment 1ES and the multiplexing of the multiplexing order 3ES will be explained with emphasis on the multiplexing order 2ES and the steps C1 to C11. To each of
By replacing the free capacities 4, 5, 6 with the free capacities 7, 8, 9 respectively, the multiplexing judgment 1 of the multiplexing order 2ES
And the operation of multiplexing is the same.

First, in the multiplexing determination step C1 of the multiplexing order 3ES, the pack selection circuit 71 determines that the multiplexing order 3ES
If the capacity of the buffer occupancy confirmation circuit of No. 3 is confirmed and it is confirmed that the amount of ES to be multiplexed does not exist in the buffer having the multiplexing order of 3ES, the multiplexing judgment 1 of the multiplexing order of 3ES is ended, and the branch D or later is performed. Proceed to the multiplexing determination 2 of the multiplexing order 1ES.

At step C1, the pack selection circuit 71
When it is confirmed that the amount of ES to be multiplexed exists in the buffer of the multiplexing order 3ES, the process proceeds to steps C2 to C6 which are the same processing as in the case of the multiplexing order 2ES, and the MB of the multiplexing order 3ES stores the ES. Judge whether overflow or underflow will occur even if input.

At the capacity determination step C6 of the multiplexing order 3ES, the pack selection circuit 71 determines whether the empty capacity 9 is within the MB capacity of the multiplexing order 3ES.

At step C6, when the free capacity 9 exceeds the MB capacity of the multiplexing order 3ES, the multiplexing order 3ES
End the multiplexing determination 1 of the
Proceed to S multiplexing determination 2.

In step C6, if the free capacity 9 is within the MB capacity of the multiplexing order 3ES, it is determined that the MB of the multiplexing order 3ES will not overflow or underflow even if the multiplexing order 3ES is multiplexed. In steps C7 to C11, which are the same processes as in the case of the rank 2ES, the multiplexing of the rank 3ES is performed.

The multiplexing circuit 72 has inputs ES, SCR, P
Multiplexing is performed based on TS, DTS, AU amount, and picture type, and PS is output.

Each MB simulation circuit changes the held value according to the changed SCR value.

Next, the multiplexing order 1E which is the processing of the branch D
6 is a flowchart showing the multiplexing decision 2 and the multiplexing of S. With reference to FIG. 6, the multiplexing decision 2 of the multiplexing order 1 ES and the multiplexing are different from the multiplexing decision 1 of the multiplexing order 1 ES and the multiplexing described above. First, in the multiplexing determination step D1 of the multiplexing order 1ES, the pack selection circuit 71 determines whether the target ES to be multiplexed exists in the buffer of the multiplexing order 1ES. The capacity confirmation request is output to the 1ES buffer occupancy confirmation circuit to confirm the capacity. That is, it is confirmed whether or not the buffer occupation amount of the multiplexing order 1ES ≧ the multiplexing amount of the multiplexing order 1ES.

If it is confirmed in step D1 that the target ES to be multiplexed does not exist in the buffer having the multiplexing order of 1ES, the multiplexing judgment 1 of the multiplexing order of 1ES is ended and the branch E is executed.
Then, the process proceeds to the multiplexing determination 1 of the multiplexing order 2ES.

If it is confirmed in step D1 that the multiplexing target ES exists in the buffer of the multiplexing order 1ES,
Proceeding to steps D2 to D4, the SCR is adjusted to increase the free space of the MB of the multiplexing order 1ES.

First, in the MB free space confirmation step D2,
The pack selection circuit 71 outputs a minimum value confirmation request to the MB simulation circuit of the multiplexing order 1ES, and when the multiplexing order 1ES is audio 1ES / ESAS2,
The corresponding MB simulation circuit responds to the minimum value confirmation request with the minimum value of PTS to the pack selection circuit 71.
When the multiplexing order 1ES is the video ES, the corresponding MB simulation circuit responds to the minimum value confirmation request with the minimum value of DTS to the pack selection circuit 71.

At the SCR change step D3, the pack selection circuit 71 inputs the PTS / DT input to the held SCR.
While setting S to change SCR, this PTS / D
The TS is output to the SCR generation circuit 79. Each MB simulation circuit changes the held value according to the changed SCR value.

At MB free space confirmation step D4, a free space confirmation request is output to the MB simulation circuit having the multiplexing order of 1ES, a free space which is a response to the free space confirmation request is input, and this value is set as the free space 10 And

In the free space determination step D5, the pack selection circuit 71 determines whether or not the free space 10 is larger than the multiplexing ES amount of the multiplexing order 1ES.

If it is determined in step D5 that the multiplexing ES amount of the multiplexing order 1ES is larger than the free space 10, step D2
The processes from to D5 are repeated until the free space 10 becomes larger than the multiplexing ES amount of the multiplexing order 1ES.

At step D5, when the free space 10 is larger than the multiplexing ES amount of the multiplexing order 1ES, step D6
~ D9, the multiplexing of the multiplexing order 1ES is performed.

In the output ES amount instructing step D6, the pack selection circuit 71 instructs the buffer of the multiplexing order 1 ES to output the ES amount to the multiplexing circuit 72. In response to the instruction, the buffer having the multiplexing order of 1 ES outputs the instructed amount of ES to the multiplexing circuit 72.

SCR and ES parameter output step D7
Then, the pack selection circuit 71 outputs the SCR added to PS to the multiplexing circuit 72. When the ES to be multiplexed includes the head of an AU, each parameter based on the type of the multiplexing order 1ES, that is, PTS, DTS, and AU similar to the case of the multiplexing determination 1 of the multiplexing order 1ES described above.
The amount and picture type are output to the multiplexing circuit 72.

In a step D8 of outputting the amount of ES to be multiplexed, the pack selection circuit 71 outputs the amount of ES to be multiplexed to the MB simulation circuit having the multiplexing order of 1 ES. When the ES to be multiplexed includes the head of AU, the following parameters are output to the MB simulation circuit having the multiplexing order of 1ES depending on the type of the multiplexing order of 1ES. Multiplexing order 1 ES is audio ESAS1 / ESA
In the case of S2, PTS and AU amount are multiplexed in MB of priority 1ES
Output to the simulation circuit. If the multiplexing order 1ES is a video ES, the DTS and AU amount are set to the multiplexing order 1ES.
Output to the MB simulation circuit.

At the step S9 of adding the SCR added value, the pack selection circuit 71 adds and holds the SCR added value to the SCR.

The multiplexing circuit 72 has inputs ES, SCR, P
Multiplexing is performed from TS, DTS, AU amount, and picture type, and PS is output.

Each MB simulation circuit changes the held value according to the changed SCR value.

Next, the multiplexing order 2E which is the processing of the branch E
7 is a flowchart showing S multiplexing decision 2 and multiplexing. With reference to FIG. 7, regarding multiplexing decision 2 and multiplexing of multiplexing order 2 ES, the difference from multiplexing decision 2 and multiplexing of multiplexing order 1 ES described above Except for the following description, the multiplexing determination 2 and the multiplexing of the multiplexing order 2ES will be explained by focusing on the multiplexing order 1ES to the multiplexing order 2ES, and steps D1, D4, D5, D6, D7, By replacing D8, D9 with steps E1, E4, E5, E6, E7, E8, E9 and free space 10 with free space 11, respectively, the same operation as the multiplexing judgment 2 and the multiplexing operation of the multiplexing order 1ES can be performed. is there.

First, in the multiplexing determination step E1 of the multiplexing order 2ES, the pack selection circuit 71 determines the multiplexing order 2ES.
When the capacity of the buffer occupancy confirmation circuit of No. 2 is confirmed and it is confirmed that the amount of ES to be multiplexed does not exist in the buffer of the multiplexing order 2ES, the multiplexing judgment 2 of the multiplexing order 2ES is finished, and the branch F and subsequent steps are performed. Proceed to the multiplexing judgment 2 of the multiplexing order 3ES.

At step E1, the pack selection circuit 71
When it is confirmed that the amount of ES to be multiplexed exists in the buffer of the multiplexing order 2ES, steps E12 to E1 described later are performed.
In step 4, it is checked whether the MB of the multiplexing order 2ES does not underflow when the free space of the MB of the multiplexing order 2ES is increased by adjusting the SCR.

First, in the MB minimum value confirmation step E12,
The pack selection circuit 71 receives the minimum value of PTS / DTS as a response to the minimum value confirmation request from the MB simulation circuit of the multiplexing order 2ES. This value is a temporary SCR
Write 1.

At the SCR changing step E13, the pack selection circuit 71 sets the temporary SCR1 as the SCR and the multiplexing order 1E.
When the DTS / PTS held therein is equal to or smaller than SCR, the MB simulation circuit adds the AU amount related to DTS / PTS to the free space of the MB. The MB simulation circuit of the multiplexing order 1ES compares the addition result with the MB capacity of the multiplexing order 1ES, and when the addition result is larger, the MB underflow is selected by the pack selection circuit 71.
Output to.

At the underflow occurrence judgment step E14,
The pack selection circuit 71 determines whether or not an MB underflow of the multiplexing order 1ES has occurred. If an underflow has occurred, the pack selection circuit 71 terminates the multiplexing decision 2 of the multiplexing order 2ES, and the multiplexing order of the branch F and thereafter. Proceed to 3ES multiplexing determination 2.

At step E14, M of multiplexing order 1ES
If the underflow of B does not occur, it is assumed that the MB does not underflow even if the SCR is adjusted, and in steps E12 and E4, the SCR is adjusted and the multiplexing order is 2E.
Increase the free space of S MB.

At step E15, the pack selection circuit 71
Is the S that the pack selection circuit 71 holds the temporary SCR1.
The value is changed by setting it to CR and is output to the SCR generation circuit 79. Each MB simulation circuit
The held value is changed according to the changed SCR value.

At the free space confirmation step E4, the pack selection circuit 71 receives the free space as a response to the free space confirmation request from the MB simulation circuit of the multiplexing order 2ES. This free space is called free space 11.

At the free space determination step E5, the pack selection circuit 71 determines whether or not the free space 11 is larger than the multiplexing ES amount of the multiplexing order 2ES, and the free space 11 indicates that the multiplexing ES amount of the multiplexing order 2ES is larger than the multiplexing ES amount. If so, step E1
2, E13, E14, E15, E4, E5 free space 1
Repeat until 1 becomes larger than the above-mentioned multiplexed ES amount.

If the free space 11 is larger than the amount of multiplexing ES in step E5, the processing is the same as in the case of multiplexing order 1ES, and in steps E6 to E9, the multiplexing order 2E is executed.
S is multiplexed.

The multiplexing circuit 72 receives the inputs ES, SCR, P
Multiplexing is performed based on TS, DTS, AU amount, and picture type, and PS is output.

Each MB simulation circuit changes the held value in accordance with the changed SCR value.

Next, the multiplexing order 3E which is the processing of the branch F
8 is a flowchart showing the multiplexing determination 2 and the multiplexing of S. With reference to FIG. 8, the multiplexing determination 2 and the multiplexing of the multiplexing priority 3 ES are different from the multiplexing determination 2 and the multiplexing of the multiplexing priority 2 ES described above. Explaining mainly, the multiplexing judgment 2 and the multiplexing of the multiplexing order 3S are performed by changing the multiplexing order 2ES to the multiplexing order 3ES, except for the following description, in steps E1, E4, E5, E6, E7, By replacing E8 and E9 with steps F1, F4, F5, F6, F7, F8, and F9, free space 11 with free space 12, and SCR1 with SCR2, the multiplexing determination 2 and multiplexing of the multiplexing order 2ES are performed. Is the same as the operation of.

First, in the multiplexing determination step F1 of the multiplexing order 3ES, the pack selection circuit 71 determines the multiplexing order 2ES.
When the capacity of the buffer occupancy confirmation circuit of No. 2 is confirmed and it is confirmed that the amount of ES to be multiplexed does not exist in the buffer of the multiplexing order 2ES, the multiplexing judgment 2 of the multiplexing order 2ES is ended, and the multiplexing process is performed. To finish.

In step F1, if the amount of ES to be multiplexed exists in the buffer having the multiplexing order of 3ES, step F1
From 2 to F16, adjust SCR and M of multiplexing order 3ES
When the free capacity of B is increased, it is confirmed whether the MBs of the multiplexing order 1ES and the multiplexing order 2ES do not underflow.

First, in the MB minimum value confirmation step F12,
The pack selection circuit 71 receives the minimum value of PTS / DTS as a response to the minimum value confirmation request from the MB simulation circuit of the multiplexing order 3ES. This value is a temporary SCR
Described as 2.

At the SCR change step F13, the pack selection circuit 71 sets the temporary SCR2 as the SCR and the multiplexing order 1E.
Output to MB simulation circuit of S, multiplexing order 1
DTS held by ES MB simulation circuit
When / PTS is equal to SCR or smaller than SCR, the AU amount related to DTS / PTS is added to the free space of the MB. The MB simulation circuit of the multiplexing order 1ES compares the addition result with the MB capacity of the multiplexing order 1ES, and outputs the MB underflow to the pack selection circuit 71 when the addition result is larger.

At the underflow occurrence determination step F14,
The pack selection circuit 71 determines whether an MB underflow of the multiplexing order 1ES has occurred, and if an underflow has occurred, the multiplexing determination 2 of the multiplexing order 3ES is completed, and the multiplexing process is completed.

At step F14, M of multiplexing order 1ES
If the underflow of B does not occur, step F1
Go to 5.

At the SCR change step F15, the pack selection circuit 71 sets the temporary SCR2 as the SCR and the multiplexing order 2E.
Output to the S MB simulation circuit. In the MB simulation circuit of the multiplexing order 2ES, when the DTS / PTS held by the SCR is the same value or smaller than the SCR,
The AU amount related to DTS / PTS is added to the free space of the MB, and the addition result is compared with the MB capacity of the multiplexing order 2ES. When the addition result is larger, the underflow of the MB is selected by the pack selection circuit. To 71.

At the underflow occurrence determination step F16,
The pack selection circuit 71 determines whether an MB underflow of the multiplexing order 2ES has occurred, and if an underflow has occurred, the multiplexing determination 2 of the multiplexing order 3ES is ended, and the multiplexing process is ended.

At step F16, M of multiplexing order 2ES
When the underflow of B does not occur, it is assumed that the MBs of the multiplexing order 1ES and the multiplexing order 2ES do not underflow even if the SCR is adjusted, and the process proceeds to steps F17 and F4 to adjust the SCR to adjust the multiplexing order 3ES. Increase the free space of MB.

In step F17, the pack selection circuit 71
Is the S that the pack selection circuit 71 holds the temporary SCR2.
The value is changed by setting it to CR and is output to the SCR generation circuit 79. Each MB simulation circuit
The held value is changed according to the changed SCR value.

At the free space confirmation step F4, the pack selection circuit 71 receives the free space as a response to the free space confirmation request from the MB simulation circuit of the multiplexing order 3ES. This free space is referred to as free space 12.

In the free capacity determination step F5, the pack selection circuit 71 determines whether or not the free capacity 12 is larger than the multiplexing ES amount of the multiplexing order 3ES, and from the free capacity 12, the multiplexing ES amount of the multiplexing order 3ES is determined. If so, step F1
2, F13, F14, F15, F16, F17, F4
F5 is repeated until the free space 12 becomes larger than the multiplexed ES amount.

If the free space 12 is larger than the amount of multiplexing ES in step F5, the same processing as in the case of multiplexing order 2ES is performed in steps F6 to F9.
S is multiplexed.

The multiplexing circuit 72 uses the inputs ES, SCR, P
Multiplexing is performed based on TS, DTS, AU amount, and picture type, and PS is output.

Each MB simulation circuit changes the held value according to the changed SCR value.

Next, referring to FIG. 9 which is an explanatory diagram showing the states of the multiplexer 7 before and after the multiplexing, the multiplexing determination and the multiplexing processing of the multiplexer of the present embodiment will be described in detail. As described above, the pack selection circuit 71 uses the maximum value of the DTS held by the VMB simulation circuit 74 and the AMB simulation circuit 7 as described above.
The maximum values of PTS1 and PTS2 held by 6 and 78 are compared, and DTS / PTS1 / PTS
DTS / PTS1 / PTS2 among ES corresponding to 2
The order in which the maximum value of each is smaller is the order in which the multiplexing determination is performed.

The multiplexing conditions of this example are shown below.

Capacity of VMB 204 = 229376 bytes.

AMB207 capacity = 4096 bytes AMB209 capacity = 4096 bytes Video ES multiplexing amount = 2015 bytes Audio ESAS1 multiplexing amount = 2015 bytes Audio ESAS2 multiplexing amount = 2015 bytes SCR added value = 159 , The contents held in the VMB simulation circuit 74 to (A), and the AMB simulation circuits 76 and 78.
The contents held in (B) and (C), the VB occupation amount confirmation circuit 7
The content held in 3 is set to (D), and the AB occupation amount confirmation circuits 75 and 7
The contents stored in No. 7 are set to (E) and (F), and the SCR generation circuit 79
(G) shows the contents stored in the pack selection circuit 71 and (H) shows the contents stored in the pack selection circuit 71. Further, the contents of the AMB simulation circuit 78, the AB occupancy confirmation circuit 77, the SCR generation circuit 79, and the pack selection circuit 71 before and after multiplexing are (CA), (CB), (FA), and (FB), respectively. ,
They are shown in (GA), (GB), (HA), and (HB), respectively.

In the example of this figure, the maximum value of DTS is 66066 for P picture, and the maximum value of PTS1 is 6480.
0, the maximum value of PTS2 is 60480, so P
TS2 <PTS1 <DTS.

Therefore, the multiplexing order 1ES is audio E
SAS2, multiplexing order 2ES is audio ESAS1,
The multiplexing order 3ES becomes a video ES.

The pack selection circuit 71 has a multiplexing order of 1 ES.
It is determined whether the occupied amount of the buffer 6 of the audio ESAS2 is larger than the multiplexed ES amount of the audio ESAS2. The ES amount (occupancy amount) existing in the buffer 6 can be confirmed by the AMB simulation circuit 78, and it can be seen that it is 2400 bytes shown in FIG. 9 (FA).

The audio ESAS2 multiplexed ES amount is
It is 2015 bytes from the multiplexing condition, and audio ES
It can be confirmed that the occupied amount of the buffer 6 of the AS2 is larger than the multiplexed ES amount of the audio ESAS2.

Next, the pack selection circuit 71 determines that the AMB 209 can be multiplexed without causing overflow or underflow even if the audio ESAS2 is multiplexed. The multiplexed ES amount of the audio ESAS2 is 2015 bytes, and it can be confirmed that the occupied amount 2400 bytes of the buffer 6 of the audio ESAS2 is larger than the multiplexed ES amount of the audio ESAS2.

Next, the pack selection circuit 71 determines that the AMB 209 can be multiplexed without causing overflow or underflow even if the audio ESAS2 is multiplexed. The pack selection circuit 71 calls the free capacity of the AMB 209 held by the AMB simulation circuit 78, and the free capacity becomes 4 as shown in FIG. 9 (CA).
Confirm that it is 000 bytes. This free space is referred to as free space 1.

Next, the pack selection circuit 71 determines the free space 1
Is subtracted from the multiplexed ES amount of audio ESAS2 = 2015 bytes, and the value of the subtraction result is described as free space 2. The free space 2 is 1985 bytes.

The pack selection circuit 71 outputs the held SCR = 60559 shown in FIG. 9 (HA) to the AMB simulation circuit 78.

The AMB simulation circuit 78 determines whether or not there is an AU amount and PTS to be deleted from the AMB simulation circuit 78 in the input SCR, or the notified SCR and P
Compare TS. The AMB simulation circuit 78 notifies the PTS = 60 which is smaller than the notified SCR = 60559.
Since 480 exists (FIG. 9 (CA)), its AU amount =
The pack selection circuit 71 is notified of 1152 bytes.

The pack selection circuit 71 adds the notified AU amount to the free space 2, and the value of the addition result is the free space 3
Write. The free space 3 is 3137 bytes. Free space 3 is A when audio ESAS2 is multiplexed.
The free space of the MB 209 is shown (FIG. 9 (CB)).

In the pack selection circuit 71, the free space 3 is AM.
It is determined whether the capacity of B207 is 0 bytes or more and 4096 bytes or less. Since it has been confirmed that the free capacity 3 is within the capacity of the AMB 209, the pack selection circuit 71 multiplexes the audio ESAS 2 as follows.

The pack selection circuit 71 notifies the ES amount output to the buffer 6 = 2015 bytes. Buffer 6
Outputs the ES amount 2015 bytes notified to the multiplexing circuit 72 and SCR = 60559 added when multiplexing is performed. In addition, since the audio ESAS2 includes the beginning of the AU, the PTS added when multiplexing is performed.
= 62460, AU amount = 1152 bytes are also output. Further, SCR = 60559 is output to the SCR generation circuit 79. Furthermore, since the multiplexed ES amount of audio ESAS2 = 2015 bytes is output to the AMB simulation circuit 78 and the head of AU is included in audio ESAS2, PTS = 62460, AU amount = 1
It also outputs 152 bytes.

The pack selection circuit 71 holds the SC it holds.
SCR addition value = 159 is added to R. The addition result is 60
758 ((FIG. 9 (HB))). Multiplexing circuit 72
Inputs audio ESAS2, PTS, AU amount, and SCR, multiplexes them, and outputs PS.

The AMB simulation circuit 78 inputs the multiplexed ES amount = 2015 of the audio ESAS2,
The multiplexed ES amount is subtracted from the free capacity of the AMB 209.
The free capacity of the AMB 209 is 1985 bytes. AM
The B simulation circuit 78 inputs and holds AU amount = 1152 bytes and PTS = 62460. Also, S
CR = 60559 is compared with the held PTS.
The AMB simulation circuit 78 has SCR = 6055.
Since there is PTS = 60480 smaller than 9, the AU amount = 1152 bytes and PTS = 60480 are deleted, and the AU amount deleted in the free space of AMB209 = 1
Add 152 bytes. AMB209 has 3 free space
It becomes 137 bytes (Fig. 9 (CB)).

The AMB simulation circuit 76 uses the SC
R = 60559 is compared with the retained PTS. Since all the held PTSs are larger than SCR = 60559, the held value is not changed. The VMB simulation circuit 74 compares SCR = 60559 with the held PTS. Since all the held PTSs and DTSs are larger than SCR = 60559, the held values are not changed.

As described above, the program stream multiplexing method and apparatus of the present embodiment manages the state of the main buffer (MB) in the P-STD, and when multiple ESs are multiplexed, the MB becomes empty. By multiplexing each ES in the following order, even if ESs having the same MB capacity are multiplexed, M
B never underflows.

Next, FIG. 10 showing the second embodiment of the program stream multiplexer according to the present invention in the same manner as in FIG. Then, the difference between the present embodiment shown in this figure and the first embodiment described above is that one video ES and three video ES
One audio ES is multiplexed, and in addition to the components of the first embodiment, the third digital audio signal AD3 is compression-coded to perform audio ESAS.
Audio encoder 8 that outputs 3 and audio E
An audio buffer 9 that temporarily holds the SAS 3 and a multiplexer 7A that multiplexes the data held in each of the video buffer 4 and the audio buffers 5, 6 and 9 instead of the multiplexer 7 and outputs the multiplexed data as a program stream PS That is.

In addition to the components of the multiplexer 7, the multiplexer 7A of the present embodiment includes an AB occupation amount confirmation circuit 81 for confirming and retaining the occupation amount of the audio ESAS3 in the audio buffer 9, and a P- AM for simulating STD third audio main buffer 213
The B simulation circuit 82, the pack selection circuit 71A in which the audio ESAS3 is added as a selection target in place of the pack selection circuit 71, and the pack selection circuit 71A in place of the multiplexing circuit 72 multiplexes the selected pack including the audio ESAS3. And a multiplexing circuit 72A.

The structure of the P-STD corresponding to the present embodiment is shown in FIG. 11 in which the same components as those in FIG. 2 are similarly indicated by blocks with common reference characters / numerals. The difference between this embodiment and the first embodiment described above is that in addition to the components of the first embodiment, the PS input instead of the demultiplexing unit 201 is used as a video ES.
A demultiplexing unit 201A that decomposes VS, audio ESAS1, AS2, AS3, system clock (SCR), PTS, and DTS, and an audio main buffer (AMB) 213 that holds audio ESAS3
And an audio signal A by decoding the audio ESAS3.
3 and an audio decoder 214 for outputting 3 are provided.

Next, referring to FIG. 11, which is a block diagram of the program stream multiplexer of the second embodiment of the present invention, the same components as those in FIG. 1 are designated by common reference characters / numerals. Then, the difference between the present embodiment shown in this figure and the first embodiment described above is that one video ES and three video ES
One audio ES is multiplexed, and in addition to the components of the first embodiment, the third digital audio signal AD3 is compression-coded to perform audio ESAS.
Audio encoder 8 that outputs 3 and audio E
An audio buffer 9 that temporarily holds the SAS 3 and a multiplexer 7A that multiplexes the data held in each of the video buffer 4 and the audio buffers 5, 6 and 9 instead of the multiplexer 7 and outputs the multiplexed data as a program stream PS That is.

In addition to the components of the multiplexer 7, the multiplexer 7A of this embodiment includes an AB occupation amount confirmation circuit 81 for confirming and retaining the occupation amount of the audio ESAS3 in the audio buffer 9, and a P- AM for simulating STD third audio main buffer 213
The B simulation circuit 82, the pack selection circuit 71A in which the audio ESAS3 is added as a selection target in place of the pack selection circuit 71, and the pack selection circuit 71A in place of the multiplexing circuit 72 multiplexes the selected pack including the audio ESAS3. And a multiplexing circuit 72A.

The structure of the P-STD corresponding to this embodiment is shown in FIG. 12 in which the same components as those in FIG. 2 are similarly indicated by blocks with common reference characters / numerals. The difference between this embodiment and the first embodiment described above is that in addition to the components of the first embodiment, the PS input instead of the demultiplexing unit 201 is used as a video ES.
A demultiplexing unit 201A that decomposes VS, audio ESAS1, AS2, AS3, system clock (SCR), PTS, and DTS, and an audio main buffer (AMB) 213 that holds audio ESAS3
And an audio signal A by decoding the audio ESAS3.
3 and an audio decoder 214 for outputting 3 are provided.

Next, with reference to FIG. 10 and FIG. 11, an outline of the operation of the present embodiment will be described focusing on the differences from the first embodiment.
The digital audio signal AD3 is input, compression-encoded according to the compression encoding protocol of the MPEG standard, and the audio ESAS3 is output and supplied to the audio buffer 9. The audio buffer 9 holds the input audio ESAS 3 and uses the pack selection circuit 7 of the multiplexer 7A.
The audio ESAS3 having the data amount designated by 1A is output and supplied to the multiplexing circuit 72A.

The AB occupation amount confirmation circuit 81 confirms and retains the audio ESAS3 occupation amount held by the audio buffer 9. The AS3 occupation amount held in response to the supply of the capacity confirmation request QCAS3 from the pack selection circuit 71A is returned.

The pack selection circuit 71A compares the maximum value of DTS and the maximum value of each of PTS1, PTS2, and PTS3 of the audio ESAS3, a total of four maximum values, and selects DTS / PTS1 / PTS2 / PTS4 having the smaller maximum value. The corresponding ESs are subjected to a multiplexing determination up to a multiplexing priority of 4 for priority multiplexing, and a multiplexing process up to this multiplexing priority of 4 is performed.

Therefore, the addition of the processing to the multiplexing algorithm of the first embodiment of this embodiment does not involve adjustment of the SCR for P-STD main buffer (MB) simulation, as will be described later in detail. Multiplexing order 4ES with multiplexing judgment of multiplexing order 4ES (multiplexing judgment 1) and multiplexing processing, and adjustment of SCR for MB simulation
Is a multiplexing determination (multiplexing determination 21) and multiplexing processing.

Next, referring to FIGS. 4, 6 and 7 and 12 to 16 which are flowcharts showing the processes of FIGS. 10 and 11 and the operation of the program stream multiplexer of the present embodiment. The program stream multiplexing method according to the present embodiment will be described focusing on the differences from the first embodiment.

First, referring to FIG. 12, in the multiplexing ordering step A1A, the pack selection circuit 71A determines that the VM
Maximum value of DTS held by B simulation circuit 74 and AMB simulation circuits 76, 78, 82
Of the PTS1, PTS2, and PTS3 held by each of the four maximum values are compared, and the ES corresponding to each of the DTS / PTS1 / PTS2 / PTS4 having the smaller maximum value is preferentially multiplexed. Multiplex ranking.

For the subsequent multiplexing determination 1 and multiplexing of the multiplexing order 1 ES, and multiplexing determination 1 and multiplexing of the multiplexing order 2 ES, the pack selection circuit 71 is operated by the pack selection circuit 71A.
Further, except that the multiplexing circuit 72 is replaced with the multiplexing circuit 72A, the processing is the same as that of the first embodiment shown in FIGS. 3 and 4, and therefore the description thereof is omitted.

Next, the multiplexing order 3E which is the processing of the branch C
12, which is a flowchart showing the multiplexing determination 1 and the multiplexing of S, the differences between the multiplexing determination 1 and the multiplexing of the multiplexing order 3 ES will be mainly described with respect to the first embodiment. In the multiplexing determination step C1A of the multiplexing order 3ES, the pack selection circuit 71A determines that the multiplexing order 3E
When the capacity is confirmed with respect to the buffer occupancy confirmation circuit of S and it is confirmed that the amount of ES to be multiplexed does not exist in the buffer with the multiplexing order of 3ES, the multiplexing judgment 1 of the multiplexing order of 3ES is ended and the branch G After that, the process proceeds to the multiplexing determination 1 of the multiplexing order 4ES.

At step C1, the pack selection circuit 71
When it is confirmed that the amount of ES to be multiplexed exists in the buffer having the multiplexing order of 3 ES, the multiplexing process of steps C2 to C11 described in the first embodiment is performed.

Next, the multiplexing order 4E which is the processing of the branch G
14 is a flowchart showing the multiplexing determination 1 and the multiplexing of S. With reference to FIG. 14, the multiplexing determination 1 and the multiplexing of 4ES are different from the multiplexing determination 1 and the multiplexing of 3ES described above. The multiplexing determination 1 and the multiplexing of the multiplexing order 4ES will be described with emphasis on the multiplexing order 3ES as the multiplexing order 4ES and steps C1 to C11 as steps G1 to G11 except for the following description. By replacing the free capacities 7, 8 and 9 with the free capacities 17, 18 and 19, respectively, the operations of the multiplexing judgment 1 and the multiplexing of the multiplexing order 3ES are the same.

First, in the multiplexing determination step G1 of the multiplexing order 4ES, the pack selection circuit 71A determines the multiplexing order 3E.
When the capacity is confirmed with respect to the buffer occupancy confirmation circuit of S and it is confirmed that the amount of ES to be multiplexed does not exist in the buffer of the multiplexing order 4ES, the multiplexing judgment 1 of the multiplexing order 4ES is ended, and the branch D Thereafter, the process proceeds to the process of the multiplexing determination 2 of the multiplexing order 1ES.

At step G1, the pack selection circuit 71A
When it is confirmed that the amount of ES to be multiplexed exists in the buffer of the multiplexing order 4ES, step G2 which is the same process as the multiplexing judgment 1 of the multiplexing order 3ES and the multiplexing step C2 to C11. G11 multiplexing processing is performed.

Next, the multiplexing order 1E of branch D and branch E
The multiplexing determination 2 and multiplexing of S, and the multiplexing determination 2 and multiplexing of the multiplexing order 2 ES are performed by the pack selection circuit 71 to the pack selection circuit 71A and the multiplexing circuit 72 to the multiplexing circuit 7.
2A is the same as the processing of the first embodiment shown in FIGS. 6 and 7 except that it is replaced by 2A, and therefore the description thereof is omitted.

Next, the multiplexing order 3E which is the processing of the branch F
With reference to FIG. 15, which is a flowchart showing S multiplexing determination 2 and multiplexing, the differences between the multiplexing determination 2 and multiplexing of the multiplexing order 3 ES will be mainly described with respect to the first embodiment. First, in the multiplexing determination step F1A of the multiplexing order 3ES, the pack selection circuit 71A confirms the capacity with respect to the buffer occupancy confirmation circuit of the multiplexing order 3ES and the buffer with the multiplexing target ES amount of the multiplexing order 3ES is confirmed. When it is confirmed that the packet does not exist, the multiplexing judgment 2 of the multiplexing order 3ES is ended, and the process advances to the multiplexing judgment 2 of the multiplexing order 4ES of the branch H.

If it is confirmed in step F1 that the amount of ES to be multiplexed exists in the buffer of the multiplexing order 3ES, steps F12 to F1 described in the first embodiment.
7. F4 to F9 multiplexing processing is performed.

Next, the multiplexing order 4E which is the processing of the branch H
16 is a flowchart showing the multiplexing judgment 2 and the multiplexing of S. With reference to FIG. 16, the multiplexing judgment 2 of the multiplexing order 4 ES and the multiplexing are different from the multiplexing judgment 2 of the multiplexing order 3 ES and the multiplexing described above. Primarily, the multiplexing determination 2 and the multiplexing of the multiplexing order 4ES will be performed by the steps H12 to F16 and F4 to F9 respectively except the following description. To
Multiplexing order 3ES to multiplex ordering 4ES, free space 12
Are replaced with the free space 22 and the provisional SCR2 is replaced with the provisional SCR3, respectively, and the operations of the multiplexing determination 2 and the multiplexing of the multiplexing order 3ES are the same.

First, in the multiplexing determination step H1 of the multiplexing order 4ES, the pack selection circuit 71A determines the multiplexing order 3E.
When the capacity of the buffer occupancy confirmation circuit of S is confirmed, and it is confirmed that the amount of ES to be multiplexed does not exist in the buffer of the multiplexing order 4ES, the multiplexing judgment 2 of the multiplexing order 4ES is ended, and the multiplexing is performed. The process ends.

If it is confirmed in step H1 that the amount of ES to be multiplexed exists in the buffer of multiplexing order 4ES, the SCR is adjusted in steps H12 to H18 to adjust the free space of the MB of multiplexing order 4ES. When increased, multiplexing order 1ES, multiplexing order 2ES, and multiplexing order 3E
Check if each MB of S does not underflow.

First, in the MB minimum value confirmation step H12,
The minimum value of PTS / DTS is received as a response to the minimum value confirmation request from the MB simulation circuit of multiplexing order 4ES. This value is described as provisional SCR3.

Thereafter, the processing of steps H13 to H16, which is the same processing as steps F13 to F16, is performed.

Underflow Occurrence Judgment Step H14 / H16 of Multiplexing Order 2ES / Multiplexing Order 3ES If an underflow occurs, the multiplexing judgment 2 of the multiplexing order 4ES is ended and the multiplexing process is ended.

At step H16, M of multiplexing order 2ES
When the underflow of B does not occur, the temporary SCR 3 is output as the SCR to the MB simulation circuit of the multiplexing order 3ES in the SCR changing step H17. In the MB simulation circuit of the multiplexing order 3ES, when the DTS / PTS held by the SCR is the same value or smaller than the SCR, the free space of the MB is related to the DTS / PTS.
U amount is added, and the addition result and MB of multiplexing order 2ES
When the addition result is larger than the capacity, the underflow of MB is output to the pack selection circuit 71A.

At the underflow occurrence judgment step H18,
It is determined whether an MB underflow of the multiplexing order 3ES occurs, and if an underflow occurs, the multiplexing order 4E
The multiplexing determination 2 of S is terminated, and the multiplexing process is terminated.

At step H18, M of multiplexing order 3ES
If the underflow of B does not occur, it is assumed that the MBs of the multiplexing order 1ES, the multiplexing order 2ES, and the multiplexing order 3ES do not underflow even if the SCR is adjusted, and the process proceeds to steps H19 and H4 to adjust the SCR. And multiplexing order 4
Increase the free space of the ES MB.

At the SCR change step H19, the temporary SCR3
Is set to the SCR held by the pack selection circuit 71A to change it and output it to the SCR generation circuit 79. Each MB simulation circuit has a modified S
The held value is changed according to the CR value.

At the free space confirmation step H4, the pack selection circuit 71A receives the free space as a response to the free space confirmation request from the MB simulation circuit of the multiplexing order 3ES. This free space is called free space 22.

In the free capacity determination step H5, the pack selection circuit 71A determines whether or not the free capacity 22 is larger than the multiplexing ES amount of the multiplexing order 4ES, and the empty ES 22 determines the multiplexing ES amount of the multiplexing order 4ES. If larger, step H
12, H13, H14, H15, H16, H17, H1
8, H19, H4, H5 are allocated to the empty space 22 by the multiplexing E
Repeat until the amount exceeds S.

If the free space 22 is larger than the amount of multiplexing ES in step H5, the same processing as in the case of multiplexing order 3ES is performed in steps H6 to H9.
S is multiplexed.

The multiplexing circuit 72A has inputs ES, SCR,
The PTS, DTS, AU amount, and picture type are multiplexed, and PS is output.

Each MB simulation circuit changes the held value according to the changed SCR value.

Next, referring to FIG. 17, which shows a block diagram of the third embodiment of the present invention in which components common to those in FIG. The difference between the program stream multiplexer of the present embodiment and the first embodiment described above is that instead of the multiplexer 7,
Pack selection circuit 71, VB occupation amount confirmation circuit 73, and VMB
The simulation circuit 74 and the AB occupation amount confirmation circuit 75,
77 and AMB simulation circuit 76, 78 and SCR
A multiplexer 7B is provided in which each function of the generation circuit 79 is replaced with a microcontroller 80 by appropriate software.

The microcontroller 80 has a processing function corresponding to the algorithm of the program stream multiplexing method described in the first embodiment,
The above functions are realized.

[0282]

As described above, according to the program stream multiplexing method and apparatus of the present invention, the VMB holding the video ES of the P-STD and the first and second audio Es.
The state of the memory occupancy of each of the first and second AMBs holding each S is managed, and the VMB and the first and second AMBs are managed.
Of the main buffer (MB) including a pack selection circuit for performing multiplexing ordering and multiplexing determination to control the multiplexing so that the video ES and the first and second audio ESs are multiplexed in the order in which the AMBs are empty. ) Status is managed and multiple E
When S is multiplexed, each ES is multiplexed in the order in which the MB becomes empty, so that even if ESs having the same MB capacity are multiplexed, the MBs can be multiplexed without underflowing. effective.

[Brief description of drawings]

FIG. 1 is a block diagram showing a first embodiment of a program stream multiplexing device of the present invention.

FIG. 2 is a block diagram showing a P-STD (program system target decoder) of the present embodiment.

FIG. 3 is a flowchart showing a process in the program stream multiplexing method of the present embodiment, which is an operation of the program stream multiplexing apparatus of the present embodiment.

FIG. 4 is a flowchart showing a process of branch B in FIG.

5 is a flowchart showing a process of branch C of FIG.

FIG. 6 is a flowchart showing a process of branch D in FIG.

FIG. 7 is a flowchart showing a process of branch E in FIG.

8 is a flowchart showing the processing of branch F in FIG. 7. FIG.

FIG. 9 is an explanatory diagram showing a state of each circuit before and after multiplexing in the present embodiment.

FIG. 10 is a block diagram showing a second embodiment of the program stream multiplexing device of the present invention.

FIG. 11 is a block diagram showing a P-STD of the present embodiment.

FIG. 12 is a flowchart showing an operation of the program stream multiplexing device of the present embodiment, showing a program stream multiplexing method of the present embodiment.

FIG. 13 is a flowchart showing a process of a branch C that characterizes this embodiment.

FIG. 14 is a flowchart showing a process of branch G in FIG.

FIG. 15 is a flowchart showing a process of a branch F that characterizes this embodiment.

16 is a flowchart showing the processing of branch H in FIG.

FIG. 17 is a block diagram showing a third embodiment of the program stream multiplexing device of the present invention.

FIG. 18 is a block diagram showing an example of a conventional first program stream multiplexer.

FIG. 19 is a block diagram showing a P-STD of a conventional first program stream multiplexer.

FIG. 20 is a flowchart showing a process in a conventional first program stream multiplexing method which is an operation of the conventional first program stream multiplexing apparatus.

21 is a flowchart showing a process of branch M of FIG. 20. FIG.

22 is a flowchart showing a process of branch N in FIG. 21. FIG.

FIG. 23 is a block diagram showing an example of a second conventional program stream multiplexer.

FIG. 24 is a block diagram showing a P-STD of a second conventional program stream multiplexer.

FIG. 25 is a flowchart showing a process in a second conventional program stream multiplexing method which is an operation of the second conventional program stream multiplexing apparatus.

FIG. 26 is a flowchart showing a process of branch P in FIG. 25.

FIG. 27 is a flowchart showing the processing of the branch MA of FIG. 26.

28 is a flowchart showing a process of branch Q in FIG. 27. FIG.

[Explanation of symbols]

1 video encoder 2,3,8 audio encoder 4 video buffer 5, 6, 9 audio buffer 7,7A, 7B, 107,107A multiplexer 71, 71A, 171, 171A pack selection circuit 72, 72A, 172 multiplexing circuit 73 VB occupancy confirmation circuit 74 VMB simulation circuit 75,77,81 AB Occupancy Check Circuit 76,78,82 AMB simulation circuit 79 SCR generation circuit 80 Microcontroller 201,301 Demultiplex part 202 system clock generator 203 Timestamp comparison section 204 VMB 205 video decoder 206,212 selector 207,209 AMB 208, 210 audio decoder 211 Rearrangement buffer

   ─────────────────────────────────────────────────── ─── Continued front page    F-term (reference) 5C059 KK35 MA00 PP04 RB01 RB10                       RC03 RC04 RC32 UA34                 5C063 AB03 AB05 AC01 AC05 AC10                       CA11 DA01 DA05 DA07 DA13                 5K028 AA14 EE05 EE08 KK01 KK12                       SS05 SS15 SS24

Claims (10)

[Claims] 1. A video elementary stream (hereinafter referred to as a video ES) and first and first digital video signals and first and second digital audio signals are inputted and compression-encoded according to a predetermined compression-encoding protocol. Each of the two audio elementary streams (hereinafter, audio ES) is generated, and these video ES and the first ES are generated.
And a program stream multiplexing method for generating a program stream by multiplexing the second audio ES, a program system target decoder (hereinafter, referred to as a virtual decoder used in encoding for creating the program stream). P-STD) video ES
And a video main buffer (hereinafter, VMB) for holding the first and second audio ESs.
Managing the state of memory occupancy of each of the audio main buffers (hereinafter referred to as AMBs) of the video ES and the first and second audio ESs in the empty order of the VMB and the first and second AMBs. A program stream multiplexing method characterized by performing multiplexing ranking and multiplexing determination so as to multiplex, and controlling multiplexing. 2. The VMB and the first and second AMBs
The VMB simulation circuit and the first and second AMB simulation circuits that hold the state of the memory occupancy of the VMB simulation circuit and the first and second AMs.
Based on the content of the occupancy held in each of the B simulation circuits, the empty order, which is the order in which the main buffer in the P-STD becomes empty, is grasped, and the empty order is set to the video ES and the first and second audio ESs. Multiplex ranking is performed as the multiplexing order of
A first multiplexing determination is made as to whether multiplexing can be performed in the multiplexing order, multiplexing is performed when multiplexing is possible, and when the video ES and all of the first and second audio ESs cannot be multiplexed, A non-multiplexing factor that is a factor that cannot be multiplexed according to the multiplexing order is confirmed, and if the non-multiplexing factor is an empty buffer state in which there is no free space in the main buffer of the ES, a second multiplexing determination As the first and second presentation time stamps (hereinafter, PTS) or decoding time stamps (hereinafter, D) held in the VMB simulation circuit and the first and second AMB simulation circuits, respectively.
By calling the minimum value of TS) and setting that value to the system clock reference (hereinafter, SCR),
2. The program stream multiplexing method according to claim 1, wherein the multiplexing is performed without depending on the size of the main buffer by creating a vacancy in the main buffer having no vacancy and performing the multiplexing. 3. The multiplexing ordering compares the maximum values of the DTS and the first and second PTS called from the VMB simulation circuit and the first and second AMB simulation circuits, respectively. , Ordering such that the maximum value is preferentially multiplexed from the smallest value to the largest value, and the first multiplexing decision is made on the SCR for simulation of each of the VMB and the first and second AMBs. The program stream multiplexing method according to claim 2, wherein the determination is a multiplexing determination without adjustment, and the second determination is a multiplexing determination with adjustment of the SCR. 4. The VMB simulation circuit and each of the first and second AMB simulation circuits include:
The free space of the VMB or the first or second AMB (hereinafter, MB) corresponding to the P-STD is held, and the data amount of the output video or the first or second audio ES (hereinafter, output) is held. In response to the supply of the ES amount), the output ES amount is subtracted from the free space of the MB to obtain the usable capacity, and the AU amount, which is the number of access units, and the DTS or the first or second PTS. Each (hereinafter, DTS / PT
S) supply of these AU amounts and DTS / PTS
Are stored in association with each other, and the value of the first SCR output from the SCR generation circuit is compared with the value of the DTS / PTS. When the DTS / PTS value is smaller, the DTS / PTS is added to the free space.
When the DTS / PTS value is larger, the DTS / PTS and the AU amount associated with the DTS / PTS that are held are deleted, In response to the supply of the second SCR, the value of the second SCR and the value of the DTS / PTS are compared, and when the value of the DTS / PTS is the same as or smaller than the second SCR value, The AU amount related to DTS / PTS is output to the pack selection circuit, the AU amount related to DTS / PTS is added to the free space of the MB, and the addition value that is the value of the addition result and the MB The capacity is compared, the underflow of the MB is output when the added value is larger, and the DTS / PTS is sent in response to the supply of the maximum value confirmation request.
Of the DTS / PTS in response to the supply of the free space confirmation request, and the free space of the MB in response to the supply of the free space confirmation request.
3. The program stream multiplexing method according to claim 2, wherein the minimum value of is output. 5. A video elementary stream (hereinafter, referred to as a video ES) and first and first digital video signals and first and second digital audio signals are input and compression-encoded according to a predetermined compression-encoding protocol. Each of the two audio elementary streams (hereinafter, audio ES) is generated, and these video ES and the first ES are generated.
And a program stream multiplexing method for generating a program stream by multiplexing the second audio ES, a program system target decoder (hereinafter, referred to as a virtual decoder used in encoding for creating the program stream). P-STD) video ES
And a video main buffer (hereinafter, VMB) for holding the first and second audio ESs.
Time stamp (hereinafter, DTS) held by each of the VMB simulation circuit and the first and second AMB simulation circuits, which holds the state of memory occupancy of each audio main buffer (hereinafter, AMB)
Alternatively, the maximum value of each of the first and second presentation time stamps (hereinafter, PTS) is compared, and the video ES corresponding to the DTS or the first and second PTS having the smallest maximum value or the first or first 2 audio ES (hereinafter,
ES) as a multiplexing order 1, and a multiplexing ordering step of performing a multiplexing ordering in which the maximum value is preferentially multiplexed in the order of increasing the maximum value; and an ES having the multiplexing order 1 (hereinafter, the multiplexing order). Rank 1E
S), a first multiplexing determination and multiplexing process, which is a multiplexing determination without adjusting a system clock reference (hereinafter, SCR) for simulating the P-STD, and an ES (where the multiplexing order is 2) Below, multiplexing order 2ES)
The first multiplexing judgment and multiplexing process of the ES, and the ES having the multiplexing order of 3 (hereinafter, the multiplexing order 2ES)
Said first multiplexing decision and multiplexing process, said second multiplexing decision and multiplexing process which is a multiplexing decision with said SCR adjustment of said multiplexing order 1ES, and said multiplex order 2ES said A program stream multiplexing method, comprising: performing a second multiplexing determination and multiplexing process and the second multiplexing determination and multiplexing process of the multiplexing order 3ES. 6. The ES of the amount of data to be multiplexed in the first multiplexing judgment and the multiplexing process of the multiplexing order 1ES (hereinafter, a multiplexing target ES) is the multiplexing order 1ES.
To determine whether or not it exists in the buffer of the multiplexing target E
If S does not exist in the buffer of the multiplexing order 1ES, the first multiplexing decision of the multiplexing order 1ES is terminated, and the process proceeds to the first multiplexing decision of the multiplexing order 2ES. In the multiplexing determination step and the multiplexing determination step of the multiplexing order 1ES, if the multiplexing target ES exists in the buffer of the multiplexing order 1ES, the main buffer of the multiplexing order 1ES inputs the ES. Even if it does not overflow or underflow, it can be multiplexed by the multiplexing possibility judgment process of the multiplexing order 1ES that judges that multiplexing is possible and the multiplexing possibility judgment process of the multiplexing order 1ES. 6. The program stream multiplexing according to claim 5, further comprising: multiplexing processing of multiplexing order 1ES for performing multiplexing of the multiplexing order 1ES. Law. 7. A first vacant capacity which is a vacant capacity of the MB simulation circuit having the multiplexing order of 1ES is called in the multiplexing possibility determination process of the multiplexing order of 1ES.
And a second free space calculation step of calculating a second free space by subtracting the multiplexing target ES amount of the multiplexing order 1ES from the first free space, and performing the multiplexing. The access unit which outputs the SCR to be added at the time to the MB simulation circuit of the multiplexing order 1ES, and the MB simulation circuit of the multiplexing order 1ES is deleted from the MB of the multiplexing order 1ES by the SCR. A deletion AU amount calling step for calling an AU amount that is the number of AU numbers, a third free capacity calculating step for adding the AU amount to the second free capacity, and calculating a third free capacity; It is judged whether the free space is within the capacity of the main buffer with the multiplexing order of 1 ES, and if it is within the capacity of the main buffer, the multiplexing with the multiplexing order of 1 ES is performed. If the capacity of the main buffer is exceeded, the multiplexing decision 1 of the multiplexing order 1ES is ended, and the process proceeds to the first multiplexing decision of the multiplexing order 2ES.
And a step of determining the MB capacity of the ES, wherein the buffer of the multiplexing order 1ES outputs the ES data amount (hereinafter, ES
Output ES amount instructing step of outputting the instructed ES amount in response to an instruction of (amount), outputting an SCR to be added to the program stream PS, and the multiplexing target ES including the head of the AU,
If the multiplexing order 1 ES is an audio ES, the P
Outputs TS and the AU amount, and in the case of a video ES, outputs the P
The SCR and the ES parameter output step of outputting the TS, the DTS, the AU amount, and the picture type, and the SCR added to the program stream to the SC
An SCR generation step for simulation that is output to an R generation circuit, and the ES amount of the ES to be multiplexed is the multiplexing order 1E.
When the multiplexing target ES includes the head of the AU, the PTS and the ATS are output to the MB simulation circuit of S.
U amount is output, and in the case of video ES, the DTS and A
The ES includes the ES, the SCR, and the PTS, and a multiplexing target ES amount / parameter output step of outputting a U amount, and an SCR addition value adding step of adding and holding a predetermined SCR addition value to the SCR. 7. The program stream multiplexing method according to claim 6, wherein multiplexing is performed based on the DTS, the AU amount, and the picture type, and the program stream is output. 8. A video encoder for inputting a digital video signal, performing compression encoding according to a predetermined compression encoding protocol and outputting a video elementary stream (hereinafter referred to as video ES), and a video buffer holding the video ES. First and second inputting each of the first and second digital audio signals, performing compression encoding according to the compression encoding protocol, respectively, and outputting each of first and second audio elementary streams (hereinafter referred to as audio ES) A second audio encoder, first and second audio buffers for holding each of the first and second audio ES, and a video buffer and each of the first and second audio buffers Video E
A program stream multiplexer comprising: a multiplexer for multiplexing S and first and second audio ES and outputting the multiplexed program ES as a program stream, wherein the multiplexer includes the video ES in the video buffer.
VB occupancy confirmation circuit that confirms and holds the occupancy of the program, and a video main of a program system target decoder (hereinafter, P-STD) that is a virtual decoder used in encoding to create the program stream. A VM that simulates a buffer (hereinafter, VMB)
A B simulation circuit; first and second AB occupation amount confirmation circuits for confirming and retaining the respective occupation amounts of the first and second audio ES in the first and second audio buffers, respectively. First and second AMB simulation circuits for simulating each of the first and second audio main buffers (hereinafter, AMB) of the P-STD, and the video ES and the first and second audio ES. On the other hand, the VMB of the P-STD and the first and second AMs
Managing the state of the memory occupancy of each of the Bs, and the video ES in the order of emptying the VMB and the first and second AMBs.
And a pack selection circuit for performing multiplexing ranking and multiplexing determination to control the multiplexing so as to multiplex the first and second audio ESs, and the P-ST based on the data from the pack selection circuit.
System clock reference for simulation of the VMB of D and the first and second AMB (hereinafter, SC
A program stream multiplexing device comprising: an SCR generation circuit for generating R); and a multiplexing circuit for performing multiplexing under the control of the pack selection circuit. 9. The multiplexer implements the functions of the pack selection circuit, the SCR generation circuit, the VMB simulation circuit, the VB occupation amount confirmation circuit, and the first and second AB occupation amount confirmation circuits by software. 9. The controller according to claim 8, further comprising:
A program stream multiplexer as described. 10. The program stream input by the P-STD includes the video ES, the first and second audio ES, the SCR, a presentation time stamp (hereinafter, PTS), and a decoding time stamp (hereinafter, A demultiplexing unit for decomposing into a DTS, and a system time clock (ST) supplied with the SCR.
C), a system clock generating unit, the VMB holding the video ES, the AMB holding each of the first and second audio ES, and the video signal V by decoding the video ES. A video decoder for outputting, and first and second first and second audio signals for decoding the first and second audio ES and outputting the first and second audio signals, respectively.
Of the audio decoder, and the STC and the PTS or the DTS are compared, and the VMB or the first or second A according to the comparison result.
A time stamp comparison unit for instructing the MB to output the ES corresponding to the PTS or DTS to the video decoder or the first or second audio decoder, and for outputting control signals of the first and second selectors described later. A rearrangement buffer for rearranging the pictures of the video signal in order to change the arrangement order of the pictures; and a first and a second switch for supplying the video signal to the rearrangement buffer in response to the supply of the control signal. 9. The program stream multiplexer according to claim 8, further comprising: