JPH11155133A - Data multiplexer and its method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To properly multiplex audio data onto a transport stream and to send the multiplexed stream by deciding the amount of voice data to be multiplexed for each video frame period, based on a multiplex object data amount and its correction amount and multiplexing the data on a prescribed transport stream. SOLUTION: A CPU 424 stores the size of data received from data size interfaces IF 30a-30c, an Ethernet(R) IF 420, and a serial IF circuit 422 to a processing RAM 426. Then based on the size of the data to be stored, the order of multiplexing the elementary stream, the timing adjustment and the scheduling or the like are planned to control the multiplexing operation of pitch circuits 34, 36. Based on the multiplexing object data amount and the corrected multiplexing object data amount, a data amount of audio data which are multiplexed for each video frame period is decided, and the audio data with the decided data amount are multiplexed on a prescribed transport stream.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a method for multiplexing video data and audio data and
The present invention relates to a data multiplexing apparatus and method for generating a transport stream of a high quality moving picture coding system by a coding expert group.

[0002]

2. Description of the Related Art Video data and audio data are MPEG
Digital television broadcasts, which are compression-encoded by a system or the like, multiplexed into predetermined transmission packets, and distributed to viewers, are being put to practical use. In the MPEG system, when multiplexing audio / video data, it is necessary to consider the remaining buffer capacity of the decompression decoding device on the receiving side on the transmitting side. In other words, the transmitting side manages the data amount and multiplexes and transmits audio / video data so that neither overflow nor underflow occurs in the reception buffer of the decompression decoding device defined in the MPEG standard. Has become
Further, the capacity of the reception buffer is defined as 4 kB for audio data.

However, in practice, even if a failure such as an overflow or an underflow does not occur in the reception buffer, if the data transmission interval is too long, for example, when the time interval of one second or more is available in the audio data, the reception side will not operate. There is a problem that audio data may not be able to be normally reproduced.
In addition, there is a problem that processing for managing compression / encoding and multiplexing of audio / video data is complicated so that the reception buffer does not break down, and also requires a floating-point operation, so that processing time is long. Therefore, there is no need to strictly manage the remaining storage capacity of the reception buffer of the decompression decoding device on the reception side, and simple processing of only integer operations without using floating-point operations does not cause a failure in the reception buffer and properly. There is a demand for a data multiplexing device that can multiplex audio / video data into transmission packets and transmit the audio / video data so that the audio / video data can be reproduced.

[0004] As a data multiplexing device that meets such a demand, there is a data multiplexing device described in Japanese Patent Application No. 9-71834 to the present applicant. In this data multiplexing apparatus, the multiplexing of each data is managed in units of one video frame period, so that the audio / video data can be appropriately transmitted without losing the reception buffer without strictly managing the reception buffer. You can send.
For this purpose, the data amount of each data to be transmitted in one video frame period can be determined by simple arithmetic processing.

Here, the simplest method of determining the transmission data amount of audio data, which is performed in the data multiplexing apparatus, will be specifically described. For example, the audio stream data rate audio_
When rate_target is Akbps, the data amount of audio stream to be transmitted during 30 video frame periods audio_frame_30fr
ame is calculated as shown in Equation 1.

[0006]

[Equation 1] audio_rate_30frame = A × 1000 × 1001/1000/8 [Byte] (1)

[0007] In the MPEG system, the data rate of audio stream audio_rate_targ
“et” is defined as a fixed value as shown in FIG. 7, and is a multiple of 8 × 1000 bps. Therefore, the result of Expression 1 is always an integer. If the data amount of the transmission audio data is controlled in units of the 30-frame period, the control can be easily and appropriately performed.

[0008] For example, the data rate of audio stream audio_rate_target is 384.
In the case of kbps, the data amount audio_frame_30frame of the audio stream output by the audio encoder during 30 video frame periods
Is 48,048 bytes according to Equation 2.

[0009]

[Equation 2] audio_rate_30frame = 384 × 1000 × 1001/1000/8 = 48048 [Byte] (2)

[0010] Thus, the data amount of the audio stream to be multiplexed into the transport stream for each video frame period is 1601 bytes or 1602 bytes. Therefore, the amount of audio data multiplexed during the past 30 video frame periods is managed, and the amount of data is calculated as the data amount audio_f calculated by Expression 2.
If it is larger than frame_30frame (48048 bytes), the audio stream of 1601 bytes is multiplexed with the transport stream of the video frame period in the next one frame period.
The 602 bytes of the audio stream may be multiplexed with the transport stream of the video frame period.

The operation of the data multiplexing device for managing such transmission audio data will be specifically described with reference to FIG. First, the data rate audio_rate_target of the audio stream is set (step S300), and based on this, the data amount frame_a of the audio stream to be multiplexed into the transport stream for each video frame period.
The audio_rate_pre is calculated by Expression 3 (Step S302).

[0012]

[Equation 3] frame_audio_rate_pre = audio_rate_target / 8 × 1001/1000/30 (3) However, the fractional part of frame_audio_rate_pre is discarded.

[0013] Next, according to Equation 1, the data amount of audio stream audio_
Calculate frame_30frame (Step S3)
04). Then, the video frame signal input from the video encoder is received, and it is determined whether or not the video frame period has been started (step S306). Is the data amount of the audio stream multiplexed in the audio stream calculated in step S304.
It is determined whether the number is greater than frame_30frame (step S308).

If the number is large, the data volume fr of audio frames to be multiplexed in the video frame period
ame_audio_rate is set to the data amount frame_audio_ra calculated in step S302.
te_pre (step S310), and if not large, the data amount of the audio frame to be multiplexed in the video frame period frame_audio_
The rate is set to a value that is one byte larger than the data amount frame_audio_rate_pre calculated in step S302 (step S312). Then, the audio data having the obtained data amount is multiplexed into a transport stream such as a video stream (step S314).

[0015]

By the way, in recent years, MP
There is an increasing demand to transmit an audio stream that has not been compressed by EG audio but has been compressed, that is, remains LPCM. This means that when material is transmitted between broadcasting stations or in SNG (Satellite News Gathering), transmission of audio data of higher sound quality corresponding to video encoding by MPEG 4: 2: 2P @ MP of high image quality is performed. Is desired.

However, unlike the MPEG audio, the data rate of such LPCM-processed audio data is not specified to be an integral multiple of 8 × 1000 [bps]. Therefore, for example, 44.1k
There is a problem that audio data sampled at Hz cannot be appropriately multiplexed into a transport stream by the above-described method. For 16-bit stereo audio data with a sampling period of 44.1 kHz, the bit rate is 17 bits.
6400 (= 44100 × 2 × 16/8) [Byte /
sec], and the amount of data per 30 video frames as described above is obtained, 1765766.4 (= 176
400 × 1001/1000) [bytes]. That is, the result is not an integer value.

Therefore, an object of the present invention is to strictly manage the storage capacity of the receiving buffer of the receiving device even for audio data whose data rate is not an integral multiple of 8 × 1000 [bps]. And a data multiplexing apparatus and method that can control transmission data so as not to cause a failure in a reception buffer by simple processing of only integer arithmetic and can appropriately multiplex and transmit audio data to a transport stream. To provide.

[0018]

In order to solve the above-mentioned problem, the above-described fractional data of audio data in a predetermined number of video frame periods, for example, 30 video frames, is accumulated, and a data amount without a fraction is accumulated. When they become multiplexed together.

Therefore, the data multiplexing device of the present invention
Data amount calculating means for calculating the audio data amount of a corresponding number of video frame periods, such as 30 frame periods, of the corresponding video data based on the bit rate of the audio data, and a predetermined unit based on the audio data amount Multiplex target data amount detection means for obtaining a multiplex target data amount of audio data for the predetermined number of video frame periods so as to be an integral multiple of the data amount, and a difference between the calculated audio data amount and the multiplex target data amount. And a correction unit detecting means for detecting the number of the predetermined number of video frame periods in which the cumulative value is an integral multiple of the unit data amount, and for each of the detected numbers, the predetermined number of video frame periods Target value correcting means for correcting the multiplexed target data amount to a data amount obtained by adding the cumulative value which is an integral multiple of the unit data amount to the original multiplexed target data amount.

In such a data multiplexing device,
First, for the audio data to be multiplexed, the data amount calculation means calculates the audio data amount of the corresponding video data in a predetermined number of video frame periods, for example, 30 frame periods, based on the bit rate of the audio data. Based on the data amount, the multiplexing target data amount detecting means sets the multiplexing target data amount of the audio data in the predetermined number of video frame periods such that the data amount becomes an integral multiple of the predetermined unit data amount. Ask for. Next, in the correction unit detecting means, an accumulated value of a difference between the calculated audio data amount and the multiplexed target data amount is obtained, and the number of the predetermined number of video frame periods in which the accumulated value is an integral multiple of the unit data amount is calculated. To detect. Then, for each predetermined number of video frame periods of the detected number, the target value correcting means calculates the accumulated value, which is an integer multiple of the unit data amount, as the original multiplexed target data amount. Correct to the added data amount.
Thereafter, based on the multiplexed target data amount and the corrected multiplexed target data amount, the data amount of audio data to be multiplexed in each video frame period of the predetermined number of video frame periods is determined. Multiplexing is performed based on the data amount.

Further, according to the data multiplexing method of the present invention, based on a bit rate of audio data to be multiplexed, an audio data amount of a predetermined number of video frame periods of corresponding video data is calculated, and the calculated audio data is calculated. Based on the amount, the multiplexed target data amount of the audio data of the predetermined number of video frame periods that is an integral multiple of the predetermined unit data amount is obtained, and the accumulated value of the difference between the audio data amount and the multiplexed target data amount is A predetermined number of video frame periods, which is an integral multiple of a predetermined unit data amount, is detected, and for each of the detected number of the predetermined number of video frame periods, the multiplex target data amount of a predetermined number of video frame periods is detected. Is corrected to a data amount obtained by adding the accumulated value that is an integral multiple of the unit data amount to the original multiplexed target data amount. Then, based on the obtained multiplexing target data amount and the corrected multiplexing target data amount, the data amount of the audio data to be multiplexed in each video frame period of the predetermined number of video frame periods is calculated and calculated. The audio data having the data amount is multiplexed into a predetermined transport stream.

Preferably, the correction of the multiplex target data amount is
Counting the data amount of the audio data multiplexed during the predetermined number of the video frame periods before the video frame period, and multiplexing the video frame period based on the counted data amount of the audio data. It is determined whether or not to correct the target data amount. Specifically, the predetermined unit data amount is a byte unit, the predetermined number of video frame periods is 30 video frame periods, and the cumulative value of fractional data less than bytes of the audio data amount is a byte. The number of 30 video frame periods that are data in units of is 100. More specifically, the audio data is audio data subjected to pulse code modulation (PCM) at a sampling frequency of 44.1 kHz.

[0023]

DESCRIPTION OF THE PREFERRED EMBODIMENTS A data multiplexing apparatus according to one embodiment of the present invention will be described with reference to FIGS. First, the configuration of the data multiplexing device will be described. FIG. 1 is a block diagram illustrating a configuration of a data multiplexing device 2 according to the present embodiment. The data multiplexing device 2 includes a video encoder 20, an audio encoder 24, a subtitle encoder 28, a multiplexing system 12, and a control system 42. Having.

FIG. 2 shows the configuration of the video encoder 20. The video encoder 20 includes an addition circuit 202, a DCT
Circuit 204, quantization circuit 206, inverse quantization circuit 208,
Inverse DCT circuit 210, addition circuit 212, frame memory circuit 216, variable length coding circuit (VLC) 218, bit rate control circuit 220, variable length code buffer (VLC)
Buffer) 222, compresses and encodes video data input from an external device by the MPEG2 method, generates a video stream having a data amount substantially equal to the target data amount video_rate_target, and generates a data size IF 30a and a FIFO of the data multiplexing device 2. Memory 3
Output to 2a.

For the bit rate control circuit 220,
The target data amount video_rate_target is set by the CPU 424 of the data multiplexing device 2. The bit rate control circuit 220 includes a variable length encoding circuit 218
Controls the quantization circuit 206 based on the data amount of the video stream actually generated, so that the data amount after the compression encoding becomes equal to the set target data amount video_rate_target.

FIG. 3 shows the configuration of the audio encoder 24. The audio encoder 24 includes a subband analysis filter bank 240, a linear quantization circuit 242, a bit compression circuit 244, an FFT (fast fourier transfer) circuit 2
46, a psychoacoustic model 248, a dynamic bit allocation circuit 250, a scale factor selection information storage circuit 252, a scale factor extraction circuit 254, a side information encoding circuit 256, and a bit stream generation circuit 258.
Audio data input from an external device is compression-encoded by the MPEG2 method to generate an audio stream,
Data size IF 30b and F of data multiplexer 2
Output to the IFO memory 32b.

The subtitle encoder 28 encodes private data such as subtitle data input from an external device. Note that the subtitle encoder 28 may directly receive the encoded subtitle data from an external device. In that case, for example, E
The NIF circuit 420 receives encoded subtitle data via a LAN such as an Ethernet network, or the SIF 422 receives encoded subtitle data via a serial line.
The data is output to the RAM 430 via the bus.

In this case, the ENIF circuit 420 or the SIF 422 detects the header size written in the header attached to the encoded subtitle data, and outputs the detected header size to the CPU 424 via the CPU bus. In addition, the RAM 430 includes a CPU 424
According to the control of (1), the input encoded subcode data is output to the input terminal d of the switch circuit 34.

The multiplexing system 12 includes an input FIFO memory 32a, 32b, 32c, a first switch circuit 34, a second switch circuit 36, an output FIFO memory 38, and a SCSI (small computer system interface) interface circuit. (SCSI IF circuit) 40.

The FIFO memories 32a, 32b, and 32c buffer the video stream, the audio stream, and the subtitle stream input from the video encoder 20, the audio encoder 24, and the subtitle encoder 28, respectively, and input terminals a, b of the switch circuit 34. , C.

The first switch circuit 34 controls the input terminals a, b, c, and c according to the control of the control system 42 through the control signal.
d, any one of the elementary streams input to each of these input terminals is selected and multiplexed, and output to the input terminal b of the switch circuit 36. The switch circuit 34 does not select any of the input terminals a, b, c, and d when there is no elementary stream to be input to any of the input terminals, or when performing a stuffing process. Outputs predetermined blank data (continuous logical value 1 or 0).

The second switch circuit 36 selects one of the input terminals a and b according to the control of the control system 42 via the control signal, and outputs an elementary stream input from the switch circuit 34 to the input terminal b. Alternatively, a private data stream input from the processing RAM 426 to the input terminal a is selected and multiplexed, and output to the FIFO memory 38 and the SCSIIF circuit 40.

The FIFO memory 38 includes a switch circuit 36
Buffers the multiplexed data stream and outputs it as a transport stream to an external device such as a communication line (not shown).

The SCSIIF circuit 40 includes a switch circuit 3
The multiplexed data stream 6 is output to a recording device (not shown) such as a hard disk device (HDD) or a magneto-optical disk device (MOD) and recorded.

The control system 42 includes a data size counting interface circuit 30a, 30b, 30c, and a RAM 43.
0, Ethernet interface circuit (ENIF) 4
20, serial interface circuit (SIF) 42
2, a CPU 424, a processing RAM 426, a control data RAM 428, and a counter 432.

Data size IF 30a, 30b, 30c
Respectively, counts the data size of each frame of the video stream, audio stream, and subtitle stream input from the video encoder 20, the audio encoder 24, and the subtitle encoder 28, and outputs the data size to the CPU 424 via the CPU bus.

The counting of the data size is performed by a counter built in the data size IF 30a, 30b, 30c. Further, the data size can be counted by detecting the data size of each frame of each elementary stream output by the video encoder 20, the audio encoder 24, and the subtitle encoder 28 by the data size IFs 30a, 30b, 30c. .

The ENIF circuit 420 receives private data input via a LAN (not shown) such as Ethernet, and outputs it to the CPU 424 via the CPU bus.

The SIF circuit 422 accepts, for example, serial private data input from a computer and outputs it to the CPU 424. Note that examples of the private data include the above-described subtitle data, closed caption data, and user data.

CPU 424 is composed of, for example, a microprocessor, a ROM for storing programs and peripheral circuits thereof, and has a data size IF 30a, 30B.
b, 30c, ENIF circuit 420 and SIF circuit 42
2 is stored in the processing RAM 426, and the remaining storage capacity (remaining buffer capacity) of the FIFO memories 32a, 32b, 32c, etc. and the processing RAM 426 are stored.
Based on the data size stored in the multiplexing method, a multiplexing method such as a multiplexing order of elementary streams, multiplexing timing adjustment and scheduling is planned, and the multiplexing operation of the switch circuits 34 and 36 according to the planned multiplexing method. Is controlled via a CPU bus.

The control data RAM 428 has C
Control data related to the processing of the PU 424 is stored. The CPU 424 creates header data on the processing RAM 426 in accordance with the scheduling using the control data stored in the control data RAM 428, and Output to terminal a. Also,
Part of the header data is stored in ENIF circuit 240 or SI
In some cases, it is created based on private data (user data) input via the F circuit 422 and stored in the RAM 426.

The counter 432 has two counters, a counter count_frame for counting the number of multiplexed frames and a counter count_second for counting the number of multiplexed seconds from a predetermined reference time (the number of multiplexed 30 frames). And count-up and reset are performed by control signals from the CPU 424.

Next, in the data multiplexing apparatus 2,
A multiplexing method for multiplexing NTSC video streams will be described. First, the CPU 424 controls the video encoder 20 input from the data amount IF 30a.
Data rate of the video stream video_rat
e_org is a fixed value (frame_vid) shown in Expression 4.
eo_rate) to perform a multiplexing plan.

[0044]

[Equation 4] frame_video_rate (byte) = video_rate_org (bps) / 8 × 1001/1000/30 (4) where video_rate_org is the video encoder 20
Is the data rate of the video stream generated by fr
ame_video_rate rounds down decimal places.

Here, the data rate fr per video frame period of the video stream calculated by equation (4)
The value of ame_video_rate is discrete,
As long as there is no restriction, the data rate of the video stream can be any value, so that there is no particular problem. In addition, Expression 4 calculates the operation [1001 / (10
00 × 30)] so that the number of frames per second of the NTSC system, 29.97, can be handled by integer arithmetic.

The value of the target data amount video_rate_target set by the CPU 424 in the bit rate control circuit 220 of the video encoder 20 is given by the following equation (5).
Is calculated by

[0047]

[Equation 5] video_rate_target (bps) = frame_video_rate (byte) / 8/30/1000 × 1001 (5)

For example, the data rate of a video stream generated by the video encoder 20 is video_rate.
When _org is 4 Mbps, the target data amount vid
The value of eo_rate_target is 3.999920
1 Mbps. In this case, the difference between the two values is very small, and even if the CPU 424 sets the data rate video_rate_org as the target data amount video_rate_target in the bit rate control circuit 220, there is no practical problem. Does not occur.

Hereinafter, with further reference to FIG.
4 is a bit rate control circuit 22 of the video encoder 20
Target data amount video_rate_ta set to 0
A method for calculating rget will be described. FIG. 4 shows the target data amount v set in the bit rate control circuit 220 of the video encoder 20 of the data multiplexing apparatus 2 in the present embodiment.
It is a flowchart figure which shows the calculation method of video_rate_target.

As shown in FIG. 4, in step S240, the CPU 424 receives the data rate video_rate_org of the video stream generated by the video encoder 20 from the data amount IF 30a, and
Set to 428. In step S242, the CPU
424 is the data rate of the video stream per video frame period frame_video_
Calculate rate. Then, in step S244, the CPU 424 determines that the video encoder 2
A target data amount video_rate_target to be set in the 0 bit rate control circuit 220 is calculated.

Next, in the data multiplexing device 2,
A method of multiplexing LPCM audio data sampled at 44.1 kHz will be described. LPCM sampled at 44.1 kHz as described above
The audio data has a data amount per 30 video frames that is not a multiple of 8 × 1000. In the present embodiment, it is considered that the fraction of the data amount is corrected at predetermined intervals.

FIG. 5 shows that the bit rate, the bit rate per 30 video frames, and the rest under various conditions of 44.1 kHz audio data are set to 100.
Shows the multiplied value. The value of “remain” shown in FIG.
Indicates the number of bytes that need to be corrected every 0 seconds. In other words, if the correction is performed by this number, the control may be simply performed in units of 30 video frames in other periods as in the related art. Therefore, in the data multiplexing apparatus 2 of the present embodiment, the counter 432 for counting this 100 sec is provided, and when the counter 432 has a predetermined value, the number of bytes for the remain is added and transmitted. I have.

Hereinafter, a method of multiplexing audio data in the data multiplexing device 2 will be described in detail with reference to FIG. FIG. 6 shows that the data multiplexing device 2 calculates the data amount of the audio stream to be multiplexed into the transport stream for each video frame period, and further adds the correction data amount to the data amount for each predetermined period. It is a flowchart figure which shows the process which multiplexes audio data with a video stream etc.

First, in step S400, the CPU
424 indicates the data rate of the audio stream generated by the audio encoder 24,
And sets it in the RAM 428. Step S402
In, the CPU 424 controls the data amount frame_audio_rate of the audio stream to be multiplexed into the transport stream for each video frame period.
_Pre is calculated by Expression 6.

[0055]

[Equation 6] frame_audio_rate_pre = audio_rate_target / 8 × 1001/1000/30 (6) However, the fractional part of frame_audio_rate_pre is rounded down.

In step S404, CPU 424
Is the data amount of audio stream audio_frame per 30 normal video frame periods according to Equation 7.
e_30frame_pre is calculated.

[0057]

Audio_rate_30frame_pre = audio_rate_target / 8 × 1001/1000 (7)

Then, in step S406, the CP
U424 is a frame counter count of the counter 423
_Frame, second counter count_second and additional correction value
The add_remain is cleared, and the correction data amount remain every 100 seconds is obtained by Expression 8.

[0059]

## EQU8 ## remain = (audio_rate_target / 8 * 1001 * 100/1000)% 100 (8) However,% 100 indicates too much of 100.

Next, at step S408, the CPU 4
24 receives the video frame signal input from the video encoder 20, and determines whether or not the video frame period has started. If the video frame period starts, C
The PU 424 proceeds to the process of step S410, and if not started, stays in the process of step S408.

When the reception of the video frame is started, in step S410, the CPU 424
The value of the frame counter count_frame of 2 is incremented by one, and the value becomes 3 in step S412.
Check whether it has become 0. If 30 frames have been reached in step S412, the process proceeds to step S414, where the frame counter count_frame is reset and the second counter count_second is incremented by one.

Then, it is checked in step S416 whether or not the second counter count_second has reached 100, and if it has reached 100, step S418 is executed.
Resets its second counter, count_second,
The correction value “remain” obtained in step S406 is set as the additional correction value “add_remain”. Step S41
In step S420, if the second counter count_second has not reached 100, an additional correction value add is added in step S420.
_Remain is set to 0.

In step S412, the frame counter co
When unt_frame has not reached 30, and in steps S418 and S420, the additional correction value add
After _remain is set, in step S422, the data amount audio obtained in step S404
_Frame_30 Frame_pre is added with an additional correction value add_remain, and the data amount for comparison audio
Generate _frame_30frame.

Then, in step S424, the CP
U424 indicates that the data amount of the audio stream multiplexed during the past 30 video frame periods is the data amount audio_frame_3 calculated in the process of S422.
It is determined whether or not the number is greater than 0 frame.
The process proceeds to the process of S426, and if not, proceeds to the process of S428. In step S426, the CPU 424 determines
Data amount of audio frames to be multiplexed in the video frame period frame_audio_rate
Is the data amount fram calculated in the process of S402.
Let e_audio_rate_pre.

At step S428, CPU 424
Is the data amount of audio frames to be multiplexed in the video frame period frame_audio_rat
e is the data amount fra calculated in the process of S302.
Set to me_audio_rate_pre + 1.

Then, the CPU 424 determines in step S430
, The data multiplexer 2 such as the switch circuit 34
Of the video data frame_video_rat calculated by the processing shown in FIG.
e, the data amount frame_a calculated in step S426 or S428
The audio stream and the private data stream of the audio_rate are multiplexed.

As described above, according to the data multiplexing apparatus 2 of the present embodiment, even if the bit rate of the audio data is not a multiple of 8 × 1000 bps, the floating-point operation or the like can be performed. Multiplexing of a video stream, an audio stream, and the like synchronized with a video frame period can be performed by a simple method that does not use.

The present invention is not limited to the present embodiment, and various modifications are possible. For example, in the present embodiment, the data amount video_frame of the video stream to be multiplexed per video frame period.
e_rate was changed for each video frame cycle,
The data amount frame_audio_rate is changed every 30 video frame periods (predetermined control periods) by the data amount f
A method of changing to one of frame_audio_rate_pre and data amount frame_audio_rate_pre + 1 may be adopted.

The data rates of the video stream and the audio stream and the unit (byte) of the data amount are mere examples, and may be changed appropriately.

[0070]

As described above, according to the present invention,
Even for audio data whose data rate is not an integral multiple of 8 × 1000 [bps], there is no need to strictly manage the storage capacity of the receiving buffer of the receiving device,
Provided is a data multiplexing apparatus and method capable of performing control of transmission data so as not to cause a failure in a reception buffer by simple processing of only an integer operation, and appropriately multiplexing and transmitting audio data to a transport stream. be able to.

[Brief description of the drawings]

FIG. 1 is a block diagram illustrating a configuration of a data multiplexing device according to an embodiment of the present invention.

FIG. 2 is a block diagram showing a configuration of a video encoder shown in FIG.

FIG. 3 is a block diagram showing a configuration of the audio encoder shown in FIG.

4 is a flowchart illustrating a method of calculating a target data amount video_rate_target set in a bit rate control circuit of a video encoder of the data multiplexer illustrated in FIG. 2;

FIG. 5 is a diagram showing a bit rate under various conditions of audio data sampled at 44.1 kHz, the number of data between 30 video frames, a correction value, and the like.

FIG. 6 is a flowchart showing a process in which the data multiplexing apparatus shown in FIG. 2 calculates a data amount of an audio stream to be multiplexed on a transport stream for each video frame period, and multiplexes the data stream with a video stream or the like. .

FIG. 7 is a diagram showing a transfer code rate of audio data specified in the MPEG system.

FIG. 8 is a flowchart showing a process in which the data multiplexing device calculates a data amount of an audio stream to be multiplexed to a transport stream for each video frame period, and multiplexes the data amount with a video stream or the like.

[Explanation of symbols]

2 Data multiplexing device, 12 Multiplexing system, 20 Video encoder, 24 Audio encoder, 28 Subtitle encoder, 32a, 32b, 32c FIF
O memory, 34, 36: switch circuit, 38: FIFO
Memory, 40: SCSIIF circuit, 42: Control system, 30
a, 30b, 30c ... data amount IF, 420 ... ENIF
Circuit, 422 ... SIF circuit, 424 ... CPU, 426 ...
RAM for processing, 428 ... RAM for control data, 430 ...
RAM, 432 ... Counter

Claims (9)

[Claims] 1. A data amount calculating means for calculating an audio data amount of a predetermined number of video frame periods of corresponding video data based on a bit rate of audio data to be multiplexed, based on the calculated audio data amount. Multiplexing target data amount detecting means for obtaining a multiplexing target data amount of audio data of the predetermined number of video frame periods, which is an integral multiple of a predetermined unit data amount, and the calculated audio data amount and the obtained multiplexing. Correction unit detecting means for detecting the number of the predetermined number of video frame periods in which the cumulative value of the difference between the normalized target data amounts is an integral multiple of the predetermined unit data amount; and the detected predetermined number of video frame periods For each number, the multiplex target data amount for the predetermined number of video frame periods is calculated by adding the cumulative value, which is an integral multiple of the predetermined unit data amount, to the multiplex target data amount. A target value correcting unit for correcting the obtained multiplexed target data amount and the corrected multiplexed target data amount, and multiplexing in each video frame period of the predetermined number of video frame periods. A data multiplexing apparatus comprising: a multiplexed data amount determining unit that determines a data amount of the audio data; and a multiplexing unit that multiplexes the determined amount of audio data into a predetermined transport stream. 2. An audio data counting means for counting a data amount of the audio data multiplexed during the predetermined number of the video frame periods before the video frame period; 2. The data multiplexing device according to claim 1, further comprising: a correction control unit that determines whether to correct a multiplexing target data amount in the video frame period based on a data amount of the audio data. 3. The predetermined unit data amount is 1 byte, and the multiplexing target data amount detecting means calculates the multiplexing target data amount in bytes based on the calculated audio data amount. The correction unit detecting means detects the number of the predetermined number of video frame periods in which the cumulative value of fractional data less than bytes of the calculated audio data amount is the data amount in bytes. Item 2. The data multiplexing device according to Item 1. 4. The multiplexing target data amount detecting means, wherein the data amount calculating means calculates the audio data amount of the corresponding video data in 30 video frame periods based on the bit rate of the audio data to be multiplexed. Calculates a multiplexing target data amount of the audio data in the 30 video frame periods in units of bytes based on the calculated audio data amount. The correction unit detecting means calculates the calculated audio data amount and the calculated The number of 30 video frame periods in which the cumulative value of the difference between the multiplexed target data amounts becomes the data amount in units of bytes is assumed to be 100, and the target value correction means sets the 30 video frame periods 10
For every 0 pieces, the multiplexing target data amount for the 30 video frame periods is added to the multiplexing target data amount, and the accumulation of the difference between the audio data amount and the multiplexing target data amount in the byte unit. The multiplexed data amount determining means, based on the obtained multiplexed target data amount and the corrected multiplexed target data amount, for each video frame of the 30 video frame period. The data multiplexing apparatus according to claim 3, wherein a data amount of the audio data to be multiplexed during a period is determined, and the multiplexing unit multiplexes the determined amount of audio data into a predetermined transport stream. . 5. The data multiplexing apparatus according to claim 4, wherein said audio data is audio data subjected to pulse code modulation (PCM) at a sampling frequency of 44.1 kHz. 6. An audio data amount for a predetermined number of video frame periods of corresponding video data is calculated based on a bit rate of audio data to be multiplexed, and a predetermined unit is calculated based on the calculated audio data amount. A multiplexing target data amount of the audio data of the predetermined number of video frame periods that is an integral multiple of the data amount is obtained, and a cumulative value of a difference between the calculated audio data amount and the obtained multiplexing target data amount is the predetermined value. Detecting the number of the predetermined number of video frame periods that is an integral multiple of the unit data amount of the multiplexed target data of the predetermined number of video frame periods for each of the detected number of the predetermined number of video frame periods. The amount is corrected to a data amount obtained by adding the cumulative value that is an integral multiple of the predetermined unit data amount to the multiplexed target data amount, and the obtained multiplexed target data amount and the corrected Calculating a data amount of the audio data to be multiplexed in each video frame period of the predetermined number of video frame periods on the basis of the multiplexed target data amount, and converting the audio data of the calculated data amount into a predetermined transformer. Data multiplexing method for multiplexing to port stream. 7. The correction of the multiplex target data amount includes counting a data amount of the audio data multiplexed during the predetermined number of the video frame periods before the video frame period, and 7. The data multiplexing method according to claim 6, wherein whether to correct the multiplexing target data amount in the video frame period is determined based on the data amount of the data. 8. The predetermined unit data amount is 1 byte, the predetermined number of video frame periods is 30 video frame periods, and the cumulative value of fractional data less than bytes of the audio data amount is bytes. 8. The data multiplexing method according to claim 7, wherein the number of 30 video frame periods which are data in units of 100 is 100. 9. The data multiplexing method according to claim 8, wherein said audio data is audio data subjected to pulse code modulation (PCM) at a sampling frequency of 44.1 kHz.