JPH1051769A - Synchronization data generator and its method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To generate an accurate PCR(program clock reference) so that no arithmetic error is accumulated in the lapse of time. SOLUTION: The arithmetic operation of [packet frame = ((SIZE FRAME + sizeremain)/188)*188] is conducted in the S308 and a numerical value frame denoting an output enable data amount in a video frame is calculated. In a S310, data are multiplexed onto a transport stream by a byte number denoted by the numerical value frame. In a S314, an arithmetic operation [size remain = packet frame - SIZE FRAME + sizeremain] is conducted to calculate a numeral size remain denoting the data amount left without multiplexing. In a S316, the arithmetic operation of [PCR FRAME = PCR FRAME + PCRbase ] is conducted and the numeral PCR FRAME denoting an offset of the PCR is calculated. In the S318, based on each numeral calculated up to now, the final PCR is calculated.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a synchronous data generating apparatus and method for generating an accurate program clock reference (PCR) from compressed audio / video data of the MPEG system, multiplexing and outputting the same, and a method therefor. .

[0002]

2. Description of the Related Art MPEG (moving pi) is an efficient coding method for video data.
cture coding experts group) method has been put to practical use. Video data (compressed video data) compressed and encoded by the MPEG system is multiplexed and transmitted to a 188-byte fixed-length TS packet composed of a 4-byte transport stream (TS) header and a 184-byte TS payload. Is done. The TS header includes a system time clock (STC) on the transmitting side and an STC on the receiving side.
A PCR used for synchronization with C is added.

The i-th PCR [PCR (i)] uses a system time clock [STC; 27 MHz (MPEG
2), data PCR _base (i) and data PCR _ext (i) defined with time accuracy (1/27 MHz, 1/90 kHz) determined by 90 kHz (MPEG1)].
Divided into two parts. In the MPEG2 system, PC
R (i), data PCR _base (i) and data PCR
_ext (i) is defined as shown in Equations 1 to 3 below.

The data PCR for each video frame is
The increase amount of _base (i) is 3,003 [= ((27,000,000 / 300) / 30) * 1.001) when the screen has a 525/60 configuration.
In the case of the 625/50 configuration, 3,600 [= ((27,000,000 /
300) / 25], which is a fixed value in each case. Also,
The PCR is used as data set in a clock oscillation circuit of a phase-locked loop (PLL) control on the receiving side, and has a transmission interval of 1 to guarantee a stable operation of the PLL.
It is defined that the data is transmitted from the transmission side to the reception side in less than 00 milliseconds.

[0005]

[Equation 1] PCR _base (i) = [(27,000,000 * t (i) / 300)]% 2 ²³ (1) where * indicates multiplication,% indicates an operator for calculating a remainder, and t ( i) indicates the time when the data arrives at the receiving side.

[0006]

[ _Equation 2] PCR _ext (i) = [(27,000,000 * t (i) / 1)]% 300 (2)

[0007]

[Equation 3] PCR (i) = PCR _base (i) * 300 + PCR _ext (i) (3)

The latest PCR (i ′) is a PCR
(I ′) Based on the previous PCR (i ″), it is defined as shown in Equation 4 below.

[0009]

## EQU00004 ## PCR (i ') = [(i'-i ") * 27,000,000] / transport-rate + PCR (i") (4) where transport-rate is the data rate of the transport stream, i, where i is an integer i "<i <i ', is
Indicates the index number (number of bytes from the beginning) of an arbitrary byte of the TS packet in the transport stream, and i ′ is the last bit of the _latest data PCR _base (i) applied to the program (program) to be decoded. And i "indicates the index number of the byte containing the last bit of data PCR _base (i) immediately following the data applied to the program to be decoded. PCR (i")
Indicates the time of encoding in the PCR _base (i) field (the field indicates the location where the data is located) and the PCR _ext (i) field. The value of STC is set to the value of PCR.

[0010] Normally, the PCR is performed by adding a numerical value [[(i'-i ") * 27,000,000] / transport-rate to the initial value of the PCR.
(Equation 4)] is sequentially calculated by software by integer addition. However, the error between the PCR value calculated by the integer arithmetic processing and the actual PCR value accumulates and expands over time. PCR is
Since the STC on the receiving side is used to synchronize with the STC on the transmitting side, if the calculation error of the PCR value increases, the STC between the receiving side and the transmitting side becomes out of synchronization. In addition, it is impossible to predict whether the calculated error of the PCR will increase or decrease the frequency of the STC on the receiving side from the frequency of the STC on the transmitting side.

In order to reduce the calculation error of the calculated PCR, the software may be changed so that the PCR is calculated by real number calculation. However, the calculation of PCR by real number operation requires much more time than by integer operation. Further, in order to reduce the error between the calculated PCR and the actual reproduction time, a method of calculating the PCR by real number calculation using dedicated hardware is conceivable. However, according to this method, the size and cost of the reproducing apparatus increase.

The present invention has been made in view of the above-described problems of the prior art, and even if a PCR is generated by integer arithmetic, a calculation error of the generated PCR does not accumulate and an accurate PCR is generated. It is an object of the present invention to provide a synchronous data generating device and a synchronous data generating method. Further, the present invention provides a synchronous data generating apparatus and a synchronous data generating apparatus capable of generating a PCR accurately and at high speed without dedicated hardware for performing a real number operation by software integer operation. Aim.

[0013]

SUMMARY OF THE INVENTION A synchronous data generating apparatus according to the present invention has been made in view of the above-mentioned problems of the prior art, and has been described with respect to audio data and video data or any one of them (audio / video data). ) Is compressed in synchronism with a predetermined clock signal by the MPEG method,
Compression means for generating video data, for each predetermined transmission cycle, synchronization data generation means for generating synchronization data used for reproduction of the clock signal, and at least the generated compressed audio / video data and the synchronization data. And multiplexing means for accommodating the remaining data in a predetermined transmission packet, wherein the synchronization data generating means calculates, for each transmission cycle, a remaining data amount remaining without being accommodated in the transmission packet. Calculating means, an offset value calculating means for calculating an offset value of the synchronization data for each transmission cycle, and calculating the synchronization data for each transmission cycle based on the calculated remaining data amount and the offset value. A synchronous data calculating unit.

The present invention generates synchronous data (PCR) used for synchronizing a system time clock between a receiving side and a transmitting side when transmitting compressed audio / video data of the MPEG system or the like. . The compression means compresses and encodes the audio / video data in accordance with a predetermined clock signal (system time clock; STC) according to the MPEG system, and generates compressed audio / video data.

[0015] The synchronous data generating means generates synchronous data (PCR) used on the receiving side to synchronize the STC on the transmitting side and the STC on the receiving side. In the synchronous data generating means, the remaining data amount calculating means subtracts the number of TS packets to be transmitted in a certain transmission cycle, that is, the amount of transmittable data, from the data quantity that can be transmitted in a certain transmission cycle, and stores the number in a transmission packet. The data amount of remaining data (remaining data amount; numerical value size remain) is calculated.

The offset value calculating means calculates, for each transmission cycle, an offset value (numerical value PCR FRAME) of a previous transmission cycle.
When the video data has a 625/50 configuration, a fixed value of 3,600 is added, and when the video data has a 525/60 configuration, a fixed value of 3,003 is added. Update the offset value used for The synchronization data calculating means adds a predetermined arithmetic processing (size) to the offset value (numeric PCR FRAME) for each transmission cycle.
The remaining data amount (numerical value size remain) subjected to remain * 27,000,000 / RATE) is added to calculate the synchronous data (PCR). Note that the arithmetic processing for the remaining data amount includes division, and the arithmetic result includes an error. However, since an offset value having a correct value is calculated for each transmission cycle, an arithmetic error is not accumulated.

Further, the synchronous data generating method according to the present invention is characterized in that the audio data and the video data or any one of them (audio / video data) is compressed in synchronization with a predetermined clock signal by the MPEG system, and Generating video data, for each predetermined transmission cycle, generating synchronization data used for reproducing the clock signal, at least
A synchronous data generation method for multiplexing the generated compressed audio / video data and synchronous data into a predetermined transmission packet, wherein a remaining data amount remaining without being accommodated in the predetermined transmission packet is calculated for each transmission cycle. Then, an offset value of the synchronization data is calculated for each transmission cycle, and the synchronization data is calculated for each transmission cycle based on the calculated remaining data amount and the offset value.

[0018]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS First Embodiment Hereinafter, a first embodiment of the present invention will be described. FIG. 1 is a diagram showing a configuration of an audio / video data generation device 1 according to the present invention in the first embodiment. As shown in FIG. 1, the audio / video data generation device 1 includes a video encoder (video
encoder) 20, Audio encoder
24 and subtitle encoder
28 and a multiplexer 10. The multiplexer 10 includes a buffer memory 100.
And transport stream generator (transport s
stream maker) 102.

With these components, the audio / video data generating apparatus 1 uses the video encoder 20, audio encoder 24 and subtitle encoder 28 to encode a movie or music promotion video stream (video stream) and audio stream ( An audio stream and a subtitle stream are multiplexed to generate a transport stream.

The video encoder 20 operates in synchronization with a system time clock (STC), and compresses digital uncompressed video data input from an external device (not shown) such as an editing device by, for example, the MPEG2 system. The multiplexed data is output to the buffer memory 100 of the multiplexer 10 as a video stream. The audio encoder 24 operates in synchronization with the STC and converts digital uncompressed audio data input from an external device into, for example, MPEG2
In accordance with a method or the like, compression encoding is performed every 1,152 sampling periods (1 audio frame) and output to the buffer memory 100 of the multiplexer 10 as an audio stream.

The subtitle encoder 28 encodes the subtitle data input from the external device for generating the subtitle data by a linear quantization process and a fixed length coding process, and sends the encoded subtitle data to the buffer memory 100 of the multiplexer 10 as a subtitle stream. Output.

The buffer memory 100 of the multiplexer 10
Is a video encoder 20, an audio encoder 24
And a video stream, an audio stream, and a subtitle stream respectively input from the subtitle encoder 28 and output to the transport stream generating apparatus 102. The transport stream generation device 102 is composed of, for example, a high-speed computer using a microprocessor (CPU) and the like, and the video stream, the audio stream, and the subtitle stream buffered in the buffer memory 100 (each is also referred to as an elementary stream). Is read out at a predetermined timing, scheduling is performed, a header or the like is added in accordance with the transport format of the transport stream, and a transport stream is generated and output.

According to the audio / video data generating apparatus 1 shown in the first embodiment, audio data such as a movie, video data and subtitle data are compression-encoded by the MPEG system or the like, and these data obtained by compression-encoding are obtained. One transport stream can be generated by multiplexing and output. Further, the audio / video data generation device 1 shown in the first embodiment, when performing digital television broadcasting of audio data and video data such as a movie or music promotion, for example, converts audio data of music into a so-called surround ( surround), audio data is prepared in several languages, and audio data, video data and additional data that have been processed such as adding subtitles in several languages to video data are multiplexed and transported. Streams can be created.

Second Embodiment Hereinafter, a second embodiment of the present invention will be described. In the audio / video data generation device 1 shown in the first embodiment, the buffer memory 100 temporarily stores all the elementary streams in the buffer memory 100, and further, the CPU of the transport stream generation device 102 performs scheduling and header A transport stream is generated by performing processing such as adding (header). As described above, the method of generating a transport stream by directly processing the words (bytes) of the elementary stream via the data bus by the CPU performing the processing is performed by the transport stream. Generator 1
High-speed processing cannot be performed due to a bus neck or the like occurring in the data bus No. 02, and cannot be applied to generation of a high-speed transport stream.

The audio / video data generation device 2 according to the present invention described in the second embodiment is a device for solving the problem of the audio / video data generation device 1, and is a device for multiplexing elementary streams. By separating a control system that performs scheduling and a multiplexing system that directly multiplexes elementary streams, a high-speed transport stream can be generated.

FIG. 2 is a diagram showing the configuration of the audio / video data generating device 2 according to the present invention in the second embodiment. FIG. 3 is a diagram showing a configuration of the video encoder 20 shown in FIG. FIG. 4 is a diagram showing a configuration of the audio encoder 24 shown in FIG. As shown in FIG.
The audio / video data generation device 2 includes a video encoder 2
0, an audio encoder 24, a subtitle encoder 28, a multiplexing system 12, and a control system 42. The multiplexing system 12 includes an input FIFO memory 32a,
32b, 32c, a first switch circuit (switch) 34, a second switch circuit 36, a FIFO memory 38 for output, and a small computer system interface (SCSI) interface circuit (SCSI IF circuit) 40.

The control system 42 includes a data size counting interface circuit (data size IF) 30a, 30b,
30c, Ethernet interface circuit (ether-ne
t interface; ENIF circuit) 420, serial interface circuit (SIF circuit) 42
2, a CPU 424, a processing RAM 426, and a control data RAM 428 are mutually connected by a CPU bus. Video encoder 20, as shown in FIG.
The addition circuit 202, the DCT circuit 204, and the quantization circuit (quant
ize) 206, inverse quantizer (I quantize) 208, inverse DC
T circuit (IDCT) 210, adder circuit 212, frame memory circuit 216, variable length encoding circuit (VLC circuit) 21
8, bit rate control circuit (bit rate control) 220,
It is composed of a variable length code buffer (VLC buffer) 222, and converts video data VideoIN input from an external device into M
The video stream (M
PEG Video stream) and the data size IF 30a and the FIFO memory 3 of the audio / video data generation device 2.
Output to 2a.

The audio encoder 24 includes a sub-band analysis filter bank (SAFB) 2 as shown in FIG.
40, a linear quantization circuit 242, a bit compression circuit 244,
It comprises an FFT (fast fourier transfer) circuit 246, a psychological auditory model 248, a dynamic bit allocation circuit 250, a scale factor selection information storage circuit 252, a scale factor extraction circuit 254, a side information encoding circuit 256, and a bit stream generation circuit 258. The audio data (Audio input) input from the external device is
Audio stream compressed and encoded by EG2 method
(MPEG Audio stream) is generated and output to the data size IF 30b and the FIFO memory 32b of the audio / video data generation device 2. Subtitle encoder 28
Encodes additional data such as subtitle data input from an external device, as in the first embodiment.

In the multiplexing system 12 (FIG. 1), the FIFO
The memories 32a, 32b, and 32c buffer the video stream, the audio stream, and the subtitle stream input from the video encoder 20, the audio encoder 24, and the subtitle encoder 28, respectively, and input the input terminals a, b, and c of the switch circuit 34. Output to

The switch circuit 34 has a control signal (control).
According to the control of the multiplexing system 12 through the input terminals a,
One of b and c is selected, and one of the elementary streams input to each of these input terminals is selected and multiplexed, and output to the input terminal b of the switch circuit 36. Note that, when there is no elementary stream input to any of the input terminals, the switch circuit 34
Alternatively, when stuffing processing is performed, predetermined blank data (consecutive logical values 1 or 0) is output without selecting any of the input terminals a, b, and c.

The switch circuit 36 controls a control signal (control).
Input terminals a and b according to the control of the multiplexing system 12 via
And multiplexing by selecting one of the elementary streams input from the switch circuit 34 to the input terminal a or the additional data stream (header information) input from the processing RAM 426 to the input terminal b. And outputs it to the FIFO memory 38 and the SCSIIF circuit 40. The FIFO memory 38 buffers the data stream multiplexed by the switch circuit 36 and outputs it as a transport stream to an external device (not shown) such as a communication line. The SCSIIF circuit 40 outputs the data stream multiplexed by the switch circuit 36 to a recording device (not shown) of a hard disk device (HDD) or a magneto-optical disk device (MOD) and records it.

In the control system 42, the data size IF3
0a, 30b and 30c are video encoders 2 respectively.
0, the data size (size) of the video stream, audio stream and subtitle stream input from the audio encoder 24 and the subtitle encoder 28 are counted, and the CPU 424 is counted via the CPU bus.
Output to The ENIF circuit 420 accepts subtitle additional data (private data) input via a LAN (not shown) such as Ethernet, and outputs the data to the CPU 424 via the CPU bus. SIF circuit 4
22 receives, for example, additional data in a serial format input from a computer and outputs it to the CPU 424.

The CPU 424 includes, for example, a microprocessor and an RO for storing programs.
M and these peripheral circuits, and stores the data size input from the data size IFs 30a, 30b, 30c in the processing RAM 426, and stores the data size in the processing RAM 426.
Based on the data size stored in the multiplexing method, a multiplexing method such as a multiplexing order of elementary streams, multiplexing timing adjustment and scheduling is planned, and the multiplexing operation of the switch circuits 34 and 36 according to the planned multiplexing method. Is controlled via a CPU bus.

The CPU 424 controls the ENIF circuit 42
0 and the additional data input from the SIF circuit 422 are stored in the processing RAM 426, the additional data stored in the processing RAM 426 is subjected to a predetermined process, and the additional data multiplexed into the header portion of the transport stream. Generate a stream (header information) and process RAM
The signal is output to the input terminal a of the switch circuit 36 via 426. The control data RAM 428 is a CPU 424
The control data for the processing of (1) is stored.

The operation of the audio / video data generating device 2 according to the second embodiment will be described below with reference to FIG.
The video encoder 20, the audio encoder 24, and the subtitle encoder 28 encode input video data, audio data, and subtitle data, respectively. Data size IF 30a, 30b, 30
c counts the data size of the video stream, the audio stream, and the subtitle stream, respectively, and outputs them to the CPU 424.

FIG. 5 is a flowchart showing the processing of the CPU 424 of the audio / video data generation device 2 (FIG. 2) in the second embodiment. As shown in FIG. 5, in step 102 (S102), upon receiving the data size from the data size IFs 30a, 30b, 30c, the CPU 424 stores the received data size in the processing RAM 426. In step 104 (S104),
The CPU 424 plans the multiplexing method described above.

In step 106 (S106), C
PU 424 determines whether the planned multiplexing method indicates to output a video stream at that time.
If the multiplexing method indicates that the video stream is to be output at that time, the CPU 424 proceeds to the process of S108, and if it does not indicate that the video stream is to be output at that time, the process proceeds to S110. Step 10
8 (S108), the CPU 424 controls the switch circuit 34 to select a video stream, and controls the switch circuit 36 to select a data stream from the switch circuit 34 to multiplex the video stream into a transport stream. .

In step 110 (S110), C
PU 424 determines whether the planned multiplexing method indicates to output an audio stream at that time. If the multiplexing method indicates that the audio stream is to be output at that time, the CPU 424 proceeds to the process of S112. If the multiplexing method does not indicate that the audio stream is to be output at that time, the process proceeds to S114. Step 112
In (S112), the CPU 424 controls the switch circuit 34 to select an audio stream, controls the switch circuit 36 to select a data stream from the switch circuit 34, and multiplexes the audio stream into the transport stream.

In step 114 (S114), C
The PU 424 controls the switch circuit 34 to select a subtitle stream, controls the switch circuit 36 to select a data stream from the switch circuit 34, and multiplexes the subtitle stream into a transport stream. In step 116 (S116),
The CPU 424 determines whether or not the planned multiplexing method indicates that header information is added at that time. If the multiplexing method indicates that the header information is to be output at that time, the CPU 424 proceeds to the process of S118, and if the multiplexing method does not indicate that the header information is to be output at that time,
The process proceeds to S120.

In step 118 (S118), C
The PU 424 pre-enables EN via the processing RAM 426.
An additional data stream (header information) generated by processing the additional data input from the IF circuit 420 or the like is output to the switch circuit 36, and further, the switch circuit 36
Is controlled to select an additional data stream and multiplex the additional data stream (header information) with the transport stream. In step 120 (S120), the CPU 424 controls the switch circuit 36 to select one of the elementary streams input from the switch circuit 34, and multiplexes the elementary stream to the transport stream.

In step 122 (S122), C
The PU 424 determines whether the input of the elementary stream to be multiplexed has been completed. When the input of the elementary stream to be multiplexed is completed, the process proceeds to S124, the generation of the transport stream is completed, and when not completed, the process proceeds to S102.

The FIFO memory 38 is a FIFO memory 3
The transport stream output from the buffer 8 is buffered and transmitted to a communication line or the like. SCSII
The F circuit 40 sends the transport stream output from the FIFO memory 38 to a hard disk device or the like, and records the transport stream.

As described above, the voice / voice according to the present invention
In the video data generation device 2, since an elementary stream having a very large amount of data is not transmitted on the CPU bus, a bus bottleneck occurs on the CPU bus and the high-speed elementary stream is multiplexed with the transport stream. Is not disturbed. Therefore, the audio / video data generation device 2 is suitable for generating a high-speed transport stream.

Further, since a multiplexing method is planned each time a transport stream is generated by software processing of the CPU 424, the multiplexing method is added, for example, as compared with a case where an elementary stream is multiplexed into a transport stream by hardware. The feature is that the type and number of additional data streams (header information) can be arbitrarily changed. Therefore, it is easy to change the type and the number of the additional data streams.

The audio / video data generating device 2 is modified to increase or decrease the number of elementary streams to be multiplexed in the transport stream, or to increase or decrease the number of ENIF circuits 420 and SIF circuits 422. It is possible. In addition to the subtitle stream shown in the second embodiment, for example,
The audio / video data generation device 2 can adopt various configurations, such as multiplexing a data stream related to editing processing and processing during broadcasting into a transport stream.

Third Embodiment Hereinafter, a third embodiment of the present invention will be described. FIG.
Configuration of transport stream, PES, pack and PS used for transmission of audio stream and video stream compressed and encoded by PEG method (ITU
-T Recommendation H.222.0 | ISO / IEC 13818-1). FIG. 7 is a diagram showing a detailed configuration of the TS header included in the transport stream shown in FIG.

For example, the audio / video data generating devices 1 and 2 shown in the first and second embodiments, respectively.
As shown in FIG. 6, the transport stream generated by (FIGS. 1 and 2) includes a clock reference PCR in an optional field of an adaptation field.
A variable length packetized elementary stream (PES) composed of a TS header section (FIG. 7) including (program clock reference) and a TS packet including a TS payload section.
Is divided into TS packets and accommodated. Further, a plurality of PESs form a pack with a pack header added thereto, and the plurality of packs have a program stream (PS: program stream) added with a program encode.
Is configured.

FIG. 8 is a diagram showing the configuration of the PES header section of the PES shown in FIG. FIGS. 9A and 9B show that the sampling frequencies are 48 kHz and 44.1 k, respectively.
FIG. 8 is a diagram illustrating a relationship between an audio frame, an audio stream data amount (SIZE), and an increase amount of a PTS in the case of Hz. As shown in FIG. 8, the PES header part of the PES has a presentation time stamp (PTS; pre).
sentation time stamp). This PTS is PES
Indicates the display time of the audio stream and the video stream contained in the PTS (hereinafter, the PTS indicating the display time of the audio stream and the video stream is represented by
Audio PTS and video PTS, respectively). When the decompression decoding device decompresses and decodes the audio stream and the video stream, the audio PTS
The video PTS is used for synchronizing audio data and video data.

On the other hand, in the audio / video data generating devices 1 and 2, the audio encoder 24 operates in synchronization with the system time clock, and converts the input digital uncompressed audio data into its 1, 152 The compression encoding is performed so that the data rate after the compression encoding becomes substantially constant with a sampling period (one audio frame) as a period. When the sampling frequency of the uncompressed audio data input to the audio encoder 24 is 48 kHz (data rate = 384 kbps), FIG.
As shown in (A), the data amount of the audio stream generated for each audio frame is 1,152 bytes, and the increase in the value of the PTS (audio PTS) of the PES containing the audio stream is 2,160.
(Equation 5 shown below). Therefore, the value of the audio PTS for each audio frame can be calculated by an integer operation.

[0050]

## EQU00005 ## 1152.times.90 (kHz) / 48 (kHz) = 2160 (5) where 90 (kHz) is the reference frequency of the PTS generation process and 48 (kHz) is the sampling frequency of the uncompressed audio data. .

On the other hand, the sampling frequency of the uncompressed audio data input to the audio encoder 24 is 44.1.
In the case of kHz (data rate = 384 kbps), the data amount of the audio stream generated for each audio frame is 12 as shown in FIG.
53.8776 bytes, the amount of increase in the value of the audio PTS is 2,351.0204 (formula 6 shown below). Therefore, in this case, the value of the audio PTS for each audio frame must be calculated by a real number operation.

[0052]

[Formula 6] 1152 × 90 (kHz) /44.1 (kHz) = 2351.0204 (6) where 44.1 (kHz) is the sampling frequency of the uncompressed audio data.

However, even when the sampling frequency of the uncompressed audio data is 44.1 kHz, it is desirable to calculate the audio PTS by an integer operation due to the hardware or software configuration for calculating the audio PTS. On the other hand, when the sampling frequency of the uncompressed audio data is 44.1 kHz, the increment of each audio frame of the audio PTS is set to the integer value 2,
If it is approximated to 351 or 2,352 and calculated by an integer operation, unless it is corrected by any method, errors accumulate and video data and audio data cannot be synchronized at the time of decompression decoding. From this point of view, in the third embodiment, the audio PTS can be calculated by the integer operation, and the error of the audio PTS calculated by the integer operation is not accumulated for a long time. A modification of the operation of the control system 42 of the video data generation device 2 (FIG. 2 shown in the second embodiment) will be described.

The operation of the audio / video data generation device 2 according to the third embodiment will be described below with reference to FIG. In the following description, processing for video data and subtitle data (same as in the second embodiment) will be omitted for simplification of description, and processing for audio data will be mainly described. The audio encoder 24 encodes input digital uncompressed audio data as in the first embodiment and the second embodiment. As in the second embodiment, the data size 30b counts the data size of the audio stream generated by the audio encoder 24 for each audio frame, and outputs the data size to the CPU 424.

FIG. 10 shows a voice / voice according to the third embodiment.
4 is a flowchart showing a process of a CPU 424 of the video data generation device 2 (FIG. 2). Step 200 (S20
0), the CPU 424, for example,
The data rate (code rate SIZE S) of the audio stream input by the user of the audio / video data generation device 2 to a terminal device (not shown) or the like connected via the circuit 420
EC) settings. That is, the CPU 424 receives the setting of the number of bytes of the audio stream generated per second by the audio encoder 24, and performs the processing in the following steps based on the code rate SIZE SEC.

As shown in step 202 (S202), the CPU 424 initializes numerical values (size sum, PTS, PTS SEC) used for calculating the audio PTS, and resets these numerical values to 0 [zero clear (size sum = 0, PTS = 0,
PTS SEC = 0)]. In step 204 (S204), the CPU 424 determines whether or not the data size of the audio stream has been input from the data amount IF 30b. The process proceeds to S206 only when the data size SIZE of the audio stream is input from the data amount IF 30b.

In step 206 (S206), C
The PU 424 changes the value of the audio PTS from the data (FIG. 8) stored in the processing RAM 426 to the value calculated in the process of S216. Step 208
In (S208), the CPU 424 controls the switch circuit 36 to first select the input terminal a and multiplex the PES header (FIG. 6) stored in the processing RAM 426 into the transport stream.

In step 210 (S210), C
The PU 424 controls the switch circuit 34 to control the input terminal b
Side, and controls the switch circuit 36 to control the input terminal b
Side to multiplex the audio stream to the transport stream. Step 212 (S21
In 2), the CPU 424 adds the data size SIZE input from the data amount IF 30b to the numerical value size sum (size sum = sizesum + SIZE) to obtain the data amount IF.
Numerical value size sum indicating the cumulative addition value of the audio stream counted by 30b is newly calculated.

In step 214 (S214), C
The PU 424 calculates the numerical value si calculated in the process of S212.
It is determined whether or not the value of ze sum is equal to or greater than the code rate SIZE SEC set in the processing of S200. CPU4
24, if the value of the numerical value size sum is equal to or greater than the code rate SIZE SEC, the process proceeds to S218, where the code rate SIZE
If it is equal to or less than SEC, the process proceeds to S216. That is, the CPU 424 determines that the value of the numerical value size sum is the code rate SI
At one second intervals equal to or longer than ZE SEC, the process proceeds to S218 and S220 for correcting the value of the audio PTS.

In step 216 (S216), C
The PU 424 performs an integer operation shown in Expression 7 below, calculates the value of the audio PTS, and proceeds to the processing of S204.

[0061]

PTS = PTS SEC + 90000 × size sum / SIZE SEC (7) In Expression 7, the numerical value 90,000 is an offset value added every second when generating the PTS.

In step 218 (S218), C
The PU 424 newly calculates the value of the numerical value size sum (si
ze sum = size sum-SIZE SEC). That is, the CPU 42
4 calculates the value of the difference between the code rate SIZE SEC set in S200 and the data amount of the audio stream actually generated for one second in the process of S218, and sets it as a new numerical value size sum. .

In step 220 (S220), C
The PU 424 is a numerical value PTS SE indicating an error in the value of the audio PTS generated in one second, that is, an offset value used in calculating the value of the audio PTS in the next one second.
Calculate C (PTS SEC = PTSSEC + 90000). That is,
The CPU 424 calculates the value PTS used for 1 second
By adding the numerical value 90,000 shown in Expression 7 to the SEC value, the value of the numerical value PTS SEC to be used in the calculation in S216 for the next one second is calculated.

The offset value for each frame of the video PTS is a fixed integer value of 3,003 or 3,600. On the other hand, when the CPU 424 generates the audio PTS as described above, the audio PTS is generated.
Is an integer. Therefore, video P
It is possible to accurately synchronize the TS and the audio PTS by an integer operation.

The FIFO memory 38 stores the FIFO memory 3
The transport stream output from the buffer 8 is buffered and transmitted to a communication line or the like. SCSII
The F circuit 40 sends the transport stream output from the FIFO memory 38 to a hard disk device or the like, and records the transport stream.

Referring to FIG. 11, audio data and video data are reproduced from the transport stream (MPEG system stream) generated by the operation of audio / video data generation device 2 shown in the third embodiment. The processing will be described. FIG. 11 shows audio / video data decompression for separating and decompressing audio and video streams and their PES from the transport stream generated by the operation of the audio / video data generation device 2 shown in the third embodiment. -It is a figure showing the composition of decoding device 3.

As shown in FIG. 11, the audio / video data decompression / decoding device 3 includes a system decoder 300, an audio decoder 302, a video decoder 304, an audio delay memory 306, a video delay memory 308, and a time stamp comparison. It comprises a circuit 310. System decoder 3 of audio / video data decompression / decoding device 3
00 is an audio stream, a video stream, an audio PTS from the PES included in the transport stream input from the audio / video data generation device 2.
(Audio time stamp) and video PTS (video time stamp), output the audio stream to the audio decoder 302, output the video stream to the video decoder 304, and compare the audio PTS and the video PTS with the time stamp. Output to the circuit 310.

The audio decoder 302 converts the audio stream input from the system decoder 300
The audio / video data generators 1 and 2 (FIGS. 1 and 2) decompress and decode by the decompression and decoding method corresponding to the audio encoder 24, generate original audio data, and output it to the delay memory 306. The video decoder 304 decompresses and decodes the video stream input from the system decoder 300 by a decompression decoding method corresponding to the video encoder 20.
The original video data is generated and output to the delay memory 308. The delay memories 306 and 308 buffer and output the audio data and the video data input from the audio decoder 302 and the video decoder 304, respectively, under the control of the time stamp comparison circuit 310.

The time stamp comparison circuit 310 controls the delay memories 306 and 308 based on the audio PTS and the video PTS input from the system decoder 300, and stores the audio data and the video data buffered by the delay memories 306 and 308. And synchronize with the display time and output. According to the audio / video data decompression / decoding device 3 described above, the audio stream and the video stream generated by the audio / video data generation device 2 can be decompressed and decoded, and output with the same display time.

As described in the third embodiment, when the sampling frequency of the uncompressed video data input to the audio encoder 24 is 44.1 kHz, the value of the PTS of the PES containing the audio stream is calculated by an integer. CPU 424 performs the processing of S218 and S220 (FIG. 10) once per second to correct the value of PTS (numerical value PTS SEC).
Errors in the value of PTS do not accumulate over a long period of time.

Therefore, the sampling frequency of the uncompressed video data input to the audio encoder 24 is 44.
Even when the frequency is 1 kHz, the PTS calculation process can be performed by an integer calculation instead of a real number calculation, and the algorithm of the PTS calculation process (hardware configuration when calculating the PTS value by hardware) is simplified. can do.

In the third embodiment, the CPU
424 describes the case where the value of the PTS of the PES is corrected at one-second intervals. By changing the code rate SIZE SEC set in S200 (FIG. 10), the data of the audio stream output by the audio encoder 24 is changed. On the condition that the rate can be set to a constant value, the time interval of correction for the PTS can be changed.

Fourth Embodiment Hereinafter, as a fourth embodiment of the present invention, referring to FIG. 12, the audio / video data generation device 2 (FIG. 2) shown in the second embodiment will be described in terms of software. A method of generating a program clock reference (PCR) at high speed and accurately by an integer operation and generating a transport stream will be described. The operation of each component other than the CPU 424 of the audio / video data generation device 2 in the fourth embodiment is as described in the second embodiment.

FIG. 12 shows the voice / voice in the fourth embodiment.
4 is a flowchart showing a process of a CPU 424 of the video data generation device 2 (FIG. 2). Note that all the operations shown in FIG. 12 are integer operations and are executed by software. As shown in FIG. 12, the audio / video data generation device 2
Can calculate the PCR without accumulating the calculation error by substantially the same processing as the calculation processing of the audio PTS shown in FIG.

In step 300 (S300), C
The PU 424 receives the setting of the data rate (code rate SIZE SEC) of the video stream. That is, the CPU 42
4 receives the setting of the number of bytes (RATE) of the video stream generated by the video encoder 20 per second, and performs the processing in the following steps based on the code rate SIZE SEC.

As shown in step 302 (S302), the CPU 424 stores the data in the processing RAM 426,
Numeric value used to calculate audio PTS (PCR FRAME,
PCR) and initialize these values to zero. In step 304 (S304), the CPU 424
It is determined from the data amount IF 30a whether or not the data size of the video stream has been input. At least, the process proceeds to S306 only when the data size SIZE of the video stream is input from the data amount IF 30a.

In step 306 (S306), C
The PU 424 updates the PCR value among the data (FIG. 7) stored in the processing RAM 426 to the value calculated in the process of S318. Step 308 (S30
In 8), the CPU 424 calculates the numerical value packet frame by performing the operation of Expression 8 shown below. The value packet
frame is the TS to output during the video frame being processed
Indicates the total number of bytes of the packet, and is a numeric
Is a number SI indicating the number of bytes per video frame
Does not exceed the value of ZE FRAME.

[0078]

[Equation 8] packet frame = ((SIZE FRAME + size remain) / 188) * 188 (8)

In step 310 (S310), C
The PU 424 controls the switch circuit 34 to select the input terminals a and b, controls the transmitting device 36 to select the input terminal b, and multiplexes the video stream and the audio stream into the transport stream.
Further, by controlling the switch circuit 36 to select the input terminal a side, the header data (FIGS. 7 and 8) stored in the processing RAM 426 is multiplexed into the transport stream, and is multiplexed through the FIFO memory 38. Output.

In step 312 (S312), C
The PU 424 calculates the numerical value packet f calculated in S308.
Data is multiplexed into the transport stream by the number of bytes indicated by rame, and it is determined whether or not the data
If data is multiplexed and output only for frame, S31
If the process proceeds to step 4 and the data is not multiplexed and output only for the numerical packet frame, the process proceeds to step S312 to repeat the data multiplexing and output process.

In step 314 (S314), C
The PU 424 performs an operation of Expression 9 shown below to obtain a numerical value size
Calculate remain. Note that, as shown in Equation 9, the numerical value si
ze remain is a difference value between the numerical value packet frame and the numerical value SIZE FRAME, and indicates the number of bytes of data not multiplexed in the transport packet in the video frame being processed. It is added to the number of bytes of data to be multiplexed in the port packet. In the calculation of S314 (Equation 9), the value of the numerical value size remain is a negative value.

[0082]

[Equation 9] size remain = packet frame-SIZE FRAME + size remain (9)

In step 316 (S316), C
The PU 424 performs an integer operation shown in Expression 10 below,
Calculate the value of PCR FRAME. The numerical PCR FRAME is
Indicates the offset value for each video frame of the numerical PCR.

[0084]

[Expression 10] PCR FRAME = PCR FRAME + PCR _base (10) In Expression 10, when the video data has a 525/60 configuration, PCR _base = 3,003 and the video data is 6
In the case of the 25/50 configuration, PCR _base = 3,600.

In step 318 (S318), C
The PU 424 performs the operation shown in Expression 11 below, then calculates the PCR to be multiplexed into the transport packet, and calculates S3
It returns to the process of 04. Since the second term on the right side of Equation 11 includes division in the operation, an arithmetic error occurs when calculated by integer operation. However, no arithmetic error occurs in the first term on the right side even by integer arithmetic, and furthermore, since the processing is updated for each video frame by the processing in S316, the arithmetic error differs with time, unlike the method shown in the related art. It accumulates and does not grow.

[0086]

[Equation 11] PCR = PCR FRAME + size remain * 27,000,000 / RATE (11)

According to the processing shown in FIG.
The value of R is set to a correct value for each video frame, and no calculation error is accumulated. Therefore, the synchronization of the STC between the receiving side and the transmitting side is not lost, so that the audio / video data can be accurately transmitted and reproduced for a long time. Further, the method described in the fourth embodiment can be applied to a case where audio / video data is transmitted between a plurality of transmitting apparatuses and a plurality of receiving apparatuses.

Further, according to the method shown in the fourth embodiment, similarly to the PCR, the SCR used for clock synchronization between the receiving side and the transmitting side can be accurately calculated. Further, in the fourth embodiment, the T
Although the case where the processing is performed with the position of the S packet (FIG. 7) at the beginning is shown, the position of the TS packet with the PCR multiplexed is not necessarily at the beginning.

[0089]

As described above, according to the synchronous data generating apparatus and method of the present invention, even if the PCR is generated by the integer operation, the operation error of the generated PCR is not accumulated, and PCR can be generated.
Further, according to the synchronous data generation device and the method thereof according to the present invention, it is possible to generate PCR accurately and at high speed by software integer operation without dedicated hardware for performing real number operation.

[Brief description of the drawings]

FIG. 1 is a diagram showing a configuration of an audio / video data generation device according to the present invention in a first embodiment.

FIG. 2 is a diagram illustrating a configuration of an audio / video data generation device according to the present invention in a second embodiment.

FIG. 3 is a diagram illustrating a configuration of a video encoder illustrated in FIG. 2;

FIG. 4 is a diagram illustrating a configuration of an audio encoder illustrated in FIG. 2;

FIG. 5 is a flowchart illustrating processing of a CPU of the audio / video data generation device (FIG. 2) according to the second embodiment.

FIG. 6 is a diagram illustrating a transport stream, a PES, a pack, and a PS used for transmitting an audio stream and a video stream compression-encoded by the MPEG method
FIG. 2 is a diagram showing a configuration of the ITU-T Recommendation H.222.0 | ISO / IEC 13818-1.

FIG. 7 is a diagram showing a detailed configuration of a TS header included in the transport stream shown in FIG.

FIG. 8 is a diagram illustrating a configuration of a PES header section of the PES illustrated in FIG. 6;

FIGS. 9A and 9B show a case where the sampling frequency is 48;
kHz, an audio frame at 44.1 kHz,
Audio stream data size (SIZE) and PTS
It is a figure showing the relation of the increase amount of.

FIG. 10 is a flowchart showing processing of a CPU of the audio / video data generation device (FIG. 2) according to the third embodiment.

FIG. 11 shows an audio / video data decompression / decompression unit that separates an audio / video stream and their PES from a transport stream generated by the operation of the audio / video data generation apparatus shown in the third embodiment, and decompresses and decodes them. It is a figure showing the composition of a decoding device.

FIG. 12 is a flowchart illustrating processing of a CPU of an audio / video data generation device (FIG. 2) according to a fourth embodiment.

[Explanation of symbols]

1, 2, audio / video data generating device, 10: multiplexing device, 100: buffer memory, 102: transport stream generating device, 12: multiplexing system, 32a, 32
b, 32c: FIFO memory, 34, 36: switch circuit, 38: FIFO memory, 40: SCSIIF circuit,
42: control system, 30a, 30b, 30c: data amount I
F, 420: ENIF circuit, 422: SIF circuit, 42
4 ... CPU, 426 ... Processing RAM, 428 ... Control Data RAM, 3 ... Audio / Video Data Decompression / Decoding Device, 3
00: system decoder, 302: audio decoder, 304: video decoder, 306, 308: delay memory, 310: time stamp comparison circuit.

Claims (2)

[Claims] An audio data and / or video data or any one of them (audio / video data) is compressed in synchronism with a predetermined clock signal by an MPEG system, and compressed audio / video data is compressed.
Compression means for generating video data; synchronization data generation means for generating synchronization data used for reproducing the clock signal for each predetermined transmission cycle; and at least the generated compressed audio / video data and the synchronization data. Multiplexing means for accommodating the remaining data in a predetermined transmission packet, wherein the synchronization data generating means calculates, for each transmission cycle, a remaining data amount remaining without being accommodated in the transmission packet. Calculating means; an offset value calculating means for calculating an offset value of the synchronization data for each transmission cycle; and calculating the synchronization data for each transmission cycle based on the calculated remaining data amount and the offset value. A synchronous data generation device having synchronous data calculation means. 2. A method for compressing audio data and / or video data or any one of them (audio / video data) in synchronism with a predetermined clock signal according to an MPEG system to generate compressed audio / video data and transmitting the predetermined data. For each cycle, a synchronous data generating method for generating synchronous data used for reproduction of the clock signal, and at least multiplexing the generated compressed audio / video data and synchronous data into a predetermined transmission packet, For each transmission cycle, calculate the remaining data amount that is not accommodated in the predetermined transmission packet, calculate the offset value of the synchronization data for each transmission cycle, and calculate the remaining data amount and the offset value. A synchronous data generation method for calculating the synchronous data for each transmission cycle based on the transmission data.