JP5677018B2 - Data multiplexing apparatus, data multiplexing method and system - Google Patents

Description

  The present invention relates to a data multiplexing apparatus and a data multiplexing method for efficiently multiplexing and outputting media data of a plurality of programs, for example.

For example, in a situation where the transmission band is limited as in digital broadcasting, it is necessary to efficiently multiplex and output a plurality of programs. Patent Document 1 below efficiently multiplexes a plurality of programs. A data multiplexing apparatus (data multiplexing apparatus that performs statistical multiplexing) is disclosed.
In general, when video data is encoded, if the complexity of the input image, the size of the motion, and the like are different, the code amount greatly fluctuates even when video encoded data having the same image quality is generated.
In the above data multiplexing device, when a plurality of programs are multiplexed and output, the complexity of the input images of each program is checked before encoding so that the video quality of the plurality of programs is equalized. A large code amount is assigned to a program with a simple input image, and a small code amount is assigned to a program with a non-complex input image.
At this time, by controlling the total code amount to match the transmission band, it is possible to multiplex a plurality of programs while maintaining the same image quality of the plurality of programs within the limited transmission band. It becomes possible.

Patent Document 2 below discloses a method of multiplexing other data in an available band when performing statistical multiplexing.
If the sum of the variable rate data cannot be matched to the output rate by statistical multiplexing, it is possible to achieve efficient multiplexing by multiplexing other data in an available band.

Patent No. 4441839 Japanese Patent No. 4206559

Since the conventional data multiplexing apparatus is configured as described above, the code amount is distributed for the purpose of equalizing the image quality of a plurality of programs at a certain time. For this reason, when a program with a complex input image and a program with a complex input image are mixed, the amount of code allocated to a program with a complex input image is reduced, and a larger amount of code is allocated to a program with a complex input image. Therefore, it is possible to suppress a deterioration in the image quality of a program having a complicated input image. However, when the input images of all the programs are complicated, there is no program that can reduce the code amount, and there is a problem that the image quality is deteriorated in all the programs.
In addition, it is possible to make the image quality of a plurality of programs uniform at a certain time, but if attention is paid to a certain program, the image quality changes with the passage of time (the complexity of the input image of the program). Since the time fluctuates sequentially, there is a problem that it is difficult to stabilize the image quality of the program in the time axis direction.
Furthermore, when the output rate is adjusted using other data, there is a problem that the transmission capacity of the other data cannot be guaranteed.

The present invention has been made in order to solve the above-described problems, and is a data that can achieve high-quality program multiplexing while suppressing temporal fluctuations in image quality while equalizing the image quality of a plurality of programs. An object is to obtain a multiplexing device and a data multiplexing method.
It is another object of the present invention to provide a data multiplexing apparatus and a data multiplexing method capable of guaranteeing the transmission capacity of other data sent simultaneously.

The data multiplexing apparatus according to the present invention multiplexes the encoded data output from the real-time data encoding means within a preset transmission rate range, and sets the data rate of the multiplexed encoded data to a predetermined value. A non-real time data output from the non-real time data output means is further multiplexed within the range of the transmission rate difference to output multiplexed data, the real time data rate control means from the multiplexing means Accumulate the amount of encoded data included in the multiplexed data from the time when output of the multiplexed data is started, and the elapsed time from the time when output of the multiplexed data is started until the present time by multiplying the reference rate which is set in advance, calculates the difference between the cumulative value of the multiplication result and the data amount, the difference is Tokoro If it exceeds the range, in which the give an instruction for controlling the code amount of the coded data so that the difference becomes smaller time data encoding means.

According to the present invention, the encoded data output from the real-time data encoding means is multiplexed within a preset transmission rate range, and the data rate of the multiplexed encoded data and a predetermined transmission rate are set. Within the range of the difference, a non-real time data output from the non real time data output means is further provided with multiplexing means for outputting multiplexed data, and the real time data rate control means receives the multiplexed data from the multiplexing means. The amount of encoded data included in the multiplexed data is accumulated from the time when the output is started, and an elapsed time from the time when the output of the multiplexed data is started until the present time is preset. by multiplying the reference rate are, calculates a difference between the accumulated value of the multiplication result and the data amount, the difference exceeds a predetermined range The case, since it is configured to provide instructions to control the code amount of the coded data so that the difference becomes smaller real time data coding means, while achieving equalization of the image quality of a plurality of programs, the quality time There is an effect that it is possible to multiplex programs with high image quality while suppressing global fluctuations.

It is a block diagram which shows the data multiplexing apparatus by Embodiment 1 of this invention. It is a flowchart which shows the processing content (data multiplexing method) of the data multiplexing apparatus by Embodiment 1 of this invention. It is explanatory drawing which shows an example of the multiplexed data output from the multiplexing part. It is explanatory drawing which shows the bit rate of the encoding data of each video data when encoding video data by making three types of video image quality substantially constant. FIG. 5 is an explanatory diagram illustrating a cumulative result of information amount of encoded data of the video data of FIG. It is explanatory drawing which shows the difference of the accumulation result of information amount, and desired accumulation information amount (desired accumulation information amount when the desired average rate of the encoding data of real-time data shall be 120 kbps). It is a block diagram which shows the data multiplexing apparatus by Embodiment 2 of this invention. It is a flowchart which shows the processing content (data multiplexing method) of the data multiplexing apparatus by Embodiment 2 of this invention.

Embodiment 1 FIG.
1 is a block diagram showing a data multiplexing apparatus according to Embodiment 1 of the present invention.
In FIG. 1, real-time data encoders 1a and 1b are composed of encoders, for example, and real-time data (for example, video data, audio data, etc.) assumed to be decoded and reproduced in real time on the decoding device side. Is encoded in real time, and the encoded data of the real time data is output.
Although FIG. 1 shows an example in which two real-time data encoding units 1a and 1b are multiplexed in the data multiplexing device, only one unit may be installed or three or more units may be installed. Also good.
The real-time data encoding units 1a and 1b constitute real-time data encoding means.

The non-real-time data output units 2a and 2b are composed of, for example, a server and the like, and are assumed to be decoded and reproduced in non-real time on the decoding device side (once stored on the decoding device side and then decoded and reproduced). Non-real-time data (for example, encoded data or files that have been accumulated previously) is output.
Although FIG. 1 shows an example in which two non-real-time data output units 2a and 2b are multiplexed in the data multiplexing device, only one or three or more may be installed. May be.
The non-real time data output units 2a and 2b constitute non-real time data output means.

The multiplexing unit 3 is composed of, for example, a semiconductor integrated circuit on which a CPU is mounted or a one-chip microcomputer, and the codes output from the real-time data encoding units 1a and 1b and the non-real-time data output units 2a and 2b. Among the encoded data, the encoded data of the real-time data output from the real-time data encoding units 1a and 1b is preferentially multiplexed, and remains within the difference between the band of the multiplexed data and a predetermined transmission rate (remaining Within the range of the transmission rate), the non-real time data output from the non-real time data output units 2a and 2b is multiplexed and output.
The multiplexing unit 3 constitutes multiplexing means.

The real-time data rate control unit 4 is composed of, for example, a semiconductor integrated circuit on which a CPU is mounted, a one-chip microcomputer or the like, and a code of real-time data included in the multiplexed data output from the multiplexing unit 3 The average rate of the encoded data is measured, and the process of controlling the code amount of the encoded data output from the real-time data encoding units 1a and 1b is performed so that the average rate approaches the desired average rate.
That is, the real time data rate control unit 4 determines the code amount of the encoded data when the average rate of the encoded data of the real time data included in the multiplexed data output from the multiplexing unit 3 is higher than the desired average rate. If the average rate of the encoded data of the real-time data is lower than the desired average rate, an instruction to increase the code amount of the encoded data is given to the real-time data encoding unit 1a, 1b. The process given to 1b is implemented.
The real time data rate control unit 4 constitutes a real time data rate control means.

In FIG. 1, each of the real-time data encoding units 1a and 1b, the non-real-time data output units 2a and 2b, the multiplexing unit 3 and the real-time data rate control unit 4 which are components of the data multiplexing apparatus is a dedicated hardware. However, when the data multiplexing apparatus is configured by a computer, the real-time data encoding units 1a and 1b, the non-real-time data output units 2a and 2b, the multiplexing unit 3, and the real-time data rate are assumed. A program describing the processing content of the control unit 4 may be stored in a memory of a computer, and the CPU of the computer may execute the program stored in the memory.
FIG. 2 is a flowchart showing the processing contents (data multiplexing method) of the data multiplexing apparatus according to Embodiment 1 of the present invention.

Next, the operation will be described.
First, the real-time data encoding units 1a and 1b set an initial image quality level (step ST1).
When the real-time data encoding units 1a and 1b input real-time data (for example, media data including video data) that is assumed to be decoded and reproduced in real time on the decoding device side, the set image quality level is obtained. As shown, the real-time data is encoded in real time, and the encoded data of the real-time data is output (step ST2).

In the real-time data encoding units 1a and 1b, in order to stabilize the image quality, it is assumed that the amount of encoded data to be output changes with time.
That is, it is assumed that if the input image is complicated, the amount of encoded data is temporarily increased, whereas if the input image is not complicated, the amount of encoded data is decreased.
The real-time data encoding units 1a and 1b correspond to devices that output an encoded data sequence (for example, MPEG-2 TS) including a plurality of media constituting one program in digital broadcasting.

The non-real-time data output units 2a and 2b are non-real-time data that is assumed to be decoded in non-real time on the decoding device side and reproduced (they are stored once on the decoding device side and then decoded and reproduced) (for example, (Encoded data, files, etc.) stored in (step ST3).
The non-real time data may be data stored in the internal memory of the non-real time data output units 2a and 2b (not shown) or may be data input from the outside.

Multiplexer 3 receives real-time encoded data from real-time data encoders 1a and 1b and receives non-real-time data from non-real-time data output units 2a and 2b. The encoded data of the real-time data output from the data encoding units 1a and 1b is preferentially multiplexed (step ST4).
However, the multiplexing unit 3 multiplexes the encoded data of the real-time data within a predetermined transmission rate TRNrate.

If the output rate OUTrate, which is a band of multiplexed data in which encoded data of real-time data is multiplexed, does not reach the predetermined transmission rate TRNrate (OUTrate <TRNrate), the multiplexing unit 3 performs the remaining transmission. Within the range of the rate (= TRNrate−OUTrate), the non-real time data output from the non-real time data output units 2a and 2b is multiplexed (steps ST5 and ST6).
If the output rate OUTrate of the multiplexed data has reached the predetermined transmission rate TRNrate (OUTrate = TRNrate), the non-realtime data output from the non-realtime data output units 2a and 2b is not multiplexed.

The multiplexing unit 3 is a non-real-time data output unit assuming that the output rate OUTrate of multiplexed data in which encoded data of real-time data is multiplexed may temporarily exceed a predetermined transmission rate TRNrate. An input buffer or the like for temporarily storing the non-real time data output from 2a and 2b is provided inside, and is configured to be able to cope with a temporary excess of encoded data.
Therefore, when the multiplexing unit 3 cannot multiplex all the non-real time data output from the non-real time data output units 2a and 2b, it temporarily stores some non-real time data in the input buffer, The non-real time data is read from the input buffer and multiplexed at a timing at which multiplexing is possible.

Multiplexer 3 multiplexes the encoded data of the real-time data and the non-real-time data, and outputs the multiplexed data (step ST7).
The multiplexing unit 3 performs the processing as described above, so that the encoded data of the real-time data output from the real-time data encoding units 1a and 1b is preferentially multiplexed while the non-real-time data output units 2a and 2b The outputted non-real time data can be multiplexed until a predetermined transmission rate TRNrate is reached.
If the output rate OUTrate of the multiplexed data does not reach the predetermined transmission rate TRNrate even if the encoded data of the real-time data and the non-realtime data are combined, for example, multiplexing is performed by multiplexing stuffing data or the like. It is also possible to make the data output rate OUTrate reach a predetermined transmission rate TRNrate.

When the multiplexing unit 3 outputs the multiplexed data, the real-time data rate control unit 4 measures the average rate AVErate of the encoded data of the real-time data included in the multiplexed data (step ST8).
When the real-time data rate control unit 4 measures the average rate AVErate of the encoded data of the real-time data, the real-time data rate control unit 4 outputs the average rate AVErate from the real-time data encoding units 1a and 1b so that the average rate AVErate approaches the desired average rate AVErate _ref. Controls the amount of encoded data.

That is, when the average rate AVErate of the encoded data of the real-time data is higher than the desired average rate AVErate _ref (step ST9), the real-time data rate control unit 4 gives an instruction to reduce the code amount of the encoded data. 1a and 1b are provided (step ST10).
As an instruction to reduce the code amount of the encoded data, for example, an instruction to lower the previously set image quality level (decrease the image quality) can be considered.
As a result, the average rate AVErate of the encoded data of the real-time data output from the real-time data encoding units 1a and 1b is lowered, so that the average rate AVErate of the encoded data of the real-time data can be made closer to the desired average rate AVErate _ref. It becomes possible.

When the average rate AVErate of the encoded data of the real-time data is lower than the desired average rate AVErate _ref (step ST9), the real-time data rate control unit 4 gives an instruction to increase the code amount of the encoded data in the real-time data encoding unit 1a, 1b (step ST11).
As an instruction to increase the code amount of encoded data, for example, an instruction to increase a previously set image quality level (improve image quality) can be considered.
As a result, the average rate AVErate of the encoded data of the real-time data output from the real-time data encoding units 1a and 1b is increased, so that the average rate AVErate of the encoded data of the real-time data can be made closer to the desired average rate AVErate _ref It becomes possible.

Here, when the average rate AVErate of the encoded data of the real-time data is higher than the desired average rate AVErate _{ref, an} instruction to reduce the code amount of the encoded data is given to the real-time data encoding units 1a and 1b, while the code of the real-time data is encoded. When the average rate AVErate of the encoded data is lower than the desired average rate AVErate _{ref, an} instruction to increase the code amount of the encoded data is given to the real-time data encoding units 1a and 1b. When the difference between the average rate AVErate and the desired average rate AVErate _ref is small (when the difference is smaller than a predetermined threshold), an instruction to change the code amount of the encoded data is not given to the real-time data encoding units 1a and 1b It may be.
In this case, since the image quality level is not changed when the average rate AVErate of the encoded data of the real-time data is close to the desired average rate AVErate _ref , the image quality can be stabilized.

The real-time data rate control unit 4 receives the notification of the rate information of the real-time data from the multiplexing unit 3 without checking the multiplexed data output from the multiplexing unit 3, and receives the real-time data encoding unit 1a. , 1b, the code amount of the encoded data may be controlled.
Thereafter, the process returns to step ST2, and the processes of steps ST2 to ST11 are repeatedly performed.

Here, FIG. 3 is an explanatory diagram showing an example of multiplexed data output from the multiplexing unit 3.
However, FIG. 3 shows an example of multiplexed data when three real-time data encoding units 1 and one non-real-time data output unit 2 are mounted on the data multiplexing device.
As described above, the multiplexing unit 3 multiplexes the encoded data (1) to (3) of the real-time data and the non-real-time data so that the output rate OUTrate of the multiplexed data matches the predetermined transmission rate TRNrate. Therefore, the total rate of the encoded data (1) to (3) of the real-time data and the non-real-time data always matches the transmission rate TRNrate.

However, the output rate of the encoded data (1) to (3) of the real-time data varies depending on the complexity of the input image.
For this reason, a desired average rate AVErate _ref (the middle dotted line in FIG. 3) is preset, and the sum of the output rates of the encoded data (1) to (3) of the real-time data is in the vicinity of the desired average rate AVErate _ref . Fluctuates.

For example, when the sum of the output rates of the encoded data (1) to (3) of the real-time data exceeds a desired average rate AVErate _ref , the real-time data rate control unit 4 encodes three real-time data By outputting an instruction to reduce the code amount of the encoded data to the unit 1, the output rate of the encoded data (1) to (3) of the real-time data is lowered, and finally the encoded data (1) to The sum of the output rates in (3) approaches the desired average rate AVErate _ref .
On the other hand, when the total output rate of the encoded data (1) to (3) of the real-time data is less than the desired average rate AVErate _ref , the real-time data rate control unit 4 uses the three real-time data codes. By outputting an instruction for increasing the code amount of the encoded data to the encoding unit 1, the output rate of the encoded data (1) to (3) of the real-time data is increased, and finally the encoded data (1) The sum of the output rates of (3) approaches the desired average rate AVErate _ref .

In this way, the sum of the output rates of the encoded data (1) to (3) of the real-time data that is actually output is monitored, and the encoded data (1) according to the difference from the desired average rate AVErate _ref. By adjusting the amount of code in (3), the total rate of real-time data and the total rate of non-real-time data can be brought close to desired rates.
At this time, since the temporary code amount of the encoded data (1) to (3) of the real-time data is allowed to increase / decrease, a real-time program with stable image quality can be multiplexed without causing a sharp image quality degradation. Yes.

However, when the real-time data rate control unit 4 measures the average rate AVErate of the encoded data of the real-time data, the real-time data rate control unit 4 measures the average rate AVErate every certain period, and the average rate AVErate approaches the desired average rate AVErate _ref . As described above, when the code amount of the encoded data output from the real-time data encoding units 1a and 1b is to be controlled, the average rate AVErate of the encoded data of the real-time data at that time is brought close to the desired average rate AVErate _ref . However, when viewed over a long period, the average rate AVErate of the encoded data of real-time data may deviate from the desired average rate AVErate _ref .

Therefore, the real-time data rate control unit 4 accumulates the data amount D of the encoded data of the real-time data included in the multiplexed data from the time T _START when the output of the multiplexed data from the multiplexing unit 3 is started. The elapsed time ΔT (= T _NOW −T _START ) from the output start time T _START to the current time T _NOW is multiplied by the desired average rate AVErate _ref .
The real-time data rate control unit 4 compares the accumulated value Daccum of the data amount D with the multiplication result ΔT × AVErate _{ref of} the elapsed time ΔT and the desired average rate AVErate _ref , and multiplies the accumulated value Daccum of the data amount D. If the result is larger than ΔT × AVErate _{ref, an} instruction to reduce the code amount of the encoded data is given to the real-time data encoding units 1a and 1b.
On the other hand, if the accumulated value Daccum of the data amount D is smaller than the multiplication result ΔT × AVErate _{ref, an} instruction to increase the code amount of the encoded data is given to the real-time data encoding units 1a and 1b.
In this way, since the accumulated value Daccum of the data amount D can be brought close to the desired total data amount, the average rate AVErate of the encoded data of the real-time data can be set to the desired average rate AVErate _ref even in a long period of time. Can be approached.

FIG. 4 is an explanatory diagram showing the bit rate of the encoded data of each video data when the video data is encoded with the three types of video image quality being substantially constant.
The bit rate of the encoded data of each video data varies from moment to moment, and the total output rate of the three encoded data also varies.
FIG. 5 is an explanatory diagram showing a cumulative result of the information amount of the encoded data of the video data in FIG.
FIG. 6 is an explanatory diagram showing the difference between the accumulated information amount and the desired accumulated information amount (desired accumulated information amount when the desired average rate of encoded data of real-time data is 120 kbps). .
As shown in FIG. 5, by allowing the rate variation of the total output rate of encoded data to some extent, it is possible to perform encoding processing with a constant image quality and to achieve a desired average rate.

As apparent from the above, according to the first embodiment, among the encoded data output from the real-time data encoding units 1a and 1b and the non-real-time data output units 2a and 2b, the real-time data encoding unit 1a, The encoded data of the real-time data output from 1b is preferentially multiplexed and output from the non-real-time data output units 2a and 2b within the range of the difference between the output rate OUTrate of the multiplexed data and a predetermined transmission rate TRNrate. A multiplexing unit 3 for multiplexing and outputting the non-real-time data, and the real-time data rate control unit 4 determines the average rate of the encoded data of the real-time data included in the multiplexed data output from the multiplexing unit 3 AVErate is measured, and the average rate AVErate is the desired average rate AVE so as to approach the ate _ref, real-time data encoding unit 1a, since it is configured to control the code amount of the coded data outputted from the 1b, while achieving equalization of the image quality of a plurality of programs, the time of image quality There is an effect that it is possible to multiplex programs with high image quality while suppressing fluctuations.

Embodiment 2. FIG.
In the first embodiment, the real-time data rate control unit 4 measures the average rate AVErate of the encoded data of the real-time data included in the multiplexed data output from the multiplexing unit 3, and the average rate AVErate is Although the control of the code amount of the encoded data output from the real-time data encoding units 1a and 1b so as to approach the desired average rate AVErate _ref has been shown, the multiplexed data output from the multiplexing unit 3 is The average rate AVErate ′ of the included non-real-time data is measured, and the encoded data output from the real-time data encoding units 1a and 1b is adjusted so that the average rate AVErate ′ approaches the desired average rate AVErate _ref ′. The code amount may be controlled.

FIG. 7 is a block diagram showing a data multiplexing apparatus according to Embodiment 2 of the present invention. In FIG. 7, the same reference numerals as those in FIG.
The real-time data rate control unit 5 is composed of, for example, a semiconductor integrated circuit on which a CPU is mounted or a one-chip microcomputer, and the non-real-time data included in the multiplexed data output from the multiplexing unit 3 The average rate AVErate ′ is measured, and the process of controlling the code amount of the encoded data output from the real-time data encoding units 1a and 1b is performed so that the average rate AVErate ′ approaches the desired average rate AVErate _ref ′ To do.
That is, if the average rate AVErate ′ of the non-real-time data included in the multiplexed data output from the multiplexing unit 3 is higher than the desired average rate AVErate _ref ′, the real-time data rate control unit 5 An instruction to increase the amount is given to the real-time data encoding units 1a and 1b, and if the average rate AVErate ′ of the non-real-time data is lower than the desired average rate AVErate _ref ′, an instruction to decrease the code amount of the encoded data is provided. The process given to the conversion units 1a and 1b is performed.
The real time data rate control unit 5 constitutes a real time data rate control means.

In FIG. 7, each of the real-time data encoding units 1a and 1b, the non-real-time data output units 2a and 2b, the multiplexing unit 3 and the real-time data rate control unit 5 which are components of the data multiplexing apparatus is a dedicated hardware. However, when the data multiplexing apparatus is configured by a computer, the real-time data encoding units 1a and 1b, the non-real-time data output units 2a and 2b, the multiplexing unit 3, and the real-time data rate are assumed. A program describing the processing contents of the control unit 5 may be stored in a memory of a computer, and the CPU of the computer may execute the program stored in the memory.
FIG. 8 is a flowchart showing the processing contents (data multiplexing method) of the data multiplexing apparatus according to the second embodiment of the present invention.

Next, the operation will be described.
However, since the components other than the real-time data rate control unit 5 are the same as those in the first embodiment, only the processing contents of the real-time data rate control unit 5 will be described.
When the multiplexing unit 3 outputs the multiplexed data, the real-time data rate control unit 5 measures the average rate AVErate ′ of the non-real time data included in the multiplexed data (step ST21).
When the real-time data rate control unit 5 measures the average rate AVErate ′ of the non-real-time data, the real-time data rate control unit 5 outputs the average rate AVErate ′ from the real-time data encoding units 1a and 1b so as to approach the desired average rate AVErate _ref ′. Controls the amount of encoded data.

That is, when the average rate AVErate ′ of the non-real-time data is higher than the desired average rate AVErate _ref ′ (step ST22), the real-time data rate control unit 5 gives an instruction to increase the code amount of the encoded data. , 1b (step ST23).
As an instruction to increase the code amount of encoded data, for example, an instruction to increase a previously set image quality level (improve image quality) can be considered.
As a result, the average rate AVErate of the encoded data of the real-time data output from the real-time data encoding units 1a and 1b increases, so that the non-real-time data output from the non-real-time data output units 2a and 2b in the multiplexing unit 3 The rate of multiplexing is reduced. Therefore, the average rate AVErate ′ of the non-real time data can be brought close to the desired average rate AVErate _ref ′.

When the average rate AVErate ′ of the non-real-time data is lower than the desired average rate AVErate _ref ′ (step ST22), the real-time data rate control unit 5 gives an instruction to reduce the code amount of the encoded data in the real-time data encoding units 1a and 1b. (Step ST24).
As an instruction to reduce the code amount of the encoded data, for example, an instruction to lower the previously set image quality level (decrease the image quality) can be considered.
As a result, the average rate AVErate of the encoded data of the real-time data output from the real-time data encoding units 1a and 1b is lowered, so that the non-real-time data output from the non-real-time data output units 2a and 2b in the multiplexing unit 3 The rate of multiplexing increases. Therefore, the average rate AVErate ′ of the non-real time data can be brought close to the desired average rate AVErate _ref ′.

Here, when the average rate AVErate ′ of the non-real-time data is higher than the desired average rate AVErate _ref ′, an instruction to increase the code amount of the encoded data is given to the real-time data encoding units 1a and 1b, while the average of the non-real-time data is averaged In the case where the rate AVErate ′ is lower than the desired average rate AVErate _ref ′, an instruction to reduce the code amount of the encoded data is given to the real-time data encoding units 1a and 1b. When the difference from the desired average rate AVErate _ref ′ is small (when the difference is smaller than a predetermined threshold), an instruction to change the code amount of the encoded data may not be given to the real-time data encoding units 1a and 1b. Good.
In this case, since the image quality level is not changed when the average rate AVErate ′ of the non-real time data is close to the desired average rate AVErate _ref ′, the image quality can be stabilized.

Further, the real-time data rate control unit 5 receives the notification of the rate information of the non-real-time data from the multiplexing unit 3 without checking the multiplexed data output from the multiplexing unit 3, and receives the real-time data encoding unit. You may make it control the code amount of the coding data output from 1a, 1b.
Thereafter, the process returns to step ST2, and the processes of steps ST2 to ST24 are repeatedly performed.

However, when the real-time data rate control unit 5 measures the average rate AVErate ′ of the non-real-time data, the real-time data rate control unit 5 measures the average rate AVErate ′ every certain period, and the average rate AVErate ′ becomes the desired average rate AVErate _ref ′. If the code amount of the encoded data output from the real-time data encoders 1a and 1b is controlled so as to approach, the average rate AVErate ′ of the non-real-time data at that time is brought close to the desired average rate AVErate _ref ′. However, when viewed over a long period of time, the average rate AVErate ′ of the non-real-time data may deviate from the desired average rate AVErate _ref ′.

Therefore, the real-time data rate control unit 5 accumulates the data amount D of the non-real-time data included in the multiplexed data from the time T _START when the output of the multiplexed data from the multiplexing unit 3 is started, and outputs the accumulated data amount D. The elapsed time ΔT (= T _NOW −T _START ) from the start time T _START to the current time T _NOW is multiplied by a desired average rate AVErate _ref ′.
Then, the real-time data rate control unit 5 compares the accumulated value Daccum of the data amount D with the multiplication result ΔT × AVErate _ref ′ of the elapsed time ΔT and the desired average rate AVErate _ref ′, and the accumulated value Daccum of the data amount D Is larger than the multiplication result ΔT × AVErate _ref ′, an instruction to increase the code amount of the encoded data is given to the real-time data encoding units 1a and 1b.
On the other hand, if the accumulated value Daccum of the data amount D is smaller than the multiplication result ΔT × AVErate _ref ′, an instruction to reduce the code amount of the encoded data is given to the real-time data encoding units 1a and 1b.
In this way, since the accumulated value Daccum of the data amount D can be brought close to the desired total data amount, the average rate AVErate ′ of the non-real-time data can be changed to the desired average rate AVErate _ref ′ even in a long period. You can get closer.

As is apparent from the above, according to the second embodiment, among the encoded data output from the real-time data encoding units 1a and 1b and the non-real-time data output units 2a and 2b, the real-time data encoding unit 1a, The encoded data of the real-time data output from 1b is preferentially multiplexed and output from the non-real-time data output units 2a and 2b within the range of the difference between the output rate OUTrate of the multiplexed data and a predetermined transmission rate TRNrate. The non-real time data is multiplexed and output by the multiplexing unit 3, and the real time data rate control unit 5 sets the average rate AVErate 'of the non-real time data included in the multiplexed data output from the multiplexing unit 3. measured, the average rate AVErate 'desired average rate AVErate _ref Since the code amount of the encoded data output from the real-time data encoding units 1a and 1b is controlled so as to approach, the temporal variation of the image quality is suppressed while equalizing the image quality of a plurality of programs. Thus, high-quality program multiplexing can be achieved, and the average rate of non-real-time data can be guaranteed.

In the second embodiment, the real-time data rate control unit 5 causes the average rate AVErate ′ of the non-real-time data included in the multiplexed data output from the multiplexing unit 3 to approach the desired average rate AVErate _ref . As described above, the control unit controls the code amount of the encoded data output from the real-time data encoding units 1a and 1b, but the non-real-time data output units 2a and 2b multiplex the non-real-time data at a fixed rate. When the multiplexing unit 3 performs multiplexing while temporarily storing non-real-time data in an input buffer (not shown), the real-time data rate control unit 5 stores the non-real-time data in the input buffer of the multiplexing unit 3. Based on the accumulated total amount of non-real time data, it is output from the real time data encoding units 1a and 1b. It may be controlled code amount of No. data.
Also in this case, the total output data amount of the non-real time data can be kept at a desired value. In addition, it is possible to suppress a sudden change in video image quality included in the real-time data.

As shown in FIG. 6, when the difference between the total amount of actual real-time data and the total amount of desired real-time data increases, the total amount of non-real-time data also deviates from the total amount of desired non-real-time data.
For example, when it is assumed that non-real-time data is transmitted at a certain rate, the transmission timing of non-real-time data is shifted by the time obtained by dividing the difference between the actual total data amount and the assumed total data amount by the assumed rate.
In the example of FIG. 6, when the assumed rate of non-real-time data is 120 kbps, and the difference between the total amount of actual real-time data and the total amount of desired real-time data is 12 kbytes, 12k × 8 / 120k = 0.8, The transmission timing of the non-real time data is shifted from the assumed timing by 0.8 seconds.

For this reason, the real-time data rate control unit 5 performs the encoding output from the real-time data encoding units 1a and 1b so that the difference between the actual total data amount of the non-real-time data and the assumed total data amount does not exceed a certain threshold. By controlling the code amount of data, the transmission timing of non-real-time data does not vary greatly from the assumed transmission timing.
In this way, by controlling the difference between the actual total amount of non-real-time data and the estimated total amount of data to be equal to or less than a predetermined threshold, the transmission timing of non-real-time data is greatly deviated from the assumed transmission timing. Therefore, it is possible to receive at the timing assumed on the decoding device side.

In the first and second embodiments, when the real-time data encoding units 1a and 1b encode media data including video data as real-time data, for example, the real-time data is obtained so that a set image quality level is obtained. Is encoded in real time (encoding so that the video image quality is constant), and when an instruction to reduce the code amount is received from the real time data rate control units 4 and 5, the encoding process is performed so that the image quality is slightly reduced. Thus, the code amount of the encoded data is reduced. Conversely, when an instruction to increase the code amount is received from the real-time data rate control units 4 and 5, encoding processing is performed so that the image quality is slightly improved, and the code amount of the encoded data is increased.
Thus, since the real-time data encoding units 1a and 1b do not directly control the output rate of the encoded data, but control the image quality level, the real-time data rate control units 4 and 5 give instructions. If not, the image quality will be constant. Further, even if an instruction is received from the real-time data rate control units 4 and 5, the image quality is stably adjusted only slightly, so that fluctuations in image quality can be suppressed.
When the real-time data encoding units 1a and 1b do not control the image quality level but try to match the output rate of the encoded data to a desired rate, the image quality level varies depending on the complexity of the input image. Therefore, it is assumed that the image quality level becomes unstable.

In the first and second embodiments, the real-time data rate control units 4 and 5 sequentially change the real-time data encoding unit so that the average rate AVErate (AVErate ′) approaches the desired average rate AVErate _ref (AVErate _ref ′). Although the control of the code amount of the encoded data output from 1a and 1b is shown, the code output from the real-time data encoding units 1a and 1b each time the real-time data rate control units 4 and 5 perform control. The image quality of the digitized data varies.
In order to suppress fluctuations in the image quality of the encoded data output from the real-time data encoding units 1a and 1b, the number of control times and the control width of the code amount in the real-time data rate control units 4 and 5 are limited. The real-time data rate controllers 4 and 5 may control the code amount of the encoded data within the control width.

For example, when the upper limit control count “N times” for 1 minute is set as the number of times of code amount control in the real-time data rate control units 4 and 5, the control count in the real-time data rate control units 4 and 5 is 1 minute. It is limited not to exceed N times.
Further, as the control range of the code amount in the real-time data rate control units 4 and 5, for example, when the upper limit fluctuation width W of the image quality by one control is set, the control in one time in the real-time data rate control units 4 and 5 Thus, the image quality is limited so as not to exceed the upper limit fluctuation range W of the image quality.

Here, what restricts the number of times of control and the control width of the code amount in the real-time data rate control units 4 and 5 has been shown, but the real-time data rate control units 4 and 5 accumulating the data amount D of the non-real-time data from the start point in time T _sTART, only if the difference between the desired total data amount and the accumulated value Daccum the data amount D exceeds a predetermined value, the real-time data encoding unit 1a, The code amount of the encoded data output from 1b may be controlled.
Thereby, while the difference between the accumulated value Daccum of the data amount D and the desired total amount of data fluctuates within a predetermined value range, the code amount of the encoded data output from the real-time data encoding units 1a and 1b is increased. Since it is not controlled, the image quality of real-time data is stabilized. Even when the difference exceeds the range of the predetermined value, the image quality change can be made to change smoothly by changing the image quality little by little without giving control that greatly changes the image quality.

  In the present invention, within the scope of the invention, any combination of the embodiments, or any modification of any component in each embodiment, or omission of any component in each embodiment is possible. .

  1a, 1b Real-time data encoding section (real-time data encoding means), 2a, 2b Non-real-time data output section (non-real-time data output means), 3 Multiplexing section (multiplexing means), 4, 5 Real-time data rate control section (Real time data rate control means).

Claims (9)

Real-time data encoding means for encoding real-time data assumed to be decoded and reproduced in real time on the decoding device side in real time, and outputting variable rate encoded data related to the real-time data;
Non-real time data output means for outputting non-real time data assumed to be reproduced in non-real time on the decoding device side;
The encoded data output from the real-time data encoding means is multiplexed within a preset transmission rate range, and the range of the difference between the data rate of the multiplexed encoded data and a predetermined transmission rate A multiplexing means for further multiplexing the non-real time data output from the non-real time data output means and outputting multiplexed data;
The amount of encoded data included in the multiplexed data is accumulated from the time when output of multiplexed data from the multiplexing means is started, and the current time from the time when output of the multiplexed data is started. Multiply the elapsed time to reach the preset reference rate, calculate the difference between the multiplication result and the accumulated value of the data amount, and if the difference exceeds a predetermined range, A data multiplexing apparatus comprising: real-time data rate control means for giving an instruction to control the code amount of the encoded data to the real-time data encoding means so that. The real-time data rate control means gives an instruction to the real-time data encoding means to reduce the code amount of the encoded data when the difference exceeds a predetermined range and the accumulated value is larger than the multiplication result, 2. The data multiplexing apparatus according to claim 1, wherein when the accumulated value is smaller than the multiplication result, an instruction to increase the code amount of the encoded data is given to the real-time data encoding means. Real-time data encoding means for encoding real-time data assumed to be decoded and reproduced in real time on the decoding device side in real time, and outputting variable rate encoded data related to the real-time data;
Non-real time data output means for outputting non-real time data assumed to be reproduced in non-real time on the decoding device side;
The encoded data output from the real-time data encoding means is multiplexed within a preset transmission rate range, and the range of the difference between the data rate of the multiplexed encoded data and a predetermined transmission rate A multiplexing means for further multiplexing the non-real time data output from the non-real time data output means and outputting multiplexed data;
The amount of the non-real time data included in the multiplexed data is accumulated from the time when output of the multiplexed data from the multiplexing means is started, and the current time from the time when output of the multiplexed data is started Multiply the elapsed time to reach the preset reference rate, calculate the difference between the multiplication result and the accumulated value of the data amount, and if the difference exceeds a predetermined range, A data multiplexing apparatus comprising: real-time data rate control means for giving an instruction to control the code amount of the encoded data to the real-time data encoding means so that. The real-time data rate control means gives an instruction to the real-time data encoding means to increase the code amount of the encoded data when the difference exceeds a predetermined range and the accumulated value is larger than the multiplication result, 4. The data multiplexing apparatus according to claim 3, wherein when the accumulated value is smaller than the multiplication result, an instruction to reduce the code amount of the encoded data is given to the real-time data encoding means.   5. The real-time data rate control means controls a code amount of encoded data output from the real-time data encoding means so that the difference becomes a predetermined value or less. The data multiplexing device according to any one of the above. When the real-time data is media data including video data, the real-time data encoding means performs encoding so that the image quality of the video data is constant when the real-time data is encoded. The data multiplexing device according to any one of claims 1 to 5 . A real-time data encoding unit that encodes real-time data assumed to be decoded and reproduced in real time on a decoding device side in real time, and outputs variable-rate encoded data related to the real-time data Processing step,
A non-real-time data output means for outputting non-real-time data that is assumed to be reproduced in non-real time on the decoding device side;
Multiplexing means multiplexes the encoded data output by the real-time data encoding processing step within a preset transmission rate range, and a data rate of the multiplexed encoded data and a predetermined value Within the range of the transmission rate difference, a multiplexing process step for further multiplexing the non-real time data output by the non-real time data output processing step and outputting multiplexed data;
The real-time data rate control means accumulates the data amount of the encoded data included in the multiplexed data from the time when the output of the multiplexed data is started by the multiplexing processing step, and the multiplexed data Multiply the elapsed time from the start of the output to the current time by a preset reference rate to calculate the difference between the multiplication result and the accumulated value of the data amount . A data multiplexing method comprising: a real-time data rate control processing step for giving an instruction to control the code amount of the encoded data to the real-time data encoding processing step so that the difference becomes small when the range is exceeded . A real-time data encoding unit that encodes real-time data assumed to be decoded and reproduced in real time on a decoding device side in real time, and outputs variable-rate encoded data related to the real-time data Processing step,
A non-real-time data output means for outputting non-real-time data that is assumed to be reproduced in non-real time on the decoding device side;
Multiplexing means multiplexes the encoded data output by the real-time data encoding processing step within a preset transmission rate range, and a data rate of the multiplexed encoded data and a predetermined value Within the range of the transmission rate difference, a multiplexing process step for further multiplexing the non-real time data output by the non-real time data output processing step and outputting multiplexed data;
The real-time data rate control means accumulates the data amount of the non-real-time data included in the multiplexed data from the time when output of the multiplexed data is started by the multiplexing processing step, and the multiplexed data Multiply the elapsed time from the start of the output to the current time by a preset reference rate to calculate the difference between the multiplication result and the accumulated value of the data amount . A data multiplexing method comprising: a real-time data rate control processing step for giving an instruction to control the code amount of the encoded data to the real-time data encoding processing step so that the difference becomes small when the range is exceeded . A data multiplexing device according to any one of claims 1 to 6 ,
The multiplexed data output from the data multiplexer is received, the real-time data is decoded from the encoded data multiplexed in the multiplexed data, the real-time data is reproduced, and the multiplexed data is converted into the multiplexed data. A decoding device for reproducing multiplexed non-real-time data ;
System with.

Images