JPH0970041A - Variable bit rate coder - Google Patents

Abstract

PROBLEM TO BE SOLVED: To obtain a variable bit rate coder by obtaining a subject evaluation value corresponding to a coding bit rate, conducting prescribed processing based thereon, and coding optimizingly the entire image quality while keeping the average bit rate to be a prescribed value. SOLUTION: A variable bit rate coding section 2 encodes digital image data 1 for a prescribed time length for each variable bit rate so that the average bit rate is a specified value or below. A subject evaluation section 4 obtains a subject evaluation value corresponding to the different coding bit rate for each unit time being division of the digital image data 1 for a prescribed time. A bit rate allocation section 5 decides the bit rate for each unit so that the average bit rate is an object rate or below based on the obtained subject evaluation value and the desired evaluation of the entire image quality is optimum. Thus, optimum bit rate arrangement is attained with respect to image data in each unit time under a prescribed condition.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a variable bit rate coding device, and more particularly, in coding image data, a variable bit which is capable of optimally coding the entire image quality while keeping an average bit rate at a predetermined value. The present invention relates to a rate coding device.

[0002]

2. Description of the Related Art Conventionally, various systems for compressing image data have been developed in order to record or transmit image data such as television images with a small amount of information. For example, MPE
In the variable bit rate coding method such as G, an image can be coded at an arbitrary bit rate. Also,
For DVD (digital video disc) and DVR (digital video recorder) that record image data,
Due to restrictions on hardware or software, the average coding bit rate needs to be maintained at a predetermined value, for example, 5 Mbps every unit time, and in addition to the maximum bit rate (for example, 8 Mbps) and the minimum bit rate (for example, 8 Mbps). For example, 1 Mbps) is set. Therefore, how to allocate the bit rate to the image data for each unit time under such a constraint becomes a problem.

Conventionally, for example, in the 1994 Television Society Annual Meeting Proceedings 18-1 (pages 285 to 286), image data of each unit time is provisionally used by using a quantized value so as to obtain a predetermined image quality. After encoding, measure the generated code amount, and for each image data, within the range of the highest and lowest bit rates, the average bit rate is below a specified value, and it is almost proportional to the code amount at the time of temporary encoding. A method of allocating such a bit rate has been proposed.

[0004]

The relationship between the bit rate (code amount) and the S / N ratio (image quality) of image data differs depending on the characteristics of the image. For example, in the case of a simple geometric pattern, if the bit rate exceeds a certain level, the S / N ratio does not change much with respect to the change of the bit rate, whereas the complicated landscape image etc. has the S / N ratio with respect to the change of the bit rate. The change in N ratio is large. However, in the above-mentioned conventional variable bit rate encoding device, the degree of change in image quality when the encoding bit rate is changed is not taken into consideration, and the bit rate allocation is not necessarily optimum. There was a problem. There is also a problem of how to obtain the image quality evaluation value by the temporary encoding. In view of the above-mentioned conventional technique, an object of the present invention is to provide a variable bit rate encoding device capable of optimally encoding the entire image quality while keeping the average bit rate at a predetermined value.

[0005]

In order to achieve the above-mentioned object, the present invention is an apparatus for encoding digital image data of a predetermined time length at a variable bit rate so that the average bit rate is below a specified value. In the above, in each unit time obtained by dividing the digital image data into predetermined time intervals, evaluation value calculation means for respectively obtaining subjective evaluation values corresponding to different encoding bit rates, and based on the obtained subjective evaluation values, the average bit It is characterized by further comprising a bit rate determining means for determining a bit rate for each unit time so that the rate becomes equal to or lower than a specified value and a desired evaluation value of the overall image quality becomes optimum.

According to the present invention, the digital image data is divided at predetermined time intervals, and the subjective evaluation values corresponding to different encoding bit rates are calculated for the images at each unit time. Then, based on the obtained subjective evaluation value, the average bit rate becomes less than or equal to the specified value,
Moreover, since the bit rate for each unit time is determined so that the desired evaluation value of the overall image quality is optimal, it becomes possible to optimally allocate the bit rate to the image data of each unit time under predetermined conditions. For example, the average image quality of DVD can be improved.

[0007]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS The present invention will be described below in detail with reference to the drawings. FIG. 1 is a functional block diagram showing the functions of the variable bit rate coding apparatus of the present invention. In FIG. 1, the dotted line shows the data flow at the time of bit rate allocation processing, and the solid line shows the data flow at the time of actual encoding. First, at the time of bit rate allocation processing, a synchronization marker is added to the source image data read by the subjective evaluation unit 4 from the image data storage device 1 that stores the source image data, and the variable bit rate encoding unit 2 is added with the synchronization marker. Is entered. The variable bit rate coding unit 2 performs, for example, known MPEG1 or MPEG2 coding, and performs motion compensation prediction processing, DCT (discrete cosine transform), quantization, and variable length coding on input image data. To do. The variable bit rate encoding unit 2 can perform encoding at an arbitrary bit rate (code amount) by externally controlling the quantization table value. During the bit rate allocation process, the encoded image data is decoded again and output to the subjective evaluation unit in the same data format as the source image data.

The subjective evaluation unit 4 has a configuration described later to compare the source image data with the encoded / decoded image data, and outputs a subjective evaluation value approximate to the human visual evaluation. The subjective evaluation unit 2 outputs a plurality of subjective evaluation values when the source image data for each unit time is encoded and decoded at different bit rates. The bit rate allocation unit 5 reads the evaluation value table from the data storage device 6 that stores the evaluation value table output from the subjective evaluation unit 4, and sequentially allocates the bit rate of each layer described below by the method described below. Go. At the time of main encoding of the image data, the variable bit rate encoding unit 2 encodes the source image data of each unit time using the quantization table based on the bit rate allocation data accumulated in the data accumulating device 7, Output to the data storage device 3. The image data are all digital signals, and all processes are executed by the computer.

FIG. 2 is an explanatory diagram showing the structure of image data. For example, the entire image data recorded on a DVD or the like needs to be divided into units U for each predetermined unit time T (for example, several seconds), and the average bit rate bav needs to be realized in each unit U. Each unit U has N GO
It is composed of P (group of pictures), and a bit rate suitable for image characteristics is allocated to each GOP. The bit rate to be distributed has an upper limit value bh and a lower limit value be. Therefore, in each GOP, the average rate in the unit U is bav in the range of bh to be,
And, for example, the bit rate is assigned so that the average image quality is the best. Each GOP is made up of n frames P, and each frame P has a hierarchical structure made up of m segments S. Each frame P and segment S is assigned a GOP by the same method as the allocation to the GOP. Bit rates that match the image characteristics are sequentially distributed.

FIG. 3 is a block diagram showing the configuration of the subjective evaluation section 4. The subjective evaluation unit 4 includes a first input unit 11 to which source image data is input, a second input unit 12 to which a decoded output signal of the variable bit rate encoding unit 2 is input, a synchronization marker addition unit 13, and a synchronization control unit. 14, delay unit 15, output unit 16, first orthogonal transform (for example DCT / WHT) calculation unit 17, second orthogonal transform (for example DCT / WHT) calculation unit 18, subtraction unit 19, WSNR calculation unit 20, subjective Evaluation value calculator 2
1 and the control unit 22. Here, the WH
T represents Hadamard transform which is one of orthogonal transforms, and WS
NR indicates a weighted S / N ratio.

Next, the operation of the subjective evaluation section 4 will be described. First, the original image data from the video source 1 is input to the first input unit 11 and output to the synchronization marker adding unit 13. The synchronization marker adding unit 13 adds a synchronization marker to the input digital image data. The data to which the synchronization marker is added is sent to the variable bit rate encoding unit 2 via the output unit 16. The variable bit rate encoding unit 2 encodes the input digital data by, for example, the MPEG2 system, then decodes it, and inputs it to the second input unit 12. The reproduced image data output from the second input unit 12 is input to the synchronization control unit 14 and is also sent to the first orthogonal transform calculation unit 17. The synchronization control unit 14 controls the delay amount of the delay unit 15 so that the original image data delayed by the delay unit 15 and the marker of the reproduced image data match. This delay amount is, for example, one frame or several frames. As a result, the reproduction image data input to the first orthogonal transform calculation unit 17 and the original image data input to the second orthogonal transform calculation unit 21 can be accurately synchronized. If it is not necessary to process the input data in real time, synchronization control is unnecessary.

When the original image is a component television signal, it is preferable that the first and second orthogonal transform calculation units 17 and 18 perform DCT transform in block units.
The subtractor 19 obtains the error value of the same coefficient in the same block. Based on the error value obtained from the subtracter 19, the WSNR calculation unit 20 calculates the WSNR reflecting the human visual characteristics. This WSNR is calculated by the subjective evaluation value calculation unit 21.
Convert to relative subjective evaluation value.

Next, the operations of the first and second orthogonal transform operation units 17 and 18 and the subtractor 19 will be described with reference to FIG. FIG. 4A shows an original image frame input to the first orthogonal transform calculation unit 17, and FIG. 4B shows a reproduced image frame input to the second orthogonal transform calculation unit 18. In these figures, c indicates the number of frames, and c = 3 (that is, T = 3) in the case of a component television signal. First
The orthogonal transformation calculation unit 17 performs orthogonal transformation on all pixels of the original image frame in block units. FIG. 6C shows the coefficient xc (bc, m, n) after the orthogonal transformation. Where bc is the position of the block in the original image frame,
m is the number in the main scanning direction in the block, and n is the number in the sub scanning direction. On the other hand, the second orthogonal transformation calculation unit 18 orthogonally transforms all the pixels of the reproduced image frame in block units. FIG. 6 (d) shows the coefficient yc (bc
, M, n) are shown. The subtracter 19 obtains the error value of the same coefficient in the same block output from the first and second orthogonal transform calculation units 17 and 21. This error value is
xc (bc, m, n) -yc (bc, m, n).

Next, the operation of the WSNR calculator 20 will be described. The average weighted S / N ratio (WSNR) of one frame is obtained from the following equation (1).

[0015]

[Equation 1] Where q ² is the average weighted noise for one frame,
It can be expressed by the following equation (2). Further, since each pixel of the television signal is represented by 8 bits, its peak value of 255 is the numerator value of the expression (1).

[0016]

[Equation 2] In the equation (2), hc ² (σc ² (bc), m, n) is 1
The visual sensitivity at each position in the block is represented, and σc ² represents the total AC power in the block bc.
In other words, σ c ² represents the degree of noise masking effect. In general, human visual sensitivity is high for images with little movement such as tree trunks and thick branches, and small for images with lots of movement such as leaves. Therefore, if hc ² is taken as the vertical axis and m ² + n ² is taken as the horizontal axis,
As shown in FIG. 5, hc ² takes a value close to 1 when m ² + n ² is small, and takes a value close to 0 when m ² + n ² is large. Also, when σc ² is small, the visual sensitivity is high, and σ
When c ² is large, the visual sensitivity is small.

The following expressions (3) and (4) can be given as specific examples of the visual sensitivity hc and the position D of the DC component in the component television signal.

[0018]

(Equation 3) Next, the subjective evaluation value calculation unit 21 calculates and outputs the subjective evaluation value based on the weighted S / N ratio (WSNR) calculated by the WSNR calculation unit 20. The subjective evaluation value calculation unit 21
Evaluation characteristic Z = f (WSNR) as shown in FIG.
From this, for example, a relative subjective evaluation value in a 5-step evaluation is obtained. The evaluation characteristic Z is a characteristic actually empirically obtained by using the person under test. The subjective evaluation unit 4 can mechanically determine the subjective evaluation value in a short time without using the person to be tested. The control unit 22 of the subjective evaluation unit 4 controls the quantization table (quantization step) of the variable bit rate encoding unit 2, and for the same unit time image data, for example, the maximum bit rate bh and the average value. bav, the minimum value b
Each WSNR corresponding to e is obtained.

FIG. 10 shows an example of the obtained evaluation value table. For each GOP having N of 1 to 5, the maximum bit rate is 4 Mbps and the average bit rate is 3 Mb.
Three subjective evaluation values (five-step evaluation) corresponding to ps and the minimum bit rate of 2 Mbps are stored.

Next, the processing of the bit rate allocation section 5 will be described. FIG. 7 is an explanatory diagram showing combinations of selectable bit rates in each GOP. For example, each GO
If the number of selectable bit rates is K between the maximum value bh and the minimum value be of P in P, the number of combinations when an arbitrary bit rate is selected in each GOP is an enormous number of K powers. Becomes a number.
Therefore, it is difficult to calculate the average bit rate and the average image quality for all combinations, and select the one having the average bit rate equal to or lower than the condition value and having the best average image quality. Therefore, the Viterbi algorithm is used to determine the optimum combination.

FIG. 8 is a trellis diagram for explaining a method of determining an optimum path (combination), and FIG. 9 is a flow chart showing a path determining process executed in the bit rate allocation unit 5. First, in step S10 of FIG. 9, the range of the next node is determined. Each G
At a depth n, which is a number corresponding to OP, a predetermined range of the cumulative rate (average rate) up to the n is divided into units of Δ (n) to form nodes. Δ (n) is determined as follows. Assuming that the cumulative rate up to the depth n is an average value and assuming (bav * n), the range of the cumulative rate at the node of depth (n + 1) is (bav * n + be).
)-(Bav * n + bh).

When this is normalized by (n + 1), (bav *
n + be) / (n + 1) to (bav * n + bh) / (n +
1), that is, bav- (bav-be) / (n + 1) to bav
It becomes + (bh-bav) / (n + 1). Here, the range that can be taken by the node centering on bav is the range (range)
Using a parameter M (eg M = 2) that controls
(Bav-be) M / (n + 1) to (bh-bav) M /
(N + 1). Then, this section is equally divided into 2L.
Let (n) be each node at depth n. That is, Δ (n) = (bh-be) M / (n + 1) 2L. It should be noted that L is a parameter for controlling accuracy (for example, L = 5), and when M and L are each set to a predetermined value or more, there are values that do not change the result, so that value is adopted.

In step S11 of FIG. 9, a path calculation is performed. First, among the nodes of depth n, the node corresponding to the selectable bit rate, for example, the node at the top of depth 1 in FIG. 8 is selectable at depth n + 1 (depth 2 in FIG. 8). The cumulative rate X in the case where any one of the bit rates is selected is calculated. Then, the cumulative evaluation value Y is calculated only for the case where the X is in any of the node ranges of the depth 2. In addition,
At depth n, X and Y are, for example, X = (1 / n)
Σb, Y = (1 / n) ΣD, and Σb and ΣD are
It is the sum of 1 to n of the selected bit rate and the corresponding subjective evaluation value at each depth.

In step S12, at each node of depth n + 1, the highest cumulative evaluation value Y of the plurality of paths reaching the range of its own node is left, and the other paths are discarded. In FIG. 8, for example, each node having a depth of 1 reaches a node in the range of Δ (1) to 2Δ (1) having a depth of 2 (the cumulative rate X is within the range) 3
The cumulative evaluation value Y is compared with respect to the book path. For example, the upper path having the highest evaluation value is left and the lower two paths are discarded. Therefore, at each depth n, only a maximum of 2L paths are left, and the number of paths to be calculated is limited. Thus, even if some paths are discarded, if L is set to be sufficiently large and Δ (n) is set to be small, the cumulative evaluation value of the paths surviving at this point is the cumulative evaluation of the paths discarded at this point. The optimum path can be selected without reversing the value.

In step S13, the depth n is N-
It is determined whether 1 has been reached, and if the result is negative, n
Is incremented by 1 and the process returns to step S10 to sequentially select the surviving path at each depth. If the determination result is affirmative, the process proceeds to step S14. In step S14, among the paths that survived at the final depth N, among the paths where the cumulative bit rate X satisfies a predetermined condition (the average bit rate is less than or equal to the target value), the cumulative evaluation value Y Choose the highest one. Note that the bit rate allocation to each frame P and segment S in FIG. 2 is processed in order from the higher order by the same method as the allocation to the GOP. For example, when allocating to each frame in the GOP, the average bit rate corresponds. The path is selected to be equal to the bit rate assigned to the GOP.

FIG. 11 is a trellis diagram showing paths selected under predetermined conditions based on the evaluation table example of FIG. In FIG. 10, three subjective evaluation values (five-step evaluation) corresponding to the maximum bit rate of 4 Mbps, the average bit rate of 3 Mbps, and the minimum bit rate of 2 Mbps are accumulated for each GOP having n of 1 to 5. There is. Here, the bit rate selectable at each depth is set to either 2, 3, or 4 Mbps, and the trellis diagram parameters M = 2 and L = 5. Then, the paths reaching the next node are limited to those described in FIG. 11, and among the paths reaching each node, only the circled ones (three for each depth) survive.

As for the paths reaching the depth 5, only those having an average bit rate of 3 Mbps or less are described, and the thick line path having the largest cumulative evaluation value Y is selected from them. Therefore, the bit rate in each GOP is determined to be 4, 2, 2, 4, 3 Mbps in order. When the cumulative evaluation function Y that maximizes the worst subjective evaluation value is adopted, the bit rates assigned to the GOPs are 3, 4, 2, 4, 2 Mbps in order.

Although the embodiment has been disclosed above, the following modifications are also possible. When the variable bit rate encoding unit can perform encoding at an arbitrary bit rate, such as the MPEG system, it is possible to divide the selectable bit rate into small pieces, but for each image data, selectable It takes a lot of time to obtain the subjective evaluation values of all bit rates by the subjective evaluation unit. Therefore, for example, the evaluation values at several places such as the upper limit value, the lower limit value, and the average value of the bit rate are obtained, and the function of the bit rate and the subjective evaluation value is obtained from the evaluation values. Then, the function may be used to obtain an evaluation value corresponding to an arbitrary bit rate.

In the embodiment, in the subjective evaluation section,
Although the evaluation value is automatically obtained, for example, by a method similar to the conventional method, a subjective evaluation test of an image by a person under test is performed, and a bit rate is assigned using the evaluation value table obtained as a result. Good. In the example,
Although the sum or average of the subjective evaluation values is used as the cumulative evaluation value, other evaluation functions may be used. For example, as the evaluation function, the worst evaluation value, that is, the worst value of the evaluation values corresponding to the bit rate selected in each node, or the reciprocal of the variance of the subjective evaluation value is adopted, and the path having the highest value is selected. You may. Further, the determination may be made by combining a plurality of conditions, and for example, the worst value may be a predetermined value or more and the average value may be the largest.

In the path determination process in the bit rate allocating unit, while the depth n is small, a path in which the accumulated bit rate X largely deviates from the average value bav may be the optimum path, but the depth n is to some extent. When it grows up,
Since there is almost no possibility that the path in which the cumulative bit rate X greatly deviates from the average value bav is the optimum path, the value of M (and the accuracy L) that determines the range of the node is increased first and then gradually decreased. You may do it.

[0031]

As described above, according to the present invention,
For digital image data for each unit time, subjective evaluation values corresponding to different encoding bit rates are obtained by calculation, and based on the obtained subjective evaluation values (functions),
The bit rate for each unit time is determined so that the average bit rate is less than or equal to the specified value and the desired evaluation value for the overall image quality is optimal. On the other hand, the optimum bit rate can be distributed, and the average image quality of a DVD can be improved, for example.

[Brief description of drawings]

FIG. 1 is a functional block diagram showing functions of a variable bit rate coding device of the present invention.

FIG. 2 is an explanatory diagram showing the structure of image data.

FIG. 3 is a block diagram showing a configuration of a subjective evaluation unit 4.

FIG. 4 is an explanatory diagram of operations of first and second orthogonal transform calculation units and a subtractor.

FIG. 5 is a graph showing a relationship between a position within a block and visual sensitivity.

FIG. 6 is a graph showing a relationship between WSNR and an evaluation value.

FIG. 7 is an explanatory diagram showing combinations of selectable bit rates in each GOP.

FIG. 8 is a trellis diagram for explaining a method of determining an optimum path.

FIG. 9 is a flowchart showing a path determination process.

FIG. 10 is an explanatory diagram showing an example of an evaluation table.

FIG. 11 is a trellis diagram showing paths selected based on the example of the evaluation table of FIG.

[Explanation of symbols]

1 ... Source image data storage device, 2 ... Variable bit rate coding unit, 3 ... Coded image data storage device, 4 ... Subjective evaluation unit, 5 ... Bit rate allocation unit, 6 ... Evaluation value table storage device, 7 ... Bit Rate allocation data storage device, 11
... 1st input part, 12 ... 2nd input part, 13 ... Synchronization marker addition part, 14 ... Synchronization control part, 15 ... Delay part, 16 ...
Output unit, 17 ... First orthogonal transform calculation unit, 18 ... Second orthogonal transform calculation unit, 19 ... Subtraction unit, 20 ... WSNR calculation unit,
21 ... Subjective evaluation value calculation unit, 22 ... Control unit

Claims (4)

[Claims] 1. Digital image data of a predetermined time length,
In a device that encodes at a variable bit rate so that the average bit rate is equal to or less than a specified value, in each unit time that divides the digital image data into predetermined time intervals, subjective evaluation values corresponding to different encoding bit rates Based on the obtained evaluation value calculation means and the obtained subjective evaluation value, the encoding bit rate for each unit time is set so that the average bit rate is equal to or less than the specified value and the evaluation value of the overall image quality is optimum. A variable bit rate coding device, comprising: 2. The subjective evaluation value calculation means synchronizes the original image data with the reproduced image data obtained by once encoding / decoding the original image data, and the synchronized original image data. Means for orthogonally transforming blocks of reproduced image data, means for calculating an error of the same coefficient of the orthogonally transformed data, position of the coefficient of the orthogonally transformed data, after orthogonal transformation in the original image The error is weighted by a weighting function that changes depending on the magnitude of the AC power in the block of
Then, a variable bit rate code including means for obtaining an average weighted S / N ratio for each frame or for each of a plurality of frames, and means for obtaining a subjective evaluation value using the weighted S / N ratio. Device. 3. A predetermined image quality evaluation function has a maximum value from a combination in which any one of the encoding bit rate values selectable for each unit time is selected by the bit rate determining means. The variable bit rate coding apparatus according to claim 1 or 2, wherein such a combination of coding bit rates for each unit time is determined. 4. The image quality evaluation function is one of an average value of subjective evaluation values, a worst subjective evaluation value, and a reciprocal of the variance of the subjective evaluation values, and the bit rate determining means uses a Viterbi algorithm, The variable bit rate encoding device according to claim 3, wherein a combination of encoding bit rates for each unit time is determined.