JPH03293882A - Image encoder - Google Patents

Abstract

PURPOSE:To perform picture quality control considering the spatial frequency characteristic of human eyes by performing encoding by frequency-separating an image signal, and deciding the quantization characteristic of each frequency component considering the spatial frequency characteristic of the human eyes. CONSTITUTION:A frequency separating means 31 separates an input image signal to plural frequency components, and supplies them to encoders 340-34n in accordance with each frequency component, and the each of the encoders 340-34n encodes the signal of an inputted frequency component according to the quantization characteristic supplied from a quantization characteristic selecting part 35. The quantization characteristic selecting part 35 distributes, designated total allowable distortion quantity to each frequency component by utilizing frequency characteristic information of visual sense, and the quantization characteristic to be supplied to each of the encoders 340-34n is selected from distributed distortion quantity. In such a manner, the picture quality control considering visual sense characteristic can be performed.

Description

DETAILED DESCRIPTION OF THE INVENTION [Field of Industrial Application] The present invention relates to an image encoder, and is particularly applicable to a variable rate moving image encoder that performs image quality control.

[Prior Art] Conventionally, there is a variable rate video encoder that performs image quality control, as shown in FIG. Information and Communication Engineers Study Group Materials, IE87-83.1) D53-60
>.

In FIG. 2, an image signal inputted through an input terminal 11 is converted into an encoding processing format by a preprocessing section 12, and then provided to a subtracter 13 as an input to be subtracted.

The subtracter 13 receives the image signal Sin after format conversion.
A prediction signal sp for the subtractor 13 is provided as a subtraction input, a prediction error signal e is obtained by the subtracter 13, and this prediction error signal e is provided to the variable quantizer 14. The variable quantizer 14 performs quantization according to quantization characteristics such as a quantization step size and the number of quantization bits given from a quantization characteristic selection section 27, which will be described later, and then immediately dequantizes it to produce a locally reproduced prediction error signal ek. is obtained and applied to the entropy encoder 15 for prediction error information. The entropy encoder 15 performs Huffman encoding, run-length encoding, or modified Huffman encoding on the reproduced prediction error signal ek, and supplies the encoded signal to a multiplexer, a packet assembler, etc. (not shown) via an output terminal 16.

In addition, the local reproduction prediction error signal e from the variable computerizer 14 is
k is given to an adder 17. The adder 17 is also supplied with the prediction signal SD of the image signal described above, and the adder 17 outputs a locally reproduced image signal 5ink. This locally reproduced image signal 5ink is transmitted to the 1-pixel delay circuit 1
8 and one frame delay circuit 19 having a memory configuration. The 1-pixel delay circuit 18 forms a predicted signal (intra-frame prediction) of the image signal by delaying one pixel, while the 1-frame delay circuit 1-9 forms a predicted signal of the image signal by delaying only one frame. (interframe prediction). These prediction signals are given to a switch circuit 20 and a prediction selection section 21 having a two-manufacturing one-output configuration. The prediction selection unit 21 compares these signals and selects the prediction signal determined to have a smaller prediction error by switching the switch circuit 2.
Controls 0.

The predicted signal (Sp) selected by the switch circuit 20 in this manner is applied to the above-mentioned subtracter 13 in the next frame.

In this conventional example, motion compensation is performed for interframe prediction. The image signal Sin passed through the preprocessing section 12 is given to a motion vector detection circuit 22 to detect the amount of motion MV. This motion amount MV is given to the 1-frame delay circuit 19, and the 1-frame delay circuit 19 receives the reproduced image signal 5in.
Writing or reading of k is controlled to perform motion compensation for the predicted signal. Note that the motion amount MV is calculated by the DPCM encoder 2.
3. The entropy encoder 24 and the output terminal 25 are sequentially operated to encode the signal, and then the signal is provided to a multiplexer, a packet assembler, etc. (not shown). The entropy encoder 24 performs Huffman encoding or run-length encoding.

This conventional example further includes a configuration that compensates for the S/N at the minimum regardless of the image content. The image signal Sin and the locally reproduced image signal 5ink are provided to the S/N calculation section 26. Based on these signals, the S/N calculation unit 26 calculates the S/N for each block obtained by dividing one image vertically and horizontally, and provides the calculated S/N to the quantization characteristic selection unit 27. The quantization characteristic selection section 27 selects a quantization characteristic for each program and sends it to the variable quantizers 1 and 4 so that S/''N of each block is approximately equal to a constant value. This quantization characteristic information is, for example, fixed-length coded by the entropy encoder 28, and is given to a multiplexing unit, bucket/node assembly unit, etc. (not shown) via two output terminals.

[Problems to be Solved by the Invention] As described above, according to the conventional video encoder, it is possible to perform encoding so as to compensate for the minimum S/N ratio, regardless of the input image content.

By the way, images appeal to the human sense of sight. Therefore, it is desirable to take human visual characteristics into consideration when encoding images. However, S/''N, which is an evaluation value of image quality in a conventional image encoder, is not necessarily proportional to the subjective evaluation value of a human. It can be said that it has not been taken into account.

The present invention has been made in consideration of the above points, and it is an object of the present invention to provide an image encoder that performs encoding so that image quality can be controlled in consideration of human visual characteristics.

[Means for solving the problem] In order to solve the problem, the image encoder of the present invention has the following features:
It has the following components.

That is, a frequency separation means that separates an input image signal into a plurality of frequency components (including at least a DC component), and a plurality of encoders corresponding to each frequency component that encodes each separated frequency component. , quantization that uses visual frequency characteristic information to distribute the specified allowable total distortion amount to each frequency component, and selects the quantization characteristic to be given to each encoder from the distributed distortion amount. and a characteristic selection section.

Note that the encoding method used by each encoder does not matter.

[Operation] In the present invention, the frequency separation means separates the input image signal into a plurality of frequency components and supplies the separated image signals to encoders corresponding to each frequency component. Each encoder encodes the input frequency component signal according to the quantization characteristic given from the quantization characteristic selection section.

The present invention is characterized by a method for selecting a quantization characteristic in a quantization characteristic selection section. The quantization characteristic selection unit distributes the specified allowable total distortion amount to each frequency component using visual frequency characteristic information, and selects a quantization characteristic to be given to each encoder from the distributed distortion amount. let them choose.

In this way, it becomes possible to control image quality in consideration of visual characteristics.

[Example] Hereinafter, an example of the present invention will be described in detail with reference to the drawings.

Here, FIG. 1 shows the configuration of this embodiment.

In FIG. 1, an image signal S input from an input terminal 30 is filtered by a QMF (Quadr
ture)lirror Filter)Bank 3
1 is given. By this QMF bank 31, the input image signal S is divided into n subband signals S having the same bandwidth.
i (i is 1 to n). Here, it is assumed that the center frequency 1 of each subband signal Si has the relationship shown in the following equation (1).

Subband signal S1 containing the lowest frequency component is provided to DC component calculation section 32 and subtracter 33. Other subband signals S2-sn are provided to corresponding encoders 342-34n, respectively. Moreover, the center frequency ct>i of each subband signal Si is determined by the quantization characteristic selection unit 3
given to 5.

The DC component calculation unit 32 calculates the DC component SO from the subband signal S1 containing the lowest frequency component. For example, the DC component SO is calculated according to the following equation 8% (2). Here, Slk is the subband signal S
is a sample value within 1, and M is the number of samples used for calculation.

The DC component SO obtained by the DC component calculation unit 32 in this manner is provided to the encoder 340 and the subtracter 33.

The subtracter 33 subtracts the DC component SO from the subband signal S1 from the QMF bank 31, and provides the encoder 341 with the lowest frequency subband signal Sla from which the DC component has been removed.

Each encoder 34j (j is 0 to n) performs encoding based on a quantization characteristic provided by the quantization characteristic selection section 35, for example, consisting of a quantization bit number Rj and a step size Δj. The encoded signal Ij thus obtained is outputted to a multiplexer, a packet assembler, etc. (not shown) via the corresponding output terminal 36j.

Here, from the lowest subband signal S1 to the DC component S
The reason why the DC component SO is separately encoded and transmitted after obtaining O is that the DC component SO is the most important component for the image signal, and as described later, this DC component SO
This is because the amount of distortion DO with respect to O can be minimized. In addition, the reason why the DC component SO of the lowest subband signal S1 is removed and then encoded and transmitted is because the DC component SO is separately encoded and transmitted with a predetermined amount of distortion taken into consideration. This is because it is possible to allocate different distortion amounts to the subband signal S1a from which the DC component SO has been removed. Since the subband signal S1 has the highest level, the DC component SO is obtained from this subband signal S1.

The quantization characteristic selection unit 35 determines the quantization characteristics Rj, Δj for each encoder 34j in a software manner (according to a predetermined algorithm), but functionally, it Consisting of a distortion amount calculation unit 35b and a quantization characteristic calculation unit 35e, it calculates quantization characteristics Rj, Δj for each encoder 34j in consideration of visual spatial frequency characteristics.
Determine.

The total distribution ratio determining unit 35a converts the allowable total average distortion iD, which is given via the input terminal 39, into distortion amounts D1,
D2, ..., Dn (DI is subband signal Sla
distortion distribution ratios W1, W2 for distributing to
, . . . determines Wn.

Originally, it would be necessary to distribute the total average distortion to the DC component SO, but as mentioned above, the DC component SO is the most important component for the image signal, so
It is necessary to keep this amount of distortion very small, so
In this embodiment, the distortion distribution ratio WO or the distortion amount DO for the DC component SO is taken in from the input terminal 40 and set.

Therefore, as described above, the distortion distribution rate determination unit 35a outputs each subband signal Sla, S2, . . . other than the DC component SO.
Strain distribution ratios W1, W2, . . . for sn.

Determine wn.

The distortion distribution rate determination unit 35a has a built-in table that associates, for example, frequencies with relative sensitivity values for those frequencies on a contrast sensitivity curve that indicates visual spatial frequency characteristics, and stores each subband signal si. When the center frequency ωi is given, a relative sensitivity value with respect to the center frequency ωi is obtained and this is multiplied by a predetermined value to obtain the distortion distribution ratio Wi. Each distortion distribution rate Wi is given to the distribution distortion amount calculation section 35b.

In addition, each distortion distribution rate Wi is expressed by the following formula: % formula % (3) There is a relationship shown in

The distributed distortion amount calculation unit 35b executes the calculation shown in the following formula (%) (4), and calculates each subband signal Sl (S1a) from the total average distortion measurement and each distortion distribution rate Wi.
, S2, ..., distributed distortion amounts D1, D2, for Sn
..., find Dn. When information regarding distortion is set in the form of distortion distribution ratio WO for DC component SO, equation (4) is also executed for this distortion distribution ratio WO.

I understand that.

Each distributed distortion amount Di (including the distortion amount DO in some cases) obtained by the distributed distortion amount calculation section 35b in this way is given to the quantization characteristic calculation section 35c.

The quantization characteristic calculation unit 35c calculates the DC component SO and each subband signal Sla, S2, . . . , s as follows.
Quantization characteristic Rj for n (j is 0.1 (la), 2~
n), determine Δj. Note that the following determination method ignores overload distortion that occurs when the input to the quantizer exceeds the quantization range.

If the range of the input signal Dividing by 2RJ gives the step size Δj, so the following formula (6) holds true. Further, the distributed distortion amount Dj has a functional relationship with the step size, and the distortion amount is changed to the specified distributed distortion amount D.
When the step size for j is expressed as Δ0PTJ, the following formula % formula % (7) holds true.

Therefore, the maximum step size Δj that satisfies the following equation (8) and the number of quantization bits Rj corresponding to this maximum step size Δj are determined. In practice, the optimum step size Δj and the optimum number of quantization bits Rj are determined by varying the number of quantization bits Rj. Therefore, although the illustration of the signal line is omitted, the signal input to the quantizer in each encoder 34j is given to the quantization characteristic calculating section 35c.

Each of the quantization characteristics Δj, Rj determined in this manner is provided to a corresponding encoder 34j and used for encoding.

Further, each of the determined quantization characteristics Δj and Rj is supplied to the encoder 37 and encoded itself, and the encoded signal J is sent to a multiplexing unit and a packet assembling unit (not shown) via the corresponding output terminal 38. etc. is output. This encoded transmission is performed in order to be appropriately decoded on the decoder side.

Therefore, according to the above-described embodiment, an encoded image signal that takes into consideration the spatial layer wavenumber characteristics of the human eye can be obtained, and image quality control that is proportional to the human subjective evaluation value is possible.

Although image encoders that divide an image signal into subband signals and encode each subband signal have existed in the past, conventional ones do not take visual characteristics into consideration, and It also does not have an S/N control structure like an encoder. An image decoder compatible with such a conventional image encoder can be directly applied as an image decoder compatible with the image encoder of the embodiment.

Note that although the encoding method in each encoder 34j was not mentioned in the above embodiment, the encoding method is not limited to a specific encoding method, and various types such as interframe encoding and intraframe encoding can be used. Encoding methods can be applied.

Further, in the above-described embodiment, the number of quantization bits and the step size are shown as the quantization characteristics that are varied according to the visual characteristics, but other parameters such as switching between linear and non-linear, etc. may be used.

Furthermore, in the above embodiment, Q
Although the MF bank is shown, other band division filters may be used, and unlike the embodiment, a two-dimensional filter may be applied.

In the above embodiment, the contrast sensitivity curve is used as the human visual characteristic, but the quantization characteristic may be changed depending on other visual characteristics (subjective evaluation method) such as the brightness characteristic. The quantization characteristics may be changed by comprehensively considering the visual characteristics of the image.

Moreover, the range of such a signal is set in advance and the quantization characteristic is calculated without inputting the signal input to the quantizer in each encoder 34j to the quantization characteristic calculating section 35c. Also good.

[Effects of the Invention] As described above, according to the present invention, an image signal is separated into frequencies, and the quantization characteristics for each frequency component are determined and encoded in consideration of the spatial frequency characteristics of the human eye. As a result, an encoded image signal that takes into consideration the spatial frequency characteristics of the human eye can be obtained, and image quality control that is proportional to the human subjective evaluation value is possible.

[Brief explanation of the drawing]

FIG. 1 is a block diagram showing an embodiment of an image encoder according to the present invention, and FIG. 2 is a block diagram showing a conventional image encoder. 31... QIVIF bank, 32... DC component calculation unit, 33... Subtractor, 340 to 34n... Encoder for image signal components, 35... Quantization characteristic selection unit,
35a... Distortion distribution rate determination unit, 35b... Distribution distortion amount calculation unit, 35c... Quantization characteristic calculation unit, 37... Encoder for quantization characteristics. Conventional 7'o book diagram Figure 2

Claims (1)

[Claims] Frequency separation means that separates an input image signal into a plurality of frequency components; a plurality of encoders corresponding to each frequency component that encodes each separated frequency component; Quantization characteristic selection that distributes the specified allowable total distortion amount to each frequency component using frequency characteristic information, and selects the quantization characteristic to be given to each encoder from the distributed distortion amount. An image encoder comprising: