JPH08331557A - Image signal feature quantity processing unit and image signal coder - Google Patents

Abstract

PURPOSE: To provide the image signal feature quantity processing unit and the image signal coder by improving image quality under a prescribed information amount so as to reduce visual deterioration. CONSTITUTION: Image data of an image signal are divided into small blocks by a small block division section 402 and a feature quantity extract section 409 obtains plural characteristic quantities for each block. Then a feature quantity space division area in existence in the vicinity of a feature vector comprising plural feature amounts is sought by a vicinity division area discrimination section 411 and a feature quantity space vector dependence degree calculation section 414 calculates the degree of dependence. A quantization control parameter based on the proper visual characteristic is calculated based on the degree of dependence and the visual sensitivity allocated to the division area. Thus, the rapid change in the quantization control parameter is suppressed regardless of the presence of a limit of the feature quantity and more proper quantization control is conducted, resulting that the image quality is enhanced under a prescribed information amount thereby reducing visual deterioration.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image signal for improving the image quality by adaptively allocating a limited amount of information according to the local property of the image in high efficiency coding such as television. The present invention relates to a feature amount processing device and an image signal encoding device.

[0002]

2. Description of the Related Art In high-efficiency coding of image signals such as television, a region in which coding noise is easily detected is quantized by a fine quantization step, and a region in which coding noise is difficult to detect is coarsely quantized. The method of quantization is often used.

FIG. 7 is a block diagram for explaining an example of a conventional coding control method. In this example, it is assumed that standard motion compensation and discrete cosine transform are used as the image encoding method. Also, as one of the human visual characteristics, there is a masking effect that a flat area with little change will be noticed when deterioration occurs even a little, and a region with a lot of change will be inconspicuous even if it deteriorates to some extent. The variance of the pixel values in the frame is used as the feature amount for this.

First, the image input from the input terminal 101 is divided into N × N small blocks by the small block dividing unit 102. From the image signal 103 of a small block and the image signal of one frame before stored in the frame memory 114, the motion vector 105 is calculated in the motion vector detecting unit 104.
Based on the motion vector 105, motion compensation is performed in the motion compensation unit 106, and the subtractor 107
After the motion-compensated prediction error signal 108 is obtained at, the discrete cosine transform unit 1 is applied to the motion-compensated prediction error signal 108.
At 09, discrete cosine transform is performed.

On the other hand, the image signal 1 divided into small blocks
Intra-frame variance 116 is obtained by the following calculation in variance calculation unit 115 for 03, and sent to quantization step calculation unit 117.

[0006]

[Equation 1]

However, s _ij is the pixel value in the small block, S
Represents the average.

In the quantization step calculator 117,
In-frame distribution 116 and buffer memory occupancy 11
Intraframe variance 1 based on 8 and constant information
When 16 is large, a coarse quantization step is set, and when it is small, fine processing is set. However, when the buffer memory occupation amount 118 increases, the quantization step is increased, and when the buffer memory occupation amount 118 decreases, the quantization step is reduced. Using the set quantization step 119, the discrete cosine transform coefficient 120
Is quantized by the quantization unit 110, and the variable length coding unit 121
After variable-length coding with, the data is input to the buffer memory 122, and the coded data output terminal 12 has a constant bit rate.
3 is output.

The discrete cosine transform coefficient quantized by the quantizer 110 is inversely quantized by the inverse quantizer 111, inversely discrete cosine transformed by the inverse discrete cosine transform unit 112, and then motion compensated. The data of the previous frame is added by the adder 113 and stored in the frame memory 114 for motion compensation prediction of the next frame.

According to such an encoding control method, since a flat portion can be finely quantized and a portion having a large change can be roughly quantized, a limited amount of information can be distributed according to the masking effect. As a result, the quality of the encoded image can be improved.

[0011]

However, in the above-described conventional image signal coding control method, it is assumed that the feature amount is a parameter that monotonically increases or monotonically decreases with respect to susceptibility to deterioration (hereinafter referred to as visual sensitivity). Since the quantization control is performed, there is a problem that the control may not be properly performed when a feature amount that is not monotonically increasing or monotonically decreasing is used as a parameter. For example, in the case of the feature amount as shown in FIG. 8, the visual sensitivity can be expressed by a monotonically increasing relationship when the feature amount is 0 to an intermediate value, but it decreases monotonically after the intermediate value, and the relationship from 0 to an intermediate value is obtained. Different from the formula. Therefore, if the control is performed based on the relational expression from 0 to the intermediate value, the appropriate control cannot be performed after the intermediate value.

However, in the case of the characteristic amount as shown in FIG. 8, it is possible to appropriately control even if the characteristic amount is not monotonically increasing or monotonically decreasing. It can be monotonically increased by taking the absolute value of the difference from the intermediate value of the feature amount, and can be controlled by the conventional method.

However, when a feature quantity having a plurality of points that change from decrease to increase or points that change from increase to decrease (hereinafter, extreme values) is used, it is very difficult to delete a plurality of extreme values. There is a possibility that the extreme value cannot be deleted and appropriate control may not be performed.

Further, when a case is performed by the feature quantity to delete a plurality of extreme values, there is a possibility that the quantization control characteristic may change abruptly at the boundary of cases. For example,
As shown in FIG. 9A, the small blocks A, which have different features,
Consider C and a small block B having an intermediate feature amount between both small blocks. The small blocks A and C belong to the case-divided feature amount regions α and β, respectively. Small block B
Has an intermediate feature amount, so the actual visibility of deterioration is at the x position in the figure. However, small block B
Despite having an intermediate feature amount, will belong to one of the feature amount regions, and a determination different from the actual visibility of deterioration will be made.

As described above, when the value of the characteristic amount does not monotonically increase or decrease with respect to the visual sensitivity, it is difficult to perform appropriate control. In addition, when control is performed depending on the case, the control may change abruptly at the boundary of the case, and it is determined that the small block having the feature amount near the boundary of the case is different from the visibility of actual deterioration. There was a problem that was carried out.

The present invention has been made to solve the above problems, and an object thereof is to improve image quality under a certain amount of information and to reduce image deterioration. A feature amount processing device and an image signal encoding device are provided.

[0017]

In order to achieve the above object, an image signal feature amount processing apparatus of the present invention is an image signal feature for dividing a feature amount space relating to a digital image signal such as a television signal into a plurality of regions. A quantity processing device, means for detecting a plurality of types of feature quantities from an image signal, means for dividing a feature quantity space composed of the detected feature quantities into a plurality of areas using visual sensitivity as an index, It is characterized in that means for obtaining a vector of a representative point of the area is assigned to each of the divided areas, and visual sensitivity is assigned to each of the divided areas.

Similarly, the image signal coding apparatus of the present invention is an image signal coding apparatus having means for dividing the current image data into small blocks for a digital image signal such as a television signal. Means for obtaining a plurality of types of feature quantities for each, and means for storing the visual sensitivity corresponding to the vector of the representative point of each region previously divided into a plurality of areas using the visual sensitivity as an index in the feature amount space of the plurality of types of feature quantities A neighborhood division area determining means for finding the divided areas existing in the vicinity of the feature quantity vector composed of the plurality of types of feature quantities; and a division quantity existing in the vicinity of the feature quantity vector and the feature quantity vector. Means for calculating the degree of belonging of the feature amount vector by obtaining the distance from the representative point of the region, the feature amount vector found and the feature amount Quantization control parameter calculation means for calculating a new visual sensitivity based on the calculated degree of belonging and the visual sensitivity corresponding to a divided area existing in the vicinity of the cutout, the quantization control parameter calculation means The quantization characteristic of the small block to be encoded is switched based on the new visual sensitivity calculated by

In the above-mentioned image signal encoding device, it is preferable to use the luminance dispersion and the moving speed as the feature amount for efficient processing.

[0020]

In the image signal feature amount processing apparatus of the present invention, a plurality of feature amounts are detected from the image signal, the detected feature amount space is divided into a plurality of regions using the visual sensitivity as an index, and the visual sensitivity is set in each region. And the representative point vector of each region is obtained, the information necessary for quantization control based on appropriate visual characteristics is provided in image signal coding and the like.

Further, in the image signal coding apparatus of the present invention,
The image data of the image signal is divided into small blocks, a plurality of feature quantities are obtained for each block, and the feature quantity space division area existing in the vicinity of the feature quantity vector composed of the plurality of feature quantities is searched for. Even if the feature amount is not monotonically increasing or monotonically decreasing from the degree of affiliation, even if the feature amount is not monotonically increasing or decreasing, by calculating the quantization control parameter based on an appropriate visual characteristic, The ease is not determined only by one feature amount space division area, but is determined based on the information of a plurality of feature amount space division areas and the degree of belonging to each of the division areas. For local features of, regardless of the presence or absence of the extreme value of the feature amount, and suppress the rapid change of the quantization control parameter,
Quantization control can be performed more adaptively, and as a result, image quality is improved and visual deterioration is reduced under a certain amount of information.

[0022]

Embodiments of the present invention will now be described in detail with reference to the drawings.

First, a first embodiment of the present invention will be described. This embodiment shows the basic configuration of the present invention. 1 and 2
FIG. 3 is a diagram showing the configuration of this embodiment.

FIG. 1 shows a schematic configuration of this embodiment. The image signal feature amount processing device 201 performs feature amount space division using the image signal 202 for feature amount space division, and obtains the representative vector and visual sensitivity of each divided region to obtain the feature amount space division region information (representative Vector) 412 and the feature space index-visual sensitivity conversion table 417 are calculated. Therefore, the image signal processing device 201 includes a unit that detects a plurality of feature amounts from the image signal, a unit that divides the feature amount space formed of the detected feature amounts into a plurality of regions using visual sensitivity as an index, It comprises means for obtaining a representative point vector of the area for each of the obtained areas and means for assigning a visual sensitivity to each of the obtained divided areas.

On the other hand, in the image signal encoding device 203, the feature amount space division area information (representative vector) 412 calculated by the image signal feature amount processing device 201 and the feature amount space index-visual sensitivity conversion table 417 are used appropriately. The quantization control parameter based on the visual characteristic is calculated, the quantization control is performed by this, the input image signal 204 is encoded, and the encoded signal 205 is output.

FIG. 2 shows the details of the basic construction centering on the image signal coding apparatus. The number of feature quantities used is n. The image signal input from the input terminal 401 is divided into small blocks by the small block dividing unit 402, and the feature amount extracting unit 409 divides each of the divided small blocks from the image signal 408 into a plurality of feature amounts f _i. (I = 1, 2, ..., N) are extracted, and the feature amount space vector v _f * = (f ₁ , f ₂ , ..., F _n )
(410). In the text that follows, a symbol is followed by * to represent a vector, but in the figure
Is attached to indicate a vector.

In the image signal processing device (not shown), as the feature amount space division region information 412, a region based on visual sensitivity is used to integrate regions having similar visual senses, the feature amount space is divided, and an index is given. To do. Furthermore, for each divided area, feature amount space index from the visual sensitivity of the region - obtaining visual sensitivity conversion table fh _tbl (417). For example, consider a case where the feature quantity space is divided as shown in FIG. Information indicating the range of the divided areas is obtained from the feature quantity space divided region information 412, feature space index as the visibility of the image quality deterioration of each divided region in FIG. 4 (b) - visual sensitivity conversion table fh _tbl It is calculated from (417).

The generated feature quantity space vector v _f * (41
0) and the predetermined feature amount space division area information f _sj (j = 1, 2, ..., M), the neighborhood division area determination unit 411 determines the feature quantity space vector v _f * (410). P divided regions N _k (k = 1, 2, ... P) (41
Find 3). Then, the feature amount space vector belonging degree calculating unit 414 calculates the degree of belonging b _k (k = 1, 2, ... P) (415) to each neighborhood divided region N _k (413).
Here, the degree of belonging b _k is an index indicating how close the feature space vector is to each neighboring divided area.

Next, in the visual sensitivity conversion unit 418, the index f _ak (416) given to each neighborhood divided area N _k (413) and the feature amount space index-visual which is the visual sensitivity of each divided area Sensitivity conversion table fh _tbl (4
17), the visual sensitivity PQD _k (419) of each neighboring divided area N _k (413) is _obtained . Then, in the quantization control parameter calculation unit 420, the visual sensitivity P of each neighborhood divided area is calculated.
From QD _k (k = 1, 2, ... P) (419) and the degree of belonging to each neighboring divided area b _k (415), the quantization control parameter z (421) of the block to be coded is obtained from the following equation.

Z = func (b _k , PQD _k ), where func is the visual sensitivity PQD _k (419) of the neighboring divided area according to the degree of belonging b _k (415), and The effect is to change the quantization control parameter z (421) of the block, and as the degree of belonging b _k (415) increases, the effect decreases.

In the quantization control parameter calculation unit 420,
The quantization step 424 is determined from the calculated quantization control parameter z (421) and the occupied state 422 of the encoded data buffer memory 406. Using the set quantization step 424, the quantization unit 404 quantizes the orthogonal transformation coefficient of the orthogonal transformation 403, and the code assignment unit 405 assigns a code to the quantized orthogonal transformation coefficient. The encoded data is input to the buffer memory 406,
It is output to the encoded data output terminal 407 at a constant bit rate.

As an example, consider a case where the degree of belonging is determined by the ratio of the distance between the representative vector of the divided area and the feature quantity space vector in the two-dimensional feature quantity space shown in FIG. The feature space vector v _f * belongs to the divided area F, but is also close to the divided area A. The distances between the divided area A or the divided area F and the feature amount space vector are l _A and l _F , respectively.
Then, the degree of belonging b _A and b _F of the feature amount space vector v _f * to each divided region is as follows: b _A = l _F / (l _A + l _F ) b _F = l _A / (l _A + l _F ) Become. Further, assuming that the visual sensitivities of the divided areas A and F are PQD _A and PQD _F , respectively, and the distances between the divided areas and the feature amount space vector are l _A and l _F , respectively, the visual perception of the feature amount space vector v _f * The sensitivity PQD _vf is PQD _vf = PQD _A · b _A + PQD _F · b _F.

Next, a second embodiment of the present invention will be shown. Figure 5
FIG. 3 is a diagram showing its block configuration. In this embodiment, 2
Quantization control is performed based on the visual sensitivity obtained from the feature amount space that is composed of both feature amounts in the image to be encoded, using various types of feature amounts (intra-frame luminance signal variance, small block motion vector size). It is a thing. It is assumed that a motion-compensated discrete cosine transform is used as the image coding method. Further, the number k of neighboring divided areas is set to 4, and the degree of belonging of each feature amount space vector to each neighboring divided area is calculated from the Euclidean distance.

First, the image input from the input terminal 601 is divided by the small block dividing unit 602 into N × N small blocks. A motion vector 605 is obtained in the motion vector detection unit 604 from the image signal 603 of the small block and the image of one frame before stored in the frame memory 614, and the motion compensation unit 60 is based on the motion vector 605.
6, the motion compensation is performed, the subtractor 607 obtains the motion compensation prediction error signal 608, and then the motion compensation prediction error signal 608 is subjected to the discrete cosine transform in the discrete cosine transform unit 609.

On the other hand, two types of feature quantities are calculated as follows. First, from the image signal 603 divided into small blocks,
The luminance variance calculation unit 615 uses the luminance variance f ₁ in the small block.
To calculate.

[0036]

[Equation 2]

Where s _ij is the pixel value in the small block, S
Represents the average. Further, from the motion vector mv * = (mv _x , mv _y ) (605) of the small block to be encoded, the motion velocity calculator 616 calculates the magnitude f ₂ of the motion vector from the following equation.

F ₂ = | mv * | = √ (mv _x ² + mv _y ² ) Next, in order to use the Euclidean distance in the neighborhood divided region determination and the membership degree calculation in the feature amount space,
Two types of feature quantities f ₁ and f _{2 of} the frame to be encoded
A normalization unit 617 performs normalization on each of the above to obtain f ₁ ′ and f ₂ ′.

Further, in the image signal feature amount processing device (not shown), in the feature amount space as shown in FIG. 6A, neighboring regions having similar visual sensitivities are integrated, and the representative vector f _{si of the} region is integrated. * (620) is calculated in advance. In addition, FIG.
As shown in (b), a representative vector-visual sensitivity conversion table (623) is also generated in advance as information indicating the visual sensitivity of each divided area.

A feature space vector v _f * = (f ₁ ′, f ₂ ′) (618) formed by _two feature quantities f ₁ ′ and f ₂ ′ obtained from the small block to be encoded and representative of each divided area From the vector f _si * (620), the neighborhood division area determination unit 6
At 19, the four neighboring divided areas of the feature quantity space vector v _f * (618) of the encoding target small block are detected. First, the distance d _i = || f _si * between the representative vector f _si * (620) of each divided region and the feature amount space vector v _f * (618)
-V _f * || is calculated, and the smallest distance is 4
Representative vector f _ak * (k = 1, 2, 3,
4) (621) is detected. Also, the representative vector f _ak * (621) of each neighborhood divided region and the feature amount space vector (6
18) with the distance l _k (k = 1, 2, 3, 4) (622)
And

Further, in the visual sensitivity conversion unit 624, the visual sensitivity PQD _k (k = 1, 2, 3, 4) (62) of each neighborhood divided area is calculated from the representative vector-visual sensitivity conversion table 623.
5) is extracted, and the quantization control parameter calculation unit 626 calculates the following expression from the visual sensitivity PQD _k (625) and the distance l _k (k = 1, 2, 3, 4) (622) of each neighborhood divided region. Then, the quantization control parameter z (627) is obtained.

[0042]

(Equation 3)

Subsequently, in the quantization step calculation unit 629, the quantization step 630 is performed based on the quantization control parameter z (627) and the buffer memory occupation amount 628.
Set. The discrete cosine transform coefficient 631 is quantized by the quantizer 610 using the set quantization step 630, variable-length coded by the variable-length coding unit 632, and then input to the buffer memory 633. It is output to the encoded data output terminal 634 at the bit rate.

The quantized discrete cosine transform coefficient is inversely quantized by an inverse quantizer 611, inverse discrete cosine transform by an inverse discrete cosine transform unit 612, and then added with the motion-compensated previous frame data. It is added by the device 613 and stored in the frame memory 614 for motion compensation prediction of the next frame.

With such a structure, the presence or absence of the extreme value of the feature amount with respect to the visual sensitivity becomes irrelevant to the quantization control, and the visual sensitivity of the neighboring region is taken into consideration. Therefore, in the vicinity of the boundary of the divided regions in the feature amount space. It is possible to suppress a drastic change in the quantization control parameter.

In the above-described embodiment, the feature amount is two kinds of luminance dispersion and motion speed, and the number of neighboring divided areas is four, but the present invention is not limited to this. Further, although the Euclidean distance is used for the determination of the neighboring divided region and the divided region having a small distance is set as the neighboring divided region, other distances indicating the distance of the feature amount space may be used. Further, the method of dividing the feature amount space, the index of the visual sensitivity, and the method of calculating the quantization control parameter from the visual sensitivity can be arbitrarily determined. Further, in this embodiment, the division information of the feature amount space is obtained in advance and fixed, but the calculation time of the division information of the feature amount space and the rewriting during the encoding process are arbitrary. Furthermore, each of the embodiments is not limited to the encoding system shown in the figure.

[0047]

As described above, according to the image signal characteristic amount processing apparatus and the image signal encoding apparatus of the present invention, the local characteristic of the image signal is irrelevant to the presence or absence of the extreme value of the characteristic amount. In addition, it becomes possible to use it for the quantization control while suppressing the abrupt change of the control parameter, and the quantization control is performed more adaptively as compared with the case of using only the feature value having no extreme value. You can That is, in a region where deterioration is apt to stand out, deterioration can be made inconspicuous by performing fine quantization, and in a region where deterioration is not noticeable, coarse quantum can be performed to reduce the amount of information. As a result, it is effective as a means for improving the image quality under a certain amount of information, and visual deterioration can be reduced.

In the above, when the luminance dispersion and the motion speed are used as the feature amount, there is an advantage that the processing can be efficiently performed by the two feature amounts. In some cases, the motion speed calculated there can be used, which is even more efficient.

[Brief description of drawings]

FIG. 1 is a diagram showing an outline of a first embodiment showing a basic configuration of the present invention.

FIG. 2 is a diagram showing a detailed block configuration of the first embodiment.

FIG. 3 is a diagram showing an example of calculating a degree of belonging in the first embodiment.

FIG. 4A and FIG. 4B are diagrams showing division of a feature quantity space by visual sensitivity in the first embodiment.

FIG. 5 is a diagram showing a block configuration of a second exemplary embodiment of the present invention.

6A and 6B are diagrams showing an example of division of a feature quantity space by visual sensitivity in the second embodiment.

FIG. 7 is a diagram showing a configuration example of conventional quantization control by image signal dispersion.

FIG. 8 is a diagram showing an example of extremum deletion of a feature value having an extremum.

9A and 9B are diagrams showing quantization control characteristics depending on the case of feature value.

[Explanation of symbols]

401 ... Input terminal 402 ... Small block division unit 403 ... Orthogonal transformation 404 ... Quantization unit 405 ... Code assignment unit 406 ... Buffer memory 407 ... Output terminal 409 ... Feature amount extraction unit 410 ... Feature amount space vector 411 ... Neighborhood division region determination Part 412 ... Feature amount space division area information 414 ... Feature amount space vector belonging degree calculation unit 417 ... Feature amount space index-visual sensitivity conversion table 418 ... Visual sensitivity conversion unit 420 ... Quantization control parameter calculation unit 423 ... Quantization step calculation Unit 601 ... Input terminal 602 ... Small block division unit 604 ... Motion vector detection unit 606 ... Motion compensation unit 607 ... Subtractor 609 ... Discrete cosine transform unit 610 ... Quantization unit 611 ... Inverse quantization unit 612 ... Inverse discrete cosine transform unit 613 ... Adder 614 ... Frame memory 615 ... Luminance dispersion calculation 616 ... Moving speed calculation unit 617 ... Normalization unit 619 ... Neighborhood division region determination unit 620 ... Each division region representative vector 623 ... Representative vector-visual sensitivity conversion table 624 ... Visual sensitivity conversion unit 626 ... Quantization control parameter calculation unit 629 ... Quantization step calculation unit 632 ... Variable length coding unit 633 ... Buffer memory 634 ... Output terminal

Claims (3)

[Claims] 1. An image signal feature amount processing apparatus for dividing a feature amount space relating to a digital image signal such as a television signal into a plurality of regions, said means for detecting a plurality of types of feature amounts from the image signal, and said detection. Means for dividing a feature amount space composed of the divided feature amounts into a plurality of regions using visual sensitivity as an index, means for obtaining a vector of a representative point of the region for each of the divided regions, and each of the divided regions An image signal feature amount processing device characterized by assigning visual sensitivity to a region. 2. An image signal encoding apparatus having means for dividing the current image data into small blocks for a digital image signal such as a television signal, and means for obtaining a plurality of types of characteristic amounts for each small block to be encoded. , A means for storing the visual sensitivity corresponding to the vector of the representative point of each region divided in advance using the visual sensitivity as an index with respect to the feature amount space based on the plurality of types of feature amounts, and configured by the plurality of types of feature amounts Neighborhood division area determining means for finding the divided area existing in the vicinity of the feature amount vector, and obtaining a distance between the feature amount vector and a representative point of the divided area existing in the vicinity of the feature amount vector Means for calculating the degree of belonging of the feature quantity vector, the searched feature quantity vector, and the divided regions existing in the vicinity of the feature quantity vector Quantization control parameter calculation means for calculating a new visual sensitivity based on the corresponding visual sensitivity and the calculated degree of membership, and based on the new visual sensitivity calculated by the quantization control parameter calculation means, An image signal coding apparatus, characterized in that the quantization characteristic of the small block to be coded is switched. 3. The image signal coding apparatus according to claim 2, wherein the feature quantities are luminance dispersion and motion speed.