KR20180099369A - Method for video rate control, and video coding method and apparatus the same - Google Patents

Abstract

The present invention discloses a method and a device for determining a bit amount of an image encoding process. According to an embodiment of the present invention, the method comprises the following processes of: calculating a weight of a predetermined first size unit by using a cognitive characteristic element; confirming a residual signal value reflecting a difference value between an original block and a prediction block to the calculated weight of the first size unit; and reflecting a ratio of the residual signal value of the first size unit compared to the residual signal of a picture unit, and allocating the bit amount for the first size unit.

Description

TECHNICAL FIELD [0001] The present invention relates to a video bit rate control method, a video bit rate control method, and a video encoding method and apparatus using the same.

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a video encoding method and apparatus, and more particularly to a bit rate control method and apparatus.

Recently, the demand for high resolution and high quality images such as high definition (HD) image and ultra high definition (UHD) image is increasing in various applications. As the image data has high resolution and high quality, the amount of data increases relative to the existing image data. Therefore, when the image data is transmitted using a medium such as a wired / wireless broadband line or stored using an existing storage medium, The storage cost is increased. High-efficiency image compression techniques can be used to solve these problems caused by high-resolution and high-quality image data.

Meanwhile, in the image compression or coding process, a technique of controlling the size of the compressed or encoded data in consideration of the transmission or storage environment of the image data may be applied. Generally, in order to control the size of compressed or coded data, a technique of allocating a bit amount for a unit such as a picture, a slice, or a block included in an image is used.

Generally, a method of controlling the size of compressed or coded data is a method of uniformly allocating a bit amount to a unit of a picture, a slice, or a block included in an image, or a method of allocating a residual amount of a unit such as a picture, a slice, A method of allocating a bit amount in consideration of a signal size or a complexity is used.

SUMMARY OF THE INVENTION It is an object of the present invention to provide an image encoding method and apparatus for controlling a bit amount in accordance with a person's cognitive characteristics.

According to another aspect of the present invention, there is provided an image encoding method and apparatus capable of efficiently controlling a bit amount while minimizing deterioration of image quality perceived by a human by controlling a bit amount by reflecting a human perception characteristic.

It is another object of the present invention to provide a method and apparatus for encoding an image, which reflects a person's cognitive characteristics and reflects more weight on a cognitively important part, thereby minimizing degradation of a perceived image quality of a person.

The technical objects to be achieved by the present disclosure are not limited to the above-mentioned technical subjects, and other technical subjects which are not mentioned are to be clearly understood from the following description to those skilled in the art It will be possible.

According to an aspect of the present disclosure, a method of determining the bit amount of the image encoding process can be provided. The method includes the steps of calculating a predetermined weight of a first size unit using a cognitive characteristic element and determining a residual signal value reflecting a difference value between the original block and the prediction block in the weight of the first unit of magnitude And allocating a bit amount for the first size unit by reflecting a ratio of the residual signal value of the first size unit to the residual signal value of the picture unit.

The features briefly summarized above for this disclosure are only exemplary aspects of the detailed description of the disclosure which follow, and are not intended to limit the scope of the disclosure.

According to the present disclosure, an image encoding method and apparatus capable of controlling a bit amount by reflecting human perception characteristics can be provided.

According to the present disclosure, it is possible to provide an image encoding method and apparatus capable of efficiently controlling the bit amount while minimizing the deterioration of image quality perceived by a person.

The effects obtainable from the present disclosure are not limited to the effects mentioned above, and other effects not mentioned can be clearly understood by those skilled in the art from the description below will be.

1 is a block diagram illustrating a configuration of an image encoding apparatus to which a rate-control method according to an embodiment of the present disclosure is applied.
FIG. 2 is a diagram illustrating a hierarchical coding structure to which a rate-control method according to an embodiment of the present disclosure is applied, and an operation in which bit allocation of a picture unit is processed.
3 is a flow chart showing the sequence of the rate-control method according to one embodiment of the present disclosure;
4A is a diagram showing a relationship between a spatial frequency and contrast.
4B is a graph showing the relationship between spatial frequency and contrast sensitivity.
5 is a diagram illustrating a weight matrix that reflects contrast sensitivity characteristics by a rate-control method in accordance with an embodiment of the present disclosure.
FIG. 6 is a diagram illustrating a weight that reflects luminance adaptation characteristics by a rate-control method according to an embodiment of the present disclosure; FIG.
7 is a diagram illustrating a weight that reflects contrast masking characteristics by a rate-control method according to one embodiment of the present disclosure.
8 is a diagram illustrating a protrusion map and weights applied in a rate-controlling method according to an embodiment of the present disclosure.

Hereinafter, embodiments of the present disclosure will be described in detail with reference to the accompanying drawings, which will be easily understood by those skilled in the art. However, the present disclosure may be embodied in many different forms and is not limited to the embodiments described herein.

In the following description of the embodiments of the present invention, a detailed description of known functions and configurations incorporated herein will be omitted when it may make the subject matter of the present disclosure rather unclear. Parts not related to the description of the present disclosure in the drawings are omitted, and like parts are denoted by similar reference numerals.

In the present disclosure, when an element is referred to as being "connected", "coupled", or "connected" to another element, it is understood that not only a direct connection relationship but also an indirect connection relationship May also be included. Also, when an element is referred to as " comprising "or" having "another element, it is meant to include not only excluding another element but also another element .

In the present disclosure, the terms first, second, etc. are used only for the purpose of distinguishing one element from another, and do not limit the order or importance of elements, etc. unless specifically stated otherwise. Thus, within the scope of this disclosure, a first component in one embodiment may be referred to as a second component in another embodiment, and similarly a second component in one embodiment may be referred to as a first component .

In the present disclosure, the components that are distinguished from each other are intended to clearly illustrate each feature and do not necessarily mean that components are separate. That is, a plurality of components may be integrated into one hardware or software unit, or a single component may be distributed into a plurality of hardware or software units. Thus, unless otherwise noted, such integrated or distributed embodiments are also included within the scope of this disclosure.

In the present disclosure, the components described in the various embodiments are not necessarily essential components, and some may be optional components. Thus, embodiments consisting of a subset of the components described in one embodiment are also included within the scope of the present disclosure. Also, embodiments that include other elements in addition to the elements described in the various embodiments are also included in the scope of the present disclosure.

Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings.

1 is a block diagram illustrating a configuration of an image encoding apparatus to which a rate-control method according to an embodiment of the present disclosure is applied.

1, the image encoding apparatus 100 includes a motion prediction unit 111, a motion compensation unit 112, an intra prediction unit 120, a switch 115, a subtractor 125, a transform unit 130, A quantization unit 140, an entropy encoding unit 150, an inverse quantization unit 160, an inverse transformation unit 170, an adder 175, a filter unit 180, and a reference picture buffer 190.

Each of the components shown in FIG. 1 is shown independently to represent different characteristic functions in the image encoding apparatus, and does not mean that each component is composed of separate hardware or one software configuration unit. That is, each constituent unit is included in each constituent unit for convenience of explanation, and at least two constituent units of the constituent units may be combined to form one constituent unit, or one constituent unit may be divided into a plurality of constituent units to perform a function. The integrated and separate embodiments of the components are also included in the scope of the present disclosure without departing from the spirit of the present disclosure.

In addition, some of the elements are not essential components to perform essential functions in the present disclosure, but may be optional components only to improve performance. This disclosure can be implemented only with components that are essential to implementing the essentials of the present disclosure, except for those components used for performance enhancement, and only those components that include only the essential components, Are also included in the scope of the present disclosure.

The image encoding apparatus 100 may perform an encoding process by dividing an input image into at least one processing unit. At this time, the processing unit may be a prediction unit (PU), a transform unit (TU), or a coding unit (CU). A picture is divided into a plurality of encoding units, a prediction unit, and a conversion unit for one picture, and a single encoding unit, a prediction unit, and a conversion unit combination are selected with a predetermined criterion (e.g., cost function) .

For example, one picture may be divided into a plurality of coding units. In order to divide a coding unit in a picture, a recursive tree structure such as a quad tree structure can be used. In a coding or decoding scheme in which one picture or a largest coding unit is used as a root and divided into other coding units A unit can be divided with as many child nodes as the number of divided coding units. Under certain constraints, an encoding unit that is no longer segmented becomes a leaf node. That is, when it is assumed that only one square division is possible for one coding unit, one coding unit can be divided into a maximum of four different coding units.

Hereinafter, in the embodiment of the present invention, a coding unit may be used as a unit for performing coding, or may be used as a unit for performing decoding.

The prediction unit may be one divided into at least one square or rectangular shape having the same size in one coding unit, and one of the prediction units in one coding unit may be divided into another prediction Or may have a shape and / or size different from the unit.

When generating a prediction unit performing intra prediction on the basis of an encoding unit, intra prediction can be performed without dividing the prediction unit into a plurality of prediction units N x N when the minimum encoding unit is not used.

The image encoding apparatus 100 may encode an input image in an intra mode or an inter mode and output a bit stream. In the intra mode, the switch 115 is switched to the intra mode, and in the inter mode, the switch 115 can be switched to the inter mode. The image encoding apparatus 100 may generate a prediction block for a processing unit of the input image and then code the residual between the input block and the prediction block.

In the intra mode, the intra prediction unit 120 may generate a prediction block by performing spatial prediction using the pixel value of the already coded block around the current block.

In the inter mode, the motion predicting unit 111 can find a motion vector by searching an area of the reference picture stored in the reference picture buffer 190 that is best matched with the input block. The motion compensation unit 112 may generate a prediction block by performing motion compensation using a motion vector.

The subtractor 125 may generate a residual block by a difference between the input block and the generated prediction block.

The transforming unit 130 may perform a transform on the residual block to output a transform coefficient. The quantization unit 140 may quantize the input transform coefficient according to the quantization parameter to output a quantized coefficient.

The entropy encoding unit 150 may output a bit stream by performing entropy encoding based on the values calculated by the quantization unit 140 or the encoding parameter values calculated in the encoding process.

When entropy encoding is applied, a small number of bits are allocated to a symbol having a high probability of occurrence, and a large number of bits are allocated to a symbol having a low probability of occurrence, thereby expressing symbols, The size of the column can be reduced. Therefore, the compression performance of the image encoding can be enhanced through the entropy encoding.

The entropy encoding unit 150 may use an encoding method such as exponential golomb, context-adaptive variable length coding (CAVLC), and context-adaptive binary arithmetic coding (CABAC) for entropy encoding.

Since the image encoding apparatus according to the embodiment of FIG. 1 performs inter prediction encoding, that is, inter-view prediction encoding, the currently encoded image needs to be decoded and stored for use as a reference image. Accordingly, the quantized coefficients are inversely quantized in the inverse quantization unit 160 and inversely transformed in the inverse transformation unit 170. The inverse quantized and inverse transformed coefficients are added to the prediction block through the adder 175 and a reconstructed block is generated.

The restoration block passes through the filter unit 180 and the filter unit 180 applies at least one of a deblocking filter, a sample adaptive offset (SAO), and an adaptive loop filter (ALF) can do. The filter unit 180 may be referred to as an adaptive in-loop filter. The deblocking filter can remove block distortion occurring at the boundary between the blocks. The SAO may add a proper offset value to the pixel value to compensate for coding errors. ALF can perform filtering based on the comparison between the reconstructed image and the original image. The reconstructed block having passed through the filter unit 180 may be stored in the reference picture buffer 190.

FIG. 2 is a diagram illustrating a hierarchical coding structure to which a rate-control method according to an embodiment of the present disclosure is applied, and an operation in which bit allocation of a picture unit is processed.

Referring to FIG. 2, an intra section 210 in which I pictures are periodically inserted can be defined. In addition, the hierarchical coding structure according to the embodiment of FIG. 2 may be configured as a GOP (Group Of Picture) unit 220 in order to provide a high-efficiency coding performance. The GOP 220 may include an I ₀ picture 230 or a P ₀ picture 240, a B ₁ picture 250, a B ₂ picture 260 and a B ₃ picture 270.

The image encoding apparatus can derive an average bit amount for one picture obtained by dividing a given target bit amount by the frame rate (281). In order to allocate bits for pictures existing in the GOP, the image encoding apparatus can allocate a bit amount to the current GOP in consideration of the number of pictures existing in the GOP (282). The image encoding apparatus can again allocate the bit amount of the allocated GOP to the pictures in the GOP to be encoded (283). At this time, the image encoding apparatus can allocate a target bit amount to each picture in consideration of the buffer status and the weight for each picture in each time layer. The encoder may consider the amount of bits required for the current buffer state to reach the target buffer state in performing bit allocation for the pictures in the GOP and the weights between the pictures may be allocated to each time layer of the hierarchical encoding structure It can be applied differently depending on the picture type.

Further, the image encoding apparatus can reallocate the bit amount allocated for each picture in units of CTU. In the embodiment of the present disclosure, the coder uses a cognitive characteristic element to determine a weight for each CTU unit, applies the determined weight to the amount of bits allocated to the corresponding picture (for example, multiplication operation) . ≪ / RTI > The cognitive characteristic element may include at least one of a contrast sensitivity characteristic, a luminance adaptation characteristic, a contrast masking characteristic, and a saliency map characteristic.

3 is a flow chart showing the sequence of the rate-control method according to one embodiment of the present disclosure;

First, in step S301, the image encoding apparatus calculates a weight value in units of CTU using a cognitive characteristic element.

First, in the embodiment of the present disclosure, cognitive chracteristics are characteristics in which a person accepts external stimuli, and cognitive chracteristic factors represent elements corresponding to external stimuli.

The cognitive characteristic element may include a contrast sensitivity characteristic, a luminance adaptation characteristic, a contrast masking characteristic, and a saliency map characteristic.

With respect to the contrast sensitivity characteristic, in general, a person has a high sensitivity to contrast in a region where the spatial frequency is low in an image, and a low sensitivity to contrast in a region where the spatial frequency is high (see FIG. That is, a person has a characteristic that the change of a low frequency element is more sensitive than a change of a high frequency element in a frequency domain of an image. In addition, the change in contrast is hardly felt above a predetermined frequency size. The contrast sensitivity characteristic reflecting the cognitive characteristic can be expressed as shown in Equation 1 below, and can be expressed as shown in FIG. In FIG. 4, the x-axis represents the spatial frequency magnitude and the y-axis represents the contrast sensitivity.

In Equation (1), w denotes a spatial frequency magnitude.

With respect to the luminance adaptation characteristic, a person generally has a perception rate (or cognitive sensitivity) with respect to an object depending on a luminance change of a background included in the image. Particularly, the recognition rate (or cognitive sensitivity) for recognizing an object existing in a region where luminance is relatively low (i.e., a dark region) or a relatively high region (i.e., a bright region) in an image exists in an intermediate region of luminance (Or cognitive sensitivity) that recognizes an object. The luminance adaptation characteristic reflecting such a cognitive characteristic can be expressed by the following Equation (2). Equation (2) represents the recognition rate (or cognitive sensitivity) according to the intensity range of the pixel, F _lum indicates the value of the recognition rate (or cognitive sensitivity), and the larger the value of F _lum , It means falling.

In relation to the contrast masking characteristic, a person can perceive a distortion or change of an image. Generally, a recognition rate (or cognitive sensitivity) for recognizing a change in a distortion in a flat region is high, (Or cognitive sensitivity) may be low. A cognitive characteristic factor reflecting such cognitive characteristics can be defined as a contrast masking characteristic.

In order to confirm the recognition rate (or cognitive sensitivity) of the contrast masking characteristic, an edge pixel pixel density (rho edge1) for a predetermined block (e.g., NxN size block) And the type of the block can be determined based on the edge pixel density by applying Equation (4). In Equation (4), the larger the value of?, The lower the recognition rate (or cognitive sensitivity).

Here, an edge detection method such as a Sobel method or a Canny method can be used to discriminate edge pixels.

With respect to saliency map characteristics, a person may generally have a high perception rate (or cognitive sensitivity) for an area of interest in the image (e.g., face, contour, color difference, A saliency map can be formed by reflecting information on an object of interest, that is, face information (recognition or dictionary information), outline information, color difference, motion information, and the like.

On the other hand, in step S302, the image encoding apparatus may calculate a residual signal value reflecting the difference between the original block and the prediction block in the weight of the CTU unit calculated in step S301.

The residual signal value in the unit of CTU can be calculated by calculating the residual signal value for at least one block included in the CTU unit and adding the residual signal value for at least one block.

Next, in step S303, when the residual signal magnitude value for the CTU is determined, the image encoding apparatus checks the sum of the residual signal magnitudes of all the CTUs included in the picture, The residual signal size value ratio of the current CTU can be determined as the residual signal size value of the CTU, and the bit amount for the CTU unit can be allocated according to this ratio.

Hereinafter, a concrete method of allocating a bit amount in CTU units will be described in detail considering the above-mentioned cognitive characteristic elements.

1. Weight calculation using contrast sensitivity

The weight matrix value can be determined in consideration of the contrast sensitivity characteristic.

The weighting matrix using the contrast sensitivity characteristic can be calculated by the following equation (5).

ω _{i, j} denotes the spatial frequency content in the i, j positions on the frequency domain, a, b, c illustrate a constant of predetermined value as a = 1.33, b = 0.11, c = 0.18.

Through the above equation (5), an 8x8 weighting matrix can be derived, and the weighted matrix can be normalized based on the maximum weighted value having the maximum value. The normalized weight value reflecting the maximum weight value can be illustrated as shown in FIG.

Next, the image encoding apparatus performs transform (DCT or Hadamard) by dividing the original block and the prediction block into blocks of CTU units (or blocks of a predetermined size).

In the embodiment of the present disclosure, it is illustrated that the CTU unit is 64 × 64 pixel units, and the unit of the block capable of performing the conversion is 4 × 4, 8 × 8, 16 × 16, and 32 × 32. The CTU unit may be variously changed, and the unit for performing the conversion may be variously changed.

The residual signal value in units of CTU, which reflects the weight of the contrast sensitivity characteristic, can be calculated as shown in Equation (6) below.

In the following Equation (6), it is exemplified that the conversion is performed in 8x8 block units.

The difference between the transformed original block (Tr_org _{8 × 8} ) and the predicted block (Tr_pred _{8 × 8} ) is found and an absolute value is obtained. The weighted matrix thus calculated is multiplied by the weight matrix calculated by the above-described equation (5) The weighted MAD _{8 × 8} can be calculated.

The residual signal size value may be calculated in the same manner for the remaining blocks and the sum or average of the residual signal size values for the eight blocks may be determined as the residual signal size value for the corresponding CTU (Weighted MAD _{8 × 8} ).

The CTU unit bits can be allocated according to the ratio of the residual signal size determined in the determined CTU unit. Specifically, when the residual signal magnitude value is determined in units of CTU, the residual signal magnitude values in one picture are totally summed, and the ratio of the residual signal magnitude value of the current CTU to the total sum of the residual signal magnitude values is reflected, A bit amount can be allocated.

For example, it is illustrated that a predetermined picture is composed of four CTUs, the size value of the residual signal of each CTU is illustrated in Table 1 below, and the bit amount allocated to the picture is 1000 bits. In this case, the residual signal size value ratio of the current CTU can be determined by the residual signal size value of the current CTU to the sum of the residual signal size values of the entire CTU, and the bit amount for the CTU unit can be allocated according to this ratio.

CTU N. Residual signal magnitude value ratio Bit assigned to CTU One 200 0.2 2000 2 400 0.4 4000 3 100 0.1 1000 4 300 0.3 3000

2. Weight calculation using luminance adaptation characteristics

The weight matrix value can be determined using the contrast luminance adaptation characteristic.

The weighting matrix using the luminance characteristic can determine the weight by considering the range of the average luminance value of the original block (or the original signal).

That is, when the range of the average luminance value of the original block belongs to the dark region or the bright region, the weight can be set small.

To this end, an average luminance range is set, and a weight for a region relatively darker than the average luminance and a region relatively bright for the average luminance can be set.

For example, when the average luminance range is set to 60 to 170 as in Equation (2), if the average luminance value of the original block is greater than 60 and less than or equal to 170, the weighting value (H _LA ) have. If the average luminance value of the original block is relatively smaller than 60, the value obtained by calculating 1- (60-I ^) / 150 can be set as the weight (H _LA ), and the average luminance value If the value is relatively larger than 170, a value obtained by calculating 1- (I ^ -170) / 450 can be set as a weight (H _LA ).

Next, if the weight value (H _LA ) is determined by the range of the average luminance value of the input block, the weighting matrix can be calculated using the determined weighting factor.

As an example, a weight determined in input block units may be applied to the input block. 6, when the weight of the first input block 612 is determined to be 1 in the CUT having the first through fourth input blocks 611, 612, 613, and 614, the first input block (Or sub-blocks) included in the weighted matrix 612. [

When the weight matrix for the input block unit is determined as described above, the residual signal magnitude value for the CTU can be calculated by the following equation (7).

When the residual signal magnitude value for the CTU is determined, the sum of the residual signal magnitudes of all the CTUs in the picture is checked. The residual signal magnitude value of the current CTU is compared with the sum of the residual signal magnitudes of all the CTUs, The signal size value ratio can be determined, and the bit amount for the CTU unit can be allocated according to this ratio.

3. Weight calculation using contrast sensitivity and luminance adaptation characteristics

The weights can be calculated by combining the above-described contrast sensitivity characteristic and luminance adaptation characteristic.

Specifically, as shown in the following equation (8), the weight matrix calculated by the contrast sensitivity characteristic and the weight matrix calculated by the luminance adaptation characteristic are multiplied by the input block unit, The residual signal magnitude value can be calculated by calculating the residual signal magnitude value and summing the residual signal magnitude values in the input block units included in the CTU.

When the residual signal magnitude value for the CTU is determined, the sum of the residual signal magnitudes of all the CTUs in the picture is checked. The residual signal magnitude value of the current CTU is compared with the sum of the residual signal magnitudes of all the CTUs, The signal size value ratio can be determined, and the bit amount for the CTU unit can be allocated according to this ratio.

As another example, as shown in the following Equation (9), by reflecting the sum operation of the weight matrix calculated by the contrast sensitivity characteristic and the weight matrix calculated by the luminance adaptation characteristic for the input block unit, The residual signal magnitude value can be calculated by calculating the residual signal magnitude value and summing the residual signal magnitude values in the input block units included in the CTU.

When the residual signal magnitude value for the CTU is determined, the sum of the residual signal magnitudes of all the CTUs in the picture is checked. The residual signal magnitude value of the current CTU is compared with the sum of the residual signal magnitudes of all the CTUs, The signal size value ratio can be determined, and the bit amount for the CTU unit can be allocated according to this ratio.

4. Weight calculation using contrast masking characteristic

The weight according to the edge pixel density existing on the original signal (or block) can be calculated by using the contrast masking characteristic of the input block.

That is, when there are many textures in the original signal or the original block, the recognition rate for image quality degradation or distortion may be relatively low. In consideration of this, the weight (H _CM ) of the original signal or the original block can be relatively set.

First, the edge pixel density (rho edge) of an original signal or an original block can be calculated through an operation of the following expression (10). The block type of the original signal or the original block can be calculated based on the original signal or the edge pixel density of the original block thus calculated (see Equation (11)).

Then, a predetermined weight value can be determined according to the block type calculated as described above. For example, the predetermined weight value according to the block type can be illustrated as Equation (12) below.

If the density of edge pixels is relatively high in the block determined by the texture, it can be determined that edge pixels exist. If the density of edge pixels is relatively low, there is no edge pixel You can decide. A block having an edge pixel can be given a relatively higher weight than a block having no edge pixel. In other words, a block having no edge pixel can be given a relatively low weight value as compared with a block having an edge pixel.

In one embodiment of the present disclosure, the density of edge pixels is set to 16 as a criterion for determining that edge pixels exist. That is, in the block determined by the texture, if the density of the edge pixels exceeds 16, it is determined that edge pixels are present. If the density of edge pixels is equal to or smaller than 16, .

It goes without saying that the criterion for determining that an edge pixel exists may be variously changed.

Next, when the weight value (H _CM ) is determined based on the density of edge pixels on a block-by-block basis, the weighting matrix can be calculated using the determined weighting factors.

As an example, a weight determined in input block units may be applied to the input block. 7, in the CTU having the first to fourth input blocks 711, 712, 713 and 714, when the weight of the first input block 712 is determined as 1, the corresponding first input block (Or sub-blocks) contained in the weighting matrix 712. The weighting matrix 722 for the pixel (or sub-block)

If the weight matrix for the input block unit is determined as described above, the residual signal magnitude value for the CTU can be calculated by the following equation (13).

When the residual signal magnitude value for the CTU is determined, the sum of the residual signal magnitudes of all the CTUs in the picture is checked. The residual signal magnitude value of the current CTU is compared with the sum of the residual signal magnitudes of all the CTUs, The signal size value ratio can be determined, and the bit amount for the CTU unit can be allocated according to this ratio.

5. Weight calculation using contrast sensitivity and contrast masking

The weighting can be calculated by combining the contrast sensitivity characteristic and the contrast masking characteristic described above.

Specifically, as shown in the following Equation (14), the weight matrix calculated by the contrast sensitivity characteristic and the weight matrix calculated by the contrast masking characteristic for the input block unit are multiplied by the weight matrix calculated by the contrast masking characteristic, The residual signal magnitude value can be calculated by calculating the residual signal magnitude value and summing the residual signal magnitude values in the input block units included in the CTU.

When the residual signal magnitude value for the CTU is determined, the sum of the residual signal magnitudes of all the CTUs in the picture is checked. The residual signal magnitude value of the current CTU is compared with the sum of the residual signal magnitudes of all the CTUs, The signal size value ratio can be determined, and the bit amount for the CTU unit can be allocated according to this ratio.

As another example, as shown in the following equation (15), the sum of the weight matrices calculated by the contrast sensitivity characteristic and the weight matrices calculated by the contrast masking characteristic for the input block unit is reflected, The residual signal magnitude value can be calculated by calculating the residual signal magnitude value and summing the residual signal magnitude values in the input block units included in the CTU.

When the residual signal magnitude value for the CTU is determined, the sum of the residual signal magnitudes of all the CTUs in the picture is checked. The residual signal magnitude value of the current CTU is compared with the sum of the residual signal magnitudes of all the CTUs, The signal size value ratio can be determined, and the bit amount for the CTU unit can be allocated according to this ratio.

6. Weight calculation using luminance adaptation characteristic and contrast masking characteristic

The weights can be calculated by combining the above-described luminance adaptation characteristics and contrast masking characteristics.

Specifically, as shown in the following equation (16), the weight matrix calculated by the luminance adaptation characteristic and the weight matrix calculated by the contrast masking characteristic are multiplied by the input block unit, The residual signal magnitude value can be calculated by calculating the residual signal magnitude value and summing the residual signal magnitude values in the input block units included in the CTU.

When the residual signal magnitude value for the CTU is determined, the sum of the residual signal magnitudes of all the CTUs in the picture is checked. The residual signal magnitude value of the current CTU is compared with the sum of the residual signal magnitudes of all the CTUs, The signal size value ratio can be determined, and the bit amount for the CTU unit can be allocated according to this ratio.

As another example, as shown in the following Equation (17), by reflecting the sum operation of the weighting matrix calculated by the luminance adaptation characteristic and the weighting matrix calculated by the contrast masking characteristic for the input block unit, And the residual signal magnitude value in the unit of the input block included in the CTU is added to calculate the residual signal magnitude value in the unit of CTU.

When the residual signal magnitude value for the CTU is determined, the sum of the residual signal magnitudes of all the CTUs in the picture is checked. The residual signal magnitude value of the current CTU is compared with the sum of the residual signal magnitudes of all the CTUs, The signal size value ratio can be determined, and the bit amount for the CTU unit can be allocated according to this ratio.

7. Weight calculation using saliency map characteristics

The weight according to the saliency map characteristic of the input block can be calculated.

It is possible to construct data in the form of a predetermined map based on the concentration of the person (degree of interest), and designate the data thus constructed as a saliency map.

8 is a diagram illustrating a protrusion map and weights applied in a rate-controlling method according to an embodiment of the present disclosure.

Referring to FIG. 8, a first protruding map 820 reflecting the concentration of a person (degree of interest) may be formed on the original image 810. In the first protrusion map 820, a relatively bright portion may indicate that a person has a high degree of concentration (attention).

In addition, after applying the color in consideration of the brightness information of the first protrusion map 820, the second protrusion map 830 may be formed by overlapping the original image 810 with the hue. In the second projecting map 830, the red part indicates a relatively high degree of human attention (interest level), and in the second projected map 830, the blue part indicates relatively low degree of human focus (interest level).

The weight can be calculated by reflecting the concentration of the person (interest level) on the saliency map.

The protruding map can be set on a picture-by-picture basis, and further, the picture can be set in a unit divided by a predetermined size.

The weights can be applied reflecting the concentration (interest level) of the person calculated while constructing the protrusion map.

The concentration of the person (interest level) calculated while constructing the protrusion map can be calculated in units of pixels or a predetermined block. On the other hand, in one embodiment of the present disclosure, the bit amount may be allocated in CTU units. Therefore, since the weight reflecting the projected map needs to be determined in units of CTU, the representative weight for the CTU unit can be determined. The representative weight in units of CTUs can be calculated as an average value of weights of pixels or blocks included in the CTU unit.

The weighting value reflecting the protrusion map can be calculated by the following equation (18).

As described above, if the weight reflecting the protruded map is determined, the residual signal magnitude value for the CTU can be calculated by the following equation (19).

When the residual signal magnitude value for the CTU is determined, the sum of the residual signal magnitudes of all the CTUs in the picture is checked. The residual signal magnitude value of the current CTU is compared with the sum of the residual signal magnitudes of all the CTUs, The signal size value ratio can be determined, and the bit amount for the CTU unit can be allocated according to this ratio.

Further, the weight reflecting the protrusion map characteristic may be combined with at least one of the weight matrix (H _CS ) using the contrast sensitivity characteristic, the weight matrix (H _CM ) using the luminance adaptation characteristic, and the weight matrix (H _CM ) , And the combined weight can be used to allocate the bit amount for the CTU unit. For example, a residual signal magnitude value in input block units is calculated by multiplying a weight matrix (H _CS ) using a weight reflecting the saliency map characteristic and a contrast sensitivity characteristic (or a sum operation) The residual signal magnitude value in units of CTU can be calculated by summing the residual signal magnitude values in block units. When the residual signal size value for the TU is determined, the sum of the residual signal size values of all the CTUs in the picture is checked, and the residual signal size value of the current CTU is compared with the sum of the residual signal size values of all the CTUs. The signal size value ratio can be determined, and the bit amount for the CTU unit can be allocated according to this ratio.

Likewise, the residual signal magnitude value in units of input blocks is calculated by multiplying the weight matrix (H _CM ) using the weights reflecting the protrusion map characteristics and the luminance adaptation characteristic, The residual signal magnitude value in the unit of CTU can be calculated. When the residual signal size value for the TU is determined, the sum of the residual signal size values of all the CTUs in the picture is checked, and the residual signal size value of the current CTU is compared with the sum of the residual signal size values of all the CTUs. The signal size value ratio can be determined, and the bit amount for the CTU unit can be allocated according to this ratio.

Likewise, the residual signal size value in input block units is calculated by reflecting the multiplication operation (or sum operation) of the weight matrix (H _CM ) using the weight reflecting the projected map characteristic and the contrast masking characteristic, The residual signal magnitude value in the unit of CTU can be calculated. When the residual signal size value for the TU is determined, the sum of the residual signal size values of all the CTUs in the picture is checked, and the residual signal size value of the current CTU is compared with the sum of the residual signal size values of all the CTUs. The signal size value ratio can be determined, and the bit amount for the CTU unit can be allocated according to this ratio.

9 is a diagram illustrating a computing environment to which an image encoding apparatus and method according to an embodiment of the present disclosure is applied.

The computing environment illustrated in FIG. 9 is an example of an operating environment in which an image encoding apparatus and method may be implemented, and does not limit the scope of use or functionality of the present disclosure.

Examples of computing systems, environments, and / or configurations that may be suitably employed in an image encoding apparatus and method in accordance with an embodiment of the present disclosure include, but are not limited to, a personal computer, a server computer, a handheld or laptop device, Based systems, programmable consumer electronics, network personal computers, minicomputers, main picture computers, distributed computing environments that include any of the above systems or devices, and the like.

An apparatus and method for encoding an image, according to one embodiment of the present disclosure, will be described generally in connection with computer-executable instructions, such as program modules, being executed by one or more computers or other devices. Generally, program modules may include routines, programs, objects, components, data structures, etc. that perform particular tasks or implement particular abstract data types. Typically, the functions of the program modules may be combined or distributed according to various circumstances.

9, a computing device 900 typically includes at least one processing unit 902 and a memory 904.

Depending on the configuration and type of the computing device, the memory may be a volatile memory such as a RAM, a nonvolatile memory such as a ROM, a flash memory, or the like, or a combination thereof. In addition, the computing device 900 may include additional (e.g., removable and / or stationary) storage devices such as magnetic or optical disks or tapes. Additional storage devices may include a removable storage device 908 and a stationary storage device 910.

The memory 904, the removable storage device 908, and the fixed storage device 910 may all be provided as computer storage media and may be embodied in a variety of forms, including RAM, ROM, EEPROM, flash memory or other memory technology, CD- versatile disk, or other optical disk storage, magnetic cassette, magnetic tape, magnetic disk storage or other magnetic storage device, or any other medium that can be accessed by device 900 and store the desired information .

The computing device 900 may include a communication portion 912 that enables communication with other devices, such as other computing devices, via a wired or wireless network. The communication unit 912 typically implements computer readable instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transport mechanism, Information transmission medium. The term "modulated data signal" means a signal that has one or more of its characteristics set or changed to encode information in the signal. By way of example, communication portion 912 may comprise wired media such as a wired network or direct-wired connection, and wireless media such as acoustic, radio frequency, infrared, or other wireless media.

The computing device 900 may include an input device 914 such as a keyboard, a mouse, a pen, a voice input device, a touch input device, a laser range finder, an infrared camera, In addition, the computing device 900 may include an output device 916 such as a display, a speaker, and a printer.

At least one processing unit 902 included in the computing device 900 may process operations corresponding to the image encoding apparatus and method according to an embodiment of the present disclosure. In particular, the processing unit 902 of the computing device 900 according to an embodiment of the present disclosure may be configured to perform operations of calculating a weight of a predetermined first size unit using a cognitive characteristic element, The method comprising: determining a residual signal value that reflects a difference value between a source block and a prediction block in a weight; and calculating a residual signal value of the first size unit based on a ratio of a residual signal value of the first size unit to a residual signal value of the picture unit. It is possible to handle an operation of allocating a bit amount.

Although the exemplary methods of this disclosure are represented by a series of acts for clarity of explanation, they are not intended to limit the order in which the steps are performed, and if necessary, each step may be performed simultaneously or in a different order. In order to implement the method according to the present disclosure, the illustrative steps may additionally include other steps, include the remaining steps except for some steps, or may include additional steps other than some steps.

The various embodiments of the disclosure are not intended to be all-inclusive and are intended to illustrate representative aspects of the disclosure, and the features described in the various embodiments may be applied independently or in a combination of two or more.

In addition, various embodiments of the present disclosure may be implemented by hardware, firmware, software, or a combination thereof. In the case of hardware implementation, one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays A general processor, a controller, a microcontroller, a microprocessor, and the like.

The scope of the present disclosure is to be accorded the broadest interpretation as understanding of the principles of the invention, as well as software or machine-executable instructions (e.g., operating system, applications, firmware, Instructions, and the like are stored and are non-transitory computer-readable medium executable on the device or computer.

Claims (1)

Calculating a weight of a predetermined first size unit using a cognitive characteristic element;
Determining a residual signal value that reflects a difference value between the original block and the prediction block in the calculated weight of the first size unit;
And allocating a bit amount for the first magnitude unit by reflecting a ratio of a residual signal value of the first magnitude unit to a residual signal value of the picture unit.

Images