JP2015167423A - Image encoding method, image decoding method, image encoder, image decoder, image encoding program and image decoding program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image encoding method capable of automatically establishing an operation expression of a filter.SOLUTION: An image encoding method which encodes an input image signal comprises: a filter generation step in which an operation expression of an image quality improvement filter used inside an encoding loop is adaptively configured for each input image signal using genetic programming; encoding step in which operational expression is encoded; a filtering step in which filter processing is performed to the input image signal with an operational expression of an image quality improvement filter; estimating step in which an encoding amount R for expressing procedure of adaptive configuration is estimated; a deriving step in which a square error sum D between original images is derived based on a restoration image and an input image signal derived by using an operational expression; and a cost calculation step in which lagrange cost C is obtained by C=D+λR using lagrange undetermined multiplier λ, and minimization standard when adaptively configuring the image quality improvement filter is the lagrange cost C.

Description

  The present invention relates to an image encoding method, an image decoding method, an image encoding device, an image decoding device, an image encoding program, and an image for improving image quality and reducing an encoding bit rate in lossy encoding of images and videos. The present invention relates to a decryption program.

  Conventionally, an image quality improvement filter called “in-loop filter” is one of the components of video encoding such as HEVC (High Efficiency Video Coding). This is a filter that reduces distortion included in a locally decoded image immediately after decoding. In-loop filters include a deblocking filter, Sample Adaptive Offset (hereinafter referred to as SAO; see, for example, Non-Patent Document 1), Adaptive Loop Filter (hereinafter referred to as ALF; see, for example, Non-Patent Document 2), and the like. There are places in the image where the effect of improving the image quality can be obtained even if the filter is applied, and there are places where the image quality is not improved. Therefore, the filter can be applied according to predetermined rules or additional information such as a quadtree. The area to which is applied and the area not to be applied are separated.

  The deblocking filter effectively removes the distortion (this is called block distortion) generated at the boundary between the encoded block and adaptively changing according to the position. SAO corrects a trend of vertical fluctuation (offset) of a pixel value caused by encoding / decoding according to a luminance band (band) to which a pixel of interest belongs and an edge direction. ALF effectively approximates a decoded image on which encoding distortion is superimposed by a linear spatial filter to an original image signal.

Here, ALF will be described with reference to FIG. The pixel value of the pixel of interest (the pixel to be improved in image quality from now on) is set to x2 (see FIG. 16A). The surrounding pixel values are set to x0, x1, x3, and x4 (see FIG. 16B). Separately, ALF filter coefficients C0, C1, and C2 are transmitted. The value after image quality improvement at x2 is
C0 × x0 + C1 × x1 + C2 × x2 + C1 × x3 + C0 × x4
= C0 * (x0 + x4) + C1 * (x1 + x3) + C2 * x2
It becomes.

  Any of the above image quality improvement techniques has the effect of improving the quality of locally decoded images. In motion predictive coding, if the image quality of the reference image increases, the amplitude of the differential signal after motion compensation decreases and less information needs to be transmitted, which contributes to improving video coding efficiency. Yes.

Chih-Ming Fu, Ching-Yeh Chen, Chia-Yang Tsai, Yu-Wen Huang, and Shawmin Lei: “Sample Adaptive Offset with Zero Pixel Line Buffers for LCU-based Decoding”, ITU-T and ISO / IEC, JCTVC- F055, July, 2011 W. Lai, FCA Fernandes, H. Guermazi F. Kossentini and M. Horowitz: "CE8 Subtest 4: ALF using vertical-size 5 filters with up to 9 coefficients", ITU-T and ISO / IEC, JCTVC-F303, July , 2011

  However, since the calculation formula of the in-loop filter for improving the conventional image quality is linear conversion and the procedure is fixed, the quality of the reference image is further improved, and the decoded video of higher quality with a smaller code amount. There is a problem that you can not get.

  The present invention has been made in view of such circumstances, and by automatically constructing a filter arithmetic expression, it is possible to improve the quality of a reference image and obtain a higher quality decoded video with a smaller code amount. An object is to provide an image encoding method, an image decoding method, an image encoding device, an image decoding device, an image encoding program, and an image decoding program.

  The present invention relates to an image encoding method performed by an image encoding apparatus that encodes an input image signal, and uses an arithmetic expression of an image quality improvement filter used inside an encoding loop by using genetic programming. A filter generation step that adaptively configures every time, a coding step that encodes the arithmetic expression, a filtering step that performs filtering on the input image signal by the arithmetic expression of the image quality improvement filter, and a procedure of the adaptive configuration An estimation step for estimating the amount of code R for expressing, a derivation step for deriving a square error sum D between the restored image derived using the arithmetic expression and the original image based on the input image signal, and a Lagrange A cost calculating step of obtaining a Lagrangian cost C by C = D + λR using an undetermined multiplier λ, A minimum criterion for adaptive configuration is the Lagrangian cost C, and in the encoding step, a plurality of pixels separated by a predetermined distance or more from a target pixel that is a target of image quality improvement by the image quality improvement filter are specified. The pixel is regarded as one pixel having a pixel value, and at least one of the pixels regarded as one pixel having the specific pixel value is obtained by regarding a plurality of adjacent pixels as one pixel. It is characterized by being.

  The present invention is an image decoding method performed by an image decoding apparatus that decodes an encoded image signal, a decoding step of decoding an arithmetic expression of an image quality improvement filter from the encoded image signal, and the encoded image signal A filtering step of performing a filtering process on the image signal according to an arithmetic expression of the image quality improvement filter, and in the decoding step, a predetermined distance or more from a pixel of interest that is a pixel of an image quality improvement target by the image quality improvement filter A plurality of distant pixels are regarded as one pixel having a specific pixel value, and at least one of the pixels regarded as one pixel having the specific pixel value is a plurality of adjacent pixels. Is regarded as one pixel.

  The present invention relates to an image encoding apparatus for encoding an input image signal, and a filter that adaptively configures an arithmetic expression of an image quality improvement filter used in an encoding loop for each of the input image signals using genetic programming. A generating means; an encoding means for encoding the arithmetic expression; a filter means for filtering the input image signal by the arithmetic expression of the image quality improvement filter; and a code for expressing the procedure of the adaptive configuration Using estimation means for estimating the quantity R, derivation means for deriving a square error sum D between the restored image derived using the arithmetic expression and the original image based on the input image signal, and a Lagrange undetermined multiplier λ And a cost calculation means for obtaining a Lagrangian cost C by C = D + λR, and a minimization criterion for adaptively configuring the image quality improvement filter is the Lagrangian cost. C, and the encoding means regards a plurality of pixels separated by a predetermined distance or more from a pixel of interest, which is a pixel targeted for image quality improvement by the image quality improvement filter, as one pixel having a specific pixel value. Among the pixels regarded as one pixel having the specific pixel value, at least one is characterized in that a plurality of adjacent pixels are regarded as one pixel.

  The present invention is an image decoding apparatus that decodes an encoded image signal, the decoding means for decoding an arithmetic expression of an image quality improvement filter from the encoded image signal, and the encoded image signal Filter means for performing a filter process using an arithmetic expression of the image quality improvement filter, and the decoding means includes a plurality of pixels separated by a predetermined distance or more from a pixel of interest that is a target of image quality improvement by the image quality improvement filter. Are regarded as one pixel having a specific pixel value, and at least one of the pixels regarded as one pixel having the specific pixel value regards a plurality of adjacent pixels as one pixel. It is characterized by that.

  The present invention is characterized by causing a computer to execute the image encoding method.

  The present invention causes a computer to execute the image decoding method.

  According to the present invention, since the quality of a reference image can be improved in motion compensated video coding, an effect that a higher quality decoded video can be obtained with a smaller code amount is obtained.

It is a block diagram which shows the structure of the image coding apparatus in one Embodiment of this invention. It is a flowchart which shows the processing operation of the filter processing part 110 in a loop shown in FIG. 3 is a flowchart showing details of a filter generation processing operation shown in FIG. 2. It is a block diagram which shows the structure of the image decoding apparatus in one Embodiment of this invention. 5 is a flowchart showing a processing operation of an in-loop filter processing unit 208 shown in FIG. 4. FIG. 3 is an explanatory diagram showing a simple average value filter arithmetic expression by a tree and showing upper and lower nodes, the highest node, and a terminal node. It is explanatory drawing which shows the estimation process (algorithm 1) of the code amount of an arithmetic expression. It is explanatory drawing which shows the process (algorithm 2) which calculates | requires the filter output value based on the given tree. It is explanatory drawing which shows the encoding process (algorithm 3) of a computing equation. It is explanatory drawing which shows the decoding process (algorithm 4) of a computing equation. It is explanatory drawing which shows the example of a response | compatibility of a focused pixel and a surrounding pixel value. It is explanatory drawing which shows the example of a response | compatibility of a focused pixel and a surrounding pixel value. It is explanatory drawing which shows the example of two patterns of a filter. It is explanatory drawing which shows the example of a response | compatibility of the symmetrical pixel pair of a focused pixel and a surrounding pixel value. It is explanatory drawing which shows the simple example about integration of a focused pixel and a surrounding pixel, and the integrated example of a focused pixel and a surrounding pixel. It is explanatory drawing which showed the relationship between the coefficient which the filter which ALF applies, and a pixel position in a comparatively simple example.

  Hereinafter, an image encoding device according to an embodiment of the present invention will be described with reference to the drawings. FIG. 1 is a block diagram showing a configuration of an image encoding device according to the embodiment. This image encoding apparatus receives an input image signal of an encoding target image, divides the frame of the input image signal into blocks, encodes each block, and outputs the encoded data as encoded data. The image coding apparatus shown in FIG. 1 is different from the prior art in that an in-loop filter processing unit 110 is provided. The configuration of the image encoding device shown in FIG. 1 is briefly described because it is the same as the conventional general configuration used as H.264 / AVC and other image encoding devices.

  The prediction residual signal generation unit 103 obtains a difference between the input image signal and the prediction signal output from the inter prediction processing unit 102 or the intra prediction processing unit 101, and outputs the difference as a prediction residual signal. The transform processing unit 104 performs orthogonal transform such as discrete cosine transform (DCT) on the prediction residual signal and outputs transform coefficients. The quantization processing unit 105 quantizes the transform coefficient and outputs a value after quantization. The inverse quantization processing unit 106 receives the quantized transform coefficient and performs an inverse quantization process. The inverse transform processing unit 107 performs inverse orthogonal transform on the transform coefficient after inverse quantization, which is the output of the inverse quantization processing unit 106, and outputs a prediction residual decoded signal. The decoded signal generation unit 108 adds the prediction residual decoded signal and the prediction signal output from the inter prediction processing unit 102 or the intra prediction processing unit 101, and generates a decoded signal of the encoded target block. This decoded signal is stored in the frame memory 109 for use as a reference image in the intra prediction processing unit 101. The intra prediction processing unit 101 sets a prediction mode or the like from the image stored in the frame memory 109.

  Further, for reference in the inter prediction processing unit 102, the in-loop filter processing unit 110 inputs an image stored in the frame memory 109, performs a filtering process to reduce coding distortion, and stores it in the frame memory 115. . The inter prediction processing unit 102 obtains information such as a motion vector and a motion unit from the past image stored in the frame memory 115 and the current input image signal. Information such as prediction coefficients set in the in-loop filter processing unit 110 is stored in the in-loop filter information storage unit 114. Information such as the prediction mode set in the intra prediction processing unit 101 is stored in the intra prediction information storage unit 112. The entropy encoding processing unit 113 is an output of the quantization processing unit 105, information such as a post-quantization transform coefficient, a prediction coefficient stored in the in-loop filter information storage unit 114, and an inter prediction information storage unit 111. Information such as a motion vector and information such as a prediction mode stored in the intra prediction information storage unit 112 are entropy-encoded and output as encoded data.

  Next, the operation of the in-loop filter processing unit 110 shown in FIG. 1 will be described with reference to FIG. FIG. 2 is a flowchart showing the operation of the in-loop filter processing unit 110 shown in FIG. First, the in-loop filter processing unit 110 generates an image quality improvement filter suitable for an input image (step S1). The processing content of step S1 will be described later. Next, the in-loop filter processing unit 110 encodes the filter (step S2). This can be realized by a processing operation of algorithm 3 described later. Then, the in-loop filter processing unit 110 generates a filtered pixel value using the filter (step S3). This can be realized by processing operation of algorithm 2 described later.

  Next, the detailed operation of the filter generation process (step S1) shown in FIG. 2 will be described with reference to FIG. FIG. 3 is a flowchart showing a detailed operation of the filter generation process (step S1) shown in FIG. First, the in-loop filter processing unit 110 generates a group of filter arithmetic expressions as a source of evolution through population generation processing (step S11). Next, the in-loop filter processing unit 110 performs parent selection and child individual generation in the replication selection / child generation process (step S12). “Child generation” is performed by processing such as crossover, mutation, and inversion.

  Next, the in-loop filter processing unit 110 uses the individual generated in step S12 to generate an image composed of filtered pixel values in the filter image generation process (step S13). This can be realized by processing operation of algorithm 2 described later. Next, the in-loop filter processing unit 110 calculates the error sum of the entire screen in the square error sum D calculation process (step S14), and calculates the information amount of the individual in the tree information amount R calculation process (step S15). . This can be realized by the processing operation of algorithm 1 described later.

  Next, the in-loop filter processing unit 110 determines whether or not to survive in the survival selection process using the value of the Lagrangian cost C = D + λR as an evaluation value (step S16). The Lagrange undetermined multiplier λ may be the same value as that used in the RD optimization in the encoding process, or a different one may be designated separately. Then, the in-loop filter processing unit 110 determines whether the evolution has converged (step S17). For example, as the convergence condition, the reduction ratio of C is less than a certain value (eg, 0.1%), the number of evaluations is more than a certain value (eg, 10,000 times), and the like can be applied. If it is determined that the in-loop filter processing unit 110 has not yet converged, the process returns to step S12 and repeats the processing. By this processing operation, the image quality improvement filter is automatically generated.

  Note that the processing operation shown in FIG. 3 may be used together with the conventional in-loop filter processing (deblocking filter, SAO, ALF, etc.), or a part thereof may be replaced.

  Next, an image decoding apparatus according to an embodiment of the present invention will be described. FIG. 4 is a block diagram showing the configuration of the image decoding apparatus in the embodiment. This image decoding apparatus outputs a video signal of a decoded image by inputting and decoding the encoded data encoded by the image encoding apparatus shown in FIG. The image decoding apparatus shown in FIG. 4 is different from the prior art in that an in-loop filter processing unit 208 is provided. The configuration of the image decoding apparatus shown in FIG. 4 is briefly described because it is the same as the conventional general configuration used for H.264 and other image decoding apparatuses.

  In order to perform decoding, the entropy decoding processing unit 201 inputs encoded data, entropy-decodes the quantized transform coefficient of the block to be decoded, and decodes information related to intra prediction, inter prediction, and in-loop filter, They are stored in the intra prediction information storage unit 210, the inter prediction information storage 109, and the in-loop filter information storage unit 211, respectively. The inverse quantization processing unit 204 inputs the quantized transform coefficient, inversely quantizes it, and outputs a decoded transform coefficient. The inverse transform processing unit 205 performs inverse orthogonal transform on the decoded transform signal and outputs a prediction residual decoded signal. The prediction residual decoded signal and the prediction signal output from the inter prediction processing unit 203 or the intra prediction processing unit 202 are added to generate a decoded signal of the decoding target block. This decoded signal is stored in the frame memory 207 for use as a reference image in the intra prediction processing unit 202. Further, in order to refer to the inter prediction processing unit 203, an in-loop filter processing unit 208 inputs an image stored in the frame memory 207, performs a filtering process to reduce coding distortion, and outputs it as an output signal. At the same time, it is stored in the frame memory 206. The in-loop filter processing unit 208 decodes the image quality improvement filter, uses the obtained image quality improvement filter, generates a filtered pixel value, and outputs it as an output signal.

  Next, the processing operation of the in-loop filter processing unit 208 shown in FIG. 4 will be described with reference to FIG. FIG. 5 is a flowchart showing the processing operation of the in-loop filter processing unit 208 shown in FIG. First, the in-loop filter processing unit 208 decodes the image quality improvement filter (step S21). This can be realized by processing operation of algorithm 4 described later. Subsequently, the in-loop filter processing unit 208 generates and outputs a filtered pixel value using the decoded filter (step S22). This can be realized by processing of algorithm 2 described later.

  Next, details of the in-loop filter processing unit 110 shown in FIG. 1 will be described. The in-loop filter processing unit 110 automatically constructs a filter arithmetic expression by a technique called “genetic programming”. In addition, the relationship between the amount of code required for transmitting the filter and the image quality obtained by using the filter is optimized through minimizing the Lagrangian cost.

  Genetic programming is a method of “optimization search for procedures” inspired by biological evolution, and represents procedures using trees. Then, (1) a large number of such trees are generated to form a population, and (2) a new tree (child) is generated by genetic techniques (crossover or mutation) from a tree (parent) appropriately extracted from the population. ), And (3) evaluate the tree (child) and add it to the population if it is more suitable. It is. The “evaluation” in the above (3) is performed by Lagrange's undetermined multiplier method widely used in video coding. This is to obtain a Lagrange undetermined multiplier λ input separately, a code amount R [bits] for transmitting the filter, and a Lagrange cost C = D + λR obtained from the square error sum D between the filtered image and the original signal, and C The smaller the value, the higher the rating.

  Next, correspondence between the tree structure and the arithmetic expression will be described. An arithmetic expression representing a filter can be described by a tree structure as follows. For example, an average value filter of (x1 + x5) / 2 can be expressed by a tree structure composed of non-terminal nodes and terminal nodes as shown in FIG. Here, a non-terminal node is a so-called “function”, for example, conditional branching, addition / subtraction / division / trigonometric function, square, square root, exponent / logarithm, absolute value, minimum value, maximum value, etc., from one or more inputs (arguments) It returns a single value. These functions are called “non-terminal nodes” because they take arguments and appear at the end of the tree.

  In addition, since the node itself has a value (no argument is required), a numerical value such as “0.148” or a value such as x0, x1, x2,. There are the pixel value of interest and surrounding pixel values, and other information that can be shared by the encoder and decoder. Terminal nodes and non-terminal nodes may be prepared in advance or automatically defined function (ADF) (eg, J. Koza: '' Genetic Programming II, Automatic Discovery of Reusable Programs) '', The MIT Press, 1998).

Next, estimation of the code amount of the arithmetic expression will be described. The number of bits necessary to represent the tree structure can be obtained by a recursive function (algorithm 1) shown in FIG. Here, the numerical value represented by the end node of the tree is expressed by, for example, a 10-bit fixed-point integer, and each function has a unique serial number from 0 to N-1. FUNCINFO is the following amount representing the code amount when the function is fixed-length encoded when N types of functions are used.
FUNCINFO = log_2 (N + 1)

Here, N + 1 is used to include a numerical value (such as 0.148) in addition to a function. Although fixed-length encoding is assumed here, variable-length encoding may be performed in consideration of the occurrence frequency for each function. When the highest node of the procedure (tree) of interest is root, R = tree_info (root)
Is executed, the information amount R of the tree is obtained.

Next, filter output values based on a given tree will be described. The filter output value can be obtained by a recursive procedure (algorithm 2) as shown in FIG. Although FIG. 8 shows an example where the number of arguments is up to 3, even if the upper limit of the number of arguments is increased to 4 or 5, the processing is the same and can be easily expanded. When the highest node of the procedure (tree) of interest is root, filter output = tree_eval (root)
As described above, the filter output value at the current pixel of interest can be obtained.

Next, encoding / decoding of arithmetic expressions will be described. Encoding can also be performed by the recursive procedure (algorithm 3) shown in FIG. 9, as in the information amount estimation (algorithm 1). When the highest node of the procedure (tree) of interest is root, tree_encode (root)
Can be used to encode the tree. The lower limit of the necessary code amount at this time coincides with tree_info (root).

Similarly, the decryption can be executed by a recursive function (algorithm 4) shown in FIG. In FIG. 10, “the number of arguments required by F” means that the function gives a value such as 2 if F is a binary operator such as add, 3 if it is a ternary operator, etc. The number of values to be used (known on both the encoding and decoding sides). F has that number of lower nodes immediately below that number. And
tree_decode ()
Is executed, the tree is decoded from the bitstream and returned.

  In the input image shown in FIG. 11, the image quality improvement filter encoding is performed by using a restoration value (filter output value) of an image quality improvement target pixel (target pixel) as its own pixel value (x0) and surrounding pixel values x1,. , X8 are used as candidates for “terminal node”. Although nine pixels are shown here, they may be wider or narrower, and may not be square. The pixel value may be a value immediately after local decoding in video encoding or a value after applying another loop filter.

  In addition, when the encoding method uses the above filters (deblocking filter, SAO, ALF, etc.) and the filter processing of the present embodiment is performed after these filters, the decoding device also uses information derived from them ( Since filter application / non-application area information, pixel values after filtering, and the like can be used, those values may be included in the terminal node candidates.

  As described above, in the conventional loop filter, the filter result is obtained only by linear processing related to the pixel value of the input image. In this embodiment, filters that are not confined to the linear category are generated one after another by genetic programming. Since it did in this way, the freedom degree of filter design can be improved greatly. In addition, since the generated filter is evaluated in terms of code amount-distortion by Lagrange cost, and survival selection is performed, the filter with improved performance always survives, which is not achieved using a conventional loop filter. Coding efficiency can be achieved.

In the above description, the image quality improvement filter encoding is performed by changing the pixel values of x1,..., X8 into four average values a1,. You may make it degenerate to an individual. For example, the decoded pixel value shown in FIG. 12 is a1 = (x1 + x5) / 2.
a2 = (x2 + x6) / 2
a3 = (x3 + x7) / 2
a4 = (x4 + x8) / 2
become that way. In the example of this figure, x0, a1,..., A4 are end node candidates for generating arithmetic expressions. Of course, it may be reduced or increased outside.

As a simple example, when the pixel values that can be referred to are only x1, x2, y1, and y2, 2 bits are required to represent each pixel value.
x1 01
x2 00
y1 10
y2 11

When degenerated, it can be expressed by 1 bit as follows.
a1 0
a2 1
Thus, it can be expressed by a shorter code word.

  In general, even if the image is reversed left and right, the pattern hardly changes. If viewed locally, the pattern does not change even if the image is flipped upside down. From this, it can be considered that the relationship from a pair of symmetric pixels to the relationship of two pixels in a symmetric position with respect to a certain pixel of interest or the value of the pixel of interest is the same from both. In view of such image properties, for example, in ALF, the filter coefficient values are the same for symmetrical pixel pairs, and the number of coefficients to be transmitted is reduced to about half. In FIG. 13, among the two filter patterns prepared, pattern A transmits 9 coefficients of C0... C8. For example, C0 is multiplied by the target pixel pair at the upper left and lower right, and the product is the filter result. Used for derivation. Similarly in pattern B, 8 coefficients C0... C7 are transmitted. For example, C0 is multiplied by the symmetrical pixel pair at the upper end and the lower end, and the product is used to derive the filter result.

  In this way, by reducing the number of terminal node candidates, it is possible to suppress the explosion of arithmetic expressions, speed up the generation of excellent arithmetic expressions by genetic programming, and The amount of code required can also be reduced.

  In order to express a symmetric pixel pair, it was expressed as two terminal nodes in the above description, but here it is expressed as one terminal node (average value of pixel pairs) (to restrict the degree of freedom of node expression). Therefore, the number of terminal nodes is reduced, and the information amount of the arithmetic expression can be reduced. Therefore, a filter can be generated with a smaller code amount.

  Further, since the types of terminal nodes are reduced, the variations of filters that can be generated by genetic programming are limited, and filters can be generated at higher speed. In addition, attention is paid to the image-specific properties (statistically symmetrical pixel properties are equal), so the image quality improvement performance of the generated filter is comparable to a filter with a higher degree of freedom. Therefore, a filter having substantially the same image quality improvement performance with a smaller code amount can be generated at a higher speed.

In addition, as shown in FIG. 14, the image quality improvement filter encoding names symmetric pixel pairs (x1 and y1, x2 and y2, x3 and y3,...) With respect to the target pixel position as xn and yn. An arithmetic expression to be expressed is 0.2 (x1 + y1) +0.1 (x2 + y2),
exp ((x1 + y1) / tan (x2 × y2))
As described above, it may be limited to a formula that is symmetrical with respect to the pixel pair xn, yn (the value does not change even if xn and yn are exchanged), and the efficiency of evolution may be improved. Such an expression will be referred to as an “xy symmetric expression”. The xy symmetric formula has the same contribution from the symmetric pixel pair (the contribution is the same from xn and yn), and is suitable for image filtering for the reasons described above.

The so-called “symmetric polynomial” is a symmetric formula limited to a polynomial. Here, the “xy symmetric formula” indicates a more general form including a polynomial. In the example of the above arithmetic expression, when x1 and y1 and x2 and y2 are exchanged, 0.2 (y1 + x1) +0.1 (y2 + x2),
exp ((y1 + x1) / tan (y2 × x2))
However, it is clear that it is completely equivalent to that before the exchange.

For example, if it is not xy symmetric,
x1 + 0.1 × y1,
x1 / y1
Etc.

The method for generating the xy symmetric formula is as follows.
(1) Generation by difference and even function: e (f (xn, yn) -f (yn, xn))
(2) Generation by variable exchange and product: f (xn, yn) × f (yn, xn)
(3) Variable exchange and generation by sum ... f (xn, yn) + f (yn, xn)
(4) Combining a plurality of xy symmetric expressions... F (s1 (xn, yn), s2 (xm, ym),.
(5) Special case of (4) above ... f (s (xn, yn))

  However, f (x, y) is an arbitrary function, s (x, y), s1 (x, y), s2 (x, y), ... are xy symmetric equations, e (x) is an even function (e ( x) = e (−x))), xn, yn and xm, ym are different symmetrical pixel pairs (may be the same).

For example, in the following xy symmetric formula:
cos (x1-y1) (1) case (cos is an even function),
x1 × y1 (2) 2 cases,
The case of sin (x1) + sin (y1) (3) is generated from the case of sin (x1 × y1) (4).

The xy symmetric formula once generated can be complicated by any combination of cases (4). In the example shown at the beginning,
0.2 (x1 + y1) +0.1 (x2 + y2) ... Combination of cases (3) and (4) exp ((x1 + y1) / tan (x2 × y2)) (2), (3), This is a combination of cases (4).

  In this way, it is possible to eliminate the generation of “expressions that are not symmetrical with respect to xn and yn”, which are highly likely to be unsuitable for images, and to generate and evaluate arithmetic expressions composed of xy symmetric expressions that are more likely to be suitable for images. The generation of excellent arithmetic expressions by genetic programming can be accelerated. Also, by preparing “a code word representing a pair of xn and yn”, an arithmetic expression can be expressed with a shorter code amount than when xn and yn are given independently.

As a simple example, when the pixel values that can be referred to are only x1, x2, y1, and y2, if the first 1 bit is 0, the individual pixel value is identified, and the subsequent 2 bits identify the pixel value. It shall be identified by a bit.
x1 001
x2 000
y1 010
y2 011
(X1, y1) 10
(X2, y2) 11

  Here, 10 and 11 are “codewords representing a pair of xn and yn”, and 10 bits are enough to represent (x1, y1), whereas 001 (x1 ) 010 (y1) and 6 bits are required.

  In this way, paying attention to image-specific properties (statistically symmetrical pixel properties are equal), generalize what used the average of pixel value pairs, and handle symmetric formulas (including average) for pixel value pairs. As a result, the obtained filter has a high degree of freedom. Therefore, higher performance can be expected for the image quality improvement performance of the generated filter. In addition, since the systematic symmetrical expression generation is performed so that the generation of the asymmetric expression is automatically excluded, the image quality improvement filter including the symmetrical expression can be generated at high speed. Therefore, a filter having image quality improvement performance can be generated at high speed.

  In the image quality improvement filter encoding, a plurality of pixels that are relatively far from the target pixel position may be integrated and regarded as one pixel and set to a specific value. For example, as shown in FIG. 15A, for a pixel farther than x6, three pixels are represented by, for example, an average value such as x7 and x8. Moreover, this pattern is arbitrary, for example, as shown in FIG. 15B, it can be considered innumerablely such as appropriately integrating pixels farther than x10. In any case, the description of pixels symmetrical to x0 is omitted. Here, it is possible to use an average value with the pixels at the symmetric position or to handle the symmetric pixel pair specially.

  In this way, by collecting distant pixels that contribute relatively little to the pixel of interest, it is possible to reduce the number of filter coefficients, for example, and to reduce the types of pixel values while referring to a wide area. In addition to suppressing the explosion of expression diversity, it is possible to speed up the generation of excellent arithmetic expressions by genetic programming, and to reduce the amount of code required to express the arithmetic expressions.

  In this way, focusing on the image-specific properties (correlation between pixels decreases according to the distance), unlike one in which one terminal node corresponds to one pixel value or one pixel value pair, the image quality restoration target pixel In a region some distance away from each other, a plurality of pixels are collectively represented by an average value, so the image quality improvement performance of the generated filter is comparable to a filter with a higher degree of freedom. Further, since the types of terminal nodes are reduced, the variations of filters that can be generated by genetic programming are limited, and filters can be generated at higher speed.

  A program for realizing the functions of the processing units in FIGS. 1 and 4 is recorded on a computer-readable recording medium, and the program recorded on the recording medium is read by a computer system and executed to execute an image. You may perform an encoding process and an image decoding process. Here, the “computer system” includes an OS and hardware such as peripheral devices. The “computer system” includes a WWW system having a homepage providing environment (or display environment). The “computer-readable recording medium” refers to a storage device such as a flexible medium, a magneto-optical disk, a portable medium such as a ROM and a CD-ROM, and a hard disk incorporated in a computer system. Further, the “computer-readable recording medium” refers to a volatile memory (RAM) in a computer system that becomes a server or a client when a program is transmitted via a network such as the Internet or a communication line such as a telephone line. In addition, those holding programs for a certain period of time are also included.

  The program may be transmitted from a computer system storing the program in a storage device or the like to another computer system via a transmission medium or by a transmission wave in the transmission medium. Here, the “transmission medium” for transmitting the program refers to a medium having a function of transmitting information, such as a network (communication network) such as the Internet or a communication line (communication line) such as a telephone line. The program may be for realizing a part of the functions described above. Furthermore, what can implement | achieve the function mentioned above in combination with the program already recorded on the computer system, what is called a difference file (difference program) may be sufficient.

  As mentioned above, although embodiment of this invention has been described with reference to drawings, the said embodiment is only the illustration of this invention, and it is clear that this invention is not limited to the said embodiment. is there. Accordingly, additions, omissions, substitutions, and other modifications of components may be made without departing from the spirit and scope of the present invention.

  In lossy encoding of images / videos, encoding / decoding of images is performed for the purpose of improving video quality and reducing the encoding bit rate, but can be applied to indispensable applications.

  DESCRIPTION OF SYMBOLS 101 ... Intra prediction process part, 102 ... Inter prediction process part, 103 ... Prediction residual signal generation part, 104 ... Transformation process part, 105 ... Quantization process part, 106 ... Inverse quantization processing unit, 107 ... Inverse transformation processing unit, 108 ... Decoded signal generation unit, 109 ... Frame memory, 110 ... In-loop filter processing unit, 111 ... Inter prediction information storage unit 112 ... Intra prediction information storage unit, 113 ... Entropy encoding processing unit, 114 ... In-loop filter information storage unit, 115 ... Frame memory, 201 ... Entropy decoding processing unit, 202 .. Intra prediction processing unit, 203 ... Inter prediction processing unit, 204 ... Inverse quantization processing unit, 205 ... Inverse transformation processing unit, 206 ... Frame memory, 207 ... Mumemori, 208 ... loop filter processor, 209 ... inter prediction information storage unit, 210 ... intra-prediction information storage unit, 211 ... loop filter information storage unit

Claims (6)

An image encoding method performed by an image encoding device that encodes an input image signal,
A filter generating step for adaptively configuring an arithmetic expression of an image quality improvement filter used inside the encoding loop for each of the input image signals using genetic programming;
An encoding step for encoding the arithmetic expression;
A filtering step of performing a filtering process on the input image signal by an arithmetic expression of the image quality improvement filter;
An estimation step for estimating a code amount R for expressing the procedure of the adaptive configuration;
A derivation step of deriving a square error sum D between the restored image derived using the arithmetic expression and the original image based on the input image signal;
Using a Lagrange undetermined multiplier λ, and calculating a Lagrange cost C by C = D + λR,
The Lagrangian cost C is a minimization criterion for adaptively configuring the image quality improvement filter,
In the encoding step, a plurality of pixels separated by a predetermined distance or more from a pixel of interest that is a target of image quality improvement by the image quality improvement filter is regarded as one pixel having a specific pixel value. An image encoding method, wherein at least one of pixels regarded as one pixel having a pixel value of 1 is a pixel in which a plurality of adjacent pixels are regarded as one pixel. An image decoding method performed by an image decoding apparatus for decoding an encoded image signal,
A decoding step of decoding an arithmetic expression of an image quality improvement filter from the encoded image signal;
A filtering step of performing a filtering process on the encoded image signal by an arithmetic expression of the image quality improvement filter,
In the decoding step, a plurality of pixels separated by a predetermined distance or more from a pixel of interest that is a pixel that is a target of image quality improvement by the image quality improvement filter is regarded as one pixel having a specific pixel value. An image decoding method, wherein among at least one pixel regarded as one pixel having a pixel value, a plurality of adjacent pixels are regarded as one pixel. An image encoding device for encoding an input image signal,
Filter generating means for adaptively configuring the arithmetic expression of the image quality improvement filter used inside the encoding loop for each of the input image signals using genetic programming;
Encoding means for encoding the arithmetic expression;
Filter means for performing a filtering process on the input image signal by an arithmetic expression of the image quality improvement filter;
Estimating means for estimating a code amount R for expressing the procedure of the adaptive configuration;
Derivation means for deriving a square error sum D between the restored image derived using the arithmetic expression and the original image based on the input image signal;
Cost calculating means for determining a Lagrangian cost C by C = D + λR using a Lagrange undetermined multiplier λ,
The Lagrangian cost C is a minimization criterion for adaptively configuring the image quality improvement filter,
The encoding means regards a plurality of pixels separated by a predetermined distance or more from a pixel of interest as a pixel whose image quality is improved by the image quality improvement filter as one pixel having a specific pixel value. An image encoding apparatus characterized in that at least one of pixels regarded as one pixel having a pixel value of 1 is a pixel in which a plurality of adjacent pixels are regarded as one pixel. An image decoding device for decoding an encoded image signal,
Decoding means for decoding an arithmetic expression of an image quality improvement filter from the encoded image signal;
Filter means for performing a filtering process on the encoded image signal by an arithmetic expression of the image quality improvement filter,
The decoding means regards a plurality of pixels separated by a predetermined distance or more from a pixel of interest that is a target of image quality improvement by the image quality improvement filter as one pixel having a specific pixel value. An image decoding apparatus, wherein at least one of pixels regarded as one pixel having a pixel value is obtained by regarding a plurality of adjacent pixels as one pixel.   An image encoding program for causing a computer to execute the image encoding method according to claim 1.   An image decoding program for causing a computer to execute the image decoding method according to claim 2.

Images