JP4994767B2 - Image predictive encoding device, image predictive encoding method, image predictive encoding program, image predictive decoding device, image predictive decoding method, and image predictive decoding program - Google Patents

Description

  The present invention relates to an image predictive encoding device, an image predictive encoding method, an image predictive encoding program, an image predictive decoding device, an image predictive decoding method, and an image predictive decoding program, and in particular, a predictive code using a texture synthesis method. The present invention relates to an image predictive encoding device, an image predictive encoding method, an image predictive encoding program, an image predictive decoding device, an image predictive decoding method, and an image predictive decoding program.

  In order to efficiently transmit and store still images and moving image data, compression coding technology is used. In the case of moving images, MPEG-1 to 4 and ITU (International Telecommunication Union) H.264. 261-H. H.264 is widely used.

  In these encoding methods, encoding / decoding processing is performed after an image to be encoded is divided into a plurality of blocks. In predictive coding within a screen, a predicted signal is generated using an adjacent previously reproduced image signal (reconstructed compressed image data) in the same screen as the target block, and then the predicted signal is The differential signal subtracted from the signal of the target block is encoded. In predictive coding between screens, the adjacent reproduced image signal in the screen different from the target block is referred to, the motion is corrected, the predicted signal is generated, and the predicted signal is subtracted from the signal of the target block. Encode the difference signal.

  For example, H.M. H.264 intra-screen predictive encoding employs a method of generating a prediction signal by extrapolating already reproduced pixel values adjacent to a block to be encoded in a predetermined direction. FIG. 2 is a schematic diagram for explaining an intra-screen prediction method used for H.264. In FIG. 18A, a target block 1802 is a block to be encoded, and a pixel group 1801 composed of pixels A to M adjacent to the boundary of the target block 1802 is an adjacent region, and has already been obtained in past processing. It is the reproduced image signal.

  In this case, a prediction signal is generated by extending downward the pixel group 1801, which is an adjacent pixel immediately above the target block 1802. In FIG. 18B, the already reproduced pixels (I to L) on the left of the target block 1804 are stretched to the right to generate a prediction signal. A specific method for generating a prediction signal is described in Patent Document 1, for example. Thus, the difference between each of the nine prediction signals generated by the method shown in FIGS. 18A to 18I and the pixel signal of the target block is taken, and the one with the smallest difference value is used as the optimum prediction signal. . As described above, a prediction signal can be generated by extrapolating pixels. The above contents are described in Patent Document 1 below.

  In normal inter-screen predictive coding, a prediction signal is generated by searching for a signal similar to the pixel signal from a screen that has already been reproduced for a block to be coded. Then, a motion vector that is a spatial displacement amount between the target block and a region formed by the searched signal, and a residual signal between the pixel signal and the prediction signal of the target block are encoded. Such a method for searching for a motion vector for each block is called block matching.

  FIG. 4 is a schematic diagram for explaining the block matching process. Here, a procedure for generating a prediction signal will be described using the target block 402 on the encoding target screen 401 as an example. The screen 403 has already been reproduced, and the area 404 is an area in the same position as the target block 402. In the block matching, a search range 405 surrounding the region 404 is set, and a region 406 in which the absolute value error sum with the pixel signal of the target block 402 is minimized is detected from the pixel signal in this search range. The signal in this area 406 becomes a prediction signal, and the amount of displacement from the area 404 to the area 406 is detected as a motion vector 407. H. H.264 provides a plurality of prediction types having different block sizes for encoding motion vectors in order to cope with changes in local features of an image. H. H.264 prediction types are described in Patent Document 2, for example.

In the compression encoding of moving image data, the encoding order of each frame may be arbitrary. For this reason, there are three types of coding order methods for inter-screen prediction in which a prediction signal is generated with reference to a reproduced screen. The first method is forward prediction in which a prediction signal is generated with reference to a past reproduced screen in the reproduction order, and the second method is backward prediction with reference to a future reproduced screen in the reproduction order. The third method is bidirectional prediction in which both forward prediction and backward prediction are performed and two prediction signals are averaged. About the kind of prediction between screens, it describes in patent document 3, for example.
US Pat. No. 6,765,964 US Patent Publication No. 7003035 US Pat. No. 6,259,739

  However, in the prior art, a prediction signal of each pixel is generated by copying a reproduction pixel value including distortion due to encoding (for example, quantization noise). Therefore, the prediction signal also includes the distortion. A prediction signal including distortion due to encoding becomes a factor that decreases encoding efficiency, such as an increase in the code amount of a residual signal and deterioration in reproduction image quality.

  The influence of distortion due to encoding can be suppressed by smoothing the prediction signal, and can also be suppressed by bidirectional prediction in which two prediction signals are averaged. However, since bi-directional prediction requires encoding two motion vectors in order to generate two prediction signals on the playback side, the signal to be averaged to enhance the smoothing effect of the prediction signal Increasing the number also increases the number of motion vectors to be encoded.

  Therefore, in order to solve the above-described problem, the present invention provides an image predictive encoding device, an image predictive encoding method, an image predictive encoding program, an image predictive decoding device, and an image predictor that can efficiently generate a prediction signal. It is an object to provide a decoding method and an image predictive decoding program.

  An image predictive coding apparatus according to the present invention includes a region dividing unit that divides an input image into a plurality of regions, and a prediction signal for a target pixel signal of a target region that is a processing target among the plurality of regions divided by the region dividing unit. A prediction signal generation unit that generates a residual signal between the prediction signal generated by the prediction signal generation unit and the target pixel signal, and a residual signal generated by the residual signal generation unit A prediction related information generation unit that generates, as prediction related information, a method for generating a signal having a high correlation with the target pixel signal in the target region from the already reproduced signal. And a texture synthesis unit that generates a prediction signal of the target area based on the prediction related information generated by the prediction related information generation unit, and the texture synthesis unit includes the prediction related information A template signal of a designated area determined based on the already reproduced signal is generated from the already reproduced signal, a plurality of similar areas having a high correlation with the designated area are searched from previously reproduced images, and a plurality of target areas are determined based on the plurality of similar areas. And generating a prediction signal by processing the plurality of texture signals using a predetermined texture synthesis method, and the encoding means generates the prediction related information generated by the prediction related information generation means. It is characterized by encoding.

The image predictive coding method according to the present invention includes a region dividing step in which the image predictive coding device divides an input image into a plurality of regions, and a plurality of regions in which the image predictive coding device is divided in the region dividing step. A prediction signal generation step for generating a prediction signal for the target pixel signal in the target region to be processed, and a residual between the prediction signal generated in the prediction signal generation step and the target pixel signal by the image prediction encoding device A residual signal generating step for generating a signal;
The image predictive coding apparatus includes a coding step for coding the residual signal generated in the residual signal generating step, and the predictive signal generating step outputs a signal having a high correlation with the target pixel signal in the target region. A prediction related information generation step for generating a method of generating from a previously reproduced signal as prediction related information; and a texture synthesis step for generating a prediction signal of the target region based on the prediction related information generated in the prediction related information generation step. The texture synthesis step generates a template signal of the designated area determined based on the prediction related information from the already reproduced signal, searches a plurality of similar areas highly correlated with the designated area from the previously reproduced image, Generating a plurality of texture signals for a target region based on a plurality of similar regions, and predetermining the plurality of texture signals Generates a prediction signal by processing using a texture synthesis method, the encoding step is characterized by coding the prediction related information generated in the prediction related information generating step.

  The image predictive coding program according to the present invention includes a computer that divides an input image into a plurality of regions and a target region that is a processing target of the plurality of regions divided by the region dividing unit. Generated by a prediction signal generation unit that generates a prediction signal for a pixel signal, a residual signal generation unit that generates a residual signal between the prediction signal generated by the prediction signal generation unit and the target pixel signal, and a residual signal generation unit The prediction signal generating means functions as a prediction related information by generating a signal having a high correlation with the target pixel signal in the target region from the already reproduced signal. A prediction related information generating means for generating, and a texture synthesizing means for generating a prediction signal of the target region based on the prediction related information generated by the prediction related information generating means. The stitcher synthesizing unit generates a template signal of the designated area determined based on the prediction related information from the already reproduced signal, searches a plurality of similar areas having a high correlation with the designated area from the already reproduced image, Generating a plurality of texture signals for the target region based on the similar region, processing the plurality of texture signals using a predetermined texture synthesis method, generating a prediction signal, and encoding means generating prediction related information The prediction related information generated by the means is encoded.

  According to such an image predictive encoding device, an image predictive encoding method, and an image predictive encoding program, an input image is divided into a plurality of regions, and a target region that is a processing target of the plurality of divided regions is determined. A prediction signal for the target pixel signal is generated, a residual signal between the prediction signal and the target pixel signal is generated, and the residual signal is encoded. Here, the prediction signal is generated by processing a plurality of texture signals generated based on the prediction related information by a specific texture synthesis method. The generated prediction related information is also encoded in the same manner as the residual signal. Thus, since the prediction signal is generated based on a plurality of texture signals, noise contained in the prediction signal can be reduced. In addition, the plurality of texture signals can be efficiently generated with a small amount of prediction related information. Therefore, it is possible to efficiently generate a prediction signal with less noise.

  In the image predictive coding apparatus of the present invention, it is preferable that the designated region and the target region have the same shape. By making the shapes of these regions the same, a prediction signal can be generated more easily.

  In the image predictive coding apparatus of the present invention, the texture synthesis means divides the target area into a plurality of small areas, and for each of the divided small areas, the template signal of the designated area determined based on the prediction related information has already been reproduced. A plurality of similar regions that are generated from the signal and have a high correlation with the designated region are searched from predetermined reproduced images, and a plurality of texture signals for a small region are generated based on the plurality of similar regions, and the plurality of texture signals Is preferably processed by using a predetermined texture synthesis method, and a prediction signal for the target region is generated by synthesizing the generated prediction signals for each small region.

  In this case, the target region is divided, a prediction signal is generated for each divided small region, and a prediction signal for the target region is generated by synthesizing the prediction signal for each small region. Therefore, it is possible to further improve the performance of the prediction signal of the target region that is finally generated. That is, a prediction signal with less noise can be generated.

  In the image predictive coding apparatus of the present invention, it is preferable that the template signal is included in a plurality of texture signals.

  The template signal is generated based on prediction related information including a method for generating a signal having a high correlation with the target pixel signal in the target region. That is, the template signal is a signal having the smallest difference from the pixel signal in the target area. Therefore, a prediction signal with less noise can be generated by using a texture signal including the template signal.

  In the image predictive coding apparatus of the present invention, it is preferable that the texture synthesizing unit generates a prediction signal by performing a weighted average process using a predetermined weighting coefficient on the texture signal. In this case, since the texture signal is averaged when generating the prediction signal, it is possible to generate a prediction signal with a small statistical error.

  In the image predictive coding apparatus of the present invention, it is preferable that the prediction related information includes a motion vector indicating a spatial displacement amount of the target region. By using this motion vector, a signal having a high correlation with the target pixel signal in the target region can be derived more easily, so that a prediction signal can be generated more easily.

  In the image predictive coding apparatus of the present invention, it is preferable that the texture synthesis means searches for a plurality of similar regions having a high correlation with the designated region from a plurality of previously reproduced images.

  In this case, a similar area can be searched from a plurality of already-reproduced images, so that a similar area having a higher correlation with the designated area can be extracted. As a result, a prediction signal with less noise can be generated.

  The image predictive decoding apparatus according to the present invention extracts, from compressed data, data analysis means for extracting encoded data of prediction related information and encoded data of a residual signal related to a target region to be processed, and extracted by the data analyzing means. Generated by the residual signal restoration means for restoring the reproduction residual signal from the encoded data of the residual signal, the prediction signal generation means for generating the prediction signal for the target pixel signal in the target region, and the prediction signal generation means An image restoration means for restoring the pixel signal of the target region by adding the prediction signal and the reproduction residual signal restored by the residual signal restoration means, and the prediction signal generation means is extracted by the data analysis means The prediction related information is restored from the encoded data of the predicted prediction related information, and the template signal in the designated area determined based on the restored prediction related information is reproduced. A plurality of similar regions that are highly correlated with the designated region are searched from previously reproduced images, a plurality of texture signals for the target region are generated based on the plurality of similar regions, and the plurality of texture signals are A prediction signal is generated by processing using a predetermined texture synthesis method.

  Also, the image predictive decoding method of the present invention is a data analysis in which the image predictive decoding device extracts encoded data of prediction related information and encoded data of a residual signal related to a target region to be processed from compressed data. A residual signal restoration step in which the image predictive decoding apparatus restores a reproduction residual signal from the encoded data of the residual signal extracted in the data analysis step, and the image predictive decoding apparatus includes the target pixel signal in the target region A prediction signal generation step for generating a prediction signal for the image, and the image prediction decoding apparatus adds the prediction signal generated in the prediction signal generation step and the reproduced residual signal restored in the residual signal restoration step An image restoration step for restoring the pixel signal of the region, and the prediction signal generation step is extracted in the data analysis step The prediction related information is restored from the encoded data of the measurement related information, the template signal of the designated area determined based on the restored prediction related information is generated from the already reproduced signal, and a plurality of similar areas highly correlated with the designated area are generated. Search from a predetermined already-reproduced image, generate a plurality of texture signals for the target region based on the plurality of similar regions, and process the plurality of texture signals using a predetermined texture synthesis method Is generated.

  Further, the image predictive decoding program of the present invention is a data analysis means for extracting encoded data of prediction related information and encoded data of a residual signal related to a target region to be processed from compressed data, Residual signal restoration means for restoring a reproduction residual signal from encoded data of the residual signal extracted by the data analysis means, prediction signal generation means for generating a prediction signal for the target pixel signal in the target region, and prediction signal generation The prediction signal generated by the means and the reproduction residual signal restored by the residual signal restoration means are added to function as an image restoration means for restoring the pixel signal of the target region, and the prediction signal generation means The prediction related information is restored from the encoded data of the prediction related information extracted by the data analysis means, and is specified based on the restored prediction related information. Generates a template signal for a region from a previously reproduced signal, searches a plurality of similar regions having a high correlation with a designated region from predetermined reproduced images, and generates a plurality of texture signals for the target region based on the plurality of similar regions The prediction signal is generated by processing the plurality of texture signals using a predetermined texture synthesis method.

  According to such an image predictive decoding device, an image predictive decoding method, and an image predictive decoding program, a prediction signal is generated from encoded data of prediction related information extracted from compressed data and extracted from the compressed data. The reproduction residual signal is restored from the encoded data of the residual signal, and the pixel signal of the target region is restored by adding the prediction signal and the reproduction residual signal. Here, the prediction signal is generated by processing a plurality of texture signals generated based on the restored prediction related information by a specific texture synthesis method. Thus, since the prediction signal is generated based on a plurality of texture signals, noise contained in the prediction signal can be reduced. In addition, the plurality of texture signals can be efficiently generated with a small amount of prediction related information. Therefore, it is possible to efficiently generate a prediction signal with less noise.

  In the image predictive decoding device of the present invention, it is preferable that the designated region and the target region have the same shape. In this case, since the designated area and the target area can be set in the same manner, the prediction signal can be generated more easily.

  In the image predictive decoding device of the present invention, the prediction signal generation means restores the prediction related information from the encoded data of the prediction related information, divides the target region into a plurality of small regions, and predicts each divided small region. A template signal of a designated area determined based on the related information is generated from the already reproduced signal, a plurality of similar areas having a high correlation with the designated area are searched from previously reproduced images, and a small signal is generated based on the plurality of similar areas. Generate a plurality of texture signals for the region, process the plurality of texture signals using a predetermined texture synthesis method to generate a small region prediction signal, and synthesize the generated prediction signals for each small region Thus, it is preferable to generate a prediction signal of the target region.

  In this case, the target region is divided, a prediction signal is generated for each divided small region, and a prediction signal for the target region is generated by synthesizing the prediction signal for each small region. Therefore, it is possible to further improve the performance of the prediction signal of the target region that is finally generated. That is, a prediction signal with less noise can be generated.

  In the image predictive decoding device of the present invention, it is preferable that the template signal is included in a plurality of texture signals.

  The template signal is generated based on prediction related information including a method for generating a signal having a high correlation with the target pixel signal in the target region. That is, the template signal is a signal having the smallest difference from the pixel signal in the target area. Therefore, a prediction signal with less noise can be generated by using a texture signal including the template signal.

  In the image predictive decoding device of the present invention, it is preferable that the prediction signal generation unit generates a prediction signal by performing a weighted average process with a predetermined weighting coefficient on the texture signal. In this case, since the texture signal is averaged when generating the prediction signal, it is possible to generate a prediction signal with a small statistical error.

  In the image predictive decoding device of the present invention, it is preferable that the prediction related information includes a motion vector indicating a spatial displacement amount of the target region. By using this motion vector, a signal having a high correlation with the target pixel signal in the target region can be derived more easily, so that a prediction signal can be generated more easily.

  In the image predictive decoding device of the present invention, it is preferable that the prediction signal generating means searches a plurality of similar reproduced regions having a high correlation with the designated region from a plurality of previously reproduced images.

  In this case, a similar area can be searched from a plurality of already-reproduced images, so that a similar area having a higher correlation with the designated area can be extracted. As a result, a prediction signal with less noise can be generated.

  According to the image predictive encoding device, image predictive encoding method, image predictive encoding program, image predictive decoding device, image predictive decoding method, and image predictive decoding program of the present invention, a small amount of prediction related information such as one motion vector Since a large number of texture signals can be generated on the basis of the above, a prediction signal accompanied with a smoothing process of the texture signal can be efficiently generated.

  Hereinafter, this embodiment will be described in detail with reference to the accompanying drawings. In the description of the drawings, the same or equivalent elements are denoted by the same reference numerals, and redundant description is omitted.

  FIG. 1 is a block diagram showing an image predictive encoding device 100 according to this embodiment. The image predictive coding apparatus 100 includes an input terminal 101, a block divider 102, a prediction signal generator 103, a frame memory 104, a subtractor 105, a transformer 106, a quantizer 107, an inverse quantizer 108, and an inverse transformer. 109, an adder 110, an entropy encoder 111, and an output terminal 112. The converter 106 and the quantizer 107 function as an encoding unit.

  The input terminal 101 is a terminal for inputting a moving image signal composed of a plurality of images.

  The block divider 102 divides an image to be encoded, which is represented by a signal input from the input terminal 101, into a plurality of regions. In this embodiment, the block is divided into 8 × 8 pixels, but may be divided into other block sizes or shapes.

  The prediction signal generator 103 generates a prediction signal for the target region (target block) that is the target of the encoding process, outputs the prediction signal to the subtractor 105, and stores information on the prediction method in the entropy encoder 111. This is the output part. Specific processing of the prediction signal generator 103 will be described later.

  The subtractor 105 generates the prediction signal generated by the prediction signal generator 103 input via the line L103 from the target region obtained by the division by the block divider 102 input via the line L102. Is a part for generating a residual signal. The subtractor 105 outputs the residual signal obtained by the subtraction to the converter 106 via the line L105.

  The converter 106 is a part that performs discrete cosine transform on the residual signal obtained by subtraction. The quantizer 107 is a part that quantizes the transform coefficient that has been discrete cosine transformed by the transformer 106. The entropy encoder 111 encodes the transform coefficient quantized by the quantizer 107, encodes information related to the prediction method, and outputs the encoded information via the line L111. The output terminal 112 outputs information input from the entropy encoder 111 to the outside.

  The inverse quantizer 108 inversely quantizes the quantized transform coefficient. The inverse transformer 109 restores the residual signal by inverse discrete cosine transform. The adder 110 adds the restored residual signal and the prediction signal sent from the line L103, reproduces the signal of the target block, and stores it in the frame memory 104. In the present embodiment, the converter 106 and the inverse converter 109 are used, but other conversion processes in place of these converters may be used. Further, the converter 106 and the inverse converter 109 are not essential. Thus, in order to perform intra prediction or inter prediction for the subsequent target area, the compressed pixel signal of the target area is restored by inverse processing and stored in the frame memory 104.

  Next, the prediction signal generator 103 will be described. The prediction signal generator 103 generates a prediction signal for a target region (hereinafter referred to as a target block) that is a target of encoding processing. In this embodiment, two types of prediction methods are used. That is, the prediction signal generator 103 generates a prediction signal using at least one of an inter-screen prediction method and an intra-screen prediction method described later. Selection information for selecting the inter-screen prediction method and the intra-screen prediction method is encoded by the entropy encoder 111 together with the prediction-related information related to the generation of the quantized transform coefficient and the prediction signal, and is transmitted from the output terminal 112. Is done.

  FIG. 2 is a block diagram of the prediction signal generator 103 shown in FIG. The prediction signal generator 103 includes a prediction related information generator 201 and a texture synthesizer 202.

  The prediction related information generator 201 receives the signal of the target block input via the line L102, and is an image signal (reproduced image signal) that has already been reproduced in the past process and is input from the frame memory 104 via the line L104. Accept. The prediction related information generator 201 generates a prediction signal having the smallest error with respect to the target block from the reproduced image signal, and obtains prediction related information necessary for the generation. That is, the prediction related information generator 201 generates, as prediction related information, a method for generating a signal having a high correlation with the target pixel signal in the target region from the already reproduced signal. The prediction related information is sent to the texture synthesizer 202 via the line L201 and also sent to the entropy encoder 111 via the line L113.

  Here, the process of generating prediction related information will be described. When the inter-screen prediction method is used, the prediction related information generator 201 detects a motion vector by block matching processing using a reproduced image signal of a screen (frame or field) different from the target block as a reference image. The detected motion vector is output as prediction related information from the prediction related information generator 201 to the entropy encoder (via the line L113) and the texture synthesizer 202 (via the line L201).

  The generation method of prediction related information using the inter-screen prediction method will be described in more detail with reference to FIG. FIG. 4 is a schematic diagram for explaining a prediction related information generation method using the inter-screen prediction method, that is, a motion vector detection method. FIG. 4 (A) shows a reference image, and FIG. Indicates a screen to be encoded. Here, a case where a motion vector for the target block 402 is detected on the encoding target screen 401 will be described.

  First, the screen 403 that has already been reproduced in the past processing is set as a reference image, and a search area 405 is set on the reference screen. In FIG. 4, the search area 405 is set around the area 404 that is spatially the same position as the target block 402, but may be set at a spatially shifted position. Next, in the search area 405, an absolute value error sum (SAD) between corresponding pixels is obtained between the pixel group having the same shape as the target block 402, and an area 406 that gives the smallest SAD is searched. . At this time, the evaluation value for searching the area 406 may be other than SAD. Then, a displacement amount from the region 404 to the region 406 is detected as a motion vector 407 of the target block.

  Note that the motion vector may be detected in units of sub-regions obtained by sub-dividing the target block. In this case, the selected division method and the motion vector of each small region are included in the prediction related information. In addition, a plurality of previously reproduced screens may be used as reference images. Optimal motion vectors are detected for a plurality of reference images, and one that minimizes SAD is selected. In this case, the frame number of the selected reference image (may be one for each small area or one for the target block) is also included in the prediction related information.

  On the other hand, when the intra prediction method is used, a reproduced image signal in the same screen as the target block is input to the prediction related information generator 201, and the smallest prediction signal is given to the target block by the prediction method shown in FIG. A method is selected. The selected prediction method is output as prediction related information from the prediction related information generator 201 to the entropy encoder (via the line L113) and the texture synthesizer 202 (via the line L201).

  Next, texture signal generation / synthesis processing according to the present embodiment will be described with reference to FIG. FIG. 3 is a block diagram showing a configuration of the texture synthesizer 202 shown in FIG. The texture synthesizer 202 includes a template signal generator 301, a texture generator 302, and a synthesizer 303. The template signal generator 301 receives prediction related information input via the line L201 and also receives a reproduction image signal input from the frame memory 104 via the line L104. Then, the template signal generator 301 generates a template signal from the reproduced image signal based on the prediction related information, and outputs it to the texture generator 302 via the line L301.

  In the texture synthesizer 202, the texture generator 302 receives an image signal (reproduced image signal) that has been reproduced in the past processing and is input from the frame memory 104 via the line L104, and the template signal generator 301 receives the line. Prediction related information input via L201 is received. The texture generator 302 generates a signal of a predetermined region (specified region) as a template signal based on the input prediction related information by a method as described below, and from the reproduced image signal by template matching processing. A plurality of texture signals are generated. Further, the synthesizer 303 synthesizes the plurality of generated texture signals to generate a prediction signal for the target block. Then, the synthesizer 303 outputs the generated prediction signal to the subtracter 105 via L103.

  In the present embodiment, the shape of the region including the template signal is arbitrary. Since this shape is designated by the method as described above, in this embodiment, an area including the template signal is referred to as a “designated area”. An area searched by the template matching process is referred to as a “similar area”. Furthermore, a region to which a plurality of texture signals input to the synthesizer 303 belong, that is, a region having the same shape as the region to be predicted is referred to as a “texture region”.

  The shape of the designated area always matches the shape of the similar area. The shape of the texture region is the same as the region to be predicted. The positional relationship between the similar area and the texture area is defined in advance by “a displacement amount indicating the relative coordinate position of the two areas” on the same screen. This will be briefly described with reference to FIG. In the screen 403, it is assumed that an area 761 is a designated area and an area 751 is a texture area. At this time, a displacement amount between the coordinates of the upper left corner of the region 761 and the coordinates of the upper left corner of the region 751 is preset. An example in which the shape of the similar region is different from the shape of the texture region as shown in FIG. 11 will be described later. Here, as shown in FIG. 6, the case where the shape of the similar region and the shape of the texture region match will be described. When the shape of the similar region matches the shape of the texture region, the displacement amount indicating the relative coordinate position of the similar region and the texture region is zero. Note that the shape of the region including the template signal is not limited to the block.

  The texture generator 302 searches the reproduced image signal and obtains an absolute value error sum (SAD) between pixels corresponding to the template for an area having the same shape as the template. In this embodiment, since a signal having a high correlation with the pixel signal of the target block is selected as the template signal, the texture generator 302 can search for a signal close to the pixel signal of the target block.

  The template signal generation processing in the template signal generator 301 will be described for the case of the inter-screen prediction method shown in FIG. The case of the intra prediction method will be described later (FIG. 17). FIG. 4 shows that the prediction signal of the target block 402 of the encoding target screen 401 is generated. The prediction related information in the target block 402 is a motion vector 407 and is input to the template signal generator 301 via the line L201. The template signal generator 301 accesses the reference image 403 stored in the frame memory 104 of FIG. 1 via the line L104, and acquires an area 406 corresponding to the target block 402 based on the motion vector 407. The acquired signal of the area 406 (designated area) is output from the template signal generator 301 as a template signal.

  The designated area 406 indicated by the motion vector 407 is a signal having the smallest difference from the pixel signal of the target block 402. Therefore, if the designated area 406 is a template signal, the texture generator 302 can search for a plurality of texture signals similar to the pixel signal of the target block 402. Since the template signal in the designated area 406 can be acquired based on the decoded motion vector, the same texture signal can be searched by the decoding device.

  In the texture generator 302, a template signal is input from the template signal generator 301 via the line L301, and a reproduced image signal is input from the frame memory 104 via the line L104. The texture generator 302 performs template matching with the settings described below based on the input template signal and reproduced image signal, and generates N texture signals. The generated N texture signals are output to the combiner 303 via the line L302. The synthesizer 303 synthesizes these N texture signals by a predetermined method to be described later, generates a prediction signal, and outputs the prediction signal via the line L103. Hereinafter, in this embodiment, the value of N is described as 5, but the value of N can be set to an arbitrary value of 1 or more.

  Here, a method of generating a texture signal in the texture generator 302 will be described in detail with reference to FIGS. 5 and 6. FIG. 5 is a block diagram of the texture generator 302 shown in FIG. FIG. 6 is a schematic diagram regarding the matching process and the texture generation process according to the present embodiment. FIG. 6A shows a reference image, and FIG. 6B shows a screen to be encoded.

  The texture generator 302 includes a matching unit 501, a candidate texture selector 502, and a texture acquisition unit 503. First, the matching device 501 receives a template signal input via the line L301. Next, the matching unit 501 accesses the reproduced image signal in the frame memory 104 via the line L104, performs matching processing according to a predetermined procedure, and sequentially acquires pixel groups having the same shape as the designated region. Then, the matching unit 501 calculates an absolute value error sum (SAD) between corresponding pixels with respect to each of the acquired pixel groups. Each calculated SAD value is output to the candidate texture selector 502 via the line L501 together with coordinate information for accessing the acquired pixel group.

  In the present embodiment, the evaluation value for measuring the similarity to the template signal is SAD, but the evaluation value is not limited thereto, and may be a sum of squared errors, a value calculated from coordinate information and SAD, or the like.

  The candidate texture selector 502 compares a plurality of input SAD values with a predetermined threshold value, and extracts a pixel group having the SAD value smaller than the threshold value and its coordinate information. Then, the candidate texture selector 502 selects a maximum of N regions having small SAD values from the extracted pixel group in order, and outputs corresponding coordinate information to the texture acquirer 503 via the line L502. Here, the extracted N areas are similar areas. When the number of SAD values smaller than the threshold is less than N, the candidate texture selector 502 updates the value of N in the target block. Since all the input signals to the texture generator 302 are data restored by the decoding device, the value of N can be updated in the decoding device by the same method. Therefore, there is no need to encode the updated N value. Note that the selection method of the coordinate information does not have to be in ascending order of the SAD value. A method of selecting N pieces of coordinate information will be described later (FIG. 16).

  The texture acquisition unit 503 receives N pieces of coordinate information input from the line L502. The texture acquisition unit 503 accesses the reproduced image signal in the frame memory 104 via the line L104, and outputs an area having the same shape as the target block, that is, a texture area signal (texture signal) based on the N pieces of coordinate information. Are obtained and sequentially output to the combiner 303 via the line L302. In the present embodiment, since the texture area and the similar area coincide with each other, the displacement amount indicating the relative coordinate position between the similar area and the texture area is zero. Therefore, the texture acquisition unit 503 can directly acquire the texture signal from the coordinate information of the designated area.

  In the example illustrated in FIG. 6, a plurality of texture signals are generated for the target block 402 of the encoding target screen 401. In FIG. 6, the template signal used for the matching process is a signal of the region 406 (designated region) having the same shape as the target block, and is determined by the template signal generator 301 according to the motion vector 407.

  In the matching process of the matching unit 501, first, a search area for the target block 402 is set by a predetermined method. In the present embodiment, the entire screen 403 that has already been reproduced is set as a search area. Note that a part of the screen may be used as the search area instead of the entire screen as the search area.

  The matching unit 501 searches the entire search region 403, and sums absolute value error values (SAD) between corresponding pixels with a plurality of pixel groups (regions 601, 602,...) Having the same shape as the designated region 406. ) The search method may be a thinning search or the like.

  Next, the candidate texture selector 502 detects five regions (regions 601, 602, 603, 604, and 406) having SAD values smaller than the threshold value as candidates for similar regions. The threshold value used here may be a predetermined value, or may be a value determined by the texture synthesizer 202 to obtain an optimum texture signal and encoded by the entropy encoder 111 (not shown). In the inter-screen prediction method according to the present embodiment, the template matching search area includes the designated area 406, and thus the template signal is necessarily included in the texture signal candidates. This template signal is a signal having the smallest difference from the pixel signal of the target block, and is necessary for generating a prediction signal with little noise and a small difference from the target block.

  The candidate texture selector 502 detects the coordinates of the upper left corner of each of the detected similar regions 601 to 604 and 406, and outputs these coordinates to the texture acquisition unit 503 via the line L501. The coordinate information output here is information for accessing the frame memory, and does not have to be the upper left corner of each similar region, but may be the lower right corner. That is, the output coordinate information may be information that specifies the position.

  When the determined coordinate information is input to the texture acquisition unit 503 in this way, the texture acquisition unit 503 determines that the N similar regions according to the displacement amount indicating the relative coordinate position between the similar region and the texture region. Are respectively converted into coordinate information of the texture area. This conversion process may be performed by the candidate texture selector 502. Subsequently, the texture acquisition unit 503 accesses the reference image 403 of the frame memory 104 via the line L104 according to the converted coordinate information, and acquires N texture signal candidates. The acquired N texture signals are output to the combiner 303 via the line L302. In the example shown in FIG. 6, since the similar region and the texture region match, the texture generator 302 can omit the coordinate information conversion process when executing the example.

  For the search area shown in FIG. 6, for example, a partial area as shown in the search area 405 in FIG. 4 may be set so as to surround the area 406. When the search area does not include the area 406, the search target may be set so that the template signal is included in the texture signal. Further, the search area is not limited to a single screen area, and partial areas of a plurality of screens may be set as search areas. At this time, if a rule for selecting a screen to be searched is determined in advance, it is not necessary to encode the information on the screen. However, when the texture generator 302 selects a search area from a plurality of candidates, it is necessary to encode information relating to the search target screen and the search area. When encoding is performed, the selected screen information and search area information are output from the texture synthesizer 202 to the entropy encoder 111 via the line L113 (not shown). The encoding of the related information may be performed for each target block as in the prediction related information. However, when the change effect for each block is small, it may be performed in units of frames or sequences.

  The synthesizer 303 receives the N texture signals from the texture generator 302 and generates a prediction signal of the target block by weighted averaging processing. The synthesizer 303 includes a weighter 304 and an adder 305. The weight value in the weighter 304 is determined as 1 / N or the like. The weighting unit 304 applies a 1 / N weight to each of the input texture signals, and outputs the result to the adder 305 via the line L304. The adder 305 adds the weighted N texture signals, and outputs the added texture signal as a prediction signal via the line L103.

  Note that the weight value in the synthesizer 303 may be determined in advance, but the synthesizer 303 may obtain the weight value for each texture signal so that the optimum prediction signal is obtained. When it is determined in advance, the weight value may be an arbitrary value. For example, the synthesizer 303 may determine 1/2 for the template signal and 1 / (2 × (N−1)) for the other texture signals. Further, the synthesizer 303 may change the weight value according to the position in the region. When the synthesizer 303 determines the weight value, the signal of the target block is input to the synthesizer 303 via the line L102, the weight value that minimizes the prediction signal is obtained, and the value is obtained via the line L113 as shown in FIG. To the entropy encoder 111 (not shown). The entropy encoder 111 encodes the weight value and sends it out from the output terminal 112. At this time, the weight value may be changed in units of blocks, or in units of frames or sequences.

  The synthesizer 303 is not limited to the configuration shown in FIG. The weight value for the texture signal may be an integer, and the weighted N texture signals may be added, and then the added signal may be divided by the sum of the N weight values. At this time, if a rounding method for division is determined in advance, the same result can be obtained regardless of the arithmetic unit. For example, when the weight value for the template signal is defined as N + 1 and the weight values for the other N texture signals are defined as 1, N is further divided by 2 × N in the added signal.

  Thus, in the prediction process of the present embodiment, a plurality of texture signals are generated based on prediction related information such as motion vectors. Since these texture signals are acquired from different areas of the screen, the correlation of the included noise components (mainly high frequency components) is low. According to the present embodiment, since a plurality of texture signals having different noise components are averaged, a prediction signal with statistically less noise can be generated due to the smoothing effect.

  In the above description, the texture area has the same shape as the target area (target block). However, if the positional relationship between the encoding target block and the texture area (designated area) is clarified, the present invention can be implemented even if the target area and the texture area have different shapes. Some other examples are shown below. Similar to the description using FIG. 6, the texture signal generation procedure for the target block 402 on the screen 401 will be described.

  7 to 10 are schematic diagrams regarding matching processing and texture generation processing when the shape of the texture region is different from the shape of the target region. In this embodiment, it is assumed that the designated area and the similar area have the same shape as the texture area.

  In FIG. 7, the small area 411 indicates the upper left quarter portion of the target block 402, and the upper left area 412 indicates the designated area. Based on the prediction related information, the template signal generator 301 cuts out a region 412 that is the upper left quarter of the area from the region 406 and outputs the signal of the region 412 as a template signal. At this time, the displacement amount from the upper left corner of the area 406 to the upper left corner of the designated area 412 (texture area) is recorded in the texture synthesizer 202 together with information on the shape of the designated area and the texture area. The texture generator 302 accesses the reference image 403 via the line L104, acquires a plurality of pixel groups having the same shape as the designated area 412 by template matching, and calculates the SAD value between corresponding pixels with the template signal. To do. Then, the texture generator 302 detects N regions 711, 712, 713, 714, and 412 having a small SAD value as candidates for similar regions (texture regions) (in this embodiment, the value of N is 5).

  The texture signals in these N texture regions are weighted and averaged by the synthesizer 303, thereby generating a prediction signal for the small region 411. At this time, the position and shape of the prediction area in the target block can be known from the recorded displacement amount and the shape of the similar area. In addition, although description is abbreviate | omitted here about the setting methods, such as a search area | region, N value, and a threshold value, you may encode similarly to the case of FIG.

  A process for generating a prediction signal in another area of the target block will be described with reference to FIGS. Also in the case of FIGS. 8 to 10, as in FIG. 7, the template signal generator 301 in FIG. 3 cuts out a region of a quarter part from the region 406 in FIG. 7 based on the prediction related information. The cutout positions are different.

  In FIG. 8, the small area 421 indicates the upper right quarter of the target block 402, and the area 422 indicates the designated area. The template signal generator 301 in FIG. 3 cuts out a region 422 in the upper right quarter of the area from the region 406 in FIG. 6 based on the prediction related information, and outputs the region 422 as a template signal. At this time, the displacement amount from the upper left corner of the area 406 to the upper left corner of the designated area 422 (texture area) is recorded in the texture synthesizer 202 together with information on the shape of the designated area and the texture area. The texture generator 302 accesses the reference image 403 via the line L104, acquires a plurality of pixel groups having the same shape as the designated area 422 by template matching, and obtains the SAD value between corresponding pixels with the template signal. calculate. The texture generator 302 detects N regions 721, 722, 723, 724, and 422 having small SAD values as texture signal candidates (in this embodiment, the value of N is 5).

  The texture signals in these N texture regions are weighted and averaged by the synthesizer 303 to generate a prediction signal for the small region 421. At this time, the position and shape of the prediction area in the target block can be known from the recorded displacement amount and the shape of the similar area.

  In FIG. 9, the small area 431 indicates the lower left quarter portion of the target block 402, and the area 432 indicates the designated area. Based on the prediction related information, the template signal generator 301 cuts out a region 432 which is a lower left quarter portion of the area from the region 406 in FIG. 6, and outputs the region 432 as a template signal. At this time, the displacement amount from the upper left corner of the area 406 to the upper left corner of the designated area 432 (texture area) is recorded in the texture synthesizer 202 together with information on the shape of the designated area and the texture area. The texture generator 302 accesses the reference image 403 via the line L104, acquires a plurality of pixel groups having the same shape as the designated region 432 by template matching, and obtains the SAD value between corresponding pixels with the template signal. calculate. Then, the texture generator 302 detects N regions 731, 732, 733, 734, and 432 having a small SAD value as candidates for similar regions (texture regions) (in this embodiment, the value of N is 5).

  The texture signals in these N texture regions are weighted and averaged by the synthesizer 303 in FIG. 3, thereby generating a prediction signal for the small region 431. At this time, the position and shape of the prediction area in the target block can be known from the recorded displacement amount and the shape of the similar area.

  In FIG. 10, the small area 441 indicates the lower right quarter portion of the target block 402, and the area 442 indicates the designated area. Based on the prediction related information, the template signal generator 301 cuts out a region 442 that is a lower right quarter portion of the area from the region 406 in FIG. 6, and outputs the region 442 as a template signal. At this time, the displacement amount from the upper left corner of the area 406 to the upper left corner of the designated area 442 (texture area) is recorded in the texture synthesizer 202 together with information on the shape of the designated area and the texture area. The texture generator 302 accesses the reference image 403 via the line L104, acquires a plurality of pixel groups having the same shape as the designated region 442 by template matching, and calculates the SAD value between corresponding pixels with the template signal. calculate. Then, the texture generator 302 detects N regions 741, 742, 743, 744, and 442 having small SAD values as candidates for similar regions (texture regions) (in this embodiment, the value of N is 5).

  The texture signals in these N texture regions are weighted and averaged by the synthesizer 303, thereby generating a prediction signal for the small region 441. At this time, the position and shape of the prediction area in the target block can be known from the recorded displacement amount and the shape of the similar area.

  As described above, even if the shape of the texture area and the shape of the target block are different, if the positional relationship between the coordinate information input to the texture acquisition unit 503 and the target block is determined in advance, the target is determined based on the coordinate information. The signal of the area having the same shape as the block can be acquired.

  The process shown in FIGS. 7 to 10 is to divide the target block in FIG. 6 into four and generate a prediction signal for each divided region based on the template signal divided into four. Thus, in the present invention, a template signal can be generated by dividing a signal generated based on prediction-related information. Therefore, a high-performance prediction signal can be generated in a region unit smaller than the target block without increasing prediction-related information. Note that the number of divisions of the target block and the shape after the division are not limited to the above modification, and can be set arbitrarily.

  In the processes shown in FIGS. 6 to 10, the shape of the texture region and the shape of the similar region match. However, if the amount of displacement indicating the relative coordinate position between the similar area and the texture area is clarified, the texture area and the similar area, which are prediction target areas, may have different shapes. Examples in which the shape of the texture region and the similar region are different are shown in FIGS.

  FIG. 11 is a schematic diagram for explaining matching processing and texture generation processing for an example in which the shape of the texture region is different from the shape of the similar region. FIG. 11A shows a reference image, and FIG. Indicates a screen to be encoded. In FIG. 11, an area 462 indicates a designated area, and an area 461 indicates a texture area corresponding to the designated area.

  In the modification shown in FIG. 11, the shape of the texture region 461 and the shape of the target region 402 are the same. The template signal generator 301 cuts out a region 462 designated in advance from the region 406 shown in FIG. 6 based on the prediction related information, and outputs the template signal belonging to the designated region 462 to the texture generator 302 via the line L301. At this time, information on the amount of displacement from the upper left corner of the texture area 461 to the upper left corner of the designated area 462, the shape of the designated area (similar area), and the shape of the texture area are recorded in the texture synthesizer 202.

  The matching unit 501 accesses the reference image 403 via the line L104, acquires a plurality of pixel groups having the same shape as the designated region 462 by template matching, and calculates the SAD value between corresponding pixels with the template signal. To do. Then, the candidate texture selector 502 detects N regions 761, 762, 763, 764, and 462 having small SAD values as similar regions, and outputs the coordinate information to the texture acquirer 503 (N in this embodiment). The value of 5). The texture acquisition unit 503 acquires signals of N texture regions 751, 752, 753, 754, and 461 from the reference image 403 based on the input coordinate information and the recorded displacement amount. These N texture signals are weighted and averaged by the synthesizer 303 to generate a prediction signal for the target block 402. Here, the description of the setting method of the search area, N value, threshold value, and the like is omitted, but may be encoded as in the case of FIG.

  As described above, even if the texture region and the similar region are different in shape, if the displacement amount indicating the relative coordinate position between the similar region and the texture region is clarified, the coordinate information input to the texture acquisition unit 503 is obtained. The signal of the texture area (target area) can be acquired based on the above. That is, a partial signal (region 462 signal) can be cut out from the region (region 406) signal generated based on the prediction related information and set as a template signal. According to this method, when the signal of the region (region 406) obtained from the prediction related information includes a signal having a strong characteristic that is not suitable for the matching process, the signal region can be excluded from the template signal. There are advantages.

  In connection with the description using FIG. 11, in the present invention, it is possible to switch the weighting factor of the weighting process between the pixels that do not belong to the similar region and the pixels that belong to the texture region. For example, the signal of the designated area (area 462) corresponding to the texture area (area 461) is analyzed, and the weight value to the template signal of the designated area is set to 1 for an area where a smoothing process is not required. It is also possible to set the weight value for other texture signals to zero.

  FIG. 12 is a schematic diagram for explaining the matching process and the texture generation process for another example in which the shape of the texture region is different from the shape of the similar region. FIG. 12A shows a reference image, and FIG. B) shows a screen to be encoded. In FIG. 12, an area 472 indicates a designated area, and an area 471 indicates a texture area corresponding to the designated area. Thus, in the present invention, the designated area can be set outside the area 471 generated by the prediction related information. The designated area does not necessarily need to touch the area 471. In this modification, the shape of the texture region 471 and the shape of the target region 402 are the same.

  The template signal generator 301 cuts out the area 472 from the screen 403 based on the prediction related information, and outputs the designated area 472 as a template signal to the texture generator 302 via the line L301. At this time, information about the displacement amount from the upper left corner of the texture area 471 to the upper left corner of the designated area 472, the shape of the designated area (similar area), and the shape of the texture area are recorded in the texture synthesizer 202.

  The matching unit 501 accesses the reference image 403 via the line L104, acquires a plurality of pixel groups having the same shape as the designated region 472 by template matching, and calculates the SAD value between corresponding pixels with the template signal. To do. Then, the candidate texture selector 502 detects N regions 781, 782, 783, 784, and 472 having small SAD values as similar regions, and outputs the coordinate information to the texture acquirer 503 (N in this embodiment). The value of 5). The texture acquisition unit 503 acquires signals of N texture areas 771, 772, 773, 774, and 471 from the reference image 403 based on the input coordinate information and the recorded displacement amount. These N texture signals are weighted and averaged by the synthesizer 303 to generate a prediction signal for the target block 402. Here, the description of the setting method of the search area, N value, threshold value, and the like is omitted, but may be encoded as in the case of FIG.

  As described above, in the present invention, it is possible to set the signal of the adjacent region to the template signal for the region generated based on the prediction related information. For example, the method shown in FIG. 12 is effective when it is desired to maintain continuity of signals between adjacent blocks and the target block. In this case, as shown in FIG. 13, the motion vector 408 serving as the prediction related information may be detected by a matching process using the pixel signal of the adjacent region 1301 of the target block 402. In this case, in the search area 1304 set in the reproduced screen 403, an area 1305 in which the SAD between corresponding pixels is minimum is detected from the pixel group having the same shape as the area 1301. Then, a displacement amount from the block 1303 at the same spatial position as the target block 402 to the region 1305 is detected as a motion vector 408. Note that the positional relationship between the region 1305 and the texture region corresponding to this region is not limited.

  In the present invention, it is possible to combine the processing shown in FIGS. 7 to 10 with the processing shown in FIG. 11 or FIG. In the present embodiment, the displacement amount between the similar region and the texture region and the displacement amount between the texture region and the target region are obtained with reference to the upper left corner of each region. Different predetermined criteria may be used. Furthermore, the shapes of the designated area, the similar area, and the texture area are not limited to the above example, and can be arbitrarily set. For example, the shape of the designated area may be larger than the shape of the texture area.

  7 to 12, partial areas of a plurality of screens (a plurality of reference images) can be set as search areas as in FIG. 6. Even in this case, if a rule for selecting a screen to be searched is determined in advance, it is not necessary to encode the information on the screen.

  For the search area, M search areas (M is a value equal to or smaller than N) may be set, and the number of candidate textures to be searched for may be determined in advance. For example, if M = N and M search areas are set on different screens, candidate texture signals can be searched from a plurality of reference images, respectively. At this time, if a candidate texture signal that satisfies a condition such as the SAD threshold cannot be searched from a predetermined reference image, the number of N may be reduced, or priority is given to a reference image for searching for an alternative candidate texture signal. You may decide the order. A collection of search areas included in a plurality of reference images may be set as one search area. For example, a search area is provided in each of P reference images, and Q (N) sums of absolute value error values (SAD) between a pixel group of a designated area and a corresponding pixel group are small from P search areas. A setting of searching for a similar area of a smaller positive integer) may be included in the M search areas. In the case where search areas are set for a plurality of reference images, information (reference image frame number) for identifying a frame to which the reference image belongs is included in the coordinate information output from the candidate texture selection unit 502 to the texture acquisition unit 503. included.

  Next, the case of the intra prediction method will be described. Similar to the description using FIG. 6, a texture signal generation procedure for the target block 402 on the screen 401 will be described. In the present embodiment, as shown in FIG. 14, since intra-screen prediction processing is performed in units of 4 × 4 blocks, attention is paid to a 4 × 4 pixel region 1402 in the target block.

  FIG. 14 is a schematic diagram for explaining the matching process and the candidate texture generation process in the case of the intra prediction method. An area 1402 indicates the designated area. The template signal generator 301 generates a signal of the area 1402 from the reproduced signal of the adjacent area 1401 of the target block 402 according to the prediction method given to the prediction related information. The signal in region 1402 is output as a template signal. At this time, the displacement amount from the upper left corner of the area 402 to the upper left corner of the designated area 1402 (texture area) and the shapes of the designated area (similar area) and the texture area are recorded in the texture synthesizer 202.

  The texture generator 302 uses a reproduced signal on the same screen 401 as the target block 402 as a search area. In the present embodiment, the search area is all the reproduced areas in the screen 401, but a part of the search area may be set as the search area. The texture generator 302 accesses via the line L104, acquires a plurality of pixel groups having the same shape as the designated region 1402 by template matching, and calculates the SAD value between corresponding pixels with the template signal. Then, the texture generator 302 detects N−1 regions 1403, 1404, 1405, and 1406 having small SAD values as texture signal candidates (in this embodiment, the value of N is 5). The N signals obtained by adding the template signal 1402 to the N−1 texture signals are weighted and averaged by the synthesizer 303, thereby generating a prediction signal of the area 1402 in the upper left quarter of the target block 402. . At this time, the shape of the texture region in the target region is known from the shape of the recorded texture region.

  Note that the template signal generated by the intra-screen prediction method can also be applied to the inter-screen prediction method. In this case, in the example shown in FIG. 14, another screen that has already been reproduced is included in the search area. The description of the setting method of the search area, N value, threshold value, etc. is omitted, but may be encoded as in the case of FIG.

  In the present embodiment, the setting of the template signal generator 301 and the texture generator 302 of the texture synthesizer 202 is changed to correspond to a plurality of texture synthesis methods as shown in FIGS. However, the image predictive coding apparatus 100 may be adapted to other methods. For example, a method may be used in which a plurality of texture synthesizers 202 having different settings of the template signal generator 301 and the texture generator 302 are prepared and used by switching adaptively. Means (for example, means for selecting a method having the smallest error from the target block) for selecting one method from a plurality of texture synthesis methods are prepared (not shown), and the selected texture synthesis method is entropy encoder 111. By configuring the image predictive coding apparatus 100 so as to perform coding, the texture synthesis method can be adapted.

  FIG. 15 is a flowchart showing the procedure of the texture generation method for the target block in the texture synthesizer 202 shown in FIG. First, the template signal generator 301 acquires a template signal of a designated region based on the prediction related information (step S302).

  Next, the absolute value sum (SAD value) with the pixel group having the same shape in the designated area and the search area is obtained by the matching unit 501 of the texture generator 302 (step S303). Then, the candidate texture selector 502 of the texture generator 302 compares the SAD value with a predetermined threshold value (step S304). If it is determined that the SAD value is smaller than the threshold value, the process proceeds to step S305, and it is determined that it is not. If YES, go to step S306.

  The candidate texture selector 502 of the texture generator 302 compares the obtained SAD value with the previous SAD value. Then, when the obtained SAD values are included in order of increasing number N, the searched similar area is added to the similar area candidates, and the similar area candidates are updated (step S305).

  Thereafter, the matching unit 501 of the texture generator 302 confirms whether or not all search areas have been searched (step S306). If it is determined that not all have been searched, the process returns to step S303, and the absolute value sum SAD between the designated region and another pixel group having the same shape in the search region is obtained by the matching unit 501 of the texture generator 302.

  When all the search has been completed, a plurality of texture signal candidates are acquired by the texture acquisition unit 503 of the texture generator 302 and weighted and averaged by the combiner 303 to generate a texture signal (step) S308). This completes the processing for one target block (step S309).

  In step S305, N small SAD values are selected as texture signal candidates, but another method may be used. In order to obtain a texture signal that is statistically less noisy by averaging the candidate texture signals, the plurality of texture signal candidates should have low correlation with each other. That is, it may be preferable that the candidate signals have diversity. As one simple method for providing diversity to candidate texture signals, N texture signals are selected in order of increasing spatial position with respect to the designated region from among similar regions having SAD values smaller than a predetermined threshold. There is a method. In addition, as shown in FIG. 16, there is a method of selecting N regions so that the spatial distances between candidate similar regions are increased.

  FIG. 16 is a flowchart showing the procedure of one method for selecting N similar regions in step S305 shown in FIG. In FIG. 16, A [p] [i] indicates the coordinate information on the reference image for the pixel area searched for the p-th among the pixel areas having the SAD value smaller than the predetermined threshold. The value i is set to 0 indicating horizontal coordinates or 1 indicating vertical coordinates. C [k] [i] indicates the coordinates of the kth similar region selected in step S305. Hereinafter, when it is not necessary to distinguish between the horizontal coordinate and the vertical coordinate, they are denoted as A [p] and C [p].

  First, the coordinates of the designated area obtained by the template generator are stored in C [1] (step S502), and the number K of similar areas selected so far is set to 1.

  Next, for a pixel region whose SAD value is smaller than a predetermined threshold, coordinates A [p] and C [1] on the reference image are compared, and A [m having the largest distance difference (vector magnitude). ] Is selected (step S503). Then, the value of K is updated to K + 1 (= 2), and C [K] = A [m] is set (step S504). At this time, since A [m] is determined as the similar region, the mth searched texture is deleted from the selection target.

  Thereafter, K and N are compared (step S505). If it is determined that K is smaller than N, the process returns to step S503, and if it is determined that K = N, the process is terminated.

  Subsequently, in step S503, each of C [1] to C [K] is compared with the remaining A [p], and A [m] having the largest sum of the distance differences is selected. According to this method, the distance between the selected candidate similar regions can be maintained.

  FIG. 17 is a flowchart illustrating a procedure of an image predictive encoding method in the image predictive encoding device 100 according to the present embodiment. First, the prediction related information generator 201 performs prediction processing using the pixel signal of the target block, and generates prediction related information (step S102). Next, the texture synthesizer 202 generates a plurality of texture signal candidates based on the prediction related information, and generates a prediction signal of the target block by the texture synthesis process (step S103). At this time, the texture synthesizer 202 may generate a prediction signal using a plurality of texture generation methods, and may select a prediction signal having the smallest error square sum between pixels corresponding to the signal of the target block.

  The residual signal indicating the difference between the prediction signal determined in this way and the signal of the target block is encoded by the transformer 106, the quantizer 107, and the entropy encoder 111 (step S104). The encoded residual signal, prediction related information, and texture generation method selection information (if necessary) are output via the output terminal 112 (step S105). Here, the prediction related information includes a motion signal and a reference image frame number in the inter-screen prediction method, selection information related to selection of the prediction method in the intra-screen prediction method, the intra-screen prediction method, and the inter-screen prediction method, and a texture signal. Search region (search for candidate texture signals) used for generation and selection of candidates, value of N (number of candidate texture signals), threshold value (evaluation value for measuring similarity between candidate texture signal and template signal) and weight value ( Various setting information such as a weighting coefficient at the time of texture synthesis may be included.

  After these processes or in parallel with these processes, the residual signal encoded to predictively encode the subsequent target block is decoded by the inverse quantizer 108 and the inverse transformer 109. Then, the adder 110 adds the prediction signal to the decoded residual signal, reproduces the signal of the target block, and stores it as a reference image in the frame memory 104 (step S106). If all the target blocks have not been processed, the process returns to step S102, and the process for the next target block is performed. If all the target blocks have been processed, the process ends (step S108).

  Next, the image predictive decoding method according to this embodiment will be described. FIG. 19 is a block diagram showing an image predictive decoding device 1900 according to this embodiment. The image predictive decoding apparatus 1900 includes an input terminal 1901, a data analyzer 1902, an inverse quantizer 1903, an inverse transformer 1904, an adder 1905, an output terminal 1906, a frame memory 1907, and a texture synthesizer 1908. The decoding means by the inverse quantizer 1903 and the inverse transformer 1904 may be performed using other than these. Further, the inverse converter 1904 may be omitted.

  The input terminal 1901 inputs compressed data that has been compression-encoded by the above-described image predictive encoding method. This compressed data includes a residual signal and prediction related information that are encoded by predicting a target block obtained by dividing an image into a plurality of blocks. Here, in the prediction related information, in addition to the prediction method in the intra prediction method, the selection information regarding the selection of the intra prediction method and the inter prediction method, the motion vector and the reference image frame number in the inter prediction method, Search region used for candidate generation and selection (search for candidate texture signals), N value (number of candidate texture signals), threshold value (evaluation value for measuring similarity between candidate texture signals and template signal), and weight value (texture) Various setting information such as a weighting factor at the time of synthesis may be included. Note that the various setting information is not necessary when the method is predetermined.

  The data analyzer 1902 extracts the residual signal, prediction related information, and various setting information (if necessary) of the target block by analyzing the compressed data input to the input terminal 1901.

  The inverse quantizer 1903 receives the residual signal of the target block via the line L1902a and performs inverse quantization. The inverse transformer 1904 performs inverse discrete cosine transform on the inversely quantized data.

  The texture synthesizer 1908 inputs prediction auxiliary signal information from the data analyzer 1902 via the line L1902b. The texture synthesizer 1908 obtains a reference image from the frame memory 907 based on the prediction related information (in a method described later), and generates a prediction signal. The texture synthesizer 1908 outputs the generated prediction signal to the adder 1905 via the line L1908.

  The adder 1905 adds the prediction signal generated by the texture synthesizer 1908 to the residual signal restored by the inverse quantizer 1903 and the inverse transformer 1904, and the reproduced image signal of the target block via the line L1905. The data is output to the output terminal 1906 and the frame memory 1907. The output terminal 1906 outputs to the outside (for example, a display).

  The frame memory 1907 stores the reproduction image output from the adder 1905 as a reference reproduction image as a reference reproduction image for the next decoding process.

  Next, the texture synthesizer 1908 will be described. The details of the texture synthesizer 1908 are shown in FIG. 20, and basically have the same configuration as the configuration shown in FIG. 3 or a function corresponding thereto.

  The texture signal generation / synthesis process according to the present embodiment will be described in detail with reference to FIG. FIG. 20 is a block diagram of the texture synthesizer 1908 shown in FIG. The texture synthesizer 1908 includes a template signal generator 2001, a texture generator 2002, and a synthesizer 2003. The template signal generator 2001 receives prediction-related information input via the line L2001 and also receives a reproduced image signal from the frame memory 1907 input via the line L1907. The template signal generator 2001 generates a template signal from the reproduced image signal based on the prediction related information, and outputs the template signal to the texture generator 2002 via the line L2001. The template signal is generated by the intra-screen prediction method or the inter-screen prediction method according to the selection information and prediction related information regarding the intra-screen prediction method and the inter-screen prediction method.

  The texture generator 2002 receives a template signal input from the template signal generator 2001 via the line L2001 and also receives a reproduced image signal input from the frame memory 1907 via the line L1907. The texture generator 2002 generates N texture signals from the template signal and the reproduced image signal in the same manner as the texture generator 302 of the encoding apparatus. The generated N texture signals are output to the combiner 2003 via the line L2002. The synthesizer 2003 receives N texture signals input from the line L2002. The synthesizer 2003 synthesizes these N texture signals by the same method as the synthesizer 303 of the encoding device, generates a prediction signal, and outputs the prediction signal via the line L1908.

  The synthesizer 2003 receives N candidate texture signals and generates a prediction signal of the target block by weighted averaging processing. The synthesizer 2003 includes a weighter 2004 and an adder 2005. The weight value in the weighter 2004 may be determined in advance or may be included in the compressed data. The weighter 2004 multiplies the input N texture signals by a determined weight value, and outputs the result to the adder 2005 via the line L2004. The adder 2005 adds the weighted N texture signals, and outputs the added texture signal as a prediction signal via the line L1908.

  The texture synthesizer 1908 has a search area (search for candidate texture signals), a value of N (number of candidate texture signals), a threshold value (evaluation value for measuring the similarity between the candidate texture signal and the template signal), and a weight value (texture synthesis). The texture synthesis process is executed in accordance with various setting information such as the time weight coefficient. The plurality of candidate texture signals to be generated must match the signals generated by the image prediction encoding device, and the prediction signals generated by the texture synthesis must also match the signals generated by the image prediction encoding device. I must. Therefore, various setting information used in the texture synthesizer 1908 needs to be completely matched with information used in the texture synthesizer 202 of the image predictive coding apparatus. Therefore, information that is not determined in advance is transmitted from the image predictive coding device 100 and is restored by the image predictive decoding device 1900.

  Note that the synthesizer 2003 in the image predictive decoding device 1900 is not limited to the configuration in FIG. 20, as is the synthesizer 303 in the image predictive coding device 100. For example, the weight value for the weighted texture signal may be an integer, the weighted N texture signals may be added, and the added signal may be divided by the sum of the N weight values.

  Next, the image predictive decoding method in the image predictive decoding apparatus 1900 shown in FIG. 19 will be described with reference to FIG. First, compressed data that has been compressed is input via the input terminal 1901 (step S1902). Then, the data analyzer 1902 performs entropy decoding on the compressed data, and the quantized transform coefficient and prediction related information are extracted (step S1903). Based on the prediction related information extracted here, a plurality of texture signals are generated by the texture synthesizer 1908, and a prediction signal is generated by combining the generated plurality of texture signals (step S1904).

  On the other hand, the quantized transform coefficient is inversely quantized using the quantization parameter in the inverse quantizer 1903, and inverse transform is performed in the inverse transformer 1904 to generate a reproduction difference signal (step S1905). Then, a reproduction signal is generated by adding the generated prediction signal and the reproduction difference signal, and this reproduction signal is stored in the frame memory 1907 to reproduce the next target block (step S1906). If there is next compressed data, this process is repeated again (step S1907), and all data is processed to the end (step S1908). If necessary, the process may return to step S1902 and take in compressed data.

  The image predictive encoding method and the image predictive decoding method according to the present embodiment can also be provided as stored in a recording medium as a program. Examples of the recording medium include a recording medium such as a floppy disk (registered trademark), CD-ROM, DVD, or ROM, or a semiconductor memory.

  FIG. 22 is a block diagram illustrating modules of a program that can execute the image predictive coding method. The image predictive coding program P100 includes a block division module P102, a prediction signal generation module P103, a storage module P104, a subtraction module P105, a transformation module P106, a quantization module P107, an inverse quantization module P108, an inverse transformation module P109, and an addition module P110. And an entropy encoding module P111. Note that the prediction signal generation module P103 includes a prediction related information generation module P201 and a texture synthesis module P202 as shown in FIG. Furthermore, as shown in FIG. 24, the texture signal module P202 includes a template signal generation module P301, a texture generation module P302, and a synthesis module P303.

  The functions realized by executing the above modules are the same as the functions of the image predictive coding apparatus 100 described above. That is, the function of each module of the image predictive coding program P100 includes a block divider 102, a prediction signal generator 103, a frame memory 104, a subtractor 105, a converter 106, a quantizer 107, an inverse quantizer 108, and an inverse quantizer. The functions of the converter 109, adder 110, entropy encoder 111, prediction related information generator 201, texture synthesizer 202, template signal generator 301, texture generator 302, and synthesizer 303 are the same.

  FIG. 25 is a block diagram illustrating modules of a program that can execute the image predictive decoding method. The image predictive decoding program P1900 includes a data analysis module P1902, an inverse quantization module P1903, an inverse transform module P1904, an addition module P1905, a storage module P1907, and a texture synthesis module P1908.

  The functions realized by executing the above modules are the same as those of the components of the image predictive decoding apparatus 1900 described above. That is, the functions of the modules of the image predictive decoding program P1900 are the same as the functions of the data analyzer 1902, inverse quantizer 1903, inverse transformer 1904, adder 1905, frame memory 1907, and texture synthesizer 1908.

  The image predictive encoding program P100 or the image predictive decoding program P1900 configured as described above is stored in the recording medium 10 and executed by a computer described later.

  FIG. 26 is a diagram illustrating a hardware configuration of a computer for executing a program recorded in a recording medium, and FIG. 27 is a perspective view of the computer for executing a program stored in the recording medium. Note that a program that executes a program stored in a recording medium is not limited to a computer, and may be a DVD player, a set-top box, a mobile phone, or the like that includes a CPU and performs processing and control by software.

  As shown in FIG. 26, the computer 30 includes a reading device 12 such as a floppy disk drive device, a CD-ROM drive device, a DVD drive device, a working memory (RAM) 14 in which an operating system is resident, and a recording medium 10. A memory 16 for storing programs stored therein, a display device 18 such as a display, a mouse 20 and a keyboard 22 as input devices, a communication device 24 for transmitting and receiving data and the like, and a CPU 26 for controlling execution of the programs. And. When the recording medium 10 is inserted into the reading device 12, the computer 30 can access the image predictive encoding / decoding program stored in the recording medium 10 from the reading device 12, and by the image encoding / decoding program, It becomes possible to operate as an image encoding device or an image decoding device according to the present embodiment.

  As shown in FIG. 27, the image predictive coding program and the image decoding program may be provided via a network as a computer data signal 40 superimposed on a carrier wave. In this case, the computer 30 can store the image predictive encoding program or the image decoding program received by the communication device 24 in the memory 16 and execute the image predictive encoding program or the image predictive decoding program.

  As described above, according to the image predictive coding apparatus according to the present embodiment, first, the moving image input to the input terminal 101 is divided into a plurality of regions by the block divider 102, and the plurality of divided regions is obtained. The prediction signal generator 103 generates a prediction signal for the target pixel signal of the target block to be processed. Then, a residual signal between the prediction signal and the target pixel signal is generated by the subtractor 105, and the residual signal is encoded by the entropy encoder 111. Here, the prediction signal is generated by adding a plurality of texture signals generated based on the prediction related information by the synthesizer 303. The generated prediction related information is also encoded by the entropy encoder 111 in the same manner as the residual signal. Thus, since the prediction signal is generated based on a plurality of texture signals, noise contained in the prediction signal can be reduced. In addition, the plurality of texture signals can be efficiently generated with a small amount of prediction related information. Therefore, it is possible to efficiently generate a prediction signal with less noise.

  In addition, the image predictive encoding device 100 according to the present embodiment can set the shape of the designated region and the shape of the target region (for example, the shape of the region 406 and the shape of the region 402) to be the same. Thus, the prediction signal can be generated more easily.

  Further, the image predictive coding apparatus 100 according to the present embodiment divides the target region 402, generates a prediction signal for each of the divided small regions (small regions 411, 421, 431, and 441), and generates a prediction signal for each small region. It is possible to generate a prediction signal for the target region 402 by synthesizing the prediction signal. Therefore, it is possible to further improve the performance of the prediction signal of the target region 402 to be finally generated, that is, to generate a prediction signal with less noise.

  In the image predictive coding device 100 according to the present embodiment, the candidate texture selector 502 selects a template signal having the smallest difference from the pixel signal in the target region as one of the texture signals. That is, the template signal is included in the texture signal. Therefore, a prediction signal with less noise can be generated.

  Further, in the image predictive encoding device 100 according to the present embodiment, since the synthesizer 303 performs weighted averaging processing on the texture signal, a plurality of texture signals are averaged. Therefore, it is possible to generate a prediction signal with a small statistical error.

  Further, in the image predictive coding device 100 according to the present embodiment, the prediction related information includes a motion vector indicating the spatial displacement amount of the target region. Therefore, a signal having a high correlation with the target pixel signal in the target region can be derived more easily, and a prediction signal can be generated more easily.

  In the image predictive decoding apparatus 1900 according to the present embodiment, the data analyzer 1902 extracts prediction related information and a residual signal from the compressed data. Thereafter, the texture synthesizer 1908 generates a prediction signal based on the prediction related information, and the inverse quantizer 1903 and the inverse transformer 1904 restore the residual signal. Then, an adder 1905 adds the prediction signal and the residual signal to restore the reproduction pixel signal in the target area. Here, the prediction signal is generated when the synthesizer 2003 processes a plurality of texture signals generated based on the prediction-related information restored by the texture generator 2002 using a weighted averaging process. Thus, since the prediction signal is generated based on a plurality of texture signals, noise contained in the prediction signal can be reduced. In addition, the plurality of texture signals can be efficiently generated with a small amount of prediction related information. Therefore, it is possible to efficiently generate a prediction signal with less noise.

  Further, the image predictive decoding device 1900 according to the present embodiment can set the shape of the designated region and the shape of the target region to be the same, and therefore can generate a prediction signal more easily.

  Also, the image predictive decoding apparatus 1900 according to the present embodiment divides the target region, generates a prediction signal for each divided small region, and synthesizes the prediction signal for each small region, thereby generating the prediction signal of the target region. It is possible to generate. Therefore, it is possible to further improve the performance of the prediction signal of the target region that is finally generated, that is, to generate a prediction signal with less noise.

  Further, in the image predictive decoding device 1900 according to the present embodiment, the texture generator 2002 can generate a texture signal including a template signal having the smallest difference from the pixel signal in the target region. A small number of prediction signals can be generated.

  Further, in the image predictive decoding device 1900 according to the present embodiment, the synthesizer 2003 performs weighted averaging processing on the texture signal, and adds the texture signals subjected to the processing to generate a prediction signal. In this way, since the texture signal is averaged when generating the prediction signal, it is possible to generate a prediction signal with a small statistical error.

  In the image predictive decoding apparatus 1900 according to the present embodiment, the data analyzer 1902 can extract prediction related information including a motion vector indicating the spatial displacement amount of the target region. Therefore, a signal having a high correlation with the target pixel signal in the target region can be derived more easily, and a prediction signal can be generated more easily.

  The present invention has been described in detail based on the embodiments. However, the present invention is not limited to the above embodiment. The present invention can be variously modified without departing from the gist thereof.

It is a block diagram which shows the image predictive coding apparatus which concerns on this embodiment. It is a block diagram which shows the prediction signal generator shown in FIG. It is a block diagram which shows the texture synthesizer shown in FIG. It is a schematic diagram for demonstrating a prediction relevant information generation process (block matching process), (A) shows a reference image, (B) shows the screen of encoding object. It is a block diagram which shows the texture generator shown in FIG. It is a schematic diagram regarding a matching process and candidate texture production | generation process, (A) shows a reference image, (B) shows the screen of encoding object. It is a 1st schematic diagram regarding the modification of a matching process and a candidate texture production | generation process, (A) shows a reference image, (B) shows the screen of encoding object. It is a 2nd schematic diagram regarding the modification of a matching process and candidate texture production | generation process, (A) shows a reference image, (B) shows the screen of encoding object. It is a 3rd schematic diagram regarding the modification of a matching process and candidate texture production | generation process, (A) shows a reference image, (B) shows the screen of encoding object. It is a 4th schematic diagram regarding the modification of a matching process and candidate texture production | generation process, (A) shows a reference image, (B) shows the screen of encoding object. It is a schematic diagram regarding the matching processing and candidate texture generation processing in the first case where the shape of the template signal and the shape of the texture signal are different, (A) shows a reference image and (B) shows a screen to be encoded. . It is a schematic diagram regarding the matching processing and candidate texture generation processing in the second case where the shape of the template signal and the shape of the texture signal are different, (A) shows a reference image and (B) shows a screen to be encoded. . It is a schematic diagram for demonstrating the modification of a prediction relevant information generation process (block matching process), (A) shows a reference image, (B) shows the screen of encoding object. It is a schematic diagram for demonstrating the modification which uses the prediction method in a screen in a matching process and a candidate texture production | generation process. It is a flowchart for demonstrating the procedure of the production | generation method and synthesis | combination method of a candidate texture signal. It is a flowchart for demonstrating the modification of the procedure of the selection method of a candidate texture signal. It is a flowchart which shows the procedure of the image prediction encoding method of the image prediction encoding apparatus shown in FIG. It is a schematic diagram for demonstrating the prediction method in a screen. It is a block diagram which shows the image prediction decoding apparatus which concerns on this embodiment. It is a block diagram which shows the texture synthesizer shown in FIG. It is a flowchart which shows the procedure of the image prediction decoding method of the image prediction decoding apparatus shown in FIG. It is a block diagram which shows the program which can perform the image predictive coding method which concerns on this embodiment. It is a block diagram which shows the prediction signal generation module shown in FIG. It is a block diagram which shows the texture synthetic | combination module shown in FIG. It is a block diagram which shows the program which can perform the image predictive decoding method which concerns on this embodiment. It is a figure which shows the hardware constitutions of the computer for performing the program recorded on the recording medium. It is a perspective view of a computer for executing a program stored in a recording medium.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 100 ... Image predictive coding apparatus, 101 ... Input terminal, 102 ... Block divider, 103 ... Prediction signal generator, 104 ... Frame memory, 105 ... Subtractor, 106 ... Converter, 107 ... Quantizer, 108 ... Inverse Quantizer, 109 ... Inverse transformer, 110 ... Adder, 111 ... Entropy encoder, 112 ... Output terminal, 201 ... Prediction related information generator, 202 ... Texture synthesizer, 301 ... Template signal generator, 302 ... Texture generator 303 ... Synthesizer 1900 Image predictive decoder 1901 Input terminal 1902 Data analyzer 1903 Inverse quantizer 1904 Inverse transformer 1905 Adder 1906 Output terminal 1907: frame memory, 1908: texture synthesizer, 2001: template signal generator, 2002: texture Formed unit, 2003 ... synthesizer.

Claims (18)

Area dividing means for dividing the input image into a plurality of areas;
A prediction signal generating unit that generates a prediction signal for a target pixel signal of a target region that is a processing target among a plurality of regions divided by the region dividing unit;
Residual signal generation means for generating a residual signal between the prediction signal generated by the prediction signal generation means and the target pixel signal;
Encoding means for encoding the residual signal generated by the residual signal generating means,
The prediction signal generating means includes
Prediction related information generating means for generating, as prediction related information, a method of generating a signal having a high correlation with the target pixel signal of the target region from the already reproduced signal;
Texture synthesis means for generating a prediction signal of the target region based on the prediction related information generated by the prediction related information generation means,
The texture synthesizing unit generates a template signal of a designated area determined based on the prediction related information from the already reproduced signal, and searches a plurality of similar areas having a high correlation with the designated area from previously reproduced images. , Generating a plurality of texture signals for the target region based on the plurality of similar regions, and generating a prediction signal by processing the plurality of texture signals using a predetermined texture synthesis method,
The image encoding / encoding device, wherein the encoding unit encodes the prediction related information generated by the prediction related information generation unit.   The image predictive coding apparatus according to claim 1, wherein the designated region and the target region have the same shape.   The texture synthesizing unit divides the target region into a plurality of small regions, generates a template signal of a designated region defined based on the prediction related information for each of the divided small regions from the already reproduced signal, A plurality of similar regions having a high correlation with the designated region are searched from predetermined reproduced images, a plurality of texture signals for the small region are generated based on the plurality of similar regions, and the plurality of texture signals are determined in advance. The prediction signal of the small area is generated by processing using a texture synthesis method, and the prediction signal of the target area is generated by synthesizing the generated prediction signal of each small area. The image predictive coding device according to 1.   The image predictive coding apparatus according to claim 1, wherein the template signal is included in the plurality of texture signals.   The said texture synthetic | combination means produces | generates the said prediction signal by performing the weighted average process by a predetermined weighting coefficient with respect to the said texture signal, The Claim 1 characterized by the above-mentioned. Image predictive coding apparatus.   The image prediction encoding apparatus according to any one of claims 1 to 5, wherein the prediction related information includes a motion vector indicating a spatial displacement amount of the target region.   The image prediction code according to claim 1, wherein the texture synthesizing unit searches a plurality of previously reproduced images for a plurality of similar regions having a high correlation with the designated region. Device. A data analysis means for extracting encoded data of prediction related information and encoded data of residual signal related to a target area to be processed from compressed data;
Residual signal restoration means for restoring a reproduction residual signal from encoded data of the residual signal extracted by the data analysis means;
Prediction signal generating means for generating a prediction signal for the target pixel signal of the target region;
Image restoration means for restoring the pixel signal of the target area by adding the prediction signal generated by the prediction signal generation means and the reproduction residual signal restored by the residual signal restoration means,
The prediction signal generation means restores prediction related information from the encoded data of the prediction related information extracted by the data analysis means, and generates a template signal in a designated area determined based on the restored prediction related information A plurality of similar regions having a high correlation with the designated region are searched from previously reproduced images, a plurality of texture signals for the target region are generated based on the plurality of similar regions, and the plurality of textures An image predictive decoding apparatus that generates a prediction signal by processing a signal using a predetermined texture synthesis method.   The image predictive decoding apparatus according to claim 8, wherein the designated region and the target region have the same shape.   The prediction signal generation means restores the prediction related information from the encoded data of the prediction related information, divides the target region into a plurality of small regions, and for each divided small region, based on the prediction related information A template signal of a designated area determined in advance is generated from the reproduced signal, a plurality of similar areas having a high correlation with the designated area are searched from predetermined reproduced images, and the small area is determined based on the plurality of similar areas Generating a plurality of texture signals, processing the plurality of texture signals using a predetermined texture synthesis method to generate a prediction signal for the small region, and combining the generated prediction signals for each small region The image predictive decoding apparatus according to claim 8, wherein a prediction signal of the target region is generated.   The image predictive decoding apparatus according to claim 8, wherein the template signal is included in the plurality of texture signals.   The said prediction signal production | generation means produces | generates the said prediction signal by performing the weighted average process by a predetermined weighting coefficient with respect to the said texture signal, The Claim signal characterized by the above-mentioned. Image predictive decoding apparatus.   The image prediction decoding apparatus according to any one of claims 8 to 12, wherein the prediction related information includes a motion vector indicating a spatial displacement amount of the target region.   The image prediction according to any one of claims 8 to 13, wherein the prediction signal generation unit searches a plurality of previously reproduced images for a plurality of similar regions having a high correlation with the designated region. Decoding device. A region dividing step in which the image predictive encoding device divides the input image into a plurality of regions;
A prediction signal generating step in which the image predictive coding device generates a prediction signal for a target pixel signal of a target region that is a processing target among a plurality of regions divided in the region dividing step;
The image predictive coding apparatus generates a residual signal between the prediction signal generated in the prediction signal generation step and the target pixel signal; and
The image predictive coding apparatus comprises: an encoding step for encoding the residual signal generated in the residual signal generating step;
The predicted signal generation step includes:
A prediction related information generating step for generating, as prediction related information, a method of generating a signal having a high correlation with the target pixel signal of the target region from the already reproduced signal;
A texture synthesis step of generating a prediction signal of the target region based on the prediction related information generated in the prediction related information generation step,
The texture synthesis step generates a template signal of a designated area determined based on the prediction-related information from the already reproduced signal, and searches a plurality of similar areas having a high correlation with the designated area from a previously reproduced image. , Generating a plurality of texture signals for the target region based on the plurality of similar regions, and generating a prediction signal by processing the plurality of texture signals using a predetermined texture synthesis method,
The encoding step encodes the prediction related information generated in the prediction related information generation step. A data analysis step in which the image predictive decoding apparatus extracts encoded data of prediction related information and encoded data of a residual signal regarding the target region to be processed from the compressed data;
The image predictive decoding apparatus, a residual signal restoration step for restoring a reproduction residual signal from encoded data of the residual signal extracted in the data analysis step;
A prediction signal generation step in which the image predictive decoding device generates a prediction signal for a target pixel signal in the target region;
Image in which the image predictive decoding apparatus restores the pixel signal of the target area by adding the prediction signal generated in the prediction signal generation step and the reproduction residual signal restored in the residual signal restoration step A restoration step, and
The prediction signal generation step restores prediction related information from the encoded data of the prediction related information extracted in the data analysis step, and generates a template signal of a designated area determined based on the restored prediction related information A plurality of similar regions having a high correlation with the designated region are searched from previously reproduced images, a plurality of texture signals for the target region are generated based on the plurality of similar regions, and the plurality of textures An image predictive decoding method, wherein a prediction signal is generated by processing a signal using a predetermined texture synthesis method. Computer
Area dividing means for dividing the input image into a plurality of areas;
A prediction signal generating unit that generates a prediction signal for a target pixel signal of a target region that is a processing target among a plurality of regions divided by the region dividing unit;
Residual signal generation means for generating a residual signal between the prediction signal generated by the prediction signal generation means and the target pixel signal;
Function as encoding means for encoding the residual signal generated by the residual signal generating means,
The prediction signal generating means includes
Prediction related information generating means for generating, as prediction related information, a method of generating a signal having a high correlation with the target pixel signal of the target region from the already reproduced signal;
Texture synthesis means for generating a prediction signal of the target region based on the prediction related information generated by the prediction related information generation means,
The texture synthesizing unit generates a template signal of a designated area determined based on the prediction related information from the already reproduced signal, and searches a plurality of similar areas having a high correlation with the designated area from previously reproduced images. , Generating a plurality of texture signals for the target region based on the plurality of similar regions, and generating a prediction signal by processing the plurality of texture signals using a predetermined texture synthesis method,
The image predictive encoding program, wherein the encoding means encodes the prediction related information generated by the prediction related information generation means. Computer
A data analysis means for extracting encoded data of prediction related information and encoded data of residual signal related to a target area to be processed from compressed data;
Residual signal restoration means for restoring a reproduction residual signal from encoded data of the residual signal extracted by the data analysis means;
Prediction signal generating means for generating a prediction signal for the target pixel signal of the target region;
Function as image restoration means for restoring the pixel signal of the target area by adding the prediction signal generated by the prediction signal generation means and the reproduction residual signal restored by the residual signal restoration means Let
The prediction signal generation means restores prediction related information from the encoded data of the prediction related information extracted by the data analysis means, and generates a template signal in a designated area determined based on the restored prediction related information A plurality of similar regions having a high correlation with the designated region are searched from previously reproduced images, a plurality of texture signals for the target region are generated based on the plurality of similar regions, and the plurality of textures An image predictive decoding program that generates a prediction signal by processing a signal using a predetermined texture synthesis method.

Images