JP5646713B2 - Image coding apparatus, method and program, and image decoding apparatus, method and program - Google Patents

Description

The present invention relates to an image encoding apparatus, method, and program, and an image decoding apparatus, method, and program .

  In order to efficiently transmit and store still image data and moving image data, image data is conventionally compressed by a compression encoding technique. Such compression encoding techniques include MPEG1-4 and H.264 in the case of moving images. 261-H. A method such as H.264 is widely used.

  In these encoding methods, image data to be encoded is divided into a plurality of blocks, and then encoding processing and decoding processing are performed. Furthermore, MPEG4 and H.264. In a system such as H.264, in order to further increase the encoding efficiency, when encoding a target block in the screen, a prediction signal is generated using adjacent reproduced pixel signals in the same screen as the target block. The already reproduced pixel signal means a signal restored from image data once compressed. Then, the difference signal obtained by subtracting the prediction signal from the pixel signal of the target block is encoded.

  In MPEG4, the image signal of the target block is encoded after discrete cosine transform. That is, for the DC component coefficient of the target block and the AC component coefficient of the first row or the first column, the coefficient of the same component of the block above or to the left of the target block is used as a predicted value, and the difference between the two is encoded. . The determination of the predicted value is performed based on the magnitude of the gradient of the DC component between the block diagonally above the target block and the block above or to the left of the target block. Such an intra-screen prediction signal generation method is described in Patent Document 1 below.

  On the other hand, H. In H.264, a method of generating a prediction signal by extrapolating already reproduced pixel values adjacent to the target block in a predetermined direction is adopted. The intra-screen prediction signal generation in this pixel region has an advantage that the details of the image can be predicted. In FIG. FIG. 19B is a schematic diagram for explaining an intra-screen prediction signal generation method used for H.264. 2 illustrates a pixel signal stretching direction in the H.264 intra-screen prediction signal generation method. In FIG. 19A, a block 1901 is a target block, and blocks 1902 to 1904 are adjacent blocks that include pixel signals that have already been reproduced in past processing. Here, using the already-reproduced pixel group 1905 adjacent to the boundary of the target block of the block 1901, nine texture signal generation methods (that is, generation of texture signals along the eight directions shown in FIG. 19B), and A texture signal is generated by generating an average value of pixel values as a texture signal). For example, in the case of the direction “0” shown in FIG. 19B, a texture signal is generated by extending the adjacent pixel directly above the block 1901 downward, and in the case of the direction “1” shown in FIG. A texture signal is generated by stretching the already reproduced pixels on the left of the block 1901 to the right. Further, in the method using the average value of the pixel values, the average value of the entire pixel values of the pixel group 1905 is generated as a texture signal. A more specific method for generating a texture signal is described, for example, in Non-Patent Document 1 below. H. In H.264, the difference between each of the nine texture signals generated in this way and the pixel signal of the target block is taken, and the generation method of the texture signal having the smallest difference value is referred to as an optimal prediction method (hereinafter referred to as “prediction method”. Also called “mode”). The difference signal is converted into a transform coefficient matrix by discrete cosine transform, quantized, and then each quantized transform coefficient is entropy coded in the scanning order.

  When transmitting or storing image data, it is necessary to send identification information indicating the optimum prediction method determined in this way to the transmitting side in order to restore the image data. The identification information is encoded by referring to the prediction methods determined for the two blocks 1902 and 1903. That is, the identification information of the prediction method of block 1902 is compared with the identification information of the prediction method of block 1903, and the one with a smaller value is determined as reference mode information. And the identification information regarding the optimal prediction method of an object block is encoded relatively from this reference mode information.

  In order to encode the identification information related to the optimal prediction method more efficiently, in Patent Document 2, H.264 is applied to the adjacent region adjacent to the target block. Reference mode information is determined after generating a prediction signal by the same extrapolation method as H.264. Here, extrapolation processing is performed using a pixel group that is in contact with the adjacent region but not in contact with the target block. FIG. 20 is a schematic diagram for explaining the method used in Patent Document 2. In encoding the identification information related to the optimal prediction method of the target region 2001, first, the optimal prediction method of the adjacent region 2005 adjacent to the target region 2001 is determined. Specifically, the pixel group 2006 adjacent to the adjacent region 2005 is used to generate an H.264 image. A plurality of prediction signals for the adjacent region 2005 are generated by the same nine extrapolation methods as H.264, and a prediction signal having the highest correlation with the pixels of the adjacent region 2005 is determined from the plurality of prediction signals and generated. The extrapolation method for this is referred to as reference mode information. In this way, the reference mode information is determined using a pixel group that is not in direct contact with the target block.

US Pat. No. 6,148,109 JP 2007-116351 A

Iain E.G. Richardson, “H.264 and MPEG-4 video compression”, Wiley 2003, pages pp.177-183.

  H. In the encoding method using the intra-screen prediction signal generation method in the pixel area as in H.264, a number of texture signal generation methods (hereinafter, the texture signal generation method is also referred to as “mode”) are provided, and appropriate for each block. By selecting the method, it is possible to improve the prediction accuracy. However, the same transform coding method is applied to the difference signal between the pixel signal of the target block and the prediction signal regardless of the method of generating the selected texture signal. The amount may not be reduced efficiently.

  The present invention solves the above-described problems, and an object thereof is to suppress the code amount of a differential signal.

  An image encoding apparatus according to the present invention includes a region dividing unit that divides an input image into a plurality of regions, and a prediction that generates a prediction signal for a target pixel signal included in a target region that is a processing target of the plurality of regions. Signal generation means, residual signal generation means for generating a residual signal between the prediction signal and the target pixel signal, and both conversion and quantization for the residual signal generated by the residual signal generation means Alternatively, the transform means for generating a transform coefficient matrix or a quantized transform coefficient matrix is performed, and the transform coefficient matrix or the quantized transform coefficient matrix generated by the transform means is scanned to obtain the transform coefficient or quantum. Scanning means for generating the converted one-dimensional data of the transform coefficient, encoding means for encoding the transform coefficient generated by the scanning means or the one-dimensional data of the quantized transform coefficient, and the conversion function Alternatively, a decoding unit that restores the one-dimensional data of the quantized transform coefficient, and a back-scan of the transform coefficient related to the residual signal decoded by the decoding unit or the one-dimensional data of the quantized transform coefficient, A reverse scanning unit that generates a matrix or a quantized transform coefficient matrix, and performs inverse transform on the transform coefficient matrix generated by the reverse scanning unit, or is quantized by the reverse scanning unit By performing both inverse quantization and inverse transform or only inverse quantization on the transform coefficient matrix, inverse transform means for restoring the reproduction residual signal, and the prediction signal and the inverse transform generated by the prediction signal generation means Image restoration means for restoring the target pixel signal by adding the reproduction residual signal restored by the means, and image storage means for storing the restored target pixel signal. The prediction signal generation unit generates a plurality of texture signals by using different prediction signal generation methods with reference to the already reproduced pixels accumulated in the image storage unit, and among the generated plurality of texture signals, a predetermined texture signal is generated. A prediction method determining unit that selects a texture signal having the highest correlation with the target pixel signal according to an evaluation rule as a prediction signal of the target region, and the encoding unit includes the texture signal selected by the prediction method determining unit. A prediction signal generation method for encoding a mode signal indicating a prediction signal generation method for generating a texture signal, wherein the scanning unit generates a texture signal selected by the prediction method determination unit from a plurality of prepared scanning methods. And a transform method corresponding to the transform block size, to generate one-dimensional data of the transform coefficient or the quantized transform coefficient, The reverse scanning means uses the reverse scanning method corresponding to the mode information and the conversion block size from among the plurality of prepared reverse scanning methods, and uses the restored conversion coefficient or the quantized conversion coefficient. When the one-dimensional data is reverse-scanned to generate a transform coefficient matrix or a quantized transform coefficient matrix of the reproduction residual signal, and the encoding means encodes the one-dimensional data of the quantized transform coefficients The quantized transform coefficient is encoded by dividing it into the number of zero coefficients that are continuous with a significant coefficient having a magnitude of 1 or more.

  An image decoding apparatus according to the present invention is an image decoding apparatus that decodes compressed data obtained by encoding by the above-described image encoding apparatus, and is a method for generating a residual signal related to a target region to be processed from the compressed data. Decoding means for extracting encoded data and restoring one-dimensional data of transform coefficients related to a residual signal; prediction signal generating means for generating a prediction signal for a target pixel signal included in the target region; and the decoding means A reverse scanning unit that reversely scans one-dimensional data of the transform coefficient or the quantized transform coefficient related to the decoded residual signal, and generates a transform coefficient matrix or a quantized transform coefficient matrix; and the reverse scanning unit Either inverse transform is performed on the generated transform coefficient matrix, or both quantized and inverse transform or inverse transform is performed on the quantized transform coefficient matrix generated by the inverse scanning unit. By performing only the child conversion, the inverse transformation means for restoring the reproduction residual signal, the prediction signal generated by the prediction signal generation means, and the reproduction residual signal restored by the inverse conversion means are added. An image restoration means for restoring the pixel signal of the target region; and an image storage means for storing the restored target pixel signal, wherein the decoding means instructs a prediction signal generation method from the compressed data Mode information is extracted and restored, and the prediction signal generation unit is configured to generate a different texture signal by referring to already reproduced pixels stored in the image storage unit. A prediction signal generation method corresponding to the mode information decoded by the decoding means is specified, a prediction signal of the target region is generated by the specified prediction signal generation method, and the reverse running The means uses a reverse scanning method corresponding to the mode information and the transform block size restored by the decoding means, from among a plurality of prepared reverse scanning methods, A quantized transform coefficient matrix is generated.

  In addition, the image encoding apparatus and the image decoding apparatus according to the present invention can employ the following aspects.

  An image encoding apparatus according to the present invention includes a region dividing unit that divides an input image into a plurality of regions, and a prediction that generates a prediction signal for a target pixel signal included in a target region that is a processing target of the plurality of regions. Signal generation means, residual signal generation means for generating a residual signal between the prediction signal and the target pixel signal, and both conversion and quantization for the residual signal generated by the residual signal generation means Alternatively, the transform means for generating a transform coefficient matrix or a quantized transform coefficient matrix is performed, and the transform coefficient matrix or the quantized transform coefficient matrix generated by the transform means is scanned to obtain the transform coefficient or quantum. Scanning means for generating the converted one-dimensional data of the transform coefficient, encoding means for encoding the transform coefficient generated by the scanning means or the one-dimensional data of the quantized transform coefficient, and the conversion function Alternatively, a decoding unit that restores the one-dimensional data of the quantized transform coefficient, and a back-scan of the transform coefficient related to the residual signal decoded by the decoding unit or the one-dimensional data of the quantized transform coefficient, A reverse scanning unit that generates a matrix or a quantized transform coefficient matrix, and performs inverse transform on the transform coefficient matrix generated by the reverse scanning unit, or is quantized by the reverse scanning unit By performing both inverse quantization and inverse transform or only inverse quantization on the transform coefficient matrix, inverse transform means for restoring the reproduction residual signal, and the prediction signal and the inverse transform generated by the prediction signal generation means Image restoration means for restoring the target pixel signal by adding the reproduction residual signal restored by the means, and image storage means for storing the restored target pixel signal. The prediction signal generation unit generates a plurality of texture signals by using different prediction signal generation methods with reference to the already reproduced pixels accumulated in the image storage unit, and among the generated plurality of texture signals, a predetermined texture signal is generated. A prediction method determining unit that selects a texture signal having the highest correlation with the target pixel signal according to an evaluation rule as a prediction signal of the target region, and the encoding unit includes the texture signal selected by the prediction method determining unit. A prediction signal generation method for encoding a mode signal indicating a prediction signal generation method for generating a texture signal, wherein the scanning unit generates a texture signal selected by the prediction method determination unit from a plurality of prepared scanning methods. A one-dimensional data of the transform coefficient or the quantized transform coefficient is generated using a scanning method corresponding to: A regenerative residual signal is obtained by reverse scanning the restored transform coefficient or the one-dimensional data of the quantized transform coefficient using a reverse scan method corresponding to the mode information among a plurality of reverse scan methods. A transform coefficient matrix or a quantized transform coefficient matrix is generated.

  The prediction signal generation unit generates one or more texture signals by an intra-screen prediction signal generation method that refers to already reproduced pixels adjacent to the target region, and the prepared scanning methods include: A scanning method suitable for the texture signal generated by the intra-screen prediction signal generation method, wherein the prepared plurality of reverse scanning methods are inverse processing suitable for the texture signal generated by the intra-screen prediction signal generation method. A configuration including a scanning method is desirable.

  The correspondence between the scanning method and the prediction signal generation method and the correspondence between the reverse scanning method and the prediction signal generation method may be determined in advance, or the encoding unit may select the selection. A reverse scanning method corresponding to the predicted signal generation method may be configured to encode identification information for identifying from the prepared reverse scanning methods.

  An image decoding apparatus according to the present invention includes: decoding means for extracting encoded data of a residual signal related to a target region to be processed from compressed data, and restoring one-dimensional data of transform coefficients related to the residual signal; A prediction signal generation unit that generates a prediction signal for a target pixel signal included in the target region; and a reverse scan of the one-dimensional data of the transform coefficient related to the residual signal decoded by the decoding unit or the quantized transform coefficient A reverse scanning means for generating a transform coefficient matrix or a quantized transform coefficient matrix, and a reverse transform is performed on the transform coefficient matrix generated by the reverse scanning means, or a quantum generated by the reverse scanning means By performing both inverse quantization and inverse transform or only inverse quantization on the transformed transform coefficient matrix, the inverse transform means for restoring the reproduction residual signal and the prediction signal generating means In addition, by adding the intra-screen prediction signal and the reproduction residual signal restored by the inverse transformation means, the image restoration means for restoring the pixel signal of the target area, and the restored target pixel signal are accumulated Image storage means, and the decoding means extracts and restores mode information instructing a prediction signal generation method from the compressed data, and the prediction signal generation means is stored in the image storage means. A prediction signal generation method corresponding to the mode information decoded by the decoding unit is identified and specified from among a plurality of prediction signal generation methods for generating different texture signals with reference to already reproduced pixels. A prediction signal of the target region is generated by a prediction signal generation method, and the reverse scanning unit is configured to restore the mode restored by the decoding unit from a plurality of prepared reverse scanning methods. Using an inverse scanning method corresponding to the information, and generates the reproduction conversion of the residual signal coefficient matrix or quantized transform coefficient matrix.

  The prediction signal generation unit generates one or more texture signals by an intra-screen prediction signal generation method that refers to already reproduced pixels adjacent to the target region, and the prepared plurality of reverse scanning methods. Is preferably configured to include a reverse scanning method suitable for the texture signal generated by the intra-screen prediction signal generation method.

  The correspondence between the reverse scanning method and the prediction signal generation method may be determined in advance, or the decoding unit may select the identified prediction from the prepared reverse scanning methods. Identification information for selecting a reverse scanning method corresponding to a signal generation method is extracted and restored, and the reverse scanning means is restored from the plurality of prepared reverse scanning methods by the decoding means. The transform coefficient matrix of the reproduction residual signal or the quantized transform coefficient matrix may be generated using a reverse scanning method determined from mode information and the identification information.

  The image encoding device and the image decoding device according to the present invention can also be regarded as a method-related invention or a program-related invention, and can be described as follows. The invention relating to the method or the invention relating to the program has the same actions and effects.

  An image encoding method according to the present invention is an image encoding method executed by an image encoding apparatus, and includes an area dividing step for dividing an input image into a plurality of areas, and a processing target among the plurality of areas. A prediction signal generation step of generating a prediction signal for a target pixel signal included in a certain target region; a residual signal generation step of generating a residual signal of the prediction signal and the target pixel signal; and the residual signal generation step A transform step of performing transform or quantization on the residual signal generated by the step of generating a transform coefficient matrix or a quantized transform coefficient matrix, and a transform coefficient matrix generated by the transform step or A scanning step of scanning the quantized transform coefficient matrix to generate one-dimensional data of the transform coefficient or the quantized transform coefficient; A decoding step for encoding the one-dimensional data of the transformed transform coefficient or the quantized transform coefficient, a decoding step for restoring the one-dimensional data of the transform coefficient or the quantized transform coefficient, and decoding by the decoding step A reverse scan step of reverse-scanning the one-dimensional data of the transform coefficient or quantized transform coefficient related to the residual signal to generate a transform coefficient matrix or a quantized transform coefficient matrix; and the reverse scan step By performing inverse transform on the transform coefficient matrix, or by performing both inverse quantization and inverse transform or only inverse quantization on the quantized transform coefficient matrix generated by the inverse scanning step, An inverse transformation step for restoring a difference signal, the prediction signal generated by the prediction signal generation step, and a reproduction residual signal restored by the inverse transformation step. An image restoration step for restoring the target pixel signal by calculation, and an image storage step for accumulating the restored target pixel signal. In the prediction signal generation step, the image restoration step accumulates the target pixel signal. A plurality of texture signals are generated by different prediction signal generation methods with reference to already reproduced pixels, and among the generated plurality of texture signals, a texture signal having the highest correlation with the target pixel signal according to a predetermined evaluation rule Is selected as a prediction signal of the target region, and in the encoding step, mode information indicating a prediction signal generation method for generating the selected texture signal is encoded, and in the scanning step, a plurality of prepared scans A scanning method corresponding to a prediction signal generation method for generating the selected texture signal is used. Generating one-dimensional data of the conversion coefficient or the quantized conversion coefficient, and in the reverse scanning step, the reverse scanning method corresponding to the mode information is used from among a plurality of prepared reverse scanning methods. The one-dimensional data of the reconstructed transform coefficient or quantized transform coefficient is reverse-scanned to generate a transform coefficient matrix or a quantized transform coefficient matrix of a reproduction residual signal.

  An image decoding method according to the present invention is an image decoding method executed by an image decoding device, and extracts encoded data of a residual signal related to a target region to be processed from compressed data, and converts it into a residual signal. A decoding step for restoring the one-dimensional data of the transform coefficient concerned, a prediction signal generating step for generating a prediction signal for the target pixel signal included in the target region, and a transform coefficient or quantum for the residual signal decoded by the decoding step A reverse scan step of generating a transform coefficient matrix or a quantized transform coefficient matrix by reverse-scanning the one-dimensional data of the converted transform coefficients, and performing an inverse transform on the transform coefficient matrix generated by the reverse scan step Or performing both inverse quantization and inverse transformation or only inverse quantization on the quantized transform coefficient matrix generated by the inverse scanning step. And adding the inverse transformation step for restoring the reproduction residual signal, the intra-screen prediction signal generated by the prediction signal generation step, and the reproduction residual signal restored by the inverse transformation step. An image restoration step for restoring the pixel signal of the region; and an image storage step for accumulating the restored target pixel signal. In the decoding step, mode information for instructing a prediction signal generation method from the compressed data In the prediction signal generation step, the decoding is performed from among a plurality of prediction signal generation methods for generating different texture signals with reference to the already reproduced pixels accumulated in the image storage step. A prediction signal generation method corresponding to the mode information decoded in the step is specified, and the target is determined by the specified prediction signal generation method. A prediction signal of a region is generated, and in the reverse scanning step, the reproduction residual is calculated using a reverse scanning method corresponding to the mode information restored by the decoding step from among a plurality of prepared reverse scanning methods. A signal transform coefficient matrix or a quantized transform coefficient matrix is generated.

  An image encoding program according to the present invention is included in an area dividing unit that divides an input image into a plurality of areas and a target area that is a processing target of the plurality of areas. Prediction signal generation means for generating a prediction signal for the target pixel signal, residual signal generation means for generating a residual signal between the prediction signal and the target pixel signal, and a residual generated by the residual signal generation means Transform means for performing transform and / or quantization on a signal to generate a transform coefficient matrix or a quantized transform coefficient matrix, and a transform coefficient matrix or quantized transform coefficient generated by the transform means Scanning means for scanning a matrix to generate one-dimensional data of transform coefficients or quantized transform coefficients, and transform coefficients generated by the scan means or first order of quantized transform coefficients An encoding means for encoding data; a decoding means for restoring one-dimensional data of the transform coefficient or the quantized transform coefficient; and a transform coefficient or quantized transform relating to a residual signal decoded by the decoding means Reverse scanning the coefficient one-dimensional data to generate a transform coefficient matrix or a quantized transform coefficient matrix; and inverse transforming the transform coefficient matrix generated by the reverse scanning means, or , Inverse transforming means for restoring a reproduction residual signal by performing both inverse quantization and inverse transform or only inverse quantization on the quantized transform coefficient matrix generated by the inverse scanning means, and the prediction signal Image restoration means for restoring the target pixel signal by adding the prediction signal generated by the generation means and the reproduction residual signal restored by the inverse transformation means; and the restored An image encoding program for functioning as an image storage unit for storing an elephant pixel signal, wherein the prediction signal generation unit refers to a previously reproduced pixel stored in the image storage unit and a different prediction signal A plurality of texture signals are generated by a generation method, and a texture signal having the highest correlation with the target pixel signal in accordance with a predetermined evaluation rule is selected as a prediction signal of the target region from among the generated plurality of texture signals, The encoding means encodes mode information indicating a prediction signal generation method for generating the selected texture signal, and the scanning means outputs the selected texture signal from a plurality of prepared scanning methods. Using the scanning method corresponding to the predicted signal generation method to be generated, one-dimensional data of the transform coefficient or the quantized transform coefficient is generated, and the reverse scanning is performed. The means reversely scans the restored transform coefficient or the one-dimensional data of the quantized transform coefficient using a reverse scan method corresponding to the mode information from among the plurality of prepared reverse scan methods. , Generating a transform coefficient matrix of a reproduction residual signal or a quantized transform coefficient matrix.

  An image decoding program according to the present invention uses a computer provided in an image decoding apparatus to extract encoded data of a residual signal related to a target region to be processed from compressed data, and to obtain one of transform coefficients related to the residual signal. Decoding means for restoring dimension data; prediction signal generating means for generating a prediction signal for the target pixel signal included in the target region; and transform coefficients or quantized transform coefficients for the residual signal decoded by the decoding means Reverse scanning the one-dimensional data and generating a transform coefficient matrix or a quantized transform coefficient matrix, and performing inverse transform on the transform coefficient matrix generated by the reverse scanning means, or The inverse transform that restores the reproduction residual signal by performing both inverse quantization and inverse transform or only inverse quantization on the quantized transform coefficient matrix generated by the inverse scanning means. And an image restoration means for restoring the pixel signal of the target area by adding the intra prediction signal generated by the prediction signal generation means and the reproduction residual signal restored by the inverse transformation means, An image decoding program for functioning as an image storage unit for storing the restored target pixel signal, wherein the decoding unit extracts mode information indicating a prediction signal generation method from compressed data. The prediction signal generation means restores the prediction signal generation means from among a plurality of prediction signal generation methods for generating different texture signals with reference to the already reproduced pixels stored in the image storage means. A prediction signal generation method corresponding to the generated mode information is identified, a prediction signal of the target region is generated by the identified prediction signal generation method, and the reverse scanning is performed The stage uses a reverse scanning method corresponding to the mode information restored by the decoding means from among a plurality of prepared reverse scanning methods, and uses a transform coefficient matrix or quantized transform of the reproduction residual signal. A coefficient matrix is generated.

  According to the present invention, a plurality of texture signals are generated by different prediction signal generation methods, and among the generated plurality of texture signals, a texture signal having the highest correlation with the target pixel signal in accordance with a predetermined evaluation rule is determined as a target region. The code amount of the prediction difference signal can be suppressed by switching the coding method of the transform coefficient in accordance with the prediction signal generation method for selecting the prediction signal and generating the selected texture signal.

It is a block diagram which shows the image coding apparatus which concerns on embodiment of this invention. It is a block diagram which shows the prediction signal production | generation method determination device in a screen used for the image coding apparatus which concerns on embodiment of this invention. It is a block diagram which shows the scanner used for the image coding apparatus which concerns on embodiment of this invention. It is a flowchart which shows the image coding process performed with the image coding apparatus which concerns on embodiment of this invention. It is a block diagram which shows the image decoding apparatus which concerns on embodiment of this invention. It is a block diagram which shows the reverse scanner used for the image coding apparatus and image decoding apparatus which concern on embodiment of this invention. It is a flowchart which shows the image decoding process performed with the image decoding apparatus which concerns on embodiment of this invention. It is a schematic diagram which shows the 1st prediction signal generation method of the target area | region which concerns on the prediction signal generation method of this invention, and the corresponding 1st conversion factor scanning method. It is a schematic diagram which shows the 2nd prediction signal production | generation method of the target area | region which concerns on the prediction signal production | generation method of this invention, and the corresponding 2nd conversion factor scanning method. It is a schematic diagram which shows the 3rd in-screen predicted signal generation method of the object area | region which concerns on the predicted signal generation method of this invention, and a corresponding 3rd conversion factor scanning method. It is a schematic diagram which shows the 4th prediction signal production | generation method of the target area | region which concerns on the prediction signal production | generation method of this invention, and the corresponding 4th conversion coefficient scanning method. It is a schematic diagram which shows the 5th prediction signal production | generation method of the target area | region which concerns on the prediction signal production | generation method of this invention, and the corresponding 5th conversion factor scanning method. It is a schematic diagram which shows the 6th prediction signal production | generation method of the object area | region which concerns on the prediction signal production | generation method of this invention, and the corresponding 6th conversion coefficient scanning method. It is a schematic diagram which shows the 7th prediction signal production | generation method of the object area | region which concerns on the prediction signal production | generation method of this invention, and the corresponding 7th conversion factor scanning method. It is a schematic diagram which shows the 8th prediction signal production | generation method of the object area | region which concerns on the prediction signal production | generation method of this invention, and the corresponding 8th conversion factor scanning method. It is a schematic diagram which shows the 9th prediction signal generation method of the target area | region which concerns on the prediction signal generation method of this invention, and the corresponding 9th conversion coefficient scanning method. It is a figure which shows the hardware constitutions of the computer for performing the program recorded on the recording medium. It is a perspective view of a computer for executing a program stored in a recording medium. It is a schematic diagram for demonstrating the prediction signal generation method in a screen used for a prior art. It is a schematic diagram for demonstrating the prediction signal generation method in a screen used for a prior art. It is a schematic diagram for demonstrating the change of the encoding efficiency by the difference in the scanning method of a conversion coefficient. It is a block diagram of the image coding program which concerns on embodiment of this invention. It is a block diagram of the image decoding program which concerns on embodiment of this invention.

  Hereinafter, embodiments of the present invention will be described with reference to FIGS.

  FIG. 1 is a block diagram showing an image encoding device 100 according to an embodiment of the present invention. The image coding apparatus 100 includes an input terminal 101, a block divider 102, an inter-screen prediction signal generation method determiner 103, an inter-screen prediction signal generator 104, an intra-screen prediction signal generation method determiner 105, and an intra-screen prediction signal generator. 106, changeover switch 107, subtractor 108, converter 109, quantizer 110, scanner 129, inverse scanner 130, inverse quantizer 111, inverse transformer 112, adder 113, frame memory (image storage means) 114, an entropy encoder 115, and an output terminal 116.

  The operation of the image coding apparatus 100 configured as described above will be described below. A moving image signal composed of a plurality of images is input to the input terminal 101. An image to be encoded is divided into a plurality of regions by the block divider 102. In the embodiment according to the present invention, the block is divided into 8 × 8 pixels, but may be divided into other block sizes or shapes. Next, a prediction signal is generated for a region to be encoded (hereinafter referred to as “target block”). In the embodiment according to the present invention, two kinds of prediction methods are used. That is, inter-screen prediction and intra-screen prediction.

  In inter-screen prediction, a reproduction image that has been encoded in the past and then restored is used as a reference image, and motion information that gives a prediction signal with the smallest error for the target block is obtained from this reference image. This process is called motion detection. Further, according to circumstances, the target block may be subdivided, and the inter-screen prediction method may be determined for the subdivided small area. In this case, the most efficient division method and the respective motion information are determined from the various division methods for the entire target block. In the embodiment according to the present invention, this process is performed by the inter-screen prediction signal generation method determiner 103, the target block is connected via the line L103, and the reference image is connected to the L121 (frame memory 114, although not shown). Not entered). As the reference image, a plurality of images encoded and restored in the past may be used as the reference image. For details, see MPEG-2, 4, H. The same as the H.264 method. The motion information and the small area dividing method determined in this way are sent to the inter-screen prediction signal generator 104 via the line L122. These pieces of information are sent to the entropy encoder 115 via the line L124, encoded, and sent from the output terminal 116. The inter-screen prediction signal generator 104 acquires a reference signal from the frame memory 114 (via the line L121) based on the subregion dividing method and the motion information corresponding to each subregion, and generates a prediction signal. The inter-screen prediction signal generated in this way is sent to the next processing block via the terminal 107a.

  In the intra prediction, an intra prediction signal is generated by a predetermined method using the already reproduced pixel values spatially adjacent to the target block. The method for generating the intra-screen prediction signal is determined by the intra-screen prediction signal generation method determiner 105. The processing of the intra-screen prediction signal generation method determiner 105 will be described later, but here, a method of interpolation using a plurality of pixels immediately adjacent to the target block is used. Information regarding the prediction method determined in this way (also referred to as mode information) is sent to the intra-screen prediction signal generator 106 via the line L123. The mode information is sent to the entropy encoder 115 via the line L125, encoded, and sent from the output terminal 116. The intra-screen prediction signal generator 106 is a pixel signal that has already been reproduced and is immediately adjacent to the target block in the same screen from the frame memory 114 (via the line L120) based on the mode information instructing the intra-screen prediction signal generation method And a prediction signal is generated according to the mode information. The inter-screen prediction signal generated in this way is sent to the next processing block via the terminal 107b.

  With respect to the inter-screen prediction signal and the intra-screen prediction signal obtained as described above, the one having the smallest error is selected by the changeover switch 107 and sent to the subtractor 108. The selection information is included in mode information that indicates a prediction signal generation method. However, since there is no past image for the first image, all target blocks are processed by intra prediction, and the switch 107 is always connected to the terminal 107b when processing the image. Similarly, when it is necessary to introduce an image of intra prediction on a screen immediately after a scene cut or periodically, only intra prediction may be performed for the entire screen. The intra-screen prediction signal generation method described below can also be applied to encoding / decoding of still images.

  The subtractor 108 subtracts the prediction signal (via line L127) from the signal of the target block (via line L102) to generate a residual signal. This residual signal is subjected to discrete cosine transform by a converter 109. Here, the transform coefficient matrix (transform block) is sequentially input to the quantizer 110 and quantized (generation of a quantized transform coefficient matrix). The quantized transform coefficient in the quantized transform coefficient matrix is scanned by the scanner 129 into one-dimensional data of the quantized transform coefficient. The scanning method in the scanner 129 will be described later, but different methods are used depending on the selected prediction method. Finally, the transform coefficient quantized by the entropy encoder 115 is encoded, information about the prediction signal generation method (mode information and motion vector indicating the prediction signal generation method), and a quantization parameter indicating the quantization method (Not shown) and output from the output terminal 116.

  In order to perform intra prediction or inter prediction for the next target block, the compressed signal of the target block is reversely processed and then restored. That is, the inverse scanner 130 performs reverse scanning from the one-dimensional data of the quantized transform coefficients into a quantized transform coefficient matrix (quantized transform block). The quantized transform coefficient is inversely quantized by the inverse quantizer 111 and then inversely discrete cosine transformed by the inverse transformer 112 to restore the residual signal. The residual signal restored by the adder 113 and the prediction signal sent from the line L128 are added, and the signal of the target block is reproduced and stored in the frame memory 114.

  Next, the intra prediction process will be described. FIG. 2 is a block diagram showing a first intra-screen prediction signal generation method determiner used in the image encoding device 100 according to the embodiment of the present invention. As shown in FIG. 2, the intra-screen prediction signal generation method determiner 105 includes a prediction method determiner 201 and a prediction signal generator 202. In the present embodiment, a prediction signal is generated by nine methods using a plurality of already reproduced pixels that are in direct contact with the target block in the prediction signal generator 202. FIG. 8 (a), FIG. 9 (a), FIG. 10 (a), FIG. 11 (a), FIG. 12 (a), FIG. 13 (a), and FIG. This will be described with reference to FIGS. 15A and 16A. FIG. 8A shows a first intra-screen prediction signal generation method for the target region. In FIG. 8A, each cell represents a pixel, and a pixel group 801 of pixels A to M is an already-reproduced pixel group, and a prediction signal generator in the intra-screen prediction signal generation method determiner 105 via a line L120. 202 is input. A pixel group 802 of pixels a to p is a pixel included in the target block. In the present embodiment, a target block composed of 4 × 4 pixels is used, but a block composed of other numbers of pixels may be used as the target block. In that case, a reproduction pixel group adjacent to the target block may be defined in accordance with the size of the target block. The first intra-screen prediction signal generation method for the target block uses the pixels A to D that are part of the already-reproduced pixel group 801, and generates a texture signal (prediction signal) by extending each pixel downward. A specific calculation is shown in Expression 806. In 806, pred (a, e, i, m) = A means that the predicted value of each pixel of a, e, i, m in the target block 802 is A. The second intra-screen prediction signal generation method for the target region is shown in FIG. 9A. In this case, I, J, K, and L, which are a part of the already-reproduced pixel group 901, are stretched to the right. Generated as a prediction signal. A specific calculation is shown in Expression 906. The third intra-screen prediction signal generation method for the target region is shown in FIG. 10A. In this case, the average value of the pixels A to D and I to L included in the already-reproduced pixel group 1001 is used as the prediction signal of the target block 1002. Used as A specific calculation is shown in Expression 1006. The remaining intra-screen prediction signal generation methods for the target block are shown in FIGS. 11 (a), 12 (a), 13 (a), 14 (a), 15 (a), and 16 (a), respectively. It is shown. Details are omitted. In addition to the nine texture signals, more or fewer texture signals may be used. Further, the texture signal may be generated by a method different from the above, for example, a method such as spline extrapolation.

  Returning to FIG. 2, the nine texture signals thus created are sent to the prediction method determiner 201 via the line L202. The signal of the target block is further input to the prediction method determiner 201 via the line L103. Therefore, the difference between the nine texture signals sent via the line L202 and the target block is obtained, and the texture signal that gives the smallest difference value is used as the prediction signal of the target block. Mode information for instructing a prediction signal generation method for generating the optimum texture signal is sent to the intra-screen prediction signal generator 106 in FIG. 1 via a line L123. The intra-screen prediction signal generator 106 generates a prediction signal of the target block using the already reproduced pixel group by the above-described method according to this mode information.

  Next, a conversion coefficient scanning method according to the present invention will be described. In entropy coding of the difference signal, the quantized transform coefficient in the block is coded by dividing it into the number of 0 coefficients that are continuous with a significant coefficient having a magnitude of 1 or more. Therefore, if the significant coefficient and the 0 coefficient are mixed in the one-dimensional data of the quantized transform coefficient input to the entropy encoder, the encoding efficiency is lowered. FIG. 21 shows an example of one-dimensional data of quantized transform coefficients. Each 枡 value represents a quantized transform coefficient. In FIG. 21A and FIG. 21B, the configuration of the quantization coefficient values is the same, but the order of encoding is different. In the example of FIG. 21B, the encoding process can be aborted by sending information indicating that the seventh and subsequent quantized transform coefficients are 0 values. On the other hand, in FIG. 21A, it is necessary to send all of the 16th quantized coefficients, and it is necessary to send the number of consecutive 0s four times. bad. Thus, the scanning order of transform coefficients affects the coding efficiency.

  As shown in FIG. 3, the scanner 129 includes scanners 301 to 309, a switch 323, and a scanning method selector 322 prepared for nine intra-screen prediction signal generation methods. Here, a case where intra prediction is selected with the switch 107 in FIG. 1 will be described. Although description when selecting inter-screen prediction is omitted, a scanner for inter-screen prediction is used (for example, zigzag scanning described in FIG. 10B is adopted as the scanning order).

  In the scanner 129 of FIG. 3, mode information for instructing the intra-screen prediction signal generation method selected first via the L 125 is sent to the scanning method selector 322. The scanning method selector 322 switches the switch 323 according to the mode information. Thereby, each transform coefficient of the quantized transform coefficient matrix (quantized transform block) is input to any of the scanners 301 to 309 via the L106 in the raster scan order. Note that the input order of the transform coefficients may be another order as long as it is determined in advance. Each of the scanners 301 to 309 rearranges the plurality of input transform coefficients into one-dimensional data by a scanning method described below, and outputs the result to the entropy encoder 115 of FIG. 1 via L130. Further, if the mode information includes the inter-screen prediction and the determination result of the inter-screen prediction, the inter-screen prediction can be handled. A scanner for prediction between screens may be prepared separately.

  Here, FIG. 8B, FIG. 9B, FIG. 10B, FIG. 11B, FIG. 12B, FIG. 13B, FIG. 14B, and FIG. 15B. Referring to FIG. 16B, a scanning method of the conversion coefficient in each of the scanners 301 to 309 will be described. In the present embodiment, nine intra-screen prediction signal generation methods are defined. In these prediction methods, except for FIG. 10 (a), as shown in FIGS. 8 (a) to 16 (a), a part of 13 already reproduced pixels (A to M) is placed in a specific direction. The prediction signal in the target region is generated by stretching. For this reason, the generated prediction signal has a strong correlation in a specific direction, and this characteristic also affects the transform coefficient of the block obtained by discrete cosine transform of the difference signal. FIG. 8B shows a transform coefficient scanning method when the first in-screen prediction signal generation method shown in FIG. 8A is selected. In this case, the difference signal has a strong correlation in the vertical direction, and a transform coefficient having a large value is likely to appear in the horizontal direction where the correlation is weak, that is, in the direction of 90 degrees with respect to the prediction direction. Therefore, the transform coefficients in the quantized transform coefficient matrix (quantized transform block) 803 are scanned from the left to the right and from the top to the bottom as indicated by the arrows, and the one-dimensional data in which the coefficient 0 to the coefficient 15 are arranged is scanned. Generate. A block 903 in FIG. 9B shows a transform coefficient scanning method when the second intra prediction signal generation method shown in FIG. 9A is selected. As in the case of FIG. 8B, since there is a high possibility that a transform coefficient having a large value in the direction of 90 degrees in the prediction direction, that is, in the vertical direction, the transform coefficient in the quantization transform block 903 is indicated by an arrow. Are scanned from top to bottom and from left to right to generate one-dimensional data in which coefficients 0 to 15 are arranged. In FIG. 10B, the average value of the eight reproduced pixels is the prediction signal for each pixel in the target block. In this case, since there is no bias in the prediction direction, scanning is performed zigzag from the low frequency component to the high frequency component of the conversion coefficient. Similarly, FIG. 11B, FIG. 12B, FIG. 13B, FIG. 14B, FIG. 15B, and FIG. The conversion coefficient is scanned in the direction of 90 degrees with respect to the prediction direction in the same manner as in FIG.

  Next, the conversion coefficient reverse scanning method of the present invention will be described. Here, a case where intra prediction is selected with the switch 107 of FIG. 1 will be described. As shown in FIG. 6, the reverse scanner 130 (also shown in FIG. 1) includes reverse scanners 601 to 609, a reverse scanning method selector 622, and switches prepared for nine intra-screen prediction signal generation methods. 623. The reverse scanning method selector 622 has the same function as the scanning method selector of FIG. That is, when the mode information instructing the intra-screen prediction signal generation method selected via L125 is sent to the reverse scanning method selector 622, the reverse scanning method selector 622 switches the switch 623 according to the mode information. As a result, the transform coefficient sequence of the one-dimensional data related to the quantized transform coefficients is input to any of the reverse scanners 601 to 609 via L130. Each of the reverse scanners 601 to 609 is shown in FIG. 8B, FIG. 9B, FIG. 10B, FIG. 11B, FIG. 12B, FIG. b) In accordance with the order of the arrows shown in FIGS. 15B and 16B, the transform coefficients of the one-dimensional data related to the input quantized transform coefficients are displayed on the quantized transform coefficient matrix (quantized transform block). Map. The quantized transform block is output to the inverse quantizer 111 of FIG. 1 via L131. In addition, although the explanation when selecting the inter-screen prediction is omitted (not shown), if the mode information indicating the prediction method includes the determination result of the inter-screen prediction and the inter-screen prediction, the inter-screen prediction is also included. Yes. A reverse scanner for inter-screen prediction is used (as a scanning order, for example, a reverse scanner for inter-screen prediction using zigzag scanning described in FIG. 10B may be prepared separately).

  FIG. 4 is a flowchart showing the intra-frame encoding processing in the image encoding device 100 according to the embodiment of the present invention. In step 402, a plurality of already-reproduced pixels (for example, 801 in FIG. 8A) immediately adjacent to the target block are acquired from the frame memory, and prediction signals of the target block are generated for nine intra-screen prediction signal generation methods. To do. Then, one prediction signal generation method that minimizes the difference value with the signal of the target block is selected. In step 403, a difference signal between the signal of the target block and the prediction signal is generated, and in step 404, the difference signal is subjected to DCT to generate a transform coefficient matrix. In step 405, each transform coefficient of the transform coefficient matrix is quantized. Subsequently, in step 406, a scanning method corresponding to the prediction signal generation method selected in step 402 is selected from FIG. 8B to FIG. 16B, and the transform coefficient in the quantized transform coefficient matrix is selected as one-dimensional data. Sort by. The quantized transform coefficient and the mode information indicating the prediction signal generation method selected in step 402 are entropy encoded (step 407). Finally, in step 408, the encoded mode information and quantized transform coefficients are output.

  Next, an image decoding apparatus and a decoding method that implement the reverse scanning method of the present invention will be described. FIG. 5 shows a block diagram of an image decoding apparatus 500 according to the embodiment of the present invention. The image decoding apparatus 500 includes an input terminal 500A, an entropy decoder 501, an inverse scanner 513, an inverse quantizer 502, an inverse transformer 503, an adder 504, an intra-screen prediction signal generator 505, a frame memory (image storage means). 506, an intra-screen prediction signal generation method acquisition unit 507, an inter-screen prediction signal generator 508, a changeover switch 509, and an output terminal 512 are provided.

  The operation of the image decoding apparatus 500 configured as described above will be described below. The compressed data compressed and encoded by the method described above is input from the input terminal 500A. The compressed data includes encoded data of a prediction residual signal of a target block obtained by dividing an image into a plurality of blocks and mode information indicating a prediction signal generation method. In the entropy decoder 501, from the compressed data, encoded data of the residual signal of the target block, mode information indicating the prediction signal generation method (in the case of intra-frame prediction, mode information indicating the intra-screen prediction signal generation method) In the case of quantization parameter and inter-frame prediction, motion information (motion vector) is further extracted and decoded. The encoded data of the residual signal of the target block, that is, the one-dimensional data of the quantized transform coefficient, is reverse-scanned to the quantized transform coefficient matrix by the inverse scanner 513 and output to the inverse quantizer 502 via the line L513. Is done. Note that the reverse scanning method in the reverse scanner 513 will be described later, but different methods are used depending on the selected prediction method. The quantized transform coefficients in the quantized transform matrix are dequantized by the inverse quantizer 502 based on the quantization parameter (via the line L511c), and output to the inverse transformer 503 via the line L503. The inverse transformer 503 performs inverse discrete cosine transform on the transform coefficient matrix obtained by inverse quantization. Next, switching information between intra prediction and inter prediction included in mode information instructing a prediction signal generation method is sent to the changeover switch 509 via the line L511b. When the inter-screen prediction is performed, the changeover switch 509 is brought down to the terminal 510a. The motion vector for the target block to be restored is sent to the inter-screen prediction signal generator 508 via the line L511a, and the frame memory 506 is accessed to generate a prediction signal specified by the motion vector. When performing intra prediction, the changeover switch 509 is brought down to the terminal 510b. In this case, the mode information indicating the intra-screen prediction signal generation method included in the mode information indicating the prediction signal generation method is sent to the intra-screen prediction signal generator 505 via the line L511d. The intra-screen prediction signal generator 505 acquires the already reproduced pixel group that is in direct contact with the target block from the frame memory 506, and generates the intra-screen prediction signal generation method corresponding to the mode information (FIGS. 8A to 16A). A prediction signal is generated. The prediction signals generated from the inter-screen prediction signal generator 508 and the intra-screen prediction signal generator 505 are sent to the adder 504 via the line L510, added to the restored residual signal, and reproduce the target block signal. The data is output via the line L512 and stored in the frame memory 506 at the same time.

  Next, the reverse scanner 513 according to the present invention will be described. The function of the reverse scanner 513 is the same as that of the reverse scanner 130 of FIG. First, a case where intra prediction is selected with the switch 509 in FIG. 5 will be described. As shown in FIG. 6, the reverse scanner 513 includes reverse scanners 601 to 609, a reverse scanning method selector 622, and a switch 623 prepared for nine intra-screen prediction signal generation methods. When mode information indicating the intra-screen prediction signal generation method selected via L511d is sent to the reverse scanning method selector 622, the reverse scanning method selector 622 switches the switch 623 according to the mode information. As a result, the transform coefficient sequence of one-dimensional data related to the quantized transform coefficients output from the entropy decoder 501 in FIG. 5 is input to any of the reverse scanners 601 to 609 via L503. Each of the reverse scanners 601 to 609 is shown in FIG. 8B, FIG. 9B, FIG. 10B, FIG. 11B, FIG. 12B, FIG. b) In accordance with the order of the arrows shown in FIG. 15B and FIG. 16B, the transform coefficients of the one-dimensional data of the input quantized transform coefficients are put on the quantized transform coefficient matrix (quantized transform block). Map. The quantized transform block is output to the inverse quantizer 502 of FIG. 5 via L513. The case where the inter-screen prediction is selected by the switch 509 in FIG. 5 can be dealt with by including the switching information between the intra-screen prediction and the inter-screen prediction in the mode information indicating the prediction signal generation method. In accordance with the order of the arrows shown in FIG. 10B, the input one-dimensional data transformation coefficient of the quantized transformation coefficient is mapped on the transformation block (not shown). At this time, a reverse scanner for inter-screen prediction may be separately prepared for the reverse scanner 513.

  FIG. 7 is a flowchart showing an intra-screen decoding process in the image decoding apparatus 500 according to the embodiment of the present invention. In step 702, data required for decoding the target block is input from the compressed data. In step 703, the compressed data of the target block acquired in step 702 is entropy decoded, and the mode information indicating the quantized transform coefficient and the prediction signal generation method is restored. In step 704, the one-dimensional data of the quantized transform coefficients is reverse-scanned by the inverse scanning method corresponding to the restored mode information, and restored to the quantized transform coefficient matrix. In step 705, the restored quantized transform coefficient is inversely quantized to restore the transform coefficient matrix. In step 705, the restored transform coefficient matrix is subjected to inverse DCT processing in step 706, and the difference signal of the target block is restored. Finally, in step 707, the restored difference signal and the intra prediction signal generated according to the restored mode information are added to generate a reproduction signal of the target block.

  Next, an image encoding program for operating a computer as an image encoding device according to the present invention and an image decoding program for operating a computer as an image decoding device according to the present invention will be described. FIG. 22 is a diagram showing the configuration of the image encoding program P100 together with the recording medium 10, and FIG. 23 is a diagram showing the configuration of the image decoding program P500 together with the recording medium 10. As described above, the image encoding program P100 and the image decoding program P500 are provided by being stored in the recording medium 10. Examples of the recording medium 10 include a recording medium such as a flexible disk, CD-ROM, DVD, or ROM, or a semiconductor memory.

  As shown in FIG. 22, an image encoding program P100 includes an area dividing module P101 corresponding to an area dividing unit included in an image encoding apparatus according to the present invention, a prediction signal generating module P102 corresponding to a prediction signal generating unit, Residual signal generation module P103 corresponding to residual signal generation means, conversion module P104 corresponding to conversion means, scanning module P105 corresponding to scanning means, encoding module P106 corresponding to encoding means, and decoding means A decoding module P107 corresponding to the above, a reverse scanning module P108 corresponding to the reverse scanning means, an inverse conversion module P109 corresponding to the reverse conversion means, an image restoration module P110 corresponding to the image restoration means, and an image storage means. And an image storage module P111.

  As shown in FIG. 23, the image decoding program P500 includes a decoding module P501 corresponding to decoding means included in the image decoding apparatus according to the present invention, a prediction signal generation module P502 corresponding to prediction signal generation means, and reverse scanning. A reverse scanning module P503 corresponding to the means, an inverse conversion module P504 corresponding to the reverse conversion means, an image restoration module P505 corresponding to the image restoration means, and an image saving module P506 corresponding to the image saving means.

  FIG. 17 is a diagram illustrating a hardware configuration of a computer for executing a program recorded in a recording medium, and FIG. 18 is a perspective view of the computer for executing a program stored in the recording medium. Examples of the computer include a DVD player, a set-top box, a mobile phone, and the like that have a CPU and perform processing and control by software.

  As shown in FIG. 17, the computer 30 includes a reading device 12 such as a flexible disk drive device, a CD-ROM drive device, and a DVD drive device, a working memory (RAM) 14 in which an operating system is resident, and a recording medium 10. A memory 16 for storing programs stored therein, a display device 18 such as a display, a mouse 20 and a keyboard 22 as input devices, a communication device 24 for transmitting and receiving data and the like, and a CPU 26 for controlling execution of the programs. And. When the recording medium 10 is inserted into the reading device 12, the computer 30 can access the image encoding program P100 and the image decoding program P500 stored in the recording medium 10 from the reading device 12, and the image encoding program P100. The image decoding program P500 can operate as an image encoding device / image decoding device according to the present invention.

  As shown in FIG. 18, the image encoding program or the image decoding program may be provided via a network as a computer data signal 40 superimposed on a carrier wave. In this case, the computer 30 can store the image encoding program or image decoding program received by the communication device 24 in the memory 16 and execute the image encoding program or image decoding program.

  As described above, according to the transform coefficient scanning method and the reverse scanning method of the present invention, in the differential encoding of a prediction signal having a strong correlation in a specific direction, the transform coefficient scanning method and the Since the reverse scanning method can be switched, the encoding efficiency is improved.

  In the above, the transform coefficients are scanned in the direction of 90 degrees with respect to the prediction direction. However, the scanning method determines the order in which transform coefficients in the quantized transform coefficient matrix are encoded. Any scanning method may be used as long as the efficiency is improved. Therefore, the optimum angle differs depending on the prediction method and the image characteristics, and is not limited. Further, in FIGS. 8B to 16B, the quantization transform coefficient matrix is scanned on a single line, but the optimum scanning method differs depending on the prediction method and the image characteristics. In some cases, one-dimensional data of conversion coefficients may be scanned with several combinations. Also, when a specific quantized transform coefficient is set to 0, it is not always necessary to scan all the transform coefficients in the quantized transform coefficient. The reverse scanning method also reverses the scanning method and restores the one-dimensional data of the quantized transform coefficients into the quantized transform coefficient matrix, and is not limited to the method of the above-described embodiment, like the scan method.

  In the above embodiment, a scanning method and a reverse scanning method for a plurality of prediction signal generation methods are predetermined, but encoding may be performed in sequence units, frame units, or block units.

  In sequence units and frame units, a flag indicating whether to use a predetermined reverse scanning method or to send data instructing the reverse scanning method is sent. At this time, a flag may be sent for each prediction signal generation method, or a flag (including all prediction signal generation methods) may be sent for each of a plurality of prediction signal generation methods. As a data format for instructing the reverse scanning method, there is a case where the correspondence between each transform coefficient of the one-dimensional data and the coordinates in the quantized transform coefficient matrix after the reverse scan is arranged in the decoding order of the quantized transform coefficients. The format is not limited to this. For example, the index of each coefficient in the quantized transform coefficient matrix may be determined in advance, and the index may be encoded instead of the coordinates, or each coefficient in the quantized transform coefficient matrix is arranged in a predetermined order, The correspondence with the one-dimensional data may be encoded in order. Further, instead of sending data instructing the reverse scanning method, identification numbers are given to a plurality of predetermined reverse scanning methods, and the identification information of the reverse scanning method corresponding to each prediction signal generation method is selectively selected. A method of sending can also be considered.

  In the block unit, the reverse scanning method may be encoded as described above, but the reverse scanning method corresponding to the prediction signal generation method is used, or the default reverse scanning method (for example, FIG. 10B) is used. Or identification information indicating either of them may be sent. The identification information may include data indicating the reverse scanning method sent in sequence units or frame units, and the identification information may include the option of transmitting data indicating the reverse scanning method. In the block indicating that the identification information transmits data indicating the reverse scanning method, the data indicating the reverse scanning method is transmitted following the identification information. Note that the identification information may be configured by any two or three combinations of these four types.

  In this way, when information on the reverse scanning method is sent in units of frames or blocks, the encoding side needs to determine a reverse scanning method corresponding to each prediction signal generation method based on the characteristics of the texture signal. As the procedure, first, a candidate scanning method is actually tried and a method with a small code amount is selected. Then, there is a method of notifying a decoder of a reverse scanning method that is a pair of appropriate scanning methods. On the decoding side, a reverse scanning method corresponding to each prediction signal generation method is selected from mode information indicating the decoded prediction signal generation method and information on the reverse scanning method, and the restored one-dimensional data of the quantized transform coefficients is selected. Generate a quantized transform coefficient matrix.

  The implementation of the present invention is not limited to the number of prediction signal generation methods, and when the generated prediction signal has specific characteristics, the effect of switching between the scanning method and the reverse scanning method can be expected.

  In this embodiment, the quantization transformation coefficient matrix (quantization transformation block) is 4 × 4 pixels as in the case of the prediction signal generation, but a block composed of other numbers of pixels (for example, 8 × 8 pixels) is used as the quantization transformation block. Also good. Further, the size of the transform block may be different from the target block for generating the prediction signal. When the size of the transform block is smaller than the size of the target block, the influence of the difference in the prediction direction on the transform coefficient becomes smaller as it is spatially separated from the already-reproduced pixels used for prediction. As described above, the scanning method may be set to use a general zigzag scanning for the quantized transform block having a small influence on the prediction direction.

  In the above description, discrete cosine transform (DCT) is used as a differential signal conversion method, but the conversion method is not limited to DCT. H. Integer conversion as defined in H.264 may be used, another conversion method such as KLT may be used, or frequency conversion may not be performed. At this time, the applied scanning method of the quantized transform coefficient matrix and the inverse scanning method of the one-dimensional data of the quantized transform coefficients differ depending on the transform method. For example, when frequency conversion is not performed, the prediction difference value tends to increase as the pixel is spatially separated from the already-reproduced pixel. Therefore, the arrows shown in FIGS. 8B to 16B are reversed. It is considered effective to scan in the direction (scanning method) and map the one-dimensional data of the quantized transform coefficients from the reverse direction of the arrow to the quantized transform coefficient matrix (reverse scan method). Further, the present invention is not limited to the entropy encoding method. All methods such as arithmetic coding and variable-length coding are applicable. The present invention is also applicable when no quantizer is present. In this case, the one-dimensional data of the quantized transform matrix and the quantized transform coefficient are replaced with the one-dimensional data of the transform matrix and the transform coefficient, respectively. Further, when frequency conversion is not performed, the difference matrix (difference block) and the one-dimensional data of the difference signal are replaced.

  In FIG. 1, the scanner is provided separately from the converter and the quantizer. However, the scanner may be included in the converter or the quantizer, or the quantizer may be included in the converter. However, the scanning method of the present invention is applicable. Similarly, the inverse scanner of FIGS. 1 and 5 may be configured to be included in an inverse transformer or an inverse quantizer, or may be configured to include an inverse transformer in the inverse quantizer. A reverse scanning method is applicable. In FIGS. 1 and 5, the reproduced signal is input to the intra prediction signal generator from the frame memory, but the adder 113 (FIG. 1) or the adder is used without going through the frame memory. The reverse scanning method of the present invention can be applied even in the configuration performed from 504 (FIG. 5).

  DESCRIPTION OF SYMBOLS 100 ... Image coding apparatus, 101 ... Input terminal, 102 ... Block divider, 103 ... Inter-screen prediction signal generation method determiner, 104 ... Inter-screen prediction signal generator, 105 ... In-screen prediction signal generation method determiner, 106 ... Intra-screen prediction signal generator 107 ... Changeover switch 108 ... Subtractor 109 ... Converter 110 ... Quantizer 111 ... Inverse quantizer 112 ... Inverse transformer 113 ... Adder 114 ... Frame Memory: 115 ... Entropy encoder, 116 ... Output terminal, 129 ... Scanner, 130 ... Inverse scanner, 500 ... Image decoding device, 500A ... Input terminal, 501 ... Entropy decoder, 502 ... Inverse quantizer, 503 ... Inverter, 504 ... Adder, 505 ... In-screen prediction signal generator, 506 ... Frame memory, 508 ... Inter-screen prediction signal generator, 509 ... Switching switch Chi, 512 ... output terminal, 513 ... inverse scanner.

Claims (12)

Area dividing means for dividing the input image into a plurality of areas;
Prediction signal generation means for generating a prediction signal for a target pixel signal included in a target region that is a processing target of the plurality of regions;
Residual signal generating means for generating a residual signal between the prediction signal and the target pixel signal;
Transforming means for performing transforming and / or quantization on the residual signal generated by the residual signal generating means and generating a transform coefficient matrix or a quantized transform coefficient matrix;
Scanning means for scanning the transform coefficient matrix or quantized transform coefficient matrix generated by the transform means to generate one-dimensional data of the transform coefficient or quantized transform coefficient;
Encoding means for encoding the transform coefficient generated by the scanning means or the one-dimensional data of the quantized transform coefficient;
Decoding means for restoring one-dimensional data of the transform coefficient or the quantized transform coefficient;
Reverse scanning means for reverse-scanning the one-dimensional data of the transform coefficient or quantized transform coefficient related to the residual signal decoded by the decoding means, and generating a transform coefficient matrix or a quantized transform coefficient matrix;
Perform inverse transform on the transform coefficient matrix generated by the inverse scanning means, or perform both inverse quantization and inverse transform on the quantized transform coefficient matrix generated by the inverse scanning means, or only inverse quantization By performing inverse transformation means for restoring the reproduction residual signal,
Image restoration means for restoring the target pixel signal by adding the prediction signal generated by the prediction signal generation means and the reproduction residual signal restored by the inverse transformation means;
Image storage means for storing the restored target pixel signal;
With
The prediction signal generation unit generates a plurality of texture signals by using different prediction signal generation methods with reference to the already reproduced pixels accumulated in the image storage unit, and among the generated plurality of texture signals, a predetermined texture signal is generated. A prediction method determining means for selecting a texture signal having the highest correlation with the target pixel signal in light of the evaluation rule as a prediction signal of the target region;
The encoding means encodes in-screen prediction mode information indicating a prediction signal generation method for generating a texture signal selected by the prediction method determination means,
The scanning means uses a prescribed scanning method when the conversion block size for performing the conversion is smaller than a reference, and from among a plurality of scanning methods prepared for the conversion block size otherwise. Generating a one-dimensional data of the transform coefficient or the quantized transform coefficient by using a scanning method determined based on a predicted signal generating method for generating a texture signal selected by the prediction method determining means;
The inverse scanning means uses a prescribed inverse scanning method when the inverse transformation block size when performing the inverse transformation is smaller than the reference, and a plurality of inverses prepared when the transformation block size is otherwise. Using the inverse scanning method determined based on the intra prediction mode information among the scanning methods, the restored transformation coefficient or the one-dimensional data of the quantized transformation coefficient is inversely scanned, and the reproduction residual signal Generate a transform coefficient matrix or a quantized transform coefficient matrix of
An image encoding apparatus characterized by that.   The prediction signal generation unit generates one or more texture signals by an intra-screen prediction signal generation method that refers to already reproduced pixels adjacent to the target region,
  The plurality of prepared scanning methods include a scanning method suitable for a texture signal generated by the intra-screen prediction signal generation method,
  The prepared plurality of reverse scanning methods include a reverse scanning method suitable for a texture signal generated by the intra prediction signal generation method.
  The image coding apparatus according to claim 1.   The image coding apparatus according to claim 1, wherein the correspondence between the scanning method and the prediction signal generation method and the correspondence between the reverse scanning method and the prediction signal generation method are determined in advance.   The encoding means encodes identification information for identifying a reverse scanning method corresponding to the selected prediction signal generation method from the prepared reverse scanning methods. The image encoding device described in 1. Decoding means for extracting from the compressed data the in-screen prediction mode information related to the target region to be processed and the one-dimensional data of the quantized transform coefficient, and restoring them;
Prediction signal generating means for generating an intra-screen prediction signal by referring to already reproduced pixels in a method specified by the intra-screen prediction mode information from among a plurality of intra-screen prediction signal generation methods;
The one-dimensional data of the quantized transform coefficient is reverse-scanned to generate a quantized transform coefficient matrix, and the reproduction residual signal is restored by performing inverse quantization and inverse transform on the generated transform coefficient matrix. And image restoration means for restoring the pixel signal of the target area by adding the intra-screen prediction signal and the reproduction residual signal,
With
The image restoration means uses a prescribed inverse scanning method when the inverse transform block size for performing the inverse transform is smaller than a certain reference, and a plurality of inverses prepared when the transform block size is otherwise. Using the inverse scanning method determined based on the in-screen prediction mode information among the scanning methods, the quantized transform coefficient matrix is generated by reverse scanning the one-dimensional data of the quantized transform coefficients. To
An image decoding apparatus characterized by that.   The prediction signal generation unit generates one or more texture signals by an intra-screen prediction signal generation method that refers to already reproduced pixels adjacent to the target region,
  The prepared plurality of reverse scanning methods include a reverse scanning method suitable for a texture signal generated by the intra prediction signal generation method.
  The image decoding apparatus according to claim 5.   6. The image decoding apparatus according to claim 5, wherein the correspondence between the reverse scanning method and the intra prediction signal generation method is determined in advance.   The decoding means extracts and restores identification information for selecting a reverse scanning method corresponding to the specified prediction signal generation method from the prepared multiple reverse scanning methods,
  The image restoration means generates the quantized transform coefficient matrix using a reverse scanning method determined from the intra prediction mode information and the identification information from among the plurality of prepared reverse scanning methods. ,
  The image decoding apparatus according to claim 5. An image encoding method executed by an image encoding device,
A region dividing step for dividing the input image into a plurality of regions;
A prediction signal generating step for generating a prediction signal for a target pixel signal included in a target region to be processed among the plurality of regions;
A residual signal generating step for generating a residual signal between the prediction signal and the target pixel signal;
A transformation step of performing transformation and / or quantization on the residual signal generated by the residual signal generation step to generate a transformation coefficient matrix or a quantized transformation coefficient matrix;
Scanning the transform coefficient matrix or quantized transform coefficient matrix generated by the transform step to generate one-dimensional data of the transform coefficient or quantized transform coefficient; and
An encoding step for encoding the transform coefficient generated by the scanning step or the one-dimensional data of the quantized transform coefficient;
Decoding to restore one-dimensional data of the transform coefficient or the quantized transform coefficient;
A reverse scanning step of generating a conversion coefficient matrix or a quantized conversion coefficient matrix by reverse scanning the one-dimensional data of the conversion coefficient or the quantized conversion coefficient relating to the residual signal decoded by the decoding step;
Perform inverse transform on the transform coefficient matrix generated by the inverse scan step, or perform both inverse quantization and inverse transform on the quantized transform coefficient matrix generated by the inverse scan step, or only inverse quantization By performing an inverse transformation step to restore the playback residual signal,
An image restoration step of restoring the target pixel signal by adding the prediction signal generated by the prediction signal generation step and the reproduction residual signal restored by the inverse transformation step;
An image storage step for accumulating the restored target pixel signal;
With
In the prediction signal generation step, a plurality of texture signals are generated by different prediction signal generation methods with reference to the already reproduced pixels accumulated in the image storage step, and a predetermined texture signal among the generated plurality of texture signals is generated. A prediction method determining step of selecting a texture signal having the highest correlation with the target pixel signal in accordance with an evaluation rule as a target region prediction signal;
In the encoding step, in-screen prediction mode information indicating a prediction signal generation method for generating the texture signal selected in the prediction method determination step is encoded,
In the scanning step, a prescribed scanning method is used when the conversion block size for carrying out the conversion is smaller than a reference, and the conversion block size is selected from a plurality of scanning methods prepared otherwise. Generating a one-dimensional data of the transform coefficient or the quantized transform coefficient using a scanning method determined based on a predicted signal generation method for generating a texture signal selected by the prediction method determination step;
In the inverse scanning step, when the inverse transform block size when performing the inverse transform is smaller than the reference, a prescribed inverse scanning method is used, and when the transform block size is other than that, a plurality of inverses prepared. Using the inverse scanning method determined based on the intra prediction mode information among the scanning methods, the restored transformation coefficient or the one-dimensional data of the quantized transformation coefficient is inversely scanned, and the reproduction residual signal Generate a transform coefficient matrix or a quantized transform coefficient matrix of
An image encoding method characterized by the above. An image decoding method executed by an image decoding device,
A decoding step of extracting and restoring in-screen prediction mode information regarding the target region to be processed and one-dimensional data of the quantized transform coefficient from the compressed data;
A prediction signal generating step of generating an intra-screen prediction signal with reference to a previously reproduced pixel by a method specified by the intra-screen prediction mode information from among a plurality of intra-screen prediction signal generation methods;
The one-dimensional data of the quantized transform coefficient is reverse-scanned to generate a quantized transform coefficient matrix, and the reproduction residual signal is restored by performing inverse quantization and inverse transform on the generated transform coefficient matrix. And an image restoration step of restoring the pixel signal of the target region by adding the intra prediction signal and the reproduction residual signal;
With
In the image restoration step, when the inverse transform block size for performing the inverse transform is smaller than a reference, a prescribed inverse scanning method is used, and when the transform block size is other than that, a plurality of inverses prepared. Using the inverse scanning method determined based on the in-screen prediction mode information among the scanning methods, the quantized transform coefficient matrix is generated by reverse scanning the one-dimensional data of the quantized transform coefficients. To
An image decoding method characterized by the above. Computer
Area dividing means for dividing the input image into a plurality of areas;
Prediction signal generation means for generating a prediction signal for a target pixel signal included in a target region that is a processing target of the plurality of regions;
Residual signal generating means for generating a residual signal between the prediction signal and the target pixel signal;
Transforming means for performing transforming and / or quantization on the residual signal generated by the residual signal generating means and generating a transform coefficient matrix or a quantized transform coefficient matrix;
Scanning means for scanning the transform coefficient matrix or quantized transform coefficient matrix generated by the transform means to generate one-dimensional data of the transform coefficient or quantized transform coefficient;
Encoding means for encoding the transform coefficient generated by the scanning means or the one-dimensional data of the quantized transform coefficient;
Decoding means for restoring one-dimensional data of the transform coefficient or the quantized transform coefficient;
Reverse scanning means for reverse-scanning the one-dimensional data of the transform coefficient or quantized transform coefficient related to the residual signal decoded by the decoding means, and generating a transform coefficient matrix or a quantized transform coefficient matrix;
Perform inverse transform on the transform coefficient matrix generated by the inverse scanning means, or perform both inverse quantization and inverse transform on the quantized transform coefficient matrix generated by the inverse scanning means, or only inverse quantization By performing inverse transformation means for restoring the reproduction residual signal,
Image restoration means for restoring the target pixel signal by adding the prediction signal generated by the prediction signal generation means and the reproduction residual signal restored by the inverse transformation means;
Image storage means for storing the restored target pixel signal;
An image encoding program for functioning as
The prediction signal generation unit generates a plurality of texture signals by using different prediction signal generation methods with reference to the already reproduced pixels accumulated in the image storage unit, and among the generated plurality of texture signals, a predetermined texture signal is generated. A prediction method determining means for selecting a texture signal having the highest correlation with the target pixel signal in light of the evaluation rule as a prediction signal of the target region;
The encoding means encodes in-screen prediction mode information indicating a prediction signal generation method for generating a texture signal selected by the prediction method determination means,
The scanning means uses a prescribed scanning method when the conversion block size for performing the conversion is smaller than a reference, and from among a plurality of scanning methods prepared for the conversion block size otherwise. Generating a one-dimensional data of the transform coefficient or the quantized transform coefficient by using a scanning method determined based on a predicted signal generating method for generating a texture signal selected by the prediction method determining means;
The inverse scanning means uses a prescribed inverse scanning method when the inverse transformation block size when performing the inverse transformation is smaller than the reference, and a plurality of inverses prepared when the transformation block size is otherwise. Using the inverse scanning method determined based on the intra prediction mode information among the scanning methods, the restored transformation coefficient or the one-dimensional data of the quantized transformation coefficient is inversely scanned, and the reproduction residual signal Generate a transform coefficient matrix or a quantized transform coefficient matrix of
An image encoding program characterized by the above. Computer
Decoding means for extracting from the compressed data the in-screen prediction mode information related to the target region to be processed and the one-dimensional data of the quantized transform coefficient, and restoring them;
Prediction signal generating means for generating an intra-screen prediction signal by referring to already reproduced pixels in a method specified by the intra-screen prediction mode information from among a plurality of intra-screen prediction signal generation methods;
The one-dimensional data of the quantized transform coefficient is reverse-scanned to generate a quantized transform coefficient matrix, and the reproduction residual signal is restored by performing inverse quantization and inverse transform on the generated transform coefficient matrix. And image restoration means for restoring the pixel signal of the target region by adding the intra prediction signal and the reproduction residual signal,
An image decoding program for functioning as
The image restoration means uses a prescribed inverse scanning method when the inverse transform block size for performing the inverse transform is smaller than a certain reference, and a plurality of inverses prepared when the transform block size is otherwise. Using the inverse scanning method determined based on the in-screen prediction mode information among the scanning methods, the quantized transform coefficient matrix is generated by reverse scanning the one-dimensional data of the quantized transform coefficients. To
An image decoding program characterized by the above.

Images