JP2012029213A - Dynamic image encoding device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To temporally homogenize image quality of decoded images in a dynamic image encoding device to which lossy compression of a locally decoded image is applied.SOLUTION: A dynamic image encoding device comprises: an encoding processing unit for encoding an original image of each frame of a dynamic image for each macroblock; a locally decoded image generation unit for generating a locally decoded image from compressed data obtained in the encoding process; a block compression unit for performing lossy compression on the locally decoded image for each macroblock; a block decompression unit for decompressing the compressed data to generate a restored locally decoded image; a differential image generation unit for generating a differential image between the restored locally decoded image and the locally decoded image; a statistic extraction unit for extracting a statistic indicating a statistical property of a distribution of errors appearing in the differential image; a statistic tallying unit for cumulatively adding the extracted statistics for each area; and an area determination unit for determining areas to apply intra-image encoding in encoding processing of the next frame of the current frame based on the cumulative addition result.

Description

  The present invention relates to a moving image encoding apparatus that encodes a moving image by combining intra prediction and inter prediction.

  In the video encoding device, a prediction image that gives a smaller prediction error is selected from the prediction image obtained by the intra-screen prediction process and the prediction image obtained by the inter-screen prediction, and is used for generation of encoded stream data. .

  In the inter-screen prediction, for example, a local decoded image obtained by decoding image encoded data of the (n-1) th frame is used to detect a motion vector for the image of the nth frame. Then, the moving image code of the nth frame encoded using inter-screen prediction is decoded using the reference image obtained by decoding the encoded data of the (n−1) th frame in the moving image decoding apparatus. Is done. As described above, the inter-picture prediction encoding / decoding process is based on the premise that the local decoded image used in the encoding process matches the reference image used in the decoding process.

  In the conventional moving image encoding apparatus, a local decoded image is generated in parallel with the encoding process of each frame, stored in an external memory such as SDRAM, and used for the encoding process of the next frame. For example, when detecting a motion vector for inter-screen prediction, a desired region of a local decoded image is read from the external memory and used.

  By the way, with the widespread use of high-definition broadcasting, the size of image data handled by a moving image encoding device is increasing. The data size of the local decoded image described above is equivalent to the image data size to be encoded and is much larger than the encoded data. For this reason, the amount of data transferred from the external memory during the motion vector detection process also increases, leading to an increase in power consumption.

  As a method for suppressing the amount of data read from the external memory during the motion vector detection process, a method of compressing the local decoded image and storing it in the external memory has been proposed.

  For example, a method has been proposed in which a reduced image and a difference image between the enlarged image of the reduced image and the local decoded image are generated from the local decoded image, and the reduced image and the difference image are stored in an external memory (patent). Reference 1). In such a technique using lossless compression, the original local decoded image is restored by combining the difference image with the enlarged image of the reduced image read from the external memory.

  A method is also proposed in which DCT conversion processing is performed on a local decoded image to generate compressed data, the compressed data is stored in an external memory instead of the local decoded image, and used for inter-frame prediction processing of the next frame. (See Patent Document 2). In such a technique using irreversible compression, a high compression rate can be obtained, so that the effect of reducing the amount of data transferred from the external memory at the time of motion vector detection processing is high.

JP 2007-036738 A Japanese Patent Laid-Open No. 10-79941

  In the technique of reversibly compressing a local decoded image, the original local decoded image can be restored from the compressed image when performing inter-screen prediction processing for the next frame. On the other hand, the compression ratio varies greatly depending on the characteristics of the local decoded image to be subjected to lossless compression, and the compression ratio is generally not very high. Further, depending on the characteristics of the local decoded image, there is a possibility that the data size after the lossless compression is larger than the original local decoded image.

  On the other hand, in the method using lossy compression, a high compression rate can be guaranteed, but an error occurs between the restored local decoded image obtained from the compressed data and the original local decoded image. Such an error propagates to encoded data generated by inter-screen prediction using the restored local decoded image. It is also propagated to a restored image obtained by decoding the encoded data. Further, since the restored image becomes a reference image for decoding the next encoded data, the above-described errors are accumulated.

  Such an accumulated error is eliminated by periodically inserting an I picture in which all macroblocks included in one frame are encoded by intra prediction processing. For example, in a typical moving picture coding apparatus, an I picture is inserted every 30 frames, so that the cumulative error is eliminated at this rate. However, the accumulated error may be very large in the frame immediately before the I picture. That is, immediately after the restored image whose image quality has deteriorated due to the accumulated error, the accumulated error is eliminated by inserting the I picture, and the image quality of the restored image is suddenly improved. Therefore, in a configuration that relies on the insertion of an I picture to eliminate the accumulated error, there is a possibility that the image quality periodically repeats deterioration and improvement. Such temporal fluctuations in image quality give the user an unnatural impression.

  It is an object of the present disclosure to provide a moving image encoding apparatus capable of temporally homogenizing the image quality of a decoded image in a moving image encoding apparatus to which irreversible compression of a local decoded image is applied. And

  The object described above can be achieved by a moving picture encoding apparatus disclosed below.

  A moving image encoding apparatus according to one aspect includes an encoding processing unit that compresses and encodes each macroblock included in an original image of each frame of a moving image by applying intra-screen encoding or inter-screen encoding. For each macroblock included in the original image, a local decoded image generating unit that generates a local decoded image from compressed data obtained in the encoding process by the encoding processing unit, and a local decoded image corresponding to each macroblock are irreversibly A block compression unit that generates block compressed data by compression, a block decompression unit that decompresses block compressed data corresponding to each macroblock and generates a restored local decoded image corresponding to each macroblock, and a macroblock Difference image showing error between corresponding restored local decoded image and local decoded image A difference image generation unit to be generated, a statistic extraction unit for extracting a statistic indicating a statistical property of an error distribution appearing in a difference image corresponding to each macroblock, and a statistic extracted by the statistic extraction unit Based on the cumulative error up to each frame indicated by the cumulative addition result by the statistical totaling unit and a statistical totaling unit that cumulatively adds each of a plurality of regions obtained by dividing the frame. And a region discriminating unit that discriminates a region to which intra-screen coding is applied from among a plurality of regions.

  In a moving image encoding apparatus to which irreversible compression of a local decoded image is applied, the image quality of a restored image can be made uniform in time.

It is a figure which shows one Embodiment of a moving image encoder. It is a figure explaining the reference range at the time of decoding the code data of the inter-screen coding mode. It is a figure explaining totaling of statistics. It is a flowchart showing the statistics totaling operation. It is a figure which shows one Embodiment of an area | region discrimination | determination part. It is a flowchart showing area | region discrimination | determination operation | movement. It is a flowchart (the 1) showing encoding operation | movement. It is a flowchart (the 2) showing encoding operation | movement. It is a figure which shows another embodiment of an area | region discrimination | determination part. It is a flowchart showing operation | movement of an area | region discrimination | determination part. It is a figure which shows another embodiment of a moving image encoder. It is a figure which shows another embodiment of a moving image encoder.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.
(One embodiment)
FIG. 1 shows an embodiment of a moving image encoding apparatus.

  The encoding core unit 100 illustrated in FIG. 1 includes an intra-screen prediction unit 101, an inter-screen prediction unit 102, an encoding mode determination unit 103, a DCT quantum unit 104, an entropy encoding unit 105, and a local decoding (LD). And an image generation unit 111. The encoding core unit 100 reads the original image stored in the original image holding unit of the external memory, for example, in units of macroblocks of 16 pixels × 16 pixels, and performs encoding processing.

  The encoding mode determination unit 103 is applied to the macroblock based on the prediction residuals obtained by the intra-screen prediction unit 101 and the inter-screen prediction unit 102 according to the input of each macroblock. To decide. The DCT quantization unit 104 receives a prediction residual corresponding to the coding mode, performs DCT transform processing and quantization processing on the prediction residual, and generates compressed data. The compressed data is encoded by the entropy encoding unit 105, and the generated encoded data is output.

  The decompression processing unit 106 of the local decoded image generation unit 111 performs inverse quantization processing and inverse DCT transform processing on the quantized compressed data received from the DCT compression unit 104 to restore prediction residual data. The restored prediction residual data is combined with the predicted image passed from the encoding mode determination unit 103 by the restored image generation unit 107. The local decoded image generated in this manner is transferred to the intra-screen prediction unit 101 and used for the prediction process of the next macroblock. In addition, this local decoded image is passed to the block compression unit 112 through the deblocking process by the deblocking processing unit 108. The deblocking process is a process for removing block noise that appears near the boundary between the macroblock of the generated local decoded image and the surrounding macroblocks.

  The block compression unit 112 compresses a local decoded image input in units of macroblocks by an irreversible compression process in which a target compression rate can be set, and generates block compressed data. As an example of lossy compression processing, a prediction residual is calculated from a prediction block by DPCM (Differential Pulse Code Modulation), and quantization processing and variable length are performed on the prediction residual so as to obtain a predetermined target compression rate. There is a method of performing an encoding process.

  The block decompression unit 113 performs inverse transformation processing corresponding to the processing of the block compression unit 112 on the block compressed data generated by the block compression unit 112. As the decompression process, for example, the block decompression unit 113 can perform variable length decoding on the block compressed data, perform an inverse quantization process on the decoding result, and further perform an inverse DPCM process. By such decompression processing, a corresponding macroblock of the restored local decoded image is generated from the block compressed data described above.

  The difference image generation unit 114 generates a difference image between the local decoded image generated by the deblocking processing unit 108 and the restored local decoded image generated by the block decompression unit 113 in units of macroblocks.

  This difference image is an error generated newly at the position of the macroblock to be encoded between the local decoded image of the nth frame to be encoded and the restored local decoded image generated from this local decoded image. Show.

  Hereinafter, as described above, based on the difference image obtained for each macroblock, among the plurality of areas obtained by dividing one frame, the accumulated local decoded image of the area including the macroblock is accumulated. Consider how to evaluate the error. In addition, as an area serving as a unit for evaluating the accumulated error, for example, a full HD size (1920 pixels × 1088 pixels) is divided into 15 equal parts and 17 equal parts in the horizontal direction and the vertical direction, respectively. Regions can be used. In this case, 32 macroblocks each having 16 pixels × 16 pixels are included in one area, and the number of areas included in one frame is 255.

Here, the difference between the pixel values of each pixel included in the local decoded image and the restored local decoded image is most often a numerical value “0”, and the appearance frequency decreases as the difference value increases. Therefore, the above-described probability distribution of error values can be approximated by a normal distribution. As a characteristic of normal distribution, it is known that the sum of independent normal distributions also becomes normal distribution. For example, the two probability density functions Ga (diff, μ _a , σ _a ² ) and the probability density function Ga (diff, μ _b , σ _b ² ), which are normal distributions, and the sum are the difference diff and the difference average value μ _a And μ _b , and difference variances σ _a ² and σ _b ² , are expressed as in equation (1).

  That is, assuming that the difference value of each region appears according to the probability distribution function of the normal distribution, the shape of the local probability density function of each region is determined by the error sample mean and error sample variance of that region . In other words, by accumulating the average value and variance of the difference image of each macroblock, which is an error sample, over several frames, it is possible to estimate the state of error accumulation in each region. Note that the average of the difference values corresponding to each pixel included in the difference image tends to converge to the numerical value “0”. Therefore, assuming that the sample error average is 0, the local probability density distribution of each region can be estimated based on the error variance. Here, the error variance is proportional to the sum of squares of the difference value diff, as shown in Expression (2), where the sampling error average μ is 0.

  From these facts, the process of calculating the error sampling error variance can be replaced with the process of calculating the sum of squared errors that is almost directly proportional to the error variance. That is, by accumulating the sum of squared differences of each macroblock, which is a sample of errors in the region j of each frame, it is possible to estimate the magnitude of the error accumulated so far in the local decoded image generated in that frame. it can. That is, by calculating the sum of squares of the pixel values for each macro block of the difference image, a statistic indicating the statistical nature of the error distribution can be extracted. Therefore, in the example illustrated in FIG. 1, the statistic extraction unit 115 can be realized by using a difference square sum calculation unit that calculates a square sum of pixel values for each macroblock of the difference image. In this difference square sum calculation unit, for example, the difference sum of squares is calculated independently for the luminance component Y and the color difference components Cb and Cr, which are the component of the pixel value of each pixel included in the macroblock of the difference image. Further, a value obtained by adding these values is calculated by the difference square sum calculation unit, and this value is provided to the processing of the statistic totaling unit 116 as a statistic extracted corresponding to the macroblock.

  Here, in the intra-frame encoding process, the local decoded image generated corresponding to the encoding process for each macroblock is referred to in the intra-frame prediction process for the next macroblock. As described above, the magnitude of the error accumulated in the local decoded image referred to in the intra-picture encoding process is estimated with high accuracy based on the accumulated value of the sum of squared differences having a high correlation with the variance of the accumulated error. Can do. In the intra-screen coding mode, the accumulated error in the referenced local decoded image is directly propagated to the decoding side via the coded data. Therefore, for example, the cumulative error sum of the difference images of each macroblock generated corresponding to the encoding process of each frame is cumulatively added for each region, so that the magnitude of the cumulative error can be estimated with high accuracy. Is possible.

  FIG. 2 is a diagram illustrating a reference range when decoding code data in the inter-screen coding mode. In the example shown in FIG. 2, when decoding the encoded data corresponding to the macro block MB (i) of the (n + 1) th frame, the range referred to in the local decoded image of the nth frame is indicated by a solid line rectangle. Yes. In the local decoded image of the Nth frame, the macroblock located at the position corresponding to the macroblock MB (i) to be decoded is indicated by a broken-line rectangle.

  As shown in FIG. 2, when the encoded data generated in the inter-picture encoding mode is decoded, the accumulated error appearing in the macroblock MB (i) of the (n + 1) th frame is caused by the following two factors: Overlay. One of the factors to be superimposed is an error caused by its own lossy compression. Another factor is the accumulated error existing at the reference destination in the local decoded image of the nth frame.

  Here, the past picture region referred to based on the motion vector has a high probability of being a neighborhood region of the decoding target region. For example, in the example shown in FIG. 2, the actual reference area and the area corresponding to the area to be decoded overlap with each other in many parts. Therefore, there is a high possibility that the statistical property of the error in the reference region is almost the same as the statistical property of the region corresponding to the position to be decoded in the nth frame. Therefore, the accumulated error of each region can be approximated by the accumulation of statistics corresponding to the region at the same position in the past picture. If this approximation is applied, the cumulative error variance of the decoded image generated from the code data encoded in the inter-screen encoding mode can be simplified as in the above-described cumulative error variance in the intra-screen encoding mode. Can be estimated. That is, it is possible to estimate the magnitude of the accumulated error by accumulating the difference square sum of the difference images of the macroblocks generated corresponding to the encoding process of each frame for each region.

  In the example of the moving image encoding apparatus illustrated in FIG. 1, the statistic extraction unit 115 calculates the sum of squares of the difference values corresponding to each pixel of the difference image generated by the difference image generation unit 114. Then, the difference sum of squares is totaled by the statistical amount counting unit 116 as a statistical amount related to the error of each macroblock included in the local decoded image corresponding to each frame.

  In the example illustrated in FIG. 1, the statistic totaling unit 116 includes an adding unit 122, an update processing unit 123, and a cumulative statistic holding unit 124. The statistic of each macroblock extracted by the statistic extraction unit 115 is added to the accumulated statistic of the corresponding region by the addition unit 122 in accordance with an instruction from the update processing unit 123. The addition result is held in the accumulated statistic holding unit 124 corresponding to the above-described area, and is used for the subsequent accumulation adding process.

  FIG. 3 is a diagram for explaining the aggregation of statistics. FIG. 4 is a flowchart showing statistics extraction and tabulation operations.

In the example shown in FIG. 3A, one frame is divided into N areas. Each area is given an area number from No. 1 to No. N in order from the upper left of the frame. In this example, m areas are arranged in the scanning line direction. In the example shown in FIG. 3B, the above-described statistical total values S _{1 to} S _N are stored in the cumulative statistical quantity holding unit 124 corresponding to the area numbers 1 to _N. As the accumulated statistic holding unit 124, for example, if the total value of the statistic corresponding to each area is expressed by 4 bytes and the number of areas N = 255, a capacity of about 1 Kbyte may be prepared.

  Corresponding to the encoding process of the i-th macroblock included in the original image of the encoding target frame by the local decoded image generation unit 111 shown in FIG. 1, the corresponding macroblock MB (i) of the local decoded image Is generated. In response to this, the statistic extraction and counting operations shown in FIG. 4 are started.

  The macroblock MB (i) of the local decoded image is first irreversibly compressed by applying a desired compression rate by the block compression unit 112 (step 301). The block compression data obtained by the block compression unit 112 is expanded by the block expansion unit 113, thereby generating a macroblock MBr (i) of the restored local decoded image (step 302). Then, the difference image generation unit 114 generates a difference image D (i) from the macroblock MBr (i) of the restored local decoded image and the macroblock MB (i) of the local decoded image described above (step 303). ).

  In response to the input of the difference image D (i), the statistic extraction unit 115 calculates the square sum Rs (i) of the pixel values of each pixel of the difference image D (i) (step 304). This square sum Rs (i) is totalized by the statistic totaling unit 116 as follows.

  The position of each macro block in the frame is indicated by a block number given to the macro block. This block number is given in the scanning order from the upper left of the frame, for example. Therefore, the update processing unit 123, based on the block number i passed from the local decoded image generation unit 111, the region number indicating the region to which the macroblock MB (i) belongs among the regions illustrated in FIG. j can be identified (step 305).

  The statistical total value Sj held in the cumulative statistic holding unit 124 corresponding to the area number specified in step 305 is passed to the adding unit 122 via the update processing unit 123. Then, the adder 122 adds the square sum Rs (i), which is the statistic newly obtained in step 304, and the aggregate value Sj of the statistic (step 306). Next, the update processing unit 123 updates the aggregate value Sj of the statistic corresponding to the region j using this addition result (step 307). As a result, the statistical amount relating to the newly obtained error can be cumulatively added to the total value of the corresponding region.

  For example, in the example shown in FIG. 3A, the region number 2 is specified from the block number of the macroblock MB (i) by the process of step 305 described above. Then, as shown in FIG. 3B, the square sum Rs (i) is added as a newly obtained statistic to the aggregate value Sj of the statistic corresponding to the area number 2.

  The above-described processing from step 301 to step 307 is repeated until it is determined in step 308 that the aggregation processing has been completed for all macroblocks included in one frame. Such an aggregation process is performed in the course of the encoding process for each frame. As a result, in the encoding process of these frames, the accumulated amount of error caused by irreversible compression of the local decoded image is used as the total value stored in the accumulated statistics holding unit 124 corresponding to each area. Obtainable.

  Therefore, the area discriminating unit 117 shown in FIG. 1 forcibly encodes the in-screen code in order to eliminate the accumulated error, for example, based on the total value of errors accumulated in each area until the nth frame. The area to be converted can be determined.

  The block compressed data generated by the block compression unit 112 is held in the compressed local decoded image (LD image) holding unit 141 of the external memory 140 as a part of the compressed local decoded image. Then, during the encoding process for the original image of the next frame, the search area restoration unit 110 generates a portion corresponding to the search area of the restored local decoded image from the block compressed data. This restored local decoded image of the search area is provided to the process of the inter-screen prediction unit 102 via the search area holding unit 109.

  Next, a method for controlling the encoding process for the next picture based on the accumulated statistics aggregated for the past pictures as described above will be described.

  FIG. 5 shows an embodiment of the area determination unit. Note that among the components shown in FIG. 5, components equivalent to those shown in FIG. 1 are denoted by the same reference numerals, and description thereof is omitted.

  In the example illustrated in FIG. 5, the region determination unit 117 includes a region detection unit 125, a threshold adjustment unit 126, and a determination flag holding unit 127. The area detection unit 125 applies an intra-frame encoding process to which the aggregate value of the accumulated statistics held in the accumulated statistics holding unit 124 is equal to or greater than a predetermined threshold Thv among the N areas described above. Detected as a region. Then, the region detection unit 125 operates the determination flag of the determination flag holding unit 127 for the detected application target region. Further, the threshold adjustment unit 126 adjusts the value of the threshold Thv used for detection by the region detection unit 125 based on the number of application target regions indicated by the determination flag of the determination flag holding unit 127.

  FIG. 6 is a flowchart showing the region discrimination operation. Note that the processing from step 311 to step 318 shown in FIG. 6 is completed, for example, with the irreversible compression processing of the local decoded image in parallel with the encoding processing of each frame, and the accumulation of the accumulated statistics up to that frame is completed. Done every time.

  First, the value k of the number of application target areas is initialized by, for example, the threshold adjustment unit 126 (step 311). At the same time, the area number j indicating the area to be discriminated is also initialized to, for example, area number 1 indicating the upper left area of the frame.

  Next, the total value S (j) corresponding to the region number j is read from the cumulative statistic holding unit 124 by the region detection unit 125 (step 312). Then, the read total value S (j) is compared with the threshold Thv by the region detection unit 125 (step 313).

  When the total value S (j) is determined to be equal to or greater than the threshold value Thv (affirmative determination in step 313), the region detection unit 125 sets the determination flag corresponding to the region number j of the determination flag holding unit 127 as the region. A value indicating that it is an application target is set (step 314). For example, in step 314, the region detection unit 125 may set a value indicating “TRUE” to the determination flag corresponding to the region number j. In accordance with the operation of the determination flag, the threshold value adjustment unit 126 adds 1 to the number k of application target regions (step 315).

  On the other hand, when the total value S (j) is determined to be smaller than the threshold value Thv (negative determination in step 313), the region detection unit 125 sets the determination flag corresponding to the region number j of the determination flag holding unit 127 to A value indicating that the area is not applicable is set (step 317). For example, the area detection unit 125 may set a value indicating “FALSE” in the determination flag corresponding to the area number j in step 317.

  As described above, after the processing according to the determination result in step 313 is performed, the region detection unit 125 determines whether or not the processing for all the regions has been completed (step 316). In the case of negative determination in step 316, the area detection unit 125 increments the area number j and returns to step 312 to perform processing based on the total value S (j) corresponding to the new area number j.

  In this way, Steps 312 to 317 are repeated, and after the determination processing for all the regions included in one frame is completed, the threshold adjustment unit 126 determines the threshold Thv according to the value of the number k of application target regions. Is adjusted (step 318). For example, the threshold adjustment unit 126 uses the number k of application target regions, the total number N of regions, and a natural number md smaller than N as a threshold Thv using a coefficient α expressed by Equation (3). By multiplying, a new threshold value Thv can be calculated.

α = 1 + (k−md) / N (3)
The coefficient α represented by the expression (3) has a value larger than 1 when the number of application target areas k is larger than the natural number md, and the value of the threshold Thv increases accordingly. Adjusted. On the contrary, when the number of application target areas k is smaller than the natural number md, the value is smaller than 1, and the value of the threshold Thv is adjusted in a decreasing direction accordingly. Note that the value of the natural number md described above can be determined in advance in consideration of limitations such as the transfer rate of encoded data. For example, when md areas are forcibly intra-coded in each frame, the amount of code data per picture is determined to be an amount that can be transmitted at the transfer rate of the encoded data. Can do.

  Next, a method for reflecting the determination result obtained as described above in the encoding process of the next frame will be described.

  The coding mode determination unit 103 illustrated in FIG. 5 includes a mode determination control unit 131 and a discrimination flag reading unit 132. Further, the cumulative statistic adjustment unit 118 illustrated in FIG. 5 adjusts the value of the cumulative statistic held in the cumulative statistic holding unit 124 according to the encoding mode determined by the encoding mode determination unit 103. . In the example illustrated in FIG. 5, the cumulative statistic adjustment unit 118 includes a selection adjustment unit 128 and a collective adjustment unit 129.

  FIG. 7 is a flowchart (part 1) showing the encoding operation. FIG. 8 is a flowchart (part 2) showing the encoding operation. The flowcharts shown in FIGS. 7 and 8 show the encoding operation of the picture of each frame. That is, the encoding operation shown in FIGS. 7 and 8 is repeated for each picture.

  An original image (picture) to be encoded is read into the encoding core unit 100 in units of macroblocks. Each time the i-th macroblock MB (i) is read (step 321), the mode determination control unit 131 first determines whether or not the picture to be encoded is an I picture (step 322).

  If the picture to be encoded is not an I picture (negative determination in step 322), the discrimination unlove reading unit 132 determines from the discrimination flag holding unit 127 the discrimination flag F (corresponding to the region j including the macroblock MB (i). j) is read (step 323). When the read discrimination flag F (j) is “TRUE”, the mode determination control unit 131 determines that the macro block MB (i) to be encoded is included in the application target region of the intra-frame encoding process. Judgment (affirmative determination in step 324).

In this case, in response to the notification of the area number j from the mode determination control unit 131, the cumulative statistic held in correspondence with the cumulative statistic holding unit 124 by the selection adjusting unit 128 of the cumulative statistic adjusting unit 118 is stored. Initialization is performed (step 325). At this time, for example, the selection adjustment unit 128 can rewrite the value of the statistical total result S _j held in the cumulative statistic holding unit 124 corresponding to the region number j to the initial value 0.

  Thereafter, the intra prediction unit 101 calculates the intra prediction residual for the above-described macroblock MB (i) (step 326). Then, the DCT quantization unit 104 and the entropy encoding unit 105 perform processing for encoding the calculated intra prediction residual (step 327), and the generated encoded data is output (step 328). .

  Next, the mode determination control unit 131 determines whether or not the encoding of all macroblocks included in one frame is completed (step 329). If the determination is negative, the process returns to step 321 to return to the next macro. A block is read.

  In this way, when the read macroblock MB (i) is included in the region to which the intraframe encoding process is applied, regardless of the intraframe prediction residual and the interframe prediction residual, This macro block MB (i) is forcibly intra-coded. That is, all the macroblocks included in the region where the determination flag held in the determination flag holding unit 127 is “TRUE” are forcibly intra-coded. Then, by intra-coding these areas, the accumulation of errors due to irreversible compression of the local decoded image accompanying the encoding process so far is selectively eliminated for these areas.

Also, the selection adjustment unit 128 of the cumulative statistic adjustment unit 118 selectively returns the aggregation result S _j of the statistic corresponding to the region j to which the in-screen encoding process is applied to the initial value 0, as described above. The selective elimination of the accumulated error can be reflected in the contents of the accumulated statistics holding unit 124. Therefore, in the encoding process for the next picture, there is a high probability that an area other than the area in which the accumulated error has been eliminated in the encoding process for the previous picture will be the target area for the intra-picture encoding process.

  In addition, as described with reference to FIGS. 1 to 3, the accumulated statistic held in the accumulated statistic holding unit 124 reflects with high fidelity the error accumulated with the encoding process up to n frames. Has been. Therefore, the cumulative error can be effectively eliminated by sequentially applying the intra-screen coding processing from the region where the cumulative statistic is increased by controlling the coding mode as described above. In such a cumulative error elimination method, the cumulative error is eliminated for each area based on the accumulated error estimated for each area. That is, elimination of the accumulated error for N areas included in one frame is performed in a distributed manner during the encoding process of a plurality of frames. In the encoding process of each frame, the image quality of the image restored on the decoding side is improved in the area where the accumulated error is eliminated by applying the compulsory intra-screen encoding process. Accordingly, it is possible to suppress temporal fluctuations in the image quality of a moving image reproduced on the decoding side, compared to a case where the accumulated error is eliminated only by inserting an I picture. That is, in the moving image encoding apparatus having the above-described configuration, the moving image provided to the user on the decoding side while eliminating the accumulation of errors caused when irreversible compression of the local decoded image is applied. It is possible to achieve uniform image quality.

  In addition, by dynamically adjusting the threshold Thv used for detection of the application target region in the region determination unit 117, the number of regions determined as application targets in each frame can be leveled. Thereby, in order to eliminate the accumulated error, it is possible to make the code amount increased by applying the intra-screen coding almost constant. Therefore, the compression rate in the quantization process in the DCT quantization unit 104 and the encoding process in the entropy encoding unit 105 is also maintained at a substantially constant level.

  On the other hand, when the picture to be encoded is an I picture (affirmative determination in step 322), the mode determination control unit 131 notifies the collective adjustment unit 129 of the cumulative statistic adjustment unit 118 to that effect. In response to this, the collective adjustment unit 129 initializes the aggregate value of the statistics corresponding to all the regions held in the accumulated statistics holding unit 124 (step 330).

  Thereafter, in the same manner as in step 326, the intra prediction residual for the above-described macroblock MB (i) is calculated (step 331), and a process for encoding the calculated intra prediction residual is performed. (Step 332). Next, the process proceeds to step 328, and the generated encoded data is output.

  Note that, in the encoding process for pictures other than the I picture, when the read determination flag F (j) is “FALSE” (No determination in step 324), the mode determination control unit 131 sets the encoding target. A coding mode determination process is performed on the macroblock MB (i).

  In this case, in step 334 shown in FIG. 8, the search area restoration unit 110 performs a process of restoring the search area of the restored local decoded image from the compressed local decoded image. At this time, the search area restoration unit 110 reads block compressed data corresponding to the search area set for the macroblock MB (i) from the compressed local decoded image holding unit 141 shown in FIG. Then, decompression processing similar to that performed by the block decompression unit 113 shown in FIG. 1 is performed on these block compressed data to restore the search area described above. The restored search area is provided to the process of the inter-screen prediction unit 102 via the search area holding unit 109.

  Next, the prediction residual is calculated by the intra-screen prediction unit 101 and the inter-screen prediction unit 102 (step 335), and the encoding mode is determined by the mode determination control unit 131 based on these prediction residuals. (Step 336). If the intra prediction residual is smaller, the process proceeds to step 338 as an affirmative determination in step 337, and the macro block MB (i) to be encoded is subjected to the intra encoding process. On the other hand, when the inter-screen prediction residual is smaller, the process proceeds to step 339 as a negative determination in step 337, and the macro block MB (i) to be encoded is subjected to inter-screen encoding processing. Thereafter, in either case, the process proceeds to step 328 shown in FIG. 7, and the generated encoded data is output.

As shown in FIG. 8, in the process of encoding each macroblock included in a region other than the application target region, the cumulative statistic held in the cumulative statistic holding unit 124 is not initialized. Accordingly, the error before and after compression of the local decoded image generated along with the encoding is the statistical total value S indicating the accumulated error corresponding to the area to which the macro block MB (i) to be encoded belongs. is added to _j . Then, after the encoding process of all the macroblocks is completed, it is subjected to a process of determining an area to which the intra-picture encoding process is applied in the encoding of the next picture.

  By the way, when encoding each picture other than the I picture, the method of keeping the number of application target areas included in each frame substantially constant is to adjust the threshold Thv used for detection of the application target area as described above. It is not limited to the method of doing. In addition, in the encoding process of each picture, a statistic other than the above-described difference square sum can be extracted as a statistic indicating an error before and after compression of the local decoded image.

Next, a modified example of the application target area determination process will be described.
(Another embodiment)
FIG. 9 shows another embodiment of the region discriminating unit. Note that among the constituent elements shown in FIG. 9, the same constituent elements as those shown in FIG. 5 are denoted by the same reference numerals, and description thereof is omitted.

  In the example illustrated in FIG. 9, the region determination unit 117 includes a region selection unit 134 instead of the threshold adjustment unit 126. The region selection unit 134 selects a predetermined number of regions from the regions detected by the region detection unit 125, and operates the determination flag so as to indicate the selected region as an application target.

  FIG. 10 is a flowchart showing the region discrimination operation. Of the steps shown in FIG. 10, the same steps as those shown in FIG. 6 are denoted by the same reference numerals, and the description thereof is omitted.

  In the example shown in FIG. 10, whether the number k of application target areas is smaller than a predetermined number K before the processing of step 312 for reading the accumulated statistics held in the accumulated statistics holding unit 124 corresponding to each area. It is determined whether or not (step 319). Then, only when the number k of application target areas is smaller than the predetermined number K (Yes in step 319), the accumulated statistics are read (step 312). Then, the determination flag is operated based on the comparison between the accumulated statistic and the threshold value Thv (steps 313 to 317).

  For example, as long as the number k of application target areas is smaller than the predetermined number K described above, the accumulated statistics are read in the order of area numbers, and whenever the read accumulated statistics are larger than the threshold Thv, A region corresponding to this cumulative statistic is selected as an application target. Each time a new area is selected as an application target, the number k of application target areas is incremented.

  After the application target area is selected in this way and the number becomes equal to the predetermined number K described above, a negative determination is made in step 319. Then, the value of the determination flag corresponding to each subsequent area is set to “FALSE” indicating that it is not the application target area regardless of the value of each accumulated statistic.

  In such an area determination process, the number of areas detected as application target areas in each frame can be limited to a predetermined number K or less. Thereby, in order to eliminate the accumulated error, it is possible to suppress the code amount that is increased by applying the intra-screen coding to a certain amount or less. Note that the accumulated error up to the nth frame of the application target area selected in the above-described area determination process is to forcibly encode the application target area in the next n + 1 frame encoding process. Is eliminated. At this time, the accumulated statistics held in the accumulated statistics holding unit 124 corresponding to these application target areas are initialized. Then, based on the accumulated statistic held in the accumulated statistic holding unit 124 at the time when the encoding of the (n + 1) th frame is completed, an area to which forcible intra-frame coding processing is applied in the next n + 2 frame is provided. Selected. In other words, by combining the process of the cumulative statistic adjustment unit 118 and the area determination process described above, the application target area selected in the previous frame is excluded and the application target area in the next frame is selected. Can do. Therefore, as described above, even when the area determination process using up to K areas detected first as the application target area is adopted, the probability of being selected as the application target area is biased depending on the position in the screen. Absent. Note that it is also possible to select the application target region in descending order of the cumulative statistic value held in the cumulative statistic holding unit 124.

Next, various modified examples of the process of extracting a statistic indicating an error before and after compression of a local decoded image will be described.
(Another embodiment)
FIG. 11 shows another embodiment of the moving image encoding apparatus. Note that among the components shown in FIG. 11, components equivalent to those shown in FIGS. 1 and 5 are denoted by the same reference numerals and description thereof is omitted.

  The statistic extraction unit 115 illustrated in FIG. 11 includes a difference absolute value sum calculation unit 135 and a weight addition unit 136. The difference absolute value sum calculation unit 135 includes a luminance component Y (x, y) and a color difference component Cb included in the difference image D (x, y) (x, y = 1 to 16) generated by the difference image generation unit 114. For each of (x, y) and Cr (x, y), the absolute value sum is calculated. The weight addition unit 136 multiplies the difference absolute value sum obtained for each component by the difference absolute value ring calculation unit 135 by the weights wy, wb, wr, respectively, and adds the multiplication results. In the example shown in FIG. 11, the addition result by the weight addition unit 136 is passed to the statistic counting unit 116 as a statistic indicating an error before and after compression of the local decoded image corresponding to the macroblock to be encoded.

  Here, there is a high correlation between the difference square sum and the difference absolute value sum. Therefore, as described above, instead of the sum of squared differences, a sum of absolute differences can be calculated and used as a statistic indicating the distribution of errors before and after compression of the local decoded image. And the process which calculates a difference absolute value sum can be implement | achieved by a simple structure compared with the process which calculates a difference square sum. That is, in the statistic extraction unit 115, the statistic indicating the distribution of errors before and after compression of the local decoded image is realized with a small amount of hardware resources by calculating the sum of absolute differences instead of the sum of squared differences. be able to.

  Further, the weight adding unit 136 can add a weight corresponding to each component to the sum of absolute differences of each component. For example, a larger weight can be added to the sum of absolute differences of the luminance component Y than to the sum of absolute differences of the color difference components Cb and Cr. As described above, the accumulated error regarding the luminance component can be emphasized in the statistic, reflecting that the noise of the luminance component is easily perceived as degradation of image quality in human vision. As a result, the area where the accumulated statistics of the luminance component is large is preferentially determined as the application target area, and the accumulated error is eliminated. It can be resolved with priority.

In addition, it is possible to simplify the process of extracting a statistic indicating an error before and after compression of a local decoded image using the above-described human visual characteristics.
(Another embodiment)
FIG. 12 shows another embodiment of the moving image encoding apparatus. 12 that are the same as those shown in FIG. 1 are denoted by the same reference numerals and description thereof is omitted.

  In the coding core unit 100 illustrated in FIG. 12, the deblocking processing unit 108 is omitted, and the local decoded image generated in units of macroblocks by the restored image generation unit 107 is input to the block compression unit 112 as it is. .

  The block compression unit 112 performs the above-described compression processing on the luminance component and the color difference component included in each pixel value of the local decoded image, and generates block compression data for each component. These block compressed data are held in the compressed local decoded image holding unit 141 and used in the encoding process of the next frame.

  The luminance block decompression unit 138 illustrated in FIG. 12 selectively performs decompression processing on the block compressed data for the luminance component among the block compressed data generated by the block compressing unit 112. By this expansion processing, the luminance component of the restored local decoded image is generated in units of macro blocks. Then, the difference luminance image generation unit 139 generates a difference luminance image for each macroblock from the luminance component of the restored local decoded image and the luminance component of the local decoded image before compression.

  The difference absolute value sum calculation unit 136 illustrated in FIG. 12 calculates the sum of absolute values of the difference luminance values of each pixel for the difference luminance image generated by the difference luminance image generation unit 139. The absolute value sum of the difference luminance values calculated in this way corresponds to the case where the luminance component weight wy is set to 1 and the color difference component weights wb and wr are set to 0 in the statistic extraction unit 115 shown in FIG. To do. Then, the sum of absolute values of the difference luminance values is used as a statistic relating to an error before and after compression of the local decoded image, and is subjected to a counting process of the statistic counting unit 116.

  In this way, it is possible to evaluate the statistics related to the error before and after compression of the local decoded image by paying attention only to the luminance value. In this case, the amount of hardware required for block decompression processing, difference image generation processing, and difference absolute value sum calculation processing, compared to the case of calculating statistics regarding errors in consideration of all components included in pixel values Can be reduced.

  In addition, the hardware reduction effect due to the omission of the deblocking processing unit 108 is great. This is because the deblocking processing unit 108 holds macroblocks for one row up to immediately before a new macroblock in order to remove block noise for the local decoded image. In each embodiment described above, the local decoded image is irreversibly compressed. Therefore, the omission of the deblocking processing unit 108 that acts on the local decoded image before lossy compression has little influence on the image quality of the restored local decoded image restored after compression.

Regarding the above description, the following items are further disclosed.
(Supplementary Note 1) An encoding processing unit that compresses and encodes each macroblock included in an original image of each frame of a moving image by applying intra-screen encoding or inter-screen encoding;
For each macroblock included in the original image, a local decoded image generation unit that generates a local decoded image from compressed data obtained in the encoding process by the encoding processing unit,
A block compression unit for irreversibly compressing the local decoded image corresponding to each macroblock to generate block compressed data;
A block decompression unit that decompresses the block compressed data corresponding to each macroblock and generates a restored local decoded image corresponding to each macroblock;
A difference image generation unit for generating a difference image indicating an error between the restored local decoded image and the local decoded image corresponding to each macroblock;
A statistic extracting unit that extracts a statistic indicating a statistical property of the distribution of the error that appears in the difference image corresponding to each macroblock;
A statistic totaling unit that cumulatively adds the statistic extracted by the statistic extraction unit for each of a plurality of regions obtained by dividing the frame;
Based on the cumulative error up to each frame indicated by the cumulative addition result by the statistic counting unit, a region to which intra-screen coding is applied among the plurality of regions is determined in the coding process of the next frame of the frame. An area discriminating unit to perform,
A moving picture encoding apparatus comprising:
(Supplementary note 2) In the video encoding device according to supplementary note 1,
A cumulative statistic adjustment that adjusts a value indicating a cumulative error that has been aggregated for each of the areas by the statistic aggregating unit based on a discrimination result by the region discriminating unit and an encoding mode that is applied by the encoding processing unit. And
A moving picture encoding apparatus comprising:
(Supplementary Note 3) In the moving picture encoding apparatus according to Supplementary Note 2,
The cumulative statistic adjustment unit
A selection adjustment unit that selectively returns the accumulated statistic obtained by the statistic totaling unit to a predetermined initial value for the region that is the application target of the intra-screen code by the region discriminating unit;
When the encoding mode indicates that all macroblocks included in one frame are encoded in the screen, cumulative statistics corresponding to all areas obtained by the statistics totaling unit are collectively displayed. And a batch adjustment unit for returning to the initial value,
A moving picture encoding apparatus comprising:
(Supplementary Note 4) In the video encoding device according to Supplementary Note 1,
The region discriminating unit
Based on the counting result by the statistic counting unit, an area where the cumulative statistic is larger than a predetermined threshold Thv, an area detecting unit,
Based on the result of comparing the number of regions detected by the threshold detection unit and the natural number md smaller than the number N of regions included in the one frame, the threshold comparison unit compares the accumulated statistics. A threshold adjustment unit for changing the predetermined threshold Thv;
With
An area detected by the area detection unit is an application target of intra-frame encoding.
(Supplementary note 5) In the moving picture encoding apparatus according to supplementary note 1,
The region discriminating unit
An area detection unit that detects an area in which the accumulated statistic is greater than a predetermined threshold Thv based on a counting result by the statistic counting unit;
When the number of regions detected by the region detection unit is larger than another natural number K smaller than the number N of regions included in the one frame, the natural number K of the regions detected by the region detection unit An area selection unit that selects an area as an application target of the intra-screen encoding process;
A moving picture encoding apparatus comprising:
(Supplementary note 6) In the moving picture encoding apparatus according to supplementary note 1,
The moving picture encoding apparatus, wherein the statistic extraction unit includes a square sum calculation unit that calculates a square sum of pixel values of the difference image for each of a plurality of regions.
(Supplementary note 7) In the video encoding device according to supplementary note 1,
The moving image encoding apparatus, wherein the statistic extraction unit includes an absolute value sum calculation unit that calculates an absolute value sum of pixel values of the difference image for each of a plurality of regions.
(Supplementary note 8) In the moving picture encoding apparatus according to supplementary note 1,
The statistic extraction unit includes:
A component statistic calculation unit for obtaining a statistic for each of the luminance component and the color component included in each pixel value of the difference image;
A weight adding unit that adds weights to the luminance component and the color component respectively corresponding to the statistic, and adds the weighted statistics together;
A moving picture encoding apparatus comprising:

100 Coding core unit 101 Intra-screen prediction unit 102 Inter-screen prediction unit 103 Coding mode determination unit 104 DCT quantization unit 105 Entropy coding unit 106 Decompression processing unit 107 Restored image generation unit 108 Deblocking processing unit 109 Search region holding unit 110 Search region restoration unit 111 Local decode (LD) image generation unit 112 Block compression unit 113 Block decompression unit 114 Difference image generation unit 115 Statistics extraction unit 116 Statistics aggregation unit 117 Region determination unit 118 Cumulative statistics adjustment unit 122 Addition unit 123 update processing unit 124 cumulative statistic holding unit 125 region detection unit 126 threshold adjustment unit 127 discrimination flag holding unit 128 selection adjustment unit 129 collective adjustment unit 131 mode determination control unit 132 discrimination flag reading unit 134 region selection unit 135 sum of absolute differences Calculation unit 136 Weight addition unit 138 Degrees block extender 139 difference luminance image generation unit 140 external memory 141 compressed locally decoded (LD) image holding unit 142 the original image holding unit

Claims (5)

An encoding processing unit that compresses and encodes each macroblock included in the original image of each frame of the moving image by applying intra-screen encoding or inter-screen encoding;
For each macroblock included in the original image, a local decoded image generation unit that generates a local decoded image from compressed data obtained in the encoding process by the encoding processing unit,
A block compression unit for irreversibly compressing the local decoded image corresponding to each macroblock to generate block compressed data;
A block decompression unit that decompresses the block compressed data corresponding to each macroblock and generates a restored local decoded image corresponding to each macroblock;
A difference image generation unit for generating a difference image indicating an error between the restored local decoded image and the local decoded image corresponding to each macroblock;
A statistic extracting unit that extracts a statistic indicating a statistical property of the distribution of the error that appears in the difference image corresponding to each macroblock;
A statistic totaling unit that cumulatively adds the statistic extracted by the statistic extraction unit for each of a plurality of regions obtained by dividing the frame;
Based on the cumulative error up to each frame indicated by the cumulative addition result by the statistic counting unit, a region to which intra-screen coding is applied among the plurality of regions is determined in the coding process of the next frame of the frame. An area discriminating unit to perform,
A moving picture encoding apparatus comprising: The moving image encoding device according to claim 1,
A cumulative error adjusting unit that adjusts a value indicating a cumulative error that is aggregated for each region by the statistic totaling unit based on a determination result by the region determining unit and an encoding mode that is applied by the encoding processing unit. When,
A moving picture encoding apparatus comprising: The moving image encoding device according to claim 1,
The region discriminating unit
Based on the counting result by the statistic counting unit, an area where the cumulative error is larger than a predetermined threshold Thv,
Based on the result of comparing the number of regions detected by the threshold detection unit and the natural number md smaller than the number N of regions included in the one frame, the threshold comparison unit compares the predetermined error with the accumulated error. A threshold adjustment unit for changing the threshold Thv of
With
An area detected by the area detection unit is an application target of intra-frame encoding. The moving image encoding device according to claim 1,
The region discriminating unit
An area detection unit that detects an area in which the accumulated error is larger than a predetermined threshold Thv based on a counting result by the statistic counting unit;
When the number of regions detected by the region detection unit is larger than another natural number K smaller than the number N of regions included in the one frame, the natural number K of the regions detected by the region detection unit An area selection unit that selects an area as an application target of the intra-screen encoding process;
A moving picture encoding apparatus comprising: The moving image encoding device according to claim 1,
The statistic extraction unit includes:
A component statistic calculation unit for obtaining a statistic for each of the luminance component and the color component included in each pixel value of the difference image;
A weight adding unit that adds weights to the luminance component and the color component respectively corresponding to the statistic, and adds the weighted statistics together;
A moving picture encoding apparatus comprising:

Images