JP2007067526A - Image processor - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image processor in which power consumption is reduced by reducing wasteful memory transfer between a video decoder and a display frame memory. <P>SOLUTION: A variable length decoding section 12 performs variable length decoding of a bit stream inputted from a buffer control section 11 and delivers encoded information to a display control section 17. The display control section 17 determines whether the content varied more than a predetermined amount from the last display update or not for each block based on the encoded information. A block that has been determined that the content varied more than a predetermined amount is transferred from a prediction frame memory 22 or 23 to a display frame memory 40 and is updated. Consequently, wasteful memory transfer between the display control section 17 and the display frame memory 40 can be reduced, and power consumption can be reduced. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to a technique for decoding and reproducing encoded digital video, and more particularly, to an image processing apparatus that detects a change in the contents of a macroblock (MB) and reduces processing related to display.

  In recent years, technologies for compressing and encoding digital video with high efficiency have been developed, and digital encoded not only in STB (set top box) and PC (personal computer) but also in mobile devices such as mobile phones. It has become possible to decode and play back video.

  In a digital video encoding method commonly used in these devices, such as MPEG (Moving Picture Experts Group), each image (frame) constituting the video is divided into small rectangular areas (blocks), and each block is divided into blocks. Predictive data is generated using spatial correlation or temporal correlation. Then, the transform coefficient (difference coefficient) obtained by orthogonal transform of the difference between the image data of the block and the prediction data is quantized, and after performing the scan transform, variable length coding is performed to further reduce spatial redundancy. Thus, high encoding efficiency is obtained.

  Here, a method of coding using only spatial correlation when coding a block is referred to as an intra coding method. In addition, encoding by the intra encoding method is called intra encoding (I encoding), and a frame encoded only by intra encoding is called an intra frame (I frame). In the intra coding method, only the correlation in the frame to which the block being coded belongs is used, so it is also called an intra-frame coding method.

  On the other hand, a coding scheme that uses not only spatial correlation but also temporal correlation is called an inter coding scheme. Also, encoding with the inter encoding scheme is called inter encoding, and a frame encoded using inter encoding is called an inter frame. In the inter coding method, prediction data is generated not from the frame to which the block being coded belongs, but from the previous and subsequent frames, and is also referred to as an inter-frame coding method.

  For example, when predictive data is generated from the frame that was encoded immediately before and only the difference between the currently encoded frame and the predictive data is encoded, this occurs when the video is stationary for several frames. The amount of codes can be greatly reduced. Here, the frame used to generate the prediction data is referred to as a reference frame, forward prediction using the frame before the current encoding frame in the display order as the reference frame, and the frame subsequent to the current encoding in the display order. There are three types of prediction methods, namely, backward prediction using the above frame as a reference frame, and bidirectional prediction using both forward prediction and backward prediction. Hereinafter, inter encoding using only forward prediction is referred to as P encoding, and inter encoding using bidirectional prediction is referred to as B encoding.

  In the encoding method described in Non-Patent Document 1 below, highly efficient encoding is realized by appropriately switching between intra encoding and inter encoding. In this encoding method, encoding is performed by dividing a frame into MB units of 16 pixels vertically and 16 pixels horizontally. In the encoding of one frame, the encoding method can be changed in units of MB. I-MB is I-MB, P-coded MB is P-MB, and B-coded MB is changed. Called B-MB.

  Also, a frame in which one frame is composed only of I-MB or P-MB is called a P frame, and a frame in which at least one MB is B-coded is called a B frame. Depending on the items to be encoded, the MB is divided into smaller blocks, for example, a block of 8 pixels in the vertical direction and 8 pixels in the horizontal direction, and is encoded. In inter coding, a technique called motion compensation is introduced. In motion compensation, for example, when encoding an image in which an object moves in parallel, instead of simply encoding a difference between frames, a parallel movement amount (motion vector) is detected, and a motion vector + This is a technique for further improving the encoding efficiency by performing encoding in the form of a difference between frames.

  However, reproduction of digital video encoded by this encoding method requires enormous computation and memory transfer. Especially in portable devices, reproducible video data from the viewpoint of processing capability and power consumption. Are often subject to constraints such as resolution, frame rate, and bit rate.

  In view of such a situation, the following Patent Document 1 proposes an image processing apparatus that can reduce memory transmission amount and power consumption.

  FIG. 13 is a block diagram illustrating a schematic configuration of the image processing apparatus described in Patent Document 1. As illustrated in FIG. The image processing apparatus includes a video decoder 100 that decodes an input bit stream and outputs a display image, and a memory 200 that stores the input bit stream and the decoded image.

  The video decoder 100 includes a buffer control unit 111, a variable length decoding unit 112, a scan conversion unit 113, an inverse quantization unit 114, an inverse DCT (Discrete Cosine Transform) unit 115, and a motion compensation image reproduction unit 116. A display control unit 117, a memory control unit 118, and a memory update control unit 119.

  In addition, the memory 200 includes a buffer memory 221 that accumulates an input bit stream, a first prediction frame memory 222, and a second prediction frame memory 223.

  Each of the first prediction frame memory 222 and the second prediction frame memory 223 has a capacity capable of storing a decoded image for one frame. The prediction frame memory in which the image being decoded is written is switched every time one frame of video is decoded by the video decoder 100. For example, when the first frame, the second frame, and the third frame, which are three consecutive frames, are decoded, the decoded image of the first frame is stored in the first prediction frame memory 222. The decoded image of the second frame is stored in the second predicted frame memory 223, and the decoded image of the third frame is stored in the first predicted frame memory 222 in the form of overwriting the decoded image of the first frame. The

  The memory control unit 118 controls various data transfers between the buffer control unit 111, the motion compensated image reproduction unit 116 and the display control unit 117, and the memory 200.

  The buffer control unit 111 receives an input bit stream from the outside, and accumulates the input bit stream in the buffer memory 221 via the memory control unit 118. Then, the buffer control unit 111 takes out one frame of the bit stream stored in the buffer memory 221 again via the memory control unit 118 and outputs it to the variable length decoding unit 112.

  The variable length decoding unit 112 performs variable length decoding on the bit stream for one frame input from the buffer control unit 111, and scan conversion unit 113 stores information such as motion vectors and conversion coefficients obtained by the variable length decoding. And output to the motion compensated image reproduction unit 116 and the memory update control unit 119. The transform coefficients output from the variable length decoding unit 112 are arranged in the order of low frequency components to high frequency components with the DC component at the head. Note that not all bitstreams are variable-length encoded, so the variable-length decoding unit 112 also performs decoding on fixed-length codes.

  The scan conversion unit 113 converts the arrangement order of the transform coefficients input from the variable length decoding unit 112 in raster order, and outputs the result to the inverse quantization unit 114.

  The inverse quantization unit 114 inversely quantizes the transform coefficient after scan conversion input from the scan conversion unit 113 and outputs the transform coefficient after inverse quantization to the inverse DCT unit 115.

  The inverse DCT unit 115 performs inverse DCT on the transform coefficient after inverse quantization input from the inverse quantization unit 114, and outputs the transform coefficient after inverse DCT to the motion compensated image reproduction unit 116.

  The motion compensated image reproduction unit 116, based on information (encoding information) such as a motion vector input from the variable length decoding unit 112, the first prediction frame memory 222 (second prediction frame) via the memory control unit 118. The already decoded image (predicted data) stored in the frame memory 223) is read out and added to the transform coefficient after the inverse DCT input from the inverse DCT unit 115 to generate the decoded data, and then the generated decoded data Is written into an appropriate address in the second prediction frame memory 223 (first prediction frame memory 222) different from the prediction frame memory from which the prediction data is read out via the memory control unit 118.

  In addition to the motion vector, the encoded information includes a skipped MB signal, a not-coded block signal indicating the presence / absence of a transform coefficient for each of the six blocks in the macroblock, and the like. Note that image data of a processed block among images currently being decoded is used as prediction data at the time of decoding of an intra-coded block.

  The display control unit 117 reads out a reproduced image from the first prediction frame memory 222 or the second prediction frame memory 223 via the memory control unit 118, and performs display image output control.

  The memory update control unit 119 controls the operation of the motion compensated image reproduction unit 116 according to the content of the encoded information input from the variable length decoding unit 112.

  Here, consider a case where the video decoder 100 decodes a digital video composed of a first frame, a second frame, and a third frame, which are three consecutive frames.

  When the memory update control unit 119 decodes a block (third block) at a predetermined position in the third frame, the content of the third block is decoded in the second frame decoded immediately before the third frame. It is determined whether or not the content of the block (second block) at the same position as the predetermined position is the same (first determination). For the first determination, encoded information about the third block is used. Specifically, when the encoding information for the third block is a skipped MB, or when the motion vector is zero and a not-coded block, it is determined that they are the same. In other cases, it is determined that they are not the same.

  If it is determined as the same as a result of the first determination, the content of the second block is a block at the same position as the predetermined position in the first frame decoded immediately before the second frame. It is determined whether or not the content is the same as (first block) (second determination).

  The second determination is performed by the same method as the first determination, using the encoded information about the second block. However, during the decoding of the third frame, the encoding information input to the memory update control unit 119 is only the encoding information for the third frame, and the second frame which is the frame immediately before the third frame. No encoding information is input for this frame. For this reason, the memory update control unit 119 internally holds at least only information used for the second determination among the encoded information for all the blocks for one frame decoded immediately before the frame currently being decoded. .

  As a result of the second determination, if it is determined that they are the same, the prediction from the first prediction frame memory 222 (second prediction frame memory 223) in the motion compensated image reproduction unit 116 for the third block is performed. Reading of data and writing of decoded data to the second prediction frame memory 223 (first prediction frame memory 222) are omitted.

  The reason why such omission is possible will be briefly described. Since the first frame, the second frame, and the third frame are sequentially decoded, for example, if the decoding result of the first frame is written in the first prediction frame memory 222, the second frame The decoding result of the second frame is written in the second prediction frame memory 223, and the decoding result of the third frame is written in the first prediction frame memory 222.

  On the other hand, the first block is included in the first frame, the third block is included in the third frame, and the first block and the third block are in the same position in the frame. The decoding result of the first block is written in the area of the first prediction frame memory 222 in which the decoding result of the third block is written when the third block is decoded. Become. In addition, it is determined that the result of the first determination and the result of the second determination are the same. In other words, the content of the third block is the same as the content of the first block. It means that there is. Therefore, whether or not such omission is performed, there is no difference in the contents of the position of the third block in the first prediction frame memory 222 after the third block decoding.

  Further, when it is determined that the result of the first determination or the result of the second determination is not the same, the above omission is not performed.

  In this way, by providing the memory update control unit 119, when decoding a digital video having still blocks continuous over three or more frames, unnecessary memory transfer is omitted, and as a result, A limited memory bandwidth can be used effectively, and power consumption required for memory transfer can be reduced.

  In the image processing apparatus shown in FIG. 13, it is assumed that the first prediction frame memory 222 and the second prediction frame memory 223 are also used as a direct display frame memory. In many cases, the memory bandwidth between the video decoder 100 and the display frame memory is not sufficient.

  In such a case, the first prediction frame memory 222 and the second prediction frame memory 223 that are frequently accessed from the video decoder 100 are separately secured in a memory area having a memory bandwidth wider than that of the display frame memory. It is common to do. Each time the display is updated, the display control unit 117 transfers the memory from the first prediction frame memory 222 and the second prediction frame memory 223 to the display frame memory only once, so that the display frame memory with a narrow memory bandwidth is obtained. Can reduce access to and improve video playback performance.

  There are also cases where it is desired to overwrite another display object on the reproduced image. Even in this case, the display object cannot be directly overwritten on the contents of the first prediction frame memory 222 and the second prediction frame memory 223, and the decoded image is once transferred to another display frame memory for display. It is necessary to perform an overwrite operation on the frame memory. This is because the contents of the first prediction frame memory 222 and the second prediction frame memory 223 may be used as a reference image in the decoding of a subsequent frame, and the contents are changed from the outside of the video decoder 100. This is because it adversely affects the decoding result of the subsequent frame.

  FIG. 14 is a block diagram showing a schematic configuration of an image processing apparatus when a display frame memory is provided. This image processing apparatus is different from the image processing apparatus shown in FIG. 13 only in that a second memory control unit 300 and a display frame memory 400 for storing display images are added. Therefore, detailed description of overlapping configurations and functions will not be repeated.

The second memory control unit 300 receives the display image data from the display control unit 117 and writes it to the display frame memory 400. Note that an area of the display frame memory 400 may be secured in the memory 200. In this case, the second memory control unit 300 becomes unnecessary, and the first memory control unit 118 writes the display image data in the display frame memory 400.
JP 2000-115777 A ISO / IEC 14496-2 (video coding technology) ISO / IEC 14496-10 (video coding technology)

  However, in the image processing apparatus shown in FIG. 14, when the decoded image is transferred to the display frame memory 400, the encoded information is not input to the display control unit 117. Even if it can be detected that the frame is stationary, the entire frame must be updated every time, and there is a problem that unnecessary memory transfer and power consumption are required.

  The present invention has been made to solve the above problems, and an object of the present invention is to reduce an unnecessary memory transfer between a video decoder and a display frame memory, and to reduce power consumption. Is to provide.

  According to an aspect of the present invention, there is provided an image processing device that decodes and displays an encoded image, and decodes the encoded image in units of blocks to generate image data and encoding information. Decoding means, prediction frame storage means for storing image data generated by the decoding means as a prediction frame, display frame storage means for storing display frames, and encoding information generated by the decoding means Based on the determination means for determining whether or not there has been a content change of a predetermined amount or more from the previous display update for each block, and the block determined to have a content change of the predetermined amount or more by the determination means is predicted frame Display update means for transferring and updating from the storage means to the display frame storage means.

  Preferably, the determination unit updates the change amount storage unit on the basis of the change amount storage unit for storing the change amount of the content from the previous display update for each block and the encoded information generated by the decoding unit. And an update necessity determining means for determining whether or not display update is necessary for each block with reference to the change amount storage means.

  More preferably, the update unit updates the change amount storage unit with information indicating whether or not both of the motion vector and the difference coefficient included in the encoded information are zero, and the update necessity determination unit includes the change amount storage unit. With reference to the means, if the motion vector is not zero, or if the difference coefficient is not zero, it is determined that there has been a content change of a predetermined amount or more since the previous display update.

  More preferably, when the block is an intra-coded macro block, the update unit sets information indicating that the motion vector or the difference coefficient is not zero in the change amount storage unit corresponding to the block.

  Preferably, the update unit accumulates the code amount included in the encoded information for each block and updates the change amount storage unit, and the update necessity determination unit determines that the accumulated code amount is equal to or greater than a first threshold value. It is determined that there has been a change in content by a predetermined amount or more since the previous display update.

  More preferably, when the block is an intra-coded macro block, the update unit sets a value equal to or greater than the first threshold value in the change amount storage unit corresponding to the block.

  Preferably, the update unit updates the change amount storage unit by accumulating a value obtained by adding a predetermined coefficient to the code amount of the difference coefficient included in the encoded information for each block, and the update necessity determination unit is accumulated. When the measured value is equal to or greater than the second threshold, it is determined that there has been a change in content by a predetermined amount or more since the previous display update.

  Preferably, when the block is an intra-coded macro block, the updating unit sets a value equal to or larger than the second threshold value in the change amount storage unit corresponding to the block.

More preferably, the predetermined coefficient is a quantization parameter value of the block.
Preferably, the predetermined coefficient is a power of 2 (quantization parameter value of block ÷ 6).

  Preferably, the update necessity determination unit initializes the content of the change amount storage unit corresponding to the block determined to have changed by a predetermined amount or more since the previous display update.

  According to another aspect of the present invention, an image processing apparatus that decodes and displays an encoded image and decodes the encoded image in units of blocks to generate image data and encoding information Decoding means, prediction frame storage means for storing image data generated by the decoding means as a prediction frame, display frame storage means for storing display frames, and encoding generated by the decoding means Based on the information, it is determined for each block whether there has been a content change of a predetermined amount or more from the previous display update, and the number of blocks having a content change of a predetermined amount or more from the previous display update is greater than or equal to the predetermined number If it is determined by the determining means for determining whether or not the number of blocks whose content has changed by a predetermined amount or more by the determining means is a predetermined number or more, all blocks The updates are transferred to the display frame storage means from the prediction frame storage means, in other cases, and a display update unit does not update the display frame storage means.

  Preferably, the predicted frame storage means includes two predicted frame memories in which image data of consecutive frames are alternately stored, and the image processing apparatus further includes the same contents of corresponding blocks in the three consecutive frames. In this case, the block of the third frame in the prediction frame memory is not updated, and means for updating other blocks is included.

  According to an aspect of the present invention, the display update unit transfers the block determined to have changed by a predetermined amount or more by the determination unit from the predicted frame storage unit to the display frame storage unit, and updates it. Blocks other than are not transferred to the display frame storage means, wasteful memory transfer between the display update means and the display frame storage means can be reduced, and power consumption can be reduced.

  In addition, since the update necessity determination unit refers to the change amount storage unit and determines whether or not display update is required for each block, it is possible to easily determine whether or not display update is necessary.

  In addition, the update necessity determination unit refers to the change amount storage unit, and determines that there has been a content change of a predetermined amount or more from the previous display update when the motion vector is not zero or the difference coefficient is not zero. Therefore, it is possible to more easily determine whether or not display updating is necessary.

  In addition, when the block is an intra-coded macroblock, the update unit sets information indicating that the motion vector or the difference coefficient is not zero in the change amount storage unit corresponding to the block. It is always possible to update the display.

  In addition, when the accumulated code amount is equal to or greater than the first threshold value, the update necessity determination unit determines that the content has changed by a predetermined amount or more since the previous display update. However, the display update can be omitted.

  In addition, when the block is an intra-coded macro block, the updating unit sets a value equal to or larger than the first threshold value in the change amount storing unit corresponding to the block, so that the display of the intra-coded macro block is always updated. Can be performed.

  In addition, when the accumulated necessity value is equal to or greater than the second threshold value, the update necessity determination unit determines that the content has changed by a predetermined amount or more since the previous display update. It becomes possible to detect.

  In addition, when the block is an intra-coded macro block, the update unit sets a value equal to or greater than the second threshold value in the change amount storage unit corresponding to the block, so that the display of the intra-coded macro block is always updated. Can be performed.

  Further, since the predetermined coefficient is the quantization parameter value of the block, it is possible to detect a block with a smaller amount of change more accurately.

  Further, since the predetermined coefficient is a power of 2 (quantization parameter value of block ÷ 6), it is possible to detect a block with a small amount of change more accurately.

  In addition, since the update necessity determination unit initializes the content of the change amount storage unit corresponding to the block that has been determined to have changed by a predetermined amount or more since the previous display update, the content change can be obtained more accurately. It becomes possible.

  According to another aspect of the present invention, the display update unit determines that all blocks are predicted frames when the determination unit determines that the number of blocks whose content has changed by a predetermined amount or more is a predetermined number or more. The data is transferred from the storage means to the display frame storage means and updated. In other cases, the display frame storage means is not updated. Therefore, the display frame storage means is updated in accordance with changes in the entire screen. It becomes possible.

  If the contents of the corresponding blocks in the three consecutive frames are all the same, the block of the third frame in the prediction frame memory is not updated, and the other blocks are updated. Since it includes means for performing, it is possible to reduce useless memory transfer with the prediction frame memory and further reduce power consumption.

  Assume that the image processing apparatus according to the embodiment of the present invention described below reproduces a bitstream in which all frames are encoded as I frames or P frames.

(First embodiment)
FIG. 1 is a block diagram showing a schematic configuration of an image processing apparatus according to the first embodiment of the present invention. The image processing apparatus includes a video decoder 1 that decodes an input bit stream and outputs a display image, a memory 2 that stores the input bit stream and the decoded image, and a display frame memory that stores the display image 40 and a second memory control unit 30 that controls writing of a display image to the display frame memory 40.

  The second memory control unit 30 receives display image data from the video decoder 1 and writes it into the display frame memory 40.

  The video decoder 1 includes a buffer control unit 11, a variable length decoding unit 12, a scan conversion unit 13, an inverse quantization unit 14, an inverse DCT unit 15, a motion compensated image reproduction unit 16, and a display control unit 17. And a first memory control unit 18.

  In addition, the memory 2 includes a buffer memory 21 that accumulates an input bit stream, a first prediction frame memory 22, and a second prediction frame memory 23.

  Each of the first predicted frame memory 22 and the second predicted frame memory 23 has a capacity capable of storing a decoded image for one frame. The prediction frame memory in which the image being decoded is written is switched every time one frame of video is decoded by the video decoder 1.

  The first memory control unit 18 controls various data transfers among the buffer control unit 11, the motion compensated image reproduction unit 16 and the display control unit 17, and the memory 2.

  The buffer control unit 11 receives an input bit stream from the outside, and accumulates the input bit stream in the buffer memory 21 via the first memory control unit 18. Then, the buffer control unit 11 takes out one frame of the bit stream stored in the buffer memory 21 again via the first memory control unit 18 and outputs it to the variable length decoding unit 12.

  The variable length decoding unit 12 performs variable length decoding on the bit stream for one frame input from the buffer control unit 11, and scan conversion unit 13 obtains information such as motion vectors and conversion coefficients obtained by the variable length decoding. And output to the motion compensated image reproduction unit 16 and the display control unit 17.

  The scan conversion unit 13 converts the arrangement order of the conversion coefficients input from the variable length decoding unit 12 in raster order, and outputs the result to the inverse quantization unit 14.

  The inverse quantization unit 14 inversely quantizes the transform coefficient after the scan conversion input from the scan conversion unit 13, and outputs the transform coefficient after the inverse quantization to the inverse DCT unit 15.

  The inverse DCT unit 15 performs inverse DCT on the transform coefficient after inverse quantization input from the inverse quantization unit 14, and outputs the transform coefficient after inverse DCT to the motion compensated image reproduction unit 16.

  The motion compensated image reproduction unit 16 receives the first prediction frame memory 22 (second prediction frame) via the first memory control unit 18 based on the encoded information such as the motion vector input from the variable length decoding unit 12. The decoded data generated after the decoded image (predicted data) stored in the frame memory 23) is read and added to the transform coefficient after the inverse DCT input from the inverse DCT unit 15 to generate the decoded data. Is written to an appropriate address in the second predicted frame memory 23 (first predicted frame memory 22) different from the predicted frame memory from which the predicted data is read out via the first memory control unit 18.

  The display control unit 17 reads out the reproduced image from the first prediction frame memory 22 or the second prediction frame memory 23 via the first memory control unit 18 and controls the output of the display image. At this time, the display control unit 17 uses the encoded information input from the variable length decoding unit 13 in the reproduced image stored in the first prediction frame memory 22 (second prediction frame memory 23). Only the blocks that have changed significantly from the previous display image are extracted, the contents of the blocks are read out via the first memory control unit 18, and the contents of the display frame memory 40 are updated via the second memory control unit 30. To do.

  FIG. 2 is a block diagram showing a detailed configuration of the display control unit 17 of the image processing apparatus shown in FIG. The display control unit 17 includes a block change amount table update unit 51, a block change amount table 52, a display update necessity determination unit 53, and a display data update unit 54.

  Based on the encoded information input from the variable length decoding unit 12, the block change amount table updating unit 51 obtains a change amount from a frame decoded before the frame being decoded in units of blocks, and uses this change amount. Accordingly, the block change amount table 52 is updated. In addition to the motion vector, the skipped MB signal, and the not-coded block signal, the encoded information includes a code amount value of each block. The code amount value of each block is obtained when the variable length decoding unit 12 performs variable length decoding, and the value is acquired by the display control unit 17.

  The block change amount table 52 holds a block change amount for a block of at least one frame. Here, the block change amount is an objective measurement of how much difference there is between the content of a predetermined block and the content of the block in the same position as the predetermined block in the display frame memory 40. It is a numerical value that can be evaluated, and is calculated using encoded information.

  FIG. 3 is a diagram for explaining the contents stored in the block change amount table 52. FIG. 3A shows an example of a specific storage format of the block change amount table 52. The block change amount table 52 holds data as a two-dimensional array for one frame with the block change amount for one block as one element. The storage method is not limited to a two-dimensional array, and may be a one-dimensional array, for example.

  FIGS. 3B to 3D are diagrams illustrating an example of the storage format of the block change amount. FIG. 3B shows binary data in which a block with zero motion vector and zero difference coefficient is “no change” and other blocks are “with change” (hereinafter referred to as motion vector zero / difference coefficient zero flag). .) Is stored as a block change amount.

  FIG. 3C shows a case where the accumulated value of the code amount of the block since the previous display update (hereinafter referred to as the accumulated block code amount) is stored as the block change amount.

  FIG. 3D shows a flag indicating zero / non-zero of the motion vector of the block (hereinafter referred to as the motion vector zero flag), and (code amount of the difference coefficient of the block) × ( A case is shown in which two fields of accumulated values (hereinafter referred to as accumulated scaled difference coefficient code amounts) of quantization parameter values for blocks are stored as block change amounts. The storage format of the block change amount is not limited to the above, and any format may be used.

  The display update necessity determination unit 53 refers to the block change amount table 52 and instructs the display data update unit 54 to update the display of a block that is determined to have a large change. The display update necessity determination unit 53 also performs an operation of setting (clearing) the elements in the block change amount table 52 for the blocks for which the display has been updated without change amounts.

  When an instruction is input from the display update necessity determination unit 53, the display data update unit 54 receives information from the first predicted frame memory 22 (second predicted frame memory 23) via the first memory control unit 18. The reproduced image is read out and written into the display frame memory 40 via the second memory control unit 30.

  FIG. 4 is a flowchart for explaining a processing procedure when the image processing apparatus according to the first embodiment of the present invention decodes a bit stream for one frame. First, when the variable length decoding unit 12 receives a bit stream of a frame from the buffer control unit 11, it decodes one block (S11). Next, the block change amount table updating unit 51 receives the encoding information obtained in the block decoding process, and updates the element for the block in the block change amount table 52 (S12).

  Next, the variable length decoding unit 12 determines whether or not the processing of all the blocks of one frame has been completed (S13). If all the blocks of one frame have not been processed (S13, No), the process returns to step S11 to process the next block. When all the blocks of one frame have been processed (S13, Yes), the process is terminated.

  In the flowchart shown in FIG. 4, decoding and update of the block change amount table 52 are alternately performed for each block, but after decoding all the blocks of one frame, the encoding information for one frame is used as the source. Alternatively, the block change amount table 52 may be updated.

  FIG. 5 is a flowchart for explaining a processing procedure when the image processing apparatus according to the first embodiment of the present invention updates the display for one frame. First, the display update necessity determination unit 53 refers to an element for one block in the block change amount table 52 and determines whether or not a large change has occurred in the content since the previous display update (S21). .

  When it is determined that a large change has occurred (S21, Yes), the display data update unit 54 stores the block stored in the first prediction frame memory 22 (second prediction frame memory 23). The decoded image is read out via the first memory control unit 18, and the contents of the corresponding part of the display frame memory 40 are updated via the second memory control unit 30 (S22). Next, the display update necessity determination unit 53 clears the element for the block in the block change amount table 52, and ends the process for the block (S23).

  If it is determined that no significant change has occurred (No in S21), the process of the block is terminated as it is, and the process proceeds to step S24.

  When the processing for one block is completed, it is determined whether or not the processing for all the blocks in one frame has been completed (S24). If all the blocks have not been processed (S24, No), the process returns to step S21 to process the next block. If all the blocks have been processed (S24, Yes), the process ends.

  In the flowchart shown in FIG. 5, display update / non-update is controlled in units of blocks, but first, the amount of change for all blocks for one frame is evaluated, and a predetermined number of one frame is evaluated. If it is determined that the content change is large in the above blocks, the entire frame may be updated and the elements for all the blocks in the block change amount table 52 may be cleared.

  As described above, in the image processing apparatus according to the present embodiment, transfer to the display frame memory 40 can be omitted for a block that remains stationary for two frames or more. The memory bandwidth can be used effectively and the power consumption can be reduced.

  Further, when the memory bandwidth between the video decoder 1 and the display frame memory 40 is narrower than the memory bandwidth between the video decoder 1 and the memory 2, the time required for memory transfer can be shortened. It has become possible to improve performance in video playback.

(Second Embodiment)
FIG. 6 is a block diagram showing a schematic configuration of an image processing apparatus according to the second embodiment of the present invention. This image processing apparatus is different from the image processing apparatus according to the first embodiment shown in FIG. 1 only in that a memory update control unit 19 is added. Therefore, detailed description of overlapping configurations and functions will not be repeated.

  The memory update control unit 19 controls the operation of the motion compensated image reproduction unit 16 according to the content of the encoded information input from the variable length decoding unit 12. The function of the memory update control unit 19 is the same as the function of the memory update control unit 119 of the conventional image processing apparatus shown in FIG.

  Since the display control unit 17 and the memory update control unit 19 operate independently of each other, unnecessary memory transfer between the video decoding unit 1 and the memory 2 in addition to the effects described in the first embodiment. As a result, the limited memory bandwidth can be effectively used, and the power consumption required for memory transfer can be reduced.

(Third embodiment)
In the image processing apparatus according to the third embodiment of the present invention, the motion vector zero / difference coefficient zero flag shown in FIG. 3B is used as the format of the block variation stored in the block variation table 52. It is done. The schematic configuration of the image processing apparatus in the present embodiment is the same as the schematic configuration of the image processing apparatus in the first embodiment shown in FIG. 1 or the image processing apparatus in the second embodiment shown in FIG. is there.

  FIG. 7 is a flowchart for explaining a processing procedure when the image processing apparatus according to the third embodiment of the present invention decodes a bit stream for one frame. First, when the variable length decoding unit 12 receives a bit stream of a frame from the buffer control unit 11, it decodes one block (S31). Next, the block change amount table update unit 51 receives the encoded information (at least the skipped MB signal, the motion vector, and the not-coded block signal) obtained in the block decoding process, and The element about the block is updated (S32). When the motion vector is zero and the difference coefficient is zero, “no change” is set as the motion vector zero / difference coefficient zero flag, otherwise “change” is set as the motion vector zero / difference coefficient zero flag. Yes "is set.

  Whether the motion vector is zero and the difference coefficient is zero can be determined from the fact that the block is a skipped MB, and can also be determined from the fact that the motion vector is zero and a knot-coded block.

  Next, the variable length decoding unit 12 determines whether or not the processing of all the blocks of one frame has been completed (S33). If all the blocks of one frame have not been processed (No at S33), the process returns to step S31 to process the next block. When all the blocks of one frame have been processed (S33, Yes), the process is terminated.

  FIG. 8 is a flowchart for explaining a processing procedure when the image processing apparatus according to the third embodiment of the present invention updates the display for one frame. First, the display update necessity determination unit 53 refers to an element for one block in the block change amount table 52, and determines whether the block is “changed” or “no change” (S41).

  When it is determined that there is a change (S41, Yes), the display data update unit 54 decodes the block stored in the first prediction frame memory 22 (second prediction frame memory 23). The image is read out via the first memory control unit 18, the contents of the corresponding part of the display frame memory 40 are updated via the second memory control unit 30, and the process for the block is terminated (S 42).

  If it is determined that there is no change (S41, No), the process for the block is terminated as it is, and the process proceeds to step S43.

  When the process for one block is completed, it is determined whether or not the process for all the blocks in one frame is completed (S43). If all the blocks have not been processed (S43, No), the process returns to step S41 to process the next block. If all the blocks have been processed (S43, Yes), the process ends.

  As described above, in the image processing apparatus according to the present embodiment, the display is not updated for a block that has been decoded immediately before and whose content is “no change” from the displayed frame. Transfers can be reduced, and wasteful power consumption can be reduced.

(Fourth embodiment)
In the image processing apparatus according to the fourth embodiment of the present invention, the accumulated block code amount shown in FIG. 3C is used as the format of the block variation stored in the block variation table 52. The schematic configuration of the image processing apparatus in the present embodiment is the same as the schematic configuration of the image processing apparatus in the first embodiment shown in FIG. 1 or the image processing apparatus in the second embodiment shown in FIG. is there.

  FIG. 9 is a flowchart for explaining a processing procedure when the image processing apparatus according to the fourth embodiment of the present invention decodes a bit stream for one frame. First, when the variable length decoding unit 12 receives a bit stream of a frame from the buffer control unit 11, it decodes one block (S51). Next, the block change amount table updating unit 51 receives the encoded information (at least the type of MB to which the block belongs and the code amount of the block) obtained in the process of decoding the block, and Update the elements for the block as follows: At this time, it is determined whether the block type is I-MB or P-MB (S52).

  If the block belongs to the I-MB (S52, I-MB), the value of the accumulated block code amount for the block is set to a predetermined threshold Ts or more (S53), and the process proceeds to step S55.

  If the block belongs to the P-MB (S52, P-MB), the code amount of the block is added to the accumulated block code amount for the block (S54), and the process proceeds to step S55.

  In step S55, it is determined whether or not the processing for all the blocks in one frame has been completed. If all the blocks of one frame have not been processed (S55, No), the process returns to step S51 to process the next block. If all the blocks of one frame have been processed (S55, Yes), the process is terminated.

  FIG. 10 is a flowchart for explaining a processing procedure when the image processing apparatus according to the fourth embodiment of the present invention updates the display for one frame. First, the display update necessity determination unit 53 refers to an element for one block in the block change amount table 52, and whether the cumulative block code amount from the previous display update for the block is equal to or greater than a predetermined threshold Ts. It is determined whether or not (S61).

  When it is determined that the threshold value Ts is exceeded (S61, Yes), the display data update unit 54 decodes the decoded image for the block stored in the first prediction frame memory 22 (second prediction frame memory 23). Is read via the first memory control unit 18, and the contents of the corresponding part of the display frame memory 40 are updated via the second memory control unit 30 (S62). Then, the accumulated block code amount for the block is cleared to zero (S63), and the processing for the block is terminated.

  If it is determined that the value is less than the threshold Ts (S61, No), the process for the block is terminated as it is, and the process proceeds to step S64.

  When the process for one block is completed, it is determined whether or not the process for all the blocks in one frame is completed (S64). If all the blocks have not been processed (S64, No), the process returns to step S61 to process the next block. If all the blocks have been processed (S64, Yes), the process ends.

  For I-MB, there is no correlation between the amount of code and the change in content from the previous frame. In other words, even if the code amount is small, the change from the previous frame is not always small. Therefore, if the display update is omitted, the subjective image quality in the reproduced video may be deteriorated. Therefore, when a block belonging to the I-MB is decoded, the display update is always performed by setting the accumulated block code amount for the block to be equal to or greater than the threshold value Ts regardless of the actual code amount.

  On the other hand, regarding P-MB, the smaller the code amount, the smaller the change in content from the immediately preceding frame, and the larger the code amount, the greater the change in content from the immediately preceding frame. For example, in the case of a skipped MB with no change from the previous frame, the code amount per MB is at most 1 bit. Therefore, even if the display update of a block with a small code amount is omitted, the subjective image quality does not deteriorate immediately.

  However, for example, when a slight change continues over several tens of frames, even if the change in the contents of the block between each frame is slight, the first and last frames of the tens of frames There is a possibility that a large difference will occur with this frame. Therefore, not only the blocks whose code amount generated from the immediately preceding frame is equal to or greater than the threshold Ts are displayed and updated, but the blocks whose cumulative block code amount from the previous display update is equal to or greater than the threshold Ts are displayed. It should be noted that the threshold value Ts is desirably set as large as possible within a range where subjective image quality is not deteriorated regardless of what video is reproduced.

  As described above, according to the image processing apparatus of the fourth embodiment of the present invention, the display is not updated for blocks whose cumulative block code amount after the previous display update is less than the predetermined threshold Ts. As a result, display updating can be omitted not only for blocks having no change but also for blocks having little change, and unnecessary memory transfer is further performed as compared with the image processing apparatus according to the third embodiment of the present invention. In addition to being able to reduce, it has become possible to further reduce wasteful power consumption.

(Fifth embodiment)
In the image processing apparatus according to the fifth embodiment of the present invention, the motion vector zero flag and the cumulative scaled difference coefficient shown in FIG. A code amount is used. The schematic configuration of the image processing apparatus in the present embodiment is the same as the schematic configuration of the image processing apparatus in the first embodiment shown in FIG. 1 or the image processing apparatus in the second embodiment shown in FIG. is there.

  FIG. 11 is a flowchart for explaining a processing procedure when the image processing apparatus according to the fifth embodiment of the present invention decodes a bit stream for one frame. First, when the variable length decoding unit 12 receives a bit stream of a frame from the buffer control unit 11, it decodes one block (S71). Next, the block change amount table updating unit 51 receives the encoding information (at least the type of the MB to which the block belongs, the motion vector, the code amount of the difference coefficient, and the quantization parameter value) obtained in the process of decoding the block. The elements of the block in the block change amount table 52 are updated as follows. At this time, it is determined whether the block type is I-MB or P-MB (S72).

  If the block belongs to I-MB (S72, I-MB), the motion vector zero flag for the block is set to non-zero (motion vector non-zero) (S73), and the process proceeds to step S76.

  If the block belongs to the P-MB (S72, P-MB), whether the motion vector is zero for the block is set in the motion vector zero flag for the block (S74). Then, (the code amount of the difference coefficient of the block) × (the quantization parameter value for the block) is added to the accumulated scaled difference coefficient code amount for the block (S75), and the process proceeds to step S76.

  Next, it is determined whether or not the processing of all the blocks in one frame has been completed (S76). If all the blocks of one frame have not been processed (S76, No), the process returns to step S71 to process the next block. If all the blocks of one frame have been processed (S76, Yes), the process ends.

  FIG. 12 is a flowchart for explaining the processing procedure when the image processing apparatus according to the fifth embodiment of the present invention updates the display for one frame. First, the display update necessity determination unit 53 refers to an element for one block in the block change amount table 52, and the motion vector zero flag for the block is not zero (non-zero) or for the block. It is determined whether the accumulated scaled difference coefficient code amount is equal to or greater than a predetermined threshold Td (S81).

  If the motion vector zero flag for the block is non-zero, or if the cumulative scaled difference coefficient code amount for the block is equal to or greater than the predetermined threshold Td (S81, Yes), the display data update unit 54 The decoded image of the block stored in the first prediction frame memory 22 (second prediction frame memory 23) is read out via the first memory control unit 18 and displayed via the second memory control unit 30. The contents of the corresponding portion of the frame memory 40 are updated (S82). Then, the cumulative scaled difference coefficient code amount for the block is cleared to zero (S83), and the process for the block is terminated.

  If the motion vector zero flag for the block is zero and the cumulative scaled difference coefficient code amount for the block is less than the predetermined threshold Td (S81, No), the processing for the block is performed as it is. finish.

  When the process for one block is completed, it is determined whether or not the process for all the blocks in one frame is completed (S84). If all the blocks have not been processed (S84, No), the process returns to step S81 to process the next block. If all the blocks have been processed (S84, Yes), the process ends.

  For I-MB, there is no correlation between the amount of code and the change in content from the previous frame. In other words, even if the code amount is small, the change from the previous frame is not always small, and thus the subjective image quality in the reproduced video may be degraded if the display update is omitted. Therefore, when a block belonging to I-MB is decoded, the display vector is always updated by setting the motion vector zero flag for the block to non-zero. Instead of setting the motion vector zero flag to non-zero, the cumulative scaled difference coefficient code amount may be set to be equal to or greater than the threshold value Td, and the display may be updated without fail.

  On the other hand, regarding P-MB, the smaller the code amount, the smaller the change in content from the immediately preceding frame, and the larger the code amount, the greater the change in content from the immediately preceding frame. Here, the two most dominant codes are the code quantity of the motion vector and the code quantity of the difference coefficient.

  However, for example, when the encoding method described in Non-Patent Document 1 is used, the motion vector does not encode the value itself, but encodes only the difference between the motion vectors of the neighboring MBs. Just because the code amount of a vector is small does not mean that the amount of change from the previous frame is small. Therefore, regarding the motion vector, it is desirable to measure the magnitude of the change from the immediately preceding frame not by the code amount but by the magnitude of the motion vector value itself as a result of variable length decoding.

  Also, the code amount of the difference coefficient depends not only on the magnitude of the change in the contents of the block but also on the magnitude of the quantization parameter value when the difference coefficient is derived. For example, in the encoding method described in Non-Patent Document 1, when the change in the contents of a block is the same, regardless of the quantization parameter value, (code amount of difference coefficient) × (quantization parameter value) is substantially constant become. This is because (the difference coefficient code amount) × (quantization parameter value) is more strongly correlated with the change in the block content than the difference coefficient code amount itself. Means that.

  Therefore, as described in the fourth embodiment of the present invention, whether or not the motion vector is zero, rather than simply determining whether to update or not update the display based on the cumulative value of the code amount of all blocks. If the display update / non-update is determined based on whether the cumulative value of the cumulative scaled difference coefficient code amount is large or small, it is possible to update only the display of the block having a large visual change more accurately. Note that the value of the threshold value Td is desirably set by the same method as the value of the threshold value Ts in the fourth embodiment of the present invention.

  In the above description, a case has been described where (difference coefficient code amount) × (quantization parameter value) is a cumulative scaled difference coefficient code amount. For example, in the encoding method described in Non-Patent Document 2, When the changes in the block contents are the same, the value of (difference coefficient code amount) × (2 ^ (quantization parameter value ÷ 6)) is substantially constant, so this multiplied value is used as the cumulative scaled difference coefficient code amount. It is better to set to. As described above, it is preferable to appropriately derive the accumulated scaled difference coefficient code amount according to the encoding method to be used.

  As described above, according to the image processing device of the fifth embodiment of the present invention, the block whose motion vector is zero and the accumulated scaled difference coefficient code amount since the previous display update is a predetermined value. By not updating the display for the block that is less than the threshold value Td, the block with less change from the previous display update is detected more accurately and displayed as compared with the image processing apparatus in the fourth embodiment of the present invention. It became possible to omit the update.

  The embodiment disclosed this time should be considered as illustrative in all points and not restrictive. The scope of the present invention is defined by the terms of the claims, rather than the description above, and is intended to include any modifications within the scope and meaning equivalent to the terms of the claims.

1 is a block diagram illustrating a schematic configuration of an image processing apparatus according to a first embodiment of the present invention. It is a block diagram which shows the detailed structure of the display control part 17 of the image processing apparatus shown in FIG. FIG. 6 is a diagram for explaining contents stored in a block change amount table 52; It is a flowchart for demonstrating the process sequence at the time of the image processing apparatus in the 1st Embodiment of this invention decoding the bit stream for 1 frame. It is a flowchart for demonstrating the process sequence in case the image processing apparatus in the 1st Embodiment of this invention updates the display for 1 frame. It is a block diagram which shows schematic structure of the image processing apparatus in the 2nd Embodiment of this invention. It is a flowchart for demonstrating the process sequence at the time of the image processing apparatus in the 3rd Embodiment of this invention decoding the bit stream for 1 frame. It is a flowchart for demonstrating the process sequence in case the image processing apparatus in the 3rd Embodiment of this invention updates the display for 1 frame. It is a flowchart for demonstrating the process sequence at the time of the image processing apparatus in the 4th Embodiment of this invention decoding the bit stream for 1 frame. It is a flowchart for demonstrating the process sequence in case the image processing apparatus in the 4th Embodiment of this invention updates the display for 1 frame. It is a flowchart for demonstrating the process sequence at the time of the image processing apparatus in the 5th Embodiment of this invention decoding the bit stream for 1 frame. It is a flowchart for demonstrating the process sequence in case the image processing apparatus in the 5th Embodiment of this invention updates the display for 1 frame. 10 is a block diagram illustrating a schematic configuration of an image processing apparatus described in Patent Literature 1. FIG. It is a block diagram which shows schematic structure of the image processing apparatus at the time of providing the frame memory for a display.

Explanation of symbols

  1,100 video decoder, 2,200 memory, 11,111 buffer control unit, 12,112 variable length decoding unit, 13,113 scan conversion unit, 14,114 inverse quantization unit, 15,115 inverse DCT unit, 16 , 116 motion compensation image reproduction unit, 17, 117 display control unit, 18, 118 first memory control unit, 19, 119 memory update control unit, 21, 221 buffer memory, 22, 222 first prediction frame memory, 23 , 223 Second prediction frame memory, 30, 300 Second memory control unit, 40, 400 Display frame memory, 51 Block change amount table update unit, 52 Block change amount table, 53 Display update necessity determination unit, 54 Display data update unit.

Claims (13)

An image processing device for decoding and displaying an encoded image,
Decoding means for decoding the encoded image in units of blocks to generate image data and encoding information;
Prediction frame storage means for storing the image data generated by the decoding means as a prediction frame;
Display frame storage means for storing display frames;
Based on the encoding information generated by the decoding means, determination means for determining whether there has been a content change of a predetermined amount or more from the previous display update for each block;
An image processing apparatus, comprising: a display update unit configured to transfer and update a block determined by the determination unit to have a content change of a predetermined amount or more from the predicted frame storage unit to the display frame storage unit. The determination means includes a change amount storage means for storing a change amount of contents from the previous display update for each block;
Updating means for updating the variation storage means based on the encoded information generated by the decoding means;
The image processing apparatus according to claim 1, further comprising: an update necessity determination unit that determines whether or not display update is necessary for each block with reference to the change amount storage unit. The update unit updates the change amount storage unit with information indicating whether or not both of the motion vector and the difference coefficient included in the encoded information are zero,
The update necessity determination means refers to the change amount storage means, and when the motion vector is not zero, or when the difference coefficient is not zero, there has been a content change of a predetermined amount or more from the previous display update. The image processing apparatus according to claim 2, wherein the determination is performed.   4. The image according to claim 3, wherein, when the block is an intra-coded macro block, the update unit sets information indicating that a motion vector or a difference coefficient is not zero in the change amount storage unit corresponding to the block. Processing equipment. The update unit updates the change amount storage unit by accumulating the code amount included in the encoding information for each block,
The image processing apparatus according to claim 2, wherein the update necessity determination unit determines that a content change of a predetermined amount or more has occurred since the previous display update when the accumulated code amount is equal to or greater than a first threshold.   The image processing apparatus according to claim 5, wherein when the block is an intra-coded macro block, the update unit sets a value equal to or greater than the first threshold value in the change amount storage unit corresponding to the block. The update unit updates the change amount storage unit by accumulating a value obtained by adding a predetermined coefficient to the code amount of the difference coefficient included in the encoding information for each block,
The image processing apparatus according to claim 2, wherein the update necessity determination unit determines that a content change of a predetermined amount or more has occurred since the previous display update when the accumulated value is equal to or greater than a second threshold value.   8. The image processing apparatus according to claim 7, wherein when the block is an intra-coded macro block, the update unit sets a value equal to or greater than the second threshold value in the change amount storage unit corresponding to the block.   The image processing apparatus according to claim 7, wherein the predetermined coefficient is a quantization parameter value of a block.   The image processing apparatus according to claim 7, wherein the predetermined coefficient is a power of 2 (quantization parameter value of block ÷ 6).   The update necessity determination means initializes the contents of the change amount storage means corresponding to a block determined to have changed by a predetermined amount or more since the last display update. Image processing apparatus. An image processing device for decoding and displaying an encoded image,
Decoding means for decoding the encoded image in units of blocks to generate image data and encoding information;
Prediction frame storage means for storing the image data generated by the decoding means as a prediction frame;
Display frame storage means for storing display frames;
Based on the encoded information generated by the decoding means, it is determined for each block whether there has been a content change of a predetermined amount or more from the previous display update, and there has been a content change of a predetermined amount or more from the previous display update. Determination means for determining whether or not the number of blocks is equal to or greater than a predetermined number;
When it is determined by the determination means that the number of blocks whose content has changed by a predetermined amount or more is a predetermined number or more, all blocks are transferred from the prediction frame storage means to the display frame storage means. An image processing apparatus comprising: a display update unit that updates and otherwise does not update the display frame storage unit. The predicted frame storage means includes two predicted frame memories in which image data of consecutive frames are alternately stored,
The image processing apparatus further does not update the block of the third frame in the prediction frame memory when the contents of the corresponding blocks in the three consecutive frames are all the same, and the other blocks The image processing apparatus according to claim 1, further comprising means for performing an update.

Images