JP5156655B2 - Image processing device - Google Patents

Description

  The present invention relates to an image processing apparatus, and more particularly to a technique for enabling low-power consumption and low-cost and simultaneously enabling high-quality image processing.

  In recent years, the amount of data processing in a signal processing system including an object recognition display device tends to increase as the resolution and accuracy of images and sounds increase. For example, with respect to images, pixel data for horizontal 1920 pixels × vertical 1080 lines per frame exists due to the recent spread of HD (High Definition) image quality, and the conventional NTSC level data (720 pixels × 480 lines) exists. In comparison, the processing amount is about 6 times. Furthermore, in the frame double speed technology for improving the image quality of the liquid crystal display, since the frame conversion processing is performed to 120 fields / SEC, what has been processed at 60 fields / SEC so far, the signal conversion is performed accordingly. The processing amount and the data storage amount in the middle tend to increase. In addition, in the processing related to audio, in order to create a surround feeling from the conventional two stereo channels, the subwoofer channel is added to the 5.1 channel and the 7 channel. The amount is getting bigger.

  On the other hand, in the field of home appliances where system cost is an important factor, signal processing is often performed by combining a signal processing LSI and a data storage memory. H. H.264 and MPEG2 image compression techniques, gamma (γ) correction, image scaling, so-called baseband image processing such as OSD superposition, and audio compression / decompression processing and surround mixing processing There is a tendency to reduce the number of memory LSIs and reduce the system cost by processing with a shared SDRAM or the like. Further, by reducing the number of memory LSIs, the number of address lines and data lines required for connection to the memory of the signal processing LSI to be connected can be saved, thereby reducing the system LSI package cost and chip. The area can be reduced, and the LSI price can also be reduced.

In such a system configuration, the increase in the amount of image and sound signal processing described above leads to an increase in memory access bandwidth and power consumption. In order to deal with these problems, for example, Patent Document 1 below discloses a technique that compresses and decompresses the data amount on the memory side when accessing the memory. According to this, when accessing a memory that performs compression / decompression, the data bandwidth can be effectively reduced, however, there is no mention of guaranteeing the minimum value. When actually designing a real-time system, a real-time system that does not cause processing failure without a guaranteed value such as the minimum guaranteed bandwidth and the maximum required memory storage capacity. It becomes extremely difficult to realize. On the other hand, when processing such as compressing / decompressing the amount of data, it is possible to increase memory efficiency and reduce power consumption by reducing the amount of data after compression of the original data. However, in that case, there is a problem that image quality deterioration due to image information deficiency becomes remarkable. In particular, there is a problem that it is difficult to easily apply when high image quality is required.
JP 2001-86460 A

  In an image processing apparatus that performs image processing and outputs, in order to achieve high image quality, it is necessary to perform some processing for high image quality, which is accompanied by an increase in power consumption. . On the other hand, in battery-powered equipment, an increase in power consumption leads to a reduction in the operating time of the apparatus, which impairs user convenience. Accordingly, high image quality and low power consumption are in a trade-off relationship with each other, and the present invention eliminates the trade-off between the high image quality and low power consumption in consideration of user convenience. This is the issue.

  That is, an object of the present invention is to provide an image processing apparatus capable of solving the above-described problems in the prior art, and in particular, the entire apparatus including the amount of memory to be stored and the memory bandwidth is high. A balance between image quality processing and low power consumption, comprehensively resolved including real-time operation, image deterioration, and power consumption reduction, and adaptive according to the situation in which the device is used Another object of the present invention is to provide an image processing apparatus capable of controlling the image processing operation.

  That is, in order to solve the above-described problem, in the present invention, first, a memory in which compressed intermediate information is recorded, an external memory, a signal processing unit that performs signal processing and output of image information, and the signal processing unit When storing the intermediate information in the external memory, the data compression means for compressing the data amount of the data, and when the signal processing means reads the compressed intermediate information from the memory, the data is expanded. An image processing apparatus comprising a data decompression unit, wherein the data compression unit is further provided with a unit for defining a worst compression rate when the data compression efficiency is the worst.

  Further, according to the present invention, in order to solve the above-described problem, a signal processing unit that performs signal processing on image information and an image output unit that outputs image information output by the signal processing unit to a display device are provided. An image processing apparatus is provided, wherein the signal processing unit further includes means for reducing power consumption when the importance of the image quality output from the image output unit is low. .

  As described above, according to the present invention, it is possible to provide an image processing apparatus that can flexibly cope with both power consumption and high image quality according to the purpose of image processing. Even achieve an excellent effect.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

  FIG. 1 is a block diagram of an image processing apparatus according to the first embodiment of the present invention. In this circuit, image data having a frame resolution of 1920 × 1080 pixels is input at a throughput of 60 frames per second, and after performing signal processing while storing intermediate data in an external memory, This circuit outputs data. Reference numeral 1 in the figure denotes a data processing circuit. The data processing circuit 1 further includes an information processing circuit 2, a host processor (Host Processor) 20, a data compression circuit unit (Encode) 13, A data decompression processing unit (Decode) 17 and a memory interface unit (Memory I / F) 14 are provided. Reference numeral 11 in the figure indicates an image input terminal of the data processing circuit 1, and reference numeral 19 indicates a data output terminal. Reference numeral 15 in the figure denotes an external memory terminal of the data processing circuit 1, and is connected to an external memory (SDRAM) via the terminal. The information processing circuit 2 includes a first half processing unit (Image Signal Processor) 12 and a second half processing unit (Image Signal Processor) 18 therein.

  Next, the operation of the image processing apparatus whose circuit configuration has been described above, particularly the data processing circuit 1 whose configuration has been described above, will be described with reference to FIGS. First, data input from the input terminal 11 of the data processing circuit 1 is input for each pixel in the scanning line direction. For example, the luminance and color difference data are input to the input terminal 11 as 8-bit quantized data, respectively, and horizontal and vertical synchronizations are input simultaneously with the pixel clock for synchronization. Here, for example, REC. As represented by 656, an interface that decodes a synchronization signal portion from data input together with a specific synchronization code (SAV, EAV) and extracts a desired valid period can be cited as a typical example.

  Next, the data input to the input terminal 11 is then input to the signal processing unit 2. Here, for example, in the first half 12 of the signal processing unit 2, processing for removing image noise and data collection of luminance distribution (histogram) in the frame are performed, and in the second half 18, the collected data is collected. Based on this, gamma (γ) correction is performed. That is, the data input from the input terminal 11 is subjected to noise removal processing and totalization of the luminance distribution in the frame in the first half processing unit 12 of the signal processing unit 2, and sequentially in accordance with the scanning line direction, the data compression unit ( Encode) 13.

  On the other hand, the data compression unit (Encode) 13 performs data compression on the transmitted data by using, for example, coding compression used in so-called image coding theory such as DCT processing, quantization processing, and Wavelet transform. I do. At this time, when performing the above-described encoding compression, the host processor 20 sets the compression rate (the amount of data after compression / the amount of data before compression) when the compression efficiency is reduced to the upper limit value (worst value: (Worst Ratio). Then, referring to the upper limit value, the data compression unit 13 performs compression so that the compression rate is equal to or lower than the upper limit value (worst value) (≦ Worst Ratio).

  FIG. 2 attached here shows an example of a compression method in the data compression unit (Encode) 13. In this example, every time eight pixel samples are input, a one-dimensional DCT (Discrete Cosine Transform) is performed, and a process of decomposing into eight-order frequency components is performed. In the example of FIG. 2, the processing for the Y component among the Y, Cb, and Cr components is shown. In the graph 21 shown in FIG. 2, the horizontal axis represents the position (pixel position) of the pixel sample, and the vertical axis represents the pixel value of each pixel. In the figure, the hatched portion indicates each pixel value (from the left, “130”, “35”, “31”, “5”, “7”, “3”, “2”, “6”). This indicates the number of bits necessary for expression (more specifically, the range below the most significant bit where “1” stands when expressing a pixel value). In order to express an arbitrary pixel value, 8 × 8 = 64 bits are required for 8 pixels.

  After the 8-pixel sample is subjected to DCT processing, it is expressed by an 8th-order frequency component (horizontal axis: frequency dimension) as shown in the graph 22 of FIG. As is apparent from this graph, each component is expressed as data of 12 bits at maximum. Therefore, at this stage, in order to express a DCT coefficient for an arbitrary pixel, a bit number of 12 bits × 8 = 96 bits is required.

  Then, for example, as shown in the graph 23 of FIG. 2, each component is, for example, “4”, “16”, “32”, “32”, “64”, “128”, “256” sequentially from the left. , Each component after quantization is expressed by (each pixel value / quantization step). On the other hand, when truncation to an integer value is performed, the data indicated by the shaded portion in the graph 21 is converted into the bit area of the shaded portion in the graph 23 after DCT conversion. As shown in the graph 24 of FIG. 2, the number of bits necessary to represent the arbitrary value of each component is specifically 10 bits, 8 bits, 7 bits for each component in order from the left of the figure. 7 bits, 6 bits, 6 bits, 5 bits, 4 bits, 4 bits are required, and 51 bits are required in total. In other words, if there is 51 bits of information, it is possible to sufficiently express any value of the combination of the quantization steps, and other bit information can be reduced. Become. Therefore, if the data compression unit (Encode) 13 transmits only the bit amount of 51 bits, the data amount is always reduced by the compression rate (after compression / original data) = 79.7%. It is possible to do.

  Further, by adding a bit indicating the presence / absence of data for each component and performing processing such as not transmitting data for the component of “0” data, overhead of the added bits (= 8 bits) is added. (51 + 8) /64=92.1%, which causes the worst compression ratio to deteriorate. For example, in the above data, a flag (specifically, a flag indicating that data exists only in three components from the left end (specifically, (The data “1110000” in binary number) may be transmitted, and the total data amount is (10 + 8 + 7) bit + 8 bits = 33 bits. In this case, 33/64 = 51.5%, which is more efficient compression. It can be realized.

  In the above-described method, the amount of data after compression depends on the original image. However, the worst compression ratio (Worst Ratio) can be guaranteed as 92.1%. In the above sequence of quantization steps, for example, if the quantization step value is further increased, the worst compression rate is reduced, that is, the compression efficiency is increased. growing. On the other hand, when the quantization step value is decreased, the worst compression rate increases, that is, the compression efficiency decreases, but on the other hand, the degradation of image quality decreases. Therefore, for example, a plurality of types of quantization step sequences are prepared in the data compression unit (Encode) 13, and the quantization step sequence is changed according to the worst compression ratio (Worst Ratio) instruction from the host processor 20. Process to select uniquely. According to this, in accordance with the worst compression rate instruction from the host processor 20, it is possible to perform data compression suitable for the instruction. Then, the compressed data is arbitrated by the memory I / F 14 so as to access the data in a time division manner with other memory accesses (for example, data reading of a data decompression unit to be described later), and the external memory terminal 15 Are stored in a memory area in the external memory 16.

  On the other hand, the data stored in the memory area in the external memory 16 is read by the data decompression unit (Decode) 17 through the memory I / F 14 and compressed by the data compression unit (Encode) 13. The data decompression process is performed assuming the encoding rules. At this time, the same worst compression rate (Worst Ratio) as instructed to the data compression unit (Encode) 13 is instructed from the host processor 20 to the data decompression (Decode) unit 17. Accordingly, the data decompression unit (Decode) 17 selects the same quantization step table as that selected by the data compression unit (Encode) 13. Then, the bit allocation amount of each component corresponding to the selected quantization step can be uniquely determined as in the compression, and the data after quantization of each frequency component is acquired based on the rule. . As described above, even when “0” data is not sent, it is possible to identify the position where the data exists by first decoding the data. And defined as data after quantization. Thereafter, each component is inversely quantized using the determined quantization step (data after quantization × quantization step of each component), so that the DCT coefficients before quantization can be reproduced. . Thereafter, by performing IDCT processing for every eight pixels, it is possible to restore a pixel sample corresponding to the original pixel data. By repeating this in units of 8 pixels, all data can be expanded and output.

  The data expanded as described above is then transferred to the latter half processing unit 18 of the signal processing unit 2. The latter half processing unit 18 performs gamma (γ) correction processing according to the histogram information separately transferred from the first half processing unit 13 to make the luminance level in the frame visually easy to view. Similarly to the input, data is output in the scanning line direction.

  In the above description, the quantization processing after DCT is specifically described as an example. However, according to the algorithm for generating the data amount corresponding to the designated worst compression rate, the intended effect of the present invention is described. Is obtained. For example, ADPCM can be performed instead of the DCT process. In this ADPCM, for example, even if the difference data between each adjacent pixel is calculated and then quantized using the quantization step selected based on the worst compression rate for each difference data, the same as above An effect is obtained. Further, a bit shift process may be used instead of the quantization process.

  For example, when the worst compression rate is defined as 50%, 8-bit ADPCM data is bit-shifted to the LSB side up to 4 bits. Thereby, the data amount is compressed to 50%. In order to reduce image quality degradation, for example, by indicating the position of a difference value having data of 4 bits or more with a flag and performing bit shift only on that portion, the data of the position where the difference value is small It is possible to prevent deterioration. In this case, an overhead of 8 bits is generated for every 8 pixels. However, the worst compression rate may be calculated in consideration of the overhead. Also, when the data is bit-shifted, the bit-shifted data is derived from the worst compression rate, the bit shift amount is derived, the data is expanded by the number of bits after the bit shift, and each data is shifted upward by a predetermined number of bits. Thereafter, by adding each adjacent data, the data can be expanded to the original state. When a negative value is output during data processing such as DCT conversion or ADPCM described above, the bit is divided into a sign bit and an absolute value, and the encoded bit is rearranged on the LSB side, so that the upper bit is obtained. By avoiding the concentration of 1 and performing quantization processing and bit shift processing in the same manner as described above, the compression processing is not hindered.

  Next, processing when data is transferred from the data compression (Encode) unit 13 to the memory I / F 14 will be described in detail with reference to FIG. In this description, it is assumed that the access to the external memory (DRAM) 16 is 2048-bit transfer as the basic burst access unit. That is, the data input to the data compression unit (Encode) 13 is assumed to contain data of 8 bits × 256 pixels, as indicated by reference numerals 30 to 33 in the figure.

  First, data compression processing is performed for each 256 pixels. In this case, it is assumed that the worst compression rate is defined as 75%. At this time, the data for each 256 pixel is always compressed to a compression rate of 75% or less of the original data, as indicated by reference numerals 41 to 44 in the figure. In the example of FIG. 3, the states compressed to 50%, 43%, 75%, and 25% are illustrated. These compressed data are arranged in a bit unit in the data compression (Encode) unit 13 and are output to the memory I / F 14 when the data amount exceeds 8192 bits, which is a burst unit. . In the example of FIG. 3, the data denoted by reference numerals 41 to 44 can be output as two 8192-bit data 46 and 47, that is, within two bursts. Conversely, at the time of data decompression, each burst is read sequentially, data 41 to 44 is decompressed from the data string of data 46 and 47, and further decompression processing is performed, so that the number of pixels corresponding to original data 30 to 33 Can be restored.

  Further, FIG. 4 attached will describe an example of utilization of the memory in the external memory (SDRAM) indicated by reference numeral 16 in FIG. For example, an area in which data processed by the first half processing unit 12 of the signal processing unit 2 in FIG. 1 is written and an area read by the second half processing unit 18 are processed in two frames, and buffered every frame period. Is assumed to be switched. Specifically, for example, during the period in which the first half processing unit 12 writes to “buf0” shown in “Uncompressed” on the left side of FIG. 4, the second half processing unit 18 reads data from “buf1”, and the next frame In the period, it is assumed that the buffer is exchanged and the toggle processing is repeated. At that time, when data compression is not performed, when recording a 1920 (horizontal) × 1080 (vertical) HD image having Y, U, and V components in a ratio of 4: 2: 0, A data area of about 4.15 Mbytes is required.

  On the other hand, if the quantization step is defined or the bit shift amount is defined so that the worst compression rate is 50% and the data is compressed by the above-described mechanism, The maximum data amount can be 4.15 Mbyte × 1/2 = 2.07 Mbyte even at the maximum. Here, in the case of “worst compression rate 50% compression” shown in the central part of FIG. 4, an example is shown in which data is stored in the lower order for each buffer in the same memory map as in the uncompressed state. If the areas starting from “adr0” and “adr2” are sequentially accessed, data expansion can be performed even when the compression ratio changes for each image. Thus, the data access amount to each buffer is reduced by the amount of data compression, and the data bandwidth, which is the access amount per unit time, is also reduced corresponding to the compression rate.

  Furthermore, since the worst compression rate is defined as described above, the data is equal to or lower than the worst compression rate, as shown in “Worst compression rate 50% compression, data packed and arranged” on the left side of FIG. By utilizing this, it is possible to delete the surplus data area by arranging “buf0” and “buf1”.

  As mentioned above, based on the worst compression rate, estimate the total amount of memory to be stored and the total system bandwidth of the memory bandwidth, and design the entire system without breaking real-time processing, that is, ensuring real-time processing. Each operation can be specified. In addition, by specifying the worst value of the compression ratio, it becomes possible to more accurately estimate the power consumption and bandwidth consumption required for data retention in the memory. In the case of driving, it is possible to further improve the accuracy of estimation of the driving time. That is, according to the present embodiment, it is possible to adaptively control the data compression rate while guaranteeing real-time operation.

  Next, an image processing apparatus according to a second embodiment of the present invention, particularly, an image processing operation in the apparatus will be described with reference to FIG. In the second embodiment, the processing block is the same as the block diagram of FIG. 1 described above. However, the data compression (Encode) unit indicated by reference numeral 13 uses a lossless compression algorithm. Also in this embodiment, as in the first embodiment, data compression is performed every time 8-bit × 256 pixel data is input.

  At that time, for example, a difference value between adjacent pixels of each pixel is taken as in ADPCM, and Huffman coding is performed. From Shannon's theorem, theoretically, when lossless compression coding is performed, there is no method that always realizes compression of less than 100% for any data. That is, depending on the data, for example, depending on the compression encoding based on a certain probability, there is a possibility that 120% and 130% may be larger than the original data amount. Therefore, in this embodiment, when the compression rate is 100% or less for every 8 pixels, the ADPCM and Huffman coding are performed to compress the data, and when the compression rate is 100% or more, The original data is transferred as it is to keep the increase rate to a minimum. However, to indicate whether the compressed data string is compressed data or original data. A 1-bit identification flag is added (for example, 0 = compressed data, 1 = original data).

  Thereafter, the data is packed and arranged, and the data is transmitted to the memory interface 14 every 2048 bits as in the first embodiment. In the example of FIG. 5, reference numeral 50 corresponds to an identification bit, and 51, 52, and 54 can be compressed to 50%, 43%, and 25%, respectively. Shows the case. As for the data 53, since the compression result is larger than 100%, the identification flag “1” is added and the original data is transmitted. The data decompression unit (Decode) 17 first analyzes the identification flag every time when each data is received. If the flag is “0”, the Huffman coding is solved, and then ADPCM inverse processing is performed. I do. On the other hand, when the identification flag is “1”, it is handled as original data.

  According to this method, the overhead increases by the amount of data of the identification flag described above, but the memory capacity and bandwidth surely allow for data of 2049 bits × burst, so that the worst compression rate can be obtained. It is possible to specify the behavior of Furthermore, when handling image data, it is possible to reduce the effective bandwidth and memory capacity by using a compression algorithm that uses a probabilistic model, and image quality is particularly important. In such applications, this method also has the advantage of not causing image quality degradation. That is, the present embodiment particularly relates to a system using a reversible algorithm, and even when the reversible algorithm is used, it is possible to avoid a significant deterioration in the worst-case data compression rate.

  Next, attached FIG. 6 is a diagram for explaining an image processing apparatus according to the third embodiment of the present invention, and in particular, a method for storing data in a memory in such an apparatus. The image processing apparatus of the present embodiment also has the same configuration as the block diagram of FIG. 1, except that when data is stored in the external memory (SDRAM) 16, the data compression is performed in the image data area. For each area divided by the worst compression ratio (Worst Ratio) × n bursts (n is an integer) instructed by the (Encode) unit 13, the amount of data that was data for n bursts before compression is stored.

  In FIG. 6, for example, the area is divided every four bursts (n = 4) of original data before compression. As described in FIG. 3, the data that was 8192 bits before compression is converted into 3953 bits. When compressed, it is stored in the area of the column address (000 to 0EO (hexadecimal)) with the row address (00 (hexadecimal)). Next, assuming that 5632-bit data is compressed and stored, the row address (00 (hexadecimal)) is used instead of “0E0”, and the column address (180-2EO (hexadecimal)) is stored. Skipped to the area. Hereinafter, similarly, for each data segmented by 8192 bits, data corresponding to four bursts before compression is arranged.

  According to this method, if the data amount before compression is Mbyte, the required data amount is M (byte) × worst compression rate (%) data, and the required memory is reduced while reducing the memory storage amount. It is possible to calculate. In addition, for example, when performing random access to a memory required for partial cut-out display of image data, the data storage position can be calculated with an accuracy of n burst units.

  In addition, when all data is arranged in the memory without using this method, even if the data is needed from the middle, the position of the necessary data cannot be specified unless the compression encoding is solved. . On the other hand, in this method, at the time of random access, necessary information is read by address calculation for the necessary area with an accuracy of n burst units, and is decompressed by the data decompression unit 17 without depending on the previous data. Is possible. Therefore, it is possible to reduce the data read access from the memory I / F 14 when the GUI, graphics, and image data are enlarged / reduced, and it is possible to reduce the bandwidth. That is, this embodiment relates to storage of data in a memory. In particular, when data compression processing is used, it is possible to avoid random access in the memory to increase the amount of data access and to reduce the GUI. Enable the system to utilize data compression processing.

  FIG. 7 attached herewith is a diagram showing a block configuration of an image processing apparatus according to the fourth embodiment of the present invention. In the image processing apparatus of this embodiment, the above-described image processing apparatus of the present invention is applied to an image processing system including a display display device, and a video bit stream (video HD image quality) stored in the memory IC 200 in advance. (1920 × 1080 60i recorded in MPEG2 Video (ISO / IEC13818-2)) is reproduced and displayed on a small LCD 219 in the apparatus or a large display 224 connected externally.

  Hereinafter, the configuration of the image processing apparatus according to the fourth embodiment shown in the drawing will be described using symbols in the drawing. First, a pre-recorded video stream is stored in the memory IC 200. When a user I / F circuit (not shown) instructs the apparatus to reproduce a video stream, data is first read by the card I / F unit 201. The read data includes data management information in addition to the video stream. The read video stream is subjected to system decoding processing in the Demux block 204, packetized, and multiplexed with other audio data or the like, from the next video decoder (Decode) 205. Are processed into a decodable stream format. For example, in the case of conforming to the MPEG2 Systems standard (ISO / IEC13818-1), headers are analyzed and removed from a stream that has been made into a transport stream (TS) packet, and an MPEG2 elementary stream (ES) level is obtained. Extract data. The extracted ES data (video ES) is once stored in the large-capacity memory (SDRAM) 203 via the memory I / F 202 and read again by the video decoder (Decode) 205.

  The data read by the video decoder (Decode) 205 described above is decoded in accordance with the MPEG2 Video standard, and the decoded image is stored in the memory (SDRAM) 203 via the memory I / F 202. Next, the decoded image is sequentially transferred to so-called image processing blocks such as a noise removal block 207, an edge enhancement block 209, and a gamma correction block 211, and each block performs noise removal processing and edge portion enhancement processing, respectively. Color gradation adjustment processing by gamma correction is performed.

  In this case, when each data process accesses the memory unit 203, the data compression / decompression process as shown in the first and second embodiments is performed, and the bandwidth is compressed. When performing these processes, particularly when image quality is important, for example, when decoding is temporarily stopped and displayed on a still screen, when outputting an image to a high-definition TV, Further, as will be described later, when an external high-definition display is connected and output, the host CPU 223 performs the worst bandwidth compression ratio (Worst Ratio) for each of the data compression / decompression blocks 206, 208, and 210. By performing the setting for raising the image quality, it is possible to perform processing giving priority to the image quality. Conversely, when the device is in a battery-powered state and it is desired to reduce its power consumption, the amount of access data to the memory (SDRAM) 203 via the memory I / F 202 is set by reducing the worst compression rate. Therefore, power consumption can be reduced.

  Further, consider a case where content with a low frame rate is decoded and displayed. For example, a content of 1280 × 720 × 30p exists in the memory card, and after decoding, the frame rate conversion unit 213 performs frame rate conversion. Here, by performing frame rate conversion on 30p (30 Hz progressive image) and outputting it as a 60p (60 Hz progressive image) image, it is possible to produce a smooth motion. Also in this case, in the FRC, when 30-Hz data is read and converted to 60 Hz and output, the bandwidth compression processing is performed by the data compression / decompression 214 block, so that power consumption can be reduced.

  As described above, the worst compression ratio (Worst Ratio) can be controlled from the host CPU 223 in accordance with the importance of the output image. The image after the gamma correction or the frame rate conversion is normally decompressed by the data decompression unit 216, and then transferred to the LCD driver 218 by the scaler 217 according to the display size. It is output to the LCD 219. For example, when a 1920 × 1080 image is output to a WVGA size LCD, the horizontal and vertical are reduced, converted to a size of 800 × 480 pixels, and output.

  On the other hand, when outputting to a so-called large display 224 such as an externally connected plasma TV or a liquid crystal large display, the image data is unencoded for bandwidth compression in the data decompression 220, The scaler 221 converts the resolution to the size of the external display, and then outputs it to the external large display 224 via the HDMI interface circuit 222. Furthermore, OSD (on-screen display) processing added at the time of image output is performed by the OSD 215. That is, the image to be output is read from the memory (SDRAM) 203, and the graphics data to be combined is also read from the memory (SDRAM) 203, and α blending is performed again, and the memory (SD (SDRAM) 203. At this time, similarly to the other image data described above, the data compression / decompression unit 214 can perform bandwidth compression on both the OSD data and the decoded image.

  As described above, by performing data compression / decompression in the memory access data path related to each image processing, it is possible to reduce the amount of data access related to the image processing, thereby reducing the power consumption of the circuit. Is possible. Further, by controlling the worst compression ratio (Worst Ratio) according to the importance of the image, it is possible to balance image quality and data access amount reduction.

  For example, when a 60i image (4: 2: 2) image is read / written, 1920 × 1080 × 8 bits × 30 × 2 = approximately 995 Mbps data access occurs. Further, when no compression is performed, memory access (via data compression / decompression blocks 206, 208, 210, 214, 216, and 220) from noise removal processing to LCD output and further output to the HDMI-I / F 222 is performed. A bandwidth of 9 × 995 Mbps = 8995 Mbps is required for the code image data and the frame rate conversion unit. For example, in the case where the memory I / F 202 is a 64-bit data bus, 8995/64 = 140 MHz is required even when the data is only the number of valid access clocks.

  On the other hand, however, if the worst compression ratio (Worst Ratio) is set to 65%, the number of clocks of 140−140 × 0.65 = about 49 MHz can be reduced. Therefore, in this case, after the host CPU 223 designates the worst compression ratio (Worst Ratio) to each data compression / decompression unit (specifically, the data compression / decompression blocks 206, 208, 210, 214, 216, 220), By reducing the operation clock supplied to the memory I / F unit 203 by 49 MHz in the clock generation circuit 225, the amount of electricity consumed by the flip-flop or the like with the constant clock supply can be greatly reduced. Is possible.

  Similarly, if the memory (SDRAM) 203 is a 32-bit DDR-SDRAM, the clock consumption for 49 MHz can be reduced, so that the supplied clock can also be reduced. As a result, the power consumed by the I / O terminal with the memory (SDRAM) 203 can be reduced, and the power consumption reduction effect obtained by the bandwidth compression circuit of the present invention is great. In addition, the worst compression rate has the advantage of guaranteeing a bandwidth that does not cause real-time data processing to fail. In addition to the above, when the above bandwidth compression processing is performed in OSD processing and frame rate processing, the power reduction effect can be further increased.

  In this embodiment, an external display (that is, a large display 224) is connected via HDMI. However, in the HDMI standard, as described in CEA-861E, the number of display pixels and the image size are as follows. As EDID (Extended Display Identification Data), a means for outputting to a signal source is provided. In this embodiment, the HDMI-IF block 222 receives this EDID information and notifies the host CPU 223 of the number of pixels and the image size. The correspondence between the notified information and the worst bandwidth compression ratio (Worst Ratio) is set in advance, and control is performed so that the worst compression ratio approaches 100% as the image size or the number of pixels increases. By doing so, it is possible to reduce the power consumption while reducing the perceived change in the image. That is, it is possible to achieve reduction in power consumption and automation according to the connection status of the external display.

  In this embodiment, a system for reading and decoding a stream from the memory IC 200 has been described. However, the present invention is not limited to this. For example, the input of a stream can be received via broadcast reception or via a network. It is apparent that the same effects as those described in the present invention can be obtained even when input to the image processing system described in the present invention and decoding.

  In the present embodiment, the decoded image generated by the video decoder (Decode) 205 is stored in the memory (SDRAM) 203, and when this is read, the bandwidth compression / decompression processing is not performed. However, on this data path, for example, a data compression / decompression processing unit may be inserted between the decode (Decode) 205 and the memory I / F 202 to perform the bandwidth compression processing. In this case, when the decoded image is changed from the original picture by data compression, an error is accumulated by referring to the image. Therefore, there is a concern that the image is disturbed as decoding progresses. Therefore, in this case, for those that can be referred from other frames such as I picture and P picture, by using lossless compression, the expanded data is prevented from changing, so that such a concern can be avoided. It can be avoided. In addition, for pictures that are not used as other reference images, such as B pictures, compression effects may be obtained using lossy compression. For the above-described I picture and P picture, lossy compression may be used as long as image quality deterioration is allowed.

  Note that when accessing reference image data by motion compensation, which is used in the moving image compression standard, a reference image region may be randomly accessed according to a motion vector. Even in such a case, when the bandwidth compression processing is performed and stored in the memory as in the above-described second embodiment, an address is arranged for each data segment defined by the worst compression rate, and the compressed data is randomly selected. It is possible to perform access, thereby reducing the data access overhead during MC.

  Next, an image processing apparatus according to a fifth embodiment of the present invention will be described with reference to FIG. The image processing apparatus shown in FIG. 8 is a diagram showing the details of the video decoding unit 205 shown in FIG. 7. In this embodiment, the image processing apparatus will be described in the first to fourth embodiments. The video decoding (Decode) unit 205 has a function of performing the above-described image processing block configuration and further performing a video decoding reduction decoding process.

  Hereinafter, the decoding process of a stream having a resolution of 1920 × 1080 will be described with reference to FIG. The detailed processing according to the MPEG2 standard requires more complicated processing, but only the main data flow related to the present invention will be described below.

  The processing for obtaining the decoded image quality with high image quality is the decoding processing defined in normal MPEG2. In other words, the video ES signal to be decoded is supplied to the VLD block 410 from the previous stage, that is, the Demux block 204 shown in FIG. 7, and is processed. That is, the variable length encoded data is decoded to obtain quantization step information for each MB, DCT coefficients after quantization, DCTtype, and motion vector information. Among these pieces of information, the quantization step information is transmitted to the subsequent inverse quantization block 411, where it is inversely quantized according to the MPEG2 standard, and the DCT coefficients after each quantization are output.

  Next, the DCT coefficient is subjected to inverse DCT processing in the inverse DCT block 412 in accordance with DCType (such as FilledDCT / FrameDCT) specified in each MB, and image sample values (for I pictures, image sample values, P, For the B picture, difference information from the reference image) is obtained. Thereafter, with respect to the P and B pictures, the reference image is processed by an MC (Motion Competition) block 413 according to the motion vector information, and a decoded image is obtained. In this embodiment, a low power decoding process is provided.

  When the host CPU 223 instructs the decoding process to be in the low power mode, the VLD block outputs all the original pixels as shown in FIG. 9 when outputting each quantized DCT image sample. Instead of (showing the DCT component of 8 × 8 = 64 samples indicated by reference numeral 420 in the figure), some data shown in the low frequency region (4 × 4 = 16 samples low in the figure indicated by reference numeral 421) Only the frequency component) is transferred to the quantization block. In this quantization block, only 16 samples sent are subjected to inverse quantization according to the corresponding inverse quantization equations. Thereafter, the inverse DCT is performed only for 16 samples, and transferred to the MC block 413. In the MC block 413, 4 × 4 image size data is arranged in order for the I picture. As a result, data of horizontal 1/2 and vertical 1/2 image sizes (960 pixels × 540 pixels) is created.

  In the normal motion compensation process, the P and B pictures are obtained from the reference image by the motion vector (MV) for each MB (16 pixels × 16 pixels) of the current frame being decoded, as indicated by reference numeral 422. The data is read from the position that has been moved, and the pixel difference value for 16 × 16 pixels output from the inverse DCT unit is added to the reference image. However, in the low power decoding mode, the motion vector is halved, that is, 8 × 8 data is read from the decoded image. Thereafter, processing is performed using the pixel difference value indicated by reference numeral 421 and added to the decoded image. Since this processing thins out the number of samples to be processed at the stage of the DCT coefficient before quantization, the number of image processing data after that becomes ¼. Further, when it is desired to reduce the image size to 1/16, the processing amount can be further reduced by narrowing the area for performing the decoding process indicated by reference numeral 421 to 2 pixels × 2 pixels.

  Changes in processing time in each mode are shown in FIG. At the time of full decoding, for example, 1 frame for 1/30 seconds is decoded within each frame period using a Vsync signal periodically output at 1/30 Hz as a reference of start. Decoding is performed in the order of I0, B1, B2, P3, and B4 (representing I picture, B picture, B picture, P picture, and B picture, respectively).

  Here, as indicated by reference numeral 421 in FIG. 9, when the DCT coefficient is ¼ before dequantization (¼ size decoding), the processing period of each frame is approximately ¼. The processing is completed in the frame period. If the next frame decoding process is continued as it is, the memory for buffering the decoded image overflows. Therefore, the decoding process of the next frame is suspended until the Vsync signal which is the start of the next frame, Pauses the operating clock. In addition, control is performed so that the clock operates again with the next Vsync as a starting point. Similarly, when 1/16 size decoding is performed, each frame is processed in about 1/16 period of each frame (see reference numeral 427 in FIG. 10), and the clock is suspended until the subsequent Vsync. .

  As described above, the operation of the flip-flop in the decode block can be stopped by stopping the clock at the stage where the decode is started in synchronization with Vsync and the decode period is completed. In general, power consumed by an LSI is very much consumed by a flip-flop to which a system clock is supplied, regardless of whether or not signal processing is actually operating. Therefore, the power consumption can be greatly reduced by stopping the clock during the suspension period as in this embodiment. Further, as in this embodiment, the control of clock supply and stop can be simplified by concentrating the processing in a part of the period such as the first half of the Vsync period, which is also effective for reducing the cost of the circuit. It is. Further, since the throughput does not change from that at the time of full decoding, there is no need to change the decoded image area and other intermediate data areas, so that the operation mode can be changed while minimizing changes in system operation.

  On the other hand, during MC processing, a decoded image different from the original is referred to, and it is generally known that degradation of the image is larger than when the image is reduced after normal decoding. . For this reason, it is desirable to provide a means for switching the decoding processing method according to the importance of the image in the decoding processing accompanied with the size reduction. That is, when outputting only with the built-in small LCD 219, since the image is finally output with a data size of 800 × 480 pixels, even if data is generated using ¼ decoding of 960 × 540 pixels, Further, reduction processing is performed in the scaler 217. For this reason, even if a slight image error occurs in the reference image as compared with normal decoding, the level of visual recognition is small and the user is unlikely to feel uncomfortable.

  On the other hand, when output to an external display, if it has a resolution of 1920 × 1080 or higher, the scaler 221 performs an enlargement process on data of 960 × 540 pixels or less, and an image generated by performing 1/4 decoding The disturbance is emphasized and output. Therefore, when outputting to a small LCD, the host CPU 223 selects the 1/4 decode mode, and the operation of each block in the video decoder 205 so that the video decoder 205 operates in the 1/4 decode mode. Define When outputting to an external display, each block in the video decoder 205 is controlled to operate in the full decode mode. In general, when outputting only with a small LCD, there are many operations driven by a battery. On the other hand, when outputting to an external display, it is often easy to connect the device to an AC power source, such as in a house. Therefore, the decoding mode control is effective in providing a device that automatically selects an operation suitable for actual usability while balancing the image quality and the power consumption reduction efficiency.

  Next, switching control between full decoding and 1/4 decoding in this embodiment will be described with reference to FIG. During playback of a certain content, for example, when an external display that has not been connected is connected, the decoding mode is changed without stopping the playback of the video stream. At this time, the decoding mode is changed from 1/4 decoding to full decoding, but when the decoding mode is switched, data of different decoded image sizes exist together. For example, in full decoding after switching, a decoded image that should be referred to should have a resolution of 1920 × 1080, but a 960 × 480 image is stored as a reference image on the front side. In this case, the MC decoding process cannot correctly refer to the image position.

  Therefore, when switching occurs, as indicated by reference numeral 500 in FIG. 11, the switching processing waits until the switching portion of GOP (Group of Pictures), and in the switching portion, the reference from the full decoded image ahead (specifically, Continue the decoding process without performing P0 reference at the time of B2 decoding and P0 reference at the time of B3 decoding. Thereby, the mismatch of the reference image can be eliminated. Further, in the subsequent scaler 217, the image before P0 is reduced from the data of 960 × 480 to 800 × 480 (WVGA), and the image after B2 is processed from the data of 1920 × 960 to 800 × 480. Processing is performed to reduce the image. Further, at the time of displaying B2 and B3, the influence of the reference image being unidirectionally referenced may be prevented from being displayed as an image by outputting the data of P0 instead or displaying a blank screen. As a result, even if the decoding mode is changed during decoding, it is possible to output so as not to disturb the image. That is, according to the present embodiment, it is possible to reduce power consumption by adaptive control of decoding processing.

1 is a block configuration diagram of a data processing circuit in an image processing apparatus that is a first embodiment (first embodiment) of the present invention. FIG. It is a figure which shows an example of the bandwidth compression means in the said image processing apparatus (Example 1). It is a figure for demonstrating the principle of the bandwidth compression in the said image processing apparatus (Example 1). It is a figure explaining the memory storage amount at the time of the bandwidth compression in the said image processing apparatus (Example 1), especially a bandwidth reduction. It is the figure explaining the memory compression efficiency at the time of the lossless compression in the image processing apparatus which is another Example (Example 2) of this invention. It is the figure which showed the arrangement | positioning in the memory in the image processing apparatus which is the other Example (Example 3) of this invention. It is a block block diagram of the image processing circuit in the image processing apparatus which is the other Example (Example 4) of this invention. It is a figure explaining the decoding block in the image processing apparatus which becomes further another Example (Example 5) of this invention. It is a figure explaining the 1/4 decoding process in the said decoding block (Example 5). It is a figure explaining the processing time of 1/4 decoding in the said decoding block (Example 5), and 1/16 decoding process. It is a figure explaining the switching control of the decoding mode in the said decoding block (Example 5).

DESCRIPTION OF SYMBOLS 1 ... Data processing circuit, 2 ... Information processing circuit, 11 ... Image input terminal, 12 ... First half processing part in information processing circuit, 13 ... Data compression circuit part, 14 ... Memory interface part, 15 ... External memory terminal, 16 ... External memory, 17... Data decompression processing unit, 18... Second half processing unit in information processing circuit, 19... Data output terminal, 20 ... Host processor, 80 ... Image processing device, 200 ... Memory IC, 201 ... Card interface, 202 ... Memory interface, 203 ... Memory, 204 ... Demux block, 205 ... Video decoder, 206 ... Data compression / decompression unit, 207 ... Noise removal block, 208 ... Data compression / decompression unit, 209 ... Edge enhancement unit, 210 ... Data compression / decompression unit, 211: Gamma correction unit, 212: Data compression / decompression unit, 213: Frame rate conversion unit, 2 4. Data compression / decompression unit, 215 ... OSD processing unit, 216 ... Data decompression unit, 217 ... Scaler, 218 ... LCD driver, 219 ... Small LCD, 220 ... Data decompression unit, 221 ... Scaler, 222 ... HDMI interface, 223 ... Host CPU, 224... Large display, 225... Clock generation circuit, 410... VLD processing unit, 411 .. inverse quantization unit, 412 ... inverse DCT unit, 413.

Claims (16)

Signal processing means for signal processing and outputting image information;
Data compression means for compressing the amount of data output from the signal processing means;
An external memory for recording the data compressed in the data compressor as intermediate information,
And a data decompression means for decompressing the data of the intermediate information read from the external memory,
The data decompression means decompresses the data of the intermediate information when reading the intermediate information from the external memory,
The data compression means compresses the data amount of the intermediate information so that the intermediate information is stored in the external memory so as to be equal to or lower than the worst compression rate when the data compression efficiency is the worst. The data output from the signal processing means is input as a plurality of bursts, and the input data is converted into the number of burst accesses corresponding to the data amount equal to or less than the worst compression rate, and output.
An image processing apparatus according to claim 1, wherein when the compressed intermediate information is read from the external memory, a data position of the intermediate information before compression is specified for each small region by address calculation . The image processing apparatus according to claim 1, wherein the signal processing unit further includes an image decoding unit that performs a motion compensation process, and the data compression unit and the data decompression unit include: An image processing apparatus for performing data compression / decompression processing on a decoded image accessed to the external memory when performing motion compensation processing . 2. The image processing apparatus according to claim 1, wherein the signal processing means further comprises means for reading and expanding only a part of the intermediate information when reading the intermediate information. An image processing apparatus. 2. The image processing apparatus according to claim 1, wherein the data compression unit further includes a unit that performs a reversible compression process . 5. The image processing apparatus according to claim 4 , wherein, when the data compression unit determines that the data size is larger than the data before compression by the compression process, the execution of the compression is stopped, and An image processing apparatus that outputs a flag for identifying that compression is not performed . The image processing apparatus according to claim 1 , wherein the data compression unit performs irreversible compression processing . The image processing apparatus according to claim 1, further comprising an image output unit that outputs the image information output by the signal processing unit to a display device,
The signal processing means further comprises means for reducing power consumption when the importance of the image quality output from the image output unit is low . The image processing apparatus according to claim 7, further comprising attribute acquisition means for acquiring the attribute of the display device to be output from the display device to be output by communication, and the signal processing means includes: An image processing apparatus , wherein the degree of importance of the image quality is determined by an attribute acquired by the attribute acquisition unit. 8. The image processing apparatus according to claim 7 , wherein the signal processing means further includes decoding processing means for performing decoding processing of a video stream, and means for reducing the power consumption of the signal processing means. Is realized by reducing the number of processing samples during the decoding process of the video stream . 10. The image processing apparatus according to claim 9 , wherein the signal processing means further includes means for changing a reduction in the number of processing samples of the decoding processing means during execution of the decoding processing. apparatus. The image processing apparatus according to claim 7 , wherein the signal processing unit includes the margin period obtained by reducing a processing amount by reducing a processing amount of the image signal in the signal processing unit. An image processing apparatus comprising means for suspending a signal processing means, thereby reducing power consumption . 12. The image processing apparatus according to claim 11 , wherein the signal processing means further includes means for stopping a clock during a pause period, thereby reducing power consumption. Processing equipment. 12. The image processing apparatus according to claim 11 , wherein the signal processing unit concentrates the processing on a part of each frame period, so that the function is performed for another time until the next frame period. An image processing apparatus comprising a means for pausing . 8. The image processing apparatus according to claim 7 , wherein the image output unit further includes output means for a plurality of display devices, and the signal processing means further includes the plurality of display devices. An image processing apparatus comprising: means for controlling the means for reducing the power consumption in accordance with an output to a display apparatus having the highest importance of image quality . 8. The image processing apparatus according to claim 7 , wherein the means for reducing power consumption is realized by reducing an access amount for accessing the external memory by the data compression means. apparatus. 16. The image processing apparatus according to claim 15 , further comprising means for reducing the number of clocks to the external memory when the amount of access to the external memory is reduced by the data compression means. Therefore, an image processing apparatus characterized by reducing power consumption .

Images