JP4552400B2 - Image display device, image display method, and image display program - Google Patents

Description

  The present invention relates to a method for reducing the amount of image data to be displayed and transferring the image data to an image display unit in an image display device including a liquid crystal display panel or other image display unit.

  In recent years, display devices mounted on mobile terminal devices such as mobile phones and PDAs (Personal Digital Assistants) have become larger in screen size and higher resolution, and higher resolution image data with a larger number of pixels than in the past has become larger. It can be displayed on the screen.

  However, the high resolution image data corresponding to such a large screen display or high resolution display has a large amount of data. In data transfer to a display device, if the number of data lines (data lines) is increased, the number of clocks can be reduced, but there is a disadvantage that the power consumption increases accordingly.

  Further, a liquid crystal driver IC which is a semiconductor circuit has an upper limit on the number of operating clocks due to its nature. For example, when the number of data lines is 1, in order to transfer 60 frames per second of video data of 6 bits each for RGB, 6 bits for each RGB according to QCIF (Quarter Common Intermediate Format) with a resolution of 176 × 208 pixels Requires a dot clock of 39.5 MHz (176 × 208 × 18 × 60). On the other hand, when the number of data lines is 3, the dot clock is 13.2 MHz which is 1/3 of the above case. At this level, the read / write speed of the built-in circuit of the liquid crystal driver IC manufactured by the current semiconductor process, particularly the RAM, can be sufficiently handled.

  However, when the CIF (Common Intermediate Format) with a resolution of 352 × 416 pixels is used, in order to transfer moving picture data of 6 bits for each of RGB, that is, 18 bits for each pixel, a total of 18 frames per second, as described above, A dot clock of 158.1 MHz is required. In this case, even if the number of data lines is three, the dot clock becomes 52.7 MHz, which cannot be handled by the current RAM read / write speed. Further, when transferring high-quality 24-bit (full color: RGB each 8 bits) image data, the dot clock is 210 MHz, and even if the number of data lines is three, it becomes 70.3 MHz. Therefore, the current capability of the liquid crystal driver IC cannot cope.

  In addition, as described above, it is theoretically possible to reduce the dot clock by increasing the number of data lines, but the portable terminal device has a limited mounting area, and the signal input / output unit of the liquid crystal driver IC is It must be a small connector or small contact. Therefore, increasing the number of data lines may lead to a decrease in the reliability of the connector due to impact or temperature change when carrying the mobile terminal device. That is, there is a limit to the increase in the number of data lines.

  As a means for coping with such a problem, there is a method of reducing the amount of image data sent to the display device. As one of the methods, Patent Document 1 reports that the amount of data is reduced by performing color reduction processing on image data to be transferred. Specifically, a method is described in which an error diffusion method is used for the color reduction processing and the calculation is omitted for the lower bits. In addition, Japanese Patent Application Laid-Open No. 2005-228561 describes a technique of transmitting image data in which a plurality of images are reduced and combined to reduce the data amount.

  However, since the above-described data reduction method mainly uses irreversible image conversion, the same image as that before conversion is not displayed. That is, the image quality of the display image is degraded. Also, when processing for interpolating an image is performed at the time of display, extra processing time is required. Moreover, there is a limit to image quality improvement by the interpolation processing.

Japanese Patent Laid-Open No. 9-101771

Japanese Patent Laid-Open No. 2003-18401

  The present invention has been made to solve the above-described problems, and an object of the present invention is to provide image data to be displayed in a color space in an image display device including a liquid crystal display panel and other image display units. Display device, image display method, and image display capable of reducing the amount of data in consideration of the characteristics of the image, transferring the image to the image display device, and restoring and displaying an image close to the original image on the image display device To provide a program.

  In one aspect of the present invention, an image display device including an image processing unit and an image display unit includes an image data acquisition unit that acquires image data in RGB format to be displayed, and the acquired RGB image data in YUV format. YUV converting means for converting to image data, Y component image data obtained by YUV, and U / V component image data, image processing means for performing separate image processing for reducing the data amount, and separately Transfer means for transferring the image data subjected to the image processing to the image display section, and the previous image display section displays the transferred Y component image data and the U / V component image data. To do.

  The image display device described above can be various terminal devices such as a mobile phone and a portable terminal that include a liquid crystal display panel as an image display unit, for example. The image data acquisition unit acquires image data from an external server or the like, for example. This image data includes both still image data and moving image data. The acquired image data is converted from RGB format to YUV format data. In order to perform image processing that reduces the amount of data in consideration of the characteristics of the color space, a Y component (luminance component) and a U / V component (which are color difference components and indicate the U component and the V component). Divided and processed separately. The image data with the reduced data amount is transferred from the image processing unit to the image display unit and displayed. In this way, image processing is performed to reduce the amount of data in consideration of the characteristics of the color space in the image processing unit, and then the image data is transferred to the image display unit. The speed can be increased.

  In a similar aspect of the present invention, an image display method executed in an image display apparatus including an image display unit includes an image data acquisition step of acquiring RGB format image data to be displayed, and the acquired RGB format image data. A YUV conversion step for converting into image data in YUV format, a Y component image data obtained by YUV conversion, and an image processing step for performing separate image processing to reduce the amount of data to U / V component image data; A transfer step of transferring image data subjected to separate image processing from the image processing unit to the image display unit, and displaying the transferred image data of the Y component and the U / V component on the image display unit And a display process.

  In a similar aspect of the present invention, an image display program is executed in an image display device including an image processing unit and an image display unit, thereby acquiring image data acquisition means for acquiring image data in RGB format to be displayed. YUV conversion means for converting image data in RGB format into image data in YUV format, Y component image data obtained by YUV conversion, and U / V component image data, separate images for reducing the data amount The image processing unit functions as a transfer unit that transfers image data that has undergone separate image processing to the image display unit, and the transferred Y component and U / V component images The image display unit is caused to function as display means for displaying data.

  Also with these image display methods and image display programs, it is possible to transfer image data with a reduced data amount to the image display unit in consideration of the characteristics of the color space, as in the case of the image display device described above.

  In one aspect of the image display device, the image processing unit includes a color reduction processing unit that performs a color reduction process on the Y component image data, and a first quantization process that performs a quantization process on the U / V component image data. Means, and encoding processing means for encoding each of the Y component image data subjected to the color reduction processing and the U / V component image data subjected to the quantization processing.

  For image data divided into a Y component and a U / V component, the Y component is subjected to a color reduction process for reducing the data amount, and the U / V component is subjected to a first quantization process. Here, the Y component and the U / V component of the color in the visual recognition characteristics have different discrimination capabilities, and are thus processed separately. Thereby, even if the amount of data is reduced, an image without a sense of incongruity can be displayed. Next, an encoding process is performed on the image data subjected to such a process. Since this encoding process is a reversible conversion, the data before conversion can be restored later, and the data amount can be further reduced without degrading the quality of the displayed image.

  In another aspect of the image display device, the color reduction processing unit performs dither processing on the image data. In the dither process, the acquired image data can be processed to reduce the data amount while remaining close to the original image.

  In another aspect of the image display device, the color reduction processing unit performs error diffusion processing on the image data. The miscalculation diffusion processing can perform processing for reducing the amount of data with respect to the acquired image data while remaining closer to the original image than the dither method.

  In another aspect of the image display device, the first quantization processing unit increases a quantization step for image data having a gradation value smaller than a predetermined value, and the gradation value is larger than the predetermined value. The image data is quantized by reducing the quantization step. This is because, in visibility, a high saturation color has a lower discrimination ability than a low saturation color. Thereby, even if the amount of image data is reduced, it is possible to display an image without a sense of incongruity.

  In another aspect of the image display device, the encoding processing unit performs Huffman encoding processing on the Y component image data. Since the Huffman encoding process is also lossless conversion, it can be decoded and returned to the data before conversion, so that the quality of the display image is not deteriorated.

  In another aspect of the image display device, the encoding processing unit encodes only the image data between the maximum value and the minimum value of the gradation value for the U / V component image data. . Since the dynamic range of the gradation value of the U / V component is relatively small in the predetermined range of the image, the range determined from the maximum value and the minimum value of the gradation value, that is, the effective gradation range is small compared to the Y component. Therefore, the amount of data can be further reduced by encoding only the image data within the effective gradation range. Further, since this encoding process is a reversible conversion, it can be restored to the data before the conversion, so that a high-quality image can be displayed.

  In another aspect of the above image display device, if the image data of the U / V component after the encoding process is equal to or less than a predetermined number of bits, the Huffman encoding process is further performed. Thereby, the amount of data to be transferred can be further reduced.

  In another aspect of the image display device, the image display unit includes a decoding processing unit configured to decode the encoded Y component image data and the U / V component image data; The second quantization processing means for quantizing the U / V component image data subjected to the quantization processing, the Y component image data subjected to the decoding processing, and the second quantization means for quantization processing. Display means for displaying the image data of the U / V component. In the image display unit, processing for decoding mainly encoded data is performed. In this way, a high-quality image close to the original image can be displayed.

  In one aspect of the image display device, the second quantization processing unit performs quantization by using a reverse characteristic of the quantization characteristic of the first quantization unit. Thereby, it is possible to return to an image close to the original image.

  In another aspect of the image display device, the encoding processing unit performs encoding in block units of the image data, and the decoding processing unit performs the encoding in parallel with the encoding processing unit. The image data corresponding to the block previously performed by the means is decoded. Thereby, an encoding process and a decoding process can be performed efficiently.

  Preferred embodiments of the present invention will be described below with reference to the drawings.

[Configuration of mobile terminal device]
FIG. 1 shows a schematic configuration of a mobile terminal device according to an embodiment of the present invention. In FIG. 1, a mobile terminal device 210 is a terminal device such as a mobile phone or a PDA. The portable terminal device 210 includes a display device 212, a transmission / reception unit 214, a CPU 216, an input unit 218, a program ROM 220, and a RAM 224. The display device 212 includes a driver 226, a display panel 227, and the like.

  The transmission / reception unit 214 receives content such as image data from the outside. Hereinafter, data processing in the mobile terminal device 210 when image data is received will be mainly described. The image data is received, for example, when the user operates the portable terminal device 210 to connect to a server device that provides a content providing service and inputs an instruction to download desired image data. The received image data includes moving image data and still image data. Here, it is assumed that, for example, RGB 24-bit image data (8 bits for each color) is received per pixel. The image data S11 received by the transmission / reception unit 214 is supplied to the CPU 216 and the like, and can be further stored in the RAM 224.

  The input unit 218 can be composed of various operation buttons for a mobile phone, a tablet that detects contact with a touch pen for a PDA, and the like, and is used when a user performs various instructions and selections. . An instruction, selection, or the like input to the input unit 218 is converted into an electrical signal and sent to the CPU 216.

  The program ROM 220 stores various programs for executing various functions of the mobile terminal device 210. In particular, in the present embodiment, image processing is performed to reduce the data amount for the image data transmitted from the transmission / reception unit 214. An image processing program to be executed is stored. The image processing program according to the present embodiment will be described in detail below.

  The RAM 224 is used as a working memory when converting image data according to a program such as an image processing program. Further, as described above, image data from the outside received by the transmission / reception unit 214 or image data acquired from a camera (not shown) provided in the mobile terminal device 210 can be stored as necessary.

  Next, image processing by the CPU 216 will be described. First, an outline of image processing according to the present embodiment will be briefly described. The CPU 216 performs image processing for the purpose of reducing the amount of data transferred to the display device 212. First, the CPU 216 converts the acquired RGB format data into YUV format data (also referred to as YUV conversion), and performs image processing on the YUV format data in subsequent processing. In the present embodiment, separate processing is performed on the Y component (luminance component) and U / V component (pointing to the color difference component, U component, and V component) of the image data. This is because there is a difference between the luminance component and the color difference component of the color in terms of image characteristics and visual recognition characteristics. Specifically, as a characteristic of the image itself, when an image is extracted in block units (for example, 4 × 4 pixels), the dynamic range of the color difference gradation value is small. On the other hand, such a tendency is not observed in the dynamic range of the luminance gradation value. Further, the visual recognition characteristics of color differences by humans are characterized in that low chroma colors have high discrimination ability and high chroma colors have low discrimination ability. However, in the visual recognition characteristic of luminance, there is not so much tendency that the discrimination ability changes depending on the luminance level. As described above, there is a difference in characteristics between the color luminance component and the color difference component. For this reason, in the present embodiment, image processing for reducing the amount of data transferred to the display device 212 is performed by different methods that match the components of the color space.

  Hereinafter, the CPU 216 according to the present embodiment, to which the image processing performed paying attention to the above-described points is applied, will be specifically described. The CPU 216 executes various functions of the mobile terminal device 210 by executing various programs stored in the program ROM 220. Specifically, the CPU 216 includes an RGB → YUV conversion unit 230, a weighted quantization processing unit 232, a frequency distribution calculation unit 234, a maximum / minimum value table generation unit 236, an encoding processing unit 238, and a color reduction processing unit 240. A frequency calculation unit 242 and a Huffman encoding processing unit 244 are provided.

  First, the RGB → YUV conversion unit 230 receives RGB format image data S11 input from the transmission / reception unit 214 and converts the RGB format data into YUV format data. The data converted into the YUV format is divided into a Y component and a U / V component when output from the RGB → YUV conversion unit 230. The Y component (luminance component) is input to the color reduction processing unit 240, and the U component (blue color difference component) and the V component (red color difference component) are input to the weighted quantization processing unit 232, and are subjected to separate processing. Go.

  First, the weighted quantization processing unit 232 performs quantization processing on U component and V component data. Specifically, in consideration of visibility, a quantization process according to the tone value of the color difference is performed. Thereby, the amount of image data transferred to the display device 212 can be reduced. Furthermore, since the amount of data is reduced in consideration of visibility, the displayed image does not feel uncomfortable. Details of the weighted quantization processing according to the present embodiment will be described later. The data subjected to the weighted quantization process is input to the frequency distribution calculation unit 234 and the encoding processing unit 238.

  The U component and V component data subjected to the weighted quantization processing as described above are then subjected to encoding processing by the frequency distribution calculation unit 234, the maximum / minimum value table generation unit 236, and the encoding processing unit 238. Is called. First, the frequency distribution calculation unit 234 calculates the frequency (= appearance rate) of gradation values for the image data received from the weighted quantization processing unit 232. Next, the maximum / minimum value table generation unit 236 determines the maximum and minimum values of the gradation values of the image based on the frequency calculated by the frequency distribution calculation unit 234, and creates a maximum / minimum value table. To do. Next, the encoding processing unit 238 encodes the data input from the weighted quantization processing unit 232 based on the maximum value and the minimum value of the gradation values determined by the maximum / minimum value table generation unit 236. Process. Specifically, a conversion process is performed so that image data is expressed within a range determined from the maximum value and the minimum value. In other words, the gradation values that the image data does not have (that is, the area below the minimum value and the area above the maximum value) are cut. Thereby, in addition to the above-described weighted quantization processing, the amount of image data transferred to the display device 212 can be further reduced. The encoding process described above can exhibit the effect of reducing the amount of data when the image is extracted in units of blocks as described above, because the dynamic range of the color difference gradation is relatively narrow. The encoded data S13 obtained by the encoding processing unit 238 and the maximum and minimum value data S12 calculated by the maximum / minimum value table generation unit 236 necessary for decoding are represented by the display device 212. Is input to the decryption processing unit 240.

  On the other hand, the Y component of the image data output from the RGB → YUV conversion unit 230 is subjected to color reduction processing by the color reduction processing unit 240. Here, the acquired image data is processed to reduce the gradation value of the image while remaining close to the original image. Thereby, the data amount of the image data transferred to the display device 212 can be reduced. This color reduction process will be described in detail later.

  The Y component data subjected to the color reduction processing is subjected to Huffman coding processing by the frequency calculation unit 242 and the Huffman coding processing unit 244. First, the frequency calculation unit 242 calculates the frequency of gradation values for the image data input from the color reduction processing unit 240. Next, the Huffman encoding processing unit 244 determines the correspondence between the gradation value and the Huffman code based on the frequency calculated by the frequency calculation unit 242, and determines the Y of the image data supplied from the color reduction processing unit 240. The component is Huffman encoded. The encoded image data S14 output from the Huffman encoding processing unit 244 and data defining the Huffman encoding method and necessary for decoding are input to the Huffman decoding processing unit 254. With the above processing, the amount of image data transferred to the display device 212 can be reduced, as in the above-described color reduction processing. This Huffman encoding process is a reversible conversion process and does not substantially reduce image data. That is, it is possible to display an image that has been subjected only to the color reduction process by performing a decoding process that is an inverse transformation later. The details of the Huffman coding processing according to the present embodiment, which is executed by the frequency calculation unit 240 and the Huffman coding processing unit 244 described above, will be described later.

  Thus, the CPU 216 performs image processing for reducing the data amount. Thereby, the data transfer speed of the image data to the display device 212 can be increased, and low power consumption can be realized. Further, the CPU 216 divides the data into a Y component and a U / V component, and performs image processing by a method suitable for each characteristic. Thereby, although the amount of data is reduced, it is possible to display an uncomfortable image that is not far from the original image. In addition, the CPU 216 implements various functions of the mobile terminal device 210 by executing various programs other than these, but since these are not directly related to the present invention, description thereof will be omitted.

  Next, the display device 212 according to the present embodiment will be described. The display device 212 is a lightweight and thin display device such as an LCD (Liquid Crystal Display), and includes a liquid crystal display panel 227, a driver 226 which is a semiconductor, and the like. Further, the display device 212 functions as an image processing unit for image data received from the CPU 216. The display device 212 includes a decoding processing unit 250, an equal quantization processing unit 252, and a Huffman coding processing unit as an image processing unit. Note that these image processing units can be realized by, for example, a processor in the display device executing a predetermined image processing program, and can also be provided inside the driver 226.

  Image data processed separately for each color component by the CPU 216 is input to the display device 212. The Y component data is input to the Huffman decoding processing unit 254, and the U / V component is input to the decoding processing unit 250.

  First, image processing for U component and V component image data in the display device 212 will be described. The decoding processing unit 240 receives the image data S13 encoded by the encoding processing unit 238 in the CPU 216 and the maximum value and minimum value data S12 calculated by the maximum / minimum value table generation unit 236. Is done. Here, the encoded image data S13 is decoded based on the maximum value and minimum value data S12. Thereby, the image data S13 encoded for reducing the data amount is returned to the image data before encoding. Next, the equal quantization processing unit 252 performs inverse quantization processing on the data input from the decoding processing unit 250. Since the weighting quantization processing unit 232 described above performs the process of changing the quantization amount according to the gradation value, the equalization quantization processing unit 252 performs this inverse transformation, and the state data before weighting quantization. Restore to. As a result, adjustment is performed so that data expressed with a small gradation value is expressed with a larger range. Data subjected to the above processing is input to the driver 226.

  On the other hand, the Y component image data is processed by the Huffman decoding processing unit 254. The Huffman decoding processing unit 254 converts the Huffman-encoded image data S14 input from the Huffman encoding processing unit 244 into data defining a Huffman encoding method (for example, correspondence between original data and encoded data). Table). As a result, the image is converted into an image that has undergone only the color reduction process before being decoded. Data subjected to the above processing is input to the driver 226.

  As described above, the image data that has been subjected to processing such as decoding separately for the Y component and the U / V component is integrated by the driver 226. At this time, YUV format data can be converted into RGB format data. The YUV → RGB conversion may be performed in the driver 226 or may be performed by providing a separate processing unit before input to the driver 226. The image data thus processed by the driver 226 is supplied to the display panel 227 and displayed in the display area of the display panel 227. Note that the display device 212 can display at a high resolution such as 352 × 416 pixels (CIF size) as described above, for example, in which the number of pixels in the horizontal and vertical directions is as described above.

  As described above, in the present embodiment, the processing for reducing the amount of data is performed in the CPU 216, so that the data transfer rate of the image data to the display device 212 can be increased, and low power consumption can be realized. In addition, since the image processing is performed by a method that matches the characteristics of the color components (Y component, U / V component), it is possible to display an image with no sense of incongruity. Furthermore, since the amount of data is reduced by the encoding process that is reversible conversion, it is possible to decode and display a high-quality image close to the original image.

[Image processing on CPU]
Here, the image processing according to the present embodiment performed by the CPU 216 will be described in detail. First, an outline of the processing will be described. The CPU 216 focuses on reducing the amount of image data to be transferred to the display device 212 and preventing the quality of the image displayed on the display device 212 from being deteriorated (that is, making the display image not strange). To process. Specifically, the image processing of the present embodiment includes image processing performed in consideration of the characteristics of color luminance components and color difference components, and image processing that reduces the amount of data using reversible transformation. . This is because first, there is a difference in image characteristics and visual recognition characteristics regarding the luminance component (Y component) and the color difference component (U / V component) forming the color space. Therefore, a method is adopted in which the luminance component and the color difference component are separately performed and different image processing is performed. As a result, the chrominance component can reduce a larger amount of data than the luminance component. That is, the data amount can be efficiently reduced. Further, a method of reducing the data amount by image processing using reversible conversion is adopted for the data obtained by dividing the luminance component and the color difference component in this way. This is because the quality of the displayed image is not deteriorated because the data before conversion can be restored by inverse conversion.

  Specifically, the image processing according to the present embodiment performs a color reduction process and a Huffman encoding process for the Y component, and a weighting quantization process and a code based on the maximum value / minimum value for the U / V component. Process. While the above color reduction processing and weighted quantization processing are irreversible transformations, the Huffman coding processing and the coding processing based on the maximum value / minimum value are reversible transformations. That is, taking into consideration image characteristics and visual recognition characteristics, a method is used in which the amount of data is reduced to some extent by irreversible conversion, and further, lossless conversion is performed on this data to reduce the amount of data transferred to the display device 212. The processing by the CPU 216 as described above can effectively reduce the amount of image data. Furthermore, it is possible to display a high-quality image that is close to the original image and that does not feel uncomfortable.

(RGB → YUV conversion)
First, the RGB → YUV conversion unit 230 in the CPU 216 converts the RGB format image data supplied from the transmission / reception unit 214 or the like into the YUV format according to the following equation. Here, in the RGB format, R, G, and B indicate the gradation values of the three primary colors of light of Red (red), Green (green), and Blue (blue), respectively. On the other hand, in the YUV format, Y represents luminance, and U and V represent blue color difference (Cb) and red color difference (Cr), respectively. The following equation is a general equation used for RGB → YUV conversion.

Y = 0.299R + 0.587G + 0.114B
U = −0.169R−0.331G + 0.500B + 128
V = 0.500R-0.419G-0.081B + 128
In the present embodiment, simultaneously with the RGB → YUV conversion, the data amount of the input 8-bit RGB data of each color is reduced so that Y, U, and V each have 7 bits. That is, a process of dropping data representing colors with 256 gradations into half 128-gradation data is performed.

(Processing of U component and V component)
Hereinafter, image processing of U / V components (color difference components) of image data will be described.

(1) Weighted quantization processing First, the weighted quantization processing according to the present embodiment will be described. In a general quantization process, the range of output gradation values relative to the gradation value of input data, that is, the quantization step is made the same.

  In this embodiment, a method of changing the quantization step according to the gradation value of input data is adopted. This will be specifically described with reference to FIG. In FIG. 2, the horizontal axis indicates the gradation value (input gradation value) of the data before the weighted quantization process. Here, since the gradation value of the color difference is taken, the saturation becomes lower as it goes to the left, and the higher saturation becomes as it goes to the right. The vertical axis indicates the gradation value (output gradation value) after the weighted quantization process. The U / V data before processing is, for example, 128 gradations (7 bits), and can be converted to 64 gradations (6 bits) after processing. In the present embodiment, the quantization step is changed between the low saturation side and the high saturation side. The quantization step is increased on the low saturation side, and the quantization step is decreased on the high saturation side. That is, by applying weighting to the quantization characteristics, low saturation data is expressed finely, and high saturation data is expressed coarsely.

  As described above, when converting U / V data of 128 gradations (7 bits) into 64 gradations (6 bits) by weighted quantization processing, for example, the processed gradation value b1 = 48 is set. . Specifically, as shown in FIG. 2, an input gradation value of 0 to 63 is set to low saturation, and 64 to 127 is set to high saturation. 48 gradations of output gradation values 0 to 47 are assigned to input gradation values 0 to 63, and 16 gradations of output gradation values 48 to 63 are assigned to input gradation values 64 to 127. Yes. That is, the width is changed at the point where the input gradation value = 64. A gradation value a2 whose input gradation value is located on the lower saturation side than 64 is converted into an output gradation value b2 by weighted quantization processing. On the other hand, the input tone value a3 whose input tone value is located on the higher saturation side than 63 is converted into the output tone value b3 by weighted quantization processing. In this way, the gradation values of low saturation are finely quantized and the gradation values of high saturation are coarsely quantized, so that the U / V data of 128 gradations (7 bits) as a whole becomes 64 gradations (6 bits). Can be compressed.

  As described above, in the present embodiment, the weighted quantization process is used in which the quantization step is changed depending on the saturation level. This is because the high-saturation color has a lower discrimination ability than the low-saturation color in the visual recognition characteristic of the color difference. By such weighted quantization processing, data transferred to the display device 212 can be reduced, and an image without a sense of incongruity can be displayed.

(2) Encoding process Next, the encoding process which concerns on this embodiment is demonstrated using FIG. In the present embodiment, the encoding process is performed in units of image blocks as shown in FIG. For example, four rows (four lines) of data are processed once as one block, and when the processing of this block is completed, the next block can be processed. First, in the data in this block, the frequency (= appearance rate) of gradation values is calculated. This processing can be performed by the frequency distribution calculation unit 234 as described above. This frequency calculation process may be performed simultaneously with the above-described weighted quantization process. When the calculated frequency is plotted on the vertical axis and the gradation value is plotted on the horizontal axis, a histogram is created, for example, as shown in FIG.

  Based on this histogram (hereinafter also referred to as “frequency histogram”), the maximum / minimum value table generation unit 236 determines the maximum value and the minimum value of gradation values in one block. In the example of FIG. 3B, the processing is performed on data having 64 gradations (which is converted to 6-bit data by the weighted quantization processing unit 232). Based on the maximum value and the minimum value determined as described above, the image processing unit 238 compresses the image data. The encoding processing unit 238 performs encoding so that the image data is expressed only within a range determined from the maximum value and the minimum value.

  For example, if there is 6-bit image data having a minimum gradation value of 15 and a maximum value of 40, the gradation value 15 is converted to 0 by conversion, and the gradation value 40 is converted to 25 ( That is, the entire gradation value is shifted by 15 gradations), and the data can be made into 5 bits. The amount of data can be reduced by the above encoding process.

  That is, if the range defined by the minimum value and the maximum value (that is, a range where gradation values exist, hereinafter also referred to as “effective gradation range”) is 32 gradations or more, 6-bit data is obtained. However, if the effective gradation range is 32 gradations or less, it can be expressed by 5-bit data. If the effective gradation range is 16 gradations or less, it can be expressed by 4-bit data.

  As described above, such an encoding process is characterized in that, when an image is extracted in units of blocks, the dynamic range of gradation values is generally narrow with respect to color difference components, so that the effect of reducing the amount of data is achieved. is there. In addition, since this conversion is a reversible conversion, data is not lost, and can be restored by decoding. Therefore, a high-quality image can be displayed by inverse conversion.

  As described above, the U / V component image data whose data amount has been reduced by the processing of the CPU 216 is supplied to the display device 212. Specifically, the data S13 encoded by the encoding processing unit 238 and the maximum value and minimum value data S12 are transferred to the decoding processing unit 250 in the display device 212. Note that image data and other related data can be transferred from the CPU 216 to the display device 212 using a data bus and a serial bus. The encoded data S13 having a large amount of data is transferred via the data bus. On the other hand, the maximum value 12 and the minimum value data 12 may be transferred to the encoded data S13 by time division multiplexing using this data bus, or the CLK signal or the SDA signal is transmitted because the data amount is small. The data may be transferred via a serial bus used when performing the above.

(Processing of Y component)
Hereinafter, image processing relating to the Y component (luminance component) of the image data will be described. In the Y component of the image, there is no image characteristic and visual recognition characteristic as seen in the U / V component, and therefore the weighted quantization process and the encoding process based on the maximum value / minimum value are not performed. As the image processing for the Y component, if the data amount is excessively reduced (expressed by a narrow gradation value), the image quality is deteriorated. Therefore, the conversion processing is performed by a method that does not depart from the original image as much as possible.

(1) Color Reduction Process First, the color reduction process performed on the Y component according to the present embodiment will be described. In this embodiment, a dither method or an error diffusion method is used as the color reduction processing. These two processes are processes that can reduce the amount of data with respect to the acquired image data while remaining close to the original image. That is, unlike processing such as bit slicing, an uncomfortable image simulating the original image can be generated.

  First, the dither method will be described with reference to FIG. In the dithering process, for example, the original image data received by the transmission / reception unit 214 is divided into blocks of 4 pixels in the vertical direction and 4 pixels in the horizontal direction, and the process is performed for each divided block. As a specific example, FIG. 4A shows one block arbitrarily selected from the original image data, and each numerical value represents a gradation value of each pixel. FIG. 4B is an example of a dither matrix, in which numbers representing gradation values from 0 to 15 are written, and this matrix is used for comparison with the original image data (ie, the numbers are Meaning as a threshold). Specifically, the received image data is compared with the numerical value of the corresponding pixel in the dither row example, and is set to 15 (white) if it is larger than the number in the dither matrix, and 0 (black) if it is smaller. As a result, an image as shown in FIG. 4C is obtained.

  In this way, by performing the dither process using the above 4 × 4 pixel dither matrix, the gradation values from 0 to 15 can be expressed by one gradation of white or black. Therefore, the 16 gradation values of the original image data can be reduced to 1/16, that is, the gradation values can be reduced by 4 bits.

  In the present embodiment, it is possible to perform color reduction processing on the image data of the Y component using the above-described dither method. Here, it is assumed that the image data to be processed is, for example, 7-bit data per pixel, and dither processing is performed on this. In the above example, the gradation value is binarized by conversion by the dither method, but the image data can also be expressed by multiple values by the dither method. As the simplest method, a ternary dither method capable of expressing an intermediate level between black and white will be described with reference to FIG. In this case, considering a half of the luminance of one pixel in given image data (for example, 127 for 256 gradation data), the gradation existing between 0 and 127 is considered. For the value pixel, 0 or 127 is determined using the above-described binary dither method, and the gradation value pixel existing between 127 and 255 is determined using the dither method. 255 is determined. Thus, image data having gradation values from 0 to 255 can be expressed by three values of 0, 127, and 255. In the present embodiment, the data amount of the Y component can be reduced by using a multi-value dither method.

  Next, the error diffusion method will be described with reference to FIG. In the simplest error diffusion method, a process of allocating all errors that occur when binarization is performed in a pixel of interest to the right pixel. An example of the processing is shown in FIG. The upper part of FIG. 5A is original image data that has not been subjected to color reduction processing, and each numerical value represents the gradation value of each pixel. First, when the image data with the gradation value of 200 located on the left in FIG. 5A is binarized, it is larger than 127 (corresponding to a half value of 256 gradations), so the gradation value becomes 255 (white). . At this time, an error of a gradation value 55 (= 255-200) occurs with respect to the original image. In FIG. 5B, this error is distributed to the image data located in the center where the gradation value is 160, and 55 is subtracted from the pixel value to obtain the gradation value 105. Perform value processing. As a result, the tone value is converted to 0 (black). At this time as well, an error of −105 (= 0− (160−55)) occurs similarly in the gradation value. In FIG. 5C, this error is distributed to pixels having a gradation value of 180 and binarized to be converted to a gradation value of 255 (black). In this manner, by sequentially distributing errors to adjacent pixels, a pseudo gradation pattern based on a binary image can be created. Here, all errors are distributed to the right pixel, but the number of pixels to which the error is allocated can be increased (for example, allocated to the right, lower right, and lower), or the ratio can be changed depending on the allocation position. Thereby, although the amount of calculation required for processing increases, it is possible to create an image that more accurately simulates the original image.

  In the above example of the error diffusion method, the gradation value is binarized by conversion. However, in this embodiment, the image data is expressed in multiple values by the error diffusion method, and the data amount of the Y component is reduced. can do.

  Below, the characteristics of the image created by the dither method and the image created by the error diffusion method will be described. In the above-described dither method, there is a possibility that a portion that should be processed as a single continuous image is divided or an image that should be processed separately is attached. As a result, an unnatural regular pattern may appear. However, in the error diffusion method, an error is calculated based on the gradation value of each pixel, and then the offset is allocated to the vicinity. Therefore, when viewed as a whole, the image has an average error from the original image. It has become. For this reason, the unnatural regular pattern which arises by the process of a dither method does not appear. However, since the error diffusion method does not repeat the fixed processing like the dither method by patchwork, it takes time to calculate. From the above, for example, when speed of processing speed is required rather than image quality, such as moving images, dither processing is used, and when image quality is required rather than processing speed, such as still images It is preferable to use an error diffusion method.

  As described above, the CPU 216 performs the color reduction process, whereby the amount of image data transferred to the display device 212 is reduced. Although the data amount is reduced by the color reduction processing, unlike the processing such as bit slicing, the processing for simulating the original image is performed, so that the displayed image does not feel strange.

(2) Huffman Coding Process Hereinafter, the Huffman coding process according to the present embodiment performed in the CPU 216 will be described. In general, the Huffman coding process is a process of assigning a short code to data that occurs frequently and a long code to data that is difficult to generate. . That is, the original data is made variable. By this processing, the data amount of the original data can be reduced.

  A specific example will be described with reference to FIG. FIG. 6A shows the original data to be processed, which is a character string composed of the symbol “aabbccccddddeeeeeeeeffff”. Next, as shown in FIG. 6B, the frequency of the input symbol is calculated to create a frequency table. In FIG. 6B, the left column shows the input symbols, and the right column shows the frequency (also referred to as the appearance rate). Next, a Huffman tree is generated based on the frequency table thus calculated. As shown in FIG. 6C, the Huffman tree is composed of leaves 10, branches 11, internal nodes 12, and roots 13. Hereinafter, a procedure for creating a Huffman tree will be described. First, a leaf 10 is created for each symbol, and the symbol and its frequency are written on each leaf 10. Next, one internal node 12 is created for the two leaves 10 with the lowest frequency, and the internal node 12 and the two leaves 10 are connected by the branch 11. A label “0” is added to one of the branches 11 and a label “1” is added to the other. The sum of the frequencies of the two leaves 10 is written in this internal node 12, and this internal node 12 is considered as a new leaf 10. Considering that there are no branches 11 and leaves 10 below the internal node 10, the above procedure is repeated. The Huffman tree is completed when the leaf 10 (or the internal node 12) becomes one sheet.

  Through the above processing, the original data is converted into a Huffman code as shown in FIG. That is, it is possible to create a code table in which a is “1110”, b is “1111”, c is “100”, d is “101”, e is “0”, and f is “110”. Based on this encoded data or Huffman tree, the original data can be encoded as “1110 1110 1111 1111 100 100 100 100 100 101 101 101 101 0 0 0 0 0 0 0 0 0 110 110 110 110 110”. .

  Hereinafter, the Huffman encoding process specifically performed in the present embodiment will be described. First, a frequency table is calculated by the frequency calculation unit 240 for the image subjected to the color reduction processing by the color reduction processing unit 230 as described above. At this time, in the present embodiment, as shown in FIG. 3A, the Huffman encoding process is performed in units of blocks. For example, if four rows of an image are handled as one block and the encoding of one block is completed, the process can be performed by moving to the next block. That is, the frequency calculation unit 240 creates a frequency histogram for data of one block of image data as described in the encoding process using the maximum value and the minimum value. The frequency table calculation process may be performed simultaneously with the above-described color reduction process. Based on the frequency table calculated in this way, the Huffman encoding processing unit 244 creates the Huffman tree shown in FIG. Then, Huffman coding is performed on the image data based on the Huffman tree. The encoded image data S14 is supplied to the Huffman decoding processing unit 254 in the display device 212. At this time, the Huffman tree data or the table indicating the correspondence between the code and the original data (that is, the table shown in FIG. 6D) is necessary for decoding the encoded data. Is transferred to the display device 212 as data S16 indicating the conversion method.

  The amount of data transferred to the display device 212 is reduced by the Huffman encoding process in the CPU 216 described above. Although this data reduction amount depends on the image, it has been experimentally found that it can be reduced to about ½ of the input data amount. In this Huffman encoding process, the amount of data is reduced by encoding, and the image data itself is not lost, so that the quality of the decoded and displayed image does not deteriorate.

  Note that as described above, image data and other related data can be transferred from the CPU 216 to the display device 212 using a data bus and a serial bus. The encoded data can be transferred using a data bus. Further, the data S16 indicating the Huffman encoding method can be transferred via the serial bus because the data amount is small.

[Image processing on display devices]
Hereinafter, image processing performed by the display device 212 will be described. As described above, since the Y component data and the U / V component data are separately transferred from the CPU 216, different image processing is also performed in the display device 212. In the display device 212, image processing for decoding data encoded mainly by the CPU 216 is performed. Decoding processing is performed for each of the Y component and the U / V component. This decoding process is an inverse transformation to a reversible process mainly performed by the CPU 216. Thereby, it is possible to display a high-quality image closer to the original image than expected from the amount of transferred data.

(Processing of U component and V component)
Here, the image processing of the U / V component of the image data will be described. The U / V component data processed by the CPU 216 is first input to the decoding processing unit 250. At this time, the encoded image data S13 and the maximum value and minimum value data S12 are input to the decoding processing unit 250. The decoding processing unit 250 decodes the encoded image data S13 based on the maximum value and minimum value data S12. Specifically, for example, a process of adding the minimum value of the gradation value to each of the gradation values of the encoded data is performed. For example, if there is encoded 5-bit data and data with a minimum gradation value of 15 and a maximum value of 40 is sent, 15 is added to each gradation value represented by 5 bits. Perform the process. That is, the original image data of 6 bits is restored.

  In the present embodiment, as described above, since the CPU 216 performs processing in units of blocks, the encoding processing performed by the CPU 216 and the decoding processing performed by the display device 212 can be performed in parallel. This will be specifically described with reference to FIG. As shown in FIG. 7A, it is assumed that the CPU 216 performs processing from the top to the bottom of the image and performs encoding processing on the image data at the position shown in the figure at a certain time. At this time, the image above the block on which the process is being performed (the image for which the encoding process has been completed) has already been transferred to the display device 212. On the other hand, as shown in FIG. 7B, the display device 212 can perform the decoding process on the data above the image encoded by the CPU 216 at the above time. Thereby, the encoding process performed by CPU216 and the decoding process performed by the display apparatus 212 can be performed efficiently.

  The image data decoded by the decoding processing unit 250 as described above is sent to the equal quantization processing unit 252.

  Next, processing in the equal quantization processing unit 252 will be described with reference to FIG. In FIG. 8, the vertical axis represents the gradation value of the data before processing, and the horizontal axis represents the gradation value of the data after processing. In the weighted quantization processing by the CPU 216 described above, a method of changing the quantization amount according to the gradation value (that is, depending on the magnitude of the saturation) is adopted. As shown in FIG. 2, the low-saturation region is finely quantized, and the high-saturation region is coarsely quantized. In the example of FIG. 2, the U / V data before quantization has 128 gradations (7 bits) and has input gradation values of 0 to 127. With the input gradation value 64 as the boundary, the input gradation values 0 to 63 are assigned to the output gradation values 0 to 47, and the input gradation values 64 to 127 are assigned to the output gradation values 48 to 63. .

  On the other hand, the inverse quantization is performed in the uniform quantization processing unit 252 of the display device 212. That is, as shown in FIG. 8, the 64 gradation (6 bits) U / V data is converted into 128 gradation (7 bits) U / V data by the inverse characteristic of the weighted quantization characteristic shown in FIG. . Thereby, U / V data in a state before weighted quantization can be restored.

  In this way, the U / V component data processed by the decoding processing unit 250 and the equal quantization processing unit 252 is supplied to the driver 226.

(Processing of Y component)
Next, image processing of the Y component of the image data will be described. The Y component data processed by the CPU 216 is input to the Huffman decoding processing unit 254. At this time, the Huffman decoding processing unit 230 includes Huffman-encoded image data S14 and data S16 that defines a Huffman encoding method such as a table indicating the correspondence between Huffman tree data or code and the original data. Based on these, a decoding process is performed. A specific decoding method will be described with reference to FIG. For example, when a code “101” is sent, the right branch 11 having a bit “1” is first traced from the root 13. Next, the left branch 11 which is bit “0” is traced. Then, when the right branch 11 having the bit “1” is traced, the leaf 10 representing “d” is reached. Thereby, the data encoded as “101” is identified as the symbol “d”. The encoded data is decoded by the above procedure. Note that the Huffman decoding process can also be performed in block units by the CPU 216 in the same manner as described above (the process shown in FIG. 7). Can be done.

  The Y component data decoded by the Huffman decoding processing unit 254 in this way is supplied to the driver 226. The driver 226 integrates the Y component data and the U / V component data described above. At this time, YUV format data can be converted into RGB format data. This YUV → RGB conversion may be performed before the input of the driver 226. The image data thus processed by the driver 226 is supplied to the display panel 227 and displayed in the display area of the display panel 227.

[Modification]
Below, the modification of this invention is demonstrated. In the embodiment described above, the CPU 216 encodes the U / V component data based on the maximum value / minimum value of the gradation values. In one modification, if the data amount is 4 bits or less at this time, the Huffman encoding process is further performed. This is because data of 4 bits or less does not require much calculation time and memory in the Huffman encoding process. By adding this Huffman encoding process, the data amount can be further reduced for U / V data. Also, since the Huffman encoding process is a reversible conversion, it can be restored and returned to the image before the process, so the image quality of the displayed image is not degraded.

  In another modified example, the RGB → YUV conversion unit 230 of the CPU 216 performs sampling at YUV 4: 2: 0, and this data may be processed by the CPU 216 similar to the above-described embodiment. good. This is because the U / V component data is not subjected to image processing by the dither method, so that moire interference does not occur in the displayed image. As a result, data with a further reduced amount of data can be transferred to the display device 212. However, in this case, since the quality of the image is lower than the data sampled at YUV 4: 4: 4, it is preferably performed when displaying a moving image that requires a higher processing speed than the image quality.

1 shows a schematic configuration of a mobile terminal device to which image processing according to the present invention is applied. It is a figure for demonstrating a weighting quantization process. It is the figure which showed the specific example of the process performed based on the maximum value / minimum value of a gradation value with respect to an image. It is a figure for demonstrating the dither method. It is a figure for demonstrating an error diffusion method. It is a figure for demonstrating a Huffman encoding process. It is the figure which showed the relationship between the image processing in CPU, and the image processing in a display apparatus. It is a figure for demonstrating a uniform quantization process.

Explanation of symbols

  210 portable terminal device, 212 display device, 214 transmission / reception unit, 216 CPU, 218 input unit, 220 program ROM, 224 RAM, 226 driver, 227 display panel, 230 RGB → YUV conversion unit, 232 weighted quantization processing unit, 234 frequency Distribution calculation unit, 236 maximum / minimum value table generation unit, 238 encoding processing unit, 240 color reduction processing unit, 242 frequency calculation unit, 244 Huffman encoding processing unit, 250 decoding processing unit, 252 equal quantization processing circuit, 254 Huffman decoding processor

Claims (9)

In an image display device including an image processing unit and an image display unit,
The image processing unit
Image data acquisition means for acquiring image data in RGB format to be displayed;
YUV conversion means for converting the acquired RGB format image data into YUV format image data;
Image processing means for performing separate image processing for reducing data on Y component image data and U / V component image data obtained by YUV conversion;
Transfer means for transferring image data subjected to separate image processing to the image display unit,
The image processing means includes
Color reduction processing means for performing color reduction processing only on the Y component image data;
Only the image data of the U / V component is increased in a quantization step for image data whose gradation value is smaller than a predetermined value, and is quantized for image data whose gradation value is larger than the predetermined value. A quantization processing means for performing quantization with a smaller value and performing a quantization process;
A Huffman encoding process is performed on the Y-component image data subjected to the color reduction process, and a maximum value and a minimum value of a gradation value are applied only to the quantized U / V component image data. Encoding processing means for encoding only the image data between,
The image display unit displays the transferred image data of the Y component and the U / V component.   The image display apparatus according to claim 1, wherein the color reduction processing unit performs dither processing on the image data of the Y component.   The image display apparatus according to claim 1, wherein the color reduction processing unit performs error diffusion processing on the image data of the Y component.   2. The image display device according to claim 1, wherein if the image data of the U / V component after the encoding process is equal to or less than a predetermined number of bits, the encoding processing unit further performs a Huffman encoding process. . The image display unit
Decoding processing means for decoding the encoded Y component image data and the U / V component image data;
Second quantization processing means for performing quantization processing on the decoded U / V component image data;
And a display means for displaying the decoded Y component image data and the U / V component image data quantized by the second quantization processing means. 2. The image display device according to 1.   The image display apparatus according to claim 5, wherein the second quantization processing unit performs quantization based on a reverse characteristic of a quantization characteristic of the quantization processing unit of the image processing unit. The encoding processing means performs encoding in units of blocks of the image data,
6. The decoding processing unit performs decoding on image data corresponding to a block previously performed by the encoding unit in parallel with the encoding processing unit. Image display device. In an image display method executed in an image display device including an image processing unit and an image display unit,
An image data acquisition step of acquiring RGB format image data to be displayed;
A YUV conversion step of converting the acquired RGB format image data into YUV format image data;
An image processing step of performing separate image processing for reducing the amount of data on the Y component image data obtained by YUV conversion and the U / V component image data;
A transfer step of transferring image data subjected to separate image processing from the image processing unit to the image display unit;
Displaying the transferred image data of the Y component and the U / V component on the image display unit, and
The image processing step includes
A color reduction processing step for performing color reduction processing only on the image data of the Y component;
Only the image data of the U / V component is increased in a quantization step for image data whose gradation value is smaller than a predetermined value, and is quantized for image data whose gradation value is larger than the predetermined value. A quantization processing step for performing quantization by performing a small quantization,
A Huffman encoding process is performed on the Y-component image data subjected to the color reduction process, and a gradation value between the maximum value and the minimum value is only applied to the quantized U / V component image data. An image display method comprising: an encoding process step for encoding only image data. By being executed in an image display device including an image processing unit and an image display unit,
Image data acquisition means for acquiring image data in RGB format to be displayed;
YUV conversion means for converting the acquired RGB format image data into YUV format image data;
Image processing means for performing separate image processing for reducing the amount of data on the Y component image data obtained by YUV conversion and the U / V component image data;
Causing the image processing unit to function as transfer means for transferring image data that has undergone separate image processing to the image display unit;
The image processing means;
Color reduction processing means for performing color reduction processing only on the Y component image data;
Only the image data of the U / V component is increased in a quantization step for image data whose gradation value is smaller than a predetermined value, and is quantized for image data whose gradation value is larger than the predetermined value. A quantization processing means for performing quantization with a smaller value and performing a quantization process;
A Huffman encoding process is performed on the Y-component image data subjected to the color reduction process, and a gradation value between the maximum value and the minimum value is only applied to the quantized U / V component image data. It functions as an encoding processing means for encoding only image data,
An image display program that causes the image display unit to function as display means for displaying the transferred image data of the Y component and the U / V component.

Images