TWI507044B - Method of generating a gain of an image frame - Google Patents

Description

Method of generating an image picture gain

The present invention relates to a method for generating an image picture gain, and more particularly to a method for generating a gain of the image according to the sensitivity of the human eye to light.

In general, when a human visual system is watching a thing, it is simultaneously performing two kinds of resolution---color discrimination and luminance resolution. However, these two kinds of resolution capabilities are easily affected by the ambient illumination. For example, when in a very bright (high illumination) environment, the human visual system is more sensitive to color, that is, the human visual system is more likely to detect colors between each other in a highly illuminating environment. The slight difference; however, the sensitivity of the contrast is reduced, that is, the human visual system is less likely to detect the difference between light and dark in an environment with high illumination. Conversely, when in a very dark (illuminated low) environment, the human visual system is less sensitive to color, that is, the human visual system is less likely to detect colors between each other in an environment with low illumination. The difference is sensitive; however, the sensitivity of the contrast is improved, that is, the human visual system is more likely to detect subtle changes between light and dark in an environment with low illumination. Please refer to Figure 1. Figure 1 is a sensing curve of the illumination of an image taken by three different human eyes. The vertical axis of Fig. 1 represents the illuminance of the actual image (unit: candle/m2), and the horizontal axis represents the grayscale value (Least Significant Bit, LSB) perceived by the human eye. When the illuminance of the actual image changes from 0 to 50 candelas per square meter, the grayscale value perceived by the human eye represented by the three curves is about 110 (first curve: 1 to 111) to 150 ( Third song Line: between 1 and 151), and when the illumination of the actual image changes from 150 to 200 candelas per square meter, the grayscale value perceived by the human eye represented by the three curves is only about 20 (the first) Three curves: 221 to 241) to 30 (first curve: 201 to 231). Therefore, it can be seen from the change of any curve in Fig. 1 that when the human eye is low in illumination of the image, the contrast is very sensitive; when the illumination of the image is high, the contrast is less sensitive.

At present, a complementary CMOS image sensor used in an optical component has a problem of fixed pattern noise (FPN), and the fixed mode noise is obtained by the optical component. In the middle, the differences between the components are caused by each other. Please refer to Figure 2. Figure 2 is an image view of a fixed mode noise with 2.41%. As can be seen from Fig. 2, the difference in potential caused by the fixed mode noise causes a difference in illuminance between rows and columns of the image frame without causing a change in color.

Conventional optical components, such as a digital camera, must perform metering and focusing before one image is to be taken. Some cameras even have a metering area to measure the illumination of the scene to be shot to determine the desire. The gain of the captured image. A video playback component, such as a liquid crystal television, usually plays an image frame to measure the illuminance of the image frame to determine the gain of the main picture to be played next, and then the main picture that is subsequently played is adjusted according to the gain. Whether it is a camera or an LCD TV, metering must first waste an image or a metering area to achieve the purpose of measuring the illuminance of the image, in terms of efficiency and resources. It is not a very economical practice, and the image of the measured light is not the actual image to be shot or played, so this traditional practice cannot be called real-time and effective.

After the photometry, according to the measured illuminance of the image, through a look-up table, a gain value is selected to adjust the main picture to be shot or played next. In conventional practice, the look-up table is based on a shooting mode in the optical component, based on an averaging method. However, in these look-up tables, the factors of the sensitivity of the human eye to light are not taken into consideration, so that in a strong light or a very dark environment, the illumination of the environment cannot be matched to provide an optimum gain value operation.

Embodiments of the present invention disclose a method for generating an image picture gain, including setting a gain of an image frame to 1; scanning a plurality of images before the image frame; and averaging the images of the front plurality of columns to obtain the front An average of the images of the plurality of columns; a corresponding gain is generated based on the average; and an image of the remaining columns of the image frame is generated based on the corresponding gain and information of the remaining columns of the image frame.

Another embodiment of the present invention discloses a method for generating an image picture gain, comprising: scanning an image of a plurality of columns before an image frame; and averaging the images of the front plurality of columns to obtain an average of the images of the front plurality of columns; Generating a corresponding gain according to the average value and a gain of the image frame; and generating an image of the remaining columns of the image frame according to the corresponding gain and the information of the remaining columns of the image frame.

In view of the shortcomings of the conventional method, the present invention proposes to use a dummy row of the current image frame for photometry without using an additional image frame or a light metering area, and the establishment of the look-up table considers the human eye. Sensitivity to light, in order to obtain a better gain value, to improve the shortcomings of traditional practices.

If the aforementioned visual theory is applied to image processing, when the illuminance of an image is high, the overall gain of the image is increased; conversely, if the illuminance of an image is low, the overall image is The gain is reduced. The gain is controlled by a Programmable Gain Amplifier (PGA). When the overall gain of the image frame is increased, the illuminance of the image frame increases, the color contrast increases, and the fixed mode noise is amplified. However, if the illumination of the image is very high, although the fixed mode noise is increased, the human visual system is not easy to perceive the difference in illumination between the rows and columns of the image caused by the noise of the fixed mode, but the color can be easily perceived. Contrast the increased effect, and create an illusion of more color continuation on this image. Conversely, when the gain of an image frame is reduced, the illumination of the image frame is reduced, the color contrast is reduced, and the fixed mode noise is also reduced. However, if the illuminance of the image is originally low, the human visual system can easily detect the change of the illuminance of the image, so the fixed mode noise is reduced, and the human visual system can be reduced to recognize the fixed pattern noise. The interference. Please refer to Figure 3. Figure 3 is a histogram of the relationship between the gain controlled by a programmable gain control amplifier and the fixed mode noise. The vertical axis of Figure 3 is the percentage of fixed mode noise, and the horizontal axis is the gain of a programmable gain control amplifier. value. Each gain value in Figure 3 corresponds to two long columns, one long column representing the predicted percentage of fixed mode noise and the other long column representing the percentage of fixed mode noise actually obtained. It can be seen from the figure that the fixed mode noise is proportional to the gain of the programmable gain control amplifier. Next, please refer to Figure 4. Figure 4 is a graph of the relationship between an input voltage and a gain-adjusted output voltage. The slope of each point on the curve in the graph is the gain value of a programmable gain control amplifier. This gain value is generated based on the sensitivity of the human eye to light. In short, the darker the picture, the smaller the gain value; the brighter the picture, the larger the gain value. This figure is also the basis for the establishment of a gain value look-up table of the present invention.

Please refer to Figure 5. Fig. 5 is a flow chart showing a method of generating an image picture gain according to the first embodiment of the present invention. The steps in Figure 5 include: Step 10: Set a programmable gain control amplifier with a gain of one.

Step 12: Start scanning an image frame column by column.

Step 14: Obtain an image of the first 18 columns of the virtual column of the image frame by column-by-column scanning.

Step 16: Average the image of the first 18 columns of virtual columns to produce an average.

Step 18: According to the average value, look for a phase in an established gain value lookup table. Corresponding gain value.

Step 20: Continue column-by-column scanning of the remaining columns of the image frame (ie, the main screen), and adjust information of the main screen obtained by column-by-column scanning according to the corresponding gain value to generate an image of the main screen.

In the first embodiment, the gain value of the programmable gain control amplifier is set to 1 before the image image is scanned, so that the information of the scanned image frame and the image frame adjusted by the gain value are obtained. The images are the same. Then, an image frame is scanned column by column, and the image of the first 18 columns of the virtual column of the image frame is obtained. Please note that the first 18 columns of virtual columns selected in this embodiment are for illustrative purposes, and are not intended to limit the present invention. The selection of the number of imaginary columns depends on the resolution of the image frame and the like. "Virtual column" is located at the edge of an image frame. The image data contained in the "virtual column" cannot be used as the main screen because of the various effects of the edge. However, if it is used to measure the illumination of the image, it is actually enough. The more the number of imaginary columns selected, the closer the information obtained by scanning is to the entire image frame. The gain value obtained after the lookup table can adjust the image frame to a better state, but relatively, the main picture (the remaining image picture The number of columns is reduced, and the resolution is reduced. Conversely, if the number of selected virtual columns is smaller, the less the information obtained by scanning is closer to the entire image frame, the more difficult it is to adjust the image to the image. In a preferred state, the number of columns of the main picture (the remaining columns of the picture picture) increases, and the resolution increases. Therefore, how to find a balance between the number of virtual columns and the resolution depends on the designer's own judgment.

Then, the images of the first 18 columns of imaginary columns are averaged to generate an average value, and according to the average value, a corresponding gain value is found in an established gain value lookup table. Please note that the look-up table of the present invention is based on the sensitivity of the human eye to light, and the average human eye, such as a weighted average, is used to optimize the curve of sensitivity of different human eyes to light. Table, in the lower image illuminance, corresponding to Low gain, higher image illuminance, corresponding to higher gain. As described above, the gain value of the image frame obtained in the look-up table reduces the influence of the fixed mode noise on the image image when the illumination of the image image is low; and the sensitivity of the human eye to the light when the illumination of the image is high, It is not too strong to feel the effect of amplifying the fixed mode noise on the image. In the optical system of the present invention, the look-up table may have a plurality of sheets, each of which corresponds to a different shooting mode in the optical element, such as a night shooting mode, an automatic shooting mode, and the like. Then, the information of the remaining columns of the image frame (ie, the main screen) is further scanned column by column, and the information of the main screen obtained by the column scan is adjusted according to the gain value selected in the table to generate the image of the main screen. Please note that the sequence of steps used in the description of the present invention is only an example, and is not intended to limit the scope of the present invention, and the order of steps that can achieve the same purpose is also within the scope of the present invention. .

Please refer to Figure 6. Figure 6 is a schematic diagram of a circuit according to an embodiment of the present invention. Figure 6 contains a 640*488 (VGA resolution) pixel array 100, vertical scanners 102, 104, capacitor storage banks 106, 108, horizontal sweep The horizontal scanners 110, 112 (i.e., shift registers), and the programmable gain control amplifiers 114, 116. The vertical scanner 102 scans the pixel array 100 from bottom to top, scans the data of the first 18 columns of virtual columns one by one, scans the data of the first column, stores them in the capacitor storage module 106, and then passes through the horizontal scanner 110. According to the input timing, it is transmitted in a sequence, and is output after being adjusted by the programmable gain control amplifier 114. After all the data in the first column is output, the vertical scanner scans the second column until The first 18 columns of virtual data are scanned. In the first embodiment of the present invention, the gain of the programmable gain control amplifier is set to 1 and then scanned, so that the data of the first 18 columns of the dummy columns obtained by the scan is equal to the image output by the programmable gain control amplifier. After the sequence data is output, the external line synchronizes the average action. For example, in step 16 of FIG. 5, the image of the first 18 columns of the virtual column is synchronously averaged to generate an average value, and according to the average value, In the established gain value lookup table, look for a corresponding gain value. After finding the corresponding gain value, the vertical scanner continues to scan the data of the main screen (the remaining columns of the image frame) from bottom to top, and outputs the data to the programmable gain control amplifier according to the same procedure. The programmable gain control amplifier adjusts the data of the main picture scanned and outputs according to the corresponding gain value.

If the gain value of the programmable gain control amplifier is not set to 1 before the image is scanned, the image of the first 18 columns of the virtual image obtained after the column-by-column scan is averaged. After obtaining an average value, the gain value of the programmable gain control amplifier is further divided to obtain an average value of the data of the first 18 columns of the original image frame. After obtaining the average of the data of the first 18 columns of the original image, and then looking up the table, find a corresponding gain value. Then, according to the corresponding gain value, the information of the main picture (the remaining columns of the image picture) obtained by the column-by-column scanning is adjusted to generate an image of the main picture. Please refer to Figure 7. Figure 7 is a flow chart showing a method of generating an image picture gain according to a second embodiment of the present invention. The steps in FIG. 7 include: Step 30: Start scanning a video image column by column.

Step 32: Obtain an image of the first 18 columns of the virtual column of the image frame by column-by-column scanning.

Step 34: Average the images of the first 18 columns of virtual columns to produce an average value.

Step 36: Divide the average value by the gain value of a programmable gain control amplifier to obtain an average value of the data of the first 18 columns of the original column of the image frame.

Step 38: Find a corresponding gain value in an established gain value lookup table according to the average value of the data of the first 18 columns of the original image of the image frame.

Step 40: Continue column-by-column scanning of the remaining columns of the image frame (ie, the main screen), and adjust information of the main screen obtained by column-by-column scanning according to the corresponding gain value to generate an image of the main screen.

Please note that the sequence of steps used in the description of the present invention is only an example, and is not intended to limit the scope of the present invention, and the order of steps that can achieve the same purpose is also within the scope of the present invention. .

In the present invention, when establishing the gain value look-up table, in addition to considering the sensitivity of the human eye to light, it is also necessary to consider the manner of placing microlens above the pixel array of the liquid crystal television, and the manner of the process. The arrangement of the microlenses above the different pixel arrays, for example, placing a microlens above one pixel, or placing a microlens above four pixels, and the like, and different process methods, such as zero point one three process , the zero point one eight process, etc., will affect the method of establishing the lookup table in the present invention.

In summary, the present invention provides an average of one of the images of the imaginary column of the image frame by scanning with a virtual column of an image frame, and based on the average value, From a look-up table established according to the sensitivity of the human eye to light, finding a corresponding gain value, and then adjusting the data of the main picture of the image frame according to the corresponding gain value to obtain the main picture The method of imagery. The invention does not need to use another image frame for scanning, or separately establish a light metering area, and the data obtained by column-by-column scanning is the same picture, which is more valuable, and the table according to the table can provide an appropriate gain value. Adjusting the main screen to a better state to facilitate image processing on the back end is an effective method without wasting system resources.

The above are only the preferred embodiments of the present invention, and all changes and modifications made to the scope of the present invention should be within the scope of the present invention.

100‧‧‧pixel array

102,104‧‧‧Vertical Scanner

106,108‧‧‧Capacitor storage module

110,112‧‧‧ horizontal scanner

114,116‧‧‧Programmable Gain Control Amplifier

10,12,14,16,18,20,30,32,34,36,38,40‧‧

Figure 1 is a graph of the sensing curves of the illuminance of an image of three different human eyes.

Figure 2 is an image screen image with 2.4% fixed mode noise.

Figure 3 is a histogram of the relationship between the gain controlled by a programmable gain control amplifier and the fixed mode noise.

Figure 4 is a graph of the relationship between an input voltage and a gain-adjusted output voltage.

Figure 5 is a flow chart of the method of the first embodiment of the present invention.

Figure 6 is a schematic diagram of the circuit of the present invention.

Figure 7 is a flow chart of the method of the second embodiment of the present invention.

10,12,14,16,18,20‧‧ steps

Claims (21)

A method for generating an image picture gain, comprising: setting a gain of an image frame to 1; scanning an image of the plurality of columns before the image frame; and averaging the image of the front plurality of columns to obtain an average of the image of the front plurality of columns a value; generating a corresponding gain according to the average value; and generating an image of the remaining columns of the image frame according to the corresponding gain and the information of the remaining columns of the image frame. The method of claim 1, wherein the image of the plurality of columns before scanning the image frame is an image of a virtual column of a plurality of columns before scanning the image frame. The method of claim 1, wherein the corresponding gain is generated according to the average value, and the corresponding gain is generated in a table lookup manner according to the average value. The method of claim 3, wherein the lookup table of the lookup table corresponds to a preset operation mode in one of the optical systems in which the image frame exists. The method of claim 4, wherein the look-up table is established based on sensitivity of the human eye to light. The method of claim 1, wherein the lower average corresponds to a lower gain. The method of claim 1, wherein the higher average corresponds to a higher gain. The method of claim 1, wherein the image frame is an image frame played by a liquid crystal television. The method of claim 1, wherein the image frame is an image displayed by a liquid crystal panel of a digital camera. The method of claim 1, wherein the image of the remaining column of the image frame is generated according to the corresponding gain and the remaining pixels of the remaining columns of the image frame according to the corresponding gain and the information of the remaining columns of the image frame. The product of the information produces an image of the remaining columns of the image. A method for generating an image picture gain, comprising: scanning an image of a plurality of columns before an image frame; averaging the images of the plurality of previous plurality of columns to obtain an average value of the image of the front plurality of columns; and according to the average value and the image frame One gain, generating a corresponding gain; and generating the image based on the corresponding gain and information of the remaining columns of the image frame The image of the remaining columns of the screen. The method of claim 11, wherein the image of the plurality of columns before scanning the image frame is an image of a virtual column of a plurality of columns before scanning the image frame. The method of claim 11, wherein the corresponding gain is generated according to the average value, based on the average value, to generate the corresponding gain in a table lookup manner. The method of claim 13, wherein the lookup table of the lookup table corresponds to a preset operation mode of one of the optical systems in which the image frame exists. The method of claim 14, wherein the look-up table is established based on sensitivity of the human eye to light. The method of claim 11, wherein the lower average corresponds to a lower gain. The method of claim 11, wherein the higher average corresponds to a higher gain. The method of claim 11, wherein the image frame is an image frame played by a liquid crystal television. The method of claim 11, wherein the image frame is an image displayed by a liquid crystal panel of a digital camera. The method of claim 11, wherein the image of the remaining columns of the image frame is generated according to the corresponding gain and the pixels of the remaining columns of the image frame according to the corresponding gain and the remaining columns of the image frame. The product of the information produces an image of the remaining columns of the image. The method of claim 11, wherein the corresponding gain is generated by dividing the average value by the gain of the image frame to generate the corresponding gain according to the average value and the gain of the image frame. .