CN111629121B

CN111629121B - Image adjusting method and related image processing circuit

Info

Publication number: CN111629121B
Application number: CN201910145959.7A
Authority: CN
Inventors: 邱仲毅; 黄文聪
Original assignee: Realtek Semiconductor Corp
Current assignee: Realtek Semiconductor Corp
Priority date: 2019-02-27
Filing date: 2019-02-27
Publication date: 2022-05-24
Anticipated expiration: 2039-02-27
Also published as: CN111629121A

Abstract

The invention provides an image adjusting method and a related image processing circuit, which are used for performing the following operations on each pixel of an image: obtaining three primary colors (R, G, B) and infrared light corresponding to the current pixel; generating a plurality of initial compensation parameters respectively corresponding to the R, G, B values; generating an overcompensation parameter according to the initial compensation parameters corresponding to the R, G, B value and the IR value; comparing the overcompensation parameter with at least one critical value to generate a compensation adjustment coefficient; and using the compensation adjustment coefficient to perform infrared crosstalk compensation on the image.

Description

Image adjusting method and related image processing circuit

Technical Field

The invention relates to an image compensation technology, in particular to a related technology for compensating infrared Crosstalk (IR Crosstalk for short), which is suitable for three-primary-color infrared sensors (R, G, B and IR sensor) and can automatically calculate appropriate infrared interference compensation parameters for the sensors.

Background

When there is an IR component in the ambient light, it is absorbed and sensed by the IR pixels in the hybrid R, G, B, IR filter array, however, in the three primary colors (R, G, B) and Infrared (IR) spectrum, crosstalk (crosstalk) occurs, which is referred to as the ideal spectrum in fig. 1 and the spectrum with crosstalk in fig. 2, because the overlap phenomenon shown in fig. 2 is caused by the interference between the IR and R, G, B lights. The current sensor manufacturing technology cannot effectively block or absorb non-color signals, so that when the light energy has a high IR component, the object color is affected by the IR crosstalk to generate a color fading phenomenon. The IR light may come from the face recognition system of the video system or from a yellowish light, such as that emitted by a halogen lamp.

US 20100289885 a1 discloses an infrared crosstalk compensation technique, which subtracts a certain proportion of IR signal values from the output R, G, B signal value:

R_new＝R_ori-k1×IR_ori

G_new＝G_ori-k2×IR_ori

B_new＝B_ori-k3×IR_ori

IR_new＝IR_ori

wherein R is_new、G_new、B_new、IR_newAdjusted R, G, B values and IR values, R_ori、G_ori、B_ori、IR_oriThe original R, G, B value and IR value are respectively, and k1, k2 and k3 are fixed ratio parameters set by the user according to the influence of the infrared crosstalk. This prior art approach only roughly cancels the IR contribution, but not adjusts for the actual contribution, and is more likely to cause overcompensation or R, G, B color shift. Taking the red light value as an example, the infrared crosstalk compensation formula is as follows:

R__new＝R__ori-k1*IR__ori

while this compensation formula can provide a degree of compensation for most cases, when IR u_oriToo high, the compensation value will increase, resulting in k1 IR __oriThe value of (A) is even greater than the value of R _, of the red light itself_oriR obtained at this time_newIs obviously incorrect (R)_newWill exhibit a 0 or negative number). The phenomenon is easy to occur under the condition that outdoor sunlight is strong or indoor lamplight has high IR components, so that the infrared light crosstalk compensation value is too high to cause color cast, and particularly the condition that the color of a picture is green is easy to occur. Therefore, there is a need for a novel and relatively free-side effect method to improve the overcompensation problem.

Disclosure of Invention

The present invention is directed to an image adjusting method and an image processing circuit thereof, which can solve the above-mentioned problems.

An embodiment of the present invention provides an image adjusting method, for performing the following operations on each pixel of an image: obtaining three primary colors (R, G, B) and Infrared (IR) light corresponding to the current pixel; generating a plurality of initial compensation parameters respectively corresponding to the R, G, B values; generating an overcompensation parameter according to the initial compensation parameters corresponding to the R, G, B values and according to the IR value; comparing the overcompensation parameter with at least one critical value to generate a compensation adjustment coefficient; and using the compensation adjustment coefficient to perform infrared crosstalk (IR crosstalk) compensation on the image.

An embodiment of the present invention provides an image processing circuit, which includes a storage unit and a processor, wherein the storage unit is configured to temporarily store data, and the processor is configured to receive an image and perform the following operations for each pixel of the image: obtaining R, G, B, IR values corresponding to the current pixel; generating a plurality of initial compensation parameters respectively corresponding to the R, G, B values; generating an overcompensation parameter according to the initial compensation parameters corresponding to the R, G, B values and according to the IR value; comparing the overcompensation parameter with at least one critical value to generate a compensation adjustment coefficient; and using the compensation adjustment coefficient to perform infrared crosstalk compensation on the image.

The embodiment of the invention can detect the area which can be overcompensated, and apply different adjustment parameters to the compensation area to prevent overcompensation phenomenon on the infrared crosstalk.

Drawings

FIG. 1 is a flowchart illustrating an image compensation method according to an embodiment of the present invention;

fig. 2 is a schematic diagram of an image processing circuit corresponding to fig. 1 according to an embodiment of the invention.

Description of the symbols:

100 method

102 to 112 steps

200 image processing circuit

210 storage unit

220 processor

Detailed Description

Certain terms are used throughout the description and following claims to refer to particular components. As one of ordinary skill in the art will appreciate, hardware manufacturers may refer to a component by different names. In the present specification and the claims that follow, elements are distinguished not by differences in name but by differences in function. In the following description and in the claims, the terms "include" and "comprise" are used in an open-ended fashion, and thus should be interpreted to mean "include, but not limited to. Additionally, the term "coupled" is used herein to encompass any direct or indirect electrical connection. Thus, if one device is coupled to another device, that connection may be through a direct electrical connection, or through an indirect electrical connection via other devices and connections.

The present invention provides a mechanism for preventing over-compensation of IR crosstalk to solve the problems encountered in the prior art, which can be broadly divided into two parts, i.e., "detecting the possible over-compensation region" and "adjusting the compensation parameters for the over-compensation region", and the details are described below.

I. Detecting regions of possible overcompensation

For red light, the overcompensation is due to k1 IR ori having a value close to R ori or even greater than R ori, so that k1 IR ori/R ori is used as the candidate compensation characteristic value, and thus the candidate compensation characteristic values corresponding to green and blue light can be determined. Then, the largest one of the candidate compensation eigenvalues corresponding to R, G, B is selected as the overcompensation eigenvalue X, and the larger the eigenvalue is, the more likely the pixel is to be overcompensated. The following is the calculation of X:

wherein k1, k2, and k3 are the initial compensation coefficients corresponding to the value R, G, B, and IR (i, j), R _ ori (i, j), G _ ori (i, j), and B _ ori (i, j) are the original IR, R, G, and B values of the current pixel, respectively. k1(i, j), k2(i, j), and k3(i, j) are the initial compensation parameters of the infrared crosstalk of R, G, B of the current pixel, respectively.

Adjusting compensation parameters for overcompensation region

The compensation parameter is adjusted according to the overcompensation characteristic value X to obtain a compensation adjustment coefficient W _ final, and the calculation formula of the adjustment parameter is as follows

TH1 and TH2 are respectively a first overcompensation threshold and a second overcompensation threshold set by a user, and W1 and W2 are respectively an upper adjustment value and a lower adjustment value set by the user. For example, TH1 and TH2 can be set to 0.8 and 1.05 respectively, and W1 and W2 can be set to 0.8 and 0.6 respectively, but these are only one possible parameter setting, and the invention is not limited thereto. In addition, the above equation can be implemented by establishing a look-up table (LUT).

Referring to the first equation in the above column, when the overcompensation characteristic value X is lower than TH1 (e.g., lower than 0.8), the possibility of overcompensation is low due to such a value, and the upper limit of W _ final value can be limited to W1 or 1, i.e., no adjustment or only slight adjustment is possible. For example, when X is lower than TH1, if the initial compensation parameter is not to be adjusted, the value of W _ final can be made to be 1, because the final compensation parameter k _ final outputted finally is equal to the initial compensation parameter k multiplied by W _ final, so that the value of the final compensation parameter k _ final is maintained at the value of the initial compensation parameter k. In another embodiment, when X is lower than TH1, if only the initial compensation parameter is adjusted, the value of W _ final can be made to be W1, so that the difference between the final compensation parameter k _ final and the initial compensation parameter k is not large.

Then, referring to the second equation, when the overcompensation characteristic X is between TH1 and TH2, the W value is adjusted such that W _ final is between W1 and W2, wherein the closer X is to TH1, the closer W _ final is to W1; conversely, the closer X is to TH2, the closer W _ final is to W2. The determination of W _ final is not necessarily in accordance with the interpolation shown in the second row of equations, as long as substantially similar results are achieved.

Then, referring to the third equation, when the overcompensation characteristic value X is greater than TH2, which represents an environment with a large infrared value, such as an environment with strong sunlight, W _ final is modified but is limited to be at least not lower than W2, so that the pixels are not excessively modified to generate color shift.

After obtaining W _ final, the initial compensation parameters corresponding to R, G, B respectively are adjusted and multiplied by compensation adjustment coefficients W _ final to obtain adjusted compensation parameters k1_ final, k2_ final, and k3_ final corresponding to R, G, B respectively, as follows:

k1_final＝k1(i,j)*W_final；

k2 — final — k2(i, j) × W — final; and

k3_final＝k3(i,j)*W_final；

where k1_ final, k2_ final, and k3_ final are the final compensation coefficients for the corresponding R, G, B, respectively.

Then, the above final compensation parameters k1_ final, k2_ final, k3_ final are used for infrared light crosstalk compensation, and the following equations are given:

R_final(i,j)＝R_ori(i,j)-k1_final*IR(i,j)；

G_final(i,j)＝G_ori(i,j)-k2_final*IR(i,j)；

B_final(i,j)＝B_ori(i,j)-k3_final*IR(i,j)。

wherein R _ final (i, j), G _ final (i, j), B _ final (i, j) are adjusted R, G, B values to be finally outputted, which can greatly improve color shift caused by over-compensation in the conventional technology and can more accurately display colors close to actual images.

In summary, the present invention provides a method for calculating an ir crosstalk overcompensation detection characteristic value and a method for adjusting an ir crosstalk overcompensation parameter. The invention can effectively detect the area which is possibly subjected to the over-compensation in the image, and apply different adjustment parameters (such as a lower adjustment value) to the compensation area so as to prevent the over-compensation phenomenon of the infrared crosstalk. In addition, because each pixel of the image is detected by the respective infrared light value, the invention can only correct the part of the image which can excessively compensate the infrared light crosstalk, and does not influence the part of the image which needs to be properly compensated (or only slightly adjusted).

The above inventive concept can be summarized as the image compensation method 100 shown in fig. 1, and it should be noted that the steps are not necessarily performed according to the execution order shown in fig. 1 if substantially the same result can be obtained. The process shown in fig. 1 can be adopted by the image processing circuit 200 shown in fig. 2, wherein the image processing circuit 200 includes a storage unit 210 and a processor 220, and the image processing circuit 200 can be applied to various cameras and video devices. The storage unit 210 is used for temporarily storing data, and the processor 220 is used for executing the image compensation method 100 and various related operations. The image compensation method 100 can be briefly summarized as follows:

step 102: receiving an image;

step 104: obtaining R, G, B, IR values corresponding to the current pixel;

step 106: generating a plurality of initial compensation parameters respectively corresponding to the R, G, B values;

step 108: generating an overcompensation parameter based on the initial compensation parameter corresponding to the value of R, G, B and the IR value;

step 110: comparing the overcompensation parameter with a critical value to generate a compensation adjustment coefficient;

step 112: and using the compensation adjustment coefficient to perform infrared light crosstalk compensation on the image.

The above description is only a preferred embodiment of the present invention, and all equivalent changes and modifications made in accordance with the claims of the present invention should be covered by the present invention.

Claims

1. An image adjusting method is used for performing the following operations on each pixel of an image:

obtaining three primary colors R, G, B and Infrared (IR) values corresponding to the current pixel;

generating a plurality of initial compensation parameters respectively corresponding to the R, G, B values;

generating an overcompensation parameter according to the initial compensation parameters corresponding to the R, G, B values and according to the IR value;

comparing the overcompensation parameter with at least one critical value to generate a compensation adjustment coefficient; and

using the compensation adjustment coefficient to perform infrared crosstalk (IR crosstalk) compensation on the image;

wherein the step of generating the overcompensation parameter according to the initial compensation parameter and the IR value corresponding to the R, G, B values respectively comprises:

the following three products respectively corresponding to the R, G, B values are respectively generated according to the product of the initial compensation parameter corresponding to the R, G, B value and the IR value, and the maximum of the above three products is selected as the overcompensation parameter:

wherein k1, k2, and k3 are the initial compensation coefficients corresponding to the value R, G, B, respectively, (i, j) is the coordinate position of a pixel, and R _ ori, G _ ori, B _ ori, and IR are the values R, G, B, IR, respectively.

2. The image adjustment method of claim 1, wherein the at least one threshold comprises a first overcompensation threshold, a second overcompensation threshold, an adjustment upper limit, and an adjustment lower limit; the step of comparing the overcompensation parameter with at least one threshold value to generate the compensation adjustment coefficient comprises:

comparing the overcompensation parameter with the first overcompensation threshold value and the second overcompensation threshold value, and obtaining the compensation adjustment coefficient with the magnitude between the upper adjustment limit value and the lower adjustment limit value according to the comparison result.

3. The image adjustment method of claim 2, wherein the step of comparing the overcompensation parameter with the first overcompensation threshold and the second overcompensation threshold and obtaining the compensation adjustment coefficient between the upper adjustment limit and the lower adjustment limit according to the comparison result is expressed by the following equation:

wherein TH1 and TH2 are the first overcompensation threshold and the second overcompensation threshold, W1 and W2 are the upper adjustment value and the lower adjustment value, X is the overcompensation parameter, and W _ final is the compensation adjustment coefficient.

4. The image adjustment method of claim 1, wherein the at least one threshold comprises a first overcompensation threshold, a second overcompensation threshold, an adjustment upper limit, and an adjustment lower limit; the step of comparing the overcompensation parameter with at least one threshold value to generate the compensation adjustment coefficient comprises:

comparing the overcompensation parameter with the first overcompensation threshold value and the second overcompensation threshold value, and obtaining the compensation adjustment coefficient with the magnitude between 1 and the adjustment lower limit value according to the comparison result.

5. The image adjustment method of claim 4, wherein the step of comparing the overcompensation parameter with the first overcompensation threshold and the second overcompensation threshold and obtaining the compensation adjustment coefficient between 1 and the adjustment threshold according to the comparison result is expressed by the following equation:

6. The image adjustment method according to claim 3 or 5, wherein the first overcompensation threshold and the second overcompensation threshold are 0.8 and 1.05, respectively, and the upper adjustment threshold and the lower adjustment threshold are 0.8 and 0.6, respectively.

7. The image adjustment method of claim 1, wherein the step of performing IR crosstalk compensation on the image using the compensation adjustment coefficients comprises:

generating final compensation parameters corresponding to the R, G, B values according to the compensation adjustment coefficients and the initial compensation parameters corresponding to the R, G, B values, wherein the final compensation parameters corresponding to the R, G, B values are expressed by the following equations:

k1_final＝k1(i,j)*W_final；

k2 — final — k2(i, j) × W — final; and

k3_final＝k3(i,j)*W_final；

where k1_ final, k2_ final, and k3_ final are the final compensation coefficients corresponding to R, G, B, respectively, and W _ final is the compensation adjustment coefficient.

8. The image adjustment method of claim 7, wherein the step of performing IR crosstalk compensation on the image using the compensation adjustment coefficient further comprises:

the final compensation coefficients corresponding to R, G, B are used to generate R, G, B values corresponding to R, G, B final outputs, wherein R, G, B values of the final outputs are expressed by the following equation:

R_final(i,j)＝R_ori(i,j)-k1_final*IR(i,j)；

g _ final (i, j) ═ G _ ori (i, j) -k2_ final IR (i, j); and

B_final(i,j)＝B_ori(i,j)-k3_final*IR(i,j)；

wherein R _ final (i, j), G _ final (i, j), B _ final (i, j) are R, G, B values of the final output, R _ ori (i, j), G _ ori (i, j), B _ ori (i, j), IR (i, j) are the R, G, B, IR values, wherein i, j are coordinate positions of the pixels.

9. An image processing circuit, comprising:

a storage unit for temporarily storing data; and

a processor for receiving an image and performing the following operations for each pixel of the image:

the following three products respectively corresponding to the R, G, B values are respectively generated according to the products of the initial compensation parameter corresponding to the R, G, B value and the IR value, and the largest of the three products is selected as the overcompensation parameter: