CN111629192B - Image adjusting method and related image processing circuit - Google Patents

Abstract

An image adjusting method includes: receiving an image, and acquiring a tristimulus value and an infrared light value from the image; dividing the image into a plurality of blocks; generating initial compensation coefficients respectively corresponding to the tristimulus values and the infrared light values for each of the blocks; and performing the following operations on each block of the blocks: adjusting the initial compensation coefficient of the current block according to the initial compensation coefficients of a plurality of adjacent blocks around the current block to generate an adjusted compensation coefficient of the current block; and using the adjusted compensation coefficient of the current block to perform infrared crosstalk compensation on the current block.

Description

Image adjusting method and related image processing circuit

Technical Field

The present invention relates to an image compensation technique, and more particularly, to a technique for compensating infrared Crosstalk (IR Crosstalk), which is suitable for a three primary color infrared sensor (rgbeir sensor) and can automatically calculate an appropriate infrared interference compensation parameter for asymmetry of the sensor and a lens.

Background

When there is an IR component in the ambient light, the light is absorbed by the IR pixels in the hybrid rgbirh filter array, however, in the three primary colors (R, G, B) and Infrared (IR) spectrum, crosstalk (crosstalk) occurs, which is referred to as the ideal spectrum in fig. 1 and the spectrum with crosstalk in fig. 2, because the overlap phenomenon shown in fig. 2 is caused by the interference between the IR and RGB lights. The current sensor manufacturing technology cannot effectively block or absorb non-color signals, so that when the light energy has a high IR component, the object color is affected by the IR crosstalk to generate a color fading phenomenon. The IR light may come from a face recognition system of a video system or from a yellowish light, such as that emitted by a halogen lamp.

US 20100289885 a1 discloses an IR crosstalk compensation technique, which subtracts a certain proportion of IR signal values from the output R, G, B signal values:

R_new＝R_ori-k1×IR_ori

G_new＝G_ori-k2×IR_ori

B_new＝B_ori-k3×IR_ori

IR_new＝IR_ori

wherein R is_new、G_new、B_new、IR_newAdjusted RGB values and IR values, R, respectively_ori、G_ori、B_ori、IR_oriThe original RGB value and IR value are respectively, and k1, k2, and k3 are fixed ratio parameters that the user sets according to the influence of the IR crosstalk. This prior art approach only roughly cancels the influence component of the infrared light, but does not adjust for the actual influence condition, and is more likely to cause over-compensation or color shift of RGB.

In addition, the above prior art method performs compensation to all pixels of the whole image to the same extent, and the effect caused by the infrared light is not uniformly distributed on the whole image, so the compensated image will appear very unnatural. Therefore, there is a need for a novel method that is relatively free of side effects to ameliorate the above problems.

Disclosure of Invention

An embodiment of the present invention provides an image adjusting method, including: receiving an image and obtaining three primary colors (R, G, B) and Infrared (IR) light values from the image; dividing the image into a plurality of blocks; generating initial compensation coefficients corresponding to the R, G, B, IR values for each of the blocks; and performing the following operations on each block of the blocks: adjusting the initial compensation coefficient of the current block according to the initial compensation coefficients of a plurality of adjacent blocks around the current block to generate an adjusted compensation coefficient of the current block; and using the adjusted compensation coefficient of the current block to perform infrared crosstalk compensation on the current block.

An embodiment of the present invention provides an image processing circuit, including a storage unit for temporarily storing data; and a processor configured to: receiving an image and obtaining R, G, B, IR values from the image; dividing the image into a plurality of blocks; generating initial compensation coefficients corresponding to the R, G, B, IR values for each of the blocks; and performing the following operations on each block of the blocks: adjusting the initial compensation coefficient of the current block according to the initial compensation coefficients of a plurality of adjacent blocks around the current block to generate an adjusted compensation coefficient of the current block; and using the adjusted compensation coefficient of the current block to perform infrared crosstalk compensation on the current block.

Drawings

FIG. 1 is a schematic diagram of an ideal spectrum;

FIG. 2 is a schematic diagram of a spectrum with crosstalk;

FIG. 3A is a flowchart illustrating an image compensation method according to an embodiment of the invention;

FIG. 3B is a diagram of the image processing circuit of FIG. 3A according to an embodiment of the present invention;

FIG. 4 is a schematic diagram of grid partitioning an image according to the present invention;

FIG. 5 is a diagram illustrating the location relationship between a specific tile B1 and neighboring tiles;

FIG. 6 is a schematic view of the embodiment corresponding to FIG. 5;

FIG. 7 is a schematic diagram of the compensation coefficient interpolation method.

Description of the symbols:

300 method

301. 302, 304, 306, 308, 310, 312

350 image processing circuit

352 storage unit

354 processor

Detailed Description

Certain terms are used throughout the description and following claims to refer to particular components. As one of ordinary skill in the art will appreciate, hardware manufacturers may refer to a component by different names. In the present specification and the claims that follow, elements are distinguished not by differences in name but by differences in function. In the following description and in the claims, the terms "include" and "comprise" are used in an open-ended fashion, and thus should be interpreted to mean "include, but not limited to. Additionally, the term "coupled" is used herein to encompass any direct or indirect electrical connection. Thus, if a first device couples to a second device, that connection may be through a direct electrical connection, or through an indirect electrical connection via other devices and connections.

When the quality of the camera lens or the image sensor is poor, the ir crosstalk may be asymmetric, where "asymmetric" means that the ir crosstalk is not uniform but is asymmetrically distributed on the left and right sides or the top and bottom sides of the screen, and in a serious case, the ir crosstalk may be asymmetric both on the top and bottom sides, so that the conventional ir crosstalk compensation coefficient cannot properly compensate the generated ir crosstalk without any side effect.

In view of the above, the present invention provides a grid (grid) -based infrared crosstalk compensation method, which can compensate for the grid-based infrared crosstalk according to the locality, and can properly compensate the entire screen even when the infrared crosstalk is severely asymmetric. The prior art does not consider the situation that when the same IR component is subtracted from the whole image, the color of the whole image will be non-uniform,

referring to fig. 3A, fig. 3A is a flowchart illustrating an image compensation method 300 according to an embodiment of the invention, wherein the steps are not necessarily performed according to the order of execution shown in fig. 3A if substantially the same result is obtained. The method shown in fig. 3A can be adopted by the image processing circuit 350 shown in fig. 3B, wherein the image processing circuit 350 includes a storage unit 352 and a processor 354, the storage unit 352 is used for temporarily storing data, the processor 354 is used for executing the image compensation method 300 and various related operations, and the image compensation method 300 can be briefly summarized as follows:

step 301: receiving an image;

step 302: carrying out grid partition on the image;

step 304: generating a set of initial compensation coefficients for the image;

step 306: finding out corresponding adjacent blocks;

step 308: interpolating a compensation coefficient of the current pixel;

step 310: the compensation coefficient is dynamically adjusted (this step can be executed or not according to the requirement);

step 312: and using the final compensation coefficient for infrared light crosstalk compensation.

The invention provides a method for correcting a compensation coefficient, which obtains a suitable infrared light crosstalk compensation coefficient through correction in advance and uses the corrected compensation coefficient to carry out infrared light crosstalk compensation. The correcting part can obtain the flat black picture card capable of reflecting IR light, then uses IR light to irradiate, and shoots the flat black picture card image, and uses the same module image to make compensation coefficient correction, wherein the IR image affected by IR and RGB image affected by IR can be obtained by shooting the flat black picture card image. This calibration method can be applied in step 304 to obtain more accurate results, however, the present invention is not limited to the above compensation factor calibration method.

Referring to fig. 4, the detailed procedure of step 302 is shown in fig. 4, which is a schematic diagram illustrating the grid partitioning of an image according to the present invention, wherein the image is divided into m × n blocks (each block having a size of p × q).

The compensation coefficient of each Block is obtained by calculating R, G, B, IR values of a Block (i, j) to obtain the average values Ravg (i, j), Gavg (i, j), Bavg (i, j), and IRavg (i, j) corresponding to the Block, wherein i and j are the index values of the Block (i, j) (representing the Block in the ith column and jth row), and the manner of obtaining these values can be referred to the related description of step 302. The equations for the infrared crosstalk compensation coefficients of R, G, B are respectively:

k1(i,j)＝A×Ravg(i,j)/IRavg(i,j)

k2(i,j)＝A×Gavg(i,j)/IRavg(i,j)

k3(i,j)＝A×Bavg(i,j)/IRavg(i,j)

wherein k1(i, j), k2(i, j), and k3(i, j) are the infrared crosstalk compensation coefficients of R, G, B of the Block (i, j), and a is a compensation correction coefficient (ranging from 0 to 1 floating point, which can be adjusted according to different modules), which can be defined according to the user's requirements.

The specific implementation of step 306 can refer to fig. 5 to 6, where fig. 5 is a position relationship diagram of the specific block B1 and the neighboring blocks, the top left, top right, bottom left, and bottom right block indexes corresponding to the specific block B1 are (i1, j1), (i2, j2), (i3, j3), (i4, j4), the current pixel image coordinate is (x, y), and the block index obtaining manner is the following manner, where p and q are the length and width of the block, respectively.

Fig. 6 shows an embodiment corresponding to fig. 5, where the blue dot is the center point of the neighboring block, and the red dot is the current pixel, where i1 is 2, j1 is 3, i2 is 2, j2 is 4, i3 is 3, j3 is 3, i4 is 3, and j4 is 4. After the neighboring block is obtained, step 308 is performed to interpolate the compensation coefficients to obtain the final value of the compensation coefficients of the current pixel. Referring to fig. 7, fig. 7 is a schematic diagram of the compensation coefficient interpolation method, in which a current pixel (coordinates (x, y)) is interpolated with four vertices neighboring an adjacent block as reference points, in detail as follows:

(1) calculating the vertical distances (i.e. normal lengths) D1-D4 between the current pixel and any two adjacent vertices, wherein D1-D4 are defined as follows:

d1 is the normal distance from the current pixel to the top left and top right (or the shortest distance from the current pixel to the top block);

d2 is the normal distance from the current pixel to the line connecting the lower left vertex and the lower right vertex (also understood as the shortest distance from the current pixel to the lower block);

d3 is the normal distance from the current pixel to the top left vertex and the bottom left vertex (or the shortest distance from the current pixel to the left block);

d4 is the normal distance from the current pixel to the top right vertex and the bottom right vertex (also understood as the shortest distance from the current pixel to the right block).

(2) Interpolating the current pixel compensation coefficient according to the distance, the detailed equation is as follows:

k1_intp＝(D2/(D1+D2))*(D4/(D3+D4))*k1(i1,j1)+(D2/(D1+D2))*(D3/(D3+D4))*k1(i2,j2)+(D1/(D1+D2))*(D4/(D3+D4))*k1(i3,j3)+(D1/(D1+D2))*(D3/(D3+D4))*k1(i4,j4)；

k2_intp＝(D2/(D1+D2))*(D4/(D3+D4))*k2(i1,j1)+(D2/(D1+D2))*(D3/(D3+D4))*k2(i2,j2)+(D1/(D1+D2))*(D4/(D3+D4))*k2(i3,j3)+(D1/(D1+D2))*(D3/(D3+D4))*k2(i4,j4)；

k3_intp＝(D2/(D1+D2))*(D4/(D3+D4))*k3(i1,j1)+(D2/(D1+D2))*(D3/(D3+D4))*k3(i2,j2)+(D1/(D1+D2))*(D4/(D3+D4))*k3(i3,j3)+(D1/(D1+D2))*(D3/(D3+D4))*k3(i4,j4)。

wherein k1_ intp (i, j), k2_ intp (i, j), and k3_ intp (i, j) are post-interpolation compensation coefficients of the infrared crosstalk of R, G, B of the Block (i, j), respectively, and k1, k2, and k3 are initial compensation coefficients corresponding to R, G, B, respectively, (i1, j1), (i2, j2), (i3, j3), and (i4, j4) are Block indexes of upper left, upper right, lower left, and lower right blocks of the neighboring blocks, respectively. However, for pixels in the edge or corner blocks, 4 vertices are not used for interpolation. For example, the interpolation of the pixels in the corner block only considers a single vertex, and the interpolation of the pixels in the edge block only considers 2 vertices.

Step 310 may be performed by selecting whether to perform or not, which may be considered as applying the user mode additionally to further adjust the compensation coefficients, in the following example, α 1, α 2, and α 3 are corner adjustment coefficients of R, G, B respectively, which are further dynamically adjusted according to the distance between the current block and the central block; β 1, β 2, β 3 are environment adjustment coefficients for further dynamically adjusting the obtained compensation coefficients for different scenes. The following procedure can be derived when both the scene and the distribution location are considered:

k1_final＝(k1_center+(k1_intp–k1_center)*α1)*β1；

k2_final＝(k2_center+(k2_intp–k2_center)*α2)*β2；

k3_final＝(k3_center+(k3_intp–k3_center)*α3)*β3。

wherein k1_ center, k2_ center, and k3_ center are IR crosstalk compensation coefficients of R, G, B in the central block

Finally, in step 312, the final compensation coefficient is used for infrared crosstalk compensation to generate R, G, B values to be outputted, which is given by the following equation:

R_final(x,y)＝R_ori(x,y)-k1_final*IR(x,y)；

G_final(x,y)＝G_ori(x,y)-k2_final*IR(x,y)；

B_final(x,y)＝B_ori(x,y)-k3_final*IR(x,y)；

wherein, R _ final (x, y), G _ final (x, y), B _ final (x, y) are R, G, B values of final output, and R _ ori (x, y), G _ ori (x, y), B _ ori (x, y), IR (x, y) are R, G, B, IR values measured at the beginning.

In summary, the present invention provides a grid-based method for automatically correcting an infrared crosstalk compensation coefficient, a grid-based method for compensating an infrared crosstalk, and a method for dynamically adjusting an infrared crosstalk coefficient. In addition, the method provided by the invention can be realized in a hardware mode and also can be realized in a software mode.

The above description is only a preferred embodiment of the present invention, and all equivalent changes and modifications made in accordance with the claims of the present invention should be covered by the present invention.

Claims (10)

1. An image adjusting method includes:

receiving an image, and obtaining three primary colors R, G, B and IR values from the image;

dividing the image into a plurality of blocks;

generating initial compensation coefficients corresponding to the R, G, B, IR values for each of the blocks; and

performing the following operations for each of the blocks:

adjusting the initial compensation coefficient of the current block according to the initial compensation coefficients of a plurality of adjacent blocks around the current block to generate an adjusted compensation coefficient of the current block; and

using the adjusted compensation coefficient of the current block to perform IR crosstalk compensation on the current block,

the infrared crosstalk compensation is obtained by subtracting the product of the compensation coefficient and the IR value from the original R, G, B value to obtain a new R, G, B value.

2. The image adjustment method of claim 1, wherein the step of adjusting the initial compensation coefficient of the current block according to the initial compensation coefficients of each of a plurality of neighboring blocks around the current block to generate the adjusted compensation coefficient of the current block comprises:

performing the following operations on each pixel in the current block:

calculating a plurality of distances between the current pixel and a plurality of adjacent blocks; and

adjusting the initial compensation coefficient of the current pixel according to the distances between the current pixel and the neighboring blocks to generate the adjusted compensation coefficient.

3. The image adjustment method of claim 2, wherein the step of adjusting the initial compensation coefficient of the current pixel according to the distances between the current pixel and neighboring blocks to generate the adjusted compensation coefficient comprises:

and performing interpolation operation on the initial compensation coefficients of the adjacent blocks according to the distance between the current pixel and the index point pixel of each of the adjacent blocks to generate the adjusted compensation coefficient of the current pixel.

4. The image adjustment method of claim 3, wherein the interpolation operation is expressed by the following equation:

k1_intp＝(D2/(D1+D2))*(D4/(D3+D4))*k1(i1,j1)+(D2/(D1+D2))*

(D3/(D3+D4))*k1(i2,j2)+(D1/(D1+D2))*(D4/(D3+D4))*

k1(i3,j3)+(D1/(D1+D2))*(D3/(D3+D4))*k1(i4,j4)；

k2_intp＝(D2/(D1+D2))*(D4/(D3+D4))*k2(i1,j1)+(D2/(D1+D2))*

(D3/(D3+D4))*k2(i2,j2)+(D1/(D1+D2))*(D4/(D3+D4))*

k2(i3, j3) + (D1/(D1+ D2)) (D3/(D3+ D4)). k2(i4, j 4); and

k3_intp＝(D2/(D1+D2))*(D4/(D3+D4))*k3(i1,j1)+(D2/(D1+D2))*

(D3/(D3+D4))*k3(i2,j2)+(D1/(D1+D2))*(D4/(D3+D4))*

k3(i3,j3)+(D1/(D1+D2))*(D3/(D3+D4))*k3(i4,j4)；

wherein k1 u_intp、k2__intp、k3__intpRespectively corresponding to R, G, B, and k1, k2, and k3 respectively corresponding to R, G, B, wherein (i1, j1), (i2, j2), (i3, j3), (i4, j4) are block indexes of upper left, upper right, lower left, and lower right blocks of the neighboring blocks,d1, D2, D3 and D4 are the shortest distances from the current pixel to the top, bottom, left and right blocks of the neighboring blocks, respectively.

5. The image adjustment method of claim 1, wherein the step of generating the initial compensation coefficients corresponding to R, G, B, IR values for each of the blocks further comprises:

color correction is performed to obtain the infrared light image affected by infrared light and the RGB image affected by infrared light, respectively, to generate the initial compensation coefficient corresponding to R, G, B, IR.

6. The image adjustment method of claim 1, wherein the step of adjusting the initial compensation coefficient of the current block according to the initial compensation coefficients of each of a plurality of neighboring blocks around the current block to generate the adjusted compensation coefficient of the current block comprises:

and dynamically adjusting the compensation coefficient of the adjusted compensation coefficient.

7. The image adjustment method of claim 6, wherein the step of dynamically adjusting the compensation coefficient further comprises:

generating corner adjustment coefficients in response to the distance between the current block and a central block of the blocks, so as to dynamically adjust the compensation coefficients.

8. The image adjustment method of claim 7, wherein the step of dynamically adjusting the compensation coefficient further comprises:

and generating an environment adjustment coefficient of the current environment to dynamically adjust the compensation coefficient.

9. The image adjustment method of claim 8, wherein the compensation factor is dynamically adjusted by the following equation:

k1_final＝(k1_center+(k1_intp–k1_center)*α1)*β1；

k2 — final (k2 — center + (k2 — intp-k 2 — center) × α 2) × β 2; and

k3_final＝(k3_center+(k3_intp–k3_center)*α3)*β3

wherein β 1, β 2, β 3 are environment adjustment coefficients corresponding to R, G, B, k1_ center, k2_ center, k3_ center are infrared crosstalk compensation coefficients corresponding to R, G, B in the central block, k1_ intp, k2_ intp, k3_ intp are interpolated compensation coefficients for infrared crosstalk of R, G, B in the block, α 1, α 2, α 3 are corner adjustment coefficients of R, G, B, and k1_ final, k2_ final, k3_ final are final compensation coefficients.

10. An image processing circuit, comprising:

a storage unit for temporarily storing data;

a processor configured to perform the following operations:

receiving an image, and obtaining three primary colors R, G, B and IR values from the image;

dividing the image into a plurality of blocks;

generating initial compensation coefficients corresponding to the R, G, B, IR values for each of the blocks; and

performing the following operations for each of the blocks:

adjusting the initial compensation coefficient of the current block according to the initial compensation coefficients of a plurality of adjacent blocks around the current block to generate an adjusted compensation coefficient of the current block; and

using the adjusted compensation coefficient of the current block to perform IR crosstalk compensation on the current block,

the infrared crosstalk compensation is obtained by subtracting the product of the compensation coefficient and the IR value from the original R, G, B value to obtain a new R, G, B value.

Images