TWI790732B - Image correction method and image correction device - Google Patents

Abstract

The disclosure provides an image correction method and an image correction device. The method includes: obtaining a scene image, which includes a physical color card; obtaining a shooting parameter and a scene parameter of the scene image; performing a linearization operation on the scene image to generate a first image, and recording a first linear operation parameter; generating a color correction matrix based on the first image and the physical color card; performing a non-linearization operation on the first image to generate a second image, and recording a non-linear operation parameter; and defining the shooting parametera and the scene parameter to correspond to the linear operation parameter, the color correction matrix, and the non-linear operation parameter.

Description

Image correction method and image correction device

The present invention relates to a calibration method and a calibration device, and in particular to an image calibration method and an image calibration device.

Nowadays, notebook computers, all-in-one (AIO) computers, smart glasses and other products are equipped with image capture devices. Users can use this type of device to perform related applications such as image storage, video conferencing, or augmented reality (AR) image capture.

Generally speaking, the color accuracy of an image capture device is an important indicator of its performance. Most image capture devices mainly perform color accuracy calibration through an image signal processor (ISP). However, the general calibration mechanism may have some limitations. For example, considering production resources, it is not necessary to make targeted adjustments to a single image capture device during production. In addition, the environment for generating color correction parameters used for color correction may be a light box or a specific scene, so the correction result may not be completely suitable for every image capture device, nor may it be in line with the actual commonly used environment of every user.

In view of this, the present invention provides an image correction method and an image correction device, which can be used to solve the above technical problems.

The present invention provides an image correction method suitable for an image correction device. The method includes: in response to determining that the image correction device is switched to a correction mode, an image capture circuit of the image correction device obtains a first scene image, wherein The first scene image includes a physical color card; obtaining a first shooting parameter and a first scene parameter of the first scene image; performing a first linearization operation on the first scene image to generate a first image, and Recording a first linear operation parameter of the first linearization operation; generating a color correction matrix based on the first image and the physical color card; performing a first nonlinearization relative to the first linearization operation on the first image operating to generate a second image, and recording a first nonlinear operation parameter of the first nonlinearization operation; and defining first capture parameters and first scene parameters as corresponding to the first linear operation parameter, color correction matrix and the first nonlinear operation parameter.

The invention provides an image correction device, which includes an image capturing circuit and a processor. The processor is coupled to the image capturing circuit and is configured to execute: in response to determining that the image correction device is switched to a calibration mode, control the image capturing circuit to obtain a first scene image, wherein the first scene image includes a physical color card ; Obtain a first shooting parameter and a first scene parameter of the first scene image; perform a first linearization operation on the first scene image to generate a first image, and record a first linearization operation A linear operation parameter; generate a color correction matrix based on the first image and the solid color card; perform a first nonlinear operation relative to the first linear operation on the first image to generate a second image, and record A first nonlinear operation parameter of the first nonlinear operation; and defining the first shooting parameter and the first scene parameter as corresponding to the first linear operation parameter, the color correction matrix and the first nonlinear operation parameter.

Please refer to FIG. 1 , which is a schematic diagram of an image calibration device according to an embodiment of the present invention. In different embodiments, the image calibration device 100 is, for example, various computer devices and/or smart devices with image capture functions, but is not limited thereto.

In FIG. 1 , an image capture device 100 may include an image capture circuit 102 and a processor 104 . In some embodiments, the image capturing circuit 102 is, for example, a charge coupled device (CCD) lens, a complementary metal oxide semiconductor transistors (CMOS) lens, or other image capturing devices that can be used to capture images. like devices, but not limited thereto.

The processor 104 is coupled to the imaging circuit 102, and can be a general-purpose processor, a special-purpose processor, a conventional processor, a digital signal processor, a plurality of microprocessors (microprocessor), one or more combined digital signal Microprocessors, controllers, microcontrollers, Application Specific Integrated Circuits (ASICs), Field Programmable Gate Arrays (Field Programmable Gate Arrays, FPGAs), any other kind of IC Circuits, state machines, Advanced RISC Machine (ARM) based processors, and the like.

In some embodiments, the image correction device 100 may further include a storage circuit coupled to the processor 104, such as any type of fixed or removable random access memory (Random Access Memory, RAM), read-only Memory (Read-Only Memory, ROM), flash memory (Flash memory), hard disk or other similar devices or a combination of these devices can be used to record multiple program codes or modules.

In an embodiment of the present invention, the processor 104 can access the modules and program codes recorded in the storage circuit to implement the image correction method proposed by the present invention, the details of which are described below.

Please refer to FIG. 2 , which is a flowchart of an image correction method according to an embodiment of the present invention. The method of this embodiment can be executed by the image calibration device 100 shown in FIG. 1 , and the details of each step in FIG. 2 will be described below with the components shown in FIG. 1 . In addition, in order to make the concept of the present invention easier to understand, the following description will be supplemented with FIG. 3 , wherein FIG. 3 is an application scenario diagram according to an embodiment of the present invention.

In a nutshell, the image calibration device 100 of the present invention can operate in an imaging mode and a calibration mode. When the image correction device 100 is switched to the correction mode, the image correction device 100 can obtain various appropriate color correction parameters based on the characteristics of the current scene, and after being switched to the image capture mode, correct the captured images based on these color correction parameters. to the image. This will be further explained below.

First, in step S210 , in response to determining that the image calibration device 100 is switched to the calibration mode, the processor 104 may control the image capturing circuit 102 to acquire the first scene image S1 . In an embodiment, the first scene image S1 may include, for example, a solid color card C1. In one embodiment, after the image calibration device 100 is switched to the calibration mode, the processor 104 may, for example, provide a prompt message to prompt the user 199 to place the solid color card C1 in the imaging range of the imaging circuit 102, The image capturing circuit 102 can capture an image including a solid color card as the first scene image C1, but it is not limited thereto. In some embodiments, the above prompt message may also require the user 199 to place the physical color card C1 in a designated area within the imaging range of the imaging circuit 102 , but it is not limited thereto.

In some embodiments, the first scene image S1 is, for example, an image that has been processed by an image signal processor (ISP). Furthermore, after the imaging circuit 102 obtains an original scene image, the ISP can perform some preliminary corrections on the original scene image (for example, perform some non-linearization operations), so as to correct the original scene image is the first scene image S1, but is not limited thereto.

After that, in step S220 , the processor 104 can obtain the first shooting parameter P11 and the first scene parameter P12 of the first scene image S1 . In different embodiments, the first shooting parameter P11 may include at least one of a white balance parameter and an exposure value when the imaging circuit 102 is shooting the first scene image S1. For the convenience of description, the following assumes that the first shooting parameters include the white balance parameter and the exposure value when the imaging circuit 102 shoots the first scene image S1, but it is not limited thereto.

，其中

為第一場景影像S1中暗像素的數量，

為第一場景影像S1中影像像素的數量，但可不限於此。 In some embodiments, the first scene parameter P12 is, for example, the dark pixel ratio of the first scene image S1. Specifically, the first scene image S1 may include a plurality of image pixels, and the image pixels include a plurality of dark pixels, and the grayscale values of each dark pixel are smaller than a brightness threshold. In this case, the dark pixel ratio can be characterized as:

,in

is the number of dark pixels in the first scene image S1,

is the number of image pixels in the first scene image S1, but is not limited thereto.

Afterwards, in step S230 , the processor 104 may perform a first linearization operation on the first scene image S1 to generate a first image IM1 , and record a first linear operation parameter PL1 of the first linearization operation. In an embodiment, assuming that the first linearization operation is Gamma 2.2, the corresponding first linear operation parameters may include parameters of the Gamma 2.2. In another embodiment, assuming that the first linearization operation is Gamma 2.4, the corresponding first linear operation parameter may include the Gamma 2.4 parameter, but is not limited thereto.

Afterwards, in step S240 , the processor 104 can generate a color correction matrix M1 based on the first image IM1 and the solid color card C1 . In an embodiment, the physical color card C1 may include one or more single color blocks, and the actual color of each single color block may be known.

In this case, the processor 104 may find a monochrome image area corresponding to one of the above-mentioned monochrome blocks (hereinafter referred to as a specific monochrome block) in the first image IM1 . Afterwards, the processor 104 can estimate the color correction matrix M1 based on the actual color of the specific monochrome block and the region color of the monochrome image region.

)，而上述單色影像區域的區域顏色可表徵為(

)。在此情況下，所述特定單色塊的實際顏色與上述單色影像區域的區域顏色之間的關係可表徵為下式(1)。

(1) In one embodiment, the actual color of the specific single color block can be characterized as (

), and the region color of the above monochrome image region can be characterized as (

). In this case, the relationship between the actual color of the specific monochrome block and the regional color of the monochrome image region can be expressed as the following formula (1).

(1)

Therefore, the processor 104 can deduce the content of the color correction matrix M1 based on the formula (1), and its specific method can refer to the correlation matrix calculation method in the prior art, and will not be repeated here.

In one embodiment, the color correction matrix M1 obtained through the above means can be understood as being obtained based on the current scene of the user 199, so when the image correction device 100 captures an image in the same scene in the future, the image correction device 100 will be The color correction of the image can be performed based on the color correction matrix M1, so that the corrected image can provide better color performance in the above scene.

Next, in step S250, the processor 104 may perform the first nonlinearization operation relative to the first linearization operation on the first image IM1 to generate the second image IM2, and record the first nonlinearization operation of the first nonlinearization operation. Non-linear operation parameter NPL1.

In one embodiment, assuming that the first linearization operation is Gamma 2.2, the first nonlinearization operation relative to the first linearization operation is, for example, Gamma 1/2.2, and its corresponding first nonlinear operation parameter NPL1 is, for example, a parameter of Gamma 1/2.2. In another embodiment, assuming that the first linearization operation is Gamma 2.4, the first nonlinear operation relative to the first linearization operation is, for example, Gamma 1/2.4, and its corresponding first nonlinear operation The parameter NPL1 is, for example, a parameter of Gamma 1/2.4, but is not limited thereto.

Afterwards, in step S260 , the processor 104 may define the first shooting parameter P11 and the first scene parameter P12 to correspond to the first linear operation parameter PL1 , the color correction matrix M1 and the first nonlinear operation parameter NPL1 . That is, the processor 104 can establish the corresponding relationship between "the first shooting parameter P11+the first scene parameter P12" and "the first linear operation parameter PL1, the color correction matrix M1, and the first nonlinear operation parameter NPL1", but It is not limited to this.

In one embodiment, the processor 104 may define the first shooting parameter P11 and the first scene parameter P12 in a lookup table to correspond to the first linear operation parameter PL1, the color correction matrix M1 and the first nonlinear operation parameter NPL1, but may not be limited to this.

Based on this, after the image correction device 100 is switched to the image capture mode, the processor 104 can correct the captured image based on the above look-up table, and the related details will be further described with reference to FIG. 4 .

Please refer to FIG. 4 , which is an application scenario diagram according to an embodiment of the present invention. In FIG. 4 , in response to determining that the image correction device 100 is switched to the image capturing mode, the processor 104 may control the image capturing circuit 102 to obtain the second scene image S2 . After that, the processor 104 can obtain the second shooting parameter P21 and the second scene parameter P22 of the second scene image. In different embodiments, the second shooting parameter P21 may correspond to the attribute of the first shooting parameter P11 and may include at least one of a white balance parameter and an exposure value when the image capturing circuit 102 shoots the second scene image S2.

In one embodiment, since the first shooting parameter P11 is assumed to be the white balance parameter and exposure value when the image capturing circuit 102 captures the first scene image S1, the second shooting parameter P21 can be the image capturing circuit 102 shooting the second scene. The white balance parameter and exposure value of image S2.

In one embodiment, the processor 104 may obtain the second scene parameter P22 of the second scene image S2 (for example, the dark pixel ratio in the second scene image S2) based on the method of obtaining the first scene parameter P12 of the first scene image S1 ).

Afterwards, the processor 104 can determine whether the second shooting parameter P21 and the second scene parameter P22 correspond to the first shooting parameter P11 and the first scene parameter P12 respectively.

In one embodiment, the processor 104 can determine whether the first error between the second shooting parameter P21 and the first shooting parameter P11 is smaller than a first threshold value, and the difference between the second scene parameter P22 and the first scene parameter P12 Whether the second error between them is smaller than a second threshold value. In one embodiment, in response to determining that the first error between the second shooting parameter P21 and the first shooting parameter P11 is smaller than the first threshold value, and the second error between the second scene parameter P22 and the first scene parameter P12 If the error is smaller than the second threshold value, the processor 104 can determine that the second shooting parameter P21 and the second scene parameter P22 correspond to the first shooting parameter P11 and the first scene parameter P12 respectively, and vice versa.

In one embodiment, in response to determining that the second shooting parameter P21 and the second scene parameter P22 correspond to the first shooting parameter P11 and the first scene parameter P12 respectively, it means that the shooting environment of the second scene image S2 corresponds to the first shooting environment. The shooting environment of a scene image S1. In this case, the processor 104 may perform color correction on the second scene image S2 based on the previously acquired color correction matrix M1.

In one embodiment, the processor 104 can obtain the first linear operation parameter PL1, the color correction matrix M1 and the first nonlinear operation parameter NPL1 corresponding to the first shooting parameter P11 and the first scene parameter P12 from the above lookup table , and correct the second scene image S2 into a third scene image S3 based on the first linear operation parameter PL1 , the color correction matrix M1 and the first nonlinear operation parameter NPL1 .

As shown in FIG. 4 , the processor 104 may perform the first linearization operation on the second scene image S2 based on the first linear operation parameter PL1 to generate the third image IM3, and convert the third image IM3 based on the color correction matrix M1 Corrected to the fourth image IM4.

In one embodiment, the processor 104 may multiply each pixel in the third image IM3 by the color correction matrix M1 to correct the third image IM3 into the fourth image IM4 , but it is not limited thereto.

Afterwards, the processor 104 may perform the first nonlinear operation on the fourth image IM4 based on the first nonlinear operation parameter NPL1 to generate the third scene image S3. In one embodiment, the processor 104 may display the third scene image S3 for the user 199 to watch. In this way, the user 199 can watch the third scene image S3 that can provide better color performance in the current scene, so as to obtain a better viewing experience.

To sum up, in the correction mode, the image correction device of the present invention can obtain the scene image including the physical color card, according to the actual color of the single color block in the physical color card and the color of the corresponding image area in the captured image. The difference between estimates applies the color correction matrix for the current scene. Afterwards, in the imaging mode, when the image correction device shoots an image in the same scene, the image correction device can perform color correction of the image based on the above color correction matrix, so that the corrected image can be provided in the above scene Better color performance.

Although the present invention has been disclosed above with the embodiments, it is not intended to limit the present invention. Anyone with ordinary knowledge in the technical field may make some changes and modifications without departing from the spirit and scope of the present invention. The scope of protection of the present invention should be defined by the scope of the appended patent application.

100: image capture device 102: Image acquisition circuit 104: Processor 199: user C1: solid color card IM1: first image M1: Color Correction Matrix NPL1: The first nonlinear operation parameter PL1: first linear operation parameter S1: First scene image S2: Second scene image S3: The third scene image S210~S260: Steps

FIG. 1 is a schematic diagram of an image correction device according to an embodiment of the present invention. FIG. 2 is a flowchart of an image correction method according to an embodiment of the present invention. Fig. 3 is an application scenario diagram according to an embodiment of the present invention. Fig. 4 is an application scenario diagram according to an embodiment of the present invention.

S210~S260: Steps

Claims (11)

An image correction method suitable for an image correction device, the method comprising: in response to the image correction device determining that the image correction device has switched to a correction mode, obtaining a first scene by an image pickup circuit of the image correction device An image, wherein the first scene image includes a physical color card, and the physical color card includes a specific single color block; a first shooting parameter and a first scene parameter of the first scene image are obtained by the image correction device; performing a first linearization operation on the first scene image by the image correction device to generate a first image, and recording a first linear operation parameter of the first linearization operation; by the image correction device based on The first image and the physical color card generate a color correction matrix, including: finding a monochrome image area corresponding to the specific monochrome block in the first image, and based on the actual color and the specific monochrome block. estimating the color correction matrix for a region color of the monochrome image region; performing a first nonlinearization operation on the first image relative to the first linearization operation by the image correction device to generate a second image, and recording a first nonlinear operation parameter of the first nonlinear operation; and defining the first shooting parameter and the first scene parameter in a look-up table by the image correction device as corresponding to the first linear operation parameters, the color correction matrix and The first nonlinear operation parameter. The method as described in Claim 1, further comprising: acquiring a second scene image by the imaging circuit in response to determining that the image correction device is switched to an imaging mode; acquiring a second shot of the second scene image parameter and a second scene parameter; in response to determining that the second shooting parameter and the second scene parameter correspond to the first shooting parameter and the first scene parameter respectively, obtaining the corresponding first shooting parameter in the lookup table and the first linear operation parameter, the color correction matrix, and the first nonlinear operation parameter of the first scene parameter; based on the first linear operation parameter, the color correction matrix and the first nonlinear operation parameter, the The second scene image is corrected into a third scene image. The method as described in claim 2, comprising: in response to determining that a first error between the second shooting parameter and the first shooting parameter is smaller than a first threshold value, and the second scene parameter and the first A second error between the scene parameters is smaller than a second threshold value, and it is determined that the second shooting parameter and the second scene parameter correspond to the first shooting parameter and the first scene parameter respectively. The method as described in claim 2, wherein the step of correcting the second scene image based on the first linear operation parameter, the color correction matrix and the first nonlinear operation parameter comprises: based on the first linear operation parameter to performing the first linearization operation on the second scene image to generate a third image; correcting the third image to a fourth image based on the color correction matrix; The first nonlinear operation is performed on the fourth image based on the first nonlinear operation parameter to generate the third scene image. The method as claimed in claim 1, wherein the first scene image is an image processed by an image signal processor. The method according to claim 1, wherein the first shooting parameter includes at least one of a white balance parameter and an exposure value when the imaging circuit captures the first scene image. The method according to claim 1, wherein the first scene image includes a plurality of image pixels, the image pixels include a plurality of dark pixels, and the first scene parameter of the first scene image includes a parameter of the first scene image Dark pixel ratio, where the dark pixel ratio is characterized by:

, wherein N _DarkPixel is the number of these dark pixels, N _GlobalImage is the number of these image pixels, and the grayscale value of each dark pixel is smaller than a brightness threshold. The method according to claim 1, wherein the first linearization operation includes Gamma 2.2, the first linear operation parameters include parameters of the Gamma 2.2, the first non-linear operation includes Gamma 1/2.2, the first A nonlinear operation parameter includes the Gamma 1/2.2 parameter. The method according to claim 1, wherein the first linearization operation includes Gamma 2.4, the first linear operation parameters include parameters of the Gamma 2.4, the first A non-linear operation includes Gamma 1/2.4, and the first non-linear operation parameter includes parameters of the Gamma 1/2.4. The method as described in claim 1, wherein the actual color of the specific monochrome block is represented by ( R _TG , G _TG , B _TG ), and the regional color of the monochrome image area is represented by ( R _SRC , G _SRC , B _SRC ),in:

, where M1 is the color correction matrix. An image correction device, comprising: an image capturing circuit; a processor coupled to the image capturing circuit and configured to execute: responding to determining that the image correction device is switched to a calibration mode, controlling the image capturing circuit Obtain a first scene image, wherein the first scene image includes a physical color card, and the physical color card includes a specific single color block; obtain a first shooting parameter and a first scene parameter of the first scene image; Execute a first linearization operation on the first scene image to generate a first image, and record a first linear operation parameter of the first linearization operation; generate based on the first image and a physical color card A color correction matrix, comprising: finding a monochrome image region corresponding to the specific monochrome patch in the first image, and estimating a region color based on the actual color of the specific monochrome patch and the monochrome image region the color correction matrix; performing a first nonlinearity on the first image relative to the first linearization operation transform operation to generate a second image, and record a first nonlinear operation parameter of the first nonlinear operation; and define the first shooting parameter and the first scene parameter in a lookup table as corresponding to the The first linear operation parameter, the color correction matrix and the first nonlinear operation parameter.

Images