WO2022246950A1

WO2022246950A1 - Imaging correction method and apparatus, and intelligent terminal

Info

Publication number: WO2022246950A1
Application number: PCT/CN2021/103646
Authority: WO
Inventors: 胡泽
Original assignee: 东莞市小精灵教育软件有限公司
Priority date: 2021-05-28
Filing date: 2021-06-30
Publication date: 2022-12-01
Also published as: CN113301222A; CN113301222B

Abstract

An imaging correction method and apparatus, and an intelligent terminal. The method comprises: monitoring the current temperature of an image sensor by means of a temperature sensor; obtaining an adjustment value for a background image according to the current temperature of the image sensor and stored dark current calibration values of a plurality of preset temperatures; and according to the background image and the adjustment value for the background image, optimizing an original image collected by the image sensor, and outputting a target image. The influence of a dark current can be adaptively adjusted according to the temperature of an image sensor, thereby improving the quality of a formed image.

Description

An imaging correction method and device, and an intelligent terminal

technical field

The invention relates to the technical field of intelligent terminals, in particular to an imaging correction method and device, and an intelligent terminal.

Background technique

Portable smart terminals, such as smart watches and smart phones, have become indispensable in daily life. Taking photos/video calls and the like are very important functions in the smart terminal, so the smart terminal is usually equipped with a camera.

Due to the influence of impurities, heat, etc., physical devices cannot be absolutely ideal. The image sensor (such as CMOS sensor, CCD sensor) in the camera will also generate charge when there is no light shining on the pixel unit. The current generated by these charges is called dark current. Among them, the dark current and the charge generated by light are difficult to distinguish.

On the same image sensor, the dark current of each pixel is not the same, some are low, some are high. Dark current may overwhelm the real light signal, causing image quality degradation.

In order to eliminate the influence of dark current, the method of background subtraction is currently used to correct the imaging. Acquire an image in a non-illuminated environment (that is, under total darkness), that is, an image caused by dark current, as a background image. The pixel value of each pixel unit in the background image is the corresponding dark current value. When taking a sample, automatically subtract the background image and output the image after background subtraction.

The magnitude of the dark current is closely related to the temperature, the higher the temperature, the greater the dark current. Due to some operations on the smart terminal, such as long call time, playing games, watching videos, etc., the temperature of the device is high. Even if the camera is far away from the processor, the temperature of the image sensor will be high and the dark current will be large. If you shoot at this time, Even with a fixed background image, the image quality will still be poor. This phenomenon is even more serious in small devices, such as smart watches.

Therefore, it is necessary to provide a better method to solve the problem that the images collected by cameras on smart terminals such as smart watches or mobile phones are affected by temperature, the distortion effect is serious, and the obtained pictures are not clear enough.

Contents of the invention

One of the objectives of the present invention is to provide an imaging correction method and device, and an intelligent terminal in order to overcome at least some of the deficiencies in the prior art.

The technical scheme provided by the invention is as follows:

An imaging correction method for an intelligent terminal, the intelligent terminal includes a camera and a temperature sensor, and the camera includes an image sensor. The imaging correction method includes: monitoring the current temperature of the image sensor through the temperature sensor;

Obtain an adjustment value of the background image according to the current temperature of the image sensor and stored dark current calibration values of several preset temperatures;

Optimizing the original image collected by the image sensor according to the background image and the adjustment value of the background image, and outputting a target image.

Further, before obtaining the adjustment value of the background image, it also includes:

adjusting the temperature of the image sensor to a preset temperature;

measuring a dark current value at a preset point of the image sensor at the preset temperature, and using it as a dark current calibration value at the preset temperature;

The preset temperature is changed, and the above process is repeated to obtain dark current calibration values at several preset temperatures.

Further, according to the current temperature of the image sensor and stored dark current calibration values of several preset temperatures, the adjustment value of the background image is obtained, including:

If the current temperature of the image sensor is not equal to the stored preset temperature, then apply an interpolation algorithm to calculate the dark current value corresponding to the current temperature according to the dark current calibration value of the stored preset temperature; The dark current value corresponding to the temperature is used to obtain the adjusted value of the background image.

Further, the dark current value corresponding to the current temperature is calculated by applying an interpolation algorithm according to the stored dark current calibration value of the preset temperature, including:

Obtaining the first two stored preset temperatures closest to the current temperature;

According to the dark current calibration values of the two preset temperatures, the dark current value corresponding to the current temperature is calculated.

Further, the monitoring the current temperature of the image sensor by the temperature sensor includes: obtaining the current temperature of the image sensor according to the temperature measured by the temperature sensor and a temperature distribution model inside the smart terminal.

The present invention also provides an imaging correction device for an intelligent terminal, the intelligent terminal includes a camera and a temperature sensor, and the camera includes an image sensor. The imaging correction device includes: a temperature monitoring module, configured to monitor the current temperature of the image sensor through the temperature sensor;

A dark current adjustment module, configured to obtain an adjustment value of the background image according to the current temperature of the image sensor and stored dark current calibration values of several preset temperatures;

The imaging correction module is configured to optimize the original image collected by the image sensor according to the background image and the adjustment value of the background image, and output a target image.

Further, it also includes:

A measurement module, configured to adjust the temperature of the image sensor to a preset temperature; measure a dark current value at a preset point of the image sensor at the preset temperature, and use it as a dark current calibration value at the preset temperature ; Change the preset temperature, repeat the above process, and obtain dark current calibration values of several preset temperatures.

Further, the dark current adjustment module is further configured to apply an interpolation algorithm to calculate the dark current calibration value of the stored preset temperature if the current temperature of the image sensor is not equal to the stored preset temperature. The dark current value corresponding to the current temperature; the adjustment value of the background image is obtained according to the dark current value corresponding to the current temperature.

Further, the dark current adjustment module is also used to obtain the first two stored preset temperatures closest to the current temperature; and calculate the current temperature according to the dark current calibration values of the two preset temperatures. The dark current value corresponding to the temperature.

The present invention also provides an intelligent terminal, including: a memory for storing a computer program; and a processor for implementing the aforementioned imaging correction method when running the computer program.

The imaging correction method and device and the intelligent terminal provided by the present invention can at least bring about the following beneficial effects: by monitoring the working temperature of the image sensor chip in the camera and combining the temperature-dark current characteristics of the chip, the background image can be adaptively adjusted , by subtracting the background image from the original image collected by the image sensor, thereby eliminating the influence of dark current and obtaining a better imaging effect.

Description of drawings

In the following, preferred embodiments will be described in a clear and understandable manner with reference to the accompanying drawings, and the above-mentioned characteristics, technical features, advantages and implementation methods of an imaging correction method and device, and an intelligent terminal will be further described.

Fig. 1 is a flowchart of an embodiment of an imaging correction method of the present invention;

Fig. 2 is a flow chart of another embodiment of an imaging correction method of the present invention;

Fig. 3 is a structural schematic diagram of an embodiment of an imaging correction device of the present invention;

Fig. 4 is a schematic structural diagram of another embodiment of an imaging correction device of the present invention;

Fig. 5 is a schematic structural diagram of an embodiment of an intelligent terminal according to the present invention.

Explanation of reference numbers:

100. Temperature monitoring module, 200. Dark current adjustment module, 300. Imaging correction module, 400. Measurement module, 20. Intelligent terminal, 21. Memory, 22. Processor, 23. Computer program.

Detailed ways

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the specific implementation manners of the present invention will be described below with reference to the accompanying drawings. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention, and those skilled in the art can obtain other accompanying drawings based on these drawings and obtain other implementations.

In order to make the drawing concise, each drawing only schematically shows the parts related to the present invention, and they do not represent the actual structure of the product. In addition, to make the drawings concise and easy to understand, in some drawings, only one of the components having the same structure or function is schematically drawn or only one of them is marked. Herein, "a" not only means "only one", but also means "more than one".

In one embodiment of the present invention, as shown in FIG. 1 , an imaging correction method is used in a smart terminal, the smart terminal includes a camera and a temperature sensor, and the camera includes an image sensor. The imaging correction method includes:

Step S200 monitors the current temperature of the image sensor through the temperature sensor.

In order to monitor the working temperature of the image sensor, a temperature sensor can be specially provided, but this will increase the cost of the smart terminal.

There is usually a temperature sensor in the smart terminal for monitoring the temperature of the heat source point, and the heat source point is generally a CPU (Central Processing Unit) chip. The image sensor is generally set on the edge of the smart terminal, and there may be a certain distance from the temperature sensor. At this time, the current temperature of the image sensor can be obtained according to the temperature distribution model inside the smart terminal and the temperature of the heat source point measured by the temperature sensor. The temperature distribution model can be obtained according to the internal structure of the smart terminal and historical temperature distribution data. In this way, it is unnecessary to add a temperature sensor specially for monitoring the image sensor, thereby not increasing the cost of the smart terminal.

Step S300 obtains the adjustment value of the background image according to the current temperature of the image sensor and stored dark current calibration values of several preset temperatures.

The background image refers to an image collected by the image sensor at a default temperature in an environment without light, and the image is caused by dark current. A normal operating temperature of the image sensor can be used as the default temperature.

Since the magnitude of the dark current is closely related to the temperature, the higher the temperature, the greater the dark current. The background image is the dark current representation of the image sensor at the default temperature. Therefore, the pixels of the background image need to be adjusted according to the current temperature of the image sensor.

Several temperature points and dark current calibration values of these temperature points are pre-stored inside the smart terminal. According to these stored data, the dark current value corresponding to the current temperature of the image sensor is obtained. The dark current calibration value or dark current value may be an average dark current value of the image sensor at a corresponding temperature.

By storing several temperature points and their dark current calibration values, it is equivalent to storing the temperature-dark current characteristics of the image sensor inside the smart terminal.

If the current temperature of the image sensor is equal to a stored preset temperature, the dark current value corresponding to the current temperature is equal to the dark current calibration value of the preset temperature.

If the current temperature of the image sensor is not equal to any stored preset temperature, an interpolation algorithm is applied to calculate the dark current value corresponding to the current temperature according to the stored dark current calibration value of the preset temperature.

For example, a temperature-dark current fitting curve is obtained according to the stored preset temperature and its dark current calibration value, and the dark current value corresponding to the current temperature is calculated using a function expressing the fitting curve.

A simpler and more effective method is to obtain the first two stored preset temperatures closest to the current temperature; and calculate the dark current value corresponding to the current temperature according to the dark current calibration values of the two preset temperatures. For example, linear interpolation is used to obtain a straight line according to the two preset temperatures and their dark current calibration values, and the dark current value corresponding to the current temperature is calculated according to the straight line.

According to the dark current value corresponding to the current temperature, the adjustment value of the background image is obtained.

Specifically, according to the dark current value corresponding to the current temperature and the dark current value at the default temperature, the dark current deviation value caused by temperature is obtained, and the pixels of the background image are adjusted according to the dark current deviation value.

Step S400 optimizes the original image collected by the image sensor according to the background image and the adjustment value of the background image, and outputs the target image.

Specifically, according to the adjustment value of the background image, the pixels of the background image are adjusted to obtain an updated background image. Then subtract the updated background image from the original image collected by the image sensor, so as to obtain the target image that eliminates the influence of dark current.

In this embodiment, by monitoring the temperature of the image sensor in the camera, combined with the stored temperature-dark current characteristics of the image sensor, the background image is adaptively adjusted, and then the adjusted background image is deducted from the original image collected by the image sensor, thereby eliminating The effect of dark current can get better imaging effect.

In another embodiment of the present invention, as shown in FIG. 2 , an imaging correction method is used in a smart terminal, the smart terminal includes a camera and a temperature sensor, and the camera includes an image sensor.

The imaging correction method includes:

Step S100 adjusts the temperature of the image sensor to a preset temperature;

Step S110 Measure the dark current value of a preset point of the image sensor at a preset temperature, and use it as the corresponding dark current calibration value;

Step S120 changes the preset temperature, repeats the above process, and obtains dark current calibration values at several preset temperatures.

Specifically, the temperature of the image sensor can be changed by adjusting the ambient temperature around the image sensor to reach a preset temperature.

Select a preset point in the image sensor, for example, an area most affected by dark current, and measure its dark current value at a preset temperature. The dark current value needs to be measured in an environment without light.

Change the preset temperature, and measure the dark current value at the preset point under the new preset temperature. By repeating the above process, dark current calibration values at several preset temperatures are obtained, which is equivalent to obtaining the temperature-dark current characteristic of the image sensor.

It is also possible to select multiple preset points in the image sensor, for example, the point most affected by dark current, the point moderately affected by dark current, and the point slightly affected by dark current, and measure the dark current values of these preset points respectively, The dark current values at these preset points are averaged, and the average value is used as the dark current calibration value at the preset temperature. Change the preset temperature and repeat the above process to obtain dark current calibration values for several preset temperatures.

The latter way can more accurately reflect the temperature-dark current characteristics of the image sensor.

Step S210 obtains the current temperature of the image sensor according to the temperature measured by the temperature sensor and the temperature distribution model inside the smart terminal.

There is usually a temperature sensor in the smart terminal to monitor the temperature of the heat source point, and the heat source point is usually a CPU (central processing unit) chip. The image sensor is generally set on the edge of the smart terminal, and there may be a certain distance from the temperature sensor. At this time, the current temperature of the image sensor can be obtained according to the temperature distribution model inside the smart terminal and the temperature of the heat source point measured by the temperature sensor. The temperature distribution model can be obtained according to the internal structure of the smart terminal and historical temperature distribution data. In this way, it is unnecessary to add a temperature sensor specially for monitoring the image sensor, thereby not increasing the cost of the smart terminal.

A simpler and more effective method is to obtain the first two stored preset temperatures closest to the current temperature; calculate the dark current value corresponding to the current temperature according to the dark current calibration values of the two preset temperatures. For example, linear interpolation is used to obtain a straight line according to the two preset temperatures and their dark current calibration values, and the dark current value corresponding to the current temperature is calculated according to the straight line.

In this embodiment, the temperature-dark current characteristics of the image sensor are obtained through multiple measurements, and stored in the smart terminal; the smart terminal monitors the temperature of the image sensor in the camera, combined with the stored temperature-dark current characteristics of the image sensor, Adaptively adjust the background image, and then subtract the adjusted background image from the original image collected by the image sensor, thereby eliminating the influence of dark current and obtaining better imaging results.

In one embodiment of the present invention, as shown in FIG. 3 , an imaging correction device is used in a smart terminal, the smart terminal includes a camera and a temperature sensor, and the camera includes an image sensor.

The image correction device includes:

The temperature monitoring module 100 is configured to monitor the current temperature of the image sensor through the temperature sensor.

There is usually a temperature sensor in the smart terminal for monitoring the temperature of the heat source point, and the heat source point is generally a CPU (Central Processing Unit) chip. The image sensor is generally set on the edge of the smart terminal, and may have a certain distance from the temperature sensor. At this time, the temperature monitoring module 100 can obtain the current temperature of the image sensor according to the temperature distribution model inside the smart terminal and the temperature of the heat source point measured by the temperature sensor. temperature. The temperature distribution model can be obtained according to the internal structure of the smart terminal and historical temperature distribution data. In this way, it is unnecessary to add a temperature sensor specially for monitoring the image sensor, thereby not increasing the cost of the smart terminal.

The dark current adjustment module 200 is configured to obtain an adjustment value of the background image according to the current temperature of the image sensor and stored dark current calibration values of several preset temperatures.

The imaging correction module 300 is configured to optimize the original image collected by the image sensor and output the target image according to the background image and the adjustment value of the background image.

In another embodiment of the present invention, as shown in FIG. 4 , an imaging correction device is used in a smart terminal, the smart terminal includes a camera and a temperature sensor, and the camera includes an image sensor.

The image correction device includes:

The measurement module 400 is used to adjust the temperature of the image sensor to a preset temperature; measure the dark current value of a preset point of the image sensor at the preset temperature, and use it as the corresponding dark current calibration value; change the preset temperature, and repeat the above process , to get the dark current calibration value of several preset temperatures.

The temperature monitoring module 100 is configured to obtain the current temperature of the image sensor according to the temperature measured by the temperature sensor and the temperature distribution model inside the smart terminal.

Several temperature points and dark current calibration values of these temperature points are pre-stored inside the smart terminal. According to these stored data, the dark current value corresponding to the current temperature of the image sensor is obtained. By storing several temperature points and their dark current calibration values, it is equivalent to storing the temperature-dark current characteristics of the image sensor inside the smart terminal.

It should be noted that the embodiment of the imaging correction device provided by the present invention is based on the same inventive concept as the embodiment of the imaging correction method provided above, and can achieve the same technical effect. Therefore, other specific content of the embodiment of the imaging correction device can refer to the description of the content of the embodiment of the aforementioned imaging correction method.

In an embodiment of the present invention, as shown in FIG. 5 , an intelligent terminal 20 includes a memory 21 and a processor 22 . The memory 21 is used to store a computer program 23 . When the processor 22 runs the computer program 23, the imaging correction method as described above is implemented.

The smart terminal 20 can be a smart watch, a smart phone, and the like.

As an example, when the processor 21 executes the computer program, the steps S200 to S400 according to the foregoing description are realized. In addition, when the processor 21 executes the computer program, it realizes the functions of the various modules in the aforementioned imaging correction device. As yet another example, the processor implements the functions of the temperature monitoring module 100 , the dark current adjustment module 200 , and the imaging correction module 300 when executing the computer program.

Optionally, the computer program can be divided into one or more modules/units according to the specific requirements for implementing the present invention. Each module/unit can be a series of computer program instruction segments capable of performing a specific function. The computer program instruction segment is used to describe the execution process of the computer program in the imaging correction device.

As an example, the computer program may be divided into various modules/units in the virtual device, such as a temperature monitoring module, a dark current adjustment module, and an imaging correction module.

The processor is configured to realize imaging correction by executing the computer program. As required, the processor can be a central processing unit (CPU), a graphics processing unit (GPU), a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), a general purpose processor or other logic devices, etc.

The memory may be any internal storage unit and/or external storage device capable of storing data and programs. For example, the memory may be a plug-in hard disk, a smart memory card (SMC), a secure digital (SD) card, or a flash memory card. The memory is used to store computer programs and data.

According to needs, the smart terminal 20 may also include input and output devices, display devices, network access devices, buses, and the like.

The smart terminal 20 can also be a single-chip microcomputer, or a computing device integrating a central processing unit (CPU) and a graphics processing unit (GPU).

Those skilled in the art can understand that the division of the above-mentioned units and modules for realizing corresponding functions is for the purpose of convenience of description and description, and according to application requirements, the above-mentioned units and modules are further divided or combined, that is, the device / The internal structure of the equipment is re-divided and combined to realize the above-mentioned functions.

The units and modules in the above embodiments may be separate physical units, or two or more units and modules may be integrated into one physical unit. Each unit and module of the above-mentioned embodiments may use hardware and/or software functional units to implement corresponding functions. The direct coupling, indirect coupling or communication connection between multiple units, components and modules in the above embodiments can be realized through buses or interfaces; the coupling and connection between multiple units or devices can be electrical, mechanical or similar way. Correspondingly, the specific names of the various units and modules in the above embodiments are only for convenience of description and distinction, and do not limit the protection scope of the present application.

It should be noted that the above embodiments can be freely combined as required. The above is only a preferred embodiment of the present invention, it should be pointed out that, for those of ordinary skill in the art, without departing from the principle of the present invention, some improvements and modifications can also be made, and these improvements and modifications can also be made. It should be regarded as the protection scope of the present invention.

Claims

An imaging correction method for an intelligent terminal, the intelligent terminal comprising a camera and a temperature sensor, the camera comprising an image sensor, characterized in that the imaging correction method comprises:

monitoring the current temperature of the image sensor through the temperature sensor;

Obtain an adjustment value of the background image according to the current temperature of the image sensor and stored dark current calibration values of several preset temperatures;

Optimizing the original image collected by the image sensor according to the background image and the adjustment value of the background image, and outputting a target image.
The imaging correction method according to claim 1, further comprising: before said obtaining the adjustment value of the background image:

adjusting the temperature of the image sensor to a preset temperature;

measuring a dark current value at a preset point of the image sensor at the preset temperature, and using it as a dark current calibration value at the preset temperature;

The preset temperature is changed, and the above process is repeated to obtain dark current calibration values at several preset temperatures.
The imaging correction method according to claim 1, wherein the adjustment value of the background image is obtained according to the current temperature of the image sensor and stored dark current calibration values of several preset temperatures, including:

If the current temperature of the image sensor is not equal to the stored preset temperature, an interpolation algorithm is applied to calculate the dark current value corresponding to the current temperature according to the stored dark current calibration value of the preset temperature;

The adjustment value of the background image is obtained according to the dark current value corresponding to the current temperature.
The imaging correction method according to claim 3, wherein the dark current value corresponding to the current temperature is calculated by applying an interpolation algorithm according to the stored dark current calibration value of the preset temperature, including:

Obtaining the first two stored preset temperatures closest to the current temperature;

According to the dark current calibration values of the two preset temperatures, the dark current value corresponding to the current temperature is calculated.
The imaging correction method according to claim 1, wherein the monitoring the current temperature of the image sensor through the temperature sensor comprises:

The current temperature of the image sensor is obtained according to the temperature measured by the temperature sensor and the temperature distribution model inside the smart terminal.
An imaging correction device for an intelligent terminal, the intelligent terminal comprising a camera and a temperature sensor, the camera comprising an image sensor, characterized in that the imaging correction device comprises:

a temperature monitoring module, configured to monitor the current temperature of the image sensor through the temperature sensor;

A dark current adjustment module, configured to obtain an adjustment value of the background image according to the current temperature of the image sensor and stored dark current calibration values of several preset temperatures;

The imaging correction module is configured to optimize the original image collected by the image sensor according to the background image and the adjustment value of the background image, and output a target image.
The imaging correction device according to claim 6, further comprising:

A measurement module, configured to adjust the temperature of the image sensor to a preset temperature; measure a dark current value at a preset point of the image sensor at the preset temperature, and use it as a dark current calibration value at the preset temperature ; Change the preset temperature, repeat the above process, and obtain dark current calibration values of several preset temperatures.
The imaging correction device according to claim 6, characterized in that:

The dark current adjustment module is further configured to apply an interpolation algorithm to calculate the current temperature according to the stored dark current calibration value of the preset temperature if the current temperature of the image sensor is not equal to the stored preset temperature The corresponding dark current value; according to the dark current value corresponding to the current temperature, an adjustment value of the background image is obtained.
The imaging correction device according to claim 8, characterized in that:

The dark current adjustment module is also used to obtain the first two stored preset temperatures closest to the current temperature; and calculate the current temperature corresponding to the current temperature according to the dark current calibration values of the two preset temperatures. dark current value.
An intelligent terminal is characterized in that it comprises:

memory for storing computer programs;

A processor configured to implement the imaging correction method according to any one of claims 1 to 5 when running the computer program.