CN115297268A

CN115297268A - Imaging system and image processing method

Info

Publication number: CN115297268A
Application number: CN202210824473.8A
Authority: CN
Inventors: 聂鑫鑫; 於敏杰
Original assignee: Hangzhou Hikvision Digital Technology Co Ltd
Current assignee: Hangzhou Hikvision Digital Technology Co Ltd
Priority date: 2020-01-22
Filing date: 2020-01-22
Publication date: 2022-11-04
Anticipated expiration: 2040-01-22
Also published as: WO2021147804A1; CN115297268B; CN113163124A; CN113163124B

Abstract

The embodiment of the application provides an imaging system and an image processing method, wherein the imaging system comprises: the device comprises an image sensor, a statistical unit and an exposure control unit. The statistical unit respectively counts the image data of each type of channel, the exposure control unit calculates the exposure parameters corresponding to the type of channel according to the statistical data of the type of channel, controls the brightness adjustment of the image data of the type of channel based on the calculated exposure parameters, and independently exposes the image data of the type of channel aiming at the actual situation that the energy of the light components responded by the different types of channels is different, so that the brightness of the image data of the type of channel is controlled within a proper brightness range, and the final imaging effect is improved.

Description

Imaging system and image processing method

Technical Field

The present application relates to the field of computer vision technologies, and in particular, to an imaging system and an image processing method.

Background

In current imaging systems, image sensors are typically employed for imaging. The image sensor uses a photoelectric conversion Device such as a CCD (Charge-coupled Device), a CMOS (Complementary Metal Oxide Semiconductor), or the like. The image sensor comprises a plurality of types of channels, wherein the channels are respectively arranged to correspond to pixels in a pixel array, one channel responds to the passed light component to correspondingly obtain one pixel, and then the light signal can be converted into an image signal.

However, since image signals between various channels affect each other, for example, RGB (color) channels can sense energy of NIR (Near Infrared) light in addition to color light, resulting in poor imaging effect. Therefore, in the corresponding imaging system, an interpolation mode is adopted, the NIR channel is interpolated by using the spatial information of the RGB channel, the RGB channel is interpolated by using the spatial information of the NIR channel, the correlation between the RGB channel and the NIR channel is analyzed in the interpolation process, and the obtained difference image has higher resolution, so that the imaging effect is improved.

However, in an actual environment, the energy of light components responded by various channels is greatly different, and although the correlation between the channels can be analyzed by adopting the interpolation method, a uniform imaging strategy is still used during imaging, and an actual imaging effect is not ideal.

Disclosure of Invention

An object of the embodiments of the present application is to provide an imaging system and an image processing method, so as to improve an imaging effect of the imaging system. The specific technical scheme is as follows:

in a first aspect, an embodiment of the present application provides an imaging system, including: an image sensor, a statistic unit and an exposure control unit; the image sensor comprises a plurality of types of channels;

the image sensor is used for converting an optical signal into an image signal, and the optical signal comprises optical components in a plurality of wave band ranges;

the statistical unit is used for acquiring the image signal; extracting image data of various channels in the image signal; respectively counting the image data of each channel to obtain the statistical data of each channel; sending the statistical data of the various channels to the exposure control unit;

the exposure control unit is used for receiving the statistical data of the various channels sent by the statistical unit; and aiming at any type of channel, calculating an exposure parameter corresponding to the type of channel according to the statistical data of the type of channel, and controlling the brightness adjustment of the image data of the type of channel based on the exposure parameter.

Optionally, the image sensor includes: a first type of channel responsive to light components in the visible wavelength band and a second type of channel responsive to light components in the near infrared wavelength band.

Optionally, the first type of channel includes a plurality of color channels, and the second type of channel includes a near-infrared channel;

the statistical unit is specifically configured to:

calculating an image data statistic value of the first type of channel according to the image data of at least one color channel in the plurality of color channels to serve as statistic data of the first type of channel;

and calculating the image data statistic value of the second channel as the statistic data of the second channel according to the image data of the near-infrared channel.

Optionally, the statistical unit is specifically configured to:

extracting image data of each color channel and image data of the near-infrared channel from the image signals; calculating the image data mean value of each color channel and the image data mean value of the near infrared channel according to the image data of each color channel and the image data of the near infrared channel; carrying out weighted summation on the image data mean values of all the color channels; taking the result of the weighted summation as the statistical data of the first channel, and taking the image data mean value of the near-infrared channel as the statistical data of the second channel;

alternatively, the first and second liquid crystal display panels may be,

partitioning the image signal to obtain a plurality of image signal blocks; for any image signal block, extracting image data of each color channel and image data of the near infrared channel from the image signal block; calculating the image data mean value of each color channel and the image data mean value of the near-infrared channel according to the image data of each color channel and the image data of the near-infrared channel in each image signal block; carrying out weighted summation on the image data mean values of all the color channels; taking the result of the weighted summation as the statistical data of the first channel, and taking the image data mean value of the near-infrared channel as the statistical data of the second channel;

alternatively, the first and second liquid crystal display panels may be,

extracting image data of each color channel and image data of the near-infrared channel from the image signals; obtaining a histogram of each color channel and a histogram of the near-infrared channel according to the image data of each color channel and the image data of the near-infrared channel; respectively carrying out weighted average calculation on gray scale numbers in the histograms of the various color channels and the histogram of the near-infrared channel to obtain an image data mean value of the various color channels and an image data mean value of the near-infrared channel; carrying out weighted summation on the image data mean values of all the color channels; and taking the weighted summation result as the statistical data of the first channel, and taking the image data mean value of the near infrared channel as the statistical data of the second channel.

Optionally, the system further includes: a light filtering unit;

the filtering unit is used for filtering out other light components except the light component in the specified waveband range in the input light signal and transmitting the filtered light signal to the image sensor.

Optionally, the filtering unit includes a switching device;

the switching device is used for switching the filtering state of the filtering unit;

the filtering unit is used for filtering other light components except the light component within the specified waveband range in the input light signal when the filtering state is on, and transmitting the filtered light signal to the image sensor; and when the filtering state is closed, transmitting all light components in the light signal to the image sensor.

Optionally, the system further includes: a light supplement unit;

the light supplementing unit is used for performing near-infrared light supplementing on a scene so that an input optical signal comprises near-infrared light.

Optionally, the image sensor includes a second type of channel responsive to light components in a near-infrared band;

and the exposure control unit is further used for controlling the light supplementing unit to adjust the light supplementing intensity according to the statistical data of the second type of channel.

Optionally, the exposure control unit is specifically configured to:

if the statistical data of the second type of channel is larger than a first preset threshold value, controlling the light supplementing unit to reduce the intensity of transmitting the near infrared light;

and if the statistical data of the second type of channel is smaller than a second preset threshold value, controlling the light supplementing unit to improve the intensity of transmitting the near infrared light.

Optionally, the image sensor further includes: a first type of channel responsive to light components in the visible light band; the exposure control unit is specifically configured to:

acquiring first exposure time of the first type of channel, first target data corresponding to the first type of channel, second exposure time of the second type of channel and second target data corresponding to the second type of channel;

calculating a first data offset of the first type channel according to the statistical data of the first type channel and the first target data, and calculating a first exposure gain according to the statistical data of the first type channel and the first target data if the first data offset is not within a first preset range;

calculating a second data offset of the second type channel according to the statistical data and the second target data of the second type channel, and calculating a second exposure gain according to the statistical data and the second target data of the second type channel if the second data offset is not within a second preset range;

if the first exposure time is equal to the second exposure time, controlling the light supplementing unit to reduce the intensity of emitting the near infrared light when the second exposure gain is smaller than a first preset gain threshold value, and controlling the light supplementing unit to improve the intensity of emitting the near infrared light when the second exposure gain is larger than a second preset gain threshold value;

if the first exposure time is not equal to the second exposure time, when the second exposure gain is smaller than the first preset gain threshold, the second exposure time is reduced, and when the second exposure gain is larger than a second preset gain threshold, the second exposure time is increased.

Optionally, the system further includes: a processing unit;

the processing unit is used for acquiring image signals output by the image sensor, current exposure parameters corresponding to the various channels and associated information among the various channels; determining the correlation between each two types of channels according to the current exposure parameters corresponding to each type of channel and the correlation information between each type of channel; and removing the light component of the other channel contained in the one channel according to the correlation between each two channels.

Optionally, the image sensor includes: a first type of channel responsive to light components in the visible light band range and a second type of channel responsive to light components in the near infrared light band range;

the processing unit is specifically configured to:

acquiring an image signal output by the image sensor, current exposure parameters corresponding to the first type of channel and the second type of channel, and correlation information between the color of the first type of channel and the brightness of the second type of channel; or acquiring an image signal output by the image sensor, current exposure parameters corresponding to the first type of channel and the second type of channel, and associated information between the brightness of the first type of channel and the brightness of the second type of channel;

normalizing the image data of the first type channel and the image data of the second type channel in the image signal to be under the same exposure parameter according to the current exposure parameters corresponding to the first type channel and the second type channel;

determining the weight of the first type of channel and the weight of the second type of channel based on the correlation information;

and removing the light component of the second channel contained in the first channel according to the weight of the first channel and the weight of the second channel, and the normalized image data of the first channel and the normalized image data of the second channel.

the processing unit is specifically configured to:

determining the weight of the first type channel and the weight of the second type channel according to the current exposure parameters corresponding to the first type channel and the second type channel and the associated information;

and removing the light component of the second channel contained in the first channel according to the weight of the first channel, the weight of the second channel, the image data of the first channel and the image data of the second channel.

Optionally, the processing unit includes a signal decomposition module and a post-processing module;

the signal decomposition module is used for acquiring an image signal, decomposing a visible light signal and a near infrared light signal of the image signal, and outputting a first decomposed image signal and a second decomposed image signal after decomposition, wherein the first decomposed image signal is the visible light image signal, and the second decomposed image signal is the near infrared light image signal;

the post-processing module is configured to obtain the first decomposed image signal, the second decomposed image signal, the first current exposure parameter corresponding to the first type of channel, the second current exposure parameter corresponding to the second type of channel, and association information between the first type of channel and the second type of channel; determining the correlation between the first type of channel and the second type of channel according to the first current exposure parameter, the second current exposure parameter and the correlation information; and determining a first output image signal and/or a second output image signal according to the correlation, wherein the first output image signal is the first decomposition image signal without the near infrared light component.

Optionally, the signal decomposition module is specifically configured to:

acquiring an image signal; respectively carrying out up-sampling on each color component of a visible light signal and a near infrared light signal in the image signal to obtain an image signal of each color component and an image signal of near infrared light; combining the image signals of the color components to obtain a first decomposition image signal for outputting, and outputting the image signal of the near infrared light as a second decomposition image signal;

alternatively, the first and second electrodes may be,

acquiring an image signal, a first current exposure gain corresponding to the first type of channel and a second current exposure gain corresponding to the second type of channel; if the second current exposure gain is smaller than the first current exposure gain, performing edge judgment interpolation on the image data of the first type of channel according to the image data of the second type of channel in the image signal, and if the second current exposure gain is larger than the first current exposure gain, performing edge judgment interpolation on the image data of the second type of channel according to the image data of the first type of channel in the image signal; obtaining image signals of all color components of the visible light signals and image signals of near infrared light after interpolation; and combining the image signals of the color components to obtain a first decomposition image signal and outputting the first decomposition image signal, and outputting the image signal of the near infrared light as a second decomposition image signal.

Optionally, the post-processing module includes: the color recovery processing module comprises a first processing submodule, a second processing submodule, a color recovery submodule and a third processing submodule;

the first processing submodule is used for acquiring the first decomposition image signal and preprocessing the first decomposition image signal to obtain a first sub-processing image signal;

the second processing sub-module is configured to acquire the second decomposed image signal, and perform preprocessing on the second decomposed image signal to obtain a second sub-processed image signal, where the preprocessing includes at least one of dead pixel correction, black level correction, digital gain, and noise reduction;

the color recovery submodule is configured to obtain a first current exposure parameter corresponding to the first type of channel, a second current exposure parameter corresponding to the second type of channel, and association information between the first type of channel and the second type of channel; normalizing the first sub-processed image signal and the second sub-processed image signal to be under the same exposure parameter according to the first current exposure parameter and the second current exposure parameter; determining the weight of the first type of channel and the weight of the second type of channel based on the association information; obtaining a first recovery image signal according to the weight of the first type channel and the weight of the second type channel, and the normalized first sub-processing image signal and the normalized second sub-processing image signal;

and the third processing submodule is used for processing the first restored image signal to obtain a first output image signal.

Optionally, the post-processing module includes: the color recovery processing module comprises a first processing submodule, a second processing submodule, a color recovery submodule and a fourth processing submodule;

the color recovery sub-module is configured to obtain a first current exposure parameter corresponding to the first type of channel, a second current exposure parameter corresponding to the second type of channel, and association information between the first type of channel and the second type of channel; normalizing the first sub-processed image signal and the second sub-processed image signal to be under the same exposure parameter according to the first current exposure parameter and the second current exposure parameter; determining the weight of the first type of channel and the weight of the second type of channel based on the correlation information; obtaining a second recovery image signal according to the weight of the first type channel and the weight of the second type channel, and the normalized first sub-processing image signal and the normalized second sub-processing image signal;

and the fourth processing submodule is used for processing the second restored image signal to obtain a second output image signal.

Optionally, the post-processing module includes: the color recovery processing module comprises a first processing sub-module, a second processing sub-module, a color recovery sub-module, a third processing sub-module, a fourth processing sub-module and a fifth processing sub-module;

the first processing submodule is used for acquiring the first decomposed image signal and preprocessing the first decomposed image signal to obtain a first sub-processed image signal;

the color recovery submodule is configured to obtain a first current exposure parameter corresponding to the first type of channel, a second current exposure parameter corresponding to the second type of channel, and association information between the first type of channel and the second type of channel; normalizing the first sub-processed image signal and the second sub-processed image signal to be under the same exposure parameter according to the first current exposure parameter and the second current exposure parameter; determining the weight of the first type of channel and the weight of the second type of channel based on the association information; obtaining a first recovery image signal and a second recovery image signal according to the weight of the first type channel and the weight of the second type channel, and the normalized first sub-processing image signal and the normalized second sub-processing image signal;

the third processing submodule is used for processing the first restored image signal to obtain a third sub-processed image signal;

the fourth processing sub-module is configured to process the second restored image signal to obtain a fourth sub-processed image signal;

and the fifth processing submodule is used for processing the third sub-processed image signal and the fourth sub-processed image signal to obtain a first output image signal.

Optionally, the post-processing module includes: the color recovery processing module comprises a first processing submodule, a second processing submodule, a color recovery submodule, a third processing submodule, a fourth processing submodule and a fifth processing submodule;

the second processing sub-module is configured to obtain the second decomposed image signal, and perform preprocessing on the second decomposed image signal to obtain a second sub-processed image signal, where the preprocessing includes at least one of dead pixel correction, black level correction, digital gain, and noise reduction;

the color recovery submodule is configured to obtain a first current exposure parameter corresponding to the first type of channel, a second current exposure parameter corresponding to the second type of channel, and association information between the first type of channel and the second type of channel; normalizing the first sub-processed image signal and the second sub-processed image signal to be under the same exposure parameter according to the first current exposure parameter and the second current exposure parameter; determining the weight of the first type of channel and the weight of the second type of channel based on the association information; obtaining a first recovery image signal and a second recovery image signal according to the weight of the first type of channel and the weight of the second type of channel, and the normalized first sub-processing image signal and the normalized second sub-processing image signal;

the fourth processing submodule is configured to process the second restored image signal to obtain a fourth sub-processed image signal;

the fifth processing sub-module is configured to process the third sub-processed image signal and the fourth sub-processed image signal to obtain a first output image signal and a second output image signal.

Optionally, the processing unit further includes a preprocessing module;

the preprocessing module is used for acquiring the image signal output by the image sensor, preprocessing the image signal and sending the preprocessed image signal to the signal decomposition module.

Optionally, the processing unit is further configured to fuse the image data of each type of channel from which the light components of other types of channels have been removed, to obtain a fused image signal.

In a second aspect, an embodiment of the present application provides an image processing method, which is applied to an imaging system, where the system includes: an image sensor, a statistic unit and an exposure control unit; the image sensor comprises a plurality of types of channels; the method comprises the following steps:

the image sensor converts an optical signal into an image signal, the optical signal including light components in a plurality of wavelength band ranges;

the statistical unit acquires the image signal, extracts image data of various channels in the image signal, respectively counts the image data of the various channels to obtain statistical data of the various channels, and sends the statistical data of the various channels to the exposure control unit;

the exposure control unit receives the statistical data of the various channels sent by the statistical unit, calculates the exposure parameters corresponding to the channels according to the statistical data of the channels aiming at any channel, and controls the brightness adjustment of the image data of the channels based on the exposure parameters.

Optionally, the image sensor includes: a first channel type responsive to light components in a visible light band and a second channel type responsive to light components in a near-infrared light band, the first channel type including a plurality of color channels, the second channel type including a near-infrared channel;

the statistical unit respectively counts the image data of all the channels to obtain the statistical data of all the channels, and the statistical data comprises the following steps:

calculating an image data statistic value of the first type channel according to the image data of at least one color channel in the plurality of color channels to serve as statistic data of the first type channel;

Optionally, the system further includes: a light supplement unit; the image sensor comprises a second type of channel responding to light components in a near infrared light waveband range;

the method further comprises the following steps:

Optionally, the system further includes: a processing unit;

after the image sensor converts the light signal into an image signal, the method further comprises:

the processing unit acquires an image signal output by the image sensor, current exposure parameters corresponding to various channels and correlation information among the various channels, determines correlation between every two types of channels according to the current exposure parameters corresponding to the various channels and the correlation information among the various channels, and removes light components of another type of channel contained in one type of channel according to the correlation between every two types of channels.

An imaging system and an image processing method provided in an embodiment of the present application include: the device comprises an image sensor, a statistic unit and an exposure control unit. The statistical unit is used for acquiring image signals obtained by converting optical signals by the image sensor, extracting image data of various channels in the image signals, respectively performing statistics on the image data of the various channels to obtain statistical data of the various channels, and sending the statistical data of the various channels to the exposure control unit; the exposure control unit is used for receiving the statistical data of various channels sent by the statistical unit; and aiming at any type of channel, calculating an exposure parameter corresponding to the type of channel according to the statistical data of the type of channel, and controlling the brightness adjustment of the image data of the type of channel based on the exposure parameter.

In the embodiment of the application, a counting unit and an exposure control unit are additionally arranged in an imaging system, the counting unit respectively counts the image data of each type of channel, the exposure control unit calculates the exposure parameters corresponding to the type of channel according to the statistical data of the type of channel, controls the brightness adjustment of the image data of the type of channel based on the calculated exposure parameters, and independently exposes the image data of the type of channel according to the actual situation that the energy of the light components responded by the type of channel is different, so that the brightness of the image data of the type of channel is controlled within the proper brightness range, and the final imaging effect is improved.

Drawings

In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present application, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts.

Fig. 1 is a schematic structural diagram of an imaging system according to an embodiment of the present application;

FIG. 2a is a schematic diagram of an arrangement of image sensors according to an embodiment of the present application;

FIG. 2b is a schematic diagram of an arrangement of image sensors according to another embodiment of the present application;

FIG. 2c is a schematic diagram illustrating an arrangement of image sensors according to yet another embodiment of the present application;

FIG. 3 is a schematic diagram of spectral response curves for the RGB channel and NIR channel of an embodiment of the present application;

FIG. 4 is a schematic diagram of a spectral response curve of the W channel of an embodiment of the present application;

FIG. 5 is a schematic structural diagram of an imaging system according to another embodiment of the present application;

FIG. 6 is a diagram illustrating a spectral transmittance curve of a filter unit according to an embodiment of the present application;

FIG. 7 is a schematic structural diagram of an imaging system according to yet another embodiment of the present application;

FIG. 8 is a schematic structural diagram of an imaging system according to yet another embodiment of the present application;

FIG. 9 is a graph showing the 850nm near infrared energy distribution of an embodiment of the present application;

FIG. 10 is a schematic view illustrating a process of adjusting exposure gain according to an embodiment of the present application;

FIG. 11 is a schematic structural diagram of an imaging system according to yet another embodiment of the present application;

FIG. 12 is a flow chart illustrating an implementation of a processing unit according to an embodiment of the present application;

FIG. 13 is a schematic flow chart illustrating an implementation of a post-processing module according to an embodiment of the present application;

FIG. 14 is a schematic flow chart illustrating an implementation of a post-processing module according to another embodiment of the present application;

FIG. 15 is a schematic flow chart illustrating an implementation of a post-processing module according to yet another embodiment of the present application;

FIG. 16 is a schematic flow chart illustrating an implementation of a processing unit according to another embodiment of the present application;

FIG. 17 is a schematic flow chart illustrating an implementation of a post-processing module according to yet another embodiment of the present application;

FIG. 18 is a flow chart illustrating an implementation of a processing unit according to yet another embodiment of the present application;

FIG. 19 is a flow chart illustrating an implementation of a processing unit according to yet another embodiment of the present application;

fig. 20 is a flowchart illustrating an image processing method according to an embodiment of the present application.

Detailed Description

The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.

In order to improve the imaging effect of an imaging system, the embodiment of the application provides an imaging system and an image processing method.

The embodiment of the present application provides an imaging system, as shown in fig. 1, which includes an image sensor 11, a statistic unit 12, and an exposure control unit 13. The image sensor 11 includes a plurality of types of channels for obtaining one pixel in an image signal in response to the light component passing therethrough.

The image sensor 11 is configured to convert an optical signal into an image signal, where the optical signal includes light components in a plurality of wavelength ranges; the statistical unit 12 is configured to acquire an image signal, extract image data of various channels in the image signal, perform statistics on the image data of the various channels to obtain statistical data of the various channels, and send the statistical data of the various channels to the exposure control unit 13; and an exposure control unit 13, configured to receive the statistical data of each type of channel sent by the statistical unit 12, calculate, for any type of channel, an exposure parameter corresponding to the type of channel according to the statistical data of the type of channel, and control, based on the exposure parameter, to perform brightness adjustment on the image data of the type of channel.

By applying the embodiment of the application, the statistical unit respectively carries out statistics on the image data of each type of channel, the exposure control unit calculates the exposure parameters corresponding to the type of channel according to the statistical data of the type of channel, controls the brightness adjustment of the image data of the type of channel based on the calculated exposure parameters, and carries out independent exposure on the type of channel aiming at the actual situation that the energy of the light components responded by different types of channels is different, so that the brightness of the type of channel is controlled within a proper brightness range, and the final imaging effect is improved.

The imaging system provided by the embodiment of the present application may be an image capturing system (for example, a digital camera, a camcorder, a monitoring camera, etc.), and the imaging system may also be an imaging module installed on a computer, a multimedia player, a mobile phone, etc.

The image sensor 11 includes a plurality of classes of channels, each class of channels being responsive to light components in a different wavelength band. In one implementation, the image sensor 11 may include: a first type of channel responsive to light components in the visible wavelength band and a second type of channel responsive to light components in the near infrared wavelength band.

In this implementation, the image sensor 11 is an RGB-IR sensor, and specifically includes two types of channels, that is, a first type of channel is an RGB channel, and a second type of channel is an NIR channel. The image sensor arrangement shown in fig. 2a, 2b or 2c is obtained by various types of channels, wherein a plurality of pixels are formed in a pixel array. In FIG. 2a, R (red) and B (blue) channels account for 1/8 of the total pixel number, NIR channel accounts for 1/4 of the total pixel number, and G (green) channel accounts for 1/2 of the total pixel number, respectively; in FIG. 2B, the R channel and the B channel respectively account for 1/8 of the total pixel number, the G channel accounts for 1/4 of the total pixel number, and the NIR channel accounts for 1/2 of the total pixel number; in FIG. 2c, the R channel and the B channel account for 3/16 of the total pixel number, the G channel accounts for 3/8 of the total pixel number, and the NIR channel accounts for 1/4 of the total pixel number, respectively.

The spectral response of each channel is shown in fig. 3, and in order to ensure that the first type channel responds to the visible light component and the second type channel responds to the near-infrared light component, the relative response of the light component of the first type channel in the waveband range of 400 nm-700 nm is not lower than that in the waveband range of 700 nm-1000 nm; the relative response of the light component of the second type of channel is not lower than that of the light component of the first type of channel in the wave band range of 800 nm-1000 nm.

The second type of channel may be a W (White) channel, which is a channel responsive to light components in the visible wavelength band and light components in the near infrared wavelength band, in addition to the NIR channel. The spectral response of the W channel is shown in FIG. 4, where the light components of the W channel all respond in the wavelength range of 400nm to 1000 nm.

A lens (not shown in fig. 1) is provided on the input side of the image sensor 11 for receiving an optical signal reflected by a target object. The optical signal comprises a visible light component, a near infrared light component and the like, and the lens can enable the visible light component and the near infrared light component to meet the confocal requirement.

In this embodiment, the statistical unit 12 is a sensor for performing image data statistics, for example, an rgbiir sensor, and when performing image data statistics, luminance statistics is mainly performed on the image data, the image data of each type of channel is a pixel value of the channel at a corresponding position in the pixel array, and the pixel value can represent pixel luminance.

In one implementation, the first type of channel includes a plurality of color channels and the second type of channel includes a near infrared channel.

Correspondingly, the statistical unit 12 may be specifically configured to: calculating an image data statistic value of a first type channel according to the image data of at least one color channel in the plurality of color channels to serve as statistic data of the first type channel; and calculating an image data statistic value of the second channel as statistic data of the second channel according to the image data of the near-infrared channel.

The first channel is a color channel, and specifically includes a red channel, a green channel, and a blue channel, or includes a red channel, a yellow channel, and a blue channel, image data of each color channel needs to be counted, and a calculated image data statistical value (for example, an image data mean value, image data and value, etc.) of the first channel is used as statistical data of the first channel, or an image data mean value of the red channel or the green channel is calculated as statistical data of the first channel. And the second channel is a near-infrared channel, image data of the near-infrared channel is counted, and the calculated image data statistical value of the second channel is used as statistical data of the second channel.

Optionally, the statistical unit 12 may be specifically configured to:

extracting image data of each color channel and image data of a near infrared channel from the image signal; calculating the image data mean value of each color channel and the image data mean value of the near-infrared channel according to the image data of each color channel and the image data of the near-infrared channel; carrying out weighted summation on the image data mean values of all color channels; taking the weighted summation result as the statistical data of a first channel, and taking the image data mean value of the near-infrared channel as the statistical data of a second channel;

alternatively, the first and second liquid crystal display panels may be,

partitioning the image signal to obtain a plurality of image signal blocks; for any image signal block, extracting image data of each color channel and image data of a near infrared channel from the image signal block; calculating the image data mean value of each color channel and the image data mean value of the near-infrared channel according to the image data of each color channel and the image data of the near-infrared channel in each image signal block; carrying out weighted summation on the image data mean values of all color channels; taking the weighted and summed result as the statistical data of a first channel, and taking the image data mean value of the near infrared channel as the statistical data of a second channel;

alternatively, the first and second electrodes may be,

extracting image data of each color channel and image data of a near infrared channel from the image signal; obtaining a histogram of each color channel and a histogram of a near-infrared channel according to the image data of each color channel and the image data of the near-infrared channel; respectively carrying out weighted average calculation on gray scale numbers in the histograms of the color channels and the near-infrared channels to obtain image data mean values of the color channels and the image data mean values of the near-infrared channels; carrying out weighted summation on the image data mean values of all color channels; and taking the weighted and summed result as the statistical data of the first channel, and taking the image data mean value of the near infrared channel as the statistical data of the second channel.

The data statistics includes three statistical modes, namely global statistics, block statistics and histogram statistics, and the three statistical modes are introduced by taking the rgbiir sensor as an example.

Global statistics: and respectively calculating image data mean values of the R channel, the G channel, the B channel and the NIR channel to respectively obtain an image data mean value Rave of the R channel, an image data mean value Gave of the G channel, an image data mean value Bave of the B channel and an image data mean value NIRave of the NIR channel. Weighting Rave, gave and Bave according to a certain weight to obtain Y = kr + Rave + kg + Gave + kb + Bave, wherein kr is the weight of an R channel, kg is the weight of a G channel, kb is the weight of a B channel, and kr + kg + kb =1.Y is the statistical data of the first type of channel, and NIRave is the statistical data of the second type of channel.

Block counting: divide the image signalThe block is used for respectively calculating the image data mean value of the R channel, the G channel, the B channel and the NIR channel of each block to obtain the image data mean value R of the R channel _ij average value G of image data of ave and G channels _ij average value B of image data of ave and B channels _ij average NIR of image data of ave and NIR channels _ij ave (i, j represent coordinates of the block, 0<i<M,0<j<N, M, N are the number of blocks in the vertical and horizontal directions); weighted average is carried out on all blocks of each channel to obtain the image data mean value of the R channel

Image data mean of G channel

B-channel image data mean

Image data mean for NIR channel

And weighting the Rave, the Gave and the Bave according to a certain weight to obtain Y = kr Rave + kg Gave + kb Bave, wherein kr is the weight of an R channel, kg is the weight of a G channel, kb is the weight of a B channel, and kr + kg + kb =1.Y is the statistical data of the first type of channel, and NIRave is the statistical data of the second type of channel.

Histogram statistics: firstly, the number of bits of input data of an R channel, a G channel and a B channel needs to be considered, if the number of bits exceeds or is less than 8 bits, the input data needs to be converted into 8 bits, and then histograms are respectively calculated for the R channel, the G channel, the B channel and the NIR channel to obtain a Rhist, a Ghist, a Bhist and an NIRhist; the number of gray scales of the histogram is 256; the histogram for each channel is weighted and averaged according to the number of gray levels as follows:

where w (n) is the weight of each gray level. Weighting Rave, gave and Bave according to a certain weight to obtain Y = kr Rave + kg Gave + kb Bave, wherein kr is the weight of the R channel, kg is the weight of the G channel, kb is the weight of the B channel, and kr + kg + kb =1.Y is the statistical data of the first channel, and NIRave is the statistical data of the second channel.

In this embodiment, the exposure control unit 13 receives the statistical data of the various channels sent by the statistical unit 12, and calculates exposure parameters corresponding to the various channels according to the statistical data of the various channels, where the exposure parameters may include exposure time, analog gain, digital gain, and the like, and the exposure parameters may also include an aperture of a lens. And after the exposure parameters corresponding to various channels are obtained, controlling the brightness adjustment of the image data of the corresponding channel based on the exposure parameters. Specifically, when controlling to adjust the brightness of the image data of the corresponding channel, the exposure control unit 13 may send the exposure parameter to the image sensor, and the image sensor acts the exposure parameter on each channel to adjust the brightness of each channel, so that the brightness of the image data of each channel is within a preset brightness range; or the exposure parameters can be sent to the image processing unit, and the image processing unit directly adjusts the image data of the corresponding channel output by the image sensor based on the exposure parameters, so that the brightness of the image data of various channels is within a preset brightness range. Different preset brightness ranges can be set for the image data of different channels, and the same preset brightness range can also be set, which is not specifically limited herein. After the brightness of the image data of each type of channel is adjusted, the finally output image signal can be in accordance with the brightness range.

After the exposure parameters are obtained through calculation, the brightness adjustment mode is controlled based on the exposure parameters, the exposure parameters are mainly sent to the image sensor or the image processing unit, and the image sensor or the image processing unit is controlled to adjust the exposure time, the exposure gain and the like, and the specific adjustment mode is a conventional exposure adjustment mode and is not limited specifically here.

Based on the embodiment shown in fig. 1, the embodiment of the present application further provides an imaging system, as shown in fig. 5, which includes an image sensor 11, a counting unit 12, an exposure control unit 13, and a filtering unit 14. The image sensor 11 includes a plurality of types of channels for obtaining one pixel in an image signal in response to a light component passing therethrough.

The filtering unit 14 is configured to filter out other light components in the input light signal except the light component in the specified wavelength range, and transmit the filtered light signal to the image sensor 11; an image sensor 11 for converting an optical signal into an image signal, the optical signal including light components in a plurality of wavelength band ranges; the statistical unit 12 is configured to obtain an image signal, extract image data of various channels in the image signal, perform statistics on the image data of the various channels respectively to obtain statistical data of the various channels, and send the statistical data of the various channels to the exposure control unit 13; and an exposure control unit 13, configured to receive the statistical data of each type of channel sent by the statistical unit 12, calculate, for any type of channel, an exposure parameter corresponding to the type of channel according to the statistical data of the type of channel, and control, based on the exposure parameter, to perform brightness adjustment on the image data of the type of channel.

By applying the embodiment of the application, the statistical unit respectively carries out statistics on the image data of each type of channel, the exposure control unit calculates the exposure parameters corresponding to the type of channel according to the statistical data of the type of channel, controls the brightness adjustment of the image data of the type of channel based on the calculated exposure parameters, and carries out independent exposure on the type of channel aiming at the actual situation that the energy of the light components responded by different types of channels is different, so that the brightness of the image data of the type of channel is controlled within a proper brightness range, and the final imaging effect is improved. In addition, the filter unit is added at the front end of the image sensor to filter the input optical signal, so that the light components in the specified wave band range can pass through and be shot into the image sensor, and the light components in other wave band ranges can be filtered.

The filter unit 14 is capable of passing light components of near infrared light and visible light in a specified wavelength band range, and filtering light components in other wavelength band ranges. The filtering unit 14 may be a monolithic filter, that is, an optical device that filters a certain frequency band of light waves by using a coating technology, for example, a spectral transmittance curve of the filtering unit shown in fig. 6, and filters near infrared light and other near infrared light except visible light within a specified wavelength band by using a filter. In order to enable the filtering unit to enable the near infrared light and the visible light in the specified waveband range to pass through, the filtering unit can be arranged in the near infrared light waveband range of 650 nm-1000 nm, and the width of the passing waveband is smaller than the sum of the widths of the filtered wavebands; in the near infrared light wave band range of 650 nm-1000 nm, the wave band width with the passing rate of more than 30% is less than 100nm. The relative response of the light components of the RGB channels of the image sensor is less than 0.3 in the range of the near-infrared light passing wavelength band of the filter unit 14.

Based on fig. 6, a first band width corresponding to the near-infrared light at a certain passing rate (e.g. 20%) can be identified, and based on fig. 3, a second band width corresponding to the response intensity of the near-infrared light can be identified, where the first band width should be less than or equal to the second band width, and if this condition is not met, the filtering units of different coating technologies may be reselected to achieve the purpose that the first band width should be less than or equal to the second band width. Through such selection, can make in the colorama near infrared light weight less, more be favorable to the promotion of formation of image effect.

Optionally, the filtering unit may include a switching device. Wherein, the switching device is used for switching the filtering state of the filtering unit 14.

The filtering unit 14 may be configured to filter, when the filtering state is on, other light components in the input light signal except for the light component in the specified wavelength range, and transmit the filtered light signal to the image sensor 11; when the filtering state is off, all light components in the light signal are transmitted to the image sensor 11.

The filter unit 14 may have a switching device for switching the on state of the filter unit 14. When the switching device switches the filtering state of the filtering unit 14 to on, the filtering unit 14 filters out other light components except for the light component within the specified wavelength band range in the input light signal, transmits the filtered light signal to the image sensor 11, and transmits the light component within the specified wavelength band range to the image sensor 11; when the switching device switches the filtering state of the filtering unit 14 to off, all light components in the transmitted light signal are transmitted to the image sensor 11. In one case, when the filtering state of the filtering unit 14 is switched to on, the filtering unit 14 may filter out other light components except for the visible light and part of the near-infrared light in the input light signal, transmit the filtered light signal to the image sensor 11, and transmit only the light components of the visible light and part of the near-infrared light to the image sensor 11; the filtering unit 14 may filter out other light components except for the visible light from the input light signal, transmit the filtered light signal to the image sensor 11, and transmit only the light component passing through the visible light to the image sensor 11.

Based on the embodiment shown in fig. 1, the embodiment of the present application further provides an imaging system, as shown in fig. 7, the imaging system includes an image sensor 11, a statistics unit 12, an exposure control unit 13, and a light supplement unit 15. The image sensor 11 includes a plurality of types of channels for obtaining one pixel in an image signal in response to the light component passing therethrough.

The light supplementing unit 15 is configured to perform near-infrared light supplementing on a scene, so that an input optical signal includes near-infrared light; an image sensor 11 for converting an optical signal into an image signal, the optical signal including light components in a plurality of wavelength band ranges; the statistical unit 12 is configured to obtain an image signal, extract image data of various channels in the image signal, perform statistics on the image data of the various channels respectively to obtain statistical data of the various channels, and send the statistical data of the various channels to the exposure control unit 13; and an exposure control unit 13, configured to receive the statistical data of each type of channel sent by the statistical unit 12, calculate, for any type of channel, an exposure parameter corresponding to the type of channel according to the statistical data of the type of channel, and control, based on the exposure parameter, brightness adjustment on image data of the type of channel.

By applying the embodiment of the application, the statistical unit respectively carries out statistics on the image data of each type of channel, the exposure control unit calculates the exposure parameters corresponding to the type of channel according to the statistical data of the type of channel, controls the brightness adjustment of the image data of the type of channel based on the calculated exposure parameters, and carries out independent exposure on the type of channel aiming at the actual situation that the energy of the light components responded by different types of channels is different, so that the brightness of the image data of the type of channel is controlled within a proper brightness range, and the final imaging effect is improved. And the light supplementing unit is added to increase the near infrared light component in the optical signal, so that the brightness of the near infrared light channel is increased, and the near infrared light imaging is improved conveniently.

In another embodiment, as shown in fig. 8, the imaging system includes an image sensor 11, a counting unit 12, an exposure control unit 13, a filtering unit 14, and a fill-in light unit 15. The image sensor 11 includes a plurality of types of channels for obtaining one pixel in an image signal in response to the light component passing therethrough.

The light supplementing unit 15 is configured to perform near-infrared light supplementing on a scene, so that an input optical signal includes near-infrared light; a filtering unit 14, configured to filter out other light components in the input light signal except for the light component in the specified wavelength range, and transmit the filtered light signal to the image sensor 11, where the light component in the specified wavelength range includes near-infrared light emitted by the fill light unit; an image sensor 11 for converting an optical signal into an image signal, the optical signal including light components in a plurality of wavelength band ranges; the statistical unit 12 is configured to obtain an image signal, extract image data of various channels in the image signal, perform statistics on the image data of the various channels respectively to obtain statistical data of the various channels, and send the statistical data of the various channels to the exposure control unit 13; and an exposure control unit 13, configured to receive the statistical data of each type of channel sent by the statistical unit 12, calculate, for any type of channel, an exposure parameter corresponding to the type of channel according to the statistical data of the type of channel, and control, based on the exposure parameter, to perform brightness adjustment on the image data of the type of channel.

The light supplement unit 15 is used for near-infrared light supplement, and certainly, can also generate visible light supplement at the same time. The energy of the near-infrared supplementary lighting generated by the supplementary lighting unit 15 is distributed in the range of 650nm to 1000nm, and specifically, the energy is concentrated in the range of 750nm to 900nm, or in the range of 900nm to 1000 nm. In order to ensure that the filtering unit 14 can supplement light through the near-infrared light of the light supplementing unit 15, the energy distribution range of the near-infrared light supplemented by the light supplementing unit 15 is required to be not less than the near-infrared light passing range preset by the filtering unit 14.

In the embodiment of the application, a light supplement device of a near infrared light band is adopted for supplementing light. Specifically, an infrared lamp of 850nm may be used as the light supplement unit 15, or infrared lamps of 750nm, 780nm, 850nm, 860nm, and 940nm may be used as the light supplement unit 15, and taking an infrared lamp of 850nm as an example, the energy distribution curve of the infrared lamp is shown in fig. 9, and the energy distribution is mainly concentrated in the range of 830nm to 880 nm.

Alternatively, the image sensor 11 may include a second type of channel responsive to light components in the near infrared wavelength band. The exposure control unit 13 may be further configured to control the light supplementing unit 15 to adjust the light supplementing intensity according to the statistical data of the second type of channel.

The exposure control unit 13 can control the light supplement unit 15 to adjust the light supplement intensity, besides controlling the brightness of the image data of various channels, and the image data of the second channel can be in accordance with the preset brightness range through adjusting the light supplement intensity.

Optionally, the exposure control unit 13 may be specifically configured to: if the statistical data of the second channel is greater than the first preset threshold, controlling the light supplement unit 15 to reduce the intensity of transmitting the near infrared light; if the statistical data of the second channel is smaller than a second preset threshold, the light supplement unit 15 is controlled to increase the intensity of emitting the near infrared light.

The exposure control unit 13 controls the light supplementing unit 15 to adjust the light supplementing intensity, specifically, determines whether the statistical data of the second channel is greater than a first preset threshold, where the statistical data of the second channel may be a luminance statistical result of the image data of the second channel, and the corresponding first preset threshold is a maximum luminance threshold, and if the statistical data is greater than the first preset threshold, it indicates that the luminance of the second channel is too high and needs to be reduced, and the light supplementing unit 15 is controlled to reduce the intensity of emitting the near infrared light. And if the statistical data is smaller than the second preset threshold, it indicates that the brightness of the second channel is too low and needs to be increased, and the light supplementing unit 15 is controlled to increase the intensity of emitting the near infrared light.

In an implementation manner of the embodiment of the present application, the exposure control unit 13 controls the light supplementing unit 15 to adjust the light supplementing intensity, so that after the exposure control unit 13 controls to perform brightness adjustment on the image data of various channels, that is, if the exposure control unit 13 controls to perform brightness adjustment on the image data of various channels, the imaging effect is still poor (the image is too dark or too bright), at this time, the exposure control unit 13 may control the light supplementing unit 15 to adjust the light supplementing intensity, so that the brightness of the image conforms to the predetermined brightness range.

Optionally, the image sensor may further include: a first type of channel responsive to light components in the visible light band; the exposure control unit 13 may be specifically configured to:

acquiring first exposure time of a first type channel, first target data corresponding to the first type channel, second exposure time of a second type channel and second target data corresponding to the second type channel; calculating a first data offset of the first type channel according to the statistical data and the first target data of the first type channel, and calculating a first exposure gain according to the statistical data and the first target data of the first type channel if the first data offset is not within a first preset range; calculating a second data offset of the second type channel according to the statistical data and the second target data of the second type channel, and if the second data offset is not within a second preset range, calculating a second exposure gain according to the statistical data and the second target data of the second type channel; if the first exposure time is equal to the second exposure time, controlling the light supplementing unit to reduce the intensity of emitting the near infrared light when the second exposure gain is smaller than a first preset gain threshold value, and controlling the light supplementing unit to improve the intensity of emitting the near infrared light when the second exposure gain is larger than a second preset gain threshold value; if the first exposure time is not equal to the second exposure time, when the second exposure gain is smaller than a first preset gain threshold, the second exposure time is reduced, and when the second exposure gain is larger than a second preset gain threshold, the second exposure time is increased.

As shown in fig. 10, it is assumed that the exposure time is a fixed time, for example, 40ms or 10ms, the exposure time of the first type channel is T1, and the Gain is Gain1, and the exposure time of the second type channel is T2, and the Gain is Gain2. The step of calculating Gain1 and Gain2 includes: firstly, calculating a data offset according to statistical data and target data of a channel, namely Ydelta = | Ycurrent-Ytarget |; secondly, judging whether the data offset is within a preset range, and if not, executing a third step; and thirdly, calculating the updated exposure Gain, namely Gain = Ytarget/Ycurrent, according to the statistical data and the target data of the channel. And then, sending the calculated updated exposure Gain and the set exposure time to an image sensor, identifying that T1 is equal to T2 by the image sensor, when the Gain2 is smaller than a first preset Gain threshold value, controlling a light supplementing unit to reduce the intensity of emitted near infrared light to realize adjustment of light supplementing intensity, identifying that T1 is different from T2, and when the Gain2 is smaller than a preset minimum value, realizing brightness adjustment of image data of a channel by reducing T2.

Based on the embodiment shown in fig. 1, the embodiment of the present application further provides an imaging system, as shown in fig. 11, which includes an image sensor 11, a statistics unit 12, an exposure control unit 13, and a processing unit 16. The image sensor 11 includes a plurality of types of channels for obtaining one pixel in an image signal in response to the light component passing therethrough.

The image sensor 11 is configured to convert an optical signal into an image signal, where the optical signal includes light components in a plurality of wavelength ranges; a processing unit 16, configured to obtain an image signal output by the image sensor 11, current exposure parameters corresponding to various channels, and correlation information between various channels, determine a correlation between each two types of channels according to the current exposure parameters corresponding to various channels and the correlation information between various channels, and remove a light component of another type of channel included in one type of channel according to the correlation between each two types of channels; the statistical unit 12 is configured to obtain an image signal, extract image data of various channels in the image signal, perform statistics on the image data of the various channels respectively to obtain statistical data of the various channels, and send the statistical data of the various channels to the exposure control unit 13; and an exposure control unit 13, configured to receive the statistical data of each type of channel sent by the statistical unit 12, calculate, for any type of channel, an exposure parameter corresponding to the type of channel according to the statistical data of the type of channel, and control, based on the exposure parameter, brightness adjustment on image data of the type of channel.

By applying the embodiment of the application, the statistical unit respectively carries out statistics on the image data of each type of channel, the exposure control unit calculates the exposure parameters corresponding to the type of channel according to the statistical data of the type of channel, controls the brightness adjustment of the image data of the type of channel based on the calculated exposure parameters, and carries out independent exposure on the type of channel aiming at the actual situation that the energy of the light components responded by different types of channels is different, so that the brightness of the image data of the type of channel is controlled within a proper brightness range, and the final imaging effect is improved. And the processing unit removes the light components of the other channel contained in the one channel by analyzing the correlation between every two channels, wherein the correlation is related to the exposure parameters, and more accurate color information is obtained.

The Processing Unit 16 is a logic platform containing an image Processing algorithm or program, and the platform may be a Central Processing Unit (CPU), a Network Processor (NP), or the like; but also a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array) or other Programmable logic device, discrete Gate or transistor logic device, discrete hardware component.

The processing unit 16 processes the image signal to obtain image data of each channel after acquiring the image signal from the image sensor 11, and may acquire current exposure parameters (which may include exposure time, exposure gain, and the like) corresponding to each channel, and may determine a correlation between each two types of channels according to the current exposure parameters corresponding to each channel and correlation information between each type of channels, where the correlation information refers to an influence of a certain image attribute of one channel on a certain image attribute of another channel, for example, an influence of a brightness of an NIR channel and a color of an RGB channel, the correlation indicates a degree and a size of image attribute correlation information, and the image attribute correlation information may be obtained by analyzing the processing unit after acquiring the image signal, or may be sent to the processing unit by analyzing the image sensor in advance according to the image signal. According to the correlation between every two types of channels, the size and the degree of mutual influence of image data between the two channels can be clearly known, so that according to the correlation between every two types of channels, the light component of the other channel contained in one channel can be removed, and the original signal of one channel can be restored. Specifically, the method may be implemented by performing weighting processing on the image data of multiple channels by using a coefficient matrix, where the coefficient matrix may be a pre-calibrated matrix.

Alternatively, the image sensor 11 may include: a first type of channel responsive to light components in the visible wavelength band and a second type of channel responsive to light components in the near infrared wavelength band.

The processing unit 16 may be specifically configured to: acquiring an image signal output by an image sensor, current exposure parameters corresponding to a first type of channel and a second type of channel, and correlation information between the color of the first type of channel and the brightness of the second type of channel, or acquiring the image signal output by the image sensor, the current exposure parameters corresponding to the first type of channel and the second type of channel, and the correlation information between the brightness of the first type of channel and the brightness of the second type of channel; normalizing the image data of the first type channel and the image data of the second type channel in the image signal to be under the same exposure parameter according to the current exposure parameters corresponding to the first type channel and the second type channel; determining the weight of the first type of channel and the weight of the second type of channel based on the association information; and removing the light component of the second channel contained in the first channel according to the weight of the first channel and the weight of the second channel, and the normalized image data of the first channel and the normalized image data of the second channel.

For a scene in which the image sensor 11 includes a first type channel and a second type channel, image data of the first type channel and image data of the second type channel in the image signal may be normalized to the same exposure parameter according to current exposure parameters of the first type channel and the second type channel, so that the image data of the first type channel and the image data of the second type channel under the same exposure parameter are logically obtained, and the weight of the first type channel and the weight of the second type channel are determined based on correlation information between the color of the first type channel and the brightness of the second type channel, and the weight of the first type channel and the weight of the second type channel may form a coefficient matrix, which may be a 3 × 4 matrix calibrated according to the correlation information. The light component of the second channel contained in the first channel can be removed by using the weight of the first channel, the weight of the second channel, the normalized image data of the first channel and the normalized image data of the second channel. The normalization process can also be implemented by adjusting the coefficient matrix, and is not described herein again. The correlation information between the color of the first-type channel and the brightness of the second-type channel may be obtained by analyzing the correlation information between the brightness of each color channel in the first-type channel and the brightness of the second-type channel in advance, and of course, the correlation information may also be obtained by directly using the correlation information between the brightness of each color channel in the first-type channel and the brightness of the second-type channel.

The processing unit 16 may be specifically configured to: acquiring an image signal output by an image sensor, current exposure parameters corresponding to a first type of channel and a second type of channel, and correlation information between the color of the first type of channel and the brightness of the second type of channel, or acquiring the image signal output by the image sensor, the current exposure parameters corresponding to the first type of channel and the second type of channel, and the correlation information between the brightness of the first type of channel and the brightness of the second type of channel; determining the weight of the first type channel and the weight of the second type channel according to the current exposure parameters corresponding to the first type channel and the second type channel and the associated information; and removing the light component of the second channel contained in the first channel according to the weight of the first channel, the weight of the second channel, the image data of the first channel and the image data of the second channel.

For a scene in which the image sensor 11 includes a first type channel and a second type channel, the weights of the first type channel and the weights of the second type channel may be determined according to current exposure parameters of the first type channel and the second type channel and pre-analyzed correlation information between colors of the first type channel and brightness of the second type channel, and the weights of the first type channel and the weights of the second type channel may form a coefficient matrix, which may be a 3 × 4 matrix calibrated according to the correlation information. By using the weight of the first-type channel and the weight of the second-type channel, and the image data of the first-type channel and the image data of the second-type channel, the light component of the second-type channel contained in the first-type channel can be removed. The determined weight of the first type channel and the determined weight of the second type channel are related to the current exposure parameter, so that more accurate color information can be obtained.

In subsequent applications, the color components in the image signal are generally of greater interest, and therefore, the above embodiments only give an implementation of removing the light components of the second type of channels included in the first type of channels. For better imaging effect, the light components of the first-type channel contained in the second-type channel can be removed in the above manner, and then the channel fusion is performed to obtain the fused image signal.

Optionally, the processing unit 16 may be further configured to fuse image data of various channels from which light components of other channels have been removed, to obtain a fused image signal.

The processing unit 16 can fuse the image data of various channels after performing light component elimination on various channels, and can enhance color signals and improve the quality of the fused image data under low illumination. After the fused image signal is obtained, the image signal may be sent to the statistical unit 12, and the statistical unit 12 performs statistics on the image data to provide a basis for exposure control of the exposure control unit 13, or may directly output the result to a user.

In the embodiment of the application, joint noise reduction can be performed on the image data of the first-type channel and the image data of the second-type channel, and a specific joint noise reduction mode can be that the image data of the first-type channel is guided by using the image data of the second-type channel, so that effective information loss is reduced while noise is reduced.

the signal decomposition module is used for acquiring an image signal, decomposing a visible light signal and a near-infrared light signal of the image signal, and outputting a first decomposed image signal and a second decomposed image signal after decomposition, wherein the first decomposed image signal is the visible light image signal, and the second decomposed image signal is the near-infrared light image signal;

the post-processing module is used for acquiring the first decomposed image signal, the second decomposed image signal, the first current exposure parameter corresponding to the first type of channel, the second current exposure parameter corresponding to the second type of channel and the associated information between the first type of channel and the second type of channel; determining the correlation between the first type of channel and the second type of channel according to the first current exposure parameter, the second current exposure parameter and the correlation information; and determining a first output image signal and/or a second output image signal according to the correlation, wherein the first output image signal is the first decomposition image signal without the near infrared light component. The second output image signal may be a second decomposition image signal from which the visible light component is removed, or may be a second decomposition image signal from which the visible light component is not removed.

Optionally, the signal decomposition module may be specifically configured to:

acquiring an image signal; respectively carrying out up-sampling on each color component of a visible light signal and a near infrared light signal in an image signal to obtain an image signal of each color component and an image signal of near infrared light; combining the image signals of the color components to obtain a first decomposition image signal for outputting, and outputting the image signal of the near infrared light as a second decomposition image signal;

alternatively, the first and second liquid crystal display panels may be,

acquiring an image signal, a first current exposure gain corresponding to a first type of channel and a second current exposure gain corresponding to a second type of channel; if the second current exposure gain is smaller than the first current exposure gain, performing edge judgment interpolation on the image data of the first type of channel according to the image data of the second type of channel in the image signal, and if the second current exposure gain is larger than the first current exposure gain, performing edge judgment interpolation on the image data of the second type of channel according to the image data of the first type of channel in the image signal; obtaining image signals of various color components of the visible light signals and image signals of near infrared light after interpolation; and combining the image signals of the color components to obtain a first decomposition image signal and outputting the first decomposition image signal, and outputting the image signal of the near infrared light as a second decomposition image signal.

Optionally, the post-processing module may include: the color recovery processing module comprises a first processing submodule, a second processing submodule, a color recovery submodule and a third processing submodule;

the first processing submodule is used for acquiring a first decomposition image signal and preprocessing the first decomposition image signal to obtain a first sub-processing image signal;

the second processing submodule is used for acquiring a second decomposition image signal and preprocessing the second decomposition image signal to obtain a second sub-processing image signal, wherein the preprocessing comprises at least one processing mode of dead pixel correction, black level correction, digital gain and noise reduction;

the color recovery submodule is used for acquiring a first current exposure parameter corresponding to the first channel, a second current exposure parameter corresponding to the second channel and associated information between the first channel and the second channel; normalizing the first sub-processed image signal and the second sub-processed image signal to be under the same exposure parameter according to the first current exposure parameter and the second current exposure parameter; determining the weight of the first type of channel and the weight of the second type of channel based on the correlation information; obtaining a first recovery image signal according to the weight of the first type of channel and the weight of the second type of channel, and the normalized first sub-processing image signal and the normalized second sub-processing image signal;

Optionally, the post-processing module may include: the color recovery processing module comprises a first processing submodule, a second processing submodule, a color recovery submodule and a fourth processing submodule;

the color recovery submodule is used for acquiring a first current exposure parameter corresponding to the first channel, a second current exposure parameter corresponding to the second channel and associated information between the first channel and the second channel; normalizing the first sub-processed image signal and the second sub-processed image signal to be under the same exposure parameter according to the first current exposure parameter and the second current exposure parameter; determining the weight of the first type of channel and the weight of the second type of channel based on the association information; obtaining a second recovery image signal according to the weight of the first type channel and the weight of the second type channel, and the normalized first sub-processing image signal and the normalized second sub-processing image signal;

Optionally, the post-processing module may include: the color recovery processing module comprises a first processing sub-module, a second processing sub-module, a color recovery sub-module, a third processing sub-module, a fourth processing sub-module and a fifth processing sub-module;

the second processing submodule is used for acquiring a second decomposition image signal and preprocessing the second decomposition image signal to obtain a second sub-processed image signal, wherein the preprocessing comprises at least one processing mode of dead pixel correction, black level correction, digital gain and noise reduction;

the color recovery sub-module is used for acquiring a first current exposure parameter corresponding to the first channel, a second current exposure parameter corresponding to the second channel and associated information between the first channel and the second channel; normalizing the first sub-processed image signal and the second sub-processed image signal to be under the same exposure parameter according to the first current exposure parameter and the second current exposure parameter; determining the weight of the first type of channel and the weight of the second type of channel based on the correlation information; obtaining a first recovery image signal and a second recovery image signal according to the weight of the first type of channel and the weight of the second type of channel, and the normalized first sub-processing image signal and the normalized second sub-processing image signal;

the fourth processing submodule is used for processing the second recovered image signal to obtain a fourth sub-processed image signal;

Optionally, the post-processing module may include: the color recovery processing module comprises a first processing submodule, a second processing submodule, a color recovery submodule, a third processing submodule, a fourth processing submodule and a fifth processing submodule;

the color recovery submodule is used for acquiring a first current exposure parameter corresponding to the first channel, a second current exposure parameter corresponding to the second channel and associated information between the first channel and the second channel; normalizing the first sub-processed image signal and the second sub-processed image signal to be under the same exposure parameter according to the first current exposure parameter and the second current exposure parameter; determining the weight of the first type of channel and the weight of the second type of channel based on the association information; obtaining a first recovery image signal and a second recovery image signal according to the weight of the first type of channel and the weight of the second type of channel, and the normalized first sub-processing image signal and the normalized second sub-processing image signal;

and the fifth processing submodule is used for processing the third sub-processed image signal and the fourth sub-processed image signal to obtain a first output image signal and a second output image signal.

Optionally, the processing unit may further include a preprocessing module;

and the preprocessing module is used for acquiring the image signal output by the image sensor, preprocessing the image signal and sending the preprocessed image signal to the signal decomposition module.

In summary, the processing unit 16 can be implemented in various ways, which will be described separately below.

The first embodiment:

as shown in fig. 12, the processing unit 16 includes a signal decomposition module and a post-processing module. The signal decomposition module is used for logically decomposing a visible light signal and a near infrared light signal of an input image signal and decomposing the image signal into a first decomposition image signal and a second decomposition image signal; the post-processing module processes the first decomposition image signal and the second decomposition image signal and outputs a first output image signal.

The image signal transmitted to the processing unit by the image sensor simultaneously comprises a visible light signal and a near infrared light signal, so that the signal decomposition module logically decomposes the two image signals and outputs a first decomposed image signal and a second decomposed image signal after decomposition.

For the signal decomposition module, one processing manner may be to perform upsampling on R, G, and B signals of visible light and NIR signals of near infrared respectively (bilinear upsampling may be used, or other upsampling methods may be used), to obtain image signals of R, G, B, and NIR, where the four image signals are full-resolution image signals, and then combine the R, G, and B image signals into a visible light image signal to be output as a first decomposed image signal, and output the NIR image signal of near infrared as a second decomposed image signal.

For the signal decomposition module, another processing manner may be to perform interpolation operation on the rgbiir image signal, and the interpolation operation may be interpolation based on edge judgment. When interpolation is carried out, a channel with better imaging quality can be used as a guide to guide a channel with poorer imaging quality to carry out edge judgment interpolation. The imaging quality can be judged by gain, for example, when the exposure gain of the NIR channel is smaller than that of the R, G, B channels, the NIR channel is used to guide the R, G, B channels to perform edge decision interpolation; and when the exposure gain of the R, G and B channels is smaller than that of the NIR channel, the R, G and B channels are adopted to guide the NIR channel to carry out edge judgment interpolation. And obtaining R, G, B and NIR image signals after interpolation, combining the R, G and B image signals into a visible light image signal to be output as a first decomposition image signal, and outputting a near infrared NIR image signal as a second decomposition image signal.

The post-processing module is used for carrying out combined processing on the first decomposition image signal and the second decomposition image signal to obtain a first output image signal. The post-processing module can be implemented in a variety of ways.

A first implementation manner of the post-processing module is shown in fig. 13, where the first processing sub-module may perform one or more of dead pixel correction, black level correction, data gain, and noise reduction on the first decomposed image signal to obtain a first sub-processed image signal; the second processing sub-module may perform one or more of dead pixel correction, black level correction, data gain, and noise reduction on the second decomposed image signal to obtain a second sub-processed image signal. Normalizing the first sub-processed image signal and the second sub-processed image signal to be in the same gain and exposure time, one processing manner may be to adjust the second sub-processed image signal according to the gain g1 and the exposure time t1 of the RGB channel, the gain g2 and the exposure time t2 of the NIR channel as follows:

performing joint processing on the first sub-processed image signal (RGB image signal) and the adjusted second sub-processed image signal (NIR' image signal) by using a pre-calibrated coefficient matrix a to obtain a first recovered image signal with recovered color, wherein the processing manner is as follows:

furthermore, it is not excluded to use other ways to achieve the purpose of normalizing the first sub-processed image signal and the second sub-processed image signal to the same exposure time and gain, such as scaling the first sub-processed image signal, or scaling the coefficient matrix a, etc.

The third processing sub-module further processes the first restored image signal including, but not limited to, digital gain, white balance, color correction, curve mapping, noise reduction, enhancement, etc., to finally obtain a colored first output image signal.

A second implementation manner of the post-processing module is shown in fig. 14, and the first processing sub-module and the second processing sub-module may adopt the same implementation manner as the first processing sub-module and the second processing sub-module in the embodiment shown in fig. 13, and are not described herein again. One processing manner of the color restoration sub-module may be to directly output the second sub-processed image signal as a second restored image signal, or to output the first sub-processed image signal and the second sub-processed image signal as a second restored image signal after weighting them. The fourth processing sub-module further processes the second restored image signal, including but not limited to digital gain, white balance, color correction, curve mapping, noise reduction, enhancement, etc., to finally obtain a black and white second output image signal.

A third implementation manner of the post-processing module is shown in fig. 15, and the first processing sub-module and the second processing sub-module may adopt the same implementation manner as that of the first processing sub-module and the second processing sub-module in the embodiment shown in fig. 13. The color restoration sub-module may output the first restored image signal in the same implementation as the color restoration sub-module in the embodiment shown in fig. 13 and the second restored image signal in the same implementation as the color restoration sub-module in the embodiment shown in fig. 14. The third processing sub-module may obtain the third sub-processed image signal in the same implementation manner as the third processing sub-module in the embodiment shown in fig. 13, and the fourth processing sub-module may obtain the fourth sub-processed image signal in the same implementation manner as the fourth processing sub-module in the embodiment shown in fig. 14. And the fifth processing submodule processes the third sub-processed image signal and the fourth sub-processed image signal to obtain a first output image signal, and the processing mode of the fifth processing submodule includes but is not limited to noise reduction, fusion, enhancement and the like.

The second embodiment:

as shown in fig. 16, the processing unit 16 includes a signal decomposition module and a post-processing module. The signal decomposition module is used for carrying out logic decomposition on an input image signal and decomposing the image signal into a first decomposition image signal and a second decomposition image signal; the post-processing module processes the first decomposition image signal and the second decomposition image signal and outputs a first output image signal and a second output image signal.

The signal decomposition module may adopt the same implementation manner as that of the signal decomposition module in the first implementation manner, and is not described herein again.

The implementation of the post-processing module is shown in fig. 17, and the first processing sub-module, the second processing sub-module, the color recovery sub-module, the third processing sub-module, and the fourth processing sub-module may adopt the same implementation as the corresponding modules in the embodiment shown in fig. 15. The fifth processing sub-module processes the third sub-processed image signal and the fourth sub-processed image signal in a processing manner including but not limited to noise reduction, fusion, enhancement, and the like, to obtain a first color output image signal and a second black-and-white output image signal.

The third embodiment:

as shown in fig. 18, the processing unit 16 includes a preprocessing module, a signal decomposition module, and a post-processing module. The preprocessing module is used for preprocessing an input image signal and outputting a preprocessed image signal; the signal decomposition module is used for carrying out logic decomposition on the preprocessed image signal and decomposing the preprocessed image signal into a first decomposed image signal and a second decomposed image signal; the post-processing module processes the first decomposition image signal and the second decomposition image signal and outputs a first output image signal.

The preprocessing module preprocesses an input image signal to obtain a preprocessed image, wherein the preprocessing includes but is not limited to black level correction, dead pixel correction, digital gain, noise reduction and the like.

The signal decomposition module may adopt the same implementation manner as the signal decomposition module in the first embodiment, and is not described herein again.

The post-processing module may adopt the same implementation manner as that of the post-processing module in the first embodiment, and is not described herein again.

The fourth embodiment:

as shown in fig. 19, the processing unit 16 includes a preprocessing module, a signal decomposition module, and a post-processing module. The preprocessing module is used for preprocessing an input image signal and outputting a preprocessed image signal; the signal decomposition module is used for carrying out logic decomposition on the preprocessed image signal and decomposing the preprocessed image signal into a first decomposed image signal and a second decomposed image signal; the post-processing module processes the first decomposition image signal and the second decomposition image signal and outputs a first output image signal and a second output image signal.

The preprocessing module may adopt the same implementation manner as the preprocessing module in the third embodiment, and is not described herein again.

The post-processing module may adopt the same implementation manner as that of the post-processing module in the second embodiment, and is not described herein again.

The embodiment of the application provides an image processing method, which is applied to an imaging system, and the system comprises the following steps: an image sensor, a statistic unit and an exposure control unit; the image sensor comprises a plurality of channels; as shown in fig. 20, the method includes:

s201, the image sensor converts an optical signal into an image signal, wherein the optical signal comprises light components in a plurality of wave band ranges.

S202, the statistical unit obtains the image signals, extracts the image data of various channels in the image signals, respectively counts the image data of various channels to obtain the statistical data of various channels, and sends the statistical data of various channels to the exposure control unit.

S203, the exposure control unit receives the statistical data of various channels sent by the statistical unit, calculates the exposure parameters corresponding to the channels according to the statistical data of the channels aiming at any channel, and controls the brightness adjustment of the image data of the channels based on the exposure parameters.

Optionally, the image sensor includes: a first channel responsive to light components in the visible light band range and a second channel responsive to light components in the near infrared light band range, wherein the first channel comprises a plurality of color channels and the second channel comprises a near infrared channel;

in S202, the step of obtaining the statistical data of the various channels by respectively performing statistics on the image data of the various channels by the statistics unit may be specifically implemented by the following steps:

calculating an image data statistic value of a first type channel according to the image data of at least one color channel in the plurality of color channels to serve as statistic data of the first type channel;

and calculating an image data statistic value of the second channel as statistic data of the second channel according to the image data of the near-infrared channel.

Optionally, the system may further include: a light supplement unit; an image sensor including a second type of channel responsive to light components in a near infrared band range;

the method may further comprise the steps of:

if the statistical data of the second channel is larger than a first preset threshold value, controlling a light supplementing unit to reduce the intensity of transmitting the near infrared light;

Optionally, the system may further include: a processing unit;

after performing S201, the method may further include the steps of:

the processing unit acquires an image signal output by the image sensor, current exposure parameters corresponding to various channels and correlation information among the various channels, determines the correlation between every two channels according to the current exposure parameters corresponding to the various channels and the correlation information among the various channels, and removes the light component of another channel contained in one channel according to the correlation between every two channels.

Alternatively, the image sensor may include: a first type of channel responsive to light components in the visible light band range and a second type of channel responsive to light components in the near infrared light band range;

the processing unit acquires an image signal output by the image sensor, current exposure parameters corresponding to various channels and associated information among the various channels, determines the correlation between every two channels according to the current exposure parameters corresponding to the various channels and the associated information among the various channels, and removes the light component of another channel contained in one channel according to the correlation between every two channels, which can be specifically realized by the following steps:

acquiring an image signal output by an image sensor, current exposure parameters corresponding to a first channel and a second channel, and correlation information between the color of the first channel and the brightness of the second channel;

determining the weight of the first type of channel and the weight of the second type of channel based on the association information;

Alternatively, the image sensor may include: a first type of channel responsive to light components in the visible light band and a second type of channel responsive to light components in the near infrared band;

determining the weight of the first type of channel and the weight of the second type of channel according to the current exposure parameters corresponding to the first type of channel and the second type of channel and the associated information;

Optionally, after the step of obtaining, by the processing unit, the image signal output by the image sensor, the current exposure parameter corresponding to each type of channel, and the correlation information between each type of channels, determining the correlation between each two types of channels according to the current exposure parameter corresponding to each type of channel and the correlation information between each type of channel, and removing the light component of another type of channel included in one type of channel according to the correlation between each two types of channels, the method may further include the steps of:

the processing unit fuses the image data of the channels from which the light components of the other channels are removed to obtain fused image signals.

By applying the embodiment of the application, the statistical unit respectively carries out statistics on the image data of each type of channel, the exposure control unit calculates the exposure parameters corresponding to the type of channel according to the statistical data of the type of channel, controls the brightness adjustment of the image data of the type of channel based on the calculated exposure parameters, and carries out independent exposure on the image data of the type of channel aiming at the actual situation that the energy of the light components responded by different types of channels is different, so that the brightness of the image data of the type of channel is controlled within a proper brightness range, and the final imaging effect is improved.

It is noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrases "comprising a," "8230," "8230," or "comprising" does not exclude the presence of additional like elements in a process, method, article, or apparatus that comprises the element.

All the embodiments in the present specification are described in a related manner, and the same and similar parts among the embodiments may be referred to each other, and each embodiment focuses on differences from other embodiments. In particular, as for the embodiment of the image processing method, since it is basically similar to the embodiment of the imaging system, the description is relatively simple, and the relevant points can be referred to the partial description of the embodiment of the imaging system.

The above description is only for the preferred embodiment of the present application, and is not intended to limit the scope of the present application. Any modification, equivalent replacement, improvement and the like made within the spirit and principle of the present application are included in the protection scope of the present application.

Claims

1. An imaging system, characterized in that the system comprises: an image sensor, a statistic unit and an exposure control unit; the image sensor comprises a plurality of types of channels;

the exposure control unit is used for receiving the statistical data of the various channels sent by the statistical unit; aiming at any type of channel, calculating an exposure parameter corresponding to the type of channel according to the statistical data of the type of channel, and controlling the brightness adjustment of the image data of the type of channel based on the exposure parameter;

the system further comprises: a processing unit;

the processing unit is used for acquiring image signals output by the image sensor, current exposure parameters corresponding to the various channels and associated information among the various channels; determining the correlation between each two types of channels according to the current exposure parameters corresponding to each type of channel and the correlation information between each type of channel; and removing the light component of the other channel contained in the one channel according to the correlation between every two channels, wherein the correlation information is the influence of the image attribute of one channel on the image attribute of the other channel, and the correlation is used for expressing the degree and the size of the image attribute correlation information.

2. The system of claim 1, wherein the image sensor comprises: a first type of channel responsive to light components in the visible light band and a second type of channel responsive to light components in the near infrared band; the first type of channel comprises a plurality of color channels, and the second type of channel comprises a near infrared channel;

the statistical unit is specifically configured to:

3. The system according to claim 2, wherein the statistical unit is specifically configured to:

alternatively, the first and second electrodes may be,

partitioning the image signal to obtain a plurality of image signal blocks; for any image signal block, extracting image data of each color channel and image data of the near-infrared channel from the image signal block; calculating the image data mean value of each color channel and the image data mean value of the near-infrared channel according to the image data of each color channel and the image data of the near-infrared channel in each image signal block respectively; carrying out weighted summation on the image data mean values of all the color channels; taking the result of the weighted summation as the statistical data of the first channel, and taking the image data mean value of the near-infrared channel as the statistical data of the second channel;

alternatively, the first and second electrodes may be,

4. The system of claim 1, further comprising: a light supplement unit; the image sensor comprises a second type of channel responding to light components in the near infrared light wave band range;

the light supplementing unit is used for performing near-infrared light supplementing on a scene so that an input optical signal comprises near-infrared light;

5. The system according to claim 4, wherein the exposure control unit is specifically configured to:

6. The system of claim 4, wherein the image sensor further comprises: a first type of channel responsive to light components in the visible light band; the exposure control unit is specifically configured to:

calculating a first data offset of the first type channel according to the statistical data and the first target data of the first type channel, and calculating a first exposure gain according to the statistical data and the first target data of the first type channel if the first data offset is not within a first preset range;

if the first exposure time is equal to the second exposure time, controlling the light supplementing unit to reduce the intensity of emitting the near infrared light when the second exposure gain is smaller than a first preset gain threshold, and controlling the light supplementing unit to improve the intensity of emitting the near infrared light when the second exposure gain is larger than a second preset gain threshold;

7. The system of claim 1, wherein the image sensor comprises: a first type of channel responsive to light components in the visible light band range and a second type of channel responsive to light components in the near infrared light band range;

the processing unit is specifically configured to:

8. The system of claim 1, wherein the image sensor comprises: a first type of channel responsive to light components in the visible light band and a second type of channel responsive to light components in the near infrared band;

the processing unit is specifically configured to:

9. The system of claim 1, wherein the processing unit comprises a signal decomposition module and a post-processing module;

10. The system of claim 9, wherein the signal decomposition module is specifically configured to:

alternatively, the first and second electrodes may be,

acquiring an image signal, a first current exposure gain corresponding to the first type of channel and a second current exposure gain corresponding to the second type of channel; if the second current exposure gain is smaller than the first current exposure gain, performing edge judgment interpolation on the image data of the first type of channel according to the image data of the second type of channel in the image signal, and if the second current exposure gain is larger than the first current exposure gain, performing edge judgment interpolation on the image data of the second type of channel according to the image data of the first type of channel in the image signal; obtaining image signals of various color components of the visible light signals and image signals of near infrared light after interpolation; and combining the image signals of the color components to obtain a first decomposition image signal and outputting the first decomposition image signal, and outputting the image signal of the near infrared light as a second decomposition image signal.

11. The system of claim 9, wherein the post-processing module comprises: the color recovery processing module comprises a first processing submodule, a second processing submodule, a color recovery submodule and a third processing submodule;

12. The system of claim 9, wherein the post-processing module comprises: the color recovery processing module comprises a first processing submodule, a second processing submodule, a color recovery submodule and a fourth processing submodule;

the color recovery sub-module is configured to obtain a first current exposure parameter corresponding to the first type of channel, a second current exposure parameter corresponding to the second type of channel, and association information between the first type of channel and the second type of channel; normalizing the first sub-processed image signal and the second sub-processed image signal to be under the same exposure parameter according to the first current exposure parameter and the second current exposure parameter; determining the weight of the first type of channel and the weight of the second type of channel based on the association information; obtaining a second recovery image signal according to the weight of the first type channel and the weight of the second type channel, and the normalized first sub-processing image signal and the normalized second sub-processing image signal;

13. The system of claim 9, wherein the post-processing module comprises: the color recovery processing module comprises a first processing sub-module, a second processing sub-module, a color recovery sub-module, a third processing sub-module, a fourth processing sub-module and a fifth processing sub-module;

the color recovery sub-module is configured to obtain a first current exposure parameter corresponding to the first type of channel, a second current exposure parameter corresponding to the second type of channel, and association information between the first type of channel and the second type of channel; normalizing the first sub-processed image signal and the second sub-processed image signal to be under the same exposure parameter according to the first current exposure parameter and the second current exposure parameter; determining the weight of the first type of channel and the weight of the second type of channel based on the association information; obtaining a first recovery image signal and a second recovery image signal according to the weight of the first type channel and the weight of the second type channel, and the normalized first sub-processing image signal and the normalized second sub-processing image signal;

the fifth processing sub-module is configured to process the third sub-processed image signal and the fourth sub-processed image signal to obtain a first output image signal.

14. The system of claim 9, wherein the post-processing module comprises: the color recovery processing module comprises a first processing submodule, a second processing submodule, a color recovery submodule, a third processing submodule, a fourth processing submodule and a fifth processing submodule;

the color recovery sub-module is configured to obtain a first current exposure parameter corresponding to the first type of channel, a second current exposure parameter corresponding to the second type of channel, and association information between the first type of channel and the second type of channel; normalizing the first sub-processed image signal and the second sub-processed image signal to be under the same exposure parameter according to the first current exposure parameter and the second current exposure parameter; determining the weight of the first type of channel and the weight of the second type of channel based on the correlation information; obtaining a first recovery image signal and a second recovery image signal according to the weight of the first type channel and the weight of the second type channel, and the normalized first sub-processing image signal and the normalized second sub-processing image signal;

15. An image processing method applied to an imaging system, the system comprising: an image sensor, a statistic unit and an exposure control unit; the image sensor comprises a plurality of types of channels; the method comprises the following steps:

the exposure control unit receives the statistical data of the various channels sent by the statistical unit, calculates the exposure parameter corresponding to the channel according to the statistical data of the channel aiming at any channel, and controls the brightness adjustment of the image data of the channel based on the exposure parameter;

the system further comprises: a processing unit;

the processing unit acquires an image signal output by the image sensor, current exposure parameters corresponding to various channels and correlation information among the various channels, determines correlation between every two channels according to the current exposure parameters corresponding to the various channels and the correlation information among the various channels, and removes light components of another channel contained in one channel according to the correlation between every two channels, wherein the correlation information is the influence of the image attribute of one channel on the image attribute of the other channel, and the correlation is used for representing the degree and the size of the image attribute correlation information.

16. The method of claim 15, wherein the image sensor comprises: a first channel type responsive to light components in the visible light band range and a second channel type responsive to light components in the near infrared light band range, the first channel type comprising a plurality of color channels and the second channel type comprising a near infrared channel;

17. The method of claim 15, wherein the system further comprises: a light supplement unit; the image sensor comprises a second type of channel responding to light components in a near infrared light waveband range;

the method further comprises the following steps: