CN110602420A

CN110602420A - Camera, black level adjusting method and device

Info

Publication number: CN110602420A
Application number: CN201910944822.8A
Authority: CN
Inventors: 於敏杰; 聂鑫鑫; 罗丽红
Original assignee: Hangzhou Hikvision Digital Technology Co Ltd
Current assignee: Hangzhou Hikvision Digital Technology Co Ltd
Priority date: 2019-09-30
Filing date: 2019-09-30
Publication date: 2019-12-20
Anticipated expiration: 2039-09-30
Also published as: CN110602420B

Abstract

The embodiment of the application provides a camera, a black level adjusting method and a black level adjusting device, wherein the camera comprises a single image sensor and a processing chip, and the image sensor is used for generating and outputting a first image signal to the processing chip according to a first exposure parameter in a first working mode; the processing chip is used for carrying out black level adjustment on the first image signal according to a first adjustment mode to obtain a first adjustment signal; the image sensor is also used for alternately generating and outputting a second image signal and a third image signal to the processing chip through multiple exposures in a second working mode; the processing chip is further used for carrying out black level adjustment on the second image signal according to a second adjustment mode to obtain a second adjustment signal; and performing black level adjustment on the third image signal according to a third adjustment mode to obtain a third adjustment signal. The embodiment of the application can solve the problem that image picture jumping is serious when black level adjustment is carried out on multi-frame image data.

Description

Camera, black level adjusting method and device

Technical Field

The present application relates to the field of image processing technologies, and in particular, to a method and an apparatus for adjusting a camera and a black level.

Background

An image sensor is a device for converting an optical signal into an electronic signal, and image data obtained by the image sensor usually needs to be subjected to black level calibration to determine the black level of the image sensor before generating a digital image, so as to obtain the digital image with full contrast and fine details in a shadow area.

In the conventional black level adjustment method, the black level of the image data collected by each frame of image sensor is usually adjusted inside the image sensor, for example, the black level of the black pixel area in the image data is averaged to obtain the black level offset value of the image data, and then the black level of the effective pixel area of the image data is adjusted according to the black level offset value. However, in the conventional black level adjustment method, when the exposure conditions of a plurality of image signals collected by the same image sensor are different, the problem that the black level adjustment of an image is unstable is easily caused.

Disclosure of Invention

The embodiment of the application provides a camera, a black level adjusting method and a black level adjusting device, and aims to solve the problem that in the prior art, when black level adjustment is carried out on multi-frame image data, the black level adjustment is unstable easily.

In a first aspect, an embodiment of the present application provides a camera, which includes a single image sensor and a processing chip, an output terminal of the image sensor is connected to the processing chip, where:

the image sensor is used for generating and outputting a first image signal to the processing chip according to a first exposure parameter in a first working mode;

the processing chip is used for carrying out black level adjustment on the first image signal according to a first adjustment mode when the first image signal is received, so as to obtain a first adjustment signal;

the image sensor is further used for alternately generating and outputting a second image signal and a third image signal to the processing chip through multiple exposures in a second working mode, wherein the second image signal is generated according to a second exposure parameter, and the third image signal is generated according to a third exposure parameter;

the processing chip is further used for carrying out black level adjustment on the second image signal according to a second adjustment mode when the second image signal is received, so as to obtain a second adjustment signal; when the third image signal is received, carrying out black level adjustment on the third image signal according to a third adjustment mode to obtain a third adjustment signal; and alternately outputting the second adjusting signal and the third adjusting signal.

In one possible implementation, the method further includes an exposure parameter control unit, where:

the exposure parameter control unit is configured to alternately output the second exposure parameter and the third exposure parameter to the image sensor in the second operating mode.

In a possible implementation, the second exposure parameter is different from at least one of the third exposure parameters, the at least one exposure parameter is one or more of exposure time, exposure gain, aperture size, and the exposure gain includes analog gain, and/or digital gain.

In a possible implementation manner, the first adjustment manner is specifically:

determining a black level value of a current frame first image signal according to a received black level statistic value of the current frame first image signal and black level information of a received historical frame first image signal, wherein the historical frame first image signal is a previous frame first image signal of the current frame first image signal;

determining the black level offset of the current frame first image signal according to the black level value of the current frame first image signal and a first target black level value;

and performing black level adjustment on the current frame first image signal according to the black level offset of the current frame first image signal to obtain a first adjustment signal corresponding to the current frame first image signal.

In a possible implementation manner, the second adjustment manner is specifically:

determining a black level value of a current frame second image signal according to a received black level statistic value of the current frame second image signal and black level information of a received historical frame second image signal, wherein the historical frame second image signal is a previous frame second image signal of the current frame second image signal;

determining the black level offset of the current frame second image signal according to the black level value of the current frame second image signal and a second target black level value;

and performing black level adjustment on the current frame second image signal according to the black level offset of the current frame second image signal to obtain a second adjustment signal corresponding to the current frame second image signal.

In a possible implementation manner, the third adjustment manner is specifically:

determining a black level value of a current frame third image signal according to a received black level statistic value of the current frame third image signal and black level information of a received historical frame third image signal, wherein the historical frame third image signal is a previous frame third image signal of the current frame third image signal;

determining the black level offset of the current frame third image signal according to the black level value of the current frame third image signal and a third target black level value;

and performing black level adjustment on the current frame third image signal according to the black level offset of the current frame third image signal to obtain a third adjustment signal corresponding to the current frame third image signal.

In one possible implementation, the processing chip is further configured to:

processing the first adjusting signal in the first working mode to obtain and output a first processed image;

in the second working mode, processing the second adjustment signal to obtain and output a second processed image, and processing the third adjustment signal to obtain and output a third processed image; or, in the second working mode, the second adjustment signal and the third adjustment signal are subjected to fusion processing to obtain and output a processed image.

In a second aspect, an embodiment of the present application provides a black level adjustment method, which is applied to a camera, where the camera includes a single image sensor and a processing chip, and an output terminal of the image sensor is connected to the processing chip, and the method includes:

generating a first image signal according to a first exposure parameter in a first working mode through the image sensor, and performing black level adjustment on the first image signal according to a first adjustment mode through the processing chip to obtain a first adjustment signal;

alternately generating a second image signal and a third image signal by multiple exposures in a second working mode through the image sensor, wherein the second image signal is generated according to a second exposure parameter, and the third image signal is generated according to a third exposure parameter; performing black level adjustment on the second image signal through the processing chip according to a second adjustment mode to obtain a second adjustment signal; performing black level adjustment on the third image signal through the processing chip according to a third adjustment mode to obtain a third adjustment signal; and alternately outputting the second adjusting signal and the third adjusting signal through the processing chip.

In one possible implementation, the camera further includes an exposure parameter control unit, and the method further includes:

alternately outputting the second exposure parameter and the third exposure parameter to the image sensor through the exposure parameter control unit in the second working mode;

in a possible implementation manner, the performing, by the processing chip, black level adjustment on the first image signal according to a first adjustment manner to obtain a first adjustment signal includes:

according to the black level offset of the current frame first image signal, black level adjustment is carried out on the current frame first image signal to obtain a first adjustment signal corresponding to the current frame first image signal

In a possible implementation manner, the performing, by the processing chip, black level adjustment on the second image signal according to a second adjustment manner to obtain a second adjustment signal includes:

determining a black level value of a current frame second image signal according to a received black level statistic value of the current frame second image signal and black level value information of a received historical frame second image signal, wherein the historical frame second image signal is a previous frame second image signal of the current frame second image signal;

In a possible implementation manner, the performing, by the processing chip, black level adjustment on the third image signal according to a third adjustment manner to obtain a third adjustment signal includes:

In one possible implementation, the method further includes:

processing the first adjusting signal through the processing chip in the first working mode to obtain and output a first processing image;

processing the second adjustment signal by the processing chip in the second working mode to obtain and output a second processed image, and processing the third adjustment signal to obtain and output a third processed image; or, in the second working mode, the second adjustment signal and the third adjustment signal are subjected to fusion processing to obtain and output a processed image.

In a third aspect, an embodiment of the present application provides a black level adjustment apparatus, including:

the first processing module is used for receiving a first image signal when the image sensor is in a first working mode, and carrying out black level adjustment on the first image signal according to a first adjustment mode to obtain a first adjustment signal;

the second processing module is configured to alternately receive a second image signal and a third image signal when the image sensor is in a second operating mode, perform black level adjustment on the second image signal according to a second adjustment mode when the second image signal is received to obtain a second adjustment signal, perform black level adjustment on the third adjustment signal according to a third adjustment mode when the third image signal is received to obtain a third adjustment signal, and alternately output the second adjustment signal and the third adjustment signal.

In a possible implementation manner, the first processing module is specifically configured to:

In a possible implementation manner, the second processing module is specifically configured to:

In a possible implementation manner, the system further includes a third processing module, where the third processing module is configured to:

The camera and the black level adjusting method and device provided by the embodiment of the application comprise a single image sensor and a single processing chip, wherein the output end of the image sensor is connected with the processing chip, the camera comprises two working modes, in the first working mode, the image sensor generates and outputs a first image signal to the processing chip according to a first exposure parameter, and the processing chip performs black level adjustment on the first image signal according to a first adjusting mode when receiving the first image signal to obtain a first adjusting signal; in a second working mode, the image sensor generates and outputs a second image signal to the processing chip according to the second exposure parameter, and generates and outputs a third image signal to the processing chip according to a third exposure parameter; when the processing chip receives the second image signal, black level adjustment is carried out on the second image signal according to a second adjustment mode to obtain a second adjustment signal; and when the third image signal is received, carrying out black level adjustment on the third image signal according to a third adjustment mode to obtain a third adjustment signal. The camera provided by the embodiment of the application can correspondingly adjust the working mode of the camera under different conditions, when the camera is in the second working mode, the second image signal and the third image signal are firstly distinguished aiming at multiple frames of second image signals and third image signals generated by the image sensor when black level adjustment is carried out, then black level adjustment processing is carried out respectively aiming at the second image signal and the third image signal according to different adjustment modes, the situation that the black level adjustment processing is carried out in the image sensor by the existing scheme is avoided, when the black level adjustment is carried out on the image signals, the black level adjustment processing is carried out respectively according to the corresponding adjustment modes aiming at the image signals under different exposure parameters, multiple paths of image signals are separately adjusted, and the black level adjustment of the images is more stable.

Drawings

In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings needed to be used in the description of the embodiments or the prior art will be briefly introduced below, and it is obvious that the drawings in the following description are some embodiments of the present application, and for those skilled in the art, other drawings can be obtained according to these drawings without inventive exercise.

Fig. 1 is a schematic structural diagram of a camera according to an embodiment of the present disclosure;

fig. 2 is a schematic target surface view of an image sensor provided in an embodiment of the present application;

fig. 3 is a schematic view of an operating principle of a camera according to an embodiment of the present application;

fig. 4 is a schematic diagram of a distribution manner of photosensitive channels of an RGBW sensor provided in an embodiment of the present application;

fig. 5 is a schematic diagram of a distribution manner of photosensitive channels of a Bayer sensor according to an embodiment of the present disclosure;

FIG. 6 is a schematic diagram illustrating a distribution of photosensitive channels of an RGB-IR sensor according to an embodiment of the present disclosure;

FIG. 7 is a schematic diagram of an induction curve of an RGBW sensor provided by an embodiment of the present application;

fig. 8 is a schematic structural diagram of a camera according to another embodiment of the present application;

fig. 9 is a schematic view of an exposure sequence of an image sensor in a first operation mode according to an embodiment of the present disclosure;

fig. 10 is a first schematic view of an exposure sequence of the image sensor in the second operating mode according to the embodiment of the present application;

fig. 11 is a schematic diagram of an exposure sequence of the image sensor in the second operating mode according to the embodiment of the present application;

fig. 12 is a schematic diagram illustrating black level adjustment processing in a first operating mode according to an embodiment of the present disclosure;

fig. 13 is a schematic diagram illustrating a plurality of first image signals according to an embodiment of the present disclosure;

fig. 14 is a first schematic diagram illustrating black level adjustment processing in a second operating mode according to an embodiment of the present disclosure;

fig. 15 is a second schematic diagram of black level adjustment processing in the second operating mode according to the embodiment of the present application;

FIG. 16 is a first schematic view of a camera process provided in an embodiment of the present application;

fig. 17 is a second schematic view of a camera process provided in the embodiment of the present application;

fig. 18 is a third schematic view of a camera process provided in the embodiment of the present application;

fig. 19 is a fourth schematic view of a camera process provided in the embodiments of the present application;

fig. 20 is a schematic flowchart of a black level adjustment method according to an embodiment of the present application;

fig. 21 is a schematic structural diagram of a black level adjustment apparatus according to an embodiment of the present disclosure.

Detailed Description

In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, but not all, embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

First, a concept of partial nouns referred to in the present application will be explained.

Black pixel area: in an image sensor, there are some black-shaded pixels, which cannot receive the energy of the external light, and are generally used for black level statistics. The Black pixel region is also commonly referred to as a dark pixel (OB) region, an Optical Black region, a Black line, a light-shielding region, a Black level reference pixel, or the like.

Effective pixel area: the image sensor is used for responding to the area of the outside light energy normally.

Fig. 1 is a schematic structural diagram of a camera according to an embodiment of the present disclosure, and as shown in fig. 1, a camera 10 includes an image sensor 11 and a processing chip 12, wherein,

the image sensor 11 is configured to generate and output a first image signal to the processing chip according to a first exposure parameter in a first working mode;

the processing chip 12 is configured to, when receiving the first image signal, perform black level adjustment on the first image signal according to a first adjustment mode to obtain a first adjustment signal;

the image sensor 11 is further configured to alternately generate and output a second image signal and a third image signal to the processing chip through multiple exposures in a second operating mode, where the second image signal is generated according to a second exposure parameter, and the third image signal is generated according to a third exposure parameter;

the processing chip 12 is further configured to, when receiving the second image signal, perform black level adjustment on the second image signal according to a second adjustment mode to obtain a second adjustment signal; when the third image signal is received, carrying out black level adjustment on the third image signal according to a third adjustment mode to obtain a third adjustment signal; alternately outputting the second adjustment signal and the third adjustment signal.

The image sensor 11 is a photosensitive device that performs light sensing using a target surface. On the target surface of the image sensor 11, a black pixel region and an effective pixel region are generally included, and black pixel region data and effective pixel region data can be output.

The image sensor supports two working modes and can be switched between the two working modes. And in the first working mode, acquiring and outputting a first image signal according to the first exposure parameter. In the second working mode, a second image signal and a third image signal can be acquired and output simultaneously in a period of time through multiple exposures, wherein the second image signal is generated according to a second exposure parameter, and the third image signal is generated according to a third exposure parameter.

The first image signal, the second image signal, and the third image signal include both black pixel area data and effective pixel area data.

The second exposure parameter and the third exposure parameter may be the same exposure parameter or different exposure parameters. The exposure parameters include, and are not limited to, exposure time, analog gain, digital gain, and the like. When the second exposure parameter and the third exposure parameter are different exposure parameters, it is usually indicated that the gain and the exposure time are different.

Fig. 2 is a schematic diagram of a target surface of an image sensor according to an embodiment of the present disclosure, and fig. 2 shows distributions of black pixel regions and effective pixel regions on the target surface of two more common image sensors 11.

The target surface 21 includes a black pixel region 211 and an effective pixel region 212, and the black pixel region 211 on the target surface 21 is distributed around the effective pixel region 212. On the target surface 22, a black pixel region 221 and an effective pixel region 222 are included, and the black pixel region 221 on the target surface 22 is distributed over the effective pixel region 222.

In the embodiment of the present application, light can be prevented from reaching the target surface of the image sensor 11 by different methods, so that the corresponding area of the image sensor 11, that is, the black pixel area of the image sensor 11, cannot sense light. Specifically, a light-impermeable material, such as a metal, a baffle, or the like, may be used to block one or more rows, one or more columns of pixels in the target surface of the image sensor 11, so that pixel regions in one or more rows, one or more columns of pixels in the target surface do not have light passing therethrough, which are black pixel regions, and regions excluding the black pixel regions are effective pixel regions, which are normally sensitive to light.

It is understood that the method for setting the black pixel region is only an example, and other methods for setting a partial region of the target surface of the image sensor 11 to be not photosensitive so that the image sensor 11 forms the black pixel region can achieve similar effects. Meanwhile, the distribution of the black pixel regions shown in fig. 2 is merely an illustration, and the specific distribution may be set according to actual needs, and the black pixel regions of any shape, size and region may be set.

When no light passes through the black pixel region on the target surface of the image sensor 11, the pixel value of the black pixel region should be 0 theoretically, but due to some factors, the black level value of the black pixel region is not 0, and the black level correction is required.

The camera in the embodiment of the application comprises two working modes, namely a first working mode and a second working mode. Fig. 3 is a schematic view of a working principle of a camera according to an embodiment of the present disclosure, and as shown in fig. 3, the camera mainly includes an image capturing unit and a black level adjusting unit, where the image capturing unit is configured to capture an image and generate an image signal, and the image capturing unit may include a lens besides an image sensor. The black level adjusting unit is a part of the processing chip and is used for adjusting the black level of the image signal. The black level adjustment unit is a logic platform containing a signal processing algorithm or a program, and the platform may be a computer based on an X86 or ARM architecture, or may be a Field-programmable gate Array (FPGA) logic circuit.

The scheme provided by the embodiment of the application comprises two working modes and supports mutual and linkage switching of the two modes, wherein the linkage switching refers to that when the image acquisition unit is in the first working mode, the black level adjustment unit is switched to the corresponding first working mode; when the image acquisition unit is in the second working mode, the black level adjustment unit is switched to the corresponding second working mode, that is, when the working modes are switched, the image acquisition unit and the black level adjustment unit can be simultaneously switched to the corresponding working modes to cooperatively work.

When the image acquisition unit is in a first working mode, the image acquisition unit can be exposed for multiple times under a certain exposure condition and output a first image signal; and in the second working mode, two image signals can be acquired and output through time-sharing multiple exposure.

A black level adjusting unit capable of performing black level adjustment on the first image signal and outputting a first adjustment signal in the first operation mode; in the second operation mode, the black level adjustment unit can perform black level adjustment on the second image signal and the third image signal, respectively, and output a second adjustment signal and a third adjustment signal. The black level adjusting unit obtains the black level deviation value of the image by counting the black pixel area and calculating, so that the black level of the effective pixel area is adjusted, and the brightness, the color and the like of the image can be ensured to truly reflect the environmental conditions.

In a first working mode, the image acquisition unit acquires a first image signal and transmits the first image signal to the black level adjustment unit for processing, and the black level adjustment unit processes the first image signal and outputs a first adjustment signal; and in the second working mode, the image acquisition unit acquires a second image signal and a third image signal and transmits the second image signal and the third image signal to the black level adjustment unit for processing, and the black level adjustment unit processes the second image signal and the third image signal and outputs a second adjustment signal and a third adjustment signal.

The image acquisition unit mainly refers to an image sensor, and acquires a second image signal and a third image signal through multiple exposures in a second working mode. Wherein the second image signal is generated according to a second exposure condition and the third image signal is generated according to a third exposure condition. The black level adjusting unit can perform black level adjustment processing on the first image signal in the first operation mode, and the black level adjusting unit can perform black level adjustment processing on the second image signal and the third image signal in the second operation mode.

In the first working mode, the image acquisition unit generates a first image signal according to the first exposure parameter, and sends the first image signal to the black level adjustment unit in the processing chip 12, and the black level adjustment unit performs black level adjustment on the first image signal according to the first adjustment mode to obtain a first adjustment signal. Since the target surface of the image sensor 11 includes the black pixel region and the effective pixel region, the black pixel region corresponding to the first image signal and the effective pixel region corresponding to the first image signal are included in the first image signal. The first image signal may be one of a plurality of image signals in a video sequence, which are generated by the image sensor 11 under the same exposure condition, i.e. the corresponding exposure parameter is the first exposure parameter.

In the second working mode, the image acquisition unit generates a second image signal according to the second exposure parameter and sends the second image signal to the black level adjustment unit in the processing chip 12, and also generates a third image signal according to the third exposure parameter and sends the third image signal to the black level adjustment unit in the processing chip 12, and the black level adjustment unit performs black level adjustment on the second image signal according to the second adjustment mode to obtain a second adjustment signal, and performs black level adjustment on the third image signal according to the third adjustment mode to obtain a third adjustment signal. Wherein the second image signal may belong to one of a plurality of image signals in a second video sequence and the third image signal may belong to one of a plurality of image signals in a third video sequence.

The black level adjustment is generally to set an offset value and then adjust the image signal according to the offset value. After receiving the first image signal, the processing chip 12 obtains, for each current frame first image signal, the black level of the black pixel region corresponding to the current frame first image signal and the black level information of the black pixel region corresponding to the historical frame first image signal. The black level of the black pixel region corresponding to the current frame first image signal is the black level of each pixel point of the corresponding black pixel region on the current frame first image signal, and if the first image signal has a plurality of photosensitive channels, the black level on each photosensitive channel needs to be acquired. The historical frame first image signal is a previous frame first image signal of the current frame first image signal, and has a certain correlation with the current frame first image signal, for example, the current frame first image signal and the historical frame first image signal may be two frame image signals in the same video sequence. The black level information of the black pixel region corresponding to the first image signal of the history frame may be the black level of the black pixel region corresponding to the first image signal of the history frame, or may be the black level shift amount of the first image signal of the history frame.

Then, in the first operating mode, the first adjusting manner may be that the processing chip 12 obtains a black level offset of the current frame first image signal according to the black level of the black pixel region corresponding to the current frame first image signal and the black level information of the black pixel region corresponding to the historical frame first image signal, and performs black level adjusting processing on the effective pixel region of the current frame first image signal according to the black level offset of the current frame first image signal to obtain a first adjusting signal.

In the second operation mode, the processing chip 12 will process the second image signal and the third image signal separately. In the second image signal processing, the second adjustment method may be to perform black level adjustment processing on the current frame second image signal using the black level of the black pixel region corresponding to the current frame second image signal and the black level information of the black pixel region corresponding to the history frame second image signal, and similarly, in the third image signal processing, the third adjustment method may be to perform black level adjustment processing on the current frame third image signal using the black level of the black pixel region corresponding to the current frame third image signal and the black level information of the black pixel region corresponding to the history frame third image signal.

The camera provided by the embodiment of the application comprises a single image sensor and a processing chip, wherein the output end of the image sensor is connected with the processing chip, the camera comprises two working modes, in the first working mode, the image sensor generates and outputs a first image signal to the processing chip according to a first exposure parameter, and the processing chip performs black level adjustment on the first image signal according to a first adjustment mode when receiving the first image signal to obtain a first adjustment signal; in a second working mode, the image sensor generates and outputs a second image signal to the processing chip according to the second exposure parameter, and generates and outputs a third image signal to the processing chip according to a third exposure parameter; when the processing chip receives the second image signal, black level adjustment is carried out on the second image signal according to a second adjustment mode to obtain a second adjustment signal; and when the third image signal is received, carrying out black level adjustment on the third image signal according to a third adjustment mode to obtain a third adjustment signal. The camera provided by the embodiment of the application can correspondingly adjust the working mode of the camera under different conditions, when the camera is in the second working mode, the second image signal and the third image signal are firstly distinguished aiming at multiple frames of second image signals and third image signals generated by the image sensor when black level adjustment is carried out, then black level adjustment processing is carried out respectively aiming at the second image signal and the third image signal according to different adjustment modes, the situation that the black level adjustment processing is carried out in the image sensor by the existing scheme is avoided, when the black level adjustment is carried out on the image signals, the black level adjustment processing is carried out respectively according to the corresponding adjustment modes aiming at the image signals under different exposure parameters, multiple paths of image signals are separately adjusted, and the black level adjustment of the images is more stable.

The principle and usage of the camera will be described in detail below with reference to fig. 4 and by using a specific embodiment.

First, the image sensor will be explained. The image sensor in this application includes a plurality of sensitization channels, and every sensitization channel is used for responding to the light of visible light wave band, and/or, responds to the light of near infrared wave band. That is, each photosensitive channel can sense light in a visible light band and can sense light in a near-infrared band, so that the first image signal can be guaranteed to have complete resolution and a pixel value is not lost.

Illustratively, the image sensor may be an RGBW sensor, a Bayer sensor, or an RGB-IR sensor, and other arrangements of sensors may be used. Fig. 4 is a schematic diagram of a distribution manner of photosensitive channels of an RGBW sensor provided in an embodiment of the present application, fig. 5 is a schematic diagram of a distribution manner of photosensitive channels of a Bayer sensor provided in an embodiment of the present application, and fig. 6 is a schematic diagram of a distribution manner of photosensitive channels of an RGB-IR sensor provided in an embodiment of the present application, as shown in fig. 4 and fig. 5, a distribution manner of an R photosensitive channel, a G photosensitive channel, a B photosensitive channel, and a W photosensitive channel in an RGBW sensor may refer to fig. 4, and a distribution manner of an R photosensitive channel, a G photosensitive channel, and a B photosensitive channel in a Bayer sensor may refer to fig. 5. Illustratively, the image sensor may be an RGB-IR sensor, wherein each IR sensitive channel in the RGB-IR sensor may sense light in the near infrared band, but not in the visible band. The distribution of the R, G, B and W photosensitive channels in the RGB-IR sensor can be seen in fig. 6.

In other embodiments, some of the photosensitive channels may sense only light in the near infrared band and not light in the visible band, so as to ensure complete resolution in the first image signal without missing pixel values. As an example, the plurality of photosensitive channels may include at least two of an R photosensitive channel, a G photosensitive channel, a B photosensitive channel, and an IR photosensitive channel. The R light sensing channel is used for sensing light of a red light wave band and a near infrared wave band, the G light sensing channel is used for sensing light of a green light wave band and a near infrared wave band, the B light sensing channel is used for sensing light of a blue light wave band and a near infrared wave band, and the IR light sensing channel is used for sensing light of a near infrared wave band.

Further, the image sensor may also be an RGB sensor, wherein when the image sensor is an RGB sensor, the RGB information collected by the RGB sensor is more complete than that collected by other image sensors, such as an RGB-IR sensor, and a part of the photosensitive channels of the RGB-IR sensor cannot collect visible light, so that the color details of the image collected by the RGB sensor are more accurate.

It should be noted that the image sensor may include a plurality of photosensitive channels corresponding to a plurality of sensing curves. Fig. 7 is a schematic diagram of a sensing curve of an RGBW sensor provided in an embodiment of the present application, and exemplarily, referring to fig. 7, taking an example that an image sensor is an RGBW sensor, an R curve in fig. 7 represents a sensing curve of the image sensor for light in a red wavelength band, a G curve represents a sensing curve of the image sensor for light in a green wavelength band, a B curve represents a sensing curve of the image sensor for light in a blue wavelength band, and a W curve represents a sensing curve of the image sensor for sensing light in a full wavelength band.

The image sensor generates different image signals under different exposure parameters. For example, the operation modes of the camera may include two operation modes, namely a first operation mode and a second operation mode. In the first working mode, the image sensor generates a plurality of frames of first image signals according to the first exposure parameters, and the first image signals are sequentially sent to the processing chip. And in the second working mode, the image sensor generates a plurality of frames of second image signals according to the second exposure parameters and generates third image signals according to the third exposure parameters, wherein in the second working mode, the image sensor alternately generates the second image signals and the third image signals through multiple exposures and alternately outputs the second image signals and the third image signals to the processing chip. The processing chip respectively performs black level adjustment on the second image signal and the third image signal, and alternately outputs a second adjustment signal and a third adjustment signal.

Fig. 8 is a schematic structural diagram of a camera according to still another embodiment of the present application, and as shown in fig. 8, the camera 80 includes an image sensor 81, a processing chip 82, an exposure parameter control unit 83, and a light supplement device 84, where the exposure parameter control unit 83 is configured to alternately output a second exposure parameter and a third exposure parameter to the image sensor 81 in a second operation mode, and the light supplement device 84 is configured to supplement light when the image sensor 81 is in the second operation mode.

Further, the camera provided in the embodiment of the present application may further include a filter assembly 86, where the filter assembly 86 includes a first filter, a second filter, and a switching component, the first filter allows visible light and part of near-infrared light to pass through, the first filter and the second filter are both connected to the switching component, and the switching component is configured to switch the second filter to the light incident side of the image sensor 81.

After the second filter is switched to the light-entering side of the image sensor 81, the second filter passes light in the visible light band and blocks light in the near-infrared light band, and the image sensor 81 is configured to generate and output a third image signal by exposure.

Before the image sensor 81, a lens 85 is further included, and when the light supplement device 84 performs light supplement, light passing through the lens 85 includes ambient light and light supplement, and the light supplement includes visible light and near-infrared light supplement.

In some embodiments, the light supplement device 84 is specifically configured to perform near-infrared light supplement, and specifically, perform near-infrared light supplement at least during a part of the time period when the image sensor 81 performs exposure according to the third exposure parameter.

In other embodiments, the light supplement device 84 is specifically configured to supplement the visible light, and specifically, the supplement of the visible light is performed at least in a partial time period when the image sensor 81 performs the exposure according to the second exposure parameter.

The image sensor 81 is located on the light exit side of the filter assembly 86. In the second operation mode, the image sensor 81 is configured to generate and output a second image signal and a third image signal through multiple exposures. Wherein the second image signal is an image signal generated according to the second exposure parameter, and the third image signal is an image signal generated according to the third exposure parameter. The light supplement device 84 includes a first light supplement device 841, where the first light supplement device 841 is configured to perform near-infrared light supplement, and the near-infrared light supplement is performed at least in a partial time period when the image sensor 81 performs exposure according to the third exposure parameter. The filtering component 86 includes a first optical filter, and the first optical filter allows light in a visible light band and light in a near-infrared light band to pass therethrough, where an intensity of near-infrared light passing through the first optical filter when the first fill-in device 841 performs near-infrared light fill-in is higher than an intensity of near-infrared light passing through the first optical filter when the first fill-in device 841 does not perform near-infrared light fill-in.

In the embodiment of the present application, a lens 85 may be further included, in this case, the filter assembly 86 may be located between the lens 85 and the image sensor 81, and the image sensor 81 is located on the light emitting side of the filter assembly 86. Alternatively, the lens 85 is located between the filter assembly 86 and the image sensor 81, and the image sensor 81 is located on the light-emitting side of the lens 85. As an example, the first filter may be a filter film, such that the first filter may be attached to a surface of the light-emitting side of the lens 85 when the filter assembly 86 is positioned between the lens 85 and the image sensor 81, or attached to a surface of the light-entering side of the lens 85 when the lens 85 is positioned between the filter assembly 86 and the image sensor 81.

It should be noted that the light supplement device 84 may be located inside the image acquisition unit or outside the image acquisition unit. The light compensator 84 may be a part of the image capturing unit or may be a separate component from the image capturing unit. When the light supplement device 84 is located outside the image capturing unit, the light supplement device 84 may be in communication connection with the image capturing unit, so that it may be ensured that there is a certain relationship between an exposure time sequence of the image sensor 81 in the image capturing unit and a near-infrared light supplement time sequence of the first light supplement device 841 included in the light supplement device 84, for example, near-infrared light supplement is performed at least in a partial exposure time period of the third exposure parameter, and near-infrared light supplement is not performed in an exposure time period of the second exposure parameter.

In addition, first light filling device 841 is the device that can send near-infrared light, for example near-infrared light filling lamp etc. first light filling device 841 can carry out near-infrared light filling with stroboscopic mode, also can carry out near-infrared light filling with other modes of similar stroboscopic, and this application embodiment does not limit this. In some examples, when the first fill-in light device 841 performs near-infrared light filling in a stroboscopic manner, the first fill-in light device 841 may be controlled in a manual manner to perform near-infrared light filling in a stroboscopic manner, or the first fill-in light device 841 may be controlled in a stroboscopic manner by a software program or a specific device, which is not limited in this embodiment of the application. The time period of the first light supplement device 841 for performing near-infrared light supplement may coincide with the exposure time period of the third exposure parameter, or may be greater than the exposure time period of the third exposure parameter or less than the exposure time period of the third exposure parameter, as long as the near-infrared light supplement is performed in the whole exposure time period or part of the exposure time period of the third exposure parameter, and the near-infrared light supplement is not performed in the exposure time period of the second exposure parameter.

It should be noted that the near-infrared supplementary lighting is not performed in the exposure time period of the second exposure parameter, for the global exposure mode, the exposure time period of the second exposure parameter may be a time period between the exposure start time and the exposure end time, and for the rolling shutter exposure mode, the exposure time period of the second exposure parameter may be a time period between the exposure start time of the first row of effective images and the exposure end time of the last row of effective images of the second image signal, but is not limited thereto. For example, the exposure time period of the second exposure parameter may also be an exposure time period corresponding to the target image in the second image signal, the target image is a plurality of lines of effective images corresponding to the target object or the target area in the second image signal, and a time period between the starting exposure time and the ending exposure time of the plurality of lines of effective images may be regarded as the exposure time period of the second exposure parameter.

Another point to be noted is that when the first fill-in light device 841 performs near-infrared light fill-in on an external scene, near-infrared light incident on the surface of an object may be reflected by the object, and then enters the first optical filter 031. In addition, since the ambient light may include visible light and near infrared light in a normal condition, and the near infrared light in the ambient light is also reflected by the object when being incident on the surface of the object, so as to enter the first filter 031. Therefore, the near-infrared light passing through the first optical filter 031 during the near-infrared light compensation may include near-infrared light entering the first optical filter 031 after being reflected by the object when the first light compensation device 841 performs the near-infrared light compensation, and the near-infrared light passing through the first optical filter 031 during the non-near-infrared light compensation may include near-infrared light entering the first optical filter 031 without being reflected by the object when the first light compensation device 841 does not perform the near-infrared light compensation. That is, the near-infrared light passing through the first optical filter 031 during the near-infrared light compensation includes the near-infrared light emitted by the first light compensation device 841 and reflected by the object and the near-infrared light reflected by the object in the ambient light, and the near-infrared light passing through the first optical filter 031 during the non-near-infrared light compensation includes the near-infrared light reflected by the object in the ambient light.

In a possible implementation manner, referring to fig. 6, the light supplement device 84 may further include a second light supplement device 842, and the second light supplement device 842 is used for supplementing visible light. In some embodiments, the second fill-in device 842 may be used for filling in visible light in a normally bright manner; alternatively, the second light supplement device 842 may be configured to perform visible light supplement in a stroboscopic manner, where at least visible light supplement exists in a partial exposure time period of the second exposure parameter, or the second light supplement device 842 may be configured to perform visible light supplement in a stroboscopic manner, where visible light supplement exists in a partial exposure time period of the second exposure parameter.

In some embodiments, the multiple exposure refers to multiple exposures within one frame period, that is, the image sensor 81 performs multiple exposures within one frame period, so as to generate and output at least one frame of the second image signal and at least one frame of the third image signal. For example, the image sensor 81 performs exposure for a plurality of times in each frame period including 25 frame periods within 1 second, thereby generating at least one frame of the second image signal and at least one frame of the third image signal, and the second image signal and the third image signal generated in one frame period are referred to as a set of image signals, so that 25 sets of image signals are generated in 25 frame periods. The second exposure parameter and the third exposure parameter may be exposure parameters of two adjacent exposures within one frame period, or exposure parameters of two nonadjacent exposures in multiple exposures within one frame period, which is not limited in this embodiment of the present application.

Through the above arrangement, under the condition of daytime or better illumination intensity, the camera 80 may be in the first working mode, the image sensor 81 generates the first image signal under the first exposure parameter, and the processing chip 82 performs black level adjustment on the first image signal to obtain the first adjustment signal.

In the case of being in the dark or having poor illumination intensity, the camera 80 may be in the second operating mode, the image sensor 81 generates the second image signal under the second exposure parameter, and generates the third image signal under the third exposure parameter, wherein the light supplement device 84 is used to supplement visible light in a part of the time period under the second exposure parameter, and the light supplement device 84 is used to supplement near-infrared light in a part of the time period under the third exposure parameter, so as to improve the imaging quality under the extremely low illumination. The image sensor 81 alternately generates and outputs a second image signal and a third image signal to the processing chip 82 through multiple exposures in the second operating mode, and the processing chip 82 respectively performs black level adjustment on the second image signal and the third image signal in the second operating mode, and alternately generates and outputs a second adjustment signal and a third adjustment signal.

Fig. 9 is a schematic view of an exposure sequence of an image sensor in a first operating mode according to an embodiment of the present disclosure, and as shown in fig. 9, in the first operating mode, the image sensor generates multiple frames of first image signals by using a first exposure parameter, and then sends the multiple frames of first image signals to a processing chip, and the processing chip sequentially processes the multiple frames of first image signals. When multiple frames of first image signals are processed, for each current frame of first image signals, the processing chip adjusts the black level of the current frame of first image signals according to the black level of the black pixel area corresponding to the current frame of first image signals and the black level information of the black pixel area corresponding to the previous frame of first image signals of the current frame of first image signals.

Fig. 10 is a schematic diagram of an exposure sequence of the image sensor in the second operation mode according to the embodiment of the present application, and fig. 11 is a schematic diagram of an exposure sequence of the image sensor in the second operation mode according to the embodiment of the present application, as shown in fig. 10 and 11, in the second operation mode, the image sensor generates multiple frames of the second image signal by using the second exposure parameter, and generates multiple frames of the third image signal by using the third exposure parameter. In fig. 10, the image sensor alternately performs exposure under the second exposure parameter and the third exposure parameter, and alternately outputs the second image signal and the third image signal. In fig. 11, the image sensor is exposed twice according to the second exposure parameter to output two frames of second image signals, and then exposed according to the third exposure parameter to output one frame of third image signals, and this exposure process is repeated.

The two exposure modes in fig. 10 and 11 are only an example, and the image sensor may be exposed in other combinations. For example, the multiple exposures may include odd and even exposures, and thus, the exposures corresponding to the second and third exposure parameters may include, but are not limited to, the following ways:

in a first possible implementation manner, the exposure corresponding to the second exposure parameter is one exposure of odd-numbered exposures, and the exposure corresponding to the third exposure parameter is one exposure of even-numbered exposures. Thus, the multiple exposures may include exposures corresponding to the second exposure parameter and exposures corresponding to the third exposure parameter arranged in odd-even order. For example, the odd-numbered exposures such as the 1 st exposure, the 3 rd exposure, and the 5 th exposure in the multiple exposures are all exposures corresponding to the second exposure parameter, and the even-numbered exposures such as the 2 nd exposure, the 4 th exposure, and the 6 th exposure are all exposures corresponding to the third exposure parameter.

In a second possible implementation manner, the exposure corresponding to the second exposure parameter is one exposure of even-numbered exposures, and the exposure corresponding to the third exposure parameter is one exposure of odd-numbered exposures, so that the multiple exposures may include exposures corresponding to the second exposure parameter and exposures corresponding to the third exposure parameter which are arranged in an odd-even order. For example, the odd-numbered exposures such as the 1 st exposure, the 3 rd exposure, and the 5 th exposure in the multiple exposures are all exposures corresponding to the third exposure parameter, and the even-numbered exposures such as the 2 nd exposure, the 4 th exposure, and the 6 th exposure are all exposures corresponding to the second exposure parameter.

In a third possible implementation manner, the exposure corresponding to the second exposure parameter is one exposure of the specified odd number of exposures, and the exposure corresponding to the third exposure parameter is one exposure of other exposures except for the specified odd number of exposures, that is, the exposure corresponding to the third exposure parameter may be an odd number of exposures in multiple exposures or an even number of exposures in multiple exposures.

In a fourth possible implementation manner, the exposure corresponding to the second exposure parameter is one of the designated even-numbered exposures, and the exposure corresponding to the third exposure parameter is one of the other exposures except the designated even-numbered exposure, that is, the exposure corresponding to the third exposure parameter may be an odd-numbered exposure in the multiple exposures or an even-numbered exposure in the multiple exposures.

In a fifth possible implementation manner, the exposure corresponding to the second exposure parameter is one exposure in the first exposure sequence, and the exposure corresponding to the third exposure parameter is one exposure in the second exposure sequence.

In a sixth possible implementation manner, the exposure corresponding to the second exposure parameter is one exposure in the second exposure sequence, and the exposure corresponding to the third exposure parameter is one exposure in the first exposure sequence.

The multiple exposure comprises a plurality of exposure sequences, the first exposure sequence and the second exposure sequence are the same exposure sequence or two different exposure sequences in the multiple exposure sequences, each exposure sequence comprises N times of exposure, the N times of exposure comprise 1 time of exposure corresponding to the second exposure parameter and N-1 times of exposure corresponding to the third exposure parameter, or the N times of exposure comprise 1 time of exposure corresponding to the third exposure parameter and N-1 times of exposure corresponding to the second exposure parameter, and N is a positive integer greater than 2.

For example, each exposure sequence includes 3 exposures, and the 3 exposures may include 1 exposure corresponding to the second exposure parameter and 2 exposures corresponding to the third exposure parameter, so that the 1 st exposure of each exposure sequence may be the exposure corresponding to the second exposure parameter, and the 2 nd and 3 rd exposures are the exposures corresponding to the third exposure parameter. That is, each exposure sequence may be represented as: exposure corresponding to the second exposure parameter, exposure corresponding to the third exposure parameter, and exposure corresponding to the third exposure parameter. Alternatively, the 3 exposures may include 1 exposure corresponding to the third exposure parameter and 2 exposures corresponding to the second exposure parameter, so that the 1 st exposure of each exposure sequence may be the exposure corresponding to the third exposure parameter, and the 2 nd and 3 rd exposures are the exposures corresponding to the second exposure parameter. That is, each exposure sequence may be represented as: exposure corresponding to the third exposure parameter, exposure corresponding to the second exposure parameter, and exposure corresponding to the second exposure parameter.

The foregoing only provides six possible implementation manners of the exposure corresponding to the second exposure parameter and the exposure corresponding to the third exposure parameter, and in practical applications, the implementation manners are not limited to the above six possible implementation manners, and this is not limited in this application.

The exposure parameter refers to an exposure parameter adopted when an image is acquired, and when the image sensor is in a second working mode, the second exposure parameter is different from at least one exposure parameter in a third exposure parameter, wherein the at least one exposure parameter is one or more of exposure time, exposure gain and aperture size, and the exposure gain comprises analog gain and/or digital gain. The exposure time refers to a light sensing time of each pixel of the image sensor, for example, if the image sensor is a camera, the exposure time refers to a time interval from the opening of a shutter of the camera to the closing of the shutter. The length of the exposure time influences the size of the light entering quantity, when the light is strong, the exposure time can be set to be short, and when the light is dark, the exposure time can be properly prolonged, and the light entering quantity is improved.

The gain is divided into analog gain and digital gain, the analog gain is used for amplifying an electric signal output by the image sensor, and the digital gain is used for amplifying an obtained digital signal after analog-to-digital conversion is completed. When the gain is adjusted, the brightness of an output image can be adjusted, the image brightness is higher when the gain is larger, but the noise is also larger, so that the particles are increased, and an output picture is blurred. The size of the aperture can also be controlled, with a larger aperture providing a larger amount of light, and a smaller aperture providing a smaller amount of light.

The second exposure parameter and the third exposure parameter are different in terms of at least one exposure parameter, wherein one or more exposure parameters may be the same. The first exposure parameter condition may be the same as or different from the second exposure parameter condition, and the first exposure parameter condition may be the same as or different from the third exposure parameter condition.

After the first image signal, the second image signal, and the third image signal are obtained, the black level adjustment process is required, and the process will be described below with reference to fig. 12 to 15.

When the camera is in the first working mode, the first adjustment mode specifically is:

determining a black level value of a current frame first image signal according to a received black level statistic value of the current frame first image signal and black level value information of a received historical frame first image signal, wherein the historical frame first image signal is a previous frame first image signal of the current frame first image signal;

The black level adjustment process will be described below by taking as an example that the image sensor is an RGBW sensor including four photosensitive channels, i.e., an R photosensitive channel, a G photosensitive channel, a B photosensitive channel, and a W photosensitive channel, and taking as an example the first adjustment manner.

Fig. 12 is a schematic diagram of black level adjustment processing in the first operating mode according to an embodiment of the present application, and as shown in fig. 12, the specific steps include:

s1: and respectively counting the black level of the black pixel area corresponding to each photosensitive channel for the input first image signal of the current frame, and obtaining the black level statistical value of each photosensitive channel of the first image signal of the current frame.

The statistics may be one or more of averaging, weighted averaging, or the like.

S2: and taking out the black level value of each photosensitive channel of the historical frame first image signal from the memory, and storing a preset black level value in the memory if the current frame first image signal is the first frame, wherein the historical frame first image signal is the first image signal of the previous frame of the current frame first image signal, and the current frame first image signal is the first image signal being processed by the processing chip.

S3: and mixing the black level statistics value of each photosensitive channel of the current frame first image signal and the black level value of each photosensitive channel of the historical frame first image signal according to a preset condition to obtain the black level value of each photosensitive channel of the current frame first image signal, and putting the black level values into a memory for the next frame first image signal.

The preset condition may be one or more combinations of weighting coefficients, selection signals, sensor analog gain, digital gain, exposure time, temperature, deviation from the previous frame, and the like, and the mixing process may be weighted averaging or selection.

S4: and calculating the black level offset of each photosensitive channel of the current frame first image signal according to the first target black level value and the black level value of each photosensitive channel of the current frame first image signal.

The first target black level value is a preset parameter, and is not less than 0, and is used to adjust the black level of the whole image signal to the target value, and calculate the black level offset of each photosensitive channel of the current frame first image signal, which may be subtracting the first target black level value from the black level value of each photosensitive channel of the current frame first image signal.

S5: and applying the black level offset of each photosensitive channel of the current frame first image signal to achieve the effect of correcting the black level offset.

Applying the black level offset of each photosensitive channel of the current frame first image signal to the current frame first image signal means adding or subtracting the black level offset to or from the black level value of the current frame first image signal.

The following is an example.

Firstly, for a black pixel area in an input first image signal, respectively calculating an average value of data values of four channels of RGBW (or calculating an average value after performing denoising processing such as median filtering on the black pixel area), to obtain a black level statistic corresponding to each photosensitive channel of the current frame first image signal, where a black level statistic of an R photosensitive channel of the current frame first image signal is hist _ R, a black level statistic of a G photosensitive channel of the current frame first image signal is hist _ G, a black level statistic of a B photosensitive channel of the current frame first image signal is hist _ B, and a black level statistic of a W photosensitive channel of the current frame first image signal is hist _ W.

Then, the black level value of each photosensitive channel of the first image signal of the historical frame is taken out from the memory, wherein the black level value of the R photosensitive channel of the first image signal of the historical frame is blc _ pre _ R, the black level value of the G photosensitive channel of the first image signal of the historical frame is blc _ pre _ G, the black level value of the B photosensitive channel of the first image signal of the historical frame is blc _ pre _ B, the black level value of the W photosensitive channel of the first image signal of the historical frame is blc _ pre _ W, and if the first image signal of the current frame is the first frame, a group of black level values are preset in the memory.

Thirdly, weighting the black level statistics value corresponding to each photosensitive channel of the current frame first image signal and the black level value of each photosensitive channel of the historical frame first image signal according to the preset weight value weight to obtain the black level value of each photosensitive channel of the current frame first image signal, and storing the black level value of each photosensitive channel of the current frame first image signal into the memory.

blc_cur_R＝(1-weight)*blc_pre_R+weight*hist_R，

blc_cur_G＝(1-weight)*blc_pre_G+weight*hist_G，

blc_cur_B＝(1-weight)*blc_pre_W+weight*hist_B，

blc_cur_W＝(1-weight)*blc_pre_W+weight*hist_W，

Where blc _ cur _ R is a black level of the R photosensitive channel of the current frame first image signal, blc _ cur _ G is a black level of the G photosensitive channel of the current frame first image signal, blc _ cur _ B is a black level of the B photosensitive channel of the current frame first image signal, and blc _ cur _ W is a black level of the W photosensitive channel of the current frame first image signal.

Again, the black level shift amount of each photosensitive channel of the current frame first image signal is calculated from the first target black level value dst _ R, dst _ G, dst _ B, dst _ W and the black level value of each photosensitive channel of the current frame first image signal.

blc_offset_cur_R＝blc_cur_R-dst_R，

blc_offset_cur_G＝blc_cur_G-dst_G，

blc_offset_cur_B＝blc_cur_B-dst_B，

blc_offset_cur_W＝blc_cur_W-dst_W，

Wherein, blc _ offset _ cur _ R is the black level offset of the R photo channel of the first image signal of the current frame;

blc _ offset _ cur _ G is the black level offset of the G photo channel of the first image signal of the current frame;

blc _ offset _ cur _ B is the black level offset of the B photo channel of the first image signal of the current frame;

blc _ offset _ cur _ W is the black level offset of the W-sensitive channel of the first image signal of the current frame.

And finally, applying the calculated black level offset of each photosensitive channel of the current frame first image signal to the effective pixel area data value of the current frame first image signal, wherein the specific mode is that the black level offset of the corresponding photosensitive channel is subtracted from each photosensitive channel data value of the effective pixel area, and when the result is less than 0, the black level offset is set to 0, so that a first adjusting signal corresponding to the current frame first image signal is obtained.

If the camera is in the second working mode, the processing chip needs to process the second image signal and the third image signal respectively.

First, a processing chip needs to distinguish a second image signal from a third image signal, fig. 13 is a schematic diagram of distinguishing the second image signal from the third image signal provided in this embodiment of the present application, as shown in fig. 13, an image sensor alternately generates the second image signal and the third image signal according to alternate exposure under a second exposure parameter and a third exposure parameter, then the processing chip logically distinguishes an image sequence generated by the image sensor to obtain a second image sequence and a third image sequence, where the second image sequence includes multiple frames of second image signals, the third image sequence includes multiple frames of third image signals, and then the processing chip respectively processes the second image signal and the third image signal. Wherein, the second adjustment mode specifically is as follows:

The third adjustment mode specifically comprises:

The black level information of the second image signal of the history frame may be a black level value of the second image signal of the history frame, or a black level offset of the second image signal of the history frame, which will be illustrated below.

Fig. 14 is a first schematic diagram illustrating black level adjustment processing in the second operating mode according to the embodiment of the present application, and as shown in fig. 14, processing processes of the processing chip for the second image signal and the third image signal are similar.

For example, for the second image signal, firstly, the black levels of the black pixel regions corresponding to the respective light-sensing channels of the second image signal of the current frame are respectively counted, and the statistical value of the black levels of the respective light-sensing channels of the second image signal of the current frame is obtained.

Then, the black level value of each photosensitive channel of the second image signal of the historical frame is taken out from the memory, and if the second image signal of the current frame is the first frame, the preset black level value is stored in the memory.

Secondly, mixing the black level statistics value of each photosensitive channel of the current frame second image signal and the black level value of each photosensitive channel of the historical frame second image signal according to a preset condition to obtain the black level value of each photosensitive channel of the current frame second image signal, and putting the black level values into a memory for the next frame second image signal.

Then, according to a second target black level value and the black level value of each photosensitive channel of the current frame second image signal, a black level offset of each photosensitive channel of the current frame second image signal is calculated, where the second target black level value is a preset parameter, and is not less than 0, and is used to adjust the black level of the entire image signal to the target value, and calculate the black level offset of each photosensitive channel of the current frame second image signal, and may be obtained by subtracting the second target black level value from the black level value of each photosensitive channel of the current frame second image signal.

And finally, applying the black level offset of each photosensitive channel of the current frame second image signal to achieve the effect of correcting the black level offset.

The processing procedure for the third image signal is similar to that described above, and is not described here again. When the second image signal is processed, the black level value of the black pixel region corresponding to the second image signal of the previous frame of the current frame of the second image signal and the second target black level value are combined, and when the third image signal is processed, the black level value of the black pixel region corresponding to the third image signal of the previous frame of the current frame of the third image signal and the third target black level value are combined.

The black level information of the black pixel region corresponding to the second image signal may be, in addition to the black level value of each photosensitive channel of the second image signal, a black level offset of each photosensitive channel of the second image signal, and correspondingly, the processing process of the processing chip may also be changed correspondingly.

Fig. 15 is a schematic diagram illustrating a second black level adjustment processing in the second operating mode according to the embodiment of the present application, and as shown in fig. 15, for the second image signal, the black level information of the second image signal in the history frame is a black level offset of the second image signal in the history frame, and the processing procedure specifically includes the following steps:

firstly, respectively solving a median value of data values of four channels RGBW for a black pixel area corresponding to each photosensitive channel of an input current frame second image signal to obtain a black level statistic value corresponding to each photosensitive channel of the current frame second image signal, wherein the black level statistic value of an R photosensitive channel of the current frame second image signal is hist _ R, the black level statistic value of a G photosensitive channel of the current frame second image signal is hist _ G, the black level statistic value of a B photosensitive channel of the current frame second image signal is hist _ B, and the black level statistic value of a W photosensitive channel of the current frame second image signal is hist _ W;

then, calculating the black level offset of each photosensitive channel of the current frame second image signal according to the second target black level value dst _ R, dst _ G, dst _ B, dst _ W and the black level statistic value corresponding to each photosensitive channel of the current frame second image signal;

blc_offset_R＝hist_R-dst_R，

blc_offset_G＝hist_G-dst_G，

blc_offset_B＝hist_B-dst_B，

blc_offset_W＝hist_W-dst_W。

secondly, the black level offset blc _ offset _ pre _ R, blc _ offset _ pre _ G, blc _ offset _ pre _ B, blc _ offset _ pre _ W of each photosensitive channel of the second image signal of the history frame is fetched from the memory, and if the second image signal of the current frame is the first frame, a group of black level offset is preset in the memory;

thirdly, calculating the difference between the black level offset of each photosensitive channel of the second image signal of the current frame and the black level offset of each photosensitive channel of the second image signal of the previous frame to obtain the difference offset of each photosensitive channel of the second image signal;

dif_offset_R＝|blc_offset_R-blc_offset_pre_R|,

dif_offset_G＝|blc_offset_G-blc_offset_pre_G|,

dif_offset_B＝|blc_offset_B-blc_offset_pre_B|,

dif_offset_W＝|blc_offset_W-blc_offset_pre_W|,

max_dif_offset＝max(dif_offset_R,dif_offset_G,dif_offset_B,dif_offset_W)

where, | | represents the absolute value, and max represents the maximum value.

Then, calculating the black level offset according to the difference offset of each photosensitive channel of the current frame second image signal and a preset difference offset threshold dif _ thr to obtain the black level offset of each photosensitive channel of the current frame second image signal;

in this step, in addition to the black level offset of each photosensitive channel of the second image signal of the current frame obtained by the method, weighting or selection may be performed according to conditions such as analog gain, digital gain, exposure time, and temperature.

And finally, applying the calculated black level offset of each photosensitive channel of the current frame second image signal to the effective pixel area data value of the current frame second image signal, wherein the specific mode is that the black level offset of the current frame of the corresponding channel is subtracted by each channel data value of the effective pixel area of the current frame second image signal, and when the result is less than 0, the value is set to 0, so that a second adjusting signal is obtained.

The third image signal may also be processed by the above steps, where the previous frame of image signal of the current frame of third image signal is the historical frame of third image signal, when the black level of the third image signal is adjusted, the corresponding second target black level value needs to be replaced by the third target black level value, and the black level of the black pixel region corresponding to each photosensitive channel of the current frame of third image signal and the black level offset of each photosensitive channel of the previous frame of third image signal of the current frame of third image signal are obtained to perform black level adjustment processing on the current frame of third image signal, which is not described herein again.

And after black level adjustment is carried out according to the steps, obtaining a first adjustment signal in a first working mode, and obtaining a second adjustment signal and a third adjustment signal in a second working mode. Further, the processing chip may include a black level adjustment unit and a first image processing unit, where the black level adjustment unit is configured to perform black level adjustment on the image signal to obtain a corresponding adjustment signal, and the first image processing unit is configured to obtain a corresponding processing signal according to the corresponding adjustment signal.

Fig. 16 is a first schematic processing diagram of the camera provided in the embodiment of the present application, as shown in fig. 16, when the camera is in the first operating mode, the image sensor collects and generates a first image signal, and sends the first image signal to the black level adjustment unit, the black level adjustment unit processes the first image signal to generate a first adjustment signal, and sends the first adjustment signal to the first image processing unit, the first image processing unit processes the first adjustment signal to obtain a first processed image, and the first image processing unit processes the first adjustment signal, which includes but is not limited to one or more of noise reduction, gain, dead pixel correction, black level correction, color correction, luminance mapping, and sharpening.

When the camera is in the second working mode, the image sensor collects and generates a second image signal and a third image signal, and sends the second image signal and the third image signal to the black level adjusting unit, the black level adjusting unit processes the second image signal to generate a second adjusting signal, processes the third image signal to generate a third adjusting signal, and sends the second adjusting signal and the third adjusting signal to the first image processing unit, the first image processing unit processes the second adjusting signal to obtain a second processed image, and processes the third adjusting signal to obtain a third processed image, and the first image processing unit processes the second adjusting signal and the third adjusting signal, wherein the first image processing unit processes the second adjusting signal and the third adjusting signal, and the processing of the second adjusting signal and the third adjusting signal includes but is not limited to one or more of noise reduction, gain, dead pixel correction, black level correction, color correction, brightness mapping, sharpening.

The image processing unit is further configured to:

and in the second working mode, carrying out fusion processing on the second adjustment signal and the third adjustment signal to obtain and output a processed image.

Fig. 17 is a second schematic processing diagram of the camera provided in the embodiment of the present application, and as shown in fig. 17, when the camera is in the second working mode, the first image processing unit is further configured to perform fusion processing on the second processed image and the third processed image to obtain a processed image.

Fig. 18 is a third schematic processing diagram of the camera provided in the embodiment of the present application, as shown in fig. 18, further, the processing chip may further include a second image processing unit, when the camera is in the first operating mode, the second image processing unit is configured to process the first image signal to obtain a first processed signal, and send the first processed signal to the black level adjustment unit, the black level adjustment unit is configured to perform black level adjustment processing on the first processed signal to obtain a first adjusted signal, and send the first adjusted signal to the first image processing unit, and the first image processing unit is configured to process the first adjusted signal to obtain a first processed image.

When the camera is in the second working mode, the second image processing unit is used for processing the second image signal to obtain a second processing signal, processing the third image signal to obtain a third processing signal, and sending the second processing signal and the third processing signal to the black level adjusting unit. The black level adjusting unit is used for performing black level adjustment processing on the second processing signal to obtain a second adjusting signal, performing black level adjustment processing on the third processing signal to obtain a third adjusting signal, and sending the second adjusting signal and the third adjusting signal to the first image processing unit. The first image processing unit is used for processing the second adjustment signal to obtain a second processed image and processing the third adjustment signal to obtain a third processed image. The processing of the image signal by the second image processing unit includes, but is not limited to, one or more of noise reduction, gain, and the like.

Fig. 19 is a fourth schematic processing diagram of the camera provided in the embodiment of the present application, as shown in fig. 19, when the camera is in the second operating mode, the second image processing unit processes the second image signal to obtain a second processed signal, processes the third image signal to obtain a third processed signal, and then obtains a second adjustment signal and a third adjustment signal through the black level adjustment unit, respectively, and the first image processing unit is further configured to perform fusion processing on the second adjustment signal and the third adjustment signal to obtain a processed image.

The camera provided by the embodiment of the application comprises a single image sensor and a processing chip, wherein the output end of the image sensor is connected with the processing chip, the camera comprises two working modes, in the first working mode, the image sensor generates and outputs a first image signal to the processing chip according to a first exposure parameter, and the processing chip performs black level adjustment on the first image signal according to a first adjustment mode when receiving the first image signal to obtain a first adjustment signal; in a second working mode, the image sensor generates and outputs a second image signal to the processing chip according to the second exposure parameter, and generates and outputs a third image signal to the processing chip according to a third exposure parameter; when the processing chip receives the second image signal, black level adjustment is carried out on the second image signal according to a second adjustment mode to obtain a second adjustment signal; and when the third image signal is received, carrying out black level adjustment on the third image signal according to a third adjustment mode to obtain a third adjustment signal. The camera provided by the embodiment of the application can correspondingly adjust the working mode of the camera under different conditions, when the camera is in the first working mode, the black level of the first image signal of the current frame can be adjusted according to the black level information of the first image signal of the historical frame, when the camera is in the second working mode, the second image signal and the third image signal are firstly distinguished, and then the black level adjustment processing is respectively carried out on the second image signal and the third image signal according to different adjustment modes, wherein the black level of the second image signal of the current frame is adjusted according to the second image signal of the historical frame, the black level of the third image signal of the current frame is adjusted according to the third image signal of the historical frame, when each frame of image signal is processed, the black level information of the image signal of the previous frame is fully utilized, the multi-frame image signals are mutually associated, so that the black level adjustment of the image signals between frames has correlation, the problem that image pictures are easy to jump seriously when the black level adjustment is carried out on the multi-frame image signals is avoided, and the instability when the black level adjustment is carried out on the multi-frame image signals is solved.

Fig. 20 is a schematic flowchart of a black level adjustment method provided in an embodiment of the present application, where the method is applied to a camera, where the camera includes a single image sensor and a processing chip, and an output terminal of the image sensor is connected to the processing chip, as shown in fig. 20, the method includes:

s201, generating a first image signal according to a first exposure parameter in a first working mode through the image sensor, and performing black level adjustment on the first image signal according to a first adjustment mode through the processing chip to obtain a first adjustment signal;

s202, alternately generating a second image signal and a third image signal through multiple exposures in a second working mode through the image sensor, wherein the second image signal is generated according to a second exposure parameter, and the third image signal is generated according to a third exposure parameter; performing black level adjustment on the second image signal through the processing chip according to a second adjustment mode to obtain a second adjustment signal; performing black level adjustment on the third image signal through the processing chip according to a third adjustment mode to obtain a third adjustment signal; and alternately outputting the second adjusting signal and the third adjusting signal through the processing chip.

and alternately outputting the second exposure parameter and the third exposure parameter to the image sensor through the exposure parameter control unit in the second working mode.

In one possible implementation, the method further includes:

Fig. 21 is a schematic structural diagram of a black level adjustment apparatus according to an embodiment of the present application, as shown in fig. 21, including a first processing module 211 and a second processing module 212, where:

the first processing module 211 is configured to receive a first image signal when the image sensor is in a first working mode, and perform black level adjustment on the first image signal according to a first adjustment mode to obtain a first adjustment signal;

the second processing module 212 is configured to alternately receive a second image signal and a third image signal when the image sensor is in a second operating mode, perform black level adjustment on the second image signal according to a second adjustment mode when the second image signal is received to obtain a second adjustment signal, perform black level adjustment on the third adjustment signal according to a third adjustment mode when the third image signal is received to obtain a third adjustment signal, and alternately output the second adjustment signal and the third adjustment signal.

In a possible implementation manner, the first processing module 211 is specifically configured to:

In a possible implementation manner, the second processing module 212 is specifically configured to:

The apparatus provided in the embodiment of the present application may be configured to implement the technical solution of the method embodiment, and the implementation principle and the technical effect are similar, which are not described herein again.

Finally, it should be noted that: the above embodiments are only used for illustrating the technical solutions of the present application, and not for limiting the same; while the invention has been described in detail and with reference to the foregoing embodiments, it will be understood by those skilled in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some or all of the technical features may be equivalently replaced; and the modifications or the substitutions do not make the essence of the corresponding technical solutions depart from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A camera comprising a single image sensor and a processing chip, an output of the image sensor being connected to the processing chip, wherein:

the processing chip is further used for carrying out black level adjustment on the second image signal according to a second adjustment mode when the second image signal is received, so as to obtain a second adjustment signal; when the third image signal is received, carrying out black level adjustment on the third image signal according to a third adjustment mode to obtain a third adjustment signal; alternately outputting the second adjustment signal and the third adjustment signal.

2. The camera according to claim 1, further comprising an exposure parameter control unit, wherein:

3. The camera of claim 1, wherein the second exposure parameter is different from at least one of the third exposure parameters, the at least one exposure parameter being one or more of exposure time, exposure gain, aperture size, the exposure gain comprising analog gain, and/or digital gain.

4. The camera according to any one of claims 1 to 3, wherein the first adjustment manner is specifically:

5. The camera according to any one of claims 1 to 3, wherein the second adjustment manner is specifically:

6. The camera according to any one of claims 1 to 3, wherein the third adjustment manner is specifically:

7. The camera of claim 1, wherein the processing chip is further configured to:

8. A black level adjustment method applied to a camera, wherein the camera comprises a single image sensor and a processing chip, and an output end of the image sensor is connected with the processing chip, the method comprises the following steps:

9. The method of claim 8, wherein the camera further comprises an exposure parameter control unit, the method further comprising:

10. The method according to claim 8 or 9, wherein performing black level adjustment on the first image signal according to a first adjustment manner by the processing chip to obtain a first adjustment signal comprises:

11. The method according to claim 8 or 9, wherein performing black level adjustment on the second image signal according to a second adjustment manner by the processing chip to obtain a second adjustment signal comprises:

12. The method according to claim 8 or 9, wherein performing black level adjustment on the third image signal according to a third adjustment manner by the processing chip to obtain a third adjustment signal comprises:

13. The method of claim 8, further comprising:

14. A black level adjusting apparatus, comprising: