WO2021016900A1

WO2021016900A1 - Image sensor and image photosensing method

Info

Publication number: WO2021016900A1
Application number: PCT/CN2019/098481
Authority: WO
Inventors: 王晗; 杨红明; 涂娇姣
Original assignee: 华为技术有限公司
Priority date: 2019-07-31
Filing date: 2019-07-31
Publication date: 2021-02-04
Also published as: CN110574367A

Abstract

Disclosed in the embodiments of the present application are an image sensor and an image photosensing method. The sensor comprises: a filtering layer, a plurality of micro lenses, and a pixel array; the pixel array comprises red pixels, green pixels, blue pixels, and infrared pixels, wherein, the micro lenses corresponding to the red pixels, the green pixels, and the blue pixels are coated with an infrared cut-off filtering layer, such that infrared light cannot enter the red pixels, the green pixels, or the blue pixels, and the red pixels, the green pixels and the blue pixels can respectively only sense red light, green light and blue light, filtering out the IR component of the visible light photosensing result, allowing the R, G, B, and IR components to be sensed independently, and thereby significantly improving the photosensing effect of the sensor.

Description

Image sensor and image light-sensing method

Technical field

This application relates to the field of image processing, and in particular to an image sensor and a method for image light-sensing.

Background technique

The color filter array (CFA) of the traditional Bayer red, green and blue sensor (Red Green Blue Sensor, RGB Sensor) contains three components of R, G, and B, which is different from the traditional Bayer RGB sensor. Infrared, RGBIR) sensor CFA contains four components R, G, B and IR, as shown in Figure 1 is the photosensitive characteristic curve of each pixel in the RGBIR sensor, where only the IR filter layer can only transmit infrared light, R The filter layer transmits both red and infrared light, the G filter layer transmits both green and infrared light, and the B filter layer transmits both blue and infrared light. Therefore, even if the filter layer is used, the visible light The IR component still cannot be completely peeled off. The photosensitive results obtained by the R pixel, G pixel and B pixel of the photosensitive device all have a certain degree of IR component signal. Due to the influence of the IR component, the color information of the image signal obtained by the sensor is not quasi. Under certain lighting conditions, the photosensitivity of existing RGBIR sensors is not satisfactory.

Now, more and more application scenarios need to be based on both visible light signals and infrared light signals, such as live detection, night video surveillance, and color-black and white dynamic fusion technology. These application scenarios have improved the color accuracy and dynamic range of the image. With high requirements, how to improve the sensitivity of the sensor needs to be solved urgently.

Summary of the invention

The embodiment of the present application provides an image sensor and an image light-sensing method, so that the R, G, B, and IR components can be light-sensitized independently, which greatly improves the light-sensing effect.

The first aspect of the present application provides an image sensor, the sensor includes: a filter layer, a plurality of micro lenses and a pixel array, the pixel array includes red pixels, green pixels, blue pixels and infrared light pixels, each pixel corresponds to A micro lens; the filter layer includes an infrared light cut filter layer, the red pixel, the green pixel and the micro lens corresponding to the blue pixel are respectively coated with the infrared light cut filter layer, the infrared light cut filter The layer is used to cut off an optical signal with a wavelength greater than a first preset wavelength, and the optical signal with a wavelength greater than the first preset wavelength includes infrared light.

In the image sensor provided by the embodiments of the present application, an infrared light cut-off filter layer is coated on the micro lenses corresponding to the red pixels, green pixels, and blue pixels, which cuts off IR light from entering the visible light pixels, and removes the light-sensitive results of the visible light pixels. The IR component signal, the color of the photosensitive result is more accurate, and the photosensitive effect of the sensor is improved. Further, since the embodiment of the present application coats the infrared light cut filter layer on the micro lens based on the coating technology, on the one hand, there is no need to add a complicated mechanical structure; on the other hand, the pixel itself under the micro lens is not changed. The relatively simple and stable internal structure of the pixel is conducive to controlling the main light path angle of incidence (Chief Ray Angle, CRA) and other issues that affect imaging, and it improves the sensitivity of the sensor while maintaining the stability of the pixel structure.

In a possible implementation manner, the first preset wavelength is 650nm. In this case, the infrared cut filter cuts off all light with a wavelength greater than the visible light range, ensuring that infrared light in all wavelength ranges cannot enter Red pixels, green pixels, and blue pixels.

In a possible embodiment, the filter layer further includes a red filter layer, a green filter layer, a blue filter layer and an infrared filter layer; the infrared light pixel corresponding to the micro lens is coated with the infrared light Optical filter layer, the infrared filter layer can pass infrared light in a specific wavelength range; the red filter layer is also coated on the micro lens corresponding to the red pixel, and the micro lens corresponding to the green pixel is also coated Green filter layer, the blue filter layer is also coated on the micro lens corresponding to the blue pixel; the red filter layer can only pass red light and infrared light in the first wavelength range, and the green filter layer only Can pass green light and infrared light in the second wavelength range, the blue filter layer can only pass blue light and infrared light in the third wavelength range, infrared light in the first wavelength range, the second wavelength range The wavelengths of the infrared light within and the infrared light within the third wavelength range are both greater than the first preset wavelength.

In the image sensor provided by the embodiment of the present application, a red filter layer and an infrared light cut filter layer are coated on the red pixels to filter out the IR component in the light-sensing result of the red pixels, so that the red pixels can only light-receive R light. , The green pixel is coated with a green filter layer and an infrared cut-off filter layer, and the blue pixel is coated with a blue filter layer and an infrared cut-off filter layer to filter out the green pixels and blue pixels. The IR component in the result allows the green pixel to receive only G light, and the blue pixel to receive only B light. The infrared light filter layer is coated on the infrared light pixels, so that the IR pixels can only receive the IR light, which greatly improves the color accuracy of the light-sensing results obtained by the RGBIR sensor.

It should be understood that the red filter layer is above or below the infrared light cut filter layer; the green filter layer is above or below the infrared light cut filter layer; the blue filter layer is above or below the infrared light cut filter layer. Above or below the filter layer. The embodiment of the present application does not limit the coating sequence of the infrared light cut filter layer, the red filter layer, the green filter layer, and the blue filter layer on the micro lens.

In the image sensor provided by the embodiment of the application, the micro lens of the red pixel is coated with a red filter layer and an infrared light cut filter layer; the micro lens of the green pixel is coated with a green filter layer and an infrared light cut filter layer; blue The micro lens of the color pixel is coated with a blue filter layer and an infrared light cut filter layer; the micro lens of an infrared light pixel is coated with an infrared filter layer, and the infrared light cut filter layer and the red filter layer are not limited. The position relationship between the green filter layer and the blue filter layer coated on the micro lens, the red filter layer, the green filter layer and the blue filter layer can be respectively coated on the infrared cut filter layer; or The infrared cut filter layer can also be coated on the red filter layer, the green filter layer and the blue filter layer respectively, as long as the light passes through the infrared cut filter layer and any one before reaching the micro lens The filter layer for visible light components is sufficient.

And in an optional case, the infrared cut filter layer is coated on the micro lens, and the red filter layer, the green filter layer, and the blue filter layer are coated on the inner side of the micro lens or are respectively made on the red Pixels, green pixels, and blue pixels; in an optional case, the red filter layer, the green filter layer, and the blue filter layer are coated on the micro lens, and the infrared cut filter layer is coated on the micro lens. Inside the lens or inside the red pixels, green pixels, and blue pixels.

In a possible implementation, the sensor further includes a filter for filtering ultraviolet light and infrared light with a wavelength greater than a second preset wavelength, and the second preset wavelength is greater than the first preset wavelength. Set the wavelength and any wavelength within the specific wavelength range; light sequentially passes through the filter, the filter layer and the micro lens to reach the pixel array.

In the image sensor provided by the embodiment of the application, the filter can filter out the far-infrared light with a longer wavelength and the ultraviolet light with a shorter wavelength in natural light, so as to prevent the far-infrared light and ultraviolet light from affecting the photosensitive characteristics of the photosensitive device.

In a possible embodiment, the sensor further includes a charge readout module, and each pixel in the pixel array includes a photosensitive device; the photosensitive device is used to convert light into electric charge; the charge readout module uses the photosensitive device The accumulated electric charge is output, and the photosensitive result is obtained.

In a possible implementation manner, the sensor further includes: a logic control circuit for independently controlling the exposure time of the visible light pixel and the infrared light pixel, the visible light pixel includes the red pixel, the green pixel, and the blue pixel .

The exposure of the RGB visible light component and the IR component of the existing RGBIR sensor is uniformly controlled, and the problem of exposure imbalance is prone to occur when the lighting conditions are not ideal, so the dynamic range of the existing RGBIR sensor is poor. In the image sensor provided by the embodiments of the present application, the exposure time of visible light pixels and IR pixels is independently controlled. For example, when infrared light is too strong and visible light is too weak, the exposure time of visible light can be increased and the exposure time of infrared light can be reduced, so that The exposure time of visible light and infrared light tends to be balanced to avoid exposure imbalance when infrared light is the dominant component or visible light is the dominant component, and the dynamic range of the sensor's sensitivity is improved to meet the user's indicators of clarity and signal-to-noise ratio Requirements.

In a possible implementation, the logic control circuit includes a first control line and a second control line, the visible light pixels in the pixel array are coupled to the first control line, and the infrared light pixels in the pixel array are coupled to The second control line; the logic control circuit is specifically configured to: control the exposure start time of the visible light pixel based on the first control line; control the exposure start time of the infrared light pixel based on the second control line.

In a possible implementation manner, the logic control circuit is further configured to control the exposure time of the visible light pixel and the infrared light pixel to meet a preset ratio based on the first control line and the second control line.

Exemplarily, the first control line outputs a first control signal, and the second control line outputs a second control signal. When the first effective transition edge of the first control signal arrives, the visible light pixel starts to be exposed. When the second effective transition edge of the second control signal arrives, the infrared light pixel starts to be exposed; by setting the arrival time of the first effective transition edge and the second effective transition edge, the visible light pixel and the infrared light pixel The exposure time meets the preset ratio.

The image sensor provided in the embodiments of the present application can set the arrival time of the effective transition edge of the respective control signals of the visible light signal and the infrared light signal, so that the exposure time of the visible light signal and the infrared light signal meets the preset ratio, for example, when the visible light signal When the ratio of the exposure time to the infrared light signal is 2:1, the definition of the exposure result is better, and the signal-to-noise ratio is higher, so that the control signal of the visible light signal jumps first, and the infrared light signal jumps after it. The time difference between the jump time points of the signal makes the exposure time of the visible light signal and the exposure time of the infrared light signal meet the preset ratio. Therefore, by accurately setting the exposure time ratio of the visible light signal and the infrared light signal, the photosensitive effect of the sensor can be controlled more finely. Exemplarily, the valid transition edge may be a falling edge of a high-level signal, a rising edge of a low-level signal, a rising edge of a high-level signal, a falling edge of a low-level signal, and so on.

In a possible implementation manner, the sensor further includes: a logic control circuit for independently controlling the exposure time of the red pixel, the green pixel, the blue pixel, and the infrared light pixel.

In the image sensor provided by the embodiments of the present application, the exposure time of the four components of R, G, B and IR are independently controlled. When certain scenes have high requirements on the photosensitivity results of the R and G components, it is desired to reduce the photosensitivity results of B and IR. , You can flexibly control the exposure time of the four components to enhance the sensitization effect of the R and G components, and weaken the sensitization effect of the B and IR components, so that the final sensitization result is more in line with the needs of the scene. The dynamic range of the sensor's sensitivity is further improved, and the sensitivity results with sharpness or signal-to-noise ratio more in line with customer needs are provided.

In a possible implementation manner, the logic control circuit includes: a first control line, a second control line, a third control line, and a fourth control line, and the red pixels in the pixel array are coupled to the first control line, The green pixels in the pixel array are coupled to the second control line, the blue pixels in the pixel array are coupled to the third control line, and the infrared light pixels in the pixel array are coupled to the fourth control line; the logic control The circuit is specifically configured to: control the exposure start time of the red pixel based on the first control line; control the exposure start time of the green pixel based on the second control line; control the exposure of the blue pixel based on the third control line Start time; control the exposure start time of the infrared light pixel based on the fourth control line.

In a possible implementation manner, the logic control circuit is further configured to: control the red pixel, the green pixel, and the green pixel based on the first control line, the second control line, the third control line, and the fourth control line. The exposure time of the blue pixel and the infrared light pixel meets a preset ratio.

In the image sensor provided by the embodiment of the present application, the exposure time of the four components of R, G, B, and IR may be preset to meet the preset ratio, so as to achieve fine control of the sensor's light-sensing effect.

Exemplarily, the first control line outputs a first control signal, the second control line outputs a second control signal, the third control line outputs a third control signal, and the fourth control line outputs a fourth control signal; when the When the first effective transition edge of the first control signal arrives, the red pixel starts to be exposed, and when the second effective transition edge of the second control signal comes, the green pixel starts to expose. When the three effective transition edges arrive, the green pixel starts to be exposed, and when the fourth effective transition edge of the fourth control signal arrives, the infrared light pixel starts to expose. By setting the arrival time of the first effective transition edge, the second effective transition edge, the third effective transition edge and the fourth effective transition edge, the exposure time of the four components of R, G, B and IR meets the preset proportion.

In a possible implementation, the sensor further includes: a row coordinate control line, a column coordinate control line, and an exposure start control line; each pixel in the pixel array is coupled to its own row coordinate control line and column coordinate control line, The exposure start control line includes multiple branches, and each branch corresponds to a pixel; when the control signals output by the row coordinate control line and the column coordinate control line of the target pixel are both effective levels, the target pixel corresponds to the The branch of the exposure start control line outputs a control signal to control the exposure start time of the target pixel, and the target pixel is any pixel in the pixel array.

In the image sensor provided by the embodiments of the present application, the exposure time of each pixel can be independently controlled. In some scenes where pixels in the target area need to be enhanced, only the exposure time of the pixels in the target area can be increased, which further improves the flexibility of the sensor's exposure. It also further satisfies the needs of users for photosensitization results.

In a possible implementation manner, the sensor further includes: an exposure end control signal for uniformly controlling the exposure end time of all pixels in the pixel array.

In a possible implementation manner, the logic control circuit includes a first control variable x and a second control variable y, and when x and y meet the coordinate conditions of the visible light pixel, the reset signal of the logic control circuit is output to the first control The line is used as the first control signal; when x and y meet the coordinate conditions of the IR pixel, the reset signal of the logic control circuit is output to the second control line as the second control signal.

In a possible implementation manner, the logic control circuit includes a first control variable x and a second control variable y. When x and y meet the coordinate conditions of the R pixel, the reset signal of the logic control circuit is output to the first control Line as the first control signal; when x and y meet the coordinate condition of the G pixel, the reset signal of the logic control circuit is output to the second control line as the second control signal; when x and y meet the coordinate condition of the B pixel, The reset signal of the logic control circuit is output to the third control line as the third control signal; when x and y meet the coordinate conditions of the IR pixel, the reset signal of the logic control circuit is output to the fourth control line as the fourth control signal.

The second aspect of the present application provides a method of image light-sensing, which is applied to an image sensor, the sensor includes: an infrared light cut filter layer, a plurality of micro lenses and a pixel array, the pixel array includes red pixels, green pixels, Blue pixels and infrared light pixels, each pixel in the pixel array corresponds to a micro lens; the infrared light cut filter layer is respectively coated on the micro lens corresponding to the red pixel, the green pixel and the blue pixel, The method includes: light passes through the infrared light cut filter layer and the micro lens to reach the red pixel, the green pixel, and the blue pixel; wherein the infrared light cut filter layer is used to cut off infrared light so that infrared light cannot Enter the red pixel, the green pixel, and the blue pixel.

In a possible implementation, the sensor further includes a red filter layer, a green filter layer, a blue filter layer, and an infrared filter layer; the red filter layer is coated on the micro lens corresponding to the red pixel , The green filter layer is coated on the micro lens corresponding to the green pixel, the blue filter layer is coated on the micro lens corresponding to the blue pixel, and the infrared filter layer is coated on the infrared light On the micro lens corresponding to the pixel, the method specifically includes: the light sequentially passes through the infrared light filter layer and the micro lens to reach the infrared light pixel; the light sequentially passes through the infrared light cut filter layer, the red filter layer, and The micro lens reaches the red pixel; the light sequentially passes through the infrared light cut filter layer, the green filter layer and the micro lens to reach the green pixel; the light light sequentially passes through the infrared light cut filter layer and the blue filter The light layer and the micro lens reach the blue pixel; or, the light sequentially passes through the red filter layer, the infrared light cut filter layer and the micro lens to reach the red pixel; the light sequentially passes through the green filter layer, The infrared light cut filter layer and the micro lens reach the green pixel; the light sequentially passes through the blue filter layer, the infrared light cut filter layer and the micro lens to reach the blue pixel; wherein, the infrared light filter The optical layer is used to pass only infrared light in a specific wavelength range, the red filter layer is used to pass only red light and infrared light in the first wavelength range, the green filter layer is used to pass only green light and the second wavelength Infrared light within the range, the blue filter layer is used to pass only blue light and infrared light in the third wavelength range; the infrared light cut off by the infrared cut filter layer includes: infrared light in the first wavelength range , Infrared light in the second wavelength range and infrared light in the third wavelength range.

The embodiment of the application does not limit the positional relationship of the infrared cut filter layer and the red filter layer, the green filter layer and the blue filter layer coated on the micro lens, the red filter layer, the green filter layer and the blue The filter layer can be respectively coated on the infrared light cut filter layer; or the infrared light cut filter layer can also be respectively coated on the red filter layer, the green filter layer and the blue filter layer, as long as The light passes through the infrared cut filter layer and any visible light component filter layer before reaching the micro lens.

In a possible implementation manner, the sensor further includes a filter, the light being the original light in nature after passing through the filter, and the filter is used to filter ultraviolet light and far-infrared light. The wavelength of the infrared light is greater than the wavelength of the infrared light in the specific wavelength range allowed by the infrared light filter layer, and the method further includes: the original light in nature passes through the filter to obtain the light.

In a possible implementation manner, the sensor further includes a charge readout module, and each pixel in the pixel array includes a photosensitive device, and the method further includes: the photosensitive device converts light into electric charge; The accumulated electric charge is output, and the photosensitive result is obtained.

In a possible implementation manner, the method further includes: controlling the exposure start time of a visible light pixel based on the first control line, the visible light pixel including the red pixel, the green pixel, and the blue pixel; and based on the second control line The line controls the exposure start time of the infrared light pixel.

In a possible implementation manner, the method further includes: controlling the exposure time of the visible light pixel and the infrared light pixel to meet a preset ratio based on the first control line and the second control line.

In a possible implementation manner, the method further includes: controlling the exposure start time of the red pixel based on the first control line; controlling the exposure start time of the green pixel based on the second control line; controlling the exposure start time of the green pixel based on the third control line The exposure start time of the blue pixel; the exposure start time of the infrared light pixel is controlled based on the fourth control line.

In a possible implementation manner, the method further includes: controlling the red pixel, the green pixel, and the blue pixel based on the first control line, the second control line, the third control line, and the fourth control line. The exposure time of the pixel and the infrared light pixel meets a preset ratio.

In a possible implementation manner, each pixel in the sensor is coupled to a respective row coordinate control line and a column coordinate control line, and each pixel corresponds to a branch of the exposure start control line, the method further includes: When the control signals output by the row coordinate control line and the column coordinate control line of the target pixel are both effective levels, the branch of the exposure start control line corresponding to the target pixel outputs a control signal, and the target is controlled based on the control signal The exposure start time of the pixel, and the target pixel is any pixel in the pixel array.

A third aspect of the present application provides an independent exposure device, the device includes: at least two control units, each of the at least two control units is used to correspondingly control one type of the pixel array of the sensor The exposure start time of the pixel, and the pixel array of the sensor includes at least two types of pixels.

Existing sensors containing multiple types of pixels, the exposure time of different types of pixels is uniformly controlled, exposure is prone to imbalance when the lighting conditions are not ideal, exposure control flexibility is poor, and the dynamic range of sensor exposure is relatively poor. The device provided in the present application can independently control the exposure time of different types of pixels in the sensor, which improves the dynamic range and signal-to-noise ratio of the sensor. Exemplarily, the device is a control unit or logic control circuit independent of the sensor, and the corresponding product form may be a processor or a chip product containing a processor.

In a possible implementation manner, the device further includes: the pixel array.

The device may be a sensor including a control unit.

In a possible implementation, the sensor is an RGBIR sensor, the at least two types of pixels include: visible light pixels and IR pixels, the visible light pixels include: R pixels, G pixels, and B pixels; or, at least Two types of pixels include: R pixels, B pixels, G pixels, and IR pixels. The at least two control units include: a first control unit, a second control unit, a third control unit, and a fourth control unit; the first The control unit is used to control the exposure start time of the R pixel; the second control unit is used to control the exposure start time of the G pixel; the third control unit is used to control the exposure start time of the B pixel; The four control unit is used to control the exposure start time of the IR pixel.

In a possible implementation manner, the sensor is an RGBW sensor, the at least two types of pixels include: visible light pixels and W pixels, the visible light pixels include: R pixels, G pixels, and B pixels, the at least two control units It includes: a first control unit and a second control unit; the first control unit is used to control the exposure start time of the visible light pixel; the second control unit is used to control the exposure start time of the W pixel; or the at least two One type of pixel includes: R pixel, B pixel, G pixel, and W pixel. The at least two control units include: a first control unit, a second control unit, a third control unit, and a fourth control unit; the first control unit The unit is used to control the exposure start time of the R pixel; the second control unit is used to control the exposure start time of the G pixel; the third control unit is used to control the exposure start time of the B pixel; the fourth The control unit is used to control the exposure start time of the W pixel.

In a possible implementation manner, the sensor is an RCCB sensor, the at least two types of pixels include visible light pixels and C pixels, the visible light pixels include R pixels and B pixels, and the at least two control units include: A control unit and a second control unit; the first control unit is used to control the exposure start time of the visible light pixel; the second control unit is used to control the exposure start time of the C pixel; the at least two types of pixels Including: R pixel, B pixel and C pixel, the at least two control units include: a first control unit, a second control unit and a third control unit; the first control unit is used to control the exposure start time of the R pixel The second control unit is used to control the exposure start time of the B pixel; the third control unit is used to control the exposure start time of the C pixel.

In a possible implementation, based on the at least two control units controlling the exposure time of the at least two types of pixels to meet a preset ratio.

In a possible implementation manner, the device further includes: an exposure end control unit, configured to uniformly control the exposure end time of all pixels in the pixel array.

The fourth aspect of the present application provides an independent exposure method, which is applied to a sensor including at least two types of pixels, the at least two types of pixels include a first type of pixel and a second type of pixel, The method includes: controlling the exposure start time of the first type of pixel based on a first control unit; and controlling the exposure start time of the second type of pixel based on the second control unit.

In a possible implementation manner, the method further includes: controlling the exposure time of each of the at least two types of pixels to meet a preset ratio.

Exemplarily, based on the first control unit and the second control unit controlling the exposure time of the visible light pixels and the IR pixels to meet the preset ratio; or based on the first control unit, the second control unit, the third control unit, and the fourth control unit control The exposure time of R, G, B and IR pixels meets the preset ratio. Or, based on the first control unit and the second control unit to control the exposure time of the visible light pixel and the W pixel to meet the preset ratio; or, based on the first control unit, the second control unit, the third control unit, and the fourth control unit to control R , G, B and W pixels’ exposure time meets a preset ratio; or, based on the first control unit and the second control unit controlling the exposure time of the visible light pixel and the C pixel to meet the preset ratio; or, based on the first control unit, the The second control unit and the third control unit control the exposure time of the R, B and C pixels to meet the preset ratio.

In a possible implementation, the sensor is an RGBIR sensor, the first type of pixels are visible light pixels, the second type of pixels are IR pixels, and the visible light pixels include R pixels, G pixels, and B pixels; Alternatively, the sensor is an RGBW sensor, the first type of pixels are visible light pixels, the second type of pixels are W pixels, and the visible light pixels include R pixels, G pixels, and B pixels; the sensor is an RCCB sensor, and the The first type of pixels are visible light pixels, the second type of pixels are C pixels, and the visible light pixels include R pixels and B pixels.

In a possible implementation manner, the at least two types of pixels further include: a third type of pixels; the method further includes: controlling the exposure start time of the third type of pixels based on the third control unit.

In a possible implementation manner, the sensor is an RCCB sensor, the first type of pixels are R pixels, the second type of pixels are B pixels, and the third type of pixels are C pixels; this method Specifically, it includes: controlling the exposure start time of the R pixel based on the first control unit; controlling the exposure start time of the B pixel based on the second control unit; controlling the exposure start time of the C pixel based on the third control unit .

In a possible implementation manner, the at least two types of pixels further include: a third type of pixels and a fourth type of pixels, and the method further includes: controlling the third type of pixels based on a third control unit The exposure start time of the pixel; the fourth control unit controls the exposure start time of the fourth type of pixel.

In a possible implementation manner, the sensor is an RGBIR sensor, the first type of pixels are R pixels, the second type of pixels are G pixels, the third type of pixels are B pixels, and the first type of pixels are B pixels. The four types of pixels are IR pixels; the method specifically includes: controlling the exposure start time of the R pixel based on the first control unit; controlling the exposure start time of the G pixel based on the second control unit; The control unit controls the exposure start time of the B pixel; controls the exposure start time of the IR pixel based on the fourth control unit; or, the sensor is an RGBW sensor, the first type of pixel is an R pixel, and the second One type of pixel is a G pixel, the third type of pixel is a B pixel, and the fourth type of pixel is a W pixel; the method specifically includes: controlling the exposure start time of the R pixel based on the first control unit ; Based on the second control unit to control the exposure start time of the G pixel; based on the third control unit to control the exposure start time of the B pixel; based on the fourth control unit to control the exposure start time of the W pixel.

In a possible implementation manner, the method further includes: uniformly controlling the exposure end time of all pixels in the pixel array based on the exposure end control unit.

The fifth aspect of the present application provides a computer-readable storage medium with instructions stored in the computer-readable storage medium, which when run on a computer or processor, cause the computer or processor to execute the fourth aspect or The method in any of its possible implementations.

The sixth aspect of the present application provides a computer program product containing instructions, which when run on a computer or processor, causes the computer or processor to execute the fourth aspect or any one of its possible implementations. Methods.

Description of the drawings

FIG. 1 is a schematic diagram of the photosensitive characteristic curve of each pixel of an exemplary RGBIR sensor according to an embodiment of the application;

Figure 2a is a schematic diagram of an exemplary 2X2 array of RGBIR sensors;

Fig. 2b is a schematic diagram of another exemplary 2X2 array sorted RGBIR sensor;

Figure 3a is a schematic diagram of an exemplary 4X4 array of RGBIR sensors;

Figure 3b is a schematic diagram of another exemplary 4X4 array of RGBIR sensors;

Fig. 4 is a schematic structural diagram of an exemplary RGBIR sensor provided by an embodiment of the application;

Fig. 5 is a schematic structural diagram of another exemplary RGBIR sensor provided by an embodiment of the application;

Fig. 6 is a schematic structural diagram of another exemplary RGBIR sensor provided by an embodiment of the application;

FIG. 7a is a schematic diagram of an exemplary 2X2 array sorting RGBIR control connection provided by an embodiment of the application;

FIG. 7b is a schematic diagram of an exemplary 4X4 array sorting RGBIR control connection provided by an embodiment of the application;

FIG. 8 is a timing diagram of an exemplary control signal provided by an embodiment of the application;

FIG. 9a is a schematic diagram of another exemplary 2X2 array sorting RGBIR control connection provided by an embodiment of the application;

Fig. 9b is another exemplary RGBIR control connection diagram of 4X4 array sorting provided by an embodiment of the application;

FIG. 10 is a timing diagram of an exemplary control signal provided by an embodiment of the application;

FIG. 11 An embodiment of the present application provides an exemplary schematic diagram of a sensor in which the exposure time of each pixel can be independently controlled;

FIG. 12 is a timing diagram of an exemplary control signal provided by an embodiment of the application;

FIG. 13 is a graph of photosensitive characteristics of each pixel of an RGBIR sensor provided by an embodiment of the application;

Figure 14a is a schematic diagram of an exemplary 2X2 array of RGBW sensors;

Figure 14b is a schematic diagram of an exemplary 2X2 array of RCCB sensors;

15 is a framework diagram of an exemplary independent exposure control unit of an RCCB sensor provided by an embodiment of the application;

16 is a schematic diagram of the hardware architecture of an exemplary independent exposure apparatus provided by an embodiment of the application;

FIG. 17 is a schematic flowchart of an exemplary image light-sensing method according to an embodiment of the application;

FIG. 18 is a schematic flowchart of an exemplary method for independently controlling exposure time provided by an embodiment of the application.

Detailed ways

The terms "first", "second", etc. in the specification embodiments and claims of this application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. In addition, the terms "including" and "having" and any variations of them are intended to cover non-exclusive inclusion, for example, a series of steps or units are included. The method, system, product, or device is not necessarily limited to those clearly listed steps or units, but may include other steps or units that are not clearly listed or are inherent to these processes, methods, products, or devices.

It should be understood that in this application, "at least one (item)" refers to one or more, and "multiple" refers to two or more. "And/or" is used to describe the association relationship of associated objects, indicating that there can be three types of relationships, for example, "A and/or B" can mean: only A, only B, and both A and B , Where A and B can be singular or plural. The character "/" generally indicates that the associated objects are in an "or" relationship. "The following at least one item (a)" or similar expressions refers to any combination of these items, including any combination of a single item (a) or plural items (a). For example, at least one (a) of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c" ", where a, b, and c can be single or multiple.

People’s requirements for the image quality of imaging equipment are getting higher and higher. For example, in the field of security monitoring, the images acquired by security monitoring equipment must meet the upgrade requirements of the subjective perception of the human eye on the one hand, and also need to meet the requirements of various machine recognition. Demand; another example is the mobile phone camera field, whether it is to shoot portraits or scenery, there are increasingly higher requirements for image clarity and color performance. The quality of the image obtained by the imaging device is related to the light sensitivity of the image sensor and the light condition of the shooting scene. Moreover, more and more application scenarios need to be based on both visible light signals and infrared light signals, such as night video surveillance, live detection, and color-black and white dynamic fusion technology. The imaging results of only visible light do not meet the requirements and require infrared light assistance; However, the existing RGBIR sensor cannot achieve the separation of visible light signal and IR signal. The photosensitive result of visible light signal has a certain degree of IR component, and the color information is not accurate; and the exposure of visible light component and IR component is uniformly controlled, and the sensor is easy There is an exposure imbalance problem, and the photosensitive dynamic range is poor.

The embodiments of the present application provide an RGBIR image sensor, which can realize independent light-sensing of visible light and IR light, strip the IR component signal in the light-sensing result of the visible light signal, and improve the color accuracy of the light-sensing result of the sensor.

Figures 2a and 2b show two exemplary 2X2 arrays of RGBIR sensors, and Figures 3a and 3b show two exemplary 4X4 arrays of RGBIR sensors. In the figure, each grid represents a pixel, R represents a red pixel, G represents a green pixel, B represents a blue pixel, IR represents an infrared light pixel, 2X2 array sorting means that the smallest repeating unit of the RGBIR four-component arrangement is a 2X2 array. The 2X2 array unit contains all the components of R, G, B, and IR; the 4X4 array ordering means that the smallest repeating unit of the RGBIR four-component arrangement is a 4X4 array, and the 4X4 array unit contains all the components. It should be understood that there may also be other arrangements of RGBIR sensors with 2X2 array sorting and 4X4 array sorting, and the embodiment of the present application does not limit the arrangement of RGBIR sensors.

As shown in FIG. 4, it is a schematic structural diagram of an exemplary RGBIR sensor provided by an embodiment of this application.

The RGBIR sensor includes a filter layer, a microlens 403, a pixel array 404, and a charge readout module 405. The filter layer includes: an infrared cut filter layer 401, a red filter layer 402R, and a green filter The layer 402G, the blue filter layer 402B and the infrared filter layer 402IR, the pixel array 404 includes a red pixel R, a green pixel G, and a blue pixel B.

The infrared light cut filter layer 401 may also be referred to as IR-Cut, and is used to cut off optical signals with a wavelength greater than a first preset wavelength, and the optical signals with a wavelength greater than the first preset wavelength include infrared optical signals. Exemplarily, the first preset wavelength is 650 nm, the infrared light cut filter layer 401 is used to cut off optical signals with a wavelength greater than 650 nm, and the optical signals with a wavelength greater than 650 nm include infrared optical signals. Exemplarily, the typical wavelength of visible light rays is about 430 nm to 650 nm, and the typical wavelength of infrared light rays that the IR pixel is sensitive to is about 850 nm to 920 nm. IR-Cut can cut off light signals with a wavelength greater than 650nm, so that infrared light in the wavelength range of about 850nm to 920nm cannot enter the red pixels, green pixels, and blue pixels. The photosensitive characteristics of light passing through the red filter layer 402R in the red pixel are shown by the thin black solid line R in Figure 1. The red pixel has two photosensitive intensity peaks near 650nm of red light and 850nm of IR light; The light-sensitive characteristics of the green filter layer 402G in the green pixel are shown by the short dashed line G in Figure 1. The green pixel has two light-sensitive intensity peaks near 550nm of green light and 850nm of IR light, and the light passes through the blue filter. The photosensitive characteristic of the light layer 402B in the blue pixel is shown by the dotted line B in Fig. 1. The blue pixel has two light-sensitive intensity peaks near 450nm of blue light and 850nm of IR light; light passes through the infrared light filter layer The photosensitive characteristic of the 402IR in the IR pixel is shown by the long dotted line IR in Fig. 1. The IR pixel only has a photosensitive intensity peak near 850nm (910nm) of IR light. Based on this, it can be obtained that the red filter layer 402R can transmit red light and IR light in the first wavelength range at the same time, and the green filter layer 402G can transmit green light and IR light in the second wavelength range at the same time. The filter layer 402B can transmit blue light and IR light in the third wavelength range at the same time. It should be understood that the first wavelength range, the second wavelength range, and the third wavelength range may be the same or different, and infrared light in the first wavelength range, infrared light in the second wavelength range, and infrared light in the third wavelength range The wavelengths of are all greater than the first preset wavelength. The infrared light filter layer 402IR can only transmit IR light in a specific wavelength range. It should be understood that the specific wavelength range may be 850nm-920nm, or the specific wavelength range may be any specific wavelength in the range of 850nm-920nm and in the vicinity thereof. Exemplarily, the IR pixel can mainly photosensitive 850nm IR light, or can mainly photosensitive 910nm IR light, and the IR pixel can photosensitive infrared light of any specific wavelength in the range of 850nm-920nm and its vicinity. This is not limited.

The micro lens 403 is a tiny convex lens device on each photosensitive pixel of the sensor, which is used to concentrate the input light into each photosensitive pixel.

The micro lenses corresponding to the red pixels, green pixels, and blue pixels are respectively coated with an infrared cut filter layer 401, so light exceeding 650 nm cannot enter the red pixels, green pixels, and blue pixels.

The micro lens corresponding to the red pixel is also coated with a red filter layer 402R, so only red light near 650 nm enters the red pixel, and the red pixel can only receive red light.

The micro lens corresponding to the green pixel is also coated with a green filter layer 402G, so only the green light near 550 nm enters the green pixel, and the green pixel can only receive green light.

The micro lens corresponding to the blue pixel is also coated with a blue filter layer 402B, so that only blue light near 450 nm enters the blue pixel, and the blue pixel can only light blue light.

The micro lens corresponding to the infrared light pixel is coated with an infrared light filter layer 402IR, so that only near-infrared light near 850 nm or 910 nm enters the infrared light pixel, and the infrared light pixel can only receive IR light.

In the embodiment of the application, an infrared light cut filter layer is coated on the micro lens corresponding to the red pixel, the green pixel and the blue pixel, which cuts off the IR light reaching the red pixel, the green pixel and the blue pixel, and removes the visible light pixel. The IR component signal in the photosensitive result, the color of the photosensitive result is more accurate, and the photosensitive effect of the sensor is improved. Further, since the embodiment of the present application coats the infrared light cut filter layer on the micro lens based on the coating technology, on the one hand, there is no need to add a complicated mechanical structure; on the other hand, the pixel itself under the micro lens is not changed. Structure, the pixels under the micro lens only have photosensitive devices such as photodiodes, and the relatively simple and stable internal structure of the pixels is conducive to controlling the main light path angle of incidence (Chief Ray Angle, CRA) and other issues that affect imaging. The filter layer is coated on On the micro lens, the light-sensing effect of the sensor is improved while keeping the structure of the pixel itself stable.

The internal structure of the pixel itself is not a smooth inner wall. There are some protrusions on the inner wall of the pixel. If the incident angle of the light is offset from the main optical path of the micro lens, part of the light will be blocked by the protrusions on the inner wall of the pixel, and the sensor itself The photosensitivity effect will decrease. The CRA of the pixel located in the optical center of the sensor is 0 degrees, and the CRA angle of the pixel farther from the optical center is larger. Generally, if the offset distance of the pixel from the center of the screen is taken as the abscissa, the The CRA angle is taken as the ordinate, and the function between the offset distance of the pixel from the center and the CRA angle of the pixel is a linear function. This rule is called CRA consistent performance. In order to make the sensor conform to the law of consistent CRA performance, the position of the micro lens of the pixel needs to be fine-tuned according to the position of the pixel. For example, the micro lens of the pixel located in the optical center is directly above the pixel, and the micro lens of the pixel off the optical center is not in the pixel. Directly above the pixel, the further away from the optical center, the deviation of the micro lens is relatively large. If the internal structure of the pixel under the micro lens is complicated, it is easy to cause inconsistent CRA performance, and the method of fine-tuning the position of the micro lens on the pixel surface may no longer be applicable. The filter layer added by the sensor provided by the embodiment of the application is coated on the micro lens, and does not change the internal structure of the pixel. The internal result of the pixel is simple and stable, and the sensor is improved without affecting the CRA performance of the sensor. The photosensitive effect.

Each pixel in the pixel array 404 includes a photosensitive device, for example, the photosensitive device may be a photodiode, which is used to convert an optical signal into an electrical signal or convert an optical signal into an electric charge.

The charge readout module 405 is used to read out the charge accumulated by the photosensitive device and output it to the subsequent image processing circuit or image processor. The charge readout module is similar to a buffer area, the charge accumulated by the photosensitive device is transferred and temporarily buffered in the charge readout module, and the charge signal of the corresponding pixel is output under the control of the readout signal.

It should be understood that the infrared cut filter layer of the sensor shown in FIG. 4 is respectively coated on the red filter layer, the green filter layer and the blue filter layer. In an optional case, the red filter layer, The green filter layer and the blue filter layer can be respectively coated on the infrared cut-off filter layer, such as the RGBIR sensor shown in Fig. 5. The other parts of the sensor shown in Fig. 5 are the same as those shown in Fig. 4. I won't repeat it here. The image sensor provided in the embodiments of the present application does not limit the positional relationship of the infrared cut filter layer and the red filter layer, the green filter layer and the blue filter layer coated on the micro lens.

In an optional situation, only the infrared cut filter layer can be coated on the micro lens, and the red filter layer can be used in the red pixels, the green filter layer can be used in the green pixels, and the blue filter layer can be used in the green pixels. The color filter layer is made in the blue pixel, and the infrared filter layer is made in the IR pixel.

In an optional case, only the red filter layer, the green filter layer, the blue filter layer, and the infrared filter layer can be coated on the micro lens, and the infrared cut filter layer can be used as Among the red pixels, green pixels, and blue pixels.

As shown in FIG. 6, a structural diagram of another exemplary RGBIR sensor provided in an embodiment of this application.

The sensor includes a filter 601, a filter layer, a micro lens 604, a pixel array 605, and a charge readout module 606. The filter layer includes an infrared cut filter layer 602, a red filter layer 603R, and a green filter A layer 603G, a blue filter layer 603B and an infrared filter layer 603IR, the pixel array 605 includes a red pixel R, a green pixel G, and a blue pixel B.

The filter 601 is used to filter ultraviolet light and infrared light with a wavelength greater than the second preset wavelength, so that visible light and part of the infrared light pass through the filter. The second preset wavelength is greater than the first preset wavelength and any wavelength in the specific wavelength range passed by the infrared light filter layer. In an optional situation, the infrared light with a wavelength greater than the second preset wavelength may be referred to as far-infrared light, and the wavelength of the far-infrared light is greater than the wavelength of the infrared light allowed by the infrared light filter layer. Exemplarily, the visible light rays have a wavelength of about 430 nm to 650 nm, and the typical wavelength range of the infrared light rays of the IR pixel is about 850 nm to 920 nm. The second preset wavelength may be, for example, 900 nm, or 920 nm, or may also be any wavelength between 850 nm and 950 nm. Exemplarily, the filter may be an all-pass filter or a dual-pass filter. An exemplary all-pass filter is used to filter ultraviolet light with a wavelength of less than 400 nm and wavelengths greater than 900 nm. Infrared light. An exemplary double-pass filter is used to pass only visible light and infrared light in the range of 800 nm to 900 nm. At this time, the double-pass filter is equivalent to filtering out infrared light greater than 900 nm; in an optional case Among them, the double-pass filter is used to pass only visible light and infrared light in the range of 900 nm to 950 nm. At this time, the double-pass filter is equivalent to filtering out infrared light greater than 950 nm. It should be understood that the wavelength of the infrared light filtered by the all-pass filter and the wavelength of the infrared light allowed by the double-pass filter can be designed as required, which is not limited in the embodiment of the present application. The filter 601 can prevent the long-wavelength far-infrared light and ultraviolet light from affecting the photosensitive characteristics of the photosensitive device.

The filter layer is the same as that of the sensor shown in FIGS. 4 and 5, the infrared cut filter layer 602, the red filter layer 603R, the green filter layer 603G, the blue filter layer 603B and the infrared filter layer 603IR Both are coated on the micro lens, and the upper and lower positions of the infrared cut filter layer 602 and the filter layers 603R-603B of each visible light component are not limited.

The light enters the pixel array through the filter, the filter layer and the micro lens in sequence. Therefore, the wavelength of part of the infrared light filtered by the filter is greater than the wavelength of the infrared light allowed by the infrared filter layer in the filter layer, and the infrared cut filter layer can filter out all the light larger than the visible light part, thereby Avoid infrared light from entering the red pixels, blue pixels, and green pixels.

For the micro lens 604, the pixel array 605, and the charge readout module 606, please refer to the description of the embodiment corresponding to FIG. 4, which will not be repeated here.

Further, an embodiment of the present application also provides a sensor capable of independently controlling the exposure time of visible light pixels and infrared light pixels, as shown in FIG. 7a and FIG. 7b, where FIG. 7a is an exemplary 2X2 array of RGBIR The schematic diagram of the control connection, FIG. 7b is an exemplary schematic diagram of the RGBIR control connection of 4X4 array sorting.

The sensor includes a pixel array 710 and a logic control circuit 720.

The pixel array 710 is the pixel array in the sensor as shown in any of the embodiments in FIGS. 4 to 6, for example, the pixel array 404, the pixel array 504 or the pixel array 605.

The logic control circuit 720 is used to independently control the exposure time of the visible light pixels and the infrared light pixels. The visible light pixels include red pixels, green pixels, and blue pixels. Specifically, the logic control circuit 720 includes a first control line and a second control line, or in other words, includes two independent control circuits: a first control circuit and a second control circuit. The visible light pixels R, G, and B in the pixel array 710 are coupled to the first control line, and the IR pixels in the pixel array 710 are coupled to the second control line. It should be understood that the control lines with the same name in FIGS. 7a and 7b are the same line or connected to each other. For example, the first control line on the pixel array side and the first control line of the logic control circuit are the same line or Connected, the second control line on the pixel array side and the second control line of the logic control circuit are the same line or connected.

If y is the number of rows of pixels in the pixel array, x is the number of columns of pixels in the pixel array, 0≤x≤M-1,0≤y≤N-1, and M is the number of pixels in the pixel array The total number of columns, N is the total number of rows of the pixel array, then for the RGBIR sensor arranged in a 2X2 array as shown in Figure 7a,

The coordinates of visible light pixels meet the following conditions: y%2=0 and y%2=1 and x%2=1,

The coordinates of the IR pixel satisfy the following conditions: y%2=1 and x%2=0.

For the RGBIR sensor arranged in a 4X4 array as shown in Figure 7b,

The coordinates of visible light pixels meet the following conditions: y%2=0 and y%2=1 and x%2=0,

The coordinates of the IR pixel satisfy the following conditions: y%2=1 and x%2=1.

Among them, "%" is the remainder operation, "y%2=0" means that the remainder of y divided by 2 is 0, and "x%2=1" means that the remainder of x divided by 2 is 1. Among them, "%" is the remainder operation, "y%2=0" means that the remainder of y divided by 2 is 0, and "x%2=1" means that the remainder of x divided by 2 is 1.

The pixel located at the visible light pixel location is coupled to the first control line, and the pixel located at the IR pixel location is coupled to the second control line. It should be understood that when the RGBIR arrays are arranged differently, the respective coordinate conditions of the RGBIR pixels will change accordingly. Therefore, the connection between the logic control circuit and the pixel array needs to be designed correspondingly according to the arrangement of the sensors.

For the RGBIR sensor arranged in a 2X2 array as shown in Figure 2b,

For the RGBIR sensor arranged in a 4X4 array as shown in Figure 3b,

The coordinates of visible light pixels meet the following conditions: y%2=1 and y%2=0 and x%2=1,

The coordinates of the IR pixel satisfy the following conditions: y%2=0 and x%2=0.

The coordinate conditions satisfied by the pixel coordinates of the sensors of other arrangements will not be listed one by one.

The first control line outputs a first control signal, and the second control line outputs a second control signal. The first control signal is used to control the exposure start time of visible light pixels, and the second control signal is used to control the exposure start time of infrared light pixels. . The first control signal and the second control signal are independent of each other, so the exposure start time of the visible light pixel and the infrared light pixel may be different. Exemplarily, when the first effective transition edge of the first control signal arrives, the visible light pixel starts to expose, and when the second effective transition edge of the second control signal arrives, the infrared light pixel starts to expose. The effective transition edges of the first control signal and the second control signal can be both falling edges or rising edges, or one can be a falling edge and the other can be a rising edge. The effective transition edge of the control signal in the embodiment of the present application is Not limited. As shown in FIG. 8, it is an exemplary control signal timing diagram. In this figure, the effective transition edges of the first control signal and the second control signal are both falling edges. In an optional case, the first control signal and the second control signal can be reset according to the system of the logic control circuit. The signal gets. As shown in Figure 8, the first control signal and the second control signal are both high-level active signals. When the falling edge of the first control signal arrives, the visible light pixel starts to expose. When the falling edge of the second control signal arrives, The infrared light pixels are exposed.

Optionally, the logic control circuit 720 further includes a reset signal. The reset signal may be a system clock signal, and the first control signal and the second control signal may both be obtained from the reset signal. Exemplarily, the logic control circuit 720 includes a logic operation circuit. The logic operation circuit may include logic operations such as AND, OR, NOT, XOR, etc. The logic operation circuit includes three inputs: a variable x, a variable y, and a reset signal. The logic operation circuit includes two output terminals: a first control line and a second control line. If the variable x and the variable y meet the coordinate conditions of the visible light pixel, the reset signal is connected to the output terminal of the first control line; if the variable When x and variable y meet the coordinate condition of the infrared light pixel, the reset signal is connected to the output terminal of the second control line.

Optionally, the logic control circuit 720 further includes:

The exposure end control line is used to uniformly control the exposure stop time of all pixels in the pixel array.

The exposure end control line outputs the exposure end signal. The exposure end signal can be a high-level active signal or a low-level active signal. The exposure end time point can be a high-level falling edge or a low-level rising edge. along. In the control signal timing chart shown in FIG. 8, the exposure end control signal is a high-level active signal. When the falling edge of the exposure end control signal arrives, all pixels in the pixel array stop exposure. That is, the exposure time for visible light pixels and IR pixels in the pixel array is independently controlled by the first control line and the second control line, and the exposure end time is uniformly controlled by the exposure end control line. As shown in FIG. 8, the The exposure time is the time difference between the falling edge of the first control signal and the falling edge of the exposure end control signal: the first exposure time, the exposure time of the IR pixel is the difference between the falling edge of the second control signal and the falling edge of the exposure end control signal Time difference between: the second exposure time. Therefore, the exposure time of visible light pixels and IR pixels is independently controlled.

In a low-light scene, if you directly use the RGBIR sensor for light exposure, because the visible light energy is less, the signal-to-noise ratio of the light-sensing result will be smaller; if you use the IR lamp to fill light, the IR light is much stronger than the visible light, in the same exposure time situation Below, the visible light energy that can be captured is much less than the IR light energy, and if the amount of visible light information is forcibly increased, the IR light will be overexposed, and the image with unbalanced exposure will lose a lot of effective information. If visible light and IR light are exposed separately, the exposure time of IR light is reduced, and the exposure time of visible light is relatively prolonged, which can effectively improve the detailed information contained in the photosensitive result of the visible light signal.

In an exemplary embodiment, by controlling the arrival time of the first effective transition edge of the first control signal and the second effective transition edge of the second control signal, the exposure time of the visible light pixel and the infrared light pixel Meet the preset ratio. For example, when the ratio of the exposure time of the visible light signal to the infrared light signal is 2:1, the definition of the exposure result is better and the signal-to-noise ratio is higher, so that the control signal of the visible light signal jumps first, and the infrared light signal jumps later. And it is ensured that the time difference between the jump time points of the two signals makes the exposure time of the visible light signal and the exposure time of the infrared light signal meet the preset ratio.

In the image sensor provided by the embodiments of the present application, the exposure time of visible light pixels and IR pixels is independently controlled. For example, when infrared light is too strong and visible light is too weak, the exposure time of visible light can be increased and the exposure time of infrared light can be reduced, so that The exposure time of visible light and infrared light tends to be balanced to avoid exposure imbalance when infrared light is the dominant component or visible light is the dominant component, and the dynamic range of the sensor's sensitivity is improved to meet the user's indicators of clarity and signal-to-noise ratio Requirements. Further, by accurately setting the exposure time ratio of the visible light signal to the infrared light signal, the light-sensing effect of the sensor can be controlled more finely.

Optionally, the logic control circuit 720 further includes:

The charge transfer control line is used to control the time point when the charge accumulated by the photosensitive device of the pixel array is transferred to the charge readout module. The charge transfer control signal is output in the charge transfer control line, and the charge transfer control signal can be a high-level effective signal or a low-level effective signal. In the control signal timing diagram shown in FIG. 8, the charge transfer control signal is a high-level effective signal. When the falling edge of the charge transfer control signal arrives, the accumulated charge is transferred from the photosensitive device to the charge readout module. In an optional situation, after the charge transfer control signal is reset, the exposure end control signal is reset again.

It should be understood that the functions of the logic control circuit may also be implemented by software codes running on the processor, or the functions of the logic control circuit may be partially implemented by hardware circuits and partially implemented by software modules. Exemplarily, the sensor may include a pixel array and a control unit. The control unit is a software module running on a processor. The control unit includes a first control unit and a second control unit for independently controlling visible light pixels and IR pixels. The exposure start time; the control unit also includes an exposure end control unit for uniformly controlling the exposure end time of each pixel in the pixel array. The control unit also includes a charge transfer control unit and a reset unit. The reset unit is used to provide the above reset signal. The function of the charge transfer control unit is similar to the charge transfer control line, which will not be repeated here.

Further, an embodiment of the present application also provides a sensor capable of independently controlling the exposure time of the four components of RGBIR, as shown in FIG. 9a and FIG. 9b, where FIG. 9a is an exemplary 2X2 array sequenced RGBIR control connection Schematic diagram, FIG. 9b is an exemplary 4X4 array sorting RGBIR control connection diagram.

The sensor includes a pixel array 910 and a logic control circuit 920.

The pixel array 910 is the pixel array in the sensor as shown in any of the embodiments in FIGS. 4 to 6, such as the pixel array 404, the pixel array 504 or the pixel array 605.

The logic control circuit 920 is used to independently control the exposure time of the red pixel, the green pixel, the blue pixel and the infrared light pixel. Specifically, the logic control circuit 920 includes a first control line, a second control line, a third control line, and a fourth control line, or it can be said that it includes four independent control circuits: a first control circuit and a second control circuit. , The third control circuit and the fourth control circuit. The red pixels in the pixel array are coupled to the first control line, the green pixels are coupled to the second control line, the blue pixels are coupled to the third control line, and the infrared light pixels are coupled to the fourth control line. It should be understood that the control lines with the same name in FIGS. 9a and 9b are the same control line and are connected to each other. For example, the first control line on the pixel array side and the first control line of the logic control circuit are the same line, and the pixel array The fourth control line on the side is the same line as the fourth control line of the logic control circuit, and so on.

If y is the number of rows of pixels in the pixel array, x is the number of columns of pixels in the pixel array, 0≤x≤M-1,0≤y≤N-1, and M is the number of pixels in the pixel array The total number of columns, N is the total number of rows of the pixel array, then for the RGBIR sensor arranged in a 2X2 array as shown in Figure 9a,

The coordinates of the red pixel meet the following conditions: y%2=0 and x%2=0,

The coordinates of the green pixel meet the following conditions: y%2=0 and x%2=1,

The coordinates of the blue pixels meet the following conditions: y%2=1 and x%2=1,

For the RGBIR sensor arranged in a 4X4 array as shown in Figure 9b,

The coordinates of the red pixel satisfy the following conditions: y%4=0 and x%4=0, and y%4=2 and x%4=2;

The coordinates of the green pixels meet the following conditions: y%2! =x%2,

The coordinates of the blue pixel satisfy the following conditions: y%4=0 and x%4=2, and y%4=2 and x%4=0,

Among them, "%" is the remainder operation, "y%4=0" means that the remainder of y divided by 4 is 0, "x%4=2" means that the remainder of x divided by 4 is 2, and "!=" means no equal.

The coordinates of the pixels coupled to the first control line meet the coordinates of the red pixels, the coordinates of the pixels coupled to the second control line meet the coordinates of the green pixels, and the coordinates of the pixels coupled to the third control line meet the coordinates of the blue pixels Condition, the coordinate of the pixel coupled to the fourth control line meets the coordinate condition of the infrared light pixel. It should be understood that when the RGBIR arrays are arranged differently, the respective coordinate conditions of the RGBIR pixels will change accordingly. Therefore, the connection between the logic control circuit and the pixel array needs to be designed correspondingly according to the arrangement of the sensors. As shown in Figure 2b for the 2X2 array of RGBIR sensors, the coordinates of the red pixels meet the following conditions:

y%2=1 and x%2=0,

The coordinates of the green pixel meet the following conditions: y%2=0 and x%2=0,

The coordinates of the blue pixel meet the following conditions: y%2=0 and x%2=1,

For the RGBIR sensor arranged in a 4X4 array as shown in Fig. 3b, the coordinates of the red pixels meet the following conditions: y%4=1 and x%4=3, and y%4=3 and x%4=1;

The coordinates of the green pixels meet the following conditions:

y%2! =x%2,

The coordinates of the blue pixels meet the following conditions:

y%4=1 and x%4=1, and y%4=3 and x%4=3,

The first control line outputs the first control signal, the second control line outputs the second control signal, the third control line outputs the third control signal, and the fourth control line outputs the fourth control signal. The first control signal is used to control the red pixel Exposure start time, the second control signal is used to control the exposure start time of the green pixel, the third control signal is used to control the exposure start time of the blue pixel, and the fourth control signal is used to control the exposure start of the infrared light pixel time. The first to fourth control signals are independent of each other, so the exposure start time of the four components of RGBIR can be different. Exemplarily, when the first effective transition edge of the first control signal arrives, the red pixel begins to be exposed, and when the second effective transition edge of the second control signal arrives, the green pixel begins to be exposed. When the third valid transition edge arrives, the blue pixel starts to be exposed, and when the fourth valid transition edge of the fourth control signal comes, the IR pixel starts to expose. The first control signal to the fourth control signal can all be high-level effective signals, and the valid transition edges of the first control signal to the fourth control signal can all be falling edges, all rising edges, or part of falling edges. The remaining part is the rising edge, and the embodiment of the present application does not limit the effective transition edge of the control signal. As shown in Fig. 10, it is an exemplary control signal timing diagram. The first control signal to the fourth control signal are all high-level effective signals, and the valid transition edges of the first control signal to the fourth control signal are all falling edges. In an optional case, the first control signal to The fourth control signal can be obtained according to the system reset signal of the logic control circuit. As shown in Figure 10, when the falling edge of the first control signal arrives, the red pixels begin to be exposed. When the falling edge of the second control signal arrives, the green pixels begin to expose. When the falling edge of the third control signal arrives, the blue pixels begin to be exposed. The color pixel starts to be exposed, and when the falling edge of the fourth control signal arrives, the IR pixel starts to be exposed.

Optionally, the logic control circuit 920 further includes a reset signal, the reset signal may be a system clock signal, and the first control signal to the fourth control signal may all be obtained from the reset signal. Exemplarily, the logic control circuit 920 includes a logic operation circuit. The logic operation circuit may include logical operations such as AND, OR, NOT, XOR, etc. The logic operation circuit includes three inputs: a variable x, a variable y, and a reset signal. , The logic operation circuit includes four output terminals: the first control line to the fourth control line. If the variable x and variable y meet the coordinate condition of the red pixel, connect the reset signal to the output terminal of the first control line; if the variable x and variable y meet the coordinate condition of the green pixel, connect the reset signal to the coordinate condition of the second control line The output terminal is connected; if the variable x and the variable y meet the coordinate condition of the blue pixel, connect the reset signal to the output terminal of the third control line; if the variable x and the variable y meet the coordinate condition of the infrared light pixel, the reset signal Connect with the output terminal of the fourth control line. It should be understood that when the RGBIR arrays are arranged differently, the respective coordinate conditions of the RGBIR pixels will change accordingly. Therefore, the logic operation circuit inside the logic control circuit needs to be adjusted correspondingly according to the arrangement of the pixel array.

Optionally, the logic control circuit 920 further includes:

The exposure end control line outputs the exposure end signal. The exposure end signal can be a high-level active signal or a low-level active signal. The exposure end time point can be a high-level falling edge or a low-level rising edge. along. In the control signal timing chart shown in FIG. 10, the exposure end control signal is a high-level active signal. When the falling edge of the exposure end control signal arrives, all pixels in the pixel array stop exposure. That is, the time for the four types of RGBIR pixels in the pixel array to start exposure is independently controlled by the first control line to the fourth control line, and the time for ending exposure is uniformly controlled by the exposure end control line. For example, as shown in FIG. 10 In the control signal timing diagram, the exposure time of the R pixel is the time difference between the falling edge of the first control signal and the falling edge of the exposure end control signal: the first exposure time, the exposure time of the G pixel, the B pixel and the IR pixel are respectively The second exposure time, the third exposure time, and the fourth exposure time. Therefore, the exposure time of the four RGBIR components is independently controlled.

In an exemplary embodiment, the time when the first effective transition edge of the first control signal arrives to the fourth effective transition edge of the fourth control signal can be controlled, so that the exposure time of the four components of RGBIR meets the preset requirements. Set the ratio.

Optionally, the logic control circuit 920 further includes a charge transfer control line for controlling when to transfer the charge accumulated by the photosensitive device of the pixel array to the charge readout module. The charge transfer control signal is output in the charge transfer control line, and the charge transfer control signal can be a high-level effective signal or a low-level effective signal. The charge transfer control signal shown in FIG. 10 is the same as that shown in FIG. 8.

It should be understood that the functions of the logic control circuit may also be implemented by software codes running on the processor, or the functions of the logic control circuit may be partially implemented by hardware circuits and partially implemented by software modules. Exemplarily, the sensor may include a pixel array and a control unit, the control unit is a software module running on a processor, and the control unit includes a first control unit, a second control unit, a third control unit, and a fourth control unit, It is used to independently control the exposure start time of the four components of R, G, B and IR; the control unit also includes an exposure end control unit for uniformly controlling the exposure end time of the four components of the pixel. The control unit also includes a charge transfer control unit and a reset unit. The reset unit is used to provide a reset signal. The function of the charge transfer control unit is similar to the charge transfer control line, which will not be repeated here.

In the image sensor provided by the embodiment of the present application, the exposure time of the four components of R, G, B, and IR is independently controlled, which further improves the dynamic range of the sensor. When some scenes have high requirements for the sensitivity of R and G components and want to reduce the sensitivity of B and IR, you can flexibly control the exposure time of the four components to enhance the sensitivity of R and G components and weaken the B and IR. The light-sensing effect of the components makes the final light-sensing result more in line with the requirements of the scene or the sharpness or signal-to-noise ratio of the customer requirements. Further, the exposure time of the four components of R, G, B, and IR may be preset to meet a preset ratio, so as to achieve fine control of the photosensitive effect of the sensor.

Further, an embodiment of the present application also provides a sensor in which the exposure time of each pixel can be independently controlled. As shown in FIG. 11, the sensor includes a pixel array 1110 and a logic control circuit 1120.

The pixel array 1110 is the pixel array in the sensor as shown in any of the embodiments in FIGS. 4 to 6, such as the pixel array 404, the pixel array 504 or the pixel array 605.

The logic control circuit 1120 includes a row coordinate control circuit and a column coordinate control circuit, or in other words, includes a row coordinate control line and a column coordinate control line, and each pixel in the pixel array is coupled to its own row coordinate control line and column coordinate control line.

The logic control circuit 1120 also includes a reset signal and an exposure start control line. When the row coordinate control signal output from the row coordinate line of the target pixel and the column coordinate control signal output from the column coordinate line are both valid signals, the exposure start control line outputs The reset signal is sent to the target pixel, and the exposure start time of the target pixel is controlled based on the reset signal. Exemplarily, the exposure start control line has multiple branches, and each pixel is coupled to one branch. When the row coordinate control signal and column coordinate control signal of the target pixel meet the requirements, the branch corresponding to the target pixel outputs an effective control signal . The column coordinate control line and the row coordinate control line are equivalent to the switch. The reset signal is input and the exposure start control line is output. When the signals in the column coordinate control line and row coordinate control line are both valid signals, the switch is turned on and the reset signal is You can output to the target pixel through the exposure start control line, and control the exposure of the target pixel. Exemplarily, when the signals in the column coordinate control line and the row coordinate control line are both valid signals, and the valid transition edge of the reset signal arrives, the target pixel is controlled to start exposure. If there is a signal in the column coordinate control line and the row coordinate control line that does not meet the requirements, the switch is closed, and the exposure start control line has no control signal output. Since each pixel in the pixel array has its own corresponding row coordinate line and column coordinate line, the exposure time of each pixel can be independently controlled, for example, the row coordinate line and column coordinate of the pixel that needs to be exposed The signal in the line is preferentially set as a valid signal, thereby extending the exposure time of the key exposure pixels.

Optionally, the logic control circuit 1120 further includes: an exposure end control line for uniformly controlling the exposure end time of each pixel in the pixel array. For details, please refer to the exposure end control lines of the logic control circuit 720 and the logic control circuit 920, where No longer.

Optionally, the logic control circuit 1120 further includes: a charge transfer control line for controlling when the charge accumulated in the photosensitive device is transferred to the charge readout module. For details, please refer to the charge transfer of the logic control circuit 720 and the logic control circuit 920 Control line, no more details here.

It should be understood that the functions of the logic control circuit may also be implemented by software codes running on the processor, or the functions of the logic control circuit may be partially implemented by hardware circuits and partially implemented by software modules. Exemplarily, the sensor may include a pixel array and a control unit, the control unit is a software module running on a processor, the control unit includes a row control unit, a column control unit, and an exposure start control unit, a row control unit and a column control unit For indicating the row coordinates and ordinate of the pixels, the exposure start control unit is used for outputting an effective control signal to control the exposure start time of the target pixel when the row control unit and the column control unit of the target pixel meet the requirements.

The sensor provided in the embodiments of the present application can control the start time of the pixel exposure according to the control signal in the row coordinate control line and the column coordinate control line of each pixel, and the exposure end time is uniformly controlled by the exposure end control line, so The exposure time of each pixel can be different. Further, it is possible to set the time when the row coordinate control signal and the column coordinate control signal corresponding to the pixel both become valid signals, so that the exposure time of each pixel meets the preset ratio. In some scenes where pixels in the target area need to be enhanced, only the exposure time of the pixels in the target area can be increased, which further improves the flexibility of the sensor's light sensitivity and further satisfies the user's requirements for light-sensing results.

As shown in FIG. 12, it is an exemplary control signal timing diagram. FIG. 12 uses two pixels as an example to illustrate the control of the exposure start control signal on the pixel exposure start time. The signals in the timing diagram are all high-level effective, and it should be understood that each control signal may also be low-level effective.

The first pixel is coupled to the first row coordinate control line and the first column coordinate control line. The signal in the first row coordinate control line is row coordinate control signal 1, and the signal in the first column coordinate control line is column coordinate control signal 1. , The second pixel is coupled to the second row coordinate control line and the second column coordinate control line, the signal in the second row coordinate control line is row coordinate control signal 2, and the signal in the second column coordinate control line is the column coordinate control signal 2. When the row coordinate control signal 1 and the column coordinate control signal 1 of the first pixel are both high, the exposure start control signal of the first pixel is valid. Specifically, the reset signal is used as the exposure start control signal, and when the reset signal falls When the edge comes, the first pixel is controlled to start exposure; when the row coordinate control signal 2 and column coordinate control signal 2 of the second pixel are both high, the exposure start control signal of the second pixel is valid. Specifically, the reset signal is used as Exposure start control signal, and when the falling edge of the reset signal arrives, control the second pixel to start exposure. When the falling edge of the exposure end control signal arrives, both the first pixel and the second pixel stop exposure. So far, the exposure time of the first pixel is the first exposure time, and the exposure time of the second pixel is the second exposure time. It should be understood that the exposure start control signal of the first pixel and the exposure start control signal of the second pixel may be two different branches of the same signal. When the coordinate control signal of the first pixel meets the requirements, the branch corresponding to the first pixel outputs Effective control signal; when the coordinate control signal of the second pixel meets the requirements, the branch corresponding to the second pixel outputs an effective control signal.

As shown in FIG. 13, it is a graph of the photosensitive characteristic curve of each pixel of the RGBIR sensor provided by the embodiment of the application. Among them, the abscissa is the wavelength of the light, the unit is nm, and the ordinate is the sensitivity of light. The thin solid line is the photosensitive characteristic curve of the R pixel, the short dashed line is the photosensitive characteristic curve of the G pixel, the dotted line is the photosensitive characteristic curve of the B pixel, and the long dashed line is the photosensitive characteristic curve of the IR pixel. It can be seen from Figure 13 that the R pixel only has a photosensitive intensity peak near 650nm in red light, the G pixel only has a photosensitive intensity peak near 550nm in green light, and the B pixel only has a photosensitive intensity peak near blue light at 450nm. IR pixels only have a peak in infrared light. There is a photosensitive intensity peak near 850nm (which can be 910nm in some cases). Compared with the existing RGBIR sensor, the RGBIR sensor provided in the embodiments of the present application removes the IR component of the R, G, and B pixel light-sensing results, so that the R pixel can only receive red light, and the G pixel can only receive green light and B Pixels can only light blue light, which improves the color accuracy of the sensor's light-sensing results.

Exemplarily, the sensors provided in the embodiments of the present application can be used in security surveillance cameras, intelligent transportation electronic eye devices, video cameras, cameras, mobile phones, and other terminal devices with imaging, photographing, or video recording functions in a community.

As shown in FIG. 14a, it is a schematic diagram of an exemplary RGBW (red green blue white) sensor arranged in a 2X2 array. As shown in Fig. 14b, it is a schematic diagram of an exemplary RCCB (red clear clear blue) sensor arranged in a 2X2 array. In the figure, each grid represents a pixel. In the RGBW sensor, R represents a red pixel, G represents a green pixel, B represents a blue pixel, and W represents a white light pixel. In the RCCB sensor, R represents red pixels, B represents blue pixels, and C represents clear pixels. The RCCB sensor replaces G pixels in RGB sensors with C pixels. The wavelength range allowed by C pixels is between 400nm and 657nm, clear pixels The amount of light that can pass is large, and the C pixel matches the current demosaic algorithm.

The embodiment of the present application also provides an RGBW sensor capable of independently controlling the exposure time of visible light pixels and W pixels. The exposure control of the RGBW sensor can be implemented by a logic control circuit or a control unit, which can be a software module running on a processor. The control logic of the independent exposure of the RGBW sensor is similar to the control logic of the independent exposure of the RGBIR sensor, and the IR pixels can be replaced with W pixels. The logic control circuit for the independent exposure of visible light pixels and W pixels of the RGBW sensor refers to the logic control circuit of the RGBIR sensor as shown in FIG. 7a, which will not be repeated here.

Correspondingly, an embodiment of the present application also provides an RGBW sensor capable of independently controlling the exposure time of the four components of RGBW. The exposure control can be implemented by a logic control circuit or a control unit, and the control unit can be software running on a processor. Module. The control logic of RGBW sensor 4-component independent exposure is similar to the control logic of RGBIR sensor 4-component independent exposure. The logic control circuit of 4-component independent exposure RGBW sensor refers to the logic control circuit of RGBIR sensor as shown in Figure 9a. Repeat.

The embodiment of the present application also provides an RCCB sensor capable of independently controlling the exposure time of visible light pixels and C pixels. The exposure control of the RCCB sensor can be completed by a logic control circuit or a control unit. The logic control circuit of the RCCB in the embodiment of this application refers to the logic control circuit 720, where the first control line is used to control R pixels and B pixels, and the second control line is used To control 2 C pixels.

Correspondingly, the embodiment of the present application also provides an RCCB sensor with independently controllable exposure times of the three components of R, B, and C. The exposure control of the RCCB sensor can be completed by a logic control circuit or a control unit, where the control unit is Examples are explained. As shown in FIG. 15, an exemplary RCCB sensor independent exposure control unit provided by this embodiment of the application, the control unit 1500 includes a first control unit 1510, a second control unit 1520, and a third control unit 1530, Among them, the first control unit 1510 is used to control the exposure start time of R pixels, the second control unit 1520 is used to control the exposure start time of B pixels, and the third control unit 1530 is used to control the exposure start time of C pixels. . The control unit 1500 also includes an exposure end control unit 1540 for uniformly controlling the exposure end time of all pixels in the pixel array. The control unit 1500 may also include a charge transfer control unit 1550 and a reset unit 1560. For the functions of the charge transfer control unit 1550 and the reset unit 1560, please refer to the description of the charge transfer control lines and reset signals in 720 and 920, which will not be repeated here. . Exemplarily, the control unit may be a software module running on a general-purpose processor or a special-purpose processor.

The embodiment of the present application provides an independent exposure device for controlling the exposure time of the pixel array of the sensor. The device includes at least two control units, and each of the at least two control units is used to correspond to The exposure start time of one type of pixel in the pixel array of the sensor is controlled, and the pixel array of the sensor controlled by the device includes at least two types of pixels.

It should be understood that the independent exposure device can be regarded as a control device independent of the sensor, for example, it can be a general-purpose processor or a dedicated processor, or can be regarded as an independently solidified hardware logic or hardware circuit. For example, the independent exposure device can be considered as the logic control circuit in Fig. 7a and Fig. 7b, Fig. 9a and Fig. 9b, Fig. 11 or the control unit in Fig. 15.

As shown in FIG. 16, it is a schematic diagram of the hardware architecture of an independent exposure apparatus provided by an embodiment of this application. It should be understood that the logic control circuits in FIGS. 7a, 7b, 9a, 9b, and 11 described above can all be completed by software modules running on the independent exposure device shown in FIG. 16. The control unit shown in FIG. 15 can also be completed by a software module running on the exposure control device shown in FIG. 16.

The exposure control device 1600 includes: at least one central processing unit (Central Processing Unit, CPU), at least one memory, a microcontroller (Microcontroller Unit, MCU), a receiving interface, a transmitting interface, and the like. Optionally, the exposure control device 1600 further includes: a dedicated video or graphics processor, and a graphics processing unit (Graphics Processing Unit, GPU), etc.

Optionally, the CPU may be a single-CPU processor or a multi-CPU processor; optionally, the CPU may be a processor group composed of multiple processors, between multiple processors Coupled to each other through one or more buses. In an optional situation, the exposure control can be partly completed by software codes running on a general-purpose CPU or MCU, and partly completed by hardware logic circuits; or it can also be completely completed by software codes running on a general-purpose CPU or MCU. Optionally, the memory 302 may be a non-power-down volatile memory, such as Embedded MultiMedia Card (EMMC), Universal Flash Storage (UFS) or Read-Only Memory (Read-Only Memory, ROM), or other types of static storage devices that can store static information and instructions, or volatile memory (volatile memory), such as Random Access Memory (RAM), or can store information and Other types of dynamic storage devices for instructions can also be Electrically Erasable Programmable Read-Only Memory (EEPROM), Compact Disc Read-Only Memory, CD-ROM or other optical discs Storage, optical disc storage (including compact discs, laser discs, optical discs, digital universal discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or can be used to carry or store program codes in the form of instructions or data structures and can Any other computer-readable storage medium accessed by the computer, but not limited to this. The receiving interface may be a data input interface of the processor chip.

In a possible implementation, the independent exposure device further includes a pixel array. In this case, the independently exposed device includes at least two types of pixels, that is, the independently exposed device can be a sensor including a control unit or a logic control circuit, or in other words, the independently exposed device can be A sensor that independently controls exposure. Exemplarily, the independent exposure device may be an RGBIR sensor, an RGBW sensor, an RCCB sensor, etc. that independently control exposure.

It should be understood that in an optional case, visible light pixels are classified as one type of pixel, that is, R pixels, G pixels, and B pixels are classified as one type of pixels, and IR pixels, W pixels, or C pixels Pixels are considered to be another type of pixels. For example, RGBIR sensors include two types of pixels: visible light pixels and IR pixels, RGBW sensors include two types of pixels: visible light pixels and W pixels, and RCCB sensors include two types of pixels. Pixels: visible light pixels and C pixels.

In another optional case, each pixel component is considered to be a type of pixel. For example, the RGBIR sensor includes four types of pixels: R, G, B, and IR, and the RGBW sensor includes: R, G, There are four types of pixels, B and W, and the RCCB sensor includes three types of pixels: R, B and C.

In a possible implementation manner, the sensor is an RGBIR sensor. The RGBIR sensor can realize the independent exposure of visible light pixels and IR pixels, and can also realize independent exposure of the four components of R, G, B, and IR.

For an RGBIR sensor where visible light pixels and IR pixels are separately exposed, the at least two control units include: a first control unit and a second control unit; the first control unit is used to control the exposure start time of the visible light pixels; the second control unit Used to control the exposure start time of IR pixels.

For the RGBIR sensor with independent exposure of the four components of R, G, B, and IR, at least two control units include: a first control unit, a second control unit, a third control unit, and a fourth control unit; the first control unit is used for Control the exposure start time of R pixels; the second control unit is used to control the exposure start time of G pixels; the third control unit is used to control the exposure start time of B pixels; the fourth control unit is used to control the exposure of IR pixels Start time.

In a possible implementation manner, the sensor is an RGBW sensor. The RGBW sensor can realize independent exposure of visible light pixels and W pixels, or can realize independent exposure of the four components of R, G, B, and W.

For an RGBW sensor where visible light pixels and W pixels are separately exposed, the at least two control units include: a first control unit and a second control unit; the first control unit is used to control the exposure start time of the visible light pixels; the second control unit Used to control the exposure start time of W pixels.

For the R, G, B, and W four components independently exposed RGBW sensor, at least two control units include: a first control unit, a second control unit, a third control unit and a fourth control unit; the first control unit is used for Control the exposure start time of R pixels; the second control unit is used to control the exposure start time of G pixels; the third control unit is used to control the exposure start time of B pixels; the fourth control unit is used to control the exposure of W pixels Start time.

In a possible implementation, the sensor is an RCCB sensor. The RCCB sensor can realize the independent exposure of visible light pixels and C pixels respectively, and can also realize independent exposure of the three components of R, B, and C.

For an RGBW sensor with visible light pixels and C pixels exposed separately, the at least two control units include: a first control unit and a second control unit; the first control unit is used to control the exposure start time of the visible light pixels; the second control unit Used to control the exposure start time of C pixels.

For the RCCB sensor with independent exposure of the three components of R, B and C, at least two control units include: a first control unit, a second control unit and a third control unit; the first control unit is used to control the start of exposure of the R pixel Time; the second control unit is used to control the exposure start time of the B pixel; the third control unit is used to control the exposure start time of the C pixel.

In a possible implementation, the independent exposure device may also control the exposure time of the at least two types of pixels to meet a preset ratio based on at least two control units. Exemplarily, based on the first control unit and the second control unit controlling the exposure time of the visible light pixels and the IR pixels to meet the preset ratio; or based on the first control unit, the second control unit, the third control unit, and the fourth control unit control The exposure time of R, G, B and IR pixels meets the preset ratio. Or, based on the first control unit and the second control unit to control the exposure time of the visible light pixel and the W pixel to meet the preset ratio; or, based on the first control unit, the second control unit, the third control unit, and the fourth control unit to control R , G, B and W pixels’ exposure time meets a preset ratio; or, based on the first control unit and the second control unit controlling the exposure time of the visible light pixel and the C pixel to meet the preset ratio; or, based on the first control unit, the The second control unit and the third control unit control the exposure time of the R, B and C pixels to meet the preset ratio.

In a possible implementation manner, the independent exposure device further includes: an exposure end control unit for uniformly controlling the exposure end time of all pixels in the pixel array.

This application also provides a method for image sensitivity. As shown in FIG. 17, it is a schematic flowchart of an exemplary image light-sensing method provided by an embodiment of this application. The method applies an image sensor, the sensor includes: an infrared light cut filter layer, a plurality of micro lenses and a pixel array, the pixel array includes red pixels, green pixels, blue pixels and infrared light pixels, the pixel array Each pixel corresponds to a micro lens; the infrared light cut filter layer is respectively coated on the micro lens corresponding to the red pixel, the green pixel, and the blue pixel, and the method includes:

1701. The original light in nature passes through the filter to obtain the first light;

The filter is used to filter out ultraviolet light and far-infrared light. Far-infrared light is infrared light with a longer wavelength. For example, the infrared light with a wavelength greater than the second preset wavelength mentioned in the foregoing embodiment can be referred to as far-infrared light. Light. The wavelength of the far-infrared light is greater than the wavelength of the infrared light in the specific wavelength range allowed by the subsequent infrared light filter layer. For the filter, please refer to the description of the filter on the device side, which will not be repeated here.

1702. The first light passes through the infrared light filter layer and the micro lens to reach the infrared light pixel;

1703. The first light passes through the infrared light cut filter layer, the red filter layer and the micro lens to reach the red pixel.

1704. The first light passes through the infrared light cut filter layer, the green filter layer and the micro lens to reach the green pixel.

1705. The first light reaches the blue pixel through the infrared cut filter layer, the blue filter layer and the micro lens.

It should be understood that the labels of steps 1702-1705 do not limit the execution order of the method. Steps 1702-1705 can usually be executed synchronously, or the steps may not be executed strictly synchronously, but there are some time differences between them. The embodiments of this application do not limit this.

The infrared filter layer allows only infrared light in a specific wavelength range to pass, the red filter layer is used to pass only red light and infrared light in the first wavelength range, and the green filter layer is used to pass only green light and the first wavelength range. Infrared light in the second wavelength range, the blue filter layer is used to pass only blue light and infrared light in the third wavelength range, and the infrared light cut by the infrared light cut-off filter layer includes: Infrared light, infrared light in the second wavelength range, and infrared light in the third wavelength range.

Since the red filter layer, the green filter layer and the blue filter layer pass infrared light within the wavelength range of the infrared light cut by the infrared light cut filter layer, the infrared light cut filter layer cuts off the entry into the R pixel , G pixel and B pixel infrared light, so that R pixel, G pixel and B pixel can only receive R light, G light and B light respectively.

It should be understood that step 1701 is an optional step, and the original light from nature may not pass through the filter but directly enter the filter layer and the micro lens. The infrared cut filter layer can be on top of the red filter layer, the green filter layer and the blue filter layer; the red filter layer, the green filter layer and the blue filter layer can also be on the infrared cut filter layer Above, the embodiment of the present application does not limit this.

1706. The photosensitive device in the pixel converts the light entering the pixel into electric charge;

1707. Output the accumulated charge through the charge readout module to obtain a photosensitive result.

In a possible implementation, the method further includes:

Controlling the exposure start time of the visible light pixel based on the first control line, the visible light pixel including the red pixel, the green pixel, and the blue pixel;

The exposure start time of the infrared light pixel is controlled based on the second control line.

In this method, visible light pixels and infrared light pixels can be independently exposed, which improves the light-sensing effect of the sensor.

In a possible implementation, the method further includes:

Controlling the exposure start time of the red pixel based on the first control line;

Controlling the exposure start time of the green pixel based on the second control line;

Controlling the exposure start time of the blue pixel based on the third control line;

The exposure start time of the infrared light pixel is controlled based on the fourth control line.

In this method, the four pixel components can be independently exposed, which improves the sensitivity of the sensor.

In a possible implementation manner, the method further includes: controlling the exposure time of the red pixel, the green pixel, and the blue pixel to meet a preset ratio.

In this method, each pixel can individually control the exposure time.

In a possible implementation manner, the method further includes: controlling the exposure end time of all pixels in the pixel array based on the exposure end control line.

As shown in FIG. 18, an exemplary method for independently controlling exposure time provided by this embodiment of the application is a schematic flow chart of an exemplary method for independently controlling exposure time. The method is applied to a sensor including at least two types of pixels. One type of pixel and the second type of pixel, the method includes:

1801, based on the first control unit controlling the exposure start time of the first type of pixels;

1802, based on the second control unit controlling the exposure start time of the second type of pixels.

Exemplarily, the sensor may be an RGBIR sensor. Correspondingly, the first type of pixels are visible light pixels, the visible light pixels include R, G, and B pixels, and the second type of pixels are IR pixels. The sensor may be an RGBW sensor. Correspondingly, the first type of pixels are visible light pixels, the visible light pixels include R, G, and B pixels, and the second type of pixels are W pixels. The sensor may be an RCCB sensor. Correspondingly, the first type of pixels are visible light pixels, the visible light pixels include R and B pixels, and the second type of pixels are C pixels. The first control unit and the second control unit are independent of each other, so the exposure start time of the first type pixel and the second type pixel are independently controlled. It should be understood that the first control unit and the second control unit may be implemented by a hardware logic circuit, or may be implemented by a software module running on a processor.

Exemplarily, the sensor is an RCCB sensor, the first type of pixels are R pixels, the second type of pixels are B pixels, and the third type of pixels are C pixels; the method specifically includes: controlling based on the first control unit The exposure start time of the R pixel; the exposure start time of the B pixel is controlled based on the second control unit; the exposure start time of the C pixel is controlled based on the third control unit.

In a possible implementation manner, the at least two types of pixels further include: the at least two types of pixels further include: a third type of pixels and a fourth type of pixels, and the method further includes:

Controlling the exposure start time of the third type of pixels based on the third control unit;

The fourth control unit controls the exposure start time of the fourth type of pixels.

Exemplarily, the sensor is an RGBIR sensor, the pixels of the first type are R pixels, the pixels of the second type are G pixels, the pixels of the third type are B pixels, and the pixels of the fourth type are Is an IR pixel; the method specifically includes:

Controlling the exposure start time of the R pixel based on the first control unit;

Controlling the exposure start time of the G pixel based on the second control unit;

Controlling the exposure start time of the B pixel based on the third control unit;

Control the exposure start time of the IR pixel based on the fourth control unit; or,

The sensor is an RGBW sensor, the pixels of the first type are R pixels, the pixels of the second type are G pixels, the pixels of the third type are B pixels, and the pixels of the fourth type are W pixels; The method specifically includes:

Based on the fourth control unit, the exposure start time of the W pixel is controlled.

Optionally, the method further includes:

Based on the exposure end control unit controls the exposure end time of all pixels.

Optionally, the method further includes: controlling the exposure time of each of the at least two types of pixels to meet a preset ratio.

The exposure start time of different types of pixels is independently controlled, and the exposure end time is uniformly controlled. Therefore, the exposure time between the pixels can be set to meet a preset ratio by setting the exposure start time of different pixels.

Optionally, the method further includes: transferring the charge accumulated in the photosensitive device to the charge readout module based on the charge transfer control unit.

The embodiments of the present application also provide a computer-readable storage medium. The computer-readable storage medium stores instructions that, when run on a computer or processor, cause the computer or the processor to execute any one provided in the embodiments of the present application. Part or all of the steps in the method of independent exposure control.

The embodiments of the present application also provide a computer program product containing instructions, which when run on a computer or processor, cause the computer or the processor to execute part or all of any of the independent exposure control methods provided in the embodiments of the present application step.

The above-mentioned embodiments are only used to illustrate the technical solutions of the present application, not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that it can still implement the foregoing The technical solutions described in the examples are modified, or some of the technical features are equivalently replaced; these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

An image sensor, characterized in that the sensor includes: a filter layer, a plurality of micro lenses and a pixel array, the pixel array includes red pixels, green pixels, blue pixels and infrared light pixels, each pixel corresponds to one Micro lens

The filter layer includes an infrared light cut filter layer, and the micro lenses corresponding to the red pixels, the green pixels, and the blue pixels are respectively coated with the infrared light cut filter layer, and the infrared light cuts off The filter layer is used to cut off optical signals with a wavelength greater than a first preset wavelength, and the optical signals with a wavelength greater than the first preset wavelength include infrared light.
The sensor according to claim 1, wherein the filter layer further comprises a red filter layer, a green filter layer, a blue filter layer and an infrared filter layer;

The infrared light filter layer is coated on the micro lens corresponding to the infrared light pixel, and the infrared light filter layer can pass infrared light in a specific wavelength range;

The micro lens corresponding to the red pixel is further coated with the red filter layer, the micro lens corresponding to the green pixel is further coated with the green filter layer, and the micro lens corresponding to the blue pixel is also coated Covering the blue filter layer;

The red filter layer can only pass red light and infrared light in the first wavelength range, the green filter layer can only pass green light and infrared light in the second wavelength range, and the blue filter layer only It can pass blue light and infrared light in the third wavelength range. The infrared light in the first wavelength range, the infrared light in the second wavelength range, and the infrared light in the third wavelength range have wavelengths greater than The first preset wavelength.
The sensor according to claim 2, wherein the sensor further comprises a filter for filtering ultraviolet light and infrared light with a wavelength greater than a second preset wavelength, and the second preset Set the wavelength to be greater than any one of the first preset wavelength and the specific wavelength range;

Light passes through the filter, the filter layer and the micro lens in sequence to reach the pixel array.
The sensor according to any one of claims 1 to 3, wherein the sensor further comprises a charge readout module, and each pixel in the pixel array comprises a photosensitive device;

The photosensitive device is used to convert light into electric charge;

The charge readout module outputs the charge accumulated by the photosensitive device to obtain a photosensitive result.
The sensor according to any one of claims 1 to 4, further comprising: a logic control circuit for independently controlling the exposure time of the visible light pixels and the infrared light pixels, the visible light pixels including the red Pixels, the green pixels, and the blue pixels.
The sensor according to claim 5, wherein the logic control circuit comprises: a first control line and a second control line, the visible light pixels in the pixel array are coupled to the first control line, and the pixel The infrared light pixels in the array are coupled to the second control line; the logic control circuit is specifically used for:

Controlling the exposure start time of the visible light pixel based on the first control line;

The exposure start time of the infrared light pixel is controlled based on the second control line.
The sensor according to claim 6, wherein:

The logic control circuit is further configured to control the exposure time of the visible light pixel and the infrared light pixel to satisfy a preset ratio based on the first control line and the second control line.
The sensor according to any one of claims 1 to 4, further comprising: a logic control circuit for independently controlling the red pixel, the green pixel, the blue pixel and the infrared light. The exposure time of the pixel.
The sensor according to claim 8, wherein the logic control circuit comprises: a first control line, a second control line, a third control line, and a fourth control line, and the red pixels in the pixel array are coupled to The first control line, the green pixels in the pixel array are coupled to the second control line, the blue pixels in the pixel array are coupled to the third control line, and the infrared light in the pixel array The pixel is coupled to the fourth control line; the logic control circuit is specifically configured to:

Controlling the exposure start time of the red pixel based on the first control line;

Controlling the exposure start time of the green pixels based on the second control line;

Controlling the exposure start time of the blue pixel based on the third control line;

The exposure start time of the infrared light pixel is controlled based on the fourth control line.
The sensor according to claim 9, wherein:

The logic control circuit is also used to control the red pixel, the green pixel, and the red pixel based on the first control line, the second control line, the third control line, and the fourth control line. The exposure time of the blue pixel and the infrared light pixel meets a preset ratio.
The sensor according to any one of claims 1 to 4, further comprising: a row coordinate control line, a column coordinate control line, and an exposure start control line; each pixel in the pixel array is coupled to a respective row A coordinate control line and a column coordinate control line, the exposure start control line includes a plurality of branches, and each branch corresponds to a pixel;

When the control signals output by the row coordinate control line and the column coordinate control line of the target pixel are both effective levels, the branch output control signal of the exposure start control line corresponding to the target pixel controls the target pixel The target pixel is any pixel in the pixel array.
An image light-sensing method, characterized in that the method is applied to an image sensor, the sensor includes: an infrared light cut filter layer, a plurality of micro lenses and a pixel array, the pixel array includes red pixels, green pixels, Blue pixels and infrared light pixels, each pixel in the pixel array corresponds to a micro lens; the infrared light cut filter layer is respectively coated on the red pixel, the green pixel and the blue pixel corresponding On the micro lens, the method includes:

Light passes through the infrared light cut filter layer and the micro lens to reach the red pixel, the green pixel, and the blue pixel;

Wherein, the infrared light cutoff filter layer is used to cut off infrared light so that infrared light cannot enter the red pixel, the green pixel, and the blue pixel.
The method of claim 12, wherein the sensor further comprises a red filter layer, a green filter layer, a blue filter layer, and an infrared filter layer; the red filter layer is coated on the On the micro lens corresponding to the red pixel, the green filter layer is coated on the micro lens corresponding to the green pixel, and the blue filter layer is coated on the micro lens corresponding to the blue pixel, so The infrared light filter layer is coated on the micro lens corresponding to the infrared light pixel, and the method specifically includes:

The light sequentially passes through the infrared light filter layer and the micro lens to reach the infrared light pixel;

The light sequentially passes through the infrared light cut filter layer, the red filter layer and the micro lens to reach the red pixel;

The light sequentially passes through the infrared light cut filter layer, the green filter layer and the micro lens to reach the green pixels;

The light sequentially passes through the infrared light cut filter layer, the blue filter layer and the micro lens to reach the blue pixels; or,

The light sequentially passes through the red filter layer, the infrared light cut filter layer and the micro lens to reach the red pixel;

The light sequentially passes through the green filter layer, the infrared light cut filter layer and the micro lens to reach the green pixels;

The light sequentially passes through the blue filter layer, the infrared light cut filter layer and the micro lens to reach the blue pixel;

Wherein, the infrared filter layer is used to pass only infrared light in a specific wavelength range, the red filter layer is used to pass only red light and infrared light in the first wavelength range, and the green filter layer is used To pass only green light and infrared light in the second wavelength range, the blue filter layer is used to pass only blue light and infrared light in the third wavelength range;

The infrared light cut off by the infrared light cut filter layer includes: infrared light in the first wavelength range, infrared light in the second wavelength range, and infrared light in the third wavelength range.
The method according to claim 12 or 13, wherein the sensor further comprises a filter, and the light is the original light in nature after passing through the filter, and the filter is used to filter In addition to ultraviolet light and far-infrared light, the wavelength of the far-infrared light is greater than the wavelength of the infrared light in the specific wavelength range that the infrared light filter layer allows to pass, in the method of claim 12 or 13 Before, the method also includes:

The original light from the natural world passes through the filter to obtain the light.
The method according to any one of claims 12 to 14, wherein the sensor further comprises a charge readout module, each pixel in the pixel array comprises a photosensitive device, and the method further comprises:

The photosensitive device converts light into electric charge;

The accumulated charge is output through the charge readout module to obtain the photosensitive result.
The method according to any one of claims 12 to 15, further comprising:

Controlling the exposure start time of visible light pixels based on the first control line, the visible light pixels including the red pixels, the green pixels, and the blue pixels;

The exposure start time of the infrared light pixel is controlled based on the second control line.
The method of claim 16, wherein the method further comprises:

Controlling the exposure time of the visible light pixel and the infrared light pixel to satisfy a preset ratio based on the first control line and the second control line.
The method according to any one of claims 12 to 15, further comprising:

Controlling the exposure start time of the red pixel based on the first control line;

Controlling the exposure start time of the green pixels based on the second control line;

Controlling the exposure start time of the blue pixel based on the third control line;

The exposure start time of the infrared light pixel is controlled based on the fourth control line.
The method according to any one of claims 12 to 15, wherein each pixel in the sensor is coupled to its own row coordinate control line and column coordinate control line, and each pixel corresponds to an exposure start control A branch of the line, the method further includes:

When the control signals output by the row coordinate control line and the column coordinate control line of the target pixel are both effective levels, the branch of the exposure start control line corresponding to the target pixel outputs a control signal based on the The control signal controls the exposure start time of the target pixel, and the target pixel is any pixel in the pixel array.
An independent exposure device, characterized in that the device comprises: at least two control units, each of the at least two control units is used to correspondingly control one type of pixel in the pixel array of the sensor The pixel array of the sensor includes at least two types of pixels.
The device according to claim 20, further comprising: the pixel array.
The device according to claim 20 or 21, wherein the sensor is an RGBIR sensor,

The at least two types of pixels include: visible light pixels and IR pixels, the visible light pixels include: R pixels, G pixels, and B pixels, said;

Alternatively, the at least two types of pixels include: R pixels, B pixels, G pixels, and IR pixels, and the at least two control units include: a first control unit, a second control unit, a third control unit, and a fourth control unit. control unit;

The first control unit is used to control the exposure start time of the R pixel;

The second control unit is used to control the exposure start time of the G pixel;

The third control unit is used to control the exposure start time of the B pixel;

The fourth control unit is used to control the exposure start time of the IR pixel.
The device according to claim 20 or 21, wherein the sensor is an RGBW sensor,

The at least two types of pixels include: visible light pixels and W pixels, the visible light pixels include: R pixels, G pixels, and B pixels, and the at least two control units include: a first control unit and a second control unit;

The first control unit is used to control the exposure start time of the visible light pixel;

The second control unit is used to control the exposure start time of the W pixel; or

The at least two types of pixels include: R pixels, B pixels, G pixels, and W pixels, and the at least two control units include: a first control unit, a second control unit, a third control unit, and a fourth control unit ；

The first control unit is used to control the exposure start time of the R pixel;

The second control unit is used to control the exposure start time of the G pixel;

The third control unit is used to control the exposure start time of the B pixel;

The fourth control unit is used to control the exposure start time of the W pixel.
The device according to claim 20 or 21, wherein the sensor is an RCCB sensor,

The at least two types of pixels include: visible light pixels and C pixels, the visible light pixels include: R pixels and B pixels, and the at least two control units include: a first control unit and a second control unit;

The first control unit is used to control the exposure start time of the visible light pixel;

The second control unit is used to control the exposure start time of the C pixel;

The at least two types of pixels include: R pixels, B pixels, and C pixels, and the at least two control units include: a first control unit, a second control unit, and a third control unit;

The first control unit is used to control the exposure start time of the R pixel;

The second control unit is used to control the exposure start time of the B pixel;

The third control unit is used to control the exposure start time of the C pixel.
The device according to any one of claims 20 to 24, wherein the at least two control units control the exposure time of the at least two types of pixels to meet a preset ratio.
The device according to any one of claims 20 to 25, further comprising: an exposure end control unit, configured to uniformly control the exposure end time of all pixels in the pixel array.
An independent exposure method, characterized in that the method is applied to a sensor including at least two types of pixels, the at least two types of pixels include a first type of pixel and a second type of pixel, so The methods include:

Controlling the exposure start time of the first type of pixels based on the first control unit;

The second control unit controls the exposure start time of the second type of pixels.
The method of claim 27, wherein the method further comprises:

The exposure time of each of the at least two types of pixels is controlled to meet a preset ratio.
The method according to claim 27 or 28, wherein the sensor is an RGBIR sensor, the first type of pixels are visible light pixels, the second type of pixels are IR pixels, and the visible light pixels are Including R pixels, G pixels and B pixels; or,

The sensor is an RGBW sensor, the pixels of the first type are visible light pixels, the pixels of the second type are W pixels, and the visible light pixels include R pixels, G pixels, and B pixels;

The sensor is an RCCB sensor, the pixels of the first type are visible light pixels, the pixels of the second type are C pixels, and the visible light pixels include R pixels and B pixels.
The method according to claim 27 or 28, wherein the at least two types of pixels further comprise: a third type of pixels; and the method further comprises:

The third control unit controls the exposure start time of the third type of pixels.
The method of claim 30, wherein the sensor is an RCCB sensor, the first type of pixels are R pixels, the second type of pixels are B pixels, and the third type The pixel of is C pixel; the method specifically includes:

Controlling the exposure start time of the R pixel based on the first control unit;

Controlling the exposure start time of the B pixel based on the second control unit;

The exposure start time of the C pixel is controlled based on the third control unit.
The method according to claim 27 or 28, wherein the at least two types of pixels further comprise: a third type of pixels and a fourth type of pixels, and the method further comprises:

Controlling the exposure start time of the third type of pixels based on the third control unit;

The fourth control unit controls the exposure start time of the fourth type of pixels.
The method of claim 32, wherein the sensor is an RGBIR sensor, the first type of pixels are R pixels, the second type of pixels are G pixels, and the third type The pixels of are B pixels, and the fourth type of pixels are IR pixels; the method specifically includes:

Controlling the exposure start time of the R pixel based on the first control unit;

Controlling the exposure start time of the G pixel based on the second control unit;

Controlling the exposure start time of the B pixel based on the third control unit;

Controlling the exposure start time of the IR pixel based on the fourth control unit; or,

The sensor is an RGBW sensor, the pixels of the first type are R pixels, the pixels of the second type are G pixels, the pixels of the third type are B pixels, and the pixels of the fourth type are The pixels are W pixels; the method specifically includes:

Controlling the exposure start time of the R pixel based on the first control unit;

Controlling the exposure start time of the G pixel based on the second control unit;

Controlling the exposure start time of the B pixel based on the third control unit;

The exposure start time of the W pixel is controlled based on the fourth control unit.
The method according to claims 27 to 33, further comprising: uniformly controlling the exposure end time of all pixels in the pixel array based on the exposure end control unit.
A computer-readable storage medium, the computer-readable storage medium stores instructions, when the instructions run on a computer or a processor, the computer or the processor executes any one of claims 27-34 The method described.
A computer program product containing instructions, when it runs on a computer or processor, causes the computer or processor to execute the method according to any one of claims 27-34.