WO2021036850A1

WO2021036850A1 - Image sensor, image processing method, and storage medium

Info

Publication number: WO2021036850A1
Application number: PCT/CN2020/109616
Authority: WO
Inventors: 杨鑫
Original assignee: Oppo广东移动通信有限公司
Priority date: 2019-08-26
Filing date: 2020-08-17
Publication date: 2021-03-04
Also published as: CN113709382A; CN110460781B; CN110460781A

Abstract

Disclosed is an image sensor. The image sensor comprises a plurality of pixel units, a plurality of exposure control modules, and an image processing module; the plurality of pixel units comprise three different sizes of photodiode (PD) columns used for absorbing RGB monochromatic light, each pixel unit in the plurality of pixel units comprises at least two rows of PD columns, and the plurality of pixel units are arranged according to a preset mode to form a plurality of rows of PD columns; in the plurality of exposure control modules, each exposure control module is correspondingly connected to one row of PD columns in the plurality of rows of PD columns; the plurality of exposure control modules are used for controlling the plurality of rows of PD columns to perform exposure according to a cross exposure mode of long exposure of one row of PD columns and short exposure of one row of PD columns so as to obtain a long exposure electric signal and a short exposure electric signal; and the image processing module is connected to the plurality of rows of PD columns and is used for generating a high dynamic range image by using the long exposure electric signal and the short exposure electric signal.

Description

Image sensor, image processing method and storage medium

Cross-references to related applications

This application is filed based on the Chinese patent application with the application number 201910791077.8, the filing date is August 26, 2019, and the application title is "an image sensor, image processing method and storage medium", and the priority of the Chinese patent application is claimed. The entire content of the Chinese patent application is hereby incorporated into this application by way of introduction.

Technical field

The embodiments of the present application relate to the field of image processing technologies, and in particular, to an image sensor, an image processing method, and a storage medium.

Background technique

The image generated by the high dynamic range (HDR) technology has rich layers and strong sense of realism, and can truly restore the light and shadow effects that are close to the real scene.

In the prior art, three different exposure times of long exposure, medium exposure, and short exposure can be used to synthesize a high dynamic range image by using a complementary metal oxide semiconductor (Complementary Metal Oxide Semiconductor, CMOS) image sensor combined with an image processor. As shown in Figure 1, for the triple exposure scheme, for each color light of RGB, a corresponding pixel structure composed of four pixel units needs to be set to perform exposure according to different exposure times, where L represents long exposure and M Means medium exposure, S means short exposure. In addition, a Digital Overlap (DOL) HDR solution can also be used to synthesize a high dynamic range image with images obtained multiple times.

However, in the above solution, in the image generation process, all the pixel units are actually divided into three parts, and the corresponding images are generated by different exposures. Therefore, the resolution of the obtained image is low. For the DOL HDR solution, the frame rate of the image sensor will be lost.

Summary of the invention

The embodiments of the present application provide an image sensor, an image processing method, and a storage medium. For each pixel unit, part of the PD column is subjected to long exposure, and part of the PD column is subjected to short exposure, so as to obtain a long exposure image and a short exposure image, and according to the long exposure The high dynamic range image generated by the image and the short exposure image not only has a higher resolution, but also avoids loss of frame rate.

The technical solutions of the embodiments of the present application are implemented as follows:

In a first aspect, an embodiment of the present application provides an image sensor, the image sensor including: a plurality of pixel units, a plurality of exposure control modules, and an image processing module;

The plurality of pixel units includes three different sizes of photodiode PD columns for absorbing RGB monochromatic light, each pixel unit of the plurality of pixel units includes at least two rows of PD columns, and the plurality of pixel units Arrange according to the preset method to form multiple rows of PD columns;

In the plurality of exposure control modules, each exposure control module is correspondingly connected to one row of PD columns in the plurality of rows of PD columns;

The multiple exposure control modules are configured to control the multiple rows of PD columns to perform exposure in a cross-exposure mode of long exposure of one row of PD columns and short exposure of one row of PD columns to obtain long-exposure electrical signals and short-exposure electrical signals;

The image processing module is connected to the multiple rows of PD columns, and is configured to use the long exposure electrical signal and the short exposure electrical signal to generate a high dynamic range image.

In the above image sensor, the three different sizes of PD columns include a first size PD column for absorbing red light in the RGB monochromatic light, a second size PD column for blue light, and a third size PD column for green light. column;

The plurality of pixel units include a first type pixel unit and a second type pixel unit;

In the first type of pixel unit, each row of PD columns includes one PD column of the first size and one PD column of the second size;

In the second type of pixel unit, each row of PD columns includes two PD columns of the third size.

In the above image sensor, a first color filter is provided on the absorption side of the first type pixel unit for passing the red light and the blue light;

The absorption side of the second type pixel unit is provided with a second color filter for passing the green light.

In the above image sensor, the multiple rows of PD columns include a first PD column, the first PD column is any one of the multiple rows of PD columns, and the image sensor further includes The first readout circuit corresponding to the column;

The first PD column is connected to the first exposure control module through the first readout circuit;

The first exposure control module is used for controlling the first PD column among the plurality of exposure control modules, and the PD columns in the same row as the first PD columns among the rows of PD columns perform long exposure Or a short exposure module.

In the above image sensor, the first readout circuit includes a transfer transistor connected to the first PD column, a readout area connected to the transfer transistor, an amplifier tube connected to the readout area, and A reset transistor connected to the readout area and the amplifying tube;

The first exposure control module is connected to the transfer transistor and the reset transistor to control the first PD column to perform long exposure or short exposure, so that the first PD column can obtain a first electrical signal; An electrical signal is the long exposure electrical signal or a part of the short exposure electrical signal.

In the above image sensor, the transfer transistor is used to transfer the first electrical signal from the first PD column to the readout area, so as to read the first electrical signal from the readout area. signal;

The amplifying tube is used to amplify the first electrical signal in the readout area;

The readout area is also used to read the reset level in the reset transistor;

The amplifying tube is also used to amplify the reset level.

In the above image sensor, the image processing module is configured to use the long exposure electrical signal to generate a long exposure image, and use the short exposure electrical signal to generate a short exposure image; The image is aligned and fused to generate the high dynamic range image.

In a second aspect, an embodiment of the present application provides an image processing method, which is applied to the above-mentioned image sensor, and the method includes:

Through multiple exposure control modules, the multiple rows of photodiode PD columns formed by multiple pixel units are controlled, and the exposure is carried out according to the cross-exposure method of long exposure of one row of PD columns and short exposure of one row of PD columns to obtain long-exposure electrical signals and short-exposure electrical signals. signal;

Through the image processing module, the long exposure electrical signal and the short exposure electrical signal are used to generate a high dynamic range image.

In the above-mentioned image processing method, the step of using the long exposure electrical signal and the short exposure electrical signal to generate a high dynamic range image through the image processing module includes:

Using the long-exposure electrical signal to generate a long-exposure image through the image processing module, and use the short-exposure electrical signal to generate a short-exposure image;

Through the image processing module, the long-exposure image and the short-exposure image are aligned and fused to generate the high dynamic range image.

In a third aspect, an embodiment of the present application provides a computer-readable storage medium on which a computer program is stored, which is applied to an image sensor, and when the computer program is executed by a processor, the foregoing image processing method is implemented.

The embodiment of the application provides an image sensor, including: a plurality of pixel units, a plurality of exposure control modules, and an image processing module; a plurality of pixel units, including three different sizes of photodiodes PD for absorbing RGB monochromatic light Each pixel unit in the plurality of pixel units includes at least two rows of PD columns, and the plurality of pixel units are arranged in a preset manner to form multiple rows of PD columns; among the multiple exposure control modules, each exposure control module is associated with multiple rows One row of PD columns in the PD column is correspondingly connected; multiple exposure control modules are used to control multiple rows of PD columns to expose according to the cross exposure mode of one row of PD columns and short exposure of one row of PD columns to obtain long exposure electrical signals and short exposure electrical signals Signal: The image processing module is connected with multiple rows of PD columns to use long exposure electrical signals and short exposure electrical signals to generate high dynamic range images. The image sensor proposed in the embodiment of the present application performs long exposure on part of the PD column and short exposure on part of the PD column of each pixel unit, thereby obtaining a long exposure image and a short exposure image, and generates high exposure images based on the long exposure image and the short exposure image. Dynamic range images not only have higher resolution, but also avoid loss of frame rate.

Description of the drawings

FIG. 1 is a schematic diagram of triple exposure of a pixel unit proposed in the prior art;

FIG. 2 is a schematic structural diagram of an image sensor provided by an embodiment of the application;

FIG. 3 is a schematic diagram of a pixel unit arrangement provided by an embodiment of the application;

4 is a schematic structural diagram of a first readout circuit provided by an embodiment of the application;

FIG. 5 is a schematic diagram of metal wiring of a multi-row PD column provided by an embodiment of the application;

6 is a schematic diagram of exposure of a multi-row PD column provided by an embodiment of the application;

FIG. 7 is a schematic flowchart of an image processing method provided by an embodiment of the application.

detailed description

The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. It can be understood that the specific embodiments described here are only used to explain the related application, but not to limit the application. In addition, it should be noted that, for ease of description, only the parts related to the relevant application are shown in the drawings.

Example one

The embodiment of the present application provides an image sensor. FIG. 2 is a schematic structural diagram of an image sensor provided by an embodiment of the application. As shown in Figure 2, the image sensor includes: multiple pixel units, multiple exposure control modules, and image processing modules;

Multiple pixel units, including three different sizes of photodiode PD columns for absorbing RGB monochromatic light, each pixel unit of the multiple pixel units includes at least two rows of PD columns, and the multiple pixel units are arranged in a preset manner , Forming multiple rows of PD columns;

Among the multiple exposure control modules, each exposure control module is correspondingly connected to one row of PD columns in the multiple rows of PD columns;

Multiple exposure control modules for controlling multiple rows of PD columns to perform exposure in a cross-exposure mode of one row of PD columns long exposure and one row of PD columns short exposure to obtain long exposure electrical signals and short exposure electrical signals;

The image processing module is connected with multiple rows of PD columns, and is used to generate a high dynamic range image by using the long exposure electrical signal and the short exposure electrical signal.

It should be noted that, in the embodiment of the present application, the image sensor may specifically be a complementary metal oxide semiconductor (Complementary Metal Oxide Semiconductor, CMOS) image sensor.

It should be noted that, in the embodiment of the present application, the plurality of pixel units include PD columns of three different sizes for absorbing RGB monochromatic light. The PD columns are photodiodes of the hundred nanometer level, and one pixel unit can Including multiple PD pillars, instead of a traditional pixel corresponding to a PD structure with a thickness of approximately 2um or more. The PD column exposure can absorb the corresponding light signal, and perform photoelectric conversion of the light signal to obtain an electrical signal.

Specifically, in the embodiment of the present application, the three different sizes of PD columns include a first size PD column for absorbing red light in RGB monochromatic light, a second size PD column for blue light, and a third size for green light. PD column;

The plurality of pixel units includes a first type pixel unit and a second type pixel unit;

In the first type of pixel unit, each row of PD columns includes a PD column of a first size and a PD column of a second size;

In the second type of pixel unit, each row of PD columns includes two third-size PD columns.

It should be noted that, in the embodiment of the present application, the diameter of the PD column of the first size is specifically 120nm, which can be used to absorb red light, the diameter of the PD column of the second size is specifically 60nm, which can be used to absorb blue light, and the third The diameter of the size PD column is specifically 90 nm, which can be used to absorb green light.

It should be noted that, in the embodiment of the present application, the three sizes of PD columns are determined based on the resonant wavelength of RGB monochromatic light and the refractive index of the optical signal, respectively.

It should be noted that in the embodiments of the present application, the diameters of the PD columns of three different sizes are determined based on the resonant wavelengths of red, green, and blue monochromatic light and the refractive index of the corresponding optical signal, or obtained by optical simulation Yes, the specific selection is made according to the actual situation, and the embodiments of this application do not make specific limitations.

In the embodiment of the present application, formula (1) is used to determine the size of the PD column

The size of the PD column = (resonance wavelength-preset constant)/refractive index (1)

Exemplarily, the diameter of the PD column that absorbs blue light is about 60 nm; the diameter of the PD column that absorbs green light is 90 nm; the diameter of the PD column that absorbs red light is 120 nm.

Optionally, in the embodiment of the present application, the shapes of the PD columns of three different sizes include at least a rectangle, a circle, a parallelogram, and a rhombus. The specific shape of the PD column is selected according to the actual situation, and the embodiment of the present application does not do it. Specific restrictions.

It should be noted that in the embodiments of the present application, the specific number of pixel units is not limited in the embodiments of the present application. In addition, the plurality of pixel units actually include two types of pixel units, namely, the first type of pixel unit and the second type of pixel unit. Type pixel unit, the pixel unit array is actually an array arranged using the first type pixel unit and the second type pixel unit.

It can be understood that, in the embodiment of the present application, in the first type of pixel unit, each row includes a PD column of a first size and a PD column of a second size, so that it can be exposed with a row of PD column length, and a row of PD column When the short-exposure cross-exposure method is used for exposure, both the red photoelectric signal and the blue electrical signal with a longer exposure time can be obtained, and the red photoelectric signal and the blue electrical signal with a shorter exposure time can also be obtained. In the second type of pixel unit, each row includes two PD columns of the third size, so that when the exposure is performed in the cross-exposure mode of long exposure of one row of PD columns and short exposure of one row of PD columns, a longer exposure time and a longer exposure time can be obtained. Short green photoelectric signal. The embodiments of this application are not limited.

FIG. 3 is a schematic diagram of a pixel unit arrangement provided by an embodiment of the application. As shown in FIG. 3, the pixel unit of the first type specifically includes four PD columns, where two PD columns are PD columns of a first size, and two PD columns are PD columns of a second size, and each row includes a first PD column. Size PD column and a second size PD column. The second type pixel unit includes four third-size PD columns arranged in two rows. Two pixel units of the first type and two pixel units of the second type are arranged in an intersecting manner to form 4 rows of PD columns, with 4 PD columns in each row. It should be noted that FIG. 3 is only an exemplary arrangement of multiple pixel units, and the arrangement of pixel units can also be determined according to actual needs. The specific preset arrangement of multiple pixel units is implemented in this application. The examples are not limited.

It should be noted that in the embodiment of the present application, the absorption side of the first type pixel unit is provided with a first color filter for passing red and blue light; the absorption side of the second type pixel unit is provided with a second filter. Color chip, used to pass green light.

Specifically, in the embodiment of the present application, the multiple rows of PD columns include a first PD column, the first PD column is any one of the multiple rows of PD columns, and the image sensor further includes a first PD column corresponding to the first PD column. Readout circuit

The first exposure control module is a module for controlling the first PD column and the PD columns in the same row as the first PD column in the multiple rows of PD columns to perform long exposure or short exposure.

It should be noted that, in the embodiment of the present application, the first exposure control module can control the first PD column, and the PD columns in the same row as the first PD column among the rows of PD columns to perform long exposure or short exposure. . Among the multiple rows of PD columns, when the adjacent row of PD columns in the row where the first PD column is located performs long exposure, the first exposure control module actually controls the PD column in the row where the first PD column is located to perform short exposure. When the PD column in the adjacent row of a PD column is short-exposed, the first exposure control module actually controls the PD column in the row of the first PD column to perform the long exposure.

It should be noted that, in the embodiment of the present application, each PD column corresponds to a readout circuit to read out electrical signals and connect the exposure control module corresponding to one row.

FIG. 4 is a schematic structural diagram of a first readout circuit provided by an embodiment of the application. As shown in FIG. 4, the first readout circuit includes a transfer transistor connected to the first PD column, a readout area connected to the transfer transistor, an amplifier tube connected to the readout area, and an amplifier tube connected to the readout area and the amplifier tube. Reset transistor.

It should be noted that, in the embodiment of the present application, the first exposure control module is connected to the transfer transistor and the reset transistor to control the first PD column to perform long exposure or short exposure, so that the first PD column obtains the first electrical signal; The first electrical signal is a long exposure electrical signal or a part of the short exposure electrical signal.

It is understandable that, in the embodiment of the present application, if the first exposure control module is controlling the first PD column to perform long exposure, the first electrical signal obtained by the first PD column is part of the long exposure electrical signal. Of course, if the first exposure control module is controlling the first PD column to perform the short exposure, the first electrical signal obtained by the first PD column is part of the electrical signal in the short exposure electrical signal.

It should be noted that, in the embodiment of the present application, the first PD column needs to read the first electrical signal through the corresponding first readout circuit after exposure.

Specifically, in the embodiment of the present application, the transfer transistor is used to transfer the first electrical signal from the first PD column to the readout area, so as to read the first electrical signal from the readout area;

Amplifying tube, used to amplify the first electrical signal in the readout area;

The readout area is also used to read the reset level in the reset transistor;

The amplifier tube is also used to amplify the reset level.

It should be noted that in the embodiment of the present application, the source of the transfer transistor is connected to the n region of the target pixel unit, the drain of the transfer transistor is connected to the readout region, and the first PD column focuses the first electrical signal to the transfer transistor. The n+ area is transferred to the readout area through the transfer transistor.

It should be noted that, in the embodiment of the present application, the light undergoes photoelectric conversion in the depletion region, and the absorbed light signal is converted into a first electrical signal, and then the transfer transistor gathers the first electrical signal into the n+ region of the transfer transistor In the channel; and transfer the target electrical signal in the channel of the n+ zone to the readout zone.

It should be noted that in the embodiment of the present application, the source of the reset transistor is connected to the power supply; the drain of the reset transistor is connected to the readout area, where the reset level is stored in the reset transistor, and it is read out through the readout area. Reset level.

In the embodiment of the present application, the reset level is read from the reset transistor and the first electrical signal is read from the transfer transistor. After that, the reset level and the first electrical signal are amplified, and then the amplified first electrical signal and the first electrical signal are amplified. The amplified reset level is subjected to correlated double sampling, thereby reducing the noise of reading the first electrical signal.

FIG. 5 is a schematic diagram of metal wiring of a multi-row PD column provided by an embodiment of the application. As shown in Figure 5, for each row of PD columns, the transistors in the readout circuit corresponding to each PD column will be connected together through metal wiring, and the crystal reset tubes will be connected together to further reset the crystal. The tube and the crystal transfer tube are connected with an exposure control module that controls the exposure time of the row of PD columns, that is, a row of PD columns is connected to an exposure control module. In addition, for each column of PD columns, the gate tubes of the readout circuit corresponding to each PD column can be connected together through metal traces to control the readout sequence of the electrical signals of each PD column.

FIG. 6 is a schematic diagram of exposure of a multi-row PD column provided by an embodiment of the application. As shown in FIG. 6, the exposure mode corresponding to the multiple rows of PD columns formed by the arrangement of the pixel units shown in FIG. 3, where the first row of PD columns are exposed for long, the second row of PD columns are exposed for short, and the third row of PD columns is exposed Long exposure, short exposure for the fourth row of PD columns.

Specifically, in the embodiment of the present application, the image processing module is used to generate a long exposure image using a long exposure electrical signal, and use the short exposure electrical signal to generate a short exposure image; perform alignment and fusion processing on the long exposure image and the short exposure image , Generate high dynamic range images.

It should be noted that in the embodiment of the present application, the image processing module is actually connected to the output terminal of the readout circuit corresponding to each PD column in the multiple rows of PD columns, so as to obtain the long exposure electrical signal and the short exposure electrical signal .

It is understandable that, in the embodiment of the present application, the exposure is carried out according to the cross-exposure method of one line of PD column long exposure and one line of PD column short exposure, in fact, two types of each color light in RGB monochromatic light are obtained. Exposure electrical signal, that is, the electrical signal under long exposure and short exposure processing. Therefore, the electrical signals under the long exposure processing of multiple rows of PD columns constitute the long exposure electrical signals. Correspondingly, the electrical signals under the short exposure processing of multiple rows of PD columns constitute the short exposure electrical signals, the long exposure electrical signals and the short exposure electrical signals. Both include the electrical signal after absorbing each color light in RGB monochromatic light and photoelectrically converted. The only difference is that the exposure time generated by the electrical signal is different.

Exemplarily, in the embodiment of the present application, as shown in FIG. 6, the image processing module may use the long exposure electrical signals obtained by the first row and the third row of PD columns to generate a long exposure image, and use the second row and the third row to generate a long exposure image. The short exposure electrical signal obtained by the four rows of PD columns generates the short exposure electrical signal.

It is understandable that, in the embodiment of the present application, since some areas of the long exposure image generated by the long exposure electrical signal will be overexposed, the image processing module aligns the long exposure image and the short exposure image. Fusion processing, that is, the brightness of the short-exposure image is aligned with the area of the long-exposure image without overexposure, and the part of the short-exposure image is replaced with the exposed area in the long-exposure image, thereby generating high dynamic range image.

Example two

The embodiment of the present application provides an image processing method, which is applied to the image sensor described in the first embodiment. FIG. 7 is a schematic flowchart of an image processing method provided by an embodiment of the application. As shown in Figure 7, it mainly includes the following steps:

S701. Through multiple exposure control modules, control multiple rows of photodiode PD columns formed by multiple pixel units, and perform exposure according to the cross-exposure method of long exposure of one row of PD columns and short exposure of one row of PD columns to obtain long exposure electrical signals and short exposures. Exposure to electrical signals.

In the embodiment of the present application, the image sensor includes multiple pixel units, multiple exposure control modules, and image processing modules. Multiple exposure control modules can be used to control multiple rows of PD columns to expose according to the length of one row of PD columns. The short exposure cross-exposure method is used for exposure to obtain the long exposure electrical signal and the short exposure electrical signal.

It should be noted that, in the embodiment of the present application, in the image sensor, among the multiple exposure control modules, each exposure control module is correspondingly connected to one row of PD columns in the multiple rows of PD columns, that is, one exposure control module can control multiple rows of PD columns. The exposure time of one row of PD columns in the PD column is controlled by multiple exposure control modules to control multiple rows of PD columns. The exposure is carried out according to the cross exposure method of long exposure of one row of PD columns and short exposure of one row of PD columns to obtain long exposure electrical signals and short exposures. Exposure to electrical signals.

It should be noted that in the embodiment of the present application, each pixel unit in the plurality of pixel units includes at least two rows of PD columns. Therefore, by performing PD column exposure in a cross-exposure between rows, each pixel can be exposed. Part of the unit is exposed for a long PD column, and part of the PD column is exposed for a short exposure.

It should be noted that, in the embodiment of the present application, the plurality of pixel units include a first type of pixel unit and a second type of pixel unit. In the first type of pixel unit, each row includes a PD column of a first size and a PD column of a second size, so that it can be exposed in the cross-exposure mode of long exposure of one row of PD columns and short exposure of one row of PD columns. Red photoelectric signals and blue electrical signals with longer exposure time can also be obtained with red photoelectric signals and blue electrical signals with shorter exposure time. In the second type of pixel unit, each row includes two PD pillars of the third size, so that a longer exposure time and exposure time can be obtained when exposure is performed in a cross-exposure mode of long exposure of one row of PD pillars and short exposure of one row of PD pillars. Short green photoelectric signal. The embodiments of this application are not limited.

It should be noted that, in the embodiment of the present application, since the plurality of pixel units include a first type of pixel unit and a second type of pixel unit, and the first type of pixel unit and the second type of pixel unit include three different sizes of PDs. column. Therefore, the multiple rows of PD columns actually consist of three different sizes of PD columns. Among them, the diameter of the first size PD column is 120nm, which can be used to absorb red light, and the diameter of the second size PD column is 60nm, which can be used to absorb blue light. The diameter of the third-size PD column is specifically 90nm, which can be used to absorb green light.

It is understandable that, in the embodiment of the present application, the image sensor uses multiple exposure processing modules to control multiple rows of PD columns for exposure according to the cross-exposure method of one row of PD column long exposure and one row of PD column short exposure, which actually obtains Two types of exposure electrical signals for each color light in RGB monochromatic light, namely, electrical signals under long exposure and short exposure processing. Therefore, the electrical signals under the long exposure processing of multiple rows of PD columns constitute the long exposure electrical signals. Correspondingly, the electrical signals under the short exposure processing of multiple rows of PD columns constitute the short exposure electrical signals, the long exposure electrical signals and the short exposure electrical signals. Both include electrical signals after absorbing and photoelectrically converted for each color of RGB monochromatic light. The only difference lies in the exposure time generated by the electrical signals.

S702. Using the long-exposure electrical signal and the short-exposure electrical signal to generate a high dynamic range image through the image processing module.

In the embodiment of the present application, in the image sensor, the image processing module is connected to multiple rows of PD pillars, so that the exposure electrical signals and short exposure electrical signals obtained after exposure of the multiple rows of PD pillars can be used to generate a high dynamic range image.

Specifically, in the embodiment of the present application, the image sensor uses the long exposure electrical signal and the short exposure electrical signal to generate a high dynamic range image through the image processing module, including: using the long exposure electrical signal to generate the long exposure through the image processing module The short exposure electrical signal is used to generate the short exposure image; through the image processing module, the long exposure image and the short exposure image are aligned and fused to generate a high dynamic range image.

It should be noted that, in the embodiment of the present application, the image sensor can use the long exposure electrical signal generated by at least one row of PD columns for long exposure processing among the multiple rows of PD columns through the image processing module to generate a long exposure image. The short exposure electrical signal generated by at least one row of the PD columns undergoing short exposure processing in the PD column generates the short exposure electrical signal.

The embodiment of the present application provides an image processing method that controls multiple rows of photodiode PD columns formed by multiple pixel units through multiple exposure control modules, according to a cross-exposure method of long exposure of one row of PD columns and short exposure of one row of PD columns Perform exposure to obtain long-exposure electrical signals and short-exposure electrical signals; through the image processing module, use long-exposure electrical signals and short-exposure electrical signals to generate high dynamic range images. The technical solution proposed in the embodiments of this application controls a part of the PD column of each pixel unit to perform long exposure and a part of PD column to perform short exposure, so as to obtain a long exposure image and a short exposure image, and generate high exposure images based on the long exposure image and the short exposure image. Dynamic range images not only have higher resolution, but also avoid loss of frame rate.

The embodiment of the present application also provides a computer-readable storage medium on which a computer program is stored, which is applied to an image sensor, and when the computer program is executed by a processor, the foregoing image processing method is implemented.

The above are only specific implementations of this application, but the scope of protection of this application is not limited to this. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed in this application. , Should be covered in the scope of protection of this application. Therefore, the protection scope of this application should be subject to the protection scope of the claims.

Claims

An image sensor, the image sensor comprising: a plurality of pixel units, a plurality of exposure control modules and an image processing module;

The plurality of pixel units includes three different sizes of photodiode PD columns for absorbing RGB monochromatic light, each pixel unit of the plurality of pixel units includes at least two rows of PD columns, and the plurality of pixel units Arrange according to the preset method to form multiple rows of PD columns;

In the plurality of exposure control modules, each exposure control module is correspondingly connected to one row of PD columns in the plurality of rows of PD columns;

The multiple exposure control modules are configured to control the multiple rows of PD columns to perform exposure in a cross-exposure mode of long exposure of one row of PD columns and short exposure of one row of PD columns to obtain long-exposure electrical signals and short-exposure electrical signals;

The image processing module is connected to the multiple rows of PD columns, and is configured to use the long exposure electrical signal and the short exposure electrical signal to generate a high dynamic range image.
The image sensor according to claim 1, wherein:

The three different sizes of PD columns include a first size PD column for absorbing red light in the RGB monochromatic light, a second size PD column for blue light, and a third size PD column for green light;

The plurality of pixel units include a first type pixel unit and a second type pixel unit;

In the first type of pixel unit, each row of PD columns includes one PD column of the first size and one PD column of the second size;

In the second type of pixel unit, each row of PD columns includes two PD columns of the third size.
The image sensor according to claim 2, wherein:

The absorption side of the first type pixel unit is provided with a first color filter for passing the red light and the blue light;

The absorption side of the second type pixel unit is provided with a second color filter for passing the green light.
The image sensor according to claim 1, wherein the plurality of rows of PD pillars includes a first PD pillar, the first PD pillar is any one of the plurality of rows of PD pillars, and the image sensor further includes A first readout circuit corresponding to the first PD column;

The first PD column is connected to the first exposure control module through the first readout circuit;

The first exposure control module is used for controlling the first PD column among the plurality of exposure control modules, and the PD columns in the same row as the first PD columns among the rows of PD columns perform long exposure Or a short exposure module.
The image sensor according to claim 4, wherein:

The first readout circuit includes a transfer transistor connected to the first PD column, a readout area connected to the transfer transistor, an amplifier tube connected to the readout area, and a connection with the readout area and A reset transistor connected to the amplifying tube;

The first exposure control module is connected to the transfer transistor and the reset transistor to control the first PD column to perform long exposure or short exposure, so that the first PD column can obtain a first electrical signal; An electrical signal is the long exposure electrical signal or a part of the short exposure electrical signal.
The image sensor according to claim 5, wherein:

The transfer transistor is used to transfer the first electrical signal from the first PD column to the readout area, so as to read the first electrical signal from the readout area;

The amplifying tube is used to amplify the first electrical signal in the readout area;

The readout area is also used to read the reset level in the reset transistor;

The amplifying tube is also used to amplify the reset level.
The image sensor according to claim 1, wherein:

The image processing module is configured to use the long exposure electrical signal to generate a long exposure image, and use the short exposure electrical signal to generate a short exposure image; perform alignment and fusion processing on the long exposure image and the short exposure image, The high dynamic range image is generated.
An image processing method, applied to the image sensor according to any one of claims 1-7, the method comprising:

Through multiple exposure control modules, the multiple rows of photodiode PD columns formed by multiple pixel units are controlled, and the exposure is carried out according to the cross-exposure method of long exposure of one row of PD columns and short exposure of one row of PD columns to obtain long-exposure electrical signals and short-exposure electrical signals. signal;

Through the image processing module, the long exposure electrical signal and the short exposure electrical signal are used to generate a high dynamic range image.
8. The method according to claim 8, wherein the step of using the long exposure electrical signal and the short exposure electrical signal to generate a high dynamic range image by the image processing module comprises:

Using the long-exposure electrical signal to generate a long-exposure image through the image processing module, and use the short-exposure electrical signal to generate a short-exposure image;

Through the image processing module, the long-exposure image and the short-exposure image are aligned and fused to generate the high dynamic range image.
A computer-readable storage medium with a computer program stored thereon, applied to an image sensor, and when the computer program is executed by a processor, the method according to any one of claims 8-9 is realized.