WO2022000485A1

WO2022000485A1 - Photoelectric conversion unit, image sensor, and focusing method

Info

Publication number: WO2022000485A1
Application number: PCT/CN2020/100202
Authority: WO
Inventors: 张玮; 李顺展; 池文明; 王炳文; 王磊
Original assignee: 深圳市汇顶科技股份有限公司
Priority date: 2020-07-03
Filing date: 2020-07-03
Publication date: 2022-01-06
Also published as: CN114145009A; CN114145009B

Abstract

Provided in the present application are a photoelectric conversion unit, an image sensor, and a focusing method. The photoelectric conversion unit comprises pixel units (401) in an array distribution. Each pixel unit (401) comprises a monochromatic light pixel (402) and a blended light pixel (403). The blended light pixel (403) comprises one first lens (4031) and N photoelectric converters (4032). Each photoelectric converter is capable of receiving blended lights transmitted by different transmitting areas, outputting an electric signal, and then adjusting focus on the basis of the phase difference between the electric signals outputted by the photoelectric converters. Furthermore, the electric signals outputted by the blended light pixels are applicable in generating imaging data, thus greatly increasing the proportion accounted by the number of the blended light pixels; moreover, the blended light pixels are capable of transmitting an increased amount of light to the photoelectric converters, thus increasing the accuracy of focusing when the light intensity is low.

Description

Photoelectric conversion unit, image sensor and focusing method

technical field

The present invention relates to the technical field of image processing, and in particular, to a photoelectric conversion unit, an image sensor and a focusing method.

Background technique

The image sensor uses the photoelectric conversion function of the pixel array to convert the light image of the imaging object into an electrical signal that is proportional to the light image. Among them, the core component of the image sensor is the pixel array.

Each pixel of the pixel array has a similar structure, and usually a pixel includes a lens (On Chip Lens, OCL for short), a color filter (Color Filter), and a light-emitting diode. When an image sensor is used for image acquisition, the image sensor needs to be focused. Common focusing methods include Phase Detection Auto Focus (PDAF). The principle of phase focusing is: when the photosensitive element is not at the focal point of the lens, the two photosensitive elements receive the light from the same object at different angles, and the images have a phase difference. By calculating the phase difference, it can be determined how to adjust the lens and the photosensitive element distance between them to achieve focus. In the prior art, in order to realize the automatic focusing of the image sensor, it is usually necessary to prepare additional focusing pixels. As shown in Figure 1, a focusing pixel consists of a lens and two photodiodes that receive light at different angles of incidence. When there is a phase difference between the electrical signals generated by the two photodiodes, it means that the photodiode is not in focus of the lens, and the focal length can be adjusted according to the phase difference.

However, in order to reduce the impact of the focusing pixels on the image captured by the entire pixel array, as shown in Figure 2, the focusing pixels are usually controlled between 1% and 3% of the entire pixel array, resulting in accurate focusing when the light intensity is low. low degree.

SUMMARY OF THE INVENTION

The present application provides a photoelectric conversion unit, an image sensor and a focusing method, which solve the problem of low focusing accuracy when the illumination intensity is low due to controlling the focusing pixels to be between 1% and 3% of the entire pixel array in the prior art. technical problem.

In a first aspect, the present application provides a photoelectric conversion unit, comprising: a plurality of pixel units distributed in an array; wherein each pixel unit includes at least one mixed light pixel and at least one monochromatic light pixel;

Wherein, each monochromatic light pixel is used to convert monochromatic light into an electrical signal;

Each mixed light pixel includes a first lens and N photoelectric converters distributed in a square array, the first lens is located above the N photoelectric converters, the first lens includes N exit areas, and each photoelectric converter receives A mixed light is emitted from the emitting area directly above, and the mixed light is converted into an electrical signal, N=k×k, where k is a positive integer greater than 1.

Optionally, the electrical signal output by each mixed light pixel is used to generate the distance between the photoelectric converter and the focal point of the first lens.

Optionally, the electrical signal output by each monochromatic light pixel and the electrical signal output by each mixed light pixel are jointly used to generate image data.

Optionally, the first lens is circular.

Optionally, each mixed light pixel includes 4 photoelectric converters.

Optionally, the mixed light is white light.

Optionally, the monochromatic light pixel includes a second lens and N photoelectric converters distributed in a square array;

The second lens is located above the N photoelectric converters, the second lens includes N output regions, and each photoelectric converter receives a monochromatic light emitted from the output region located just above, and converts the monochromatic light into electrical signals.

Optionally, the mixed light pixels and the monochromatic light pixels are distributed in an array.

Optionally, each pixel unit includes one mixed light pixel and three monochromatic light pixels, and the three monochromatic light pixels are red pixels, blue pixels and green pixels in sequence.

Optionally, each pixel unit includes two mixed light pixels and two monochromatic light pixels, and the two monochromatic light pixels are any two of red pixels, blue pixels and green pixels.

Optionally, the monochromatic light pixel includes a third lens and a photoelectric converter;

Wherein, the third lens is located above the photoelectric converter, and the photoelectric converter receives the monochromatic light emitted from the exit area of the third lens, and converts the monochromatic light into electrical signals.

Optionally, each pixel unit includes at least one mixed light pixel and a plurality of monochromatic light pixel subunits distributed in an array, each monochromatic light pixel subunit includes a plurality of monochromatic light pixels distributed in an array, each Monochromatic light pixel subunits are used to convert the same monochromatic light into electrical signals.

Optionally, the arrangement area of the monochromatic light pixel subunits is any one of the area of 4 monochromatic light pixels, the area of 9 monochromatic light pixels, or the area of 16 monochromatic light pixels.

Optionally, each mixed light pixel is located at a corner of four adjacent monochromatic light pixel subunits.

Optionally, each mixed light pixel is located inside the monochromatic light pixel sub-unit.

Optionally, one partial mixed light pixel is located at the corner of four adjacent monochromatic light pixel subunits, and the remaining mixed light pixels are located inside the monochromatic light pixel subunit.

In a second aspect, the present application provides an image sensor, comprising the photoelectric conversion unit according to any one of claims 1 to 16 .

In a third aspect, the present application provides a focusing method for an image sensor. The image sensor includes a photoelectric conversion unit, and the photoelectric conversion unit includes a mixed light pixel and a monochromatic light pixel. Both the mixed light pixel and the monochromatic light pixel include a lens and a square array. Distributed photoelectric converters, methods comprising:

Get the light intensity of the external environment;

If the light intensity is less than the preset threshold, the distance between the photoelectric converter and the focal point of the lens is determined according to the multi-channel mixed photoelectric signals output by the mixed light pixels and the multi-channel monochromatic photoelectric signals output by the monochromatic light pixels.

Optionally, the distance between the photoelectric converter and the focal point of the lens is determined according to the multi-channel mixed photoelectric signal output by the mixed light pixel and the multi-channel monochromatic photoelectric signal output by the monochromatic light pixel, which specifically includes:

Extract the phase of each mixed photoelectric signal and each monochromatic photoelectric signal;

The mixed photoelectric signal and the monochromatic photoelectric signal with the same phase are superimposed to generate an enhanced electric signal;

The distance between the photoelectric converter and the focal point of the lens is determined according to the enhanced electrical signal.

Optionally, if the light intensity is greater than or equal to a preset threshold, the distance between the photoelectric converter and the focal point of the lens is determined according to the multi-channel mixed photoelectric signals output by the mixed light pixels.

The present application provides a photoelectric conversion unit, an image sensor and a focusing method. The photoelectric conversion unit includes a monochromatic light pixel and a mixed light pixel. The mixed light pixel includes a first lens and N photoelectric converters. Each photoelectric converter can receive The mixed light emitted from different emission areas is converted into an electrical signal for output, and the phase difference of the electrical signal of the photoelectric converter is calculated to adjust the focal length according to the phase difference. In addition, the electrical signals output by the monochromatic light pixels and the electrical signals output by the mixed light pixels can be used together to generate imaging data, so that the proportion of the number of mixed light pixels is greatly increased, and the mixed light pixels can emit more light into the photoelectric converter. This in turn improves focusing accuracy when the light intensity is low.

Description of drawings

In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following will briefly introduce the accompanying drawings used in the description of the embodiments or the prior art. Obviously, the accompanying drawings in the following description These are some embodiments of the present application, and for those of ordinary skill in the art, other drawings can also be obtained from these drawings without any creative effort.

1 is a schematic structural diagram of a photoelectric conversion unit provided by the prior art;

2 is a schematic diagram of the distribution of focusing pixels provided by the prior art;

3 is a schematic structural diagram of an image sensor provided by the present application;

4 is a schematic structural diagram of a pixel provided by the present application;

5 is a schematic diagram of the principle of phase focusing provided by the application;

FIG. 6 is a schematic structural diagram of a photoelectric conversion unit provided in Embodiment 1 of the present application;

7 is a top view of a pixel unit provided in Embodiment 1 of the present application;

FIG. 8 is a schematic diagram of the distribution of focusing pixels in a photoelectric conversion unit provided in Embodiment 1 of the present application;

FIG. 9 is a front view of a pixel unit provided in Embodiment 1 of the present application;

10 is a main view light path diagram of a mixed light pixel provided in Embodiment 1 of the present application;

11 is a top-view light path diagram of a mixed light pixel provided in Embodiment 1 of the present application;

12 is a top view of a pixel unit provided in Embodiment 2 of the present application;

13 is a front view of a pixel unit provided in Embodiment 2 of the present application;

14 is a top view of a 3×3 pixel unit provided in Embodiment 3 of the present application;

15 is a front view of a monochromatic light pixel provided in Embodiment 3 of the present application;

16 is a top view of another 3×3 pixel unit provided in Embodiment 3 of the present application;

FIG. 17 is a top view of another 3×3 pixel unit provided in Embodiment 3 of the present application;

18 is a top view of a 4×4 pixel unit provided in Embodiment 3 of the present application;

FIG. 19 is a top view of another 4×4 pixel unit provided in Embodiment 3 of the present application;

20 is a top view of another 4×4 pixel unit provided in Embodiment 3 of the present application;

FIG. 21 is a schematic diagram of the focusing method provided in Embodiment 4 of the present application.

detailed description

In order to make the purposes, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments It is a part of the embodiments of the present application, but not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the protection scope of the present application.

The image sensor uses the photoelectric conversion function of the pixel array to convert the light image of the imaging object into an electrical signal that is proportional to the light image. As shown in FIG. 3 , the image sensor includes a pixel array 201 (Pixel Array), a signal reading circuit 202 , a signal processor 203 , a controller 204 , an interface circuit 205 and a power supply 206 . The electrical signal output terminal of the pixel array 201 is connected to the input terminal of the signal reading circuit 202 , the control terminal of the pixel array 201 is connected to the output terminal of the controller 204 , and the output terminal of the signal reading circuit 202 is connected to the output terminal of the signal processor 203 . The input terminal is connected, and the power supply 206 is used to provide power for the signal reading circuit 202 , the signal processor 203 , the controller 204 and the interface circuit 205 .

Among them, the pixel array 201 is used to collect the light signal emitted by the imaging object, convert the light signal into an electrical signal, and reflect the light image of the imaging object through the intensity of the electrical signal. The signal reading circuit 202 is used to read the electrical signal output by each pixel. The signal processor 203 is configured to perform analog-to-digital conversion on the electrical signals output by the pixel array, and output image data of the imaging object. The interface circuit 205 is used for externally transmitting image data. The controller 204 is used for outputting a control signal, and the control signal is used for controlling each pixel in the pixel array to work together.

Among them, the core component of the image sensor is the pixel array. The structure of each pixel of the pixel array is similar. As shown in FIG. 4 , a typical pixel includes a lens 301 (On Chip Lens, OCL for short), a color filter 302 (Color Filter), and a light emitting diode 303 . The lens 301 is located above the filter, and the filter 302 is located above the light-emitting diode 303 . The light emitted by the imaging object is focused by the lens 301 , and then emitted from the output area of the lens 301 , filtered by the filter 302 , and then injected into the photosensitive element of the light-emitting diode 303 . The optical signal is converted into an electrical signal by the photosensitive element. According to the type of light that the filter can transmit, the pixels can be divided into red pixels, green pixels and blue pixels. The red pixel refers to the photosensitive element that only the red light enters the light-emitting diode after being filtered by the filter. The principle of the green pixel and the blue pixel is the same as that of the red pixel, and will not be repeated here.

Among them, the principle of the image sensor generating color image data is as follows: each pixel in the pixel array can only convert one type of optical signal into an electrical signal, and then combine the optical signals collected by other types of pixels around to perform a difference operation. Restore the image color of the area captured by the current pixel. For example, if the current pixel is a red pixel, and the red pixel can only convert the red light signal into an electrical signal, you can combine the electrical signals collected by the surrounding blue pixels or green pixels to restore the current pixel. The blue and green light intensities of the pixel determine the image color of the current pixel.

In order for the image sensor to collect clearer image data, when the image sensor is used for image acquisition, the image sensor needs to be focused. Common focusing methods include Phase Detection Auto Focus (PDAF). As shown in Figure 5, the principle of phase focusing is: the imaging surface located on the upper side receives the light emitted by the incident light located below through the lens, and the imaging surface located on the lower side receives the incident light located above through the lens. . When the imaging surface is not at the focal point of the lens, the light received from different angles will form two images with phase difference. By calculating the phase difference, it is possible to determine how to adjust the distance between the lens and the photosensitive element to achieve focusing.

The following mainly describes the problems in the prior art. In order to realize the automatic focusing of the image sensor, it is usually necessary to prepare additional focusing pixels. Continuing to refer to FIGS. 1 and 2 , the image sensor includes red pixels 101 , green pixels 102 , blue pixels 103 , and focus pixels 104 . Among them, the red pixel 101 , the green pixel 102 and the blue pixel 103 have the same structure, and are composed of a lens, a filter and a photoelectric secondary light. The focusing pixel 104 is different from the other three types of pixels, including a lens, a filter, and two photoelectric secondary lights. The two light-emitting diodes can receive light from different angles. If there is a certain phase difference between the images of the optical signals received by the two pixels, the focus is adjusted according to the phase difference. However, in order to reduce the influence of the focusing pixels on the image acquisition accuracy of the entire pixel array, the number of focusing pixels is usually controlled between 1% and 3% of the entire pixel array, resulting in low focusing accuracy when the light intensity is low.

The present application provides a photoelectric conversion unit, an image sensor and a focusing method, aiming at solving the above problems. The inventive concept of the present application is to add mixed color pixels on the basis of red pixels, green pixels and blue pixels. Construct a mixed light pixel composed of a first lens and N photoelectric converters. Each photoelectric converter can receive mixed light from different output areas. By calculating the phase difference of the mixed light of the photoelectric converter, the focal length is adjusted according to the phase difference. . Since the electrical signals output by the mixed light pixels can be used to generate imaging data, the proportion of the mixed light pixels is greatly increased, and the mixed light pixels can emit more light into the photoelectric converter, thereby improving the focus when the light intensity is low Accuracy.

FIG. 6 is a schematic structural diagram of the photoelectric conversion unit provided in the first embodiment of the present application. As shown in FIG. 6 , a photoelectric conversion unit 400 provided by the present application includes a plurality of pixel units 401 .

The bottom plate is provided with a dummy pixel area (Dummy Pixel Area) and an active pixel area (Active Pixel Area), and a plurality of pixel units 401 are arranged in an array in the active area. The arrangement interval between the two pixel units 401 is determined according to the actual situation.

As shown in FIG. 7 , each pixel unit 401 includes at least one monochromatic light pixel 402 and at least one mixed light pixel 403 . The plurality of monochromatic light pixels 402 and the plurality of mixed light pixels 403 are distributed in an array. Monochromatic light pixels 402 are adjacently arranged around each mixed light pixel 403 . Each monochromatic light pixel 402 is used for converting the monochromatic light into an electrical signal, and each mixed light pixel 403 is used for converting the mixed light into an electrical signal. The electrical signal output by each mixed light pixel is used for the distance between the photoelectric converter and the focal point of the lens, and the position of the lens is adjusted according to the above distance to realize focusing. The electrical signal output by each monochromatic light pixel and the electrical signal output by each mixed light pixel are jointly used to generate image data of the imaging object.

Among them, monochromatic light refers to the spectral color light separated from white light or sunlight by the refraction of the prism, for example: red light, orange light, yellow light, green light, blue light, indigo light, violet light and other seven colors of light. The separated spectral color light passes through the prism again and will not be decomposed into other color light. The color light that cannot be decomposed is monochromatic light, and the mixed light refers to the polychromatic light formed by mixing any two or more monochromatic lights.

In this embodiment, the monochromatic light is preferably red light (Red), green light (Green) and blue light (Blue), and the mixed light is preferably white light. Accordingly, the monochromatic light pixels 402 include red pixels, green pixels and blue pixels. Mixed light pixels are white pixels.

The number of mixed light pixels 403 and monochromatic light pixels 402 is determined according to the actual imaging requirements. Since the mixed light pixels 403 can be used for imaging, the increase in the number of mixed light pixels 403 does not affect the accuracy of the image data output by the photoelectric conversion unit. As shown in FIG. 8 , when the monochromatic light pixels include red pixels, green pixels and blue pixels, and the mixed light pixels are white pixels, the ratio of the white light pixels to the entire pixel unit can reach 25% or even higher.

Wherein, as shown in FIG. 9 , each mixed light pixel 403 includes a first lens 4031 and N photoelectric converters 4032 , the circular area represents the first lens, and the square array represents the photoelectric converter array. Continuing to refer to FIG. 7 , the N photoelectric converters are distributed 4032 in a square array. That is, N=k×k, where k is a positive integer greater than 1. In this embodiment of the present application, each mixed light pixel includes four mixed light converters 4032 and one first lens 4031 . The first lens 4031 is circular. The four hybrid light converters 4032 form a 2×2 square array, and the diameter of the first lens 4031 is the same as the side length of the square array. The hybrid light converter is preferably a light emitting diode (PD).

Continuing to refer to FIG. 9 , the first lens is located above the N photoelectric converters. The mixed light pixel 403 further includes a mixed light filter 4033, which is a white light filter if the mixed light is white light. The mixed light filter 4033 is provided between the first lens 4031 and the photoelectric converter 4032 . Each mixed light pixel 403 also includes other auxiliary elements such as: a dielectric and reflective grid around the filter, a dielectric layer between the filter and the photodiode, an isolation region between the two photodiodes, and a semiconductor substrate.

Among them, the first lens 4031 is used for focusing the incident light. The mixed light filter 4033 is used to filter the incident light and output mixed light. For example, the incident light consists of red light, green light and yellow light. The mixed light filter 4033 only allows red light and green light to pass through. The light filtered by the light filter 4033 includes red light and green light. The photoelectric converter is used to convert the incoming optical signal into an electrical signal of corresponding intensity.

As shown in FIG. 10 , the light path of the mixed light pixel is as follows: the light source illuminates the imaging object, and after being reflected by the imaging object, enters the first lens 4031 , the first lens 4031 focuses the incident light, and exits through the exit area, and passes through the The light emitted from the emitting area is filtered by the mixed light filter 4033, and then enters the photosensitive element of the photoelectric converter 4032, and the photoelectric converter 4032 converts the optical signal received by the photosensitive element into an electrical signal of corresponding intensity. That is, the mixed light pixel can convert the light image of the imaging object into an electrical signal.

The following focuses on the focusing principle of the photoelectric conversion unit provided in the first embodiment of the present application. As shown in FIG. 11 , in the mixed light pixel, since the first lens 4031 is located above the N photoelectric converters 4032, each photoelectric converter 4032 Corresponding to one outgoing area of the first lens, each photoelectric converter receives the mixed light emitted by the outgoing area located just above, and converts the mixed light into an electrical signal. When the hybrid optical converter 4032 is located at the focal point of the first lens 4031, the optical signal received by each photoelectric converter has no phase difference, and thus there is no phase difference between the electrical signals output by the hybrid optical converter. When the hybrid optical converter 4032 is not located at the focal point of the first lens 4031, there is a phase difference between the optical signals received by each photoelectric converter, and thus there is a phase difference between the electrical signals output by the hybrid optical converter. And the distance between the photoelectric converter and the focal point of the lens can be determined according to the magnitude of the phase difference.

The following describes the imaging principle of the photoelectric conversion unit provided in the first embodiment of the present application by taking the monochromatic light pixels including red pixels, green pixels and blue pixels, and the mixed light pixels being white pixels as an example: the white light source or sunlight illuminates the imaging object. superior. Red pixels convert red light in white light reflected by the imaging object into electrical signals, green pixels convert green light in white light reflected by imaging objects into electrical signals, and blue pixels convert blue light in white light reflected by imaging objects into electrical signals , the mixed light pixel converts the white light reflected by the imaged object into an electrical signal. The texture data of the imaging object is generated according to the electrical signals output by the mixed light pixels, and the color data of the imaging object can be generated by combining the output signals of the green pixels, red pixels and blue pixels to determine the color data of the imaging object. The mixed light pixel includes a plurality of photoelectric converters, each photoelectric converter outputs an electrical signal, and one of the electrical signals can be arbitrarily selected to generate texture data of the imaging object.

In the photoelectric conversion unit provided in the first embodiment of the present application, the electrical signals output by the mixed light pixels can be used to generate imaging data and can also be used for phase focusing, so that the number of mixed light pixels can be greatly increased, and the mixed light pixels can More light is emitted into the photoelectric converter, which in turn improves focusing accuracy when the light intensity is low.

The following mainly describes the structural intent of the photoelectric conversion unit provided in the second embodiment of the present application. A photoelectric conversion unit 400 provided by the present application includes a plurality of pixel units 401 . Each pixel unit 401 includes at least one monochromatic light pixel 402 and at least one mixed light pixel 403 . The plurality of monochromatic light pixels 402 and the plurality of mixed light pixels 403 are distributed in an array. Monochromatic light pixels 402 are adjacently arranged around each mixed light pixel 403 .

As shown in FIG. 12 , the difference between the photoelectric conversion unit provided in the second embodiment and the photoelectric conversion unit provided in the first embodiment is that the monochromatic light pixel 402 includes a second lens 4021 and N photoelectric converters 4022 . The N photoelectric converters 4022 are distributed in a square array. N=k×k, where k is a positive integer greater than 1. That is, the structure of the monochromatic light pixel is similar to that of the mixed light pixel.

In this embodiment of the present application, preferably, each monochromatic light pixel includes four monochromatic light converters 4022 and one second lens 4021 . The second lens 4021 is circular. The four monochromatic light converters 4022 form a 2×2 square array, and the diameter of the second lens 4021 is the same as the side length of the square array. The monochromatic light converter is preferably a light emitting diode (PD).

In this embodiment, the monochromatic light is preferably red light (Red), green light (Green) and blue light (Blue), and the mixed light is preferably white light. Accordingly, the monochromatic light pixels 402 include red pixels, green pixels and blue pixels. The mixed light pixels 403 are white pixels.

As an implementation manner of the pixel unit, each pixel unit includes one mixed light pixel and three monochromatic light pixels. The three monochromatic light pixels are red pixels, blue pixels and green pixels in sequence, and the mixed light pixels are white pixels. The red, blue, green and white pixels form a 2x2 square array. In each pixel unit, the ratio of the area occupied by all mixed light pixels to the entire pixel unit is equal to 25%.

As another implementation manner of the pixel unit, each pixel unit includes 2 mixed light pixels and 2 monochromatic light pixels. The two monochromatic light pixels are any two of red pixels, blue pixels and green pixels, and the mixed light pixels are white pixels. 2 white light pixels and 2 monochromatic light pixels form a 2×2 square array. Preferably, the two white light pixels are located on opposite corners of the square array. In each pixel unit, the ratio of the area occupied by all mixed light pixels to the entire pixel unit is equal to 50%.

As shown in FIG. 13 , the second lens is located above the N photoelectric converters. The monochromatic light pixel 402 further includes a monochromatic light filter 4023 disposed between the second lens 4021 and the photoelectric converter 4022 . If the monochromatic light pixel 402 is a red pixel, the monochromatic light filter 4023 is a red color filter. The green pixel and the blue pixel are the same as the red pixel, and will not be repeated here. Each monochromatic light pixel 402 also includes other auxiliary elements such as: a dielectric and reflective grid around the filter, a dielectric layer between the filter and the photodiode, an isolation region between the two photodiodes, and semiconductor substrate.

Among them, the second lens 4021 is used for focusing the incident light. The monochromatic light filter 4023 is used to filter the incident light and output monochromatic light. For example, the incident light is composed of red light, green light and yellow light. The monochromatic light filter 4023 is a red light filter. Optical filter 4023 only allows red light to pass. The photoelectric converter is used to convert the incoming optical signal into an electrical signal of corresponding intensity.

As shown in FIG. 13 , the optical path of the monochromatic light pixel is as follows: the light source irradiates the imaging object, and after being reflected by the imaging object, it enters the second lens 4021, and the second lens 4021 focuses the incident light and exits through the exit area, After the light emitted through the exit area is filtered by the monochromatic light filter 4023, it enters the photosensitive element of the photoelectric converter 4022, and the photoelectric converter 4022 converts the light signal received by the photosensitive element into an electrical signal of corresponding intensity, so as to realize the imaging The light image of the object is converted into an electrical signal. Each photoelectric converter 4022 corresponds to one outgoing area of the second lens, so that each photoelectric converter receives the monochromatic light emitted by the outgoing area located directly above, and converts the monochromatic light into an electrical signal. When the photoelectric converter is located at the focal point of the second lens, the electrical signals output by each photoelectric converter are the same, and one of the electrical signals can be arbitrarily selected to generate color data of the imaging object.

The imaging principle and focusing principle of the photoelectric converter in this embodiment are the same as those in the first embodiment, and repeated parts will not be repeated here.

In the photoelectric conversion unit provided in the second embodiment of the present application, the structure of the mixed light pixel and the monochromatic light pixel is similar, and both include a plurality of photoelectric converters distributed in a square. The similar structure is convenient for the mixed light pixel and the monochromatic light. The electrical signals output by the pixels are demodulated to generate corresponding image data.

The following mainly describes the structural intent of the photoelectric conversion unit provided in the third embodiment of the present application. A photoelectric conversion unit 400 provided by the present application includes a plurality of pixel units 401 . As shown in FIG. 14 , the difference between the photoelectric conversion unit provided in Embodiment 3 and the photoelectric conversion unit provided in Embodiment 1 is that each pixel unit includes at least one mixed light pixel and multiple monochromatic light pixel sub-units. A plurality of monochromatic light pixel subunits are distributed in an array, each monochromatic light pixel subunit includes a plurality of monochromatic light pixels distributed in an array, and each monochromatic light pixel subunit is used to convert the same monochromatic light into electricity. Signal.

In this embodiment, the monochromatic light is preferably red light (Red), green light (Green) and blue light (Blue), and the mixed light is preferably white light. Accordingly, the monochromatic light pixels 402 include red pixels, green pixels and blue pixels. The mixed light pixels 403 are white pixels. That is, each monochromatic light pixel subunit is a red pixel subunit, a blue pixel subunit and a green pixel subunit. The red pixel subunit includes a plurality of red pixels, the blue pixel subunit includes a plurality of blue pixels, and the green pixel subunit includes a plurality of green pixels.

Wherein, as shown in FIG. 15 , the monochromatic light pixel 402 includes a second lens 4021 and a photoelectric converter 4022 . That is, the structures of the monochromatic light pixels and the mixed light pixels are not similar. The second lens is located above the photoelectric converter. The monochromatic light pixel 402 further includes a monochromatic light filter 4023 disposed between the second lens 4021 and the photoelectric converter 4022 . If the monochromatic light pixel 402 is a red pixel, the monochromatic light filter 4023 is a red color filter. The green pixel and the blue pixel are the same as the red pixel, and will not be repeated here. Each monochromatic light pixel 402 also includes other auxiliary elements such as: a dielectric and reflective grid around the filter, a dielectric layer between the filter and the photodiode, an isolation region between the two photodiodes, and semiconductor substrate.

As shown in FIG. 15 , the optical path of the monochromatic light pixel is as follows: the light source illuminates the imaging object, and after being reflected by the imaging object, enters the second lens 4021, and the second lens 4021 focuses the incident light and exits through the exit area, After the light emitted through the exit area is filtered by the monochromatic light filter 4023, it enters the photosensitive element of the photoelectric converter 4022, and the photoelectric converter 4022 converts the light signal received by the photosensitive element into an electrical signal of corresponding intensity, so as to realize the imaging The light image of the object is converted into an electrical signal.

The area of the square area where the monochromatic light pixel sub-unit is located is the area occupied by 4 monochromatic light pixels, the area occupied by 9 monochromatic light pixels, or the area occupied by 16 monochromatic light pixels any of them.

The arrangement relationship between the monochromatic light pixel subunits and the mixed light pixels may be any of the following: each mixed light pixel is located at the corner of four adjacent monochromatic light pixel subunits, and each mixed light pixel is located at the corner of four adjacent monochromatic light pixel subunits. Located inside the monochromatic light pixel subunit, a part of the mixed light pixel is located at the corner of four adjacent monochromatic light pixel subunits, and the remaining mixed light pixels are located inside the monochromatic light pixel subunit.

The following describes the combination form between the area of the square region where the monochromatic light pixel subunits are located and the arrangement relationship between the monochromatic light pixel subunits and the mixed light pixels. In the drawings, the pixels where the lenses with the same gray value are located represent the same monochromatic light pixels.

As an implementation of the pixel unit, continue to refer to FIG. 7 , the pixel unit includes 1 mixed pixel and 4 monochromatic light pixel subunits, each monochromatic light pixel subunit includes 3 monochromatic light pixels, 3 monochromatic light pixels The chromatic light pixels are arranged in a 2×2 square, and a monochromatic light pixel is missing at the corner of the square to reserve a position for the mixed light pixel. The mixed light pixels are located at the corners of four adjacent monochromatic light pixel subunits. The arrangement area of the monochromatic light pixel subunits is the area of 4 monochromatic light pixels. In each pixel unit, the ratio of the area occupied by all mixed light pixels to the entire pixel unit is equal to 25%.

14, the pixel unit includes 4 mixed pixels and 16 monochromatic light pixel subunits, each monochromatic light pixel subunit includes 8 monochromatic light pixels, 8 monochromatic light pixels The chromatic light pixels are arranged in a 3×3 square, and a monochromatic light pixel is missing at the corner of the square to reserve a place for the mixed light pixel. Each mixed light pixel is located at the corner of four adjacent monochromatic light pixel subunits. The arrangement area of the monochromatic light pixel subunits is the area of 9 monochromatic light pixels. In each pixel unit, the ratio of the area occupied by all mixed light pixels to the entire pixel unit is equal to 1/9.

As an implementation of the pixel unit, referring to FIG. 16 , the pixel unit includes 5 mixed pixels and 16 monochromatic light pixel subunits, the 12 monochromatic light pixel subunit includes 8 monochromatic light pixels, and 4 monochromatic light pixels The subunit includes 7 monochromatic light pixels, and the monochromatic light pixels in each monochromatic light pixel sub-unit are arranged in a 3×3 square, and the monochromatic light pixels are missing at corresponding positions to reserve positions for mixed light pixels. Each mixed light pixel is located at the corner of four adjacent monochromatic light pixel subunits. The arrangement area of the monochromatic light pixel subunits is the area of 9 monochromatic light pixels. In each pixel unit, the ratio of the area occupied by all mixed light pixels to the entire pixel unit is equal to 5/36.

As an implementation of the pixel unit, referring to FIG. 17 , the pixel unit includes 16 mixed pixels and 16 monochromatic light pixel subunits, each monochromatic light pixel subunit includes 5 monochromatic light pixels, 5 monochromatic light pixels The light pixels are arranged in a 3×3 square. There are 4 monochromatic light pixels missing at the corners of the square to reserve positions for mixed light pixels. Each mixed light pixel is located inside the monochromatic light pixel sub-unit. The arrangement area of the monochromatic light pixel subunits is the area of 9 monochromatic light pixels. In each pixel unit, the ratio of the area occupied by all mixed light pixels to the entire pixel unit is equal to 4/9.

As an implementation of the pixel unit, referring to FIG. 18 , the pixel unit includes 16 mixed pixels and 16 monochromatic light pixel subunits, each monochromatic light pixel subunit includes 12 monochromatic light pixels, 12 monochromatic light pixels The light pixels are arranged in a 4×4 square. 4 monochromatic light pixels are missing in the interior of the square, to reserve places for mixed light pixels. Each mixed light pixel is located inside the monochromatic light pixel sub-unit. The arrangement area of the monochromatic light pixel subunits is the area of 16 monochromatic light pixels. In each pixel unit, the ratio of the area occupied by all mixed light pixels to the entire pixel unit is equal to 1/4.

As an implementation of the pixel unit, referring to FIG. 19 , the pixel unit includes 9 mixed pixels and 16 monochromatic light pixel sub-units. The 4 monochromatic light pixel subunit includes 8 monochromatic light pixels, the 4 monochromatic light pixel subunit includes 14 monochromatic light pixels, and the 8 monochromatic light pixel subunit includes 15 monochromatic light pixels, and each monochromatic light pixel subunit includes 15 monochromatic light pixels. The pixels are arranged in a 4 × 4 square, and the monochromatic light pixels are missing in the corresponding positions to reserve positions for the mixed light pixels. Each mixed light pixel is located at a corner of the monochromatic light pixel subunit. The arrangement area of the monochromatic light pixel subunits is the area of 16 monochromatic light pixels. In each pixel unit, the ratio of the area occupied by all mixed light pixels to the entire pixel unit is equal to 9/64.

As an implementation of the pixel unit, referring to FIG. 20 , the pixel unit includes 25 mixed pixels and 16 monochromatic light pixel sub-units. The 4 monochromatic light pixel subunit includes 8 monochromatic light pixels, the 4 monochromatic light pixel subunit includes 11 monochromatic light pixels, and the 8 monochromatic light pixel subunit includes 10 monochromatic light pixels, and each monochromatic light pixel subunit includes 10 monochromatic light pixels. The pixels are arranged in a 4 × 4 square, and the monochromatic light pixels are missing in the corresponding positions to reserve positions for the mixed light pixels. 16 mixed light pixels are located inside the monochromatic light pixel subunit, and 9 mixed light pixels are located at the corners of the monochromatic light pixel subunit. The arrangement area of the monochromatic light pixel subunits is the area of 16 monochromatic light pixels. In each pixel unit, the ratio of the area occupied by all mixed light pixels to the entire pixel unit is equal to 25/64.

In the photoelectric conversion unit provided in the second embodiment of the present application, the structures of the mixed light pixels and the monochromatic light pixels are not similar, and the area of the monochromatic light pixels is smaller than that of the mixed light pixels, which is conducive to arranging the monochromatic light pixels around the mixed light pixels , and on the premise of increasing the area of mixed light pixels, each mixed light pixel can collect enough color data with the surrounding monochromatic light pixels. With the increase of mixed light pixels, the texture data of the image is more accurate, and the phase focus is more accurate. Accurate, and the arrangement of mixed light pixels and monochromatic light pixels can obtain more accurate color data, and then the collected image data more truly reflects the imaging object.

The phase focusing method provided in the third embodiment of the present application will be described in detail below with reference to FIG. 21 and the structure of the photoelectric conversion unit provided in the second embodiment. In the photoelectric conversion unit provided in the second embodiment, the mixed light pixel is similar to the monochromatic light pixel, and both include a lens and photoelectric converters distributed in a square array. The phase focusing method provided in the fourth embodiment of the present application includes the following steps:

S501. Acquire the light intensity of the external environment.

Among them, the light intensity of the external environment is collected by the photosensitive element in the image sensor.

S502: Determine whether the light intensity is less than a preset threshold, if yes, go to S503, if not, go to S504.

S503: Determine the distance between the photoelectric converter and the focal point of the lens according to the multi-channel mixed photoelectric signal output by the mixed light pixel and the multi-channel monochromatic photoelectric signal output by the monochromatic light pixel.

The phase of each mixed photoelectric signal and each monochromatic photoelectric signal is extracted, and the mixed photoelectric signal and the monochromatic photoelectric signal with the same phase are superimposed to generate an enhanced electric signal. The distance between the photoelectric converter and the focal point of the lens is determined according to the enhanced electrical signal.

This step is described by taking as an example that the pixel unit includes white pixels, red pixels, green pixels and blue pixels, and each pixel includes a lens and 4 photoelectric converters arranged in a square array.

Acquire 4 channels of white photoelectric signals output by the white pixel, and extract the phases of the 4 channels of white photoelectric signals, which are marked as phase 11, phase 12, phase 13 and phase 14 respectively.

Using the same method for red pixels, green pixels and blue pixels, extract the phase of 4 channels of red photoelectric signals, the phases of 4 channels of green photoelectric signals and the phases of 4 channels of blue electrical signals.

The phases of the 4 red photoelectric signals are marked as phase 21, phase 22, phase 23 and phase 24 respectively. The phases of the 4 green photoelectric signals are respectively marked as phase 31, phase 32, phase 33 and phase 34. The four blue electrical signals The phases are labeled Phase 41, Phase 42, Phase 43, and Phase 44, respectively.

Since the structures of the white, red, green and blue pixels are similar, and the phase 11, phase 21, phase 31 and phase 41 are the same, the electrical signals of the above four phases are superimposed to generate an enhanced signal. The electrical signals of the other three groups of phases are superimposed to generate three enhanced signals correspondingly. Calculate the phase difference between any two enhanced electrical signals, and determine the distance between the photoelectric converter and the focal point of the lens according to the phase difference.

S504. Determine the distance between the photoelectric converter and the focal point of the lens according to the multi-channel mixed photoelectric signal output by the mixed light pixel.

Among them, the phase of each mixed photoelectric signal is extracted, the phase difference between any two enhanced electric signals is calculated, and the distance between the photoelectric converter and the focal point of the lens is determined according to the phase difference.

In the focusing method provided in the fourth embodiment of the present application, when the light intensity is low, electric signals with the same phase are added to generate corresponding enhanced signals, and then the distance between the photoelectric converter and the focal point of the lens is determined according to the phase difference between the enhanced signals. This improves the focusing accuracy in low light.

Those of ordinary skill in the art can understand that all or part of the steps of implementing the above method embodiments may be completed by program instructions related to hardware. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, the steps including the above method embodiments are executed; and the foregoing storage medium includes: ROM, RAM, magnetic disk or optical disk and other media that can store program codes.

Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, but not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: The technical solutions described in the foregoing embodiments can still be modified, or some or all of the technical features thereof can be equivalently replaced; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the technical solutions of the embodiments of the present invention. Scope.

Claims

A photoelectric conversion unit, comprising: a plurality of pixel units distributed in an array; wherein each pixel unit includes at least one mixed light pixel and at least one monochromatic light pixel;

Wherein, each monochromatic light pixel is used to convert monochromatic light into an electrical signal;

Each mixed light pixel includes a first lens and N photoelectric converters distributed in a square array, the first lens is located above the N photoelectric converters, the first lens includes N exit areas, each Each of the photoelectric converters receives the mixed light emitted by an exit region located directly above, and converts the mixed light into an electrical signal, N=k×k, where k is a positive integer greater than 1.
The photoelectric conversion unit according to claim 1, wherein the electrical signal output by each mixed light pixel is used to generate the distance between the photoelectric converter and the focal point of the first lens.
The photoelectric conversion unit according to claim 1 or 2, wherein the electrical signal output by each monochromatic light pixel and the electrical signal output by each mixed light pixel are jointly used to generate image data.
The photoelectric conversion unit according to claim 1, wherein the first lens is circular.
The photoelectric conversion unit according to claim 4, wherein each mixed light pixel includes 4 photoelectric converters.
The photoelectric conversion unit according to claim 1, wherein the mixed light is white light.
The photoelectric conversion unit according to claim 1, wherein the monochromatic light pixel comprises a second lens and N photoelectric converters distributed in a square array;

The second lens is located above the N photoelectric converters, the second lens includes N outgoing regions, and each photoelectric converter receives a monochromatic light emitted from an outgoing region located just above, and transmits the Monochromatic light is converted into electrical signals.
The photoelectric conversion unit according to claim 7, wherein the mixed light pixels and the monochromatic light pixels are distributed in an array.
The photoelectric conversion unit according to claim 7 or 8, wherein each pixel unit comprises one mixed light pixel and three monochromatic light pixels, and the three monochromatic light pixels are red pixels, Blue pixels and green pixels.
The photoelectric conversion unit according to claim 7 or 8, wherein each pixel unit comprises two mixed light pixels and two monochromatic light pixels, and the two monochromatic light pixels are red pixels, blue Any two of color pixels and green pixels.
The photoelectric conversion unit according to claim 1, wherein the monochromatic light pixel comprises a third lens and a photoelectric converter;

Wherein, the third lens is located above the photoelectric converter, and the photoelectric converter receives the monochromatic light emitted from the exit area of the third lens, and converts the monochromatic light into an electrical signal.
The photoelectric conversion unit according to claim 11, wherein each pixel unit comprises at least one mixed light pixel and a plurality of monochromatic light pixel sub-units distributed in an array, and each monochromatic light pixel sub-unit comprises an array of A plurality of monochromatic light pixels are distributed, and each monochromatic light pixel sub-unit is used to convert the same monochromatic light into electrical signals.
The photoelectric conversion unit according to claim 12, wherein the arrangement area of the monochromatic light pixel sub-units is the area of 4 monochromatic light pixels, the area of 9 monochromatic light pixels, or the area of 16 monochromatic light pixels Any of the areas of the light pixel.
The photoelectric conversion unit according to claim 11 or 12, wherein each mixed light pixel is located at a corner of four adjacent monochromatic light pixel subunits.
The photoelectric conversion unit according to claim 11 or 12, wherein each mixed light pixel is located inside a monochromatic light pixel sub-unit.
The photoelectric conversion unit according to claim 11 or 12, wherein a partial mixed light pixel is located at a corner of four adjacent monochromatic light pixel subunits, and the remaining mixed light pixels are located inside the monochromatic light pixel subunit .
An image sensor, characterized by comprising the photoelectric conversion unit according to any one of claims 1 to 16.
A focusing method of an image sensor, characterized in that the image sensor includes a photoelectric conversion unit, the photoelectric conversion unit includes a mixed light pixel and a monochromatic light pixel, and the mixed light pixel and the monochromatic light pixel both include A lens and photoelectric converters distributed in a square array, the method comprising:

Get the light intensity of the external environment;

If the light intensity is less than a preset threshold, determine the distance from the photoelectric converter to the lens according to the multi-channel mixed photoelectric signal output by the mixed light pixel and the multi-channel monochromatic photoelectric signal output by the monochromatic light pixel. distance between foci.
The method according to claim 18, wherein, according to the multi-channel mixed photoelectric signal output by the mixed light pixel and the multi-channel monochromatic photoelectric signal output by the monochromatic light pixel, it is determined that the photoelectric converter is connected to the The distance between the focal points of the lens, specifically including:

Extract the phase of each mixed photoelectric signal and each monochromatic photoelectric signal;

superimposing the mixed photoelectric signal and the monochromatic photoelectric signal with the same phase to generate an enhanced electric signal;

The distance between the photoelectric converter and the focal point of the lens is determined according to the enhanced electrical signal.
The method according to claim 18 or 19, wherein,

If the light intensity is greater than or equal to a preset threshold, the distance between the photoelectric converter and the focal point of the lens is determined according to the multi-channel mixed photoelectric signals output by the mixed light pixels.