WO2022104629A1

WO2022104629A1 - Image sensor, light splitting and color filtering device, and image sensor manufacturing method

Info

Publication number: WO2022104629A1
Application number: PCT/CN2020/130020
Authority: WO
Inventors: 张友明; 孙上; 刘闯闯
Original assignee: 华为技术有限公司
Priority date: 2020-11-19
Filing date: 2020-11-19
Publication date: 2022-05-27

Abstract

An image sensor, a light splitting and color filtering device, and an image sensor manufacturing method, for improving the utilization rate of light incident on image sensors. The image sensor comprises a super surface (701), a substrate (702), and a photoelectric conversion unit (703); the array in the super surface (701) is arranged on the top of the substrate (702); the super surface (701) comprises an array consisting of at least one columnar structure, the bottom of the substrate (702) is provided on the surface of the photoelectric conversion unit (703) in the image sensor, and the super surface (701) comprises at least two mediums having different refractive indexes, the photoelectric conversion unit (703) comprises an array for photoelectric conversion, the array of the photoelectric conversion unit (703) is divided into a plurality of color areas, and the super surface (701) is used for refracting incident light and transmitting same to the corresponding color area in the array of the photoelectric conversion unit (703) by means of the substrate (702).

Description

A kind of image sensor, spectroscopic color filter device and preparation method of image sensor

technical field

The present application relates to the field of imaging, and in particular, to an image sensor, a spectroscopic color filter device and a method for preparing the image sensor.

Background technique

Image sensors convert optical images into electrical signals and are widely used in a variety of electronic devices, such as digital cameras. The hardware of a digital camera mainly includes a lens group, an image sensor, and an electrical signal processor. The lens group is used to image an optical image on the image sensor, and the image sensor is used to convert the optical signal of the image into an analog electrical signal and input it to the electrical signal. The processor, the electrical signal processor converts the analog electrical signal into a digital signal, and outputs the photo after data processing. As a photoelectric converter, the image sensor is one of the core components of a digital camera, and its performance directly determines the quality of the output photo.

The photoelectric conversion element of the image sensor can convert light signals of different intensities into electrical signals of different intensities. However, the photoelectric conversion element itself cannot distinguish the frequency of light, that is, cannot distinguish the color. Therefore, pictures obtained directly with an image sensor that does not contain a color acquisition layer are black and white. In order to obtain a colored picture, a color filter system is required as a color acquisition layer to obtain the color information of the picture. For example, using the characteristics that the human eye is sensitive to the red green blue (RGB) three primary color spectrum, RGB color filters are arranged on the photoelectric conversion element to form an RGB mosaic Bayer color filter system, which can obtain color picture. However, in the existing solutions, the utilization rate of light is relatively low, for example, the total light utilization rate for incident white light is only about 25%; the light utilization rate for incident red light or blue light is only about 15%, The light utilization rate of the incident green light is about 30%, etc. Therefore, how to improve the light utilization rate has become an urgent problem to be solved.

SUMMARY OF THE INVENTION

Embodiments of the present application provide an image sensor, a spectroscopic color filter device, and a method for preparing the image sensor, which are used to improve the utilization rate of light incident on the image sensor.

In view of this, a first aspect of the present application provides an image sensor, including: a metasurface, a substrate and a photoelectric conversion unit; the metasurface includes at least one array of columnar structures, and the array in the metasurface is arranged on top of the substrate , the bottom of the substrate is arranged on the surface (or top) of the photoelectric conversion unit, the metasurface includes at least two media with different refractive indices, the photoelectric conversion unit includes an array for photoelectric conversion, and the array of photoelectric conversion units is divided into multiple A color region, the metasurface is used to refract incident light and transmit it to the corresponding color region in the array of photoelectric conversion units through the substrate.

Therefore, in the embodiment of the present application, the incident light can be refracted through an array formed by at least two media with different refractive indices on the metasurface, so that the light of different frequency bands can be refracted to the corresponding color region in the photoelectric conversion unit, Improved light utilization. Moreover, compared with setting Bayer color filters, the metasurface of the image sensor provided by the present application can refract light of various frequency bands to corresponding color regions, avoiding the problem of low light utilization rate caused by filtering. It can be understood that the metasurface can diffract the incident light, so that the light of different frequency bands can be transmitted to the corresponding color area in the photoelectric conversion unit, and the utilization rate of light can be improved.

In a possible implementation manner, a color filter structure is further arranged between the photoelectric conversion unit and the substrate, the color filter structure is divided into a plurality of color filter regions, and each color filter region covers a corresponding color region, and each color filter region covers a corresponding color region. The color corresponding to each color area is the same as the color transmitted by the color filter area covered by each color area.

In the embodiment of the present application, a color filter structure can be further arranged between the photoelectric conversion unit and the substrate to filter different signals of light in the frequency band corresponding to the color region, thereby reducing crosstalk.

In a possible implementation manner, a lens is further arranged between each color filter region and the substrate. In the present application, a lens can also be arranged between each color filter region and the substrate to condense the light incident on the color filter region.

In a possible implementation manner, the light of multiple frequency bands corresponding to multiple color regions includes: one or more of green, red, blue, or infrared light. Therefore, in the embodiments of the present application, the photoelectric conversion unit can convert optical signals in multiple frequency bands into electrical signals, so that images with more color channels can be generated subsequently.

In a possible embodiment, the material of the metasurface includes one or more of the following: titanium dioxide, gallium nitride or silicon carbide.

In a second aspect, the present application provides a spectroscopic color filter device, comprising: a metasurface and a substrate;

The metasurface includes at least one array of columnar structures, the array in the metasurface is arranged on top of the substrate, the metasurface includes at least two media with different refractive indices, the metasurface is used to refract incident light, and the substrate is used for Transmits light refracted by the metasurface.

Therefore, in the embodiments of the present application, incident light can be refracted through an array formed by at least two media with different refractive indices on the metasurface, so that light in different frequency bands can be refracted to different regions, thereby improving light utilization. In addition, compared with the setting of Bayer color filters, the metasurface of the spectroscopic color filter device provided by the present application can refract light of various frequency bands to corresponding color regions, avoiding the problem of low light utilization rate caused by filtering.

In a possible implementation, the color-splitting filter device can be applied to an image sensor, the image sensor includes a photoelectric conversion unit, the substrate is disposed on the surface of the photoelectric conversion unit, the photoelectric conversion unit includes an array for photoelectric conversion, and the photoelectric conversion unit includes an array for photoelectric conversion. The array of conversion units is divided into a plurality of color areas, and the metasurface refracts incident light and transmits it to the corresponding color areas in the array of photoelectric conversion units through the substrate. It can be understood that the metasurface can diffract the incident light, so that the light of different frequency bands can be transmitted to the corresponding color area in the photoelectric conversion unit, and the utilization rate of light can be improved.

In a possible implementation manner, a color filter structure is further arranged between the photoelectric conversion unit and the substrate, the color filter structure is divided into a plurality of color filter regions, and each color filter region covers a corresponding color region, and each color filter region covers a corresponding color region. The color corresponding to each color area is the same as the color transmitted by the color filter area covered by each color area. In the embodiment of the present application, a color filter structure can be further arranged between the photoelectric conversion unit and the substrate to filter different signals of light in the frequency band corresponding to the color region, thereby reducing crosstalk.

In a possible implementation manner, the colors of light in multiple frequency bands corresponding to the multiple color regions include: one or more of green, red, blue, or infrared light. Therefore, in the embodiments of the present application, the photoelectric conversion unit can convert optical signals in multiple frequency bands into electrical signals, so that images with more color channels can be generated subsequently.

In a third aspect, the present application provides a method for preparing an image sensor, comprising:

preparing a photoelectric conversion unit, the photoelectric conversion unit is used for converting an optical signal into an electrical signal, the photoelectric conversion unit includes an array for photoelectric conversion, and the array of the photoelectric conversion unit is divided into a plurality of color areas;

A spectroscopic color filter device is prepared on the surface of the photoelectric conversion unit. The spectroscopic color filter device includes a metasurface and a substrate. The metasurface includes at least one array of columnar structures. The array in the metasurface is arranged on the top of the substrate and the bottom of the substrate. It is arranged on the surface of the photoelectric conversion unit, the metasurface includes at least two kinds of media with different refractive indices, the photoelectric conversion unit includes an array for photoelectric conversion, the surface of the photoelectric conversion unit is the surface that receives the light signal, and the array of the photoelectric conversion unit is divided into A plurality of color regions, the metasurface is used to refract incident light to corresponding color regions in the array of photoelectric conversion units.

Therefore, in the embodiment of the present application, the prepared image sensor has an array formed by at least two media with different refractive indices on the metasurface of the spectroscopic color filter device to refract the incident light, so that the light of different frequency bands can be refracted to The corresponding color area in the photoelectric conversion unit improves the light utilization rate. In addition, compared with the setting of Bayer color filters, the metasurface of the spectroscopic color filter device provided by the present application can refract light of various frequency bands to corresponding color regions, avoiding the problem of low light utilization rate caused by filtering.

In a possible embodiment, before preparing the spectroscopic color filter device on the surface of the photoelectric conversion unit, the method further includes: determining a plurality of arrays, and using the plurality of arrays as the structure of the metasurface of the spectroscopic color filter device to obtain a variety of Spectral structure; multiple evaluation values corresponding to various spectroscopic structures are obtained through a preset evaluation function, and the evaluation function is a function for calculating the light utilization rate of the spectroscopic structure; if the multiple evaluation values include a value higher than the preset value at least one evaluation value, the first light-splitting structure is selected from a variety of light-splitting structures as the structure of the light-splitting color filter device, and the evaluation value of the first light-splitting structure is higher than a preset value.

Therefore, in the embodiment of the present application, the light utilization rate of each array can be calculated through a pre-established simulation model, so as to obtain an array with a light utilization rate exceeding a preset value, and then prepare an image sensor with a light utilization rate exceeding the preset value. . It can be understood that, based on the set demand target of light utilization, an array that meets the demand target of light utilization can be obtained inversely through optimization algorithms such as genetic algorithm, simulated annealing algorithm or gradient descent, so as to improve the performance of spectroscopic color filter devices and The light utilization of the image sensor.

In a possible implementation manner, the method may further include: if at least one evaluation value higher than a preset value is not included in the plurality of evaluation values, re-determining the plurality of arrays, and determining the light spectrum according to the re-determined plurality of arrays The structure serves as the structure of the spectral color filter device.

Therefore, in this embodiment of the present application, if the multiple arrays do not include an array with an evaluation value higher than a preset value, the multiple arrays can be updated to obtain new multiple arrays until the evaluation value is higher than the preset value. array of values.

In a possible implementation, re-determining multiple arrays may include: determining a mutation rate corresponding to each of the multiple arrays according to the values of multiple evaluation values; The array is mutated to obtain updated multiple arrays. In the embodiment of the present application, a new array can be obtained by mutation until an array with an evaluation value higher than a preset value is obtained.

In a possible implementation manner, re-determining multiple arrays may include: determining a probability value corresponding to each of the multiple arrays according to the values of the multiple evaluation values; Each array is sampled multiple times to obtain multiple intermediate structures; the variation rates of multiple intermediate structures are determined according to the evaluation values of multiple intermediate structures; the multiple intermediate structures are mutated according to the corresponding mutation rates of multiple intermediate structures to obtain new multiple arrays.

Therefore, in the embodiment of the present application, the probability value corresponding to each array can be determined according to the evaluation value of each array, and then sampling is performed based on the probability value of each array, so as to screen out an array with a higher evaluation value, and screen The resulting array is mutated to obtain a new array until an array with an evaluation value higher than the preset value is obtained.

In a possible implementation manner, the method may further include: preparing a color filter structure between the photoelectric conversion unit and the substrate, the color filter structure is divided into a plurality of color filter regions, and each color filter region covers a corresponding color The color corresponding to each color area is the same as the color transmitted by the color filter area covered by each color area, and each color filter area is used to filter light of colors other than the color corresponding to the covered color area.

In one possible implementation, the method may further include: fabricating a lens between each color filter region and the substrate. In the present application, a lens can also be arranged between each color filter region and the substrate to condense the light incident on the color filter region.

In a fourth aspect, the present application further provides an electronic device, which may include the image sensor in the aforementioned first aspect, or the image sensor prepared by the third aspect, or the like.

In a fifth aspect, the present application provides a method for determining an array structure applied to an image sensor, the image sensor includes a spectral color filter device and a photoelectric conversion unit, the spectral color filter device includes a metasurface and a substrate, and the photoelectric conversion unit The unit includes an array for photoelectric conversion, the spectroscopic color filter device includes a metasurface and a substrate, the array in the metasurface includes at least one array of columnar structures, and the array in the metasurface is arranged on the substrate The top of the bottom, the bottom of the substrate is arranged on the surface of the photoelectric conversion unit, the metasurface includes at least two kinds of media with different refractive indices, the array of the photoelectric conversion unit is divided into a plurality of color areas, the metasurface For refracting incident light to corresponding color regions in the array of photoelectric conversion units, the method includes: determining the structure of a plurality of arrays; evaluating the structures of the plurality of arrays through an evaluation function, and obtaining a The evaluation value corresponding to each of the multiple arrays, the evaluation function is a function of calculating the light utilization rate of the spectroscopic color filter device when the multiple arrays are used as the metasurface of the spectroscopic color filter device; according to the evaluation The value determines the structure of the metasurface of the spectroscopic color filter device.

In a possible implementation manner, the determining the structure of the array of metasurfaces of the spectroscopic color filter device according to the evaluation value may include: if the plurality of arrays include at least one whose evaluation value is higher than a preset value One array, then select one of the arrays from at least one array with the evaluation value higher than the preset value as the structure of the array of the metasurface of the spectroscopic color filter device; if the multiple arrays do not include an evaluation value higher than If the array of preset values is used, the plurality of arrays are updated, and the structure of the array of metasurfaces of the spectroscopic color filter device is determined according to the updated plurality of arrays.

In a possible implementation manner, the updating the plurality of arrays may include: determining, according to the values of the plurality of evaluation values, a mutation rate corresponding to each of the plurality of arrays; The mutation rates corresponding to the multiple arrays are mutated to obtain updated multiple arrays.

In a possible implementation manner, the updating the multiple arrays may include: determining a probability value corresponding to each of the multiple arrays according to the values of the multiple evaluation values; The probability values corresponding to the plurality of arrays are sampled multiple times to obtain multiple intermediate structures; the variation rates of the multiple intermediate structures are determined according to the evaluation values of the multiple intermediate structures; The mutation rate corresponding to the intermediate structure is used to mutate the multiple intermediate structures to obtain multiple new arrays.

In a sixth aspect, the present application provides an array structure construction device, comprising:

a first determining unit, used for determining the structure of the plurality of arrays;

an evaluation unit, configured to evaluate the structure of the plurality of arrays by using an evaluation function to obtain an evaluation value corresponding to each of the plurality of arrays, the evaluation function is to calculate the plurality of arrays as the spectrometer a function of the light utilization rate of the spectroscopic color filter device when the color filter device is metasurface;

The second determining unit is configured to determine the structure of the metasurface of the color-splitting filter device according to the evaluation value, the color-splitting filter device is included in an image sensor, and the image sensor includes a color-splitting filter device and a photoelectric conversion unit. The device includes a metasurface and a substrate, the photoelectric conversion unit includes an array for photoelectric conversion, and the spectroscopic color filter device includes a metasurface and a substrate, the array in the metasurface is arranged on the top of the substrate, and the bottom of the substrate is arranged on the photoelectric conversion The surface of the unit, the metasurface includes an array composed of at least two media with different refractive indices, the array in the metasurface includes an array composed of at least one columnar structure, the array of photoelectric conversion units is divided into multiple color regions, and the metasurface is used to convert Incident light is refracted to corresponding color regions in the array of photoelectric conversion units.

In a possible implementation manner, the first determining unit is specifically configured to, if the plurality of arrays includes at least one array whose evaluation value is higher than a preset value, select from at least one array whose evaluation value is higher than a preset value Selecting one of the arrays as the structure of the metasurface array of the spectroscopic color filter device;

The apparatus for constructing an array structure may further include: an updating unit configured to update the plurality of arrays if the arrays whose evaluation values are higher than a preset value are not included in the plurality of arrays;

The second determining unit is further configured to determine the structure of the array of the metasurface of the spectroscopic color filter device according to the updated multiple arrays.

In a possible implementation manner, the updating unit is specifically configured to determine a variation rate corresponding to each of the multiple arrays according to the values of the multiple evaluation values; and according to the variation rate corresponding to each array The multiple arrays are mutated to obtain updated multiple arrays.

In a possible implementation manner, the updating unit is specifically configured to determine a probability value corresponding to each of the plurality of arrays according to the values of the plurality of evaluation values; according to the probability value corresponding to each array , sampling the multiple arrays multiple times to obtain multiple intermediate structures; determining the variation rates of the multiple intermediate structures according to the evaluation values of the multiple intermediate structures; according to the variation rates corresponding to the multiple intermediate structures The multiple intermediate structures are mutated to obtain new multiple arrays.

In a seventh aspect, an embodiment of the present application provides an array structure construction device. The array structure construction device may also be called a digital processing chip or a chip. The chip includes a processing unit and a communication interface. The processing unit obtains program instructions through the communication interface. The instructions are executed by a processing unit, and the processing unit is configured to perform processing-related functions as described in the fifth aspect or any of the optional embodiments of the fifth aspect.

In an eighth aspect, an embodiment of the present application provides a computer-readable storage medium, including instructions, which, when executed on a computer, cause the computer to execute the method in the fifth aspect or any optional implementation manner of the fifth aspect.

In a ninth aspect, an embodiment of the present application provides a computer program product including instructions, which, when run on a computer, enables the computer to execute the method in the fifth aspect or any optional implementation manner of the fifth aspect.

Description of drawings

FIG. 1 is a schematic structural diagram of an electronic device according to an embodiment of the present application;

FIG. 2 is a schematic structural diagram of an image sensor provided by the application;

Fig. 3 is the structure of a kind of Bayer color filter provided for this application;

4 is a structure of another Bayer color filter provided by the application;

5 is a schematic diagram of optical signal transmission of a spectral channel;

6 is a schematic diagram of refraction and reflection of an optical signal provided by the application;

7 is a schematic structural diagram of a spectroscopic color filter device provided by the application;

FIG. 8 is a schematic structural diagram of an image sensor provided by the application;

9A is a schematic structural diagram of another spectroscopic color filter device provided by the application;

9B is a schematic structural diagram of a metasurface provided by the application;

10 is a schematic structural diagram of another image sensor provided by the application;

11 is a schematic structural diagram of a photoelectric conversion unit provided by the application;

12 is a schematic structural diagram of another image sensor provided by the application;

13 is a schematic structural diagram of another image sensing device provided by the application;

14 is a schematic flowchart of a method for determining an array structure provided by the present application;

15 is a schematic structural diagram of another metasurface provided by the application;

16 is a schematic flowchart of a method for preparing an image sensor provided by the present application;

17 is a schematic flowchart of a method for preparing an image sensor provided by the present application;

18 is a schematic diagram of the light utilization rate spectrum of a spectroscopic color filter device provided by the application;

19 is a schematic diagram of light intensity distribution in a photoelectric conversion unit provided by the present application;

FIG. 20 is a schematic structural diagram of an array structure construction device provided by the present application.

Detailed ways

The technical solutions in the embodiments of the present application will be described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work fall within the protection scope of the present application.

The terms "first", "second", "third", "fourth", etc. (if any) in the description and claims of this application and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It is to be understood that data so used may be interchanged under appropriate circumstances so that the embodiments described herein can be practiced in sequences other than those illustrated or described herein. Furthermore, the terms "comprising" and "having", and any variations thereof, are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those expressly listed Rather, those steps or units may include other steps or units not expressly listed or inherent to these processes, methods, products or devices.

In the description of the present application, "plurality" means two or more, unless otherwise expressly and specifically defined.

In this application, unless otherwise expressly specified and limited, the terms "installed", "connected", "connected", "fixed", "arranged" and other terms should be understood in a broad sense, for example, it may be a fixed connection, or It can be a detachable connection, or integrated; it can be a mechanical connection or an electrical connection; it can be directly connected or indirectly connected through an intermediate medium, or it can be the internal connection of two elements or the interaction between the two elements. . For those of ordinary skill in the art, the specific meanings of the above terms in this application can be understood according to specific situations.

In the description of this application, it should be understood that the terms "length", "width", "upper", "lower", "front", "rear", "left", "right", "vertical", The orientation or positional relationship indicated by "horizontal", "top", "bottom", "inner", "outer", etc. is based on the orientation or positional relationship shown in the drawings, and is only for the convenience of describing the present application and simplifying the description, and It is not indicated or implied that the indicated device or element must have a particular orientation, be constructed and operate in a particular orientation, and therefore should not be construed as limiting the application.

Certain terms are used in the specification and claims to refer to particular components. It should be understood by those skilled in the art that hardware manufacturers may refer to the same component by different nouns. This specification and the following claims do not use the difference in name as a way to distinguish components, but use the difference in function of the components as a criterion for distinguishing. The inclusion or inclusion mentioned in the specification and claims is an open-ended term, so it should be interpreted as including but not limited to or including but not limited to.

For the convenience of understanding, the technical terms involved in this application are explained and described below.

Metamaterial: Broadly defined, it refers to an artificially designed complex of unit structures with physical properties that traditional natural materials do not possess. Its physical properties are mainly determined by the structure and arrangement of sub-wavelength (much smaller than wavelength) unit structures.

Metasurface: A two-dimensional form of a metamaterial, including in this application metasurface.

Focal point: When light enters the metasurface structure, the light rays converge at several points behind the metasurface structure, and these points where the light rays converge are the focal points.

Focal length: Also known as focal length, it is a measure of the concentration or divergence of light in an optical system. In the embodiment of this application, when an infinitely far scene is formed into a clear image at the focal plane through the metasurface structure, the distance from the optical center of the metasurface structure to the focal point can also be understood as the vertical distance from the optical center of the metasurface structure to the focal plane. .

The technical solutions in the present application will be described below with reference to the accompanying drawings.

The electronic devices provided in the embodiments of the present application may include handheld devices, vehicle-mounted devices, wearable devices, computing devices, or other processing devices connected to a wireless modem. May also include digital cameras, cellular phones, smart phones, personal digital assistant (PDA) computers, tablet computers, laptop computers, machines Types of communication (machine type communication, MTC) terminals, point of sales (point of sales, POS), on-board computers, headsets, wearable devices (such as bracelets, smart watches, etc.), security equipment, virtual reality (virtual reality, VR ) devices, augmented reality (AR) devices, and other electronic devices with imaging capabilities.

Taking a digital camera as an example, a digital camera is the abbreviation of digital camera, which is a camera that uses a photoelectric sensor to convert optical images into digital signals. Unlike traditional cameras, which rely on changes in the photosensitive chemicals on the film to record images, the sensor of a digital camera is a photosensitive charge-coupled device (CCD) or complementary metal oxide semiconductor (complementary metal oxide semiconductor). , CMOS). Compared with traditional cameras, digital cameras have the advantages of being more convenient, fast, repeatable, and more timely due to the direct use of photoelectric conversion image sensors. With the development of CMOS processing technology, the functions of digital cameras have become more and more powerful, and they have almost completely replaced traditional film cameras. They are widely used in consumer electronics, security, human-computer interaction, computer vision, automatic driving and other fields.

FIG. 1 shows a schematic diagram of an electronic device provided by the present application. As shown in the figure, the electronic device may include a lens group 110 , an image sensor (sensor) 120 and an electrical signal processor 130 . The electrical signal processor 130 may include an analog-to-digital (A/D) converter 131 and a digital signal processor 132 . The analog-to-digital converter 131 is an analog-to-digital signal converter, and is used for converting an analog electrical signal into a digital electrical signal.

It should be understood that the electronic device shown in FIG. 1 is not limited to include the above components, and may also include more or less other components, such as batteries, flashlights, buttons, sensors, etc. The embodiment of the present application is only provided with an image sensor installed The electronic device of 120 is taken as an example for description, but the components mounted on the electronic device are not limited to this.

The light signals reflected by the object are collected by the lens group 110 and imaged on the image sensor 120 . The image sensor 120 converts the optical signal into an analog electrical signal. The analog electrical signal is converted into a digital electrical signal by an analog-to-digital (A/D) converter 131 in the electrical signal processor 130, and the digital electrical signal is processed by the digital signal processor 132, for example, through a series of complex mathematical arithmetic operations , optimize the data electrical signal, and finally output the image. The electrical signal processor 130 may further include an analog signal pre-processor 133 for pre-processing the analog electrical signal transmitted by the image sensor and outputting it to the analog-to-digital converter 131 .

The performance of the image sensor 120 affects the quality of the final output image. The image sensor 120 may also be called a photosensitive chip, a photosensitive element, etc., and includes hundreds of thousands to millions of photoelectric conversion elements. When irradiated by light, charges are generated and converted into digital signals by an analog-to-digital converter chip.

Generally, the image sensor 120 can acquire color information of an image through a color filter system. The color filter system may be a Bayer color filter system. That is, the Bayer color filter is covered over the photoelectric conversion elements in the image sensor 120 to form a color filter system. The photoelectric conversion element may be a photodiode. Bayer filters may also be referred to as Bayer filters. FIG. 2 shows a schematic diagram of an image sensor based on a Bayer color filter system. The image sensor includes a microlens 121 , a Bayer color filter 122 and a photodiode 123 . The Bayer color filter 122 includes RGB color filters, and the RGB color filters are arranged on the squares of the photodiodes to form an RGB mosaic color filter system. Following the biological characteristics of the largest number of green photoreceptor cells in the retina of the human eye, Bayer filters are usually arranged in the form of RGGB. based on

FIG. 3 shows the structure of an image sensor based on a Bayer color filter system, and FIG. 4 shows a schematic structural diagram of a color pixel unit in the image sensor. As shown in FIG. 3 or FIG. 4 , one color pixel unit includes four color filters 122 and corresponding four photodiodes 123 pixel elements. The four color filters 122 are arranged in the form of RGGB, that is, the red color filter and the blue color filter are in the diagonal position, and the two green color filters are in the diagonal position. Since the photosensitive area of the photodiode 123 is at the center of the area occupied by the photodiode pixel unit, a color pixel unit further includes an array of microlenses 121 above the color filter 122 . The array of microlenses 121 is used for converging light signals into the photosensitive area of the photodiode 123 to ensure light utilization. The microlens 121 array condenses the incident light signals onto the four color filters 122 respectively, and after being filtered by the four color filters 122, transmits them to the photodiodes 123 covered by them, thereby simultaneously obtaining the light intensity information and The approximate color information can be optimized and restored to the closest true color image through the software difference algorithm in the later stage.

However, image sensors based on Bayer color filter systems have low light utilization. For each color pixel channel, it can also be said to be a spectral channel, more than 70% of the light signal will be filtered out by the Bayer filter, and only less than 30% of the light can reach the photodiode and be converted into an electrical signal for final use. computational imaging. Figure 5 shows a schematic diagram of the luminous flux of one spectral channel in a color pixel unit. As shown in Figure 5, for a color pixel unit arranged in the form of RGGB, when the incident light is white light, that is, it contains light signals with all wavelengths of 400-700 nanometers, in the case that the color filter has an ideal color filtering effect, the filter The theoretical maximum value of the luminous flux after color filtering is only 1/3 of the incident luminous flux; when the incident light is red or blue light, the theoretical maximum value of the filtered luminous flux is 1/4 of the incident luminous flux; when the incident light is green light, Since there are two green channels, the theoretical maximum value of the filtered luminous flux is 1/2 of the incident luminous flux. And in fact, the color filtering effect of the color filter cannot be perfect in reality, that is, its color filtering and light transmission efficiency cannot reach 100%, so the actual light utilization rate will be lower. When the incident light is white light, the total light utilization rate is only about 25%; when the incident light is red or blue light, the light utilization rate is about 15%, and when the incident light is green light, the light utilization rate is about 30%.

Alternatively, in some scenarios, the color filter system is replaced by a metasurface with spectral splitting capabilities. According to the generalized Snell's law, the direction of reflected and transmitted light depends not only on the refractive index of the interface material, but also on the phase gradient distribution at the interface. For example, as shown in Figure 6, the phase gradient distribution can be calculated by the formula, as

According to the spectroscopic function of the metasurface and the generalized Snell's law, the required spatial phase distribution can be calculated, and then the required geometric phase can be generated by the anisotropic nanofin structure at the required wavelength, and the spectral splitting function can be realized. However, both metasurfaces take advantage of the geometric phase generated by the anisotropic nanofin structure to achieve spectroscopic functions. An inherent property of the geometric phase is that only one circularly polarized light works, and the other circularly polarized light will become useless stray light and cannot be utilized, so at most 50% of the light can be utilized. In addition, another inherent characteristic of the nanofin structure is that each nanofin structure produces the same geometric phase for light of different frequencies. Nanofins of different sizes can be placed in a unit structure, and each size of nanofins acts on different frequency bands, thereby realizing the effect of light splitting. However, doing so will inevitably result in a very low light utilization rate, because nanofins of different sizes will negatively scatter light in non-corresponding operating frequency bands.

Therefore, the present application provides a spectroscopic color filter device applied to an image sensor for improving light utilization. Through the integrated pixel-level spectroscopic device, the efficient pixel-level spectral spectroscopic function is realized, and the light utilization rate of the color image sensor is improved.

The spectroscopic color filter device, the image sensor, and the preparation method of the image sensor provided by the present application will be described in detail below.

First of all, the present application provides an image sensor, the image sensor includes a color-splitting filter device and a photoelectric conversion unit, the color-splitting filter device is used to refract incident light, so that light of different frequency bands can be transmitted to the corresponding photoelectric conversion unit in the photoelectric conversion unit. color area.

The structure of the spectroscopic color filter device applied to the image sensor provided by the present application will be described.

Referring to FIG. 7 , a schematic structural diagram of a spectroscopic color filter device provided by the present application.

The spectroscopic color filter device includes: a metasurface 701 and a substrate 702 .

The array of metasurfaces 701 is arranged on top of the substrate 702, or the top of the substrate 702 is used to carry the metasurfaces.

The metasurface 701 includes at least one array of columnar structures assembled. The array of the metasurface 701 is used to refract light, and the metasurface 701 includes two media with different refractive indices. For example, a metasurface can be divided into multiple meshes, each filled with a medium, such as titanium dioxide or air.

The substrate 702 is usually composed of a material whose light transmittance is higher than a certain value, such as silicon dioxide, polymethyl methacrylate (PMMA) or polycarbonate (PC). Alternatively, the substrate 702 can also be a hollow structure to ensure high transmittance.

The structure of the image sensor including the spectroscopic color filter device may be as shown in FIG. 8 , and the bottom of the substrate is disposed on the surface or the top of the photoelectric conversion unit 703 in the image sensor. The photoelectric conversion unit 703 includes an array for photoelectric conversion, the array of photoelectric conversion units is divided into a plurality of color areas, and the metasurface is used to refract incident light and transmit it to the corresponding color area in the array of photoelectric conversion units through the substrate. For example, each pixel unit in the photoelectric conversion unit can be divided into four color regions, such as four color regions of red, green, green and blue (RGGB). After the incident light is refracted by the metasurface, the red light is transmitted through the substrate. To the R area of the photoelectric conversion unit, the green light is transmitted to the G area through the substrate, and the blue light is transmitted to the B area through the substrate.

Therefore, in the spectroscopic color filter device provided by the embodiment of the present application, the array formed by the columnar structure in the metasurface refracts light of different colors. When it is applied to the sensor, the incident light is refracted by the metasurface and transmitted to the The corresponding color area in the photoelectric conversion unit, so as to realize light splitting. It can be understood that the spectroscopic color filter device has a medium metasurface or a medium diffractive surface, has the structural characteristics of a second-order two-dimensional code pattern, and has a variety of spectral channels to achieve the function of splitting multiple colors and achieving efficient light splitting. And the array on the metasurface structure can refract the incident light, reduce the scattering phenomenon, and improve the light utilization rate. It can also be understood that the metasurface diffracts the incident light, so that the light of different frequency bands can be transmitted to the corresponding color area in the photoelectric conversion unit, especially in the sub-wavelength scene, the metasurface diffracts the incident light, which is relatively With the Bayer color filter and nano-fin metasurface structure, the metasurface provided by this application can diffract light in different frequency bands, thereby improving the utilization rate of light.

Specifically, the structure of the array of metasurfaces 701 may include at least two media with different refractive indices. Taking two media as an example, at least one of the materials may form a columnar structure and form an array of metasurfaces.

Optionally, the material of the metasurface includes one or more of the following: materials with high refractive index such as titanium dioxide, gallium nitride or silicon carbide. For example, the array of the metasurface can be a columnar structure including titanium dioxide, a plurality of columnar structures constitute the array, and the other medium in the metasurface can be air.

Exemplarily, the metasurface can be composed of titanium dioxide and air, the refractive index of titanium dioxide is higher than that of air, the structure of the metasurface can be as shown in Figure 9A, the material of the plurality of columnar structures can be titanium dioxide, and the other medium can be air. , it can be understood that the columnar structure formed by titanium dioxide and the air form an array. The top view of the array can be shown in Figure 9B, which is equivalent to meshing the metasurface and dividing it into multiple networks. The size of each mesh can be the same or different. Here, each mesh is the same size. example of a square network. Specifically, for example, the height of the columnar structures on the metasurface 701 may be 500 nm, the width of each square grid may be 100 nm, and the transparent substrate 702 is made of silica glass with a thickness of 3um. The pixel size of the sensor targeted by the metasurface is 800nm, that is, the size of the metasurface corresponding to one pixel unit is 1.6um. In the subsequent preparation of the spectroscopic device, the metasurface can be obtained by filling the medium in the grid. Each medium can have a columnar structure, and the size and shape of each columnar structure can be the same or different. Furthermore, one of the mediums may be air, and the other medium may be a material having a different refractive index than air. The material is saved, and the preparation efficiency of the spectroscopic color filter device can be improved.

Exemplarily, the transmission mode of incident light after being refracted by the metasurface is exemplified. As shown in FIG. 10 , (a) of FIG. 10 shows a schematic structural diagram of a color pixel unit in an embodiment of the present application. As shown in (a) of FIG. 10 , a color pixel unit may include a metasurface structure 701 and four adjacent two-dimensionally arranged photoelectric conversion elements located under the metasurface structure 702 . The four photoelectric conversion elements correspond to the photoelectric conversion element A, the photoelectric conversion element B, the photoelectric conversion element C, and the photoelectric conversion element D in FIG. 10( b ), respectively. The four photoelectric conversion elements can be arranged in an RGGB manner. For example, photoelectric conversion element A, photoelectric conversion element B, photoelectric conversion element C, and photoelectric conversion element D can correspond to three frequency bands of red light, green light, green light, and blue light, respectively. The metasurface structure 701 can focus red light, green light, and blue light on the photosensitive positions of the photoelectric conversion element A, the photoelectric conversion element B, the photoelectric conversion element C, and the photoelectric conversion element D, respectively. Combined with the transmission path of refracted light and reflected light when the spatial transmission phase gradient on the interface shown in FIG. 6 is 0 and the transmission path of refracted light and reflected light when the spatial transmission phase gradient on the interface is not 0. A spatial transmission phase is generated in the tangential direction of the array of metasurfaces 701 to obtain a spatial transmission phase gradient, and the spatial transmission phase gradient is used to transmit the optical signal of each frequency band to the photoelectric conversion element corresponding to each frequency band. In this embodiment of the present application, the transmission phase may also be referred to as the transmission phase.

The existence of the spatial transmission phase gradient enables the incident optical signal to form a certain resonance effect with the metasurface 701. When optical signals of different frequency bands pass through the metasurface 701, different transmission phase changes will be generated in the metasurface 701, which can further enable Changing the refraction angle of the optical signal, that is, controlling the propagation direction of the optical signal, transmits the optical signal of different frequency bands to the photoelectric conversion elements of different frequency bands.

It should be understood that the numerical values of the above-mentioned spectrum bands are only used as reference data for interpretation, and should not be regarded as limitations on the embodiments of the present application. The specific frequency range of each spectrum channel is based on the overall spectrum design of the actual image sensor and the actual spectrum response range of the photoelectric conversion element. prevail. The number of photoelectric conversion elements in one color pixel unit shown in FIG. 10 is for illustration only, and does not constitute a limitation to the embodiments of the present application.

FIG. 11 shows a schematic diagram of an array of photoelectric conversion units 220 . As shown in FIG. 11 , each photoelectric conversion unit 703 may correspond to four photoelectric conversion elements, and the four photoelectric conversion elements are arranged in an RGGB manner. In addition, an anti-emission plate may also be provided in the photoelectric conversion unit to reduce the reflection of the optical signal by the photoelectric conversion unit and further improve the light utilization rate.

It should be understood that the number of photoelectric conversion elements in one photoelectric conversion unit 703 shown in FIG. 10 and FIG. 11 is only for illustration, and in practice, the number of photoelectric conversion elements may be more or less. The number of photoelectric conversion elements does not constitute a limitation to the embodiments of the present application.

Therefore, in the embodiments of the present application, by arranging an array composed of columnar structures on the metasurface of the spectroscopic color filter device, the refraction of light in different frequency bands is realized. Compared with the traditional Bayer color filter, the present application refracts the light of different frequency bands, so that the light of each frequency band is transmitted to the corresponding area in the photoelectric conversion unit, avoiding the low utilization rate of light caused by the filtering of the color filter. .

Usually, the light is refracted by the nanofin structure, which only works on one circularly polarized light, and the other circularly polarized light will become useless stray light and cannot be used. Therefore, the light utilization rate is greatly reduced. The spectroscopic color filter device provided in the present application can refract all incident polarized light, not only one of the circularly polarized lights, and has little dependence on polarization. Therefore, the spectroscopic color filter device provided by the present application It can greatly improve the light utilization rate.

In addition, the light is refracted by the nanofin structure, because each nanofin structure has the same geometric phase for light of different frequencies, so it is desired to realize the light splitting function of different frequencies through the geometric phase generated by the nanofin structure. , only nanofins of different sizes can be placed in one unit structure, and each size of nanofins acts on different frequency bands, thereby achieving the effect of light splitting. However, doing so will inevitably result in a very low light utilization rate, because nanofins of different sizes will negatively scatter light in non-corresponding operating frequency bands. In the embodiment of the present application, the array structure of the metasurface can be composed of at least two kinds of media with different refractive indices, and the columnar structure has the required response to light in different frequency bands, which avoids the need for nanofins of different sizes to refract different frequency bands. The problem of low utilization rate of light that is not refracted caused by the refracted light further improves the utilization rate of light.

Optionally, the surface of the photoelectric conversion unit may further include an anti-reflection layer or an anti-reflection layer may be provided between the surface of the photoelectric conversion unit and the substrate to reduce reflection of incident light and improve light utilization.

In a possible implementation manner, a color filter structure is further arranged between the photoelectric conversion unit 703 and the substrate 702, the color filter structure is divided into a plurality of color filter regions, and each color filter region covers a corresponding color region, And the color corresponding to each color area is the same as the color transmitted by the color filter area covered by each color area. For example, the color filter structure can be four color filters or other mediums that transmit a specified frequency band.

Exemplarily, as shown in FIG. 12 , a color filter 704 may be provided between the photoelectric conversion unit and the substrate, and the color filter may be divided into a plurality of color filter regions for filtering light other than a specific frequency band. Light is filtered. For example, if the color area corresponding to the color filter is green, the color filter can transmit green and filter other colors except green.

Therefore, in the embodiments of the present application, by adding a color filter structure, the light transmitted to each color region in the photoelectric conversion unit can be filtered, and the light that is not in a specific frequency band can be filtered out, thereby avoiding the photoelectric conversion unit. The interference of conversion is carried out, and the light utilization rate is further improved.

In a possible implementation manner, a lens is further arranged between each color filter region and the substrate.

Exemplarily, as shown in FIG. 13 , a micro-convex lens 705 (referred to as a micro-lens) is disposed between each color filter region and the substrate. Usually, the size of each microlens is the same as the size of the color filter area it covers, so that the light transmitted to the color filter area is converged, so as to reduce the scattering of the light transmitted to the color area, and further improve the light utilization rate. Therefore, in the embodiments of the present application, a color filter and a microlens may be used in the color separation filter device to reduce the crosstalk of each channel.

In a possible implementation manner, the colors of light in multiple frequency bands corresponding to multiple color regions include: one or more of green, red, blue, or infrared (infrared radiation, IR).

For example, the spectroscopic color filter device provided by the present application has no less than two spectral spectral channels, and the frequency band range of the spectrum is in the ultraviolet to near-infrared range, depending on the application of the image sensor and the spectral response range of the photosensitive layer; The arrangement depends on the image sensor usage; for example, for visible light imaging, the spectral band range is 400-700nm visible light range, the number of spectra is 3, and the channel arrangement is RGGB; for multispectral imaging, the tiled frequency band range is 400-700 , the number of spectrum can be 7; for visible light & near-infrared imaging, the spectrum frequency range is 400-1000nm, the number of spectrum is 4, and the channel arrangement can be RGB&IR. Therefore, in the embodiments of the present application, it is possible to realize the light splitting of various kinds of visible light or invisible light, adapt to various scenarios, and has strong generalization ability.

The structure of the spectroscopic color filter device provided by the present application has been exemplarily described above. From the above structure, it can be seen that the metasurface of the spectroscopic color filter device plays an important role in improving the light utilization rate, and the array of the metasurface can be arranged in many ways. In the following, the method for determining the array structure of the metasurface of the spectroscopic color filter device will be introduced.

Referring to FIG. 14 , the present application provides a method for determining an array structure applied to an image sensor. The image sensor includes a color-splitting filter device and a photoelectric conversion unit. For the color-splitting filter device, please refer to the aforementioned color-splitting filter device in FIGS. 7 to 13 . , and details are not repeated here, and the detailed steps of the method for determining an array structure applied to an image sensor provided by the present application will be introduced below.

1401. Determine the structures of multiple arrays.

The structures of the multiple arrays may be obtained by searching from a preset search space, or may be randomly generated structures. Each array can be a gridded structure, a honeycomb structure, or the like.

For example, a variety of array structures can be generated in advance to form a search space, and then available array structures can be screened according to actual application scenarios, so as to quickly obtain multiple arrays and improve the efficiency of obtaining multiple arrays. For another example, one or more kinds of two-dimensional code images can be randomly generated, and then an array can be constructed according to the plurality of two-dimensional code images. Specifically, for example, the size of the metasurface to be constructed can be preset, and then the metasurface is meshed, divided into square meshes or hexagonal meshes, etc., and then each mesh is randomly filled to obtain an array structure, as shown in Figure 15.

1402. Evaluate the structures of the multiple arrays by using the evaluation function to obtain an evaluation value corresponding to each of the multiple arrays.

Among them, the evaluation function is a function of calculating the light utilization rate of the spectroscopic color filter device when multiple arrays are used as the metasurface of the spectroscopic color filter device, and the light utilization rate of each array is obtained, and then the metasurface of the spectroscopic color filter device is determined according to the evaluation value. The structure of the array.

For example, N initial two-dimensional code structures can be randomly generated, each two-dimensional code structure is composed of N*N square areas, and each area can be one of two dielectric materials, such as air and titanium dioxide, just A second-order matrix consisting of 0/1 of N*N characterizes the structure. In this example, 1 represents titanium dioxide, and 0 represents air. According to the spectroscopic function of RGGB, a simulation model is established, and Maxwell's equation is solved by simulation, and the average light utilization rate is calculated as the evaluation function as follows:

Among them, λ _r1 ～λ _r2 , λ _g1 ～λ _g2 , λ _b1 ～λ _b2 are the spectra of red light, green light, and blue light, respectively, _{Tr , T g} _, T _b are the corresponding red light, green light, and blue light, respectively Transmittance of the cell area. Specifically, for example, N image sensors corresponding to the two-dimensional code structure can be generated by simulation, and then the light utilization rate of each simulated image sensor can be calculated by an evaluation function to obtain the light utilization rate corresponding to each array. Of course, the light utilization rate can also be calculated by preparing an image sensor corresponding to each array, which is not limited in this application.

1403. Determine that the plurality of arrays includes at least one array whose evaluation value is higher than the preset value, if yes, go to step 1404, if not, go to step 1405.

After obtaining the evaluation value of each of the multiple arrays, it is determined whether there is an array with an evaluation value higher than a preset value in the multiple arrays, and if the multiple arrays include at least one array whose evaluation value is higher than the preset value, Then one of the arrays with the evaluation value higher than the preset value can be selected as the structure of the array of the metasurface of the spectroscopic color filter device; if the arrays with the evaluation value higher than the preset value are not included in the multiple arrays, it can be obtained. New arrays until an array with an evaluation value higher than the preset value is obtained.

1404. Select one of the arrays from the at least one array whose evaluation value is higher than a preset value as the structure of the metasurface of the spectroscopic color filter device.

Wherein, one can be selected from one or more arrays with an evaluation value higher than a preset value as the structure of the metasurface of the aforementioned spectroscopic color filter device.

For example, if there are multiple arrays with an evaluation value higher than 50%, the array with the highest evaluation value can be selected from the arrays with the evaluation value higher than 50% as the structure of the metasurface of the spectroscopic color filter device. Alternatively, one array is randomly selected from the plurality of arrays with an evaluation value higher than 50% as the structure or the like of the metasurface of the spectroscopic color filter device.

1405. Update multiple arrays.

Wherein, if there is no array whose evaluation value is higher than the preset value among the multiple arrays, the multiple arrays are updated, and the structure of the metasurface of the spectroscopic color filter device is determined according to the updated multiple arrays, that is, step 1202 is repeatedly executed. -1203 until an array with an evaluation value higher than the preset value is obtained.

The manners of updating the plurality of arrays may include various manners, and some possible implementation manners will be introduced below.

In a possible implementation, the mutation rate corresponding to each of the multiple arrays may be determined according to the values of the multiple evaluation values; then, the multiple arrays may be mutated according to the mutation rate corresponding to each array to obtain an update After multiple arrays. Usually, the evaluation value of each array can be used to represent the light utilization rate obtained after each array is substituted into the image sensor through simulation. The higher the light utilization rate, the lower the corresponding variation rate. The difference between the light utilization rate and the variation rate is The relationship can be a linear relationship or an exponential relationship, etc., which can be adjusted according to the actual application scenario. The higher the mutation rate, the greater the proportion of the array being mutated. For example, if the mutation rate is 20%, 20% of the regions in the array can be mutated to obtain a mutated array. The specific mutation method may be to rearrange the columnar structures in some regions in the array, exchange the structures of some regions in other arrays, etc. The specific mutation method can be adjusted according to the actual application scenario.

Therefore, in the embodiment of the present application, the mutation rate of each array can be determined based on the evaluation value of each array, so as to complete the mutation of the array, and the array with a higher evaluation value corresponds to a lower mutation rate. The structure of the new array is closer to the preset value, so that the array with higher light utilization can be screened out later.

In another possible implementation manner, a probability value corresponding to each of the multiple arrays may be determined according to the values of the multiple evaluation values; then, the multiple arrays are sampled according to the probability value corresponding to each array, Obtain multiple intermediate structures; then, determine the mutation rates of the multiple intermediate structures according to the evaluation values of the multiple intermediate structures; then, mutate the multiple intermediate structures according to the corresponding mutation rates of the multiple intermediate structures to obtain new multiple arrays . The specific mutation method may be to rearrange the columnar structures in some regions in the array, exchange the structures of some regions in other arrays, etc. The specific mutation method can be adjusted according to the actual application scenario.

For example, if the light utilization rate corresponding to array 1 is 25%, the light utilization rate corresponding to array 2 is 30%, the light utilization rate corresponding to array 3 is 26%, etc., then based on the value of the light utilization rate of each array, The probability of being sampled is set for each array, such as the probability of array 1 is 10, the probability of array 2 is 25%, the probability of array 3 is 13%, etc., and then one or more samplings can be performed based on the probability of each array, Get multiple of these columns, that is, multiple intermediate structures. Specifically, for example, the manner of setting the corresponding probability for each array may include:

where i represents the ith two-dimensional code structure, F _i is the evaluation function of the ith structure, and n can take an integer greater than or equal to 1.

Wherein, if multiple samplings are performed, each sampling may be based on the probability of each array, and the arrays that may be sampled by different batches of sampling may be the same or different. Therefore, in the embodiment of the present application, the probability of each array can be set based on the light utilization rate of each array, so that the array with higher light utilization rate has a higher probability of being sampled during subsequent sampling. And mutate the sampled array to obtain a new array.

In another possible implementation, if a search space is preset, if an array with an evaluation value higher than the preset value is not filtered out, multiple arrays can be obtained by searching again from the search space.

In the method provided by the present application, based on the set demand target of light utilization, an optimization algorithm such as genetic algorithm, simulated annealing algorithm or gradient descent can be used to reversely obtain an array that meets the demand target of light utilization, thereby improving the spectroscopic performance. Light utilization of color filter devices and image sensors.

The foregoing has described in detail the method for determining the array structure of the spectroscopic color filter device, the image sensor, and the application and image sensor provided by the present application. The preparation method of the image sensor provided by the application is introduced in detail.

Referring to FIG. 16 , a schematic flowchart of a method for preparing an image sensor provided by the present application is as follows.

1601. Prepare a photoelectric conversion unit. The photoelectric conversion unit is used to convert an optical signal into an electrical signal. The photoelectric conversion unit includes an array for photoelectric conversion, and the array of the photoelectric conversion unit is divided into a plurality of color regions.

1602. Prepare a spectroscopic color filter device on the surface of the photoelectric conversion unit.

Wherein, the spectroscopic color filter device includes a metasurface and a substrate, the array in the metasurface is arranged on the top of the substrate, the bottom of the substrate is arranged on the surface or top of the photoelectric conversion unit, and the metasurface includes at least one array composed of columnar structures, And the metasurface includes at least two media with different refractive indices, the photoelectric conversion unit includes an array for photoelectric conversion, the array of photoelectric conversion units is divided into a plurality of color regions, and the metasurface is used to refract incident light to the array of photoelectric conversion units in the corresponding color area.

For example, an array of photoelectric conversion units and an array of metasurface structures can be integrated and processed using a CMOS process. Specifically, for example, the metasurface can include an array composed of two media, titanium dioxide and air. After the array structure of the metasurface is determined, a columnar structure composed of titanium dioxide can be filled in the array.

Illustratively, step 1602 may include depositing a substrate on the array of photoelectric conversion units, and fabricating microstructures on the substrate.

Therefore, in the embodiment of the present application, a substrate can be prepared on the surface of the photoelectric conversion unit, and an array of metasurfaces can be prepared on the top of the substrate, and the array formed by the columnar structure in the metasurface can refract light of different colors, and the incident light is After being refracted by the metasurface, it is transmitted to the corresponding color region in the photoelectric conversion unit, thereby realizing light splitting. It can be understood that the spectroscopic color filter device has a medium metasurface or a medium diffractive surface, has the structural characteristics of a second-order two-dimensional code pattern, and has a variety of spectral channels to achieve the function of splitting multiple colors and achieving efficient light splitting. And the array on the metasurface structure can refract the incident light, reduce the scattering phenomenon, and improve the light utilization rate of the image sensor.

In a possible implementation manner, a color filter structure can also be prepared between the photoelectric conversion unit and the substrate, the color filter structure is divided into a plurality of color filter regions, and each color filter region covers a corresponding color region, and The color corresponding to each color area is the same as the color transmitted by the color filter area covered by each color area, and each color filter area is used to filter light of colors other than the color corresponding to the covered color area.

In one possible implementation, a lens may also be fabricated between each color filter region and the substrate.

In a possible implementation manner, the colors of light in multiple frequency bands corresponding to the multiple color regions include: one or more of green, red, blue, or infrared light.

In addition, before preparing the metasurface, the array structure of the metasurface can also be constructed. Exemplarily, referring to FIG. 17 , the flow of the image sensor fabrication method provided by the present application will be described in more detail below in conjunction with the method for constructing a metasurface array structure, as described below.

1701. Determine the structures of multiple arrays.

1702. Evaluate the structures of the multiple arrays by using the evaluation function to obtain an evaluation value corresponding to each of the multiple arrays.

1703. Determine that the plurality of arrays includes at least one array whose evaluation value is higher than the preset value, if yes, go to step 1705, if not, go to step 1704.

1704. Update multiple arrays.

1705. Select one of the arrays from the at least one array whose evaluation value is higher than a preset value as the structure of the metasurface of the spectroscopic color filter device.

For steps 1701-1705, reference may be made to the aforementioned steps 1401-1405, which will not be repeated here.

1706. Prepare a photoelectric conversion unit.

1707. Prepare a spectroscopic color filter device on the surface of the photoelectric conversion unit.

Wherein, steps 1706-1707 can refer to the aforementioned steps 1601-1602, which will not be repeated here.

Therefore, in the embodiments of the present application, before preparing the metasurface, the array structure of the metasurface can also be determined, thereby obtaining an image sensor with higher light utilization rate. It can be understood that, based on the set demand target of light utilization, an array that meets the demand target of light utilization can be obtained inversely through optimization algorithms such as genetic algorithm, simulated annealing algorithm or gradient descent, so as to improve the performance of spectroscopic color filter devices and The light utilization of the image sensor.

The light utilization rate of the image sensor mentioned in this application will be introduced in more detail below with more specific application scenarios.

Exemplarily, FIG. 18 is a spectrum diagram of the light utilization rate of the spectroscopic device in the visible light band range of 400-700 nm, the abscissa is the wavelength, and the ordinate is the transmittance. Tb represents the transmittance of light reaching the blue photosensitive pixel in the lower right corner, Tg and Tg2 represent the transmittance of light reaching the two photosensitive pixels in the upper right corner and the lower left corner, respectively, and Tr represents the transmittance reaching the red photosensitive pixel in the upper left corner. Rate. As shown in FIG. 18 , this embodiment increases the light utilization rate of visible light to 55.9%, which is about 224% of the light utilization rate of the conventional color filter. Among them, the utilization rate of red light is 73.7%, which is about 393% of the light utilization rate of traditional color filters; the utilization rate of green light is 47.9%, which is about 127% of the light utilization rate of traditional color filters; the utilization rate of blue light is 47.3%, which is about 252% of the light utilization rate of traditional color filters.

The light intensity distribution on a plane 3.5um away from the bottom of the dielectric light splitting device layer is shown in Figure 19. (a) (b) (c) correspond to wavelengths of 450nm, 536nm and 640nm respectively. It can be seen that the red, green and blue lights are The arrangement of RGGB is focused on the four photosensitive pixel positions of the lower right, lower left, upper right and upper left, respectively. The spectral crosstalk is eliminated by adding color filter layers, and the scattered light is less.

Obviously, according to the above analysis, the light utilization rate of the image sensor provided by the present application is obviously higher than that of the Bayer color filter or the nanofin structure. Therefore, compared with the Bayer color filter that obtains color information through the color filtering method, the embodiment of the present application uses a pixel-level spectral spectroscopic device to break through the theoretical limitation of light utilization rate of a single color filtering system through spectral splitting, thereby improving the color image in principle. Light utilization of the sensor. For the nano-fin structure metasurface technology, the present invention utilizes a function-driven reverse design algorithm to design a second-order two-dimensional code structure, which has the advantages of high spectral efficiency, low polarization dependence, and smaller matching photosensitive pixel elements. In addition, by improving the light utilization rate of the image sensor, the signal-to-noise ratio when the image sensor is used for shooting is improved, the quality of images shot under low light conditions is improved, and the photographing performance under low light conditions is improved.

Referring to Figure 20, the present application also provides a device for constructing an array structure for performing the method of Figure 14, the device may include:

a first determining unit 2001, configured to determine the structure of a plurality of arrays;

The evaluation unit 2002 is configured to evaluate the structure of the multiple arrays by using an evaluation function to obtain an evaluation value corresponding to each of the multiple arrays, and the evaluation function is to calculate the multiple arrays as the a function of the light utilization rate of the spectroscopic color filter element when the spectroscopic color filter element is metasurface;

The second determining unit 2003 is configured to determine the structure of the metasurface of the spectroscopic color filter device according to the evaluation value. The spectroscopic color filter device is included in an image sensor. The image sensor includes a spectroscopic color filter device and a photoelectric conversion unit. The color device includes a metasurface and a substrate, the photoelectric conversion unit includes an array for photoelectric conversion, the spectroscopic color filter device includes a metasurface and a substrate, the array in the metasurface is arranged on the top of the substrate, and the bottom of the substrate is arranged on the photoelectric The surface or top of the conversion unit, the metasurface includes an array composed of at least two media with different refractive indices, the array in the metasurface includes at least one array composed of columnar structures, the array of photoelectric conversion units is divided into multiple color regions, the metasurface Used to refract incident light to corresponding color regions in the array of photoelectric conversion units.

In a possible implementation manner, the first determining unit 2001 is specifically configured to, if the plurality of arrays includes at least one array whose evaluation value is higher than a preset value, select at least one array whose evaluation value is higher than the preset value from at least one array whose evaluation value is higher than the preset value. The structure of selecting one of the arrays as an array of metasurfaces of the spectroscopic color filter device;

The array structure construction device may further include: an updating unit 2004, configured to update the plurality of arrays if the arrays whose evaluation values are higher than a preset value are not included in the plurality of arrays;

In a possible implementation manner, the updating unit 2004 is specifically configured to determine, according to the values of the multiple evaluation values, a variation rate corresponding to each of the multiple arrays; The multiple arrays are mutated at a rate to obtain updated multiple arrays.

In a possible implementation manner, the updating unit 2004 is specifically configured to determine a probability value corresponding to each of the plurality of arrays according to the values of the plurality of evaluation values; according to the probability corresponding to each array value, sampling the multiple arrays multiple times to obtain multiple intermediate structures; determining the variation rate of the multiple intermediate structures according to the evaluation values of the multiple intermediate structures; according to the variation corresponding to the multiple intermediate structures The multiple intermediate structures are mutated at a rate to obtain new multiple arrays.

The above descriptions are only optional embodiments of the present application, and are not intended to limit the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application shall be included in the protection of the present application. within the range.

Claims

An image sensor, comprising: a metasurface, a substrate and a photoelectric conversion unit;

The metasurface includes at least one array of columnar structures, the arrays in the metasurface are arranged on the top of the substrate, the bottom of the substrate is arranged on the surface of the photoelectric conversion unit, and the metasurface includes at least two kinds of media with different refractive indices, the photoelectric conversion unit includes an array for photoelectric conversion, the array of photoelectric conversion units is divided into a plurality of color regions, the metasurface is used for refracting incident light and passing through the The substrates are transferred to corresponding color regions in the array of photoelectric conversion units.
The image sensor according to claim 1, wherein,

A color filter structure is also arranged between the photoelectric conversion unit and the substrate, the color filter structure is divided into a plurality of color filter regions, and each color filter region covers a corresponding color region, and each of the color filter regions covers a corresponding color region. The color corresponding to the color area is the same as the color transmitted by the color filter area covered by each color area.
The image sensor according to claim 2, wherein a lens is further provided between each of the color filter regions and the substrate.
The image sensor according to any one of claims 1-3, wherein the light of the plurality of frequency bands corresponding to the plurality of color regions comprises: one of green, red, blue or infrared light or variety.
The image sensor according to any one of claims 1-4, wherein the material of the metasurface comprises one or more of the following: titanium dioxide, gallium nitride or silicon carbide.
A spectroscopic color filter device, comprising: a metasurface and a substrate;

The metasurface includes at least one array of columnar structures, the arrays in the metasurface are arranged on top of the substrate, the metasurface includes at least two media with different refractive indices, and the metasurface is used for converting Refracts incident light, and the substrate is used to transmit light refracted by the metasurface.
The spectroscopic color filter device according to claim 6, wherein the spectroscopic color filter device is applied to an image sensor, the image sensor includes a photoelectric conversion unit, and the substrate is disposed on the surface of the photoelectric conversion unit , the photoelectric conversion unit includes an array for photoelectric conversion, the array of photoelectric conversion units is divided into a plurality of color regions, and the metasurface refracts incident light and transmits it to the photoelectric conversion unit through the substrate. The corresponding color area in the array.
The spectroscopic color filter device according to claim 6 or 7, characterized in that,

A color filter structure is also arranged between the photoelectric conversion unit and the substrate, the color filter structure is divided into a plurality of color filter regions, and each color filter region covers a corresponding color region, and each of the color filter regions covers a corresponding color region. The color corresponding to the color area is the same as the color transmitted by the color filter area covered by each color area.
A method for preparing an image sensor, comprising:

preparing a photoelectric conversion unit, the photoelectric conversion unit is used for converting an optical signal into an electrical signal, the photoelectric conversion unit includes an array for photoelectric conversion, and the array of the photoelectric conversion unit is divided into a plurality of color regions;

A spectroscopic color filter device is prepared on the surface of the photoelectric conversion unit, the spectroscopic color filter device includes a metasurface and a substrate, the metasurface includes at least one array of columnar structures, and the array in the metasurface is arranged in the The top of the substrate, the bottom of the substrate is arranged on the surface of the photoelectric conversion unit, the metasurface includes at least two kinds of media with different refractive indices, and the surface of the photoelectric conversion unit is the surface that receives the optical signal, The array of photoelectric conversion units is divided into a plurality of color regions, and the metasurface is used to refract incident light to corresponding color regions in the array of photoelectric conversion units.
The method according to claim 9, wherein before preparing the spectroscopic color filter device on the surface of the photoelectric conversion unit, the method further comprises:

determining a plurality of arrays, and using the arrays as the structure of the metasurface of the spectroscopic color filter device to obtain a variety of spectroscopic structures;

A plurality of evaluation values corresponding to the various spectroscopic structures one-to-one are obtained through a preset evaluation function, where the evaluation function is a function for calculating the light utilization rate of the spectroscopic structure;

If the plurality of evaluation values include at least one evaluation value higher than a preset value, a first spectral structure is selected from the plurality of spectral structures as the structure of the spectral color filter device, and the first spectral structure is selected. The evaluation value of the structure is higher than the preset value.
The method of claim 10, wherein the method further comprises:

If the multiple evaluation values do not include at least one evaluation value higher than the preset value, multiple arrays are re-determined, and a spectroscopic structure is determined as the structure of the spectroscopic color filter device according to the re-determined multiple arrays.
The method of claim 11, wherein the re-determining a plurality of arrays comprises:

determining a mutation rate corresponding to each of the plurality of arrays according to the values of the plurality of evaluation values;

The multiple arrays are mutated according to the mutation rate corresponding to each of the arrays to obtain updated multiple arrays.
The method of claim 11, wherein the re-determining a plurality of arrays comprises:

determining a probability value corresponding to each of the plurality of arrays according to the values of the plurality of evaluation values;

According to the probability value corresponding to each array, sampling the plurality of arrays multiple times to obtain a plurality of intermediate structures;

determining the variation rates of the plurality of intermediate structures according to the evaluation values of the plurality of intermediate structures;

The multiple intermediate structures are mutated according to the mutation rates corresponding to the multiple intermediate structures to obtain multiple new arrays.
The method according to any one of claims 9-13, wherein the method further comprises:

A color filter structure is prepared between the photoelectric conversion unit and the substrate, the color filter structure is divided into a plurality of color filter areas, and each color filter area covers a corresponding color area, and each color filter area The color corresponding to the area is the same as the color transmitted by the color filter area covered by each color area, and each color filter area is used to filter light of colors other than the color corresponding to the covered color area.
The method of claim 14, wherein the method further comprises:

A lens is fabricated between each of the color filter regions and the substrate.
The method according to any one of claims 9-15, wherein the light of multiple frequency bands corresponding to the multiple color regions comprises: one or more of green, red, blue or infrared light kind.
The method according to any one of claims 9-16, wherein the material of the metasurface comprises one or more of the following: titanium dioxide, gallium nitride or silicon carbide.
An electronic device, characterized in that, the electronic device comprises the image sensor according to any one of claims 1-5.