CN114068598A

CN114068598A - Image sensor, image pickup apparatus, and display device

Info

Publication number: CN114068598A
Application number: CN202111155234.XA
Authority: CN
Inventors: 代郁峰; 汪立; 邹松
Original assignee: Huawei Technologies Co Ltd
Current assignee: Huawei Technologies Co Ltd
Priority date: 2021-09-29
Filing date: 2021-09-29
Publication date: 2022-02-18
Also published as: WO2023051475A1

Abstract

The embodiment of the application provides an image sensor, camera equipment and a display device, relates to the technical field of optics, and is used for solving the problem that the photon utilization rate of the image sensor or the display device is low due to the existing color filter. The image sensor comprises a color filter, a first photodiode, a second photodiode, a third photodiode and a light splitting structure; the color filter comprises a first color filter unit and a second color filter unit; the first color filter unit allows the first color light and the second color light in the three primary color light to transmit, and the second color filter unit allows the second color light and the third color light in the three primary color light to transmit; the light splitting structure is used for transmitting the first color light transmitted by the first color filter unit to the first photodiode, transmitting the second color light to the second photodiode, transmitting the second color light transmitted by the second color filter unit to the second photodiode, and transmitting the third color light to the third photodiode.

Description

Image sensor, image pickup apparatus, and display device

Technical Field

The present application relates to the field of optical technologies, and in particular, to an image sensor, an image pickup apparatus, and a display device.

Background

Currently, a Color Filter (CF) is widely used in an optical device, such as an image sensor or a display device, and the existing color filter CF includes a red filter unit (red, R), a green filter unit (green, G), and a blue filter unit (blue, B). Taking the application of color filters to image sensors as an example, the image sensors are important components of image pickup apparatuses and can convert optical images into digital signals. As shown in fig. 1, the conventional image sensor mainly includes a color filter CF in which each of a red filter unit R, a green filter unit G, and a blue filter unit B is disposed corresponding to one photodiode PD, and a plurality of Photodiodes (PDs). The red filter unit R, the green filter unit G and the blue filter unit B in the color filter CF filter the natural light sent by the lens to respectively obtain red light R, green light G and blue light B. Each photodiode PD is configured to receive light filtered by the corresponding red filter unit R, green filter unit G, or blue filter unit B, and convert an optical signal corresponding to the light into an electrical signal, where the electrical signal is used for processing by an image processor to restore the optical signal into an R/G/B full-color image.

However, in both the image sensor and the display device, since the red filter unit R, the green filter unit G, and the blue filter unit B of the color filter CF can transmit only one color of light and absorb the other two colors of light, 2/3 of photons in the natural light transmitted from the lens are absorbed by the color filter CF and only 1/3 of photons is received by the photodiode PD for the image sensor; for a display device, 2/3 photons in the white light provided by the backlight are absorbed by the color filter CF, and only 1/3 photons are received by the viewer, which results in a low utilization of photons.

Disclosure of Invention

The embodiment of the application provides an image sensor, camera equipment and a display device, and is used for solving the problem that the photon utilization rate of the image sensor or the display device is low due to the existing color filter.

In order to achieve the purpose, the technical scheme is as follows:

in a first aspect, an image sensor is provided. The image sensor comprises a color filter, a photodiode array and a light splitting structure, wherein the color filter comprises a plurality of first color filter units and a plurality of second color filter units; at least any one of the plurality of first color filter units allows the first color light and the second color light of the three primary color light to transmit, and at least any one of the plurality of second color filter units allows the second color light and the third color light of the three primary color light to transmit. The photodiode array is arranged on the light-emitting side of the color filter and comprises a plurality of first photodiodes, a plurality of second photodiodes and a plurality of third photodiodes. The light splitting structure is arranged between the color filter and the photodiode array; the light splitting structure is used for separating the first color light and the second color light which are transmitted by the first color filter unit, transmitting the first color light to the first photodiode, transmitting the second color light to the second photodiode, and separating the second color light and the third color light which are transmitted by the second color filter unit, transmitting the second color light to the second photodiode, and transmitting the third color light to the third photodiode. In this application, "at least any one" means one or more.

Because the first color filter unit and the second color filter unit of the color filter in the image sensor can both allow light of two colors to transmit, specifically, the first color filter unit allows the light of the first color and the light of the second color to transmit, and the second color filter unit allows the light of the second color and the light of the third color to transmit, compared with the red filter unit, the green filter unit and the blue filter unit of the color filter in the image sensor in the prior art, the light of one color can only be transmitted, so the light-entering amount can be improved, and the photon utilization rate can also be improved. In addition, compared to the case that the color filter in the image sensor in the prior art includes a red filter unit, a green filter unit, and a blue filter unit, since the area of the first color filter unit or the second color filter unit corresponding to the first photodiode, the second photodiode, and the third photodiode is 2 times larger than the area of the red filter unit, the green filter unit, and the blue filter unit corresponding to the first photodiode, the second photodiode, and the third photodiode in the prior art in the present application, the amount of incident light received by the first photodiode, the second photodiode, and the third photodiode in the present application is increased by 2 times.

On the basis, compared with the case that the color filter in the image sensor in the prior art comprises the red filter unit, the green filter unit and the blue filter unit, since each first photodiode, each second photodiode and each third photodiode, namely each first color pixel, each second color pixel and each third color pixel in the pixel matrix all receive the light which is distributed by the white light in the pixel space corresponding to the two color filter units in the prior art, only a little of spatial resolution is lost, and the spatial resolution is 70% of the prior art.

In addition, in the image sensor provided by the application, because the first photodiode receives the first color light of the three primary color lights, the second photodiode receives the second color light of the three primary color lights, and the third photodiode receives the third color light of the three primary color lights, all the three primary color lights can be received by the image sensor, and the image sensor is beneficial to the post-image processing. In addition, the first photodiode, the second photodiode and the third photodiode receive light of a single color, specifically, the first photodiode receives light of a first color, the second photodiode receives light of a second color, and the third photodiode receives light of a third color, that is, the image sensor includes pixels of the first color, the second color and the third color, and compared with the case that the image sensor includes pixels W + R, pixels W-R, pixels W + B and pixels W-B, the difficulty of algorithm processing in the later stage of the application is reduced, and noise is not increased.

In a possible embodiment, the light splitting structure is further configured to converge the first color light transmitted from the first color filter unit and the second color light, and the light splitting structure is further configured to converge the second color light transmitted from the second color filter unit and the third color light. Due to the converging function of the light splitting structure, more first color light can be transmitted to the first photodiode, and more second color light can be transmitted to the second photodiode, so that the first photodiode can receive more first color light, and the second photodiode can receive more second color light, and the light quantity and the light intensity received by the first photodiode and the second photodiode can be improved. Likewise, the converging function of the light splitting structure enables the third photodiode to receive more light of the third color, thereby increasing the amount and intensity of light received by the third photodiode.

In one possible embodiment, the image sensor further includes a plurality of microlenses disposed on the light-incident side of the color filter; the micro lens is used for transmitting the three primary color lights to the first color filter unit and the second color filter unit after converging the three primary color lights. The plurality of micro lenses arranged on the light incident side of the color filter can converge the three primary color lights and transmit the three primary color lights to the first color filter unit and the second color filter unit, so that the light quantity and the light intensity transmitted to the color filter can be improved, and the light quantity and the light intensity transmitted to the color filter can be improved, so that the light quantity and the light intensity received by the first photodiode, the second photodiode and the third photodiode are increased.

In one possible embodiment, the plurality of first photodiodes, the plurality of second photodiodes, and the plurality of third photodiodes are used to form a plurality of 2 × 2 first photodiode matrices repeatedly arranged in the first direction and the second direction; the first direction is a row direction of the first photodiode matrix, and the second direction is a column direction of the first photodiode matrix. Each first photodiode matrix comprises one first photodiode, two second photodiodes and one third photodiode. Arranging the plurality of first photodiodes, the plurality of second photodiodes, and the plurality of third photodiodes in this manner facilitates post-image processing.

In one possible embodiment, the first photodiode is located in the upper left corner of the first photodiode matrix, the two second photodiodes are located in the upper right corner and the lower left corner of the first photodiode matrix, respectively, and the third photodiode is located in the lower right corner of the first photodiode matrix. When the plurality of first photodiodes, the plurality of second photodiodes, and the plurality of third photodiodes are arranged in this manner, they may be arranged in a bayer array.

In one possible embodiment, the plurality of first color filter units and the plurality of second color filter units are alternately arranged in a direction parallel to two diagonal lines of the first photodiode matrix; the first color filter unit covers the first photodiode and the part of the second photodiode positioned in the four neighborhoods of the first photodiode; the second color filter unit covers the third photodiode and the part of the second photodiode positioned in the four neighborhoods of the third photodiode; each second photodiode is covered by two adjacent first color filter units and two adjacent second color filter units. According to the arrangement mode of the first color filter unit and the second color filter unit and the corresponding design mode of the first color filter unit and the second color filter unit with the first photodiode, the second photodiode and the third photodiode, the first color light transmitted by the first color filter unit can be transmitted to the first photodiode, the second color light can be transmitted to the second photodiode, the second color light transmitted by the second color filter unit can be transmitted to the second photodiode, and the third color light can be transmitted to the third photodiode.

In one possible embodiment, the light splitting structure comprises a plurality of light splitting cells; each first photodiode, each second photodiode and each third photodiode correspond to four light splitting units; each first photodiode is used for receiving the first color light split by the four light splitting units corresponding to the first photodiode, each second photodiode is used for receiving the second color light split by the four light splitting units corresponding to the second photodiode, and each third photodiode is used for receiving the third color light split by the four light splitting units corresponding to the third photodiode. Each light splitting unit corresponds to one first color filter unit or one second color filter unit; the light splitting unit corresponding to the first color filter unit receives the first color light and the second color light transmitted by the first color filter unit and transmits the first color light and the second color light to a first photodiode and a second photodiode separately; the light splitting unit corresponding to the second color filter unit receives the second color light and the third color light transmitted by the second color filter unit, and splits and transmits the second color light and the third color light to a second photodiode and a third photodiode.

In one possible embodiment, the light splitting structure comprises a plurality of light splitting cells; each first photodiode, each second photodiode and each third photodiode correspond to four light splitting units; each first photodiode is used for receiving the first color light split by the four light splitting units corresponding to the first photodiode, each second photodiode is used for receiving the second color light split by the four light splitting units corresponding to the second photodiode, and each third photodiode is used for receiving the third color light split by the four light splitting units corresponding to the third photodiode. Each light splitting unit corresponds to the first color filter unit and the second color filter unit. Each light splitting unit receives the first color light and the second color light transmitted from the first color filter unit, and transmits the first color light and the second color light to one first photodiode and two second photodiodes separately, and receives the second color light and the third color light transmitted from the second color filter unit, and transmits the second color light and the third color light to one third photodiode and two second photodiodes separately; alternatively, each of the light splitting units receives the first color light and the second color light transmitted from the first color filter unit, and receives the second color light and the third color light transmitted from the second color filter unit, and transmits the first color light to one of the first photodiodes, the second color light to one of the second photodiodes, and the third color light to one of the third photodiodes.

In one possible embodiment, the light splitting structure comprises a plurality of light splitting cells; each first photodiode and each third photodiode correspond to four light splitting units; each first photodiode is used for receiving the first color light divided by the four light-splitting units corresponding to the first photodiode, and each third photodiode is used for receiving the third color light divided by the four light-splitting units corresponding to the third photodiode; each second photodiode corresponds to one light splitting unit; each second photodiode is used for receiving the second color light split by the corresponding light splitting unit. Each light splitting unit corresponds to two opposite first color filter units and two opposite second color filter units, receives light transmitted by the two opposite first color filter units and the two opposite second color filter units, transmits second color light to one second photodiode, transmits first color light to the two opposite first photodiodes, and transmits third color light to the two opposite third photodiodes.

In one possible implementation mode, the light splitting structure comprises a plurality of micro-nano medium columns distributed in an array. The micro-nano medium columns do not absorb light, phase delay can occur when light irradiates the micro-nano medium columns, the phases of a plurality of micro-nano medium columns are continuously changed, the direction angles of the light can be changed, and the direction angles of the light with different wavelengths after passing through the micro-nano medium columns are different, so that the separation of the light with different wavelengths can be realized.

In one possible embodiment, the first color filter unit is a yellow filter unit, and the second color filter unit is a cyan filter unit. After the three primary colors of light pass through the yellow filter unit, only red light and green light can pass through, and blue light can be absorbed.

In one possible embodiment, the first color light is red light, the second color light is green light, and the third color light is blue light. Red light, green light, and blue light constitute three primary colors of light, and thus post-image processing can form a color image.

In a second aspect, an image sensor is provided. The image sensor includes a color filter, a photodiode array, and a light-splitting structure. The color filter comprises a plurality of first color filter units, a plurality of second color filter units and a plurality of third color filter units; at least any one of the plurality of first color filter units allows first color light of the infrared light and the three primary colors to pass through, at least any one of the plurality of second color filter units allows second color light of the infrared light and the three primary colors to pass through, and at least any one of the plurality of third color filter units allows third color light of the infrared light and the three primary colors to pass through. The photodiode array is arranged on the light-emitting side of the color filter and comprises a plurality of first photodiodes, a plurality of second photodiodes, a plurality of third photodiodes and a plurality of fourth photodiodes. The light splitting structure is arranged between the color filter and the first photodiode, the second photodiode, the third photodiode and the fourth photodiode; the light splitting structure is used for separating first color light and infrared light transmitted by the first color filter unit, separating second color light and infrared light transmitted by the second color filter unit, separating third color light and infrared light transmitted by the third color filter unit, transmitting the first color light to the first photodiode, transmitting the second color light to the second photodiode, transmitting the third color light to the third photodiode, and transmitting the infrared light to the fourth photodiode.

In the application, the first color filter unit, the second color filter unit and the third color filter unit in the color filter can pass infrared light and light of another color, so that the light-entering amount can be improved, and the photon utilization rate can also be improved. In addition, compared to the case that the color filter in the image sensor in the prior art includes a red filter unit, a green filter unit, and a blue filter unit, in the present application, the amount of incident light received by the first photodiode, the second photodiode, the third photodiode, and the fourth photodiode is increased by 2 times. On this basis, compared with the case that the color filter in the image sensor in the prior art includes a red filter unit, a green filter unit and a blue filter unit, since each first photodiode, each second photodiode, each third photodiode and each fourth photodiode, that is, each first color pixel, each second color pixel, each third color pixel and each fourth third color pixel in the pixel matrix all receive the light after the white light distribution in the pixel space corresponding to the two color filter units in the prior art, only a little spatial resolution is lost, and the spatial resolution is 70% of the prior art. In addition, in the application, the first photodiode, the second photodiode, the third photodiode and the fourth photodiode can receive light of three primary colors, and the received light is light of a single color, so that the difficulty of post-algorithm processing can be reduced. In addition, since the image sensor provided by the second aspect is also used for receiving infrared light, imaging can be performed by using infrared light, so that an image obtained by post-image processing is more accurate.

In one possible embodiment, the light splitting structure is further configured to converge the first color light transmitted from the first color filter unit, converge the second color light transmitted from the second color filter unit, converge the third color light transmitted from the third color filter unit, and converge the infrared light transmitted from the first, second, and third color filter units. Reference may be made to the description of the related technical effects of the first aspect, which is not repeated herein.

In one possible embodiment, the image sensor further includes a plurality of microlenses disposed on the light-incident side of the color filter; the micro lens is used for converging the three primary colors and the infrared light and transmitting the converged light to the first color filter unit, the second color filter unit and the third color filter unit. Reference may be made to the description of the related technical effects of the first aspect, which is not repeated herein.

In one possible embodiment, the image sensor further includes a first infrared cut filter disposed between the light splitting structure and the first, second, and third photodiodes; the first infrared cut-off filter is used for filtering infrared light; the first infrared cut-off filter comprises a plurality of hollow areas, and the projection of the fourth photodiode on the first infrared cut-off filter and the hollow areas have overlapped areas. Because the first infrared cut-off filter is arranged between the light splitting structure and the photodiode array, infrared light transmitted from the light splitting structure can be absorbed by the first infrared cut-off filter in the process of being transmitted to the first photodiode, the second photodiode and the third photodiode, so that the first photodiode, the second photodiode and the third photodiode can be prevented from receiving the infrared light, the first photodiode, the second photodiode and the third photodiode can be ensured to receive light with single color, and the accuracy of later-stage image processing is facilitated. The first infrared cut-off filter comprises a hollow-out area, and the projection of the fourth photodiode on the first infrared cut-off filter and the hollow-out area have an overlapped area, so that infrared light transmitted from the light splitting structure can be transmitted to the fourth photodiode and received by the fourth photodiode.

In one possible embodiment, a first color filter unit covers a first photodiode and a portion of a fourth photodiode located in a fourth neighborhood of the first photodiode, a second color filter unit covers a second photodiode and a portion of a fourth photodiode located in a fourth neighborhood of the second photodiode, and a third color filter unit covers a third photodiode and a portion of a fourth photodiode located in a fourth neighborhood of the third photodiode. The design can realize that the first color light and the infrared light transmitted by the first color filter unit are respectively transmitted to the first photodiode and the fourth photodiode, and the second color light and the infrared light transmitted by the second color filter unit are respectively transmitted to the second photodiode and the fourth photodiode. The third color light and the infrared light transmitted from the third color filter unit are transmitted to the third photodiode and the fourth photodiode, respectively.

In one possible embodiment, the row of first photodiodes, the row of fourth photodiodes, the row of second photodiodes, the row of fourth photodiodes, the row of third photodiodes, and the row of fourth photodiodes are alternately arranged in sequence along the first direction. The arrangement of the plurality of first photodiodes, the plurality of second photodiodes, the plurality of third photodiodes, and the plurality of fourth photodiodes in this manner is advantageous for post-image processing.

In one possible embodiment, the plurality of first photodiodes, the plurality of second photodiodes, the plurality of third photodiodes, and the plurality of fourth photodiodes are used to form a plurality of 2 × 2 second photodiode matrices and 2 × 2 third photodiode matrices alternately arranged in the first direction and the second direction; the first direction is a row direction of the second photodiode matrix and the third photodiode matrix, and the second direction is a column direction of the second photodiode matrix and the third photodiode matrix. Each second photodiode matrix comprises a first photodiode, a second photodiode and two fourth photodiodes; each third photodiode matrix comprises one second photodiode, one third photodiode and two fourth photodiodes. The arrangement of the plurality of first photodiodes, the plurality of second photodiodes, the plurality of third photodiodes, and the plurality of fourth photodiodes in this manner is advantageous for post-image processing.

In one possible embodiment, in the second photodiode matrix, two fourth photodiodes are respectively located at the upper left corner and the lower right corner of the second photodiode matrix, and the first photodiode and the second photodiode are respectively located at the upper right corner and the lower left corner of the second photodiode matrix; in the third photodiode matrix, two fourth photodiodes are located at the upper left corner and the lower right corner of the third photodiode matrix, respectively, and the second photodiode and the third photodiode are located at the lower left corner and the upper right corner of the third photodiode matrix, respectively.

In one possible embodiment, the light splitting structure includes a plurality of light splitting units, each of the first photodiodes, each of the second photodiodes, each of the third photodiodes, and each of the fourth photodiodes correspond to four light splitting units; each first photodiode is used for receiving the first color light split by the four light splitting units corresponding to the first photodiode, each second photodiode is used for receiving the second color light split by the four light splitting units corresponding to the second photodiode, each third photodiode is used for receiving the third color light split by the four light splitting units corresponding to the third photodiode, and each fourth photodiode is used for receiving the infrared light split by the four light splitting units corresponding to the fourth photodiode. Each light splitting unit corresponds to one of the first color filter unit, the second color filter unit and the third color filter unit; the light splitting unit corresponding to the first color filter unit receives the first color light and the infrared light transmitted by the first color filter unit and separately transmits the first color light and the infrared light to the first photodiode and the fourth photodiode; the light splitting unit corresponding to the second color filter unit receives the second color light and the infrared light transmitted from the second color filter unit and separately transmits the second color light and the infrared light to the second photodiode and the fourth photodiode; the light splitting unit corresponding to the third color filter unit receives the third color light and the infrared light transmitted from the third color filter unit and separately transmits the third color light and the infrared light to the third photodiode and the fourth photodiode.

In one possible embodiment, the light splitting structure includes a plurality of light splitting units, and each of the first photodiode, each of the second photodiodes, and each of the third photodiodes corresponds to one light splitting unit; each first photodiode is used for receiving the first color light split by one light splitting unit corresponding to the first photodiode, each second photodiode is used for receiving the second color light split by one light splitting unit corresponding to the second photodiode, and each third photodiode is used for receiving the third color light split by one light splitting unit corresponding to the third photodiode; each fourth photodiode corresponds to four light splitting units and is used for receiving the infrared light split by any one of the four light splitting units corresponding to the fourth photodiode. Each light splitting unit corresponds to one of the first color filter unit, the second color filter unit and the third color filter unit; the light splitting unit corresponding to the first color filter unit receives the first color light and the infrared light transmitted by the first color filter unit and respectively transmits the first color light and the infrared light to the first photodiode and any fourth photodiode in the fourth neighborhood; the light splitting unit corresponding to the second color filter unit receives the second color light and the infrared light transmitted by the second color filter unit and respectively transmits the second color light and the infrared light to the second photodiode and any fourth photodiode in the fourth neighborhood; the light splitting unit corresponding to the third color filtering unit receives the third color light and the infrared light transmitted by the third color filtering unit, and transmits the third color light and the infrared light to the third photodiode and any fourth photodiode in the four neighborhoods thereof respectively.

In one possible embodiment, the light splitting structure includes a plurality of light splitting units, and each of the first photodiodes, each of the second photodiodes, and each of the third photodiodes corresponds to four light splitting units; each first photodiode is used for receiving the first color light split by the four light splitting units corresponding to the first photodiode, each second photodiode is used for receiving the second color light split by the four light splitting units corresponding to the second photodiode, each third photodiode is used for receiving the third color light split by the four light splitting units corresponding to the third photodiode, and each fourth photodiode corresponds to one light splitting unit and is used for receiving the infrared light split by the light splitting unit corresponding to the fourth photodiode. Each of the light splitting units corresponds to two groups of the two first color filter units, the two second color filter units, and the two third color filter units, for example, the light splitting units correspond to the two first color filter units and the two second color filter units. The light splitting units corresponding to the two first color filter units and the two second color filter units receive the first color light and the infrared light transmitted from the two first color filter units, receive the second color light and the infrared light transmitted from the two second color filter units, transmit the first color light to the two first photodiodes, transmit the second color light to the two second photodiodes, and transmit the infrared light to the fourth photodiode. The light splitting units corresponding to the two second color filter units and the two third color filter units receive the second color light and the infrared light transmitted from the two second color filter units, receive the third color light and the infrared light transmitted from the two third color filter units, transmit the second color light to the two second photodiodes, transmit the third color light to the two third photodiodes, and transmit the infrared light to one fourth photodiode. The light splitting units corresponding to the two third color filter units and the two first color filter units receive the third color light and the infrared light transmitted by the two third color filter units, receive the first color light and the infrared light transmitted by the two first color filter units, transmit the first color light to the two first photodiodes, transmit the third color light to the two third photodiodes, and transmit the infrared light to the fourth photodiode.

In one possible embodiment, the light splitting structure includes a plurality of light splitting units, and each of the first photodiodes, each of the second photodiodes, and each of the third photodiodes corresponds to four light splitting units; each first photodiode is used for receiving the first color light split by the four light splitting units corresponding to the first photodiode, each second photodiode is used for receiving the second color light split by the four light splitting units corresponding to the second photodiode, each third photodiode is used for receiving the third color light split by the four light splitting units corresponding to the third photodiode, and each fourth photodiode corresponds to one light splitting unit and is used for receiving the infrared light split by the light splitting unit corresponding to the fourth photodiode. Each light splitting unit corresponds to one first color filter unit, two second color filter units and one third color filter unit; each light splitting unit receives the first color light transmitted by the corresponding first color filter unit, transmits the first color light to one first photodiode, receives the second color light transmitted by the corresponding second color filter unit, transmits the second color light to two second photodiodes, receives the third color light transmitted by the corresponding third color filter unit, and transmits the third color light to one third photodiode; each light splitting unit also receives the infrared light transmitted by the first color filter unit, the second color filter unit and the third color filter unit and transmits the infrared light to the fourth photodiode.

In one possible embodiment, the light splitting structure includes a plurality of light splitting units, each of the first photodiodes, each of the second photodiodes, each of the third photodiodes, and each of the fourth photodiodes correspond to four light splitting units; each first photodiode is used for receiving the first color light split by the four light splitting units corresponding to the first photodiode, each second photodiode is used for receiving the second color light split by the four light splitting units corresponding to the second photodiode, each third photodiode is used for receiving the third color light split by the four light splitting units corresponding to the third photodiode, and each fourth photodiode is used for receiving the infrared light split by the four light splitting units corresponding to the fourth photodiode. Each light splitting unit corresponds to any one group of two first color filter units, two second color filter units, two third color filter units, a first color filter unit, a second color filter unit, a third color filter unit and a first color filter unit, for example, the light splitting unit corresponds to two first color filter units; for another example, the light splitting unit corresponds to the first color filter unit and the second color filter unit. The light splitting units corresponding to the two first color filter units receive the first color light and the infrared light transmitted by the first color filter units, transmit the first color light to the two first photodiodes, and transmit the infrared light to the two fourth photodiodes; the light splitting units corresponding to the two second color filter units receive the second color light and the infrared light transmitted by the second color filter units, transmit the second color light to the two second photodiodes, and transmit the infrared light to the two fourth photodiodes; the light splitting units corresponding to the two third color filter units receive the third color light and the infrared light transmitted by the third color filter units, transmit the third color light to the two third photodiodes, and transmit the infrared light to the two fourth photodiodes. The light splitting units corresponding to the first color filter unit and the second color filter unit receive the first color light and the infrared light transmitted from the first color filter unit, and receive the second color light and the infrared light transmitted from the second color filter unit, and transmit the first color light to the first photodiode, the second color light to the second photodiode, and the infrared light to the two fourth photodiodes. The light splitting units corresponding to the second color filter unit and the third color filter unit receive the second color light and the infrared light transmitted from the second color filter unit, and receive the third color light and the infrared light transmitted from the third color filter unit, and transmit the second color light to the second photodiode, the third color light to the third photodiode, and the infrared light to the two fourth photodiodes. The light splitting units corresponding to the third color filter units and the first color filter units receive the third color light and the infrared light transmitted by the two third color filter units, receive the first color light and the infrared light transmitted by the two first color filter units, transmit the first color light to the first photodiode, transmit the third color light to the third photodiode, and transmit the infrared light to the fourth photodiode.

In one possible embodiment, the light splitting structure includes a plurality of light splitting units, each of the first photodiodes, each of the second photodiodes, each of the third photodiodes, and each of the fourth photodiodes correspond to four light splitting units; each first photodiode is used for receiving the first color light split by the four light splitting units corresponding to the first photodiode, each second photodiode is used for receiving the second color light split by the four light splitting units corresponding to the second photodiode, each third photodiode is used for receiving the third color light split by the four light splitting units corresponding to the third photodiode, and each fourth photodiode is used for receiving the infrared light split by the four light splitting units corresponding to the fourth photodiode. Each light splitting unit corresponds to the first color filter unit and the second color filter unit, or corresponds to the second color filter unit and the third color filter unit; the light splitting units corresponding to the first color filter unit and the second color filter unit receive the first color light and the infrared light transmitted from the first color filter unit, and receive the second color light and the infrared light transmitted from the second color filter unit, and transmit the first color light to the first photodiode, the second color light to the second photodiode, and the infrared light to the two fourth photodiodes; the light splitting units corresponding to the second color filter unit and the third color filter unit receive the second color light and the infrared light transmitted from the second color filter unit, and receive the third color light and the infrared light transmitted from the third color filter unit, and transmit the second color light to the second photodiode, the third color light to the third photodiode, and the infrared light to the two fourth photodiodes.

In a third aspect, an image sensor is provided. The image sensor includes a color filter, a photodiode array, and a light-splitting structure. The color filter comprises a plurality of first color filter units, a plurality of second color filter units, a plurality of third color filter units and a plurality of hollow areas; at least any one of the first color filter units allows the first color light of the three primary colors to transmit, at least any one of the second color filter units allows the second color light of the three primary colors to transmit, and at least any one of the third color filter units allows the third color light of the three primary colors to transmit. The photodiode array is arranged on the light-emitting side of the color filter and comprises a plurality of first photodiodes corresponding to the first color filter units, a plurality of second photodiodes corresponding to the second color filter units, a plurality of third photodiodes corresponding to the third color filter units and a plurality of fourth photodiodes corresponding to the hollow areas. The light splitting structure is arranged on the light incident side of the color filter; the light splitting structure is used for splitting the infrared light and the three primary colors of light, the infrared light is transmitted to the fourth photodiode through the hollow area of the color filter, and the three primary colors of light are transmitted to the first color filter unit, the second color filter unit and the third color filter unit.

In the application, the light splitting structure firstly separates the three primary colors from the infrared light, the three primary colors are transmitted to the first color filter unit, the second color filter unit and the third color filter unit of the color filter, and the infrared light is transmitted to the fourth photodiode through the hollow area of the color filter. On this basis, compared with the case that the color filter in the image sensor in the prior art includes a red filter unit, a green filter unit and a blue filter unit, since each first photodiode, each second photodiode, each third photodiode and each fourth photodiode, that is, each first color pixel, each second color pixel, each third color pixel and each fourth third color pixel in the pixel matrix all receive the light after the white light distribution in the pixel space corresponding to the two color filter units in the prior art, only a little spatial resolution is lost, and the spatial resolution is 70% of the prior art. In addition, in the application, the first photodiode, the second photodiode, the third photodiode and the fourth photodiode can receive three primary colors of light, and the received lights are all single-color lights, so that the difficulty of post-algorithm processing can be reduced.

In one possible embodiment, the row of first photodiodes, the row of fourth photodiodes, the row of second photodiodes, the row of fourth photodiodes, the row of third photodiodes, and the row of fourth photodiodes are alternately arranged in sequence along the first direction. Reference may be made to the above description of related art effects, which are not repeated herein.

In one possible embodiment, the plurality of first photodiodes, the plurality of second photodiodes, the plurality of third photodiodes, and the plurality of fourth photodiodes are used to form a plurality of 2 × 2 second photodiode matrices and 2 × 2 third photodiode matrices alternately arranged in the first direction and the second direction; the first direction is a row direction of the second photodiode matrix and the third photodiode matrix, and the second direction is a column direction of the second photodiode matrix and the third photodiode matrix. Each second photodiode matrix comprises a first photodiode, a second photodiode and two fourth photodiodes; each third photodiode matrix comprises one second photodiode, one third photodiode and two fourth photodiodes. Reference may be made to the above description of related art effects, which are not repeated herein.

In one possible embodiment, in the second photodiode matrix, two fourth photodiodes are respectively located at the upper left corner and the lower right corner of the second photodiode matrix, and the first photodiode and the second photodiode are respectively located at the upper right corner and the lower left corner of the second photodiode matrix; in the third photodiode matrix, two fourth photodiodes are located at the upper left corner and the lower right corner of the third photodiode matrix, respectively, and the second photodiode and the third photodiode are located at the lower left corner and the upper right corner of the third photodiode matrix, respectively. Reference may be made to the above description of related art effects, which are not repeated herein.

In one possible embodiment, the light splitting structure includes a plurality of light splitting units, and each of the first photodiodes, each of the second photodiodes, and each of the third photodiodes corresponds to four light splitting units; each first photodiode is used for receiving first color light which is split by four light splitting units corresponding to the first photodiode and passes through the first color filter unit, each second photodiode is used for receiving second color light which is split by four light splitting units corresponding to the second photodiode and passes through the second color filter unit, each third photodiode is used for receiving third color light which is split by four light splitting units corresponding to the third photodiode and passes through the third color filter unit, and each fourth photodiode corresponds to one light splitting unit and is used for receiving infrared light which is split by the light splitting unit corresponding to the fourth photodiode.

In a fourth aspect, an image sensor is provided. The image sensor comprises a photodiode array and a light splitting structure, wherein the photodiode array comprises a plurality of first photodiodes, a plurality of second photodiodes, a plurality of third photodiodes and a plurality of fourth photodiodes; the light splitting structure is arranged on the light incident sides of the first photodiodes, the second photodiodes, the third photodiodes and the fourth photodiodes; the light splitting structure is used for transmitting the received first color light to the first photodiode, transmitting the received second color light to the second photodiode, transmitting the received third color light to the third photodiode, and transmitting the received infrared light to the fourth photodiode.

Because the light splitting structure directly transmits first color light to the first photodiode, second color light is transmitted to the second photodiode, third color light is transmitted to the third photodiode, infrared light is transmitted to the fourth photodiode, and the light splitting structure does not absorb light, so that the light incoming amount of the image sensor can be improved. On this basis, compared with the case that the color filter in the image sensor in the prior art includes a red filter unit, a green filter unit and a blue filter unit, since each first photodiode, each second photodiode, each third photodiode and each fourth photodiode, that is, each first color pixel, each second color pixel, each third color pixel and each fourth third color pixel in the pixel matrix all receive the light after the white light distribution in the pixel space corresponding to the two color filter units in the prior art, only a little spatial resolution is lost, and the spatial resolution is 70% of the prior art. In addition, in the application, the first photodiode, the second photodiode, the third photodiode and the fourth photodiode can receive three primary colors of light, and the received lights are all single-color lights, so that the difficulty of post-algorithm processing can be reduced.

In one possible embodiment, the plurality of first photodiodes, the plurality of second photodiodes, the plurality of third photodiodes, and the plurality of fourth photodiodes are used to form a plurality of 2 × 2 fourth photodiode matrices repeatedly arranged in the first direction and the second direction; the first direction is a row direction of the fourth photodiode matrix, and the second direction is a column direction of the fourth photodiode matrix. Each fourth photodiode matrix includes a first photodiode, a second photodiode, a third photodiode, and a fourth photodiode. The arrangement of the plurality of first photodiodes, the plurality of second photodiodes, the plurality of third photodiodes, and the plurality of fourth photodiodes in this manner is advantageous for post-image processing.

In one possible embodiment, in the fourth photodiode matrix, the first photodiode is located at the upper left of the fourth photodiode matrix, the second photodiode is located at the upper right of the fourth photodiode matrix, the third photodiode is located at the lower right of the fourth photodiode matrix, and the fourth photodiode is located at the lower left of the fourth photodiode matrix.

In one possible embodiment, the light splitting structure includes a plurality of light splitting units, each of the first photodiodes, each of the second photodiodes, each of the third photodiodes, and each of the fourth photodiodes correspond to four light splitting units; each light splitting unit corresponds to one first photodiode, one second photodiode, one third photodiode and one fourth photodiode, and is used for transmitting the received first color light to the first photodiode, transmitting the received second color light to the second photodiode, transmitting the received third color light to the third photodiode and transmitting the received infrared light to the fourth photodiode.

In a fifth aspect, an image pickup apparatus is provided. The image pickup apparatus includes a lens and an image sensor; the lens is used for converging light emitted by a shot object or reflected light to the image sensor; the image sensor is used for converting the received optical image into a digital signal; wherein the image sensor is the image sensor provided in the first aspect; the image pickup apparatus further includes a second infrared cut filter; the second infrared cut-off filter is arranged between the lens and the image sensor and used for filtering infrared light; alternatively, the image sensor is the image sensor provided in the second, third or fourth aspect described above.

In one possible embodiment, the image pickup apparatus further includes an image processor; the image processor is used for processing the digital signal and outputting an image of the shot object. The color image can be obtained after being processed by the image processor.

In a sixth aspect, a display device is provided. The display device comprises a liquid crystal display panel, a backlight source and a light splitting structure, wherein the liquid crystal display panel comprises a plurality of first color sub-pixels, a plurality of second color sub-pixels and a plurality of third color sub-pixels; the backlight source is arranged on the light incident side of the liquid crystal display panel and used for providing a light source for the liquid crystal display panel; the light splitting structure is arranged between the backlight source and the liquid crystal display panel and is used for splitting white light emitted from the backlight source into first color light, second color light and third color light, transmitting the first color light to the first color sub-pixels, transmitting the second color light to the second color sub-pixels and transmitting the third color light to the third color sub-pixels. The display device provided by the application comprises a liquid crystal display panel and a backlight source, and further comprises a light splitting structure arranged between the liquid crystal display panel and the backlight source, wherein the light splitting structure can split white light emitted from the backlight source into first color light, second color light and third color light, the first color light is transmitted to a first color sub-pixel, the second color light is transmitted to a second color sub-pixel, and the third color light is transmitted to a third color sub-pixel, so that the first color sub-pixel of the liquid crystal display panel can transmit the first color light, the second color sub-pixel can transmit the second color light, and the third color sub-pixel can transmit the third color light. On this basis, for current display device, through set up color filter on liquid crystal display panel, absorb the light of two kinds of colours in the white light through color filter, see through the light of another kind of colour, in order to ensure to pass through first colour light from liquid crystal display panel's first colour subpixel, second colour subpixel passes through second colour light, third colour subpixel passes through third colour light, because this application utilizes the beam splitting structure to divide light, and the beam splitting structure does not absorb light, therefore be favorable to improving the utilization ratio of the photon that the backlight provided.

In one possible embodiment, the liquid crystal display panel includes a color filter; the color filter comprises a plurality of first color filter units corresponding to the first color sub-pixels, a plurality of second color filter units corresponding to the second color sub-pixels and a plurality of third color filter units corresponding to the third color sub-pixels; at least any one of the first color filter units only allows the first color light to transmit, at least any one of the second color filter units only allows the second color light to transmit, and at least any one of the third color filter units only allows the third color light to transmit. The color filter is arranged in the liquid crystal display panel, so that light transmitted by the first color sub-pixel, the second color sub-pixel and the third color sub-pixel of the liquid crystal display panel is ensured to be light of a single color, color cross is avoided, and the display effect of the display device is improved.

In one possible embodiment, the display device further includes an upper polarizer and a lower polarizer disposed at both sides of the liquid crystal display panel.

Drawings

Fig. 1 is a schematic structural diagram of an image sensor provided in the prior art;

fig. 2 is a schematic structural diagram of an image capturing apparatus according to an embodiment of the present application;

fig. 3a is a schematic structural diagram of an image sensor according to an embodiment of the present application;

fig. 3b is a schematic diagram of light splitting of a micro-nano medium column in the image sensor provided in fig. 3 a;

fig. 4a is a schematic structural diagram of an image sensor according to another embodiment of the present application;

FIG. 4b is a schematic diagram of the structure of the image sensor provided in FIG. 4a, in which each pixel receives light;

fig. 5 is a schematic structural diagram of an image sensor according to yet another embodiment of the present application;

fig. 6 is a schematic diagram illustrating an arrangement structure of a plurality of first photodiodes, a plurality of second photodiodes, and a plurality of third photodiodes according to an embodiment of the present application;

fig. 7a is a schematic structural diagram of a first color filter unit and a second color filter unit in a color filter according to an embodiment of the present disclosure;

fig. 7b is a schematic diagram of a corresponding design structure of a photodiode array and a color filter according to an embodiment of the present disclosure;

fig. 8 is a schematic structural diagram of a light splitting structure provided in an embodiment of the present application;

fig. 9 is a schematic structural diagram of an image sensor according to yet another embodiment of the present application;

fig. 10a is a schematic structural diagram of a corresponding design of a light splitting unit and a photodiode array according to an embodiment of the present application;

fig. 10b is a schematic light splitting diagram of a light splitting structure according to an embodiment of the present application;

fig. 11a is a schematic structural diagram of a corresponding design of a light splitting unit and a photodiode array according to another embodiment of the present application;

fig. 11b is a schematic light splitting diagram of a light splitting structure according to another embodiment of the present application;

fig. 11c is a schematic light splitting diagram of a light splitting structure according to another embodiment of the present application;

fig. 12a is a schematic structural diagram of a corresponding design of a light splitting unit and a photodiode array according to yet another embodiment of the present application;

fig. 12b is a schematic light splitting diagram of a light splitting structure according to another embodiment of the present application;

fig. 12c is a schematic light splitting diagram of a light splitting structure according to another embodiment of the present application;

fig. 13 is a schematic structural diagram of an image sensor according to yet another embodiment of the present application;

fig. 14 is a schematic diagram illustrating an arrangement structure of a plurality of first photodiodes, a plurality of second photodiodes, a plurality of third photodiodes, and a plurality of fourth photodiodes according to an embodiment of the present application;

fig. 15 is a schematic diagram illustrating an arrangement structure of a plurality of first photodiodes, a plurality of second photodiodes, a plurality of third photodiodes, and a plurality of fourth photodiodes according to another embodiment of the present application;

FIG. 16a is a schematic view of a corresponding design structure of a photodiode array and a color filter according to another embodiment of the present application;

FIG. 16b is a schematic diagram illustrating a corresponding design structure of a photodiode array and a color filter according to yet another embodiment of the present disclosure;

fig. 17 is a schematic structural diagram of an image sensor according to yet another embodiment of the present application;

fig. 18a is a schematic structural diagram of a corresponding design of a light splitting unit and a photodiode array according to yet another embodiment of the present application;

fig. 18b is a schematic structural diagram of a corresponding design of a light splitting unit and a photodiode array according to yet another embodiment of the present application;

fig. 18c is a schematic light splitting diagram of a light splitting structure according to another embodiment of the present application;

fig. 18d is a schematic light splitting diagram of a light splitting structure according to another embodiment of the present application;

fig. 19a is a schematic structural diagram of a corresponding design of a light splitting unit and a photodiode array according to yet another embodiment of the present application;

fig. 19b is a schematic diagram illustrating a corresponding design structure of a light splitting unit and a photodiode array according to still another embodiment of the present disclosure;

fig. 19c is a schematic light splitting diagram of a light splitting structure according to another embodiment of the present application;

fig. 19d is a schematic light splitting diagram of a light splitting structure according to another embodiment of the present application;

fig. 20a is a schematic structural diagram of a corresponding design of a light splitting unit and a photodiode array according to yet another embodiment of the present application;

fig. 20b is a schematic light splitting diagram of a light splitting structure according to another embodiment of the present application;

fig. 21a is a schematic structural diagram of a corresponding design of a light splitting unit and a photodiode array according to yet another embodiment of the present application;

fig. 21b is a schematic light splitting diagram of a light splitting structure according to another embodiment of the present application;

fig. 22a is a schematic structural diagram of a corresponding design of a light splitting unit and a photodiode array according to yet another embodiment of the present application;

fig. 22b is a schematic light splitting diagram of a light splitting structure according to another embodiment of the present application;

fig. 23a is a schematic view of a corresponding design structure of a light splitting unit and a photodiode array according to yet another embodiment of the present application;

fig. 23b is a schematic light splitting diagram of a light splitting structure according to another embodiment of the present application;

fig. 24 is a schematic structural diagram of an image sensor according to yet another embodiment of the present application;

fig. 25a is a schematic light splitting diagram of a light splitting structure according to another embodiment of the present application;

fig. 25b is a schematic light splitting diagram of a light splitting structure according to another embodiment of the present application;

fig. 26 is a schematic structural diagram of an image sensor according to yet another embodiment of the present application;

fig. 27 is a schematic diagram illustrating an arrangement structure of a plurality of first photodiodes, a plurality of second photodiodes, a plurality of third photodiodes, and a plurality of fourth photodiodes according to still another embodiment of the present application;

fig. 28 is a schematic structural diagram of a corresponding design of a light splitting unit and a photodiode array according to yet another embodiment of the present application;

fig. 29 is a schematic light splitting diagram of a light splitting structure according to another embodiment of the present application;

fig. 30 is a schematic structural diagram of a display device according to an embodiment of the present application;

fig. 31 is a schematic structural diagram of a display panel including a plurality of first color sub-pixels, second color sub-pixels, and third color sub-pixels according to an embodiment of the present disclosure;

fig. 32 is a schematic structural diagram of a display device according to another embodiment of the present application.

Reference numerals:

1-an image pickup apparatus; 10-an image sensor; 20-a lens; 30-an image processor; 40-a display device; 100-a second infrared cut filter; 101-a color filter; 102-an array of photodiodes; 103-a light splitting structure; 104-a microlens; 105-a first infrared cut filter; 401-liquid crystal display panel; 401 a-first color sub-pixel; 401 b-a second color subpixel; 401 c-a third color subpixel; 402-a backlight; 403-upper polarizer; 404-lower polarizer 404; 1011-a first color filter unit; 1012-second color filter unit; 1013-a third color filter unit; 1014-a hollowed-out area; 1031-micro nano medium column; 1032-a light splitting unit; 4011-an array substrate; 4012-pair of cassette substrates; 4013-liquid crystal layer.

Detailed Description

The technical solutions in the embodiments of the present application will be described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments.

Hereinafter, the terms "first", "second", and the like are used for descriptive convenience only and are not to be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first," "second," etc. may explicitly or implicitly include one or more of that feature. In the description of the present application, "a plurality" means two or more unless otherwise specified.

In the embodiments of the present application, unless otherwise specifically stated or limited, the term "electrically connected" may be a direct electrical connection or an indirect electrical connection through an intermediate.

In the embodiments of the present application, words such as "exemplary" or "for example" are used to mean serving as an example, instance, or illustration. Any embodiment or design described herein as "exemplary" or "e.g.," is not necessarily to be construed as preferred or advantageous over other embodiments or designs. Rather, use of the word "exemplary" or "such as" is intended to present concepts related in a concrete fashion.

In the embodiment of the present application, "and/or" describes an association relationship of an associated object, indicating that three relationships may exist, for example, a and/or B may indicate: a exists alone, A and B exist simultaneously, and B exists alone, wherein A and B can be singular or plural. The character "/" generally indicates that the former and latter associated objects are in an "or" relationship.

In the embodiments of the present application, directional indications such as up, down, left, right, front, and rear, etc., used for explaining the structure and movement of the various components in the present application, are relative. These indications are appropriate when the components are in the positions shown in the figures. However, if the specification of the position of the element changes, then the direction indications will change accordingly.

The embodiment of the present application provides an image capturing apparatus, which may be, for example, a device having an image capturing function, such as a camera, an Internet Protocol Camera (IPC), a mobile phone having a front camera and/or a rear camera, a tablet having a front camera and/or a rear camera, a digital video camera, a vehicle-mounted camera, or an industrial camera. In addition, the image pickup apparatus can be applied to the security field, the photographic field, the automotive electronics field, the industrial machine vision field, or the like.

As shown in fig. 2, the image pickup apparatus 1 described above may include an image sensor 10, a lens 20, and an image processor 30. The lens 20 is configured to condense light emitted from or reflected from a subject onto the image sensor 10, and the image sensor 10 is configured to convert a received optical image into a digital signal; the image processor 30 is configured to process the digital signal and output an image of the subject, which is typically an R/G/B full-color image.

Among them, the above-described image sensor 10 is an important component of the image pickup apparatus 1, affecting the performance of the image pickup apparatus. To solve the problem of low photon utilization of the conventional image sensor 10, the embodiments of the present application provide several alternative implementations with respect to an image sensor.

In a first alternative embodiment, as shown in fig. 3a, the image sensor 10 includes a plurality of photodiodes PD and transparent irregular micro-nano dielectric pillars. The micro-nano dielectric column can deflect incident light with different wavelengths, red R, green G and blue B light rays are deflected to corresponding pixel regions respectively, and each pixel region is correspondingly provided with a photodiode PD. Taking the red pixels R, the green pixels G and the blue pixels B arranged as shown in fig. 3B as an example, a pixel matrix is indicated by a dashed-line box in fig. 3B, and each pixel matrix includes a green pixel G located at the upper left corner and the lower right corner, a red pixel R located at the upper right corner and a blue pixel B located at the lower left corner. For a pixel matrix, white light incident on a micro-nano medium column corresponding to the pixel matrix contains red, green and blue wavelength components, the micro-nano medium column deflects green light to green pixels G at the lower left corner and the upper right corner, deflects red light to red pixels R at the upper right corner, and deflects blue light to blue pixels B at the lower left corner, so that the effect similar to that of the existing color filter CF can be realized. Because the transparent micro-nano medium column does not absorb the energy of light, all incident light can be utilized and deflected to a pixel region to be belonged to, and therefore the light incoming amount is improved.

However, for the first alternative embodiment, since each of the red pixels R, the green pixels G, and the blue pixels B in the pixel matrix receives the light obtained by distributing the white light in the pixel space corresponding to 2 × 2 pixels, the pixel spatial resolution is reduced, and compared with the case where the color filter 101 in the image sensor 10 in the prior art includes the red filter unit, the green filter unit, and the blue filter unit, the pixel spatial resolution is 25% in the prior art, which is not in accordance with the principle of spatial sampling.

In a second optional embodiment, the structure of the image sensor 10 is the same as that of the image sensor 10 in the first optional embodiment, except that in the second optional embodiment, after the light is incident on the micro-nano medium column, a diffraction effect occurs at the micro-nano medium column, so that the white light is divided into red light and blue light. Since in the second alternative embodiment, each of the red pixels and the blue pixels receives the light after the white light in the pixel space corresponding to 2 × 1 pixels is allocated, the pixel spatial resolution is reduced, and the principle of spatial sampling is not satisfied. Furthermore, since the image sensor 10 generally includes red, green and blue pixels, the second alternative embodiment lacks the most important green pixel, and thus is not conducive to post-image processing.

In a third optional embodiment, as shown in fig. 4a, the image sensor 10 includes a transparent irregular micro-nano dielectric column and a plurality of photodiodes PD, where a part of white light transmitted from the lens is directly transmitted to the photodiodes PD, and does not pass through the micro-nano dielectric column, a part of the white light is first transmitted to the micro-nano dielectric column, and is transmitted to different photodiodes PD after being dispersed by the micro-nano dielectric column, and the micro-nano dielectric column may separate the white light into light other than red light and red light, or separate the white light into light other than blue light and blue light. Based on the difference in color of the light received by the photodiode PD, as shown in fig. 4B, the pixel matrix includes a pixel W + R, a pixel W-R, a pixel W + B, and a pixel W-B; the photodiode PD corresponding to the pixel W + R receives white light and red light, the photodiode PD corresponding to the pixel W-R receives light except the red light in the white light, the photodiode PD corresponding to the pixel W + B receives the white light and the blue light, and the photodiode PD corresponding to the pixel W-B receives light except the blue light in the white light.

Since in the third alternative embodiment, the pixel matrix is not the pure red pixel R, green pixel G and blue pixel B, but the pixel W + R, the pixel W-R, the pixel W + B and the pixel W-B, the color use increases the difficulty of the post-algorithm processing, and the post-algorithm processing increases noise.

In order to solve the problems of the image sensor 10 provided in the first, second and third alternative embodiments, the embodiment of the present application further provides an image sensor 10, where the image sensor 10 can be applied to the image capturing apparatus 1, and the image sensor 10 provided in the present application is exemplarily described below by four specific examples.

Example one

As shown in fig. 5, the image sensor 10 includes a color filter 101 and a photodiode array 102.

In the case where the image sensor 10 provided in the first embodiment is applied to the image capturing apparatus 1, the image capturing apparatus 1 further includes a second infrared cut filter (IR-cut) 100 as shown in fig. 5; the second infrared cut filter 100 is disposed between the lens 20 and the image sensor 10, and filters infrared light. The wavelength of infrared light is over 700nm, far longer than that of three primary colors.

In some examples, the second infrared cut filter 100 may be separately provided from the lens 20 and the image sensor 10. In other examples, the second infrared cut filter 100 may also be integrated on the lens 20.

Since the natural light passes through the second infrared cut filter 100, the infrared light is filtered, and thus the light transmitted to the color filter 101 of the image sensor 10 is light of the natural light, which is light except for the infrared light, that is, light of three primary colors. Since the infrared light is cut off, the infrared light can be prevented from interfering with the three primary color light.

The color filter 101 includes a plurality of first color filter units 1011 and a plurality of second color filter units 1012; at least any one first color filter unit 1011 among the plurality of first color filter units 1011 allows the first color light and the second color light among the three primary color lights to transmit therethrough, and at least any one second color filter unit 1012 among the plurality of second color filter units 1012 allows the second color light and the third color light among the three primary color lights to transmit therethrough.

It is understood that the three primary colors of light may include red light, green light, and blue light, in which case the first color of light, the second color of light, and the third color of light are one of red light, green light, and blue light, and the first color of light, the second color of light, and the third color of light are not the same. For example, the first color light may be red light, the second color light may be green light, and the third color light may be blue light. For another example, the first color light may be green light, the second color light may be red light, and the third color light may be blue light.

It should be understood that the color filter 101 is coated with an absorption type color pigment organic substance that allows light of a specific wavelength band to pass therethrough and blocks light of other wavelength bands. The first color filter unit 1011 and the second color filter unit 1012 are coated with different color pigment organic substances, the first color filter unit 1011 is coated with the color pigment organic substances that allow the first color light and the second color light to pass therethrough and block the other color light, and the second color filter unit 1012 is coated with the color pigment organic substances that allow the second color light and the third color light to pass therethrough and block the other color light.

In the case that the first color light is green light, the second color light is red light, and the third color light is blue light, the first color filter unit 1011 may be, for example, a yellow filter unit, and after the red light, the green light, and the blue light pass through the yellow filter unit, only the red light and the green light may pass through, and the blue light may be absorbed; the second color filter unit 1012 may be, for example, a cyan filter unit through which only green light and blue light may pass and red light may be absorbed after red light, green light, and blue light pass.

The photodiode array 102 is disposed on the light-emitting side of the color filter 101, and the photodiode array 102 includes a plurality of first photodiodes PD1, a plurality of second photodiodes PD2, and a plurality of third photodiodes PD 3.

It is understood that the number of the first photodiode PD1, the second photodiode PD2, and the third photodiode PD3 may be set as needed.

Note that the first photodiode PD1, the second photodiode PD2, and the third photodiode PD3 in the photodiode array 102 all function to convert a received optical signal into an electrical signal.

The first photodiode PD1, the second photodiode PD2, and the third photodiode PD3 are configured to receive light of one color, respectively, and the first photodiode PD1, the second photodiode PD2, and the third photodiode PD3 receive light of different colors. For example, the first photodiode PD1 is configured to receive the first color light, the second photodiode PD2 is configured to receive the second color light, and the third photodiode PD3 is configured to receive the third color light.

It should be understood that the first photodiode PD1, the second photodiode PD2, and the third photodiode PD3 each represent a pixel, and in the case where the first photodiode PD1 is configured to receive light of a first color, the second photodiode PD2 is configured to receive light of a second color, and the third photodiode PD3 is configured to receive light of a third color, it can be considered that the first photodiode PD1 represents a pixel of the first color, the second photodiode PD2 represents a pixel of the second color, and the third photodiode PD3 represents a pixel of the third color.

In the first embodiment, the first photodiode PD1 represents a first color pixel such as red, the second photodiode PD2 represents a second color pixel such as green, and the third photodiode PD3 represents a third color pixel such as blue, and the first embodiment is not limited to the pixel matrix design, i.e., the arrangement of the first plurality of photodiodes PD1, the second plurality of photodiodes PD2, and the third plurality of photodiodes PD3 in the photodiode array 102. The plurality of first photodiodes PD1, the plurality of second photodiodes PD2, and the plurality of third photodiodes PD3 form a two-dimensional array, and the plurality of first photodiodes PD1, the plurality of second photodiodes PD2, and the plurality of third photodiodes PD3 may be arranged in different arrays according to different imaging bands, and the photodiodes at different positions are used for imaging light beams of different bands.

In an alternative embodiment, as shown in fig. 6, a plurality of first photodiodes PD1, a plurality of second photodiodes PD2, and a plurality of third photodiodes PD3 are used to constitute a plurality of 2 × 2 first photodiode matrices M repeatedly arranged in the first direction X and the second direction Y; the first direction X is a row direction of the first photodiode matrix M, the second direction Y is a column direction of the first photodiode matrix M, and the first direction X is perpendicular to the second direction Y. Each of the first photodiode matrices M includes one first photodiode PD1, two second photodiodes PD2, and one third photodiode PD 3.

The arrangement of the first photodiode PD1, the second photodiodes PD2, and the third photodiode PD3 in the first photodiode matrix M is not limited. In some examples, as shown in fig. 6, in the first photodiode matrix M, the first photodiode PD1 is located at the upper left corner of the first photodiode matrix M, the two second photodiodes PD2 are located at the upper right corner and the lower left corner of the first photodiode matrix M, respectively, and the third photodiode PD3 is located at the lower right corner of the first photodiode matrix M. In this case, the arrangement of the plurality of first photodiodes PD1, the plurality of second photodiodes PD2, and the plurality of third photodiodes PD3 is in a bayer array.

Based on the above, in the case where the plurality of first photodiodes PD1, the plurality of second photodiodes PD2, and the plurality of third photodiodes PD3 are used to constitute the plurality of 2 × 2 first photodiode matrices M (the first photodiode matrices M may be referred to above specifically) which are repeatedly arranged in the first direction X and the second direction Y, as shown in fig. 7a, the plurality of first color filter units 1011 and the plurality of second color filter units 1012 in the color filter 101 may be alternately arranged in a direction parallel to two diagonal lines of the first photodiode matrices M. On this basis, as shown in fig. 7b, the first color filter unit 1011 covers the first photodiode PD1 and the portion of the second photodiode PD2 located in the fourth neighborhood of the first photodiode PD 1; the second color filter unit 1012 covers the third photodiode PD3 and a portion of the second photodiode PD2 located in the fourth neighborhood of the third photodiode PD 3; each of the second photodiodes PD2 is covered by two adjacent first color filter units 1011 and two adjacent second color filter units 1012.

Here, the "second photodiode PD2 located in the four neighborhoods of the first photodiode PD 1" refers to the second photodiode PD2 located in four directions of up, down, left, right, and left of the first photodiode PD 1. Similarly, the "second photodiode PD2 located in the fourth neighborhood of the third photodiode PD 3" refers to the second photodiode PD2 located in four directions, i.e., up, down, left, and right, of the third photodiode PD 3.

Since the light transmitted from the first color filter unit 1011 is the mixed first color light and the second color light, the light transmitted from the second color filter unit 1012 is the mixed second color light and the third color light, and the first photodiode PD1, the second photodiode PD2, and the third photodiode PD3 are respectively used for receiving the light of one color, as shown in fig. 5, the above-mentioned image sensor 10 further includes a light splitting structure (color splitter)103, and the light splitting structure 103 is disposed between the color filter 101 and the first photodiode PD1, the second photodiode PD2, and the third photodiode PD 3. The light splitting structure 103 is configured to split the first color light and the second color light transmitted from the first color filter unit 1011 and transmit the first color light to the first photodiode PD1 and the second color light to the second photodiode PD2, and the light splitting structure 103 is further configured to split the second color light and the third color light transmitted from the second color filter unit 1012 and transmit the second color light to the second photodiode PD2 and the third color light to the third photodiode PD 3.

In some examples, as shown in fig. 8, the light splitting structure 103 may include a plurality of micro nano dielectric columns 1031 distributed in an array. On the basis, the shape of the micro-nano dielectric column 1031 can be a cylinder, a prism, other regular or irregular shapes, and the prism includes a quadrangular prism, a pentagonal prism, and the like. The shapes of the micro-nano dielectric columns 1031 may be the same or may not be completely the same. Fig. 8 illustrates an example in which the micro nano dielectric columns 1031 are cylindrical columns and the shapes of the plurality of micro nano dielectric columns 1031 are the same.

In addition, the micro-nano dielectric columns 1031 may be regularly arranged along the row direction and the column direction, in this case, an included angle between the row direction and the column direction may be a right angle or an acute angle; of course, the micro nano dielectric posts 1031 may be arranged irregularly.

The micro/nano dielectric column 1031 is a small column formed of a transparent dielectric and having dimensions in the micro/nano range in each direction. The micro-nano dielectric columns 1031 do not absorb light, phase delay occurs when light irradiates the micro-nano dielectric columns 1031, the phases of the micro-nano dielectric columns 1031 are continuously changed, the direction angle of the light can be changed, the principle of the micro-nano dielectric columns 1031 is similar to that of a phased array radar, and the direction angles of the light with different wavelengths after passing through the micro-nano dielectric columns 1031 are different, so that separation of the light with different wavelengths can be realized.

In order to increase the amount and intensity of light received by the photodiode array 102, the first embodiment can be implemented in the following two ways, for example.

The first method comprises the following steps:

the light splitting structure 103 is further configured to converge the first color light transmitted from the first color filter unit 1011 and converge the second color light, transmit the converged first color light to the first photodiode PD1, and transmit the converged second color light to the second photodiode PD 2. Due to the converging function of the light splitting structure 103, more light of the first color can be transmitted to the first photodiode PD1 and more light of the second color can be transmitted to the second photodiode PD2, so that the first photodiode PD1 can receive more light of the first color and the second photodiode PD2 can receive more light of the second color, thereby increasing the amount and intensity of light received by the first photodiode PD1 and the second photodiode PD 2.

The light splitting structure 103 is further configured to converge the second color light transmitted from the second color filter unit 1012 and converge the third color light, transmit the converged second color light to the second photodiode PD2, and transmit the converged third color light to the third photodiode PD 3. Likewise, the second photodiode PD2 may further receive more second color light, and the third photodiode PD3 may receive more third color light, so that the amount and intensity of light received by the second photodiode PD2 may be further increased, and the amount and intensity of light received by the third photodiode PD3 may be increased.

And the second method comprises the following steps:

as shown in fig. 9, the image sensor 10 further includes a plurality of micro lenses (micro lenses) 104 provided on the light incident side of the color filter 101; the micro lens 104 is used for converging the three primary colors and transmitting the three primary colors to the first color filter unit 1011 and the second color filter unit 1012.

Here, one first color filter unit 1011 or one second color filter unit 1012 may correspond to one or more microlenses 104. Considering that light transmitted from the first color filter unit 1011 and the second color filter unit 1012 is also divided into different photodiodes through the light splitting structure 103, in order to avoid that when one first color filter unit 1011 or one second color filter unit 1012 corresponds to one microlens 104, all light of the area corresponding to one first color filter unit 1011 or one second color filter unit 1012 is converged by the first microlens 104 corresponding to the first color filter unit 1011 or one second color filter unit 1012, which is not favorable for the light splitting structure 103 to divide light into different photodiodes, based on which, in some examples, one first color filter unit 1011 or one second color filter unit 1012 corresponds to a plurality of microlenses 104.

Since the plurality of microlenses 104 disposed on the light incident side of the color filter 101 can converge and transmit the three primary colors of light to the first color filter unit 1011 and the second color filter unit 1012, the amount and intensity of light transmitted to the color filter 101 can be increased, and the amount and intensity of light transmitted to the color filter 101 can be increased, which in turn increases the amount and intensity of light received by the first photodiode PD1, the second photodiode PD2, and the third photodiode PD 3.

Based on the above, in the case where the plurality of first photodiodes PD1, the plurality of second photodiodes PD2, and the plurality of third photodiodes PD3 in the photodiode array 102, and the plurality of first color filter units 1011 and the plurality of second color filter units 1012 in the color filter 101 are arranged in the arrangement manner as shown in fig. 7b, in order to ensure that the light splitting structure 103 can realize the transmission of the first color light transmitted from the plurality of first color filter units 1011 to the first photodiodes PD1, and the transmission of the second color light to the second photodiodes PD2, and can realize the transmission of the second color light transmitted from the plurality of second color filter units 1012 to the second photodiodes PD2, and the transmission of the third color light to the third photodiodes PD3, the light splitting structure 103 can be divided into the plurality of light splitting units. Several kinds of corresponding design relationships of the plurality of light splitting units to the first photodiode PD1, the second photodiode PD2, the third photodiode PD3 in the photodiode array 102, and to the first color filter unit 1011 and the second color filter unit 1012 in the color filter 101 are exemplarily provided below.

The first method comprises the following steps:

as shown in fig. 10a, the light splitting structure 103 includes a plurality of light splitting cells 1032; each of the first photodiode PD1, each of the second photodiode PD2, and each of the third photodiode PD3 corresponds to four light-splitting units 1032; each first photodiode PD1 is configured to receive the first color light split by its corresponding four light splitting units 1032, each second photodiode PD2 is configured to receive the second color light split by its corresponding four light splitting units 1032, and each third photodiode PD3 is configured to receive the third color light split by its corresponding four light splitting units 1032.

Each light splitting unit 1032 corresponds to one first color filter unit 1011 or one second color filter unit 1012; and each light splitting unit 1032 corresponds to two photodiodes, specifically, each light splitting unit 1032 corresponds to one first photodiode PD1 and one second photodiode PD2, or each light splitting unit 1032 corresponds to one second photodiode PD2 and one third photodiode PD 3. As shown in fig. 10b, the light splitting unit 1032 corresponding to the first color filter unit 1011 receives the first color light and the second color light transmitted from the first color filter unit 1011 and separately transmits the first color light and the second color light to one first photodiode PD1 and one second photodiode PD 2; the light splitting unit 1032 corresponding to the second color filter unit 1012 receives the second color light and the third color light transmitted by the second color filter unit 1012, and separately transmits the second color light and the third color light to one second photodiode PD2 and one third photodiode PD 3.

In fig. 10b, the first color filter unit 1011 and the second color filter unit 1012 are not shown. The arrows in fig. 10b indicate that the light-splitting cells 1032 split light, and as seen in fig. 10b, each light-splitting cell 1032 can split light incident thereon to two photodiodes.

And the second method comprises the following steps:

as shown in fig. 11a, the light splitting structure 103 includes a plurality of light splitting cells 1032; each of the first photodiode PD1, each of the second photodiode PD2, and each of the third photodiode PD3 corresponds to four light-splitting units 1032; each first photodiode PD1 is configured to receive the first color light split by its corresponding four light splitting units 1032, each second photodiode PD2 is configured to receive the second color light split by its corresponding four light splitting units 1032, and each third photodiode PD3 is configured to receive the third color light split by its corresponding four light splitting units 1032.

Each light splitting unit 1032 corresponds to one first color filter unit 1011 and one second color filter unit 1012, and each light splitting unit 1032 corresponds to four photodiodes, specifically, each light splitting unit 1032 corresponds to one first photodiode PD1, two second photodiodes PD2, and one third photodiode PD 3.

As shown in fig. 11b, each light splitting unit 1032 receives the first color light and the second color light transmitted from the first color filter unit 1011 and transmits the first color light and the second color light separately to one first photodiode PD1 and two second photodiodes PD2, and each light splitting unit 1032 also receives the second color light and the third color light transmitted from the second color filter unit 1012 and transmits the second color light and the third color light separately to one third photodiode PD3 and two second photodiodes PD 2.

Alternatively, as shown in fig. 11c, each light-dividing unit 1032 receives the first color light and the second color light transmitted from the first color filter unit 1011 and the second color light and the third color light transmitted from the second color filter unit 1012 from the first color filter unit 1011, and transmits the first color light to one first photodiode PD1, the second color light to one second photodiode PD2, and the third color light to one third photodiode PD 3.

Fig. 11b and 11c each show a spectral relationship between one light splitting unit 1032 and the first photodiode matrix M.

And the third is that:

as shown in fig. 12a, the light splitting structure 103 includes a plurality of light splitting cells 1032; each of the first photodiodes PD1 and each of the third photodiodes PD3 corresponds to four light splitting units 1032; each first photodiode PD1 is configured to receive the first color light split by its corresponding four light splitting units 1032, and each third photodiode PD3 is configured to receive the third color light split by its corresponding four light splitting units 1032; each of the second photodiodes PD2 corresponds to one light splitting unit 1032; each of the second photodiodes PD2 is configured to receive the second color light split by its corresponding one of the light splitting units 1032.

Each light splitting unit 1032 corresponds to two first color filter units 1011 and two second color filter units 1012 opposite to each other, and each light splitting unit 1032 corresponds to one second photodiode PD2, two first photodiodes PD1 opposite to each other, and two third photodiodes PD3 opposite to each other.

Each light splitting unit 1032 receives light transmitted by two first color filter units 1011 and two second color filter units 1012 opposite to each other, and transmits the second color light to one second photodiode PD2 corresponding to the light splitting unit 1032, the first color light to two first photodiodes PD1 opposite to each other, and the third color light to two third photodiodes PD3 opposite to each other.

Note that, when each light splitting unit 1032 receives light transmitted by two opposite first color filter units 1011 and transmits the first color light to two opposite first photodiodes PD1, here, each first photodiode PD1 receives light transmitted by its corresponding first color filter unit 1011. Similarly, when each light splitting unit 1032 receives light transmitted by two opposite second color filter units 1012 and transmits third color light to two opposite third photodiodes PD3, each third photodiode PD3 receives light transmitted by its corresponding second color filter unit 1012.

Here, the second color light transmitted from the first color filter unit 1011 and the second color filter unit 1012 passes through the light splitting unit 1032 and then directly transmits the second color light to one second photodiode PD2 corresponding to the light splitting unit 1032, and as shown in fig. 12b, the first color light incident to the light splitting unit 1032a is transmitted to two upper and lower first photodiodes PD1, and the third color light incident to the light splitting unit 1032 is transmitted to two left and right third photodiodes PD3, respectively, or as shown in fig. 12c, the first color light incident to the light splitting unit 1032b is transmitted to two left and right first photodiodes PD1, and the third color light incident to the light splitting unit 1032 is transmitted to two upper and lower third photodiodes PD3, respectively.

In addition, in the case that the light splitting structure 103 includes a plurality of micro nano dielectric posts 1031 distributed in an array, each light splitting unit 1032 may include one or more micro nano dielectric posts 1031.

On the basis, in the case that the image sensor 10 includes a plurality of microlenses 104, one microlens 104 may correspond to one light splitting unit 1032, and the microlens 104 transmits the collected light to the light splitting unit 1032 corresponding to the microlens 104 through the first color filter unit 1011 or the second color filter unit 1012. Based on this, the design relationship between the microlenses 104 and the first photodiode PD1, the second photodiode PD2, the third photodiode PD3 in the photodiode array 102, and the first color filter unit 1011 and the second color filter unit 1012 in the color filter 101 can refer to fig. 10a, fig. 11a, fig. 12a, and the design relationship between the light splitting units and the first photodiode PD1, the second photodiode PD2, the third photodiode PD3 in the photodiode array 102, and the design relationship between the light splitting units and the first color filter unit 1011 and the second color filter unit 1012 in the color filter 101 in the first, second, and third cases, which are not described herein again.

It is understood that the corresponding design relationship between the light splitting unit 1032 and the first photodiode PD1, the second photodiode PD2, the third photodiode PD3 in the photodiode array 102, and the first color filter unit 1011 and the second color filter unit 1012 in the color filter 101 includes, but is not limited to, the above three cases.

In the first embodiment, since the first color filter unit 1011 and the second color filter unit 1012 of the color filter 101 in the image sensor 10 can allow light of two colors to transmit, specifically, the first color filter unit 1011 allows light of the first color and light of the second color to transmit, and the second color filter unit 1012 allows light of the second color and light of the third color to transmit, compared with the prior art in which the red filter unit, the green filter unit, and the blue filter unit of the color filter 101 in the image sensor 10 can only transmit light of one color, the first embodiment can improve the amount of light entering, that is, the photon utilization rate.

Furthermore, compared to the case where the color filter 101 in the image sensor 10 of the prior art includes a red filter unit, a green filter unit, and a blue filter unit, since the area of the first color filter unit 1011 or the second color filter unit 1012 corresponding to the first photodiode PD1, the second photodiode PD2, and the third photodiode PD3 in the first embodiment is 2 times larger than the area of the red filter unit, the green filter unit, and the blue filter unit corresponding to the first photodiode PD1, the second photodiode PD2, and the third photodiode PD3 in the prior art, the amount of incoming light received by the first photodiode PD1, the second photodiode PD2, and the third photodiode PD3 in the first embodiment is increased by 2 times.

On this basis, compared to the case where the color filter 101 in the image sensor 10 of the related art includes a red filter unit, a green filter unit, and a blue filter unit, since in the first embodiment, each first photodiode PD1, each second photodiode PD2, and each third photodiode PD3, that is, each first color pixel, each second color pixel, and each third color pixel in the pixel matrix all receive the light after the white light distribution in the pixel space corresponding to the two color filter units of the related art, only a little spatial resolution is lost, and the spatial resolution is 70% of the related art.

In addition, in the image sensor 10 provided in the first embodiment, since the first photodiode PD1 receives the first color light of the three primary colors, the second photodiode PD2 receives the second color light of the three primary colors, and the third photodiode PD3 receives the third color light of the three primary colors, all the three primary colors can be received by the image sensor 10, thereby facilitating the post-image processing. In addition, the first photodiode PD1, the second photodiode PD2, and the third photodiode PD3 receive light of a single color, and specifically, the first photodiode PD1 receives light of a first color, the second photodiode PD2 receives light of a second color, and the third photodiode PD3 receives light of a third color, that is, the image sensor 10 includes a first color pixel, a second color pixel, and a third color pixel, and compared to the case where the image sensor includes a pixel W + R, a pixel W-R, a pixel W + B, and a pixel W-B, the difficulty of the post-algorithm processing of the present embodiment is reduced without increasing noise.

In view of the low color requirements but high profile requirements in some fields, such as security, it is desirable that the image sensor 10 be capable of imaging RGB-NIR (red, green, blue, and near infrared) light when designing the image sensor 10. Based on this, the image sensor 10 provided in the first embodiment is different from the image sensor 10 provided in the second, third and fourth embodiments described below in that the image sensor 10 provided in the first embodiment is used for imaging RGB, and therefore in the first embodiment, the light impinging on the image sensor 10 is natural light with infrared light filtered, that is, the light impinging on the image sensor 10 is light of three primary colors, while the image sensor 10 provided in the second, third and fourth embodiments described below is used for imaging RGB-NIR, and therefore in the second, third and fourth embodiments, the light impinging on the image sensor 10 is natural light, that is, the light impinging on the image sensor 10 includes light of three primary colors and infrared light.

Based on this, in the case where the image sensor 10 provided in embodiment two, embodiment three, and embodiment four is applied to the above-described image pickup apparatus 1, no infrared cut filter is provided between the lens 20 and the image sensor 10 in the image pickup apparatus 1.

The second, third and fourth embodiments are described below.

Example two

As shown in fig. 13, the image sensor 10 includes a color filter 101 and a photodiode array 102.

The color filter 101 includes a plurality of first color filter units 1011, a plurality of second color filter units 1012, and a plurality of third color filter units 1013; at least any one of the first color filter units 1011 allows a first color light of the infrared light and the three primary colors to pass therethrough, at least any one of the second color filter units 1012 allows a second color light of the infrared light and the three primary colors to pass therethrough, and at least any one of the third color filter units 1013 allows a third color light of the infrared light and the three primary colors to pass therethrough.

Here, the first color light, the second color light, and the third color light may refer to the first embodiment, and are not described herein again. Under the condition that the first color light is green light, the second color light is red light, and the third color light is blue light, the first color filter unit 1011 may be, for example, a red filter unit, and after the infrared light and the three primary colors light pass through the red filter unit, only the infrared light and the red light can pass through, and the blue light and the green light can be absorbed; the second color filter unit 1012 may be, for example, a green filter unit, and after the infrared light and the three primary colors of light pass through the green filter unit, only the infrared light and the green light may pass through, and the red light and the blue light may be absorbed; the third color filter 1013 may be, for example, a blue filter, and after the infrared light and the three primary colors of light pass through the blue filter, only the infrared light and the blue light may pass through the blue filter, and the red light and the green light may be absorbed.

The photodiode array 102 is disposed on the light-emitting side of the color filter 101, and the photodiode array 102 includes a plurality of first photodiodes PD1, a plurality of second photodiodes PD2, a plurality of third photodiodes PD3, and a plurality of fourth photodiodes PD 4.

Here, the first photodiode PD1, the second photodiode PD2, and the third photodiode PD3 may refer to the first embodiment, and are not described herein again. The fourth photodiode PD4 is for receiving infrared light, and the fourth photodiode PD4 may represent an infrared light pixel.

It is understood that the number of the first photodiode PD1, the second photodiode PD2, the third photodiode PD3, and the fourth photodiode PD4 may be set as needed.

The present embodiment is designed for a pixel matrix, that is, the arrangement of the plurality of first photodiodes PD1, the plurality of second photodiodes PD2, the plurality of third photodiodes PD3, and the plurality of fourth photodiodes PD4 in the photodiode array 102 is not limited. The plurality of first photodiodes PD1, the plurality of second photodiodes PD2, the plurality of third photodiodes PD3 and the plurality of fourth photodiodes PD4 form a two-dimensional array, the plurality of first photodiodes PD1, the plurality of second photodiodes PD2, the plurality of third photodiodes PD3 and the plurality of fourth photodiodes PD4 can be arranged in different arrays according to different imaging bands, and the photodiodes in different positions are used for imaging light rays in different bands. Since the pixel matrix includes the first color pixel, the second color pixel, the third color pixel, and the infrared light pixel, in the case where the first color pixel is a red pixel, the second color pixel is a green pixel, and the third color pixel is a blue pixel, the pixel matrix may be referred to as an RGB-NIR pixel matrix. For the RGB-NIR pixel matrix design, two alternative embodiments are provided below by way of example.

In a first alternative embodiment, as shown in fig. 14, a row of first photodiodes PD1, a row of fourth photodiodes PD4, a row of second photodiodes PD2, a row of fourth photodiodes PD4, a row of third photodiodes PD3, and a row of fourth photodiodes PD4 are alternately arranged in sequence along the first direction X.

In a second alternative embodiment, as shown in fig. 15, a plurality of first photodiodes PD1, a plurality of second photodiodes PD2, a plurality of third photodiodes PD3, and a plurality of fourth photodiodes PD4 are used to form a plurality of second photodiode matrices P of 2 × 2 and a third photodiode matrix Q of 2 × 2 alternately arranged in the first direction X and the second direction Y; the first direction X is a row direction of the second photodiode matrix P and the third photodiode matrix Q, the second direction Y is a column direction of the second photodiode matrix P and the third photodiode matrix Q, and the first direction X is perpendicular to the second direction Y. Each second photodiode matrix P includes one first photodiode PD1, one second photodiode PD2, and two fourth photodiodes PD 4; each third photodiode matrix Q includes one second photodiode PD2, one third photodiode PD3, and two fourth photodiodes PD 4.

The arrangement of one first photodiode PD1, one second photodiode PD2, and two fourth photodiodes PD4 in the second photodiode matrix P is not limited. In some examples, as shown in fig. 15, in the second photodiode matrix P, two fourth photodiodes PD4 are located at the upper left corner and the lower right corner of the second photodiode matrix P, respectively, and the first photodiode PD1 and the second photodiode PD2 are located at the upper right corner and the lower left corner of the second photodiode matrix P, respectively.

Similarly, the arrangement of the one second photodiode PD2, the one third photodiode PD3, and the two fourth photodiodes PD4 in the third photodiode matrix Q is not limited. In some examples, as shown in fig. 15, in the third photodiode matrix Q, two fourth photodiodes PD4 are located at the upper left corner and the lower right corner of the third photodiode matrix Q, respectively, and a second photodiode PD2 and a third photodiode PD3 are located at the lower left corner and the upper right corner of the third photodiode matrix Q, respectively.

In the second embodiment, the first color filter unit 1011, the second color filter unit 1012 and the third color filter unit 1013 in the color filter 101 may be designed according to the RGB-NIR pixel matrix described above. In some examples, as shown in fig. 16a and 16b, a first color filter unit 1011 covers a first photodiode PD1 and a portion of a fourth photodiode PD4 located in the fourth neighborhood of the first photodiode PD1, a second color filter unit 1012 covers a second photodiode PD2 and a portion of a fourth photodiode PD4 located in the fourth neighborhood of the second photodiode PD2, and a third color filter unit 1013 covers a third photodiode PD3 and a portion of a fourth photodiode PD4 located in the fourth neighborhood of the third photodiode PD 3. Each fourth photodiode PD4 is covered by a first color filter unit 1011, one third color filter unit 1013, and two second color filter units 1012.

Here, the explanation of the first embodiment may be referred to for the "four neighborhoods", and details thereof are not repeated here.

Referring to fig. 16a and 16b, it can be seen that, if the arrangement of the first photodiode PD1, the second photodiode PD2, the third photodiode PD3 and the fourth photodiode PD4 in the RGB-NIR pixel matrix is different, the arrangement of the first color filter unit 1011, the second color filter unit 1012 and the third color filter unit 1013 in the color filter 101 is different. In the case where the first photodiode PD1, the second photodiode PD2, the third photodiode PD3, and the fourth photodiode PD4 are arranged according to the above-described first alternative embodiment, the first color filter unit 1011, the second color filter unit 1012, and the third color filter unit 1013 in the color filter 101 are arranged as shown in fig. 16 a. In the case where the first photodiode PD1, the second photodiode PD2, the third photodiode PD3, and the fourth photodiode PD4 are arranged according to the second alternative embodiment described above, the first color filter unit 1011, the second color filter unit 1012, and the third color filter unit 1013 in the color filter 101 are arranged as shown in fig. 16 b.

With reference to fig. 13, the image sensor 10 of the second embodiment further includes a light splitting structure 103, where the light splitting structure 103 is disposed between the color filter 101 and the photodiode array 102; the light splitting structure 103 is configured to separate the first color light and the infrared light transmitted from the first color filter unit 1011, separate the second color light and the infrared light transmitted from the second color filter unit 1012, separate the third color light and the infrared light transmitted from the third color filter unit 1013, and transmit the first color light to the first photodiode PD1, transmit the second color light to the second photodiode PD2, transmit the third color light to the third photodiode PD3, and transmit the infrared light to the fourth photodiode PD 4.

It should be noted that, for a specific structure of the light splitting structure 103, reference may be made to the first embodiment, and details are not described here.

In order to increase the amount and intensity of light received by the photodiode array 102, the second embodiment can be implemented in the following two ways, for example.

The first method comprises the following steps:

the light splitting structure 103 is further configured to converge the first color light transmitted from the first color filter unit 1011 and transmit the converged first color light to the first photodiode PD1, converge the second color light transmitted from the second color filter unit 1012 and transmit the converged second color light to the second photodiode PD2, converge the third color light transmitted from the third color filter unit 1013 and transmit the converged third color light to the third photodiode PD3, converge the infrared light transmitted from the first color filter unit 1011, the second color filter unit 1013, and the third color filter unit 1013 and transmit the converged infrared light to the fourth photodiode PD 4.

In the first case, reference may be made to the first embodiment for technical effects brought by the light splitting structure 103, and details are not described here.

And the second method comprises the following steps:

as shown in fig. 17, the image sensor 10 further includes a plurality of microlenses 104 provided on the light incident side of the color filter 101; the micro lens 104 is used for converging the three primary colors and the infrared light and transmitting the converged light to the first color filter unit 1011, the second color filter unit 1012 and the third color filter unit 1013.

Here, the corresponding arrangement relationship between the micro lens 104 and the first color filter unit 1011, the second color filter unit 1012 and the third color filter unit 1013, and the function of the micro lens 104 can be referred to the first embodiment, and no further description is given here.

Since the image sensor 10 of the second embodiment is directly used for receiving the three primary colors and the infrared light, and the infrared light can directly transmit through the color filter 101 and the light splitting structure 103, it is considered that when the light splitting structure 103 splits the infrared light to transmit the infrared light to the fourth photodiode PD4, it is possible that a part of the infrared light may strike other photodiodes except the fourth photodiode PD4, and thus, the color of the light received by other photodiodes except the fourth photodiode PD4 may be interfered, and the accuracy of the post-processed image may be affected. Based on this, in some examples, as shown in fig. 17, the above-described image sensor 10 further includes a first infrared cut filter 105 disposed between the light splitting structure 103 and the photodiode array 102; the first infrared cut filter 105 is used for filtering infrared light. The first ir-cut filter 105 includes a plurality of hollow areas, and a projection of the fourth photodiode PD4 on the first ir-cut filter 105 has an overlapping region with the hollow areas.

Since the first infrared cut filter 105 is disposed between the light splitting structure 103 and the photodiode array 102, infrared light transmitted from the light splitting structure 103 is absorbed by the first infrared cut filter 105 in the process of being transmitted to the first photodiode PD1, the second photodiode PD2, and the third photodiode PD3, so that the first photodiode PD1, the second photodiode PD2, and the third photodiode PD3 can be prevented from receiving infrared light, and thus it is ensured that the first photodiode PD1, the second photodiode PD2, and the third photodiode PD3 all receive light of a single color, which is beneficial to accuracy of post-image processing. The first ir-cut filter 105 includes a hollow area, and a projection of the fourth photodiode PD4 on the first ir-cut filter 105 has an overlapping area with the hollow area, so that the infrared light transmitted from the light splitting structure 103 can be transmitted to the fourth photodiode PD4 and received by the fourth photodiode PD 4.

In order to increase the amount of infrared light received by the fourth photodiode PD4, in some examples, a projection of the fourth photodiode PD4 on the first ir-cut filter 105 is located within the hollowed-out region of the first ir-cut filter packet 105.

On this basis, in order to ensure that the light splitting structure 103 can achieve the separation of the first color light and the infrared light transmitted from the first color filter unit 1011, the second color light and the infrared light transmitted from the second color filter unit 1012, the third color light and the infrared light transmitted from the third color filter unit 1013, and the first color light is transmitted to the first photodiode PD1, the second color light is transmitted to the second photodiode PD2, the third color light is transmitted to the third photodiode PD3, and the infrared light is transmitted to the fourth photodiode PD4, so that the above-described light splitting structure 103 can be divided into a plurality of light splitting units. Taking the arrangement of the first photodiode PD1, the second photodiode PD2, the third photodiode PD3 and the fourth photodiode PD4 in the photodiode array 102 as an example, as shown in fig. 14 and fig. 15, several designs of the plurality of light splitting units 1032, the first photodiode PD1, the second photodiode PD2, the third photodiode PD3 and the fourth photodiode PD4 in the photodiode array 102, and the first color filter unit 1011, the second color filter unit 1012 and the third color filter unit 1013 in the color filter 101 are provided.

The first method comprises the following steps:

referring to fig. 18a, 18b, 18c, and 18d, the photodiode arrays 102 in fig. 18a and 18c are arranged in the arrangement shown in fig. 14, and the photodiode arrays 102 in fig. 18b and 18d are arranged in the arrangement shown in fig. 15.

Referring to fig. 18a and 18b, the light splitting structure 103 includes a plurality of light splitting units 1032, each of the first photodiodes PD1, each of the second photodiodes PD2, each of the third photodiodes PD3, and each of the fourth photodiodes PD4 corresponding to four light splitting units 1032; each first photodiode PD1 is configured to receive the first color light split by its corresponding four light splitting units 1032, each second photodiode PD2 is configured to receive the second color light split by its corresponding four light splitting units 1032, each third photodiode PD3 is configured to receive the third color light split by its corresponding four light splitting units 1032, and each fourth photodiode PD4 is configured to receive the infrared light split by its corresponding four light splitting units 1032.

Each light-splitting unit 1032 corresponds to one of the first color filter unit 1011, the second color filter unit 1012, and the third color filter unit 1013; and each light splitting unit 1032 corresponds to two photodiodes, specifically, each light splitting unit 1032 corresponds to one fourth photodiode PD4 and one of the first photodiode PD1, the second photodiode PD2 and the third photodiode PD 3.

Fig. 18c and 18d illustrate a specific light splitting case of the light splitting unit 1032, and as shown in fig. 18c and 18d, the light splitting unit 1032 corresponding to the first color filter unit 1011 receives the first color light and the infrared light transmitted from the first color filter unit 1011 and separately transmits the first color light and the infrared light to the first photodiode PD1 and the fourth photodiode PD 4; the light dividing unit 1032 corresponding to the second color filter unit 1012 receives the second color light and the infrared light transmitted from the second color filter unit 1012 and separately transmits the second color light and the infrared light to the second photodiode PD2 and the fourth photodiode PD 4; the light-dividing unit 1032 corresponding to the third color filter unit 1013 receives the third color light and the infrared light transmitted from the third color filter unit 1013, and separately transmits the third color light and the infrared light to the third photodiode PD3 and the fourth photodiode PD 4.

In fig. 18c and 18d, the first color filter unit 1011, the second color filter unit 1012, and the third color filter unit 1013 are not illustrated. Arrows in fig. 18c and 18d indicate that the light-splitting units 1032 split light, and as can be seen from fig. 18c and 18d, each light-splitting unit 1032 can split light incident thereon to two photodiodes.

And the second method comprises the following steps:

referring to fig. 19a, 19b, 19c, and 19d, the photodiode arrays 102 in fig. 19a and 19c are arranged in the arrangement shown in fig. 14, and the photodiode arrays 102 in fig. 19b and 19d are arranged in the arrangement shown in fig. 15.

Referring to fig. 19a and 19b, the light splitting structure 103 includes a plurality of light splitting units 1032, and each of the first photodiode PD1, each of the second photodiode PD2, each of the third photodiode PD3, and each of the fourth photodiode PD4 corresponds to one light splitting unit 1032. Each first photodiode PD1 is configured to receive the first color light split by its corresponding one light splitting unit 1032, each second photodiode PD2 is configured to receive the second color light split by its corresponding one light splitting unit 1032, and each third photodiode PD3 is configured to receive the third color light split by its corresponding one light splitting unit 1032; each of the fourth photodiodes PD4 corresponds to four light-splitting cells 1032, and is configured to receive the infrared light split by any light-splitting cell 1032 of the four light-splitting cells 1032 corresponding thereto.

Each of the light splitting units 1032 corresponds to one of the first color filter unit 1011, the second color filter unit 1012, and the third color filter unit 1013. Fig. 19c and 19d illustrate a specific light splitting condition of the light splitting unit 1032, and as shown in fig. 19c and 19d, the light splitting unit 1032 corresponding to the first color filter unit 1011 receives the first color light and the infrared light transmitted from the first color filter unit 1011 and transmits the first color light and the infrared light to the first photodiode PD1 and any fourth photodiode PD4 in the fourth neighborhood thereof, respectively; the light splitting unit 1032 corresponding to the second color filter unit 1012 receives the second color light and the infrared light transmitted from the second color filter unit 1012, and transmits the second color light and the infrared light to the second photodiode PD2 and any fourth photodiode PD4 in the four neighborhoods thereof, respectively; the light-dividing unit 1032 corresponding to the third color filter unit 1013 receives the third color light and the infrared light transmitted from the third color filter unit 1013, and transmits the third color light and the infrared light to the third photodiode PD3 and any fourth photodiode PD4 in the four neighborhoods thereof, respectively.

In fig. 19c and 19d, the first color filter unit 1011, the second color filter unit 1012, and the third color filter unit 1013 are not illustrated. The circles in fig. 19c and 19d indicate that the first color light transmitted from the first color filter unit 1011 directly strikes the first photodiode PD1, the second color light transmitted from the second color filter unit 1012 directly strikes the second photodiode PD2, and the third color light transmitted from the third color filter unit 1013 directly strikes the third photodiode PD 3. The arrow indicates that the light-splitting unit 1032 transmits the infrared light incident thereon to the fourth photodiode PD 4.

It should be appreciated that, in order to ensure that each fourth photodiode PD4 can receive infrared light, each light splitting unit 1032 should transmit infrared light to the same-direction fourth photodiode PD4 in the four neighborhoods of the first, second, and third photodiodes PD1, PD2, and PD3, e.g., to the fourth photodiode PD4 on the left side of the first, second, and third photodiodes PD1, PD2, and PD 3.

And the third is that:

in the case where the photodiode array 102 is arranged in the arrangement shown in fig. 14, as shown in fig. 20a, the light splitting structure 103 includes a plurality of light splitting units 1032, and each of the first photodiode PD1, each of the second photodiode PD2, and each of the third photodiode PD3 corresponds to four light splitting units 1032; each first photodiode PD1 is configured to receive the first color light split by its corresponding four light splitting units 1032, each second photodiode PD2 is configured to receive the second color light split by its corresponding four light splitting units 1032, each third photodiode PD3 is configured to receive the third color light split by its corresponding four light splitting units 1032, and each fourth photodiode PD4 corresponds to one light splitting unit 1032 and is configured to receive the infrared light split by its corresponding light splitting unit 1032.

Each light-dividing unit 1032 corresponds to two sets of the two first color filter units 1011, the two second color filter units 1012, and the two third color filter units 1013, for example, the light-dividing unit 1032 corresponds to the two first color filter units 1011 and the two second color filter units 1012, and each light-dividing unit 1032 corresponds to five photodiodes.

Fig. 20b illustrates a specific light splitting case of the light splitting unit 1032, and as shown in fig. 20b, the light splitting unit 1032 corresponding to the two first color filter units 1011 and the two second color filter units 1012 receives the first color light and the infrared light transmitted from the two first color filter units 1011, and receives the second color light and the infrared light transmitted from the two second color filter units 1012, and transmits the first color light to the two first photodiodes PD1, the second color light to the two second photodiodes PD2, and the infrared light to the one fourth photodiode PD 4.

The light-dividing unit 1032 corresponding to the two second color filter units 1012 and the two third color filter units 1013 receives the second color light and the infrared light transmitted from the two second color filter units 1012 and receives the third color light and the infrared light transmitted from the two third color filter units 1013 and transmits the second color light to the two second photodiodes PD2, the third color light to the two third photodiodes PD3, and the infrared light to the one fourth photodiode PD 4.

The light splitting unit 1032 corresponding to the two third color filter units 1013 and the two first color filter units 1013 receives the third color light and the infrared light transmitted from the two third color filter units 1013 and receives the first color light and the infrared light transmitted from the two first color filter units 1011 and transmits the first color light to the two first photodiodes PD1, the third color light to the two third photodiodes PD3, and the infrared light to the one fourth photodiode PD 4.

In fig. 20b, the first color filter unit 1011, the second color filter unit 1012, and the third color filter unit 1013 are not shown. The circles in fig. 20b indicate that the infrared light is directly transmitted through the light splitting unit 1032 and transmitted to the fourth photodiode PD4, and the arrows indicate that the first color light, the second color light, and the third color light are separated into four photodiodes above, below, on the left, and on the right of the fourth photodiode PD4 after passing through the light splitting unit 1032.

And fourthly:

in the case where the photodiode array 102 is arranged in the arrangement shown in fig. 15, as shown in fig. 21a, the light splitting structure 103 includes a plurality of light splitting units 1032, and each of the first photodiode PD1, each of the second photodiode PD2, and each of the third photodiode PD3 corresponds to four light splitting units 1032; each first photodiode PD1 is configured to receive the first color light split by its corresponding four light splitting units 1032, each second photodiode PD2 is configured to receive the second color light split by its corresponding four light splitting units 1032, each third photodiode PD3 is configured to receive the third color light split by its corresponding four light splitting units 1032, and each fourth photodiode PD4 corresponds to one light splitting unit 1032 and is configured to receive the infrared light split by its corresponding light splitting unit 1032.

Each light splitting unit 1032 corresponds to one first color filter unit 1011, two second color filter units 1012, and one third color filter unit 1013.

Fig. 21b illustrates a specific light splitting situation of the light splitting units 1032, and as shown in fig. 21b, each light splitting unit 1032 receives the first color light transmitted by the corresponding first color filter unit 1011 and transmits the first color light to one first photodiode PD1, receives the second color light transmitted by the corresponding second color filter unit 1012 and transmits the second color light to two second photodiodes PD2, receives the third color light transmitted by the corresponding third color filter unit 1013 and transmits the third color light to one third photodiode PD 3; each light-splitting unit 1032 also receives the infrared light transmitted from the first color filter unit 1011, the second color filter unit 1012, and the third color filter unit 1013, and transmits the infrared light to the fourth photodiode PD 4.

In fig. 21b, the first color filter unit 1011, the second color filter unit 1012, and the third color filter unit 1013 are not shown. The circles in fig. 21b indicate that the infrared light is directly transmitted through the light splitting unit 1032 and transmitted to the fourth photodiode PD4, and the arrows indicate that the first color light, the second color light, and the third color light are separated into four photodiodes above, below, on the left, and on the right of the fourth photodiode PD4 after passing through the light splitting unit 1032.

And a fifth mode:

in the case where the photodiode array 102 is arranged in the arrangement shown in fig. 14, as shown in fig. 22a, the light splitting structure 103 includes a plurality of light splitting units 1032, each of the first photodiodes PD1, each of the second photodiodes PD2, each of the third photodiodes PD3, and each of the fourth photodiodes PD4 corresponding to four light splitting units 1032; each first photodiode PD1 is configured to receive the first color light split by its corresponding four light splitting units 1032, each second photodiode PD2 is configured to receive the second color light split by its corresponding four light splitting units 1032, each third photodiode PD3 is configured to receive the third color light split by its corresponding four light splitting units 1032, and each fourth photodiode PD4 is configured to receive the infrared light split by its corresponding four light splitting units 1032.

Each light splitting unit 1032 corresponds to any one group of two first color filter units 1011, two second color filter units 1012, two third color filter units 1013, first color filter units 1011 and second color filter units 1012, second color filter units 1012 and third color filter units 1013, and first color filter units 1011, for example, the light splitting unit 1032 corresponds to two first color filter units 1011; for another example, the light splitting unit 1032 corresponds to the first color filter unit 1011 and the second color filter unit 1012, and each light splitting unit 1032 corresponds to four photodiodes.

Fig. 22b illustrates a specific light splitting case of the light splitting unit 1032, and as shown in fig. 22b, the light splitting unit 1032 corresponding to the two first color filter units 1011 receives the first color light and the infrared light transmitted from the first color filter units 1011 and transmits the first color light to the two first photodiodes PD1 and transmits the infrared light to the two fourth photodiodes PD 4; the light-dividing units 1032 corresponding to the two second color filter units 1012 receive the second color light and the infrared light transmitted from the second color filter units 1012, transmit the second color light to the two second photodiodes PD2, and transmit the infrared light to the two fourth photodiodes PD 4; the light-dividing units 1032 corresponding to the two third color filter units 1013 receive the third color light and the infrared light transmitted from the third color filter units 1013, and transmit the third color light to the two third photodiodes PD3 and transmit the infrared light to the two fourth photodiodes PD 4.

The light splitting unit 1032 corresponding to the first and second

color filter units

1011 and 1012 receives the first color light and the infrared light transmitted from the first color filter unit 1011, and receives the second color light and the infrared light transmitted from the second color filter unit 1012, and transmits the first color light to the first photodiode PD1, the second color light to the second photodiode PD2, and the infrared light to the two fourth photodiodes PD 4.

The light-dividing unit 1032 corresponding to the second color filter unit 1012 and the third color filter unit 1013 receives the second color light and the infrared light transmitted from the second color filter unit 1012, and receives the third color light and the infrared light transmitted from the third color filter unit 1032, and transmits the second color light to the second photodiode PD2, the third color light to the third photodiode PD3, and the infrared light to the two fourth photodiodes PD 4.

The light splitting unit 1032 corresponding to the third color filter unit 1013 and the first color filter unit 1011 receives the third color light and the infrared light transmitted from the two third color filter units 1013 and receives the first color light and the infrared light transmitted from the two first color filter units 1011 and transmits the first color light to the first photodiode PD1, the third color light to the third photodiode PD3, and the infrared light to the fourth photodiode PD 4.

In fig. 22b, the first color filter unit 1011, the second color filter unit 1012, and the third color filter unit 1013 are not shown. The arrows in fig. 22b indicate that the light-splitting cells 1032 split light, and as can be seen from fig. 22b, each light-splitting cell 1032 can split the light impinging thereon to four photodiodes.

And a sixth mode:

in the case where the photodiode array 102 is arranged in the arrangement shown in fig. 15, as shown in fig. 23a, the light splitting structure 103 includes a plurality of light splitting units 1032, and each of the first photodiode PD1, each of the second photodiode PD2, and each of the third photodiode PD3 corresponds to four light splitting units 1032; each first photodiode PD1 is configured to receive the first color light split by its corresponding four light splitting units 1032, each second photodiode PD2 is configured to receive the second color light split by its corresponding four light splitting units 1032, each third photodiode PD3 is configured to receive the third color light split by its corresponding four light splitting units 1032, and each fourth photodiode PD4 corresponds to one light splitting unit 1032 and is configured to receive the infrared light split by its corresponding light splitting unit 1032.

Each light splitting unit 1032 corresponds to the first color filter unit 1011 and the second color filter unit 1012, or the second color filter unit 1012 and the third color filter unit 1013, and each light splitting unit 1032 corresponds to four photodiodes.

Fig. 23b illustrates a specific light splitting case of the light splitting unit 1032, and as shown in fig. 23b, the light splitting unit 1032 corresponding to the first color filter unit 1011 and the second color filter unit 1012 receives the first color light and the infrared light transmitted from the first color filter unit 1011, and receives the second color light and the infrared light transmitted from the second color filter unit 1012, and transmits the first color light to the first photodiode PD1, the second color light to the second photodiode PD2, and the infrared light to the two fourth photodiodes PD 4. As shown in fig. 23b, the light-splitting unit 1032 corresponding to the second color filter unit 1012 and the third color filter unit 1013 receives the second color light and the infrared light transmitted from the second color filter unit 1012, and receives the third color light and the infrared light transmitted from the third color filter unit 1013, and transmits the second color light to the second photodiode PD2, the third color light to the third photodiode PD3, and the infrared light to the two fourth photodiodes PD 4.

In fig. 23b, the first color filter unit 1011, the second color filter unit 1012, and the third color filter unit 1013 are not shown. The arrows in fig. 23b indicate that the light-splitting cells 1032 split light, and as can be seen from fig. 23b, each light-splitting cell 1032 can split the light impinging thereon to four photodiodes.

In the second embodiment, since the first color filter unit 1011, the second color filter unit 1012 and the third color filter unit 1013 in the color filter 101 can pass infrared light and light of another color, the second embodiment can improve the amount of incident light, that is, the photon utilization rate. In addition, compared to the case where the color filter 101 in the image sensor 10 of the prior art includes a red filter unit, a green filter unit, and a blue filter unit, in the second embodiment, the amount of incoming light received by the first photodiode PD1, the second photodiode PD2, the third photodiode PD3, and the fourth photodiode PD4 is increased by 2 times, and for a specific reason, reference may be made to the first embodiment.

On this basis, compared to the case where the color filter 101 in the image sensor 10 of the related art includes a red filter unit, a green filter unit, and a blue filter unit, since each of the first photodiodes PD1, each of the second photodiodes PD2, each of the third photodiodes PD3, and each of the fourth photodiodes PD4 in the second embodiment, that is, each of the first color pixels, each of the second color pixels, each of the third color pixels, and the fourth third color pixels in the pixel matrix receives light distributed by white light in the pixel space corresponding to two color filter units in the related art, only a little spatial resolution is lost, and the spatial resolution is 70% of that in the related art.

In addition, in the second embodiment, the first photodiode PD1, the second photodiode PD2, the third photodiode PD3, and the fourth photodiode PD4 can receive light of three primary colors, and the received light is light of a single color, so that the difficulty of the post-algorithm processing can be reduced.

EXAMPLE III

As shown in fig. 24, the image sensor 10 includes a color filter 101 and a photodiode array 102.

The color filter 101 includes a plurality of first color filter units 1011, a plurality of second color filter units 1012, a plurality of third color filter units 1013, and a plurality of hollow areas 1014; at least any one of the first color filter units 1011 allows the first color light of the three primary colors to pass through, at least any one of the second color filter units 1012 allows the second color light of the three primary colors to pass through, and at least any one of the third color filter units 1013 allows the third color light of the three primary colors to pass through.

It should be noted that, the first color filter unit 1011, the second color filter unit 1012, and the third color filter unit 1013 may refer to the second embodiment, and details are not described here.

The photodiode array 102 is disposed on the light-emitting side of the color filter 101, and the photodiode array 102 includes a plurality of first photodiodes PD1, a plurality of second photodiodes PD2, a plurality of third photodiodes PD3, and a plurality of fourth photodiodes PD4, the first photodiodes PD1 correspond to the first color filter unit 1011, the second photodiodes PD2 correspond to the second color filter unit 1012, the third photodiodes PD3 correspond to the third color filter unit 1013, and the fourth photodiodes PD4 correspond to the hollow area 1014 of the color filter 101.

Here, the first photodiode PD1, the second photodiode PD2, the third photodiode PD3, and the fourth photodiode PD4 may refer to the second embodiment, and are not described herein again.

It should be noted that, for example, the arrangement of the plurality of first photodiodes PD1, the plurality of second photodiodes PD2, the plurality of third photodiodes PD3, and the plurality of fourth photodiodes PD4 in the photodiode array 102 may be as shown in fig. 14 and fig. 15, and reference may be specifically made to the second embodiment described above, which is not described herein again.

With reference to fig. 24, the image sensor 10 of the third embodiment further includes a light splitting structure 103, where the light splitting structure 103 is disposed on the light incident side of the color filter 101; the light splitting structure 103 is used for splitting the infrared light and the three primary colors, the infrared light is transmitted to the fourth photodiode PD4 through the hollow area of the color filter 101, and the three primary colors are transmitted to the first color filter unit 1011, the second color filter unit 1012 and the third color filter unit 1013.

Since the first color filter unit 1011 allows the first color light of the three primary color lights to pass through, and the first photodiode PD1 corresponds to the first color filter unit 1011, the first photodiode PD1 receives the first color light. Similarly, the second photodiode PD2 receives the second color light, and the third photodiode PD3 receives the third color light.

In order to increase the amount and intensity of light received by the photodiode array 102, this third embodiment can be implemented in the following two ways, for example.

The first method comprises the following steps:

the light splitting structure 103 is further configured to converge the three primary colors, converge the infrared light, transmit the converged three primary colors to the first color filter unit 1011, the second color filter unit 1012, and the third color filter unit 1013 of the color filter 101, and transmit the converged infrared light to the hollow area 1014 of the color filter 101.

And the second method comprises the following steps:

as shown in fig. 24, the image sensor 10 further includes a plurality of microlenses 104 disposed on the light incident side of the light splitting structure 103; the micro lens 104 is used for converging the three primary colors and the infrared light and transmitting the converged light to the light splitting structure 103.

Here, the number of the microlenses 104 can be set as needed.

In addition, the function of the microlens 104 can refer to the first embodiment, and is not described herein again.

On this basis, in order to ensure that the light splitting structure 103 can separate the infrared light and the three primary colors of light, the infrared light is transmitted to the fourth photodiode PD4 through the hollow area of the color filter 101, and the three primary colors of light are transmitted to the first color filter unit 1011, the second color filter unit 1012, and the third color filter unit 1013, so that the light splitting structure 103 may be divided into a plurality of light splitting units. Taking the arrangement of the first photodiode PD1, the second photodiode PD2, the third photodiode PD3, and the fourth photodiode PD4 in the photodiode array 102 as an example, as shown in fig. 14 and 15, a plurality of light splitting units 1032 are exemplarily provided in corresponding design relationships with the first photodiode PD1, the second photodiode PD2, the third photodiode PD3, and the fourth photodiode PD4 in the photodiode array 102, and with the first color filter unit 1011, the second color filter unit 1012, and the third color filter unit 1013 in the color filter 101.

Referring to fig. 18a, 18b, 25a, and 25b, fig. 18a and 25a exemplify the arrangement of the photodiode array 102 as shown in fig. 14, and fig. 18b and 25b exemplify the arrangement of the photodiode array 102 as shown in fig. 15.

As shown in fig. 18a and 18b, the light splitting structure 103 includes a plurality of light splitting units 1032, and each of the first photodiode PD1, each of the second photodiode PD2, each of the third photodiode PD3, and each of the fourth photodiode PD4 corresponds to four light splitting units 1032.

Fig. 25a and 25b illustrate a specific light splitting situation of the light splitting unit 1032, as shown in fig. 25a and 25b, each first photodiode PD1 is configured to receive the first color light split by the four light splitting units 1032 corresponding to the first photodiode PD1 and passed through the first color filter unit 1011, each second photodiode PD2 is configured to receive the second color light split by the four light splitting units 1032 corresponding to the second photodiode PD2 and passed through the second color filter unit 1012, each third photodiode PD3 is configured to receive the third color light split by the four light splitting units 1032 corresponding to the third color filter unit 1013, and each fourth photodiode PD4 corresponds to one light splitting unit 1032 for receiving the infrared light split by the light splitting unit 1032 corresponding to the third color filter unit 1013.

In fig. 25a and 25b, the first color filter unit 1011, the second color filter unit 1012, and the third color filter unit 1013 are not illustrated. The circles in fig. 25a and 25b indicate that the infrared light is directly transmitted to the fourth photodiode PD4 after being split by the light splitting unit 1032, the arrows indicate that the three primary color light is split into the first color filter unit 1011, the second color filter unit 1012, and the third color filter unit 1013 in the fourth domain of the fourth photodiode PD4 after being split by the light splitting unit 1032, the first color light is transmitted to the first photodiode PD1 after the three primary color light passes through the first color filter unit 1011, the second color light is transmitted to the second photodiode PD2 after the three primary color light passes through the second color filter unit 1012, and the third color light is transmitted to the third photodiode PD3 after the three primary color light passes through the third color filter unit 1013.

Note that, in the case where the image sensor 10 includes a plurality of microlenses 104, one microlens 104 may be provided corresponding to one light splitting unit 1032.

In the third embodiment, the light splitting structure 103 first splits the three primary colors of light from the infrared light, transmits the three primary colors of light to the first color filter unit 1011, the second color filter unit 1012 and the third color filter unit 1013 of the color filter 101, and transmits the infrared light to the fourth photodiode PD4 through the hollow area 1014 of the color filter 101, so that the second embodiment can improve the amount of incident light compared to the prior art in which the color filter 101 corresponding to the fourth photodiode PD4 absorbs the three primary colors of light.

On this basis, with respect to the case where the color filter 101 in the image sensor 10 of the related art includes the red filter unit, the green filter unit, and the blue filter unit, since each of the first photodiodes PD1, each of the second photodiodes PD2, each of the third photodiodes PD3, and each of the fourth photodiodes PD4 in the third embodiment, that is, each of the first color pixels, each of the second color pixels, each of the third color pixels, and the fourth third color pixels in the pixel matrix receives the light distributed by the white light in the pixel space corresponding to the two color filter units of the related art, only a little spatial resolution is lost, and the spatial resolution is 70% of the related art.

In addition, in the third embodiment, the first photodiode PD1, the second photodiode PD2, the third photodiode PD3, and the fourth photodiode PD4 can receive light of three primary colors, and the received light is light of a single color, so that the difficulty of the post-algorithm processing can be reduced.

Example four

As shown in fig. 26, the image sensor 10 includes a photodiode array 102 and a light splitting structure 103.

The photodiode array 102 includes a plurality of first photodiodes PD1, a plurality of second photodiodes PD2, a plurality of third photodiodes PD3, and a plurality of fourth photodiodes PD 4.

The light splitting structure 103 is disposed on the light incident side of the photodiode array 102; the light splitting structure 103 is configured to transmit the received first color light to the first photodiode PD1, transmit the received second color light to the second photodiode PD2, transmit the received third color light to the third photodiode PD3, and transmit the received infrared light to the fourth photodiode PD 4.

The present embodiment is not limited to the pixel matrix design, that is, the arrangement of the plurality of first photodiodes PD1, the plurality of second photodiodes PD2, the plurality of third photodiodes PD3, and the plurality of fourth photodiodes PD4 in the photodiode array 102. In some examples, as shown in fig. 27, a plurality of first photodiodes PD1, a plurality of second photodiodes PD2, a plurality of third photodiodes PD3, and a plurality of fourth photodiodes PD4 are used to constitute a plurality of 2 × 2 fourth photodiode matrices N repeatedly arranged in the first direction X and the second direction Y; the first direction X is a row direction of the fourth photodiode matrix N, the second direction Y is a column direction of the fourth photodiode matrix N, and the first direction X is perpendicular to the second direction Y. Each fourth photodiode matrix N includes one first photodiode PD1, one second photodiode PD2, one third photodiode PD3, and one fourth photodiode PD 4.

The arrangement of one first photodiode PD1, one second photodiode PD2, one third photodiode PD3, and one fourth photodiode PD4 in the fourth photodiode matrix N is not limited. In some examples, as shown in fig. 27, in the fourth photodiode matrix N, the first photodiode PD1 is located at the upper left of the fourth photodiode matrix N, the second photodiode PD2 is located at the upper right of the fourth photodiode matrix N, the third photodiode PD3 is located at the lower right of the fourth photodiode matrix N, and the fourth photodiode PD4 is located at the lower left of the fourth photodiode matrix N.

It should be noted that, in the fourth embodiment, the light quantity and the light intensity received by the photodiode array 102 can be increased by adopting the two ways provided in the third embodiment, which may be referred to specifically for the third embodiment and will not be described herein again.

On this basis, in order to ensure that the light splitting structure 103 can transmit the received first color light to the first photodiode PD1, transmit the received second color light to the second photodiode PD2, transmit the received third color light to the third photodiode PD3, and transmit the received infrared light to the fourth photodiode PD4, the light splitting structure 103 may be divided into a plurality of light splitting units. Taking the arrangement of the first photodiode PD1, the second photodiode PD2, the third photodiode PD3 and the fourth photodiode PD4 in the photodiode array 102 as an example, as shown in fig. 27, the corresponding design relationships of the plurality of light splitting units 1032 with the first photodiode PD1, the second photodiode PD2, the third photodiode PD3 and the fourth photodiode PD4 in the photodiode array 102 are exemplarily provided.

As shown in fig. 28, the light splitting structure 103 includes a plurality of light splitting units 1032, and each of the first photodiode PD1, each of the second photodiode PD2, each of the third photodiode PD3, and each of the fourth photodiode PD4 corresponds to four light splitting units 1032.

Fig. 29 illustrates the light splitting of the light splitting units 1032, and as shown in fig. 29, each light splitting unit 1032 corresponds to one first photodiode PD1, one second photodiode PD2, one third photodiode PD3, and one fourth photodiode PD4, and is configured to transmit the received first color light to the first photodiode PD1, transmit the received second color light to the second photodiode PD2, transmit the received third color light to the third photodiode PD3, and transmit the received infrared light to the fourth photodiode PD 4.

Arrows in fig. 29 indicate that after the three primary color light and the infrared light pass through the light splitting unit 1032, the first color light is transmitted to the first photodiode PD1, the second color light is transmitted to the second photodiode PD2, the third color light is transmitted to the third photodiode PD3, and the infrared light is transmitted to the fourth photodiode PD 4.

In the fourth embodiment, since the light splitting structure 103 directly transmits the first color light to the first photodiode PD1, the second color light to the second photodiode PD2, the third color light to the third photodiode PD3, and the infrared light to the fourth photodiode PD4, the light splitting structure 103 does not absorb light, and thus the light incoming amount of the image sensor 10 can be increased.

On this basis, with respect to the case where the color filter 101 in the image sensor 10 of the related art includes the red filter unit, the green filter unit, and the blue filter unit, since each of the first photodiodes PD1, each of the second photodiodes PD2, each of the third photodiodes PD3, and each of the fourth photodiodes PD4 in the fourth embodiment, that is, each of the first color pixels, each of the second color pixels, each of the third color pixels, and the fourth third color pixels in the pixel matrix receives the light distributed by the white light in the pixel space corresponding to the two color filter units in the related art, only a little spatial resolution is lost, and the spatial resolution is 70% of the related art.

In addition, in the fourth embodiment, the first photodiode PD1, the second photodiode PD2, the third photodiode PD3, and the fourth photodiode PD4 can receive light of three primary colors, and the received light is light of a single color, so that the difficulty of the post-algorithm processing can be reduced.

The embodiment of the present application further provides a display device, which may also be referred to as a Liquid Crystal Display (LCD) device, and the display device may be, for example, a product or a component having any display function, such as a liquid crystal television, a digital photo frame, a mobile phone, a tablet computer, and the like.

As shown in fig. 30, the display device 40 includes a liquid crystal display panel 401, a backlight 402, and a light splitting structure 103.

The main structure of the liquid crystal display panel 401 includes an array substrate 4011, a counter substrate 4012, and a liquid crystal layer 4013 provided between the array substrate 4011 and the counter substrate 4012. It is understood that the liquid crystal display panel 401 includes a plurality of sub-pixels, and as shown in fig. 31, the plurality of sub-pixels includes a plurality of first color sub-pixels 401a, a plurality of second color sub-pixels 401b, and a plurality of third color sub-pixels 401 c.

In some examples, the first color subpixel 401a, the second color subpixel 401B, and the third color subpixel 401c are a red subpixel R, a green subpixel G, and a blue subpixel B, respectively.

Here, the arrangement of the plurality of first color sub-pixels 401a, the plurality of second color sub-pixels 401b, and the plurality of third color sub-pixels 401c in the liquid crystal display panel 401 is not limited. In some examples, as shown in fig. 31, a column of first color sub-pixels 401a, a column of second color sub-pixels 401B, and a column of third color sub-pixels 401c are alternately arranged in sequence along the first direction X, and fig. 31 illustrates an example where the first color sub-pixels 401a are red sub-pixels R, the second color sub-pixels 401B are green sub-pixels G, and the third color sub-pixels 401c are blue sub-pixels B.

The array substrate 4011 in the liquid crystal display panel 401 includes a substrate, and a thin film transistor and a pixel electrode disposed on the substrate and located in each sub-pixel, where the thin film transistor includes an active layer, a source electrode, a drain electrode, a gate electrode, and a gate insulating layer, the source electrode and the drain electrode are respectively in contact with the active layer, and the pixel electrode is electrically connected to the drain electrode of the thin film transistor. In some examples, the array substrate 4011 further includes a common electrode. In other examples, the counter box substrate 4012 includes a common electrode. Referring to fig. 30, the backlight source 402 is disposed on the light incident side of the lcd panel 401 for providing light to the lcd panel 401.

Here, the backlight 402 may include a plurality of conventional LEDs (light emitting diodes), or a plurality of mini-LEDs. The light splitting structure 103 is disposed between the liquid crystal display panel 401 and the backlight 402, and the light splitting structure 103 is configured to split the white light emitted from the backlight 402 into a first color light, a second color light, and a third color light, transmit the first color light to the first color sub-pixel, transmit the second color light to the second color sub-pixel, and transmit the third color light to the third color sub-pixel.

In addition, the specific structure of the light splitting structure 103 may refer to the first embodiment, and is not described herein again.

The display device 40 provided in the embodiment of the present application further includes a light splitting structure 103 disposed between the liquid crystal display panel 401 and the backlight 402, in addition to the liquid crystal display panel 401 and the backlight 402, since the light splitting structure 103 can split white light emitted from the backlight 402 into first color light, second color light, and third color light, transmit the first color light to the first color sub-pixel, transmit the second color light to the second color sub-pixel, and transmit the third color light to the third color sub-pixel, the first color sub-pixel of the liquid crystal display panel 401 can transmit the first color light, the second color sub-pixel can transmit the second color light, and the third color sub-pixel can transmit the third color light. On this basis, compared with the existing display device, by providing the color filter 101 on the liquid crystal display panel 401, the color filter 101 absorbs light of two colors of white light, and transmits light of another color, so as to ensure that the first color sub-pixel of the liquid crystal display panel 401 transmits light of the first color, the second color sub-pixel transmits light of the second color, and the third color sub-pixel transmits light of the third color. Note that, in some examples, the liquid crystal display panel 401 does not include the color filter 101. In other examples, considering that when the light splitting structure 103 splits white light emitted from the backlight 402 into first color light, second color light, and third color light to be transmitted to the first color sub-pixel, the second color sub-pixel, and the third color sub-pixel of the liquid crystal display panel 401, respectively, light transmitted from the first color sub-pixel, the second color sub-pixel, and the third color sub-pixel of the liquid crystal display panel 401 may not be light of a single color, which may affect the display effect, as shown in fig. 32, the liquid crystal display panel 401 includes the color filter 101; the color filter 101 includes a plurality of first color filter units 1011 corresponding to the first color sub-pixels, a plurality of second color filter units 1012 corresponding to the second color sub-pixels, and a plurality of third color filter units 1013 corresponding to the third color sub-pixels, wherein at least any one first color filter unit 1011 of the plurality of first color filter units 1011 allows only the first color light to transmit, at least any one second color filter unit 1012 of the plurality of second color filter units 1012 allows only the second color light to transmit, and at least any one third color filter unit 1013 of the plurality of third color filter units 1013 allows only the third color light to transmit.

It should be understood that the color filter 101 is disposed in the liquid crystal display panel 401, so that it is ensured that the light transmitted from the first color sub-pixel, the second color sub-pixel and the third color sub-pixel of the liquid crystal display panel 401 is light of a single color, and color cross is not generated, thereby being beneficial to improving the display effect of the display device.

It can be understood that the arrangement of the plurality of first color filter units 1011, the plurality of second color filter units 1012 and the plurality of third color filter units 1013 in the color filter 101 is the same as the arrangement of the plurality of first color sub-pixels 401a, the plurality of second color sub-pixels 401b and the plurality of third color sub-pixels 401c in the liquid crystal display panel 401, and therefore, the description thereof is omitted.

Here, the position where the color filter 101 is provided in the liquid crystal display panel 401 is not limited, and the color filter 101 may be provided on the cassette substrate 4012, in which case the cassette substrate 4012 may also be referred to as a color filter substrate, or the array substrate 4011 may also include the color filter 101. Fig. 32 illustrates an example in which the cartridge substrate 4012 includes a color filter 101.

On this basis, the display device 40 further includes an upper polarizer 403 and a lower polarizer 404 disposed on both sides of the liquid crystal display panel 401.

In some examples, the display device 40 provided herein may further include the image sensor 10 described above.

The above description is only for the specific embodiments of the present application, but the scope of the present application is not limited thereto, and any person skilled in the art can easily conceive of the changes or substitutions within the technical scope of the present application, and shall be covered by the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims

1. An image sensor, comprising:

a plurality of first photodiodes, a plurality of second photodiodes, a plurality of third photodiodes; the color filter comprises a plurality of first color filter units and a plurality of second color filter units; at least any one of the first color filter units allows first color light and second color light of three primary color light to transmit, and at least any one of the second color filter units allows second color light and third color light of the three primary color light to transmit;

a light splitting structure for splitting the first color light and the second color light transmitted from the first color filter unit, transmitting the first color light to the first photodiode, and transmitting the second color light to the second photodiode, and for splitting the second color light and the third color light transmitted from the second color filter unit, transmitting the second color light to the second photodiode, and transmitting the third color light to the third photodiode.

2. The image sensor of claim 1, wherein the light splitting structure is further configured to converge the first color light transmitted from the first color filter unit and the second color light, and wherein the light splitting structure is further configured to converge the second color light transmitted from the second color filter unit and the third color light.

3. The image sensor of claim 1 or 2, further comprising a plurality of microlenses disposed on the light-incident side of the color filter;

the micro lens is used for transmitting the three primary color lights to the first color filtering unit and the second color filtering unit after converging the three primary color lights.

4. The image sensor according to any one of claims 1 to 3, wherein a plurality of the first photodiodes, a plurality of the second photodiodes, and a plurality of the third photodiodes are used to form a plurality of 2 x 2 first photodiode matrices repeatedly arranged in the first direction and the second direction; wherein the first direction is a row direction of the first photodiode matrix, and the second direction is a column direction of the first photodiode matrix;

each of the first photodiode matrices includes one of the first photodiodes, two of the second photodiodes, and one of the third photodiodes.

5. The image sensor of claim 4, wherein the first photodiode is located at an upper left corner of the first photodiode matrix, two of the second photodiodes are located at an upper right corner and a lower left corner of the first photodiode matrix, respectively, and the third photodiode is located at a lower right corner of the first photodiode matrix.

6. The image sensor according to any one of claims 1 to 5, wherein a plurality of the first color filter units and a plurality of the second color filter units are alternately arranged in a direction parallel to two diagonal lines of the first photodiode matrix;

the first color filter unit covers the first photodiode and a part of the second photodiode located in the four neighborhoods of the first photodiode; the second color filter unit covers the third photodiode and a part of the second photodiode located in the fourth neighborhood of the third photodiode; each of the second photodiodes is covered with two adjacent first color filter units and two adjacent second color filter units.

7. The image sensor of any of claims 1-6, wherein the light splitting structure comprises a plurality of light splitting cells; each of the first photodiodes, each of the second photodiodes, and each of the third photodiodes correspond to four of the light splitting units;

each light splitting unit corresponds to one first color filter unit or one second color filter unit; the light splitting unit corresponding to the first color filter unit receives the first color light and the second color light transmitted from the first color filter unit and splits and transmits the first color light and the second color light to one first photodiode and one second photodiode; the light splitting unit corresponding to the second color filter unit receives the second color light and the third color light transmitted by the second color filter unit, and transmits the second color light and the third color light to one second photodiode and one third photodiode separately.

8. The image sensor of any of claims 1-6, wherein the light splitting structure comprises a plurality of light splitting cells; each of the first photodiodes, each of the second photodiodes, and each of the third photodiodes correspond to four of the light splitting units;

each light splitting unit corresponds to the first color filter unit and the second color filter unit; each of the light splitting units receives the first color light and the second color light transmitted from the first color filter unit, and separately transmits the first color light and the second color light to one of the first photodiodes and two of the second photodiodes, and receives the second color light and the third color light transmitted from the second color filter unit, and separately transmits the second color light and the third color light to one of the third photodiodes and two of the second photodiodes;

or, each of the light splitting units receives the first color light and the second color light transmitted from the first color filter unit, and receives the second color light and the third color light transmitted from the second color filter unit, and transmits the first color light to one of the first photodiodes, the second color light to one of the second photodiodes, and the third color light to one of the third photodiodes.

9. The image sensor of any of claims 1-6, wherein the light splitting structure comprises a plurality of light splitting cells; each first photodiode and each third photodiode correspond to four light splitting units; each second photodiode corresponds to one light splitting unit;

each light splitting unit corresponds to two first color filter units which are opposite to each other and two second color filter units which are opposite to each other, receives light transmitted by the two first color filter units which are opposite to each other and the two second color filter units which are opposite to each other, transmits the second color light to one second photodiode, transmits the first color light to the two first photodiodes which are opposite to each other, and transmits the third color light to the two third photodiodes which are opposite to each other.

10. The image sensor according to any one of claims 1 to 9, wherein the light splitting structure comprises a plurality of micro-nano dielectric pillars distributed in an array.

11. The image sensor of any of claims 1-10, wherein the first color filter element is a yellow filter element and the second color filter element is a cyan filter element.

12. The image sensor of claim 11, wherein the first color light is red light, the second color light is green light, and the third color light is blue light.

13. An image sensor, comprising:

a plurality of first photodiodes, a plurality of second photodiodes, a plurality of third photodiodes, and a plurality of fourth photodiodes;

the color filter comprises a plurality of first color filter units, a plurality of second color filter units and a plurality of third color filter units; at least any one of the first color filter units allows a first color light of infrared light and a third primary color light to pass through, at least any one of the second color filter units allows a second color light of infrared light and the third primary color light to pass through, and at least any one of the third color filter units allows a third color light of infrared light and the third primary color light to pass through;

a light splitting structure for splitting the first color light and the infrared light transmitted from the first color filter unit, splitting the second color light and the infrared light transmitted from the second color filter unit, splitting the third color light and the infrared light transmitted from the third color filter unit, and transmitting the first color light to the first photodiode, the second color light to the second photodiode, the third color light to the third photodiode, and the infrared light to the fourth photodiode.

14. The image sensor of claim 13, wherein the light splitting structure is further configured to converge the first color light transmitted from the first color filter unit, converge the second color light transmitted from the second color filter unit, converge the third color light transmitted from the third color filter unit, and converge the infrared light transmitted from the first, second, and third color filter units.

15. The image sensor of claim 13 or 14, further comprising a plurality of microlenses disposed on the light-incident side of the color filter;

the micro lens is used for converging the three primary colors and the infrared light and transmitting the converged light to the first color filter unit, the second color filter unit and the third color filter unit.

16. The image sensor of any of claims 13-15, further comprising a first infrared cut filter disposed between the light splitting structure and the first, second, and third photodiodes; the first infrared cut-off filter is used for filtering infrared light;

the first infrared cut-off filter comprises a plurality of hollowed-out areas, and the projection of the fourth photodiode on the first infrared cut-off filter and the hollowed-out areas have overlapped areas.

17. The image sensor as claimed in any one of claims 13 to 16, wherein one of said first color filter units covers one of said first photodiodes and a portion of said fourth photodiode located in a neighborhood of four of said first photodiode, one of said second color filter units covers one of said second photodiodes and a portion of said fourth photodiode located in a neighborhood of four of said second photodiode, and one of said third color filter units covers said third photodiode and a portion of said fourth photodiode located in a neighborhood of four of said third photodiode.

18. The image sensor as claimed in any one of claims 13 to 17, wherein a row of the first photodiodes, a row of the fourth photodiodes, a row of the second photodiodes, a row of the fourth photodiodes, a row of the third photodiodes, and a row of the fourth photodiodes are alternately arranged in sequence along the first direction.

19. The image sensor according to any one of claims 13 to 17, wherein a plurality of the first photodiodes, a plurality of the second photodiodes, a plurality of the third photodiodes, and a plurality of the fourth photodiodes are used to constitute a plurality of 2 x 2 second photodiode matrices and 2 x 2 third photodiode matrices alternately arranged in the first direction and the second direction; wherein the first direction is a row direction of the second and third photodiode matrices, and the second direction is a column direction of the second and third photodiode matrices;

each of the second photodiode matrices includes one of the first photodiodes, one of the second photodiodes, and two of the fourth photodiodes; each of the third photodiode matrices includes one of the second photodiodes, one of the third photodiodes, and two of the fourth photodiodes.

20. The image sensor of claim 19, wherein in the second photodiode matrix, two of the fourth photodiodes are located at an upper left corner and a lower right corner of the second photodiode matrix, respectively, and the first photodiode and the second photodiode are located at an upper right corner and a lower left corner of the second photodiode matrix, respectively;

in the third photodiode matrix, two of the fourth photodiodes are respectively located at the upper left corner and the lower right corner of the third photodiode matrix, and the second photodiode and the third photodiode are respectively located at the lower left corner and the upper right corner of the third photodiode matrix.

21. The image sensor as claimed in any one of claims 13 to 20, wherein the light splitting structure comprises a plurality of light splitting cells, each of the first photodiode, each of the second photodiode, each of the third photodiode, and each of the fourth photodiode corresponding to four of the light splitting cells;

each light splitting unit corresponds to one of the first color filter unit, the second color filter unit and the third color filter unit; the light splitting unit corresponding to the first color filter unit receives the first color light and the infrared light transmitted from the first color filter unit and separately transmits the first color light and the infrared light to the first photodiode and the fourth photodiode; the light splitting unit corresponding to the second color filter unit receives the second color light and the infrared light transmitted from the second color filter unit and separately transmits the second color light and the infrared light to the second photodiode and the fourth photodiode; the light splitting unit corresponding to the third color filtering unit receives the third color light and the infrared light transmitted from the third color filtering unit, and separately transmits the third color light and the infrared light to the third photodiode and the fourth photodiode.

22. The image sensor according to any one of claims 13-20, wherein the light splitting structure comprises a plurality of light splitting cells, and each of the first photodiode, each of the second photodiode, and each of the third photodiode corresponds to one of the light splitting cells; each fourth photodiode corresponds to four light splitting units and is used for receiving the infrared light split by any one of the four light splitting units corresponding to the fourth photodiode;

each light splitting unit corresponds to one of the first color filter unit, the second color filter unit and the third color filter unit; the light splitting unit corresponding to the first color filter unit receives the first color light and the infrared light transmitted by the first color filter unit, and transmits the first color light and the infrared light to the first photodiode and any fourth photodiode in the fourth neighborhood thereof; the light splitting unit corresponding to the second color filter unit receives the second color light and the infrared light transmitted by the second color filter unit, and transmits the second color light and the infrared light to the second photodiode and any fourth photodiode in the fourth neighborhood thereof; the light splitting unit corresponding to the third color filtering unit receives the third color light and the infrared light transmitted from the third color filtering unit, and transmits the third color light and the infrared light to the third photodiode and any fourth photodiode in the fourth neighborhood thereof.

23. The image sensor according to claim 19 or 20, wherein the light splitting structure comprises a plurality of light splitting units, each of the first photodiode, each of the second photodiode, and each of the third photodiode corresponds to four light splitting units; each fourth photodiode corresponds to one light splitting unit;

each light splitting unit corresponds to one first color filter unit, two second color filter units and one third color filter unit; each light splitting unit receives the first color light transmitted by the corresponding first color filter unit, transmits the first color light to one first photodiode, receives the second color light transmitted by the corresponding second color filter unit, transmits the second color light to two second photodiodes, receives the third color light transmitted by the corresponding third color filter unit, and transmits the third color light to one third photodiode; each light splitting unit also receives infrared light transmitted by the first color filter unit, the second color filter unit and the third color filter unit and transmits the infrared light to the fourth photodiode.

24. The image sensor according to claim 19 or 20, wherein the light splitting structure comprises a plurality of light splitting units, each of the first photodiode, each of the second photodiode, each of the third photodiode, and each of the fourth photodiode corresponds to four light splitting units;

each light splitting unit corresponds to the first color filter unit and the second color filter unit, or corresponds to the second color filter unit and the third color filter unit;

the light splitting unit corresponding to the first color filter unit and the second color filter unit receives the first color light and the infrared light transmitted from the first color filter unit, and receives the second color light and the infrared light transmitted from the second color filter unit, and transmits the first color light to the first photodiode, the second color light to the second photodiode, and the infrared light to two fourth photodiodes;

the light splitting unit corresponding to the second color filter unit and the third color filter unit receives the second color light and the infrared light transmitted by the second color filter unit, receives the third color light and the infrared light transmitted by the third color filter unit, transmits the second color light to the second photodiode, transmits the third color light to the third photodiode, and transmits the infrared light to two fourth photodiodes.

25. An image sensor, comprising:

the color filter comprises a plurality of first color filter units, a plurality of second color filter units, a plurality of third color filter units and a plurality of hollow areas; at least any one of the first color filter units allows a first color light of the three primary colors to transmit, at least any one of the second color filter units allows a second color light of the three primary colors to transmit, and at least any one of the third color filter units allows a third color light of the three primary colors to transmit;

a plurality of first photodiodes corresponding to the first color filter units, a plurality of second photodiodes corresponding to the second color filter units, a plurality of third photodiodes corresponding to the third color filter units, and a plurality of fourth photodiodes corresponding to the hollow areas;

the light splitting structure is used for splitting infrared light and the three primary colors of light, the infrared light is transmitted to a fourth photodiode through the hollow area of the color filter, and the three primary colors of light are transmitted to the first color filter unit, the second color filter unit and the third color filter unit.

26. The image sensor of claim 25, wherein a row of the first photodiodes, a row of the fourth photodiodes, a row of the second photodiodes, a row of the fourth photodiodes, a row of the third photodiodes, and a row of the fourth photodiodes alternate in sequence along the first direction.

27. The image sensor of claim 25, wherein a plurality of the first photodiodes, a plurality of the second photodiodes, a plurality of the third photodiodes, and a plurality of the fourth photodiodes are used to form a plurality of 2 x 2 second photodiode matrices and 2 x 2 third photodiode matrices alternately arranged in the first direction and the second direction; wherein the first direction is a row direction of the second and third photodiode matrices, and the second direction is a column direction of the second and third photodiode matrices;

28. The image sensor of claim 27, wherein in the second photodiode matrix, two of the fourth photodiodes are located at an upper left corner and a lower right corner of the second photodiode matrix, respectively, and the first photodiode and the second photodiode are located at an upper right corner and a lower left corner of the second photodiode matrix, respectively;

29. The image sensor as in any one of claims 25-28, wherein the light splitting structure comprises a plurality of light splitting cells, each of the first photodiodes, each of the second photodiodes, and each of the third photodiodes corresponding to four of the light splitting cells; each first photodiode is used for receiving four light splitting units which correspond to the first photodiode and splitting the light through the first color filter unit, each second photodiode is used for receiving four light splitting units which correspond to the second photodiode and splitting the light through the second color filter unit, each third photodiode is used for receiving four light splitting units which correspond to the third photodiode and splitting the light through the third color filter unit, and each fourth photodiode corresponds to one light splitting unit and is used for receiving infrared light which is split by the light splitting unit corresponding to the fourth photodiode.

30. An image sensor, comprising:

and the light splitting structure is used for transmitting the received first color light to the first photodiode, transmitting the received second color light to the second photodiode, transmitting the received third color light to the third photodiode, and transmitting the received infrared light to the fourth photodiode.

31. The image sensor of claim 30, wherein a plurality of the first photodiodes, a plurality of the second photodiodes, a plurality of the third photodiodes, and a plurality of the fourth photodiodes are used to form a plurality of 2 x 2 fourth photodiode matrices repeatedly arranged in the first direction and the second direction; wherein the first direction is a row direction of the fourth photodiode matrix, and the second direction is a column direction of the fourth photodiode matrix;

each of the fourth photodiode matrices includes one of the first photodiodes, one of the second photodiodes, one of the third photodiodes, and one of the fourth photodiodes.

32. The image sensor of claim 31, wherein in the fourth photodiode matrix, the first photodiode is located at an upper left of the fourth photodiode matrix, the second photodiode is located at an upper right of the fourth photodiode matrix, the third photodiode is located at a lower right of the fourth photodiode matrix, and the fourth photodiode is located at a lower left of the fourth photodiode matrix.

33. The image sensor as in any one of claims 30-32, wherein the light splitting structure comprises a plurality of light splitting cells, each of the first photodiodes, each of the second photodiodes, each of the third photodiodes, and each of the fourth photodiodes corresponding to four of the light splitting cells;

each light splitting unit corresponds to one of the first photodiode, one of the second photodiode, one of the third photodiode, and one of the fourth photodiode, and is configured to transmit the received first color light to the first photodiode, transmit the received second color light to the second photodiode, transmit the received third color light to the third photodiode, and transmit the received infrared light to the fourth photodiode.

34. An image pickup apparatus includes a lens and an image sensor; the lens is used for converging light emitted by a shot object or reflected light onto the image sensor; the image sensor is used for converting the received optical image into a digital signal;

wherein the image sensor is the image sensor of any one of claims 1-12; the image pickup apparatus further includes a second infrared cut filter; the second infrared cut-off filter is arranged between the lens and the image sensor and used for filtering infrared light;

alternatively, the image sensor is as claimed in any one of claims 13-33.

35. The image capturing apparatus according to claim 34, wherein the image capturing apparatus further comprises an image processor; the image processor is used for processing the digital signal and outputting an image of a shot object.

36. A display device, comprising:

a liquid crystal display panel including a plurality of first color sub-pixels, a plurality of second color sub-pixels, and a plurality of third color sub-pixels;

the backlight source is used for providing a light source for the liquid crystal display panel;

and the light splitting structure is used for splitting the white light emitted from the backlight source into first color light, second color light and third color light, transmitting the first color light to the first color sub-pixel, transmitting the second color light to the second color sub-pixel and transmitting the third color light to the third color sub-pixel.

37. The display device according to claim 36, wherein the liquid crystal display panel comprises a color filter;

the color filter comprises a plurality of first color filter units corresponding to the first color sub-pixels, a plurality of second color filter units corresponding to the second color sub-pixels and a plurality of third color filter units corresponding to the third color sub-pixels; at least any one of the first color filter units allows only the first color light to pass through, at least any one of the second color filter units allows only the second color light to pass through, and at least any one of the third color filter units allows only the third color light to pass through.