CN212435793U - Image sensor and electronic device - Google Patents

Abstract

Provided are an image sensor and an electronic device, which can improve the performance of the image sensor. The image sensor includes: the filter unit array comprises a plurality of filter unit groups, each filter unit group in the plurality of filter unit groups comprises 2 white filter units and 2 color filter unit sets, each color filter unit set is equal to the area of each white filter unit, and each color filter unit set comprises 4 color filter units; the pixel unit array comprises a plurality of pixel units, the pixel unit array is positioned below the light filtering unit array, and a plurality of pixel units in the pixel unit array correspond to a plurality of light filtering units in the light filtering unit array one by one. Through the scheme of this application embodiment, when guaranteeing image color information, can also promote image dynamic range and light inlet quantity, and then promote image sensor's performance.

Description

Image sensor and electronic device

The present application claims priority from the chinese patent office, application No. 202010410639.2, entitled "image sensor and electronic device," filed on 15/5/2020, the entire contents of which are incorporated herein by reference.

Technical Field

The present application relates to the field of sensors, and more particularly, to an image sensor and an electronic device.

Background

An image sensor is an electronic device that converts an optical image into a digital signal, and generally includes a pixel cell array composed of a plurality of pixel cells, each pixel cell in the pixel cell array being used to form one pixel value in the image. In order to enable the image sensor to capture a color image, a Color Filter (CF) may be disposed above the pixel unit so that the pixel unit may receive a light signal of a specific color, forming a pixel value corresponding to the light signal of the specific color.

However, when the color filter is provided, the light received by each pixel unit is reduced, which results in a reduced signal-to-noise ratio (SNR) of the image, and thus affects the image quality. And if the image sensor is applied to a mobile device, the size of the image sensor is limited, and the photosensitive area of the corresponding pixel unit array is also limited, so that the image quality is further limited in a low-light environment.

SUMMERY OF THE UTILITY MODEL

The embodiment of the application provides an image sensor and electronic equipment, and aims to solve the problem of image quality reduction in a low-illumination environment.

In a first aspect, an image sensor is provided, including: the filter unit array comprises a plurality of filter unit groups, each filter unit group in the plurality of filter unit groups comprises 2 white filter units and 2 color filter unit sets, each color filter unit set is equal to the area of each white filter unit, and each color filter unit set comprises 4 color filter units; the pixel unit array comprises a plurality of pixel units, the pixel unit array is positioned below the light filtering unit array, and a plurality of pixel units in the pixel unit array correspond to a plurality of light filtering units in the light filtering unit array one by one.

Through the scheme of this application embodiment, because in every filtering unit group, include white filtering unit and colored filtering unit simultaneously, wherein, 1 area of white filtering unit is the sum of 4 colored filtering unit areas, when guaranteeing image color information, can also promote image dynamic range and light inlet quantity, and then promote image quality. In addition, in the subsequent image processing process, the pixel data amount can be reduced, and the image processing speed can be increased to further improve the performance of the image sensor.

In some possible embodiments, the filtering unit group includes first, second and third color filtering units of different colors, and the number of the first color filtering units is equal to the sum of the numbers of the second and third color filtering units.

In some possible embodiments, in the filter unit group, the number of the second color filter units is equal to the number of the third color filter units.

In some possible embodiments, in the set of filter units, the 2 white filter units are disposed adjacent to each other at a common vertex, and the 2 color filter units are disposed adjacent to each other at a common vertex.

In some possible embodiments, in the 2 color filter sets, the first color filter set includes 2 first color filter units and 2 second color filter units, and the second color filter set includes 2 first color filter units and 2 third color filter units.

In some possible embodiments, in each color filter set, the color filter units of the same color are arranged adjacent to each other at a common corner.

In some possible embodiments, in the 2 color filter sets, the relative position relationship of the color filters of the same color is the same.

In some possible embodiments, in the set of filter units, 4 first color filter units are disposed on one diagonal line, and 2 white filter units are disposed on the other diagonal line.

In some possible embodiments, in the filtering unit group, 2 second color filtering units adjacent to a common vertex angle and 2 third color filtering units adjacent to a common vertex angle are arranged on one diagonal line, and 2 white filtering units are arranged on the other diagonal line.

In some possible embodiments, in each color filter set, color filters of the same color are arranged adjacent to each other.

In some possible embodiments, the relative position relationship of the color filter units of the same color is different in the 2 color filter unit sets.

In some possible embodiments, in the filtering unit group, 1 second color filtering unit, 2 first color filtering units and 1 third color filtering unit which are adjacent to each other at a common vertex angle are sequentially disposed on one diagonal line, and 2 white filtering units are disposed on the other diagonal line.

In some possible embodiments, in the 2 color filter sets, the first color filter set includes 4 first color filter units, and the second color filter set includes 2 second color filter units and 2 third color filter units.

In some possible embodiments, in the second set of color filter units, color filter units of the same color are adjacently disposed on the same side.

In some possible embodiments, in the filtering unit group, 2 first color filtering units, 1 second color filtering unit and 1 third color filtering unit which are adjacent to each other at a common vertex angle are sequentially disposed on one diagonal line, and 2 white filtering units are disposed on the other diagonal line.

In some possible embodiments, the first color filter unit, the second color filter unit, and the third color filter unit are configured to pass light signals of three colors respectively, and the wavelength bands of the light signals of the three colors cover the visible light wavelength band.

In some possible embodiments, the first color filter unit, the second color filter unit, and the third color filter unit have three colors of red, green, blue, cyan, magenta, and yellow, respectively.

In some possible embodiments, the first color filter unit is a green filter unit, the second color filter unit and the third color filter unit are a red filter unit and a blue filter unit, respectively.

In some possible embodiments, the image sensor further includes: and the micro lens array comprises a plurality of micro lenses, is positioned above the light filtering unit array and is used for converging the optical signals returned by the shooting object to the light filtering unit array, wherein the plurality of micro lenses in the micro lens array correspond to the plurality of light filtering units in the light filtering unit array one by one.

In some possible embodiments, a first microlens in the microlens array corresponds to a white filter unit in the filter unit array, and a second microlens in the microlens array corresponds to a color filter unit in the filter unit array; wherein the radius of the first micro lens is 2 times of the radius of the second micro lens.

In some possible embodiments, a white pixel unit in the pixel unit array corresponds to a white filter unit in the filter unit array, and a color pixel unit in the pixel unit array corresponds to a color filter unit in the filter unit array; wherein the area of the first pixel unit is 4 times of the area of the second pixel unit.

In some possible embodiments, the pixel values of the color pixel cells in the pixel cell array are used for generating first image data of the photographic subject, the pixel values of the white pixel cells in the pixel cell array are used for generating second image data of the photographic subject, and the first image data and the second image data are used for synthesizing a target image of the photographic subject; the white pixel unit is a pixel unit corresponding to the white filter unit, and the color pixel unit is a pixel unit corresponding to the color filter unit.

In some possible embodiments, pixel values of color pixel cells in the pixel cell array are used to generate first image data in Bayer format by data processing.

In some possible embodiments, the resolution of the first image data is half of the resolution of the pixel cell array.

In some possible embodiments, the second image data and the first image data have the same resolution; alternatively, the resolution of the second image data is half the resolution of the first image data.

In some possible embodiments, the image sensor is a complementary metal oxide semiconductor CMOS image sensor, or a charge coupled device CCD image sensor.

In a second aspect, an electronic device is provided, comprising: the image sensor of the first aspect or any one of the possible embodiments of the first aspect.

Drawings

Fig. 1 is a schematic structural diagram of an image sensor according to an embodiment of the present disclosure.

Fig. 2 is a schematic top view of another image sensor provided in an embodiment of the present application.

Fig. 3 is a schematic cross-sectional view of the image sensor of fig. 2 along a-a'.

Fig. 4 is another schematic cross-sectional view of the image sensor of fig. 2 along the direction a-a'.

Fig. 5 is a schematic arrangement diagram of a filter unit in a filter unit array according to an embodiment of the present application.

Fig. 6 to 17 are schematic arrangement diagrams of filter units in several filter unit groups provided in embodiments of the present application.

Fig. 18 is a schematic flowchart of an image processing method according to an embodiment of the present application.

Fig. 19 is an image schematic diagram of the image processing method of fig. 18.

Detailed Description

The technical solutions in the embodiments of the present application will be described below with reference to the accompanying drawings.

It should be understood that the specific examples are provided herein only to assist those skilled in the art in better understanding the embodiments of the present application and are not intended to limit the scope of the embodiments of the present application.

It should also be understood that the various embodiments described in this specification can be implemented individually or in combination, and the examples in this application are not limited thereto.

The technical solution of the embodiment of the present application may be applied to various image sensors, such as a Complementary Metal Oxide Semiconductor (CMOS) image sensor (CIS) or a Charge Coupled Device (CCD) image sensor, but the embodiment of the present application is not limited thereto.

As a common application scenario, the image sensor provided in the embodiment of the present application may be applied to a smart phone, a camera, a tablet computer, and other mobile terminals or other terminal devices having an imaging function.

Fig. 1 shows a schematic structural diagram of an image sensor. As shown in fig. 1, the image sensor 100 includes: a pixel array (pixel array)110, a row selection circuit 120, a column selection circuit 130, a control circuit 140, an analog to digital converter (ADC) circuit 150, a front-end signal processing circuit 160, and a back-end signal processing circuit 170.

Specifically, as shown in fig. 1, a plurality of square pixel cells in the pixel cell array 110 are arranged in M rows × N columns, where M, N is a positive integer. Generally, the row direction of the M rows and the column direction of the N columns are perpendicular to each other on the plane of the pixel unit array 110. In some cases, for convenience of description, in one plane, two directions perpendicular to each other, such as a row direction and a column direction in the present application, may be referred to as a horizontal direction and a vertical direction.

In the pixel cell array 110 shown in fig. 1, any one side of each square pixel cell is parallel or perpendicular to the row direction or the column direction.

Optionally, the pixel unit may include a photodiode, a field effect switching transistor, and other devices for receiving an optical signal and converting the optical signal into a corresponding electrical signal.

Alternatively, if the image sensor needs to collect a color image, a Color Filter Array (CFA) may be disposed above the pixel unit array 110, wherein one color filter unit may be disposed above each pixel unit, and for the purpose of description, the pixel unit above which the color filter unit is disposed is also referred to as a color pixel unit, for example, the pixel unit above which the red filter unit is disposed is referred to as a red pixel unit (denoted by R in fig. 1), the pixel unit above which the green filter unit is disposed is referred to as a green pixel unit (denoted by G in fig. 1), and the pixel unit above which the blue filter unit is disposed is referred to as a blue pixel unit (denoted by B in fig. 1).

Currently, most CFAs of image sensors use a Bayer (Bayer) format based on three primary colors of RGB, for example, as shown in fig. 1, a CFA in the Bayer format is disposed above the pixel cell array 110, and the pixel cell array 110 uses 2 × 2 pixel cells as basic cells, each of which includes 1 red pixel cell, 1 blue pixel cell, and 2 green pixel cells, wherein 2 green pixel cells are disposed adjacent to each other at a common vertex angle.

The row selection circuit 120 is connected to each row of pixel cells in the pixel cell array 110 through M row control lines, and may be used to turn on and off each pixel cell in each row of pixel cells. For example, the row selection circuit 120 is connected to the gate of the fet of each pixel in the first row of the pixel array 110 via a row control line, and controls the operating state of the photodiode by turning on or off the fet. Wherein, M row control lines are all parallel to the horizontal direction.

The column selection circuit 130 is connected to each column of pixel cells in the pixel cell array 110 through N column control lines, and may be configured to select a signal value output of each pixel cell in each column. For example, the column selection circuit 130 is connected to the source of the field effect switch tube of each pixel unit in the first column in the pixel unit array 110 through a column control line, and controls the output of the electrical signal converted by the photodiode. Wherein, N column control lines are all parallel to the vertical direction.

The control circuit 140 is connected to the row selection circuit 120 and the column selection circuit 130, and is configured to provide timing for the row selection circuit 120 and the column selection circuit 130, control the row selection circuit 120 and the column selection circuit 130 to select a pixel unit in the pixel unit array 110, and output a pixel value of the pixel unit.

Optionally, after the row selecting circuit 120, the column selecting circuit 130 and the control circuit 140 cooperate to output the pixel value generated by the pixel unit array 110, the pixel value of the pixel unit array 110 is transmitted to the ADC circuit 150 for analog-to-digital conversion, and the analog pixel value is converted into a digital pixel value to form a digital image, so that the subsequent signal processing circuit 160 can perform image processing conveniently to output an optimized color image.

Alternatively, the signal processing circuit 160 may include, but is not limited to, an Image Signal Processor (ISP) for performing linearization processing, dead pixel removal, noise removal, color correction, demosaic (demosaic), Automatic Exposure Control (AEC), Automatic Gain Control (AGC), Automatic White Balance (AWB), and the like on the digital image.

With the above-described image sensor 100 employing the Bayer format CFA, the red pixel cell can receive only the red light signal, the green pixel cell can receive only the green light signal, the blue pixel cell can receive only the blue light signal, and the intensity of the light signal received by each pixel cell is small, resulting in a large SNR of an image, thereby affecting image quality.

Based on the above problem, the present application provides an image sensor, in which a white filtering unit is added in a CFA, a part of pixel units in a pixel unit array receives color light signals, and a part of pixel units receives white light signals, so as to increase the intensity of the light signals received by the part of pixel units, and on this basis, the pixel values of a plurality of pixel units in the pixel unit array are processed, so that on the basis of ensuring image color information, image quality parameters such as SNR and dynamic range of an image are improved, and an optimized color image is obtained.

Fig. 2 is a schematic top view of an image sensor 200 according to an embodiment of the present disclosure, and fig. 3 is a schematic cross-sectional view of the image sensor 200 along a direction a-a'.

As shown in fig. 2 and 3, the image sensor 200 includes:

the filter unit array 210 includes a plurality of filter unit groups 211, each filter unit group in the plurality of filter unit groups 211 includes 2 white filter units and 2 color filter unit sets, wherein each color filter unit set is equal to an area of each white filter unit, and each color filter unit set includes 4 color filter units, that is, a sum of the areas of the white filter units and the areas of the 4 color filter units is equal;

the pixel unit array 220 is located below the filter unit array 210, and includes a plurality of pixel units, and the pixel units in the pixel unit array correspond to the filter units in the filter unit array one to one.

In one possible implementation, as shown in fig. 3, a plurality of filter units in the filter unit array 210 may be disposed on an upper surface of a plurality of pixel units in the pixel unit array 220; in another possible implementation, the plurality of filter units in the filter unit array 210 may be disposed above the plurality of pixel units in the pixel unit array 220 in a floating manner.

Further, as shown in fig. 3, as an example, each filter unit in the filter unit array 210 is correspondingly disposed right above each pixel unit in the pixel unit array 220, in other words, the center of each filter unit coincides with the center of its corresponding pixel unit in the vertical direction. Except for this way, each filter unit in the filter unit array 210 is correspondingly disposed above and obliquely above each pixel unit in the pixel unit array 220, at this time, each pixel unit in the pixel unit array 220 may receive an optical signal in an oblique direction, and the embodiment of the present application does not limit the specific position of the filter unit array 210.

The pixel unit corresponding to the color filter unit in the pixel unit array 220 is configured to receive the color light signal passing through the color filter unit and output a color pixel value correspondingly; the pixel unit corresponding to the white filter unit in the pixel unit array 220 is configured to receive the white light signal passing through the white filter unit and output a corresponding white pixel value; the colored light signal and the white light signal are used together to generate a target image of a photographic subject. For example, a red pixel cell corresponding to the red filter cell receives a red light signal, and a pixel value output correspondingly may be referred to as a red pixel value; the white pixel unit corresponding to the white filter unit receives the white light signal, and the pixel value output correspondingly may be referred to as a white pixel value. Alternatively, the area of the white pixel cell is equal to the sum of the areas of the 4 color pixel cells.

Further, as shown in FIG. 4, FIG. 4 shows a schematic cross-sectional view of another image sensor 200 along the A-A' direction.

As shown in fig. 4, the image sensor 200 includes, in addition to the filter cell array 210 and the pixel cell array 220 described above:

and a microlens array 230 disposed above the filter unit array 210, and configured to converge an optical signal returned by the photographic subject to the filter unit array 210, and reduce crosstalk of the optical signal between adjacent pixel units, where the microlens array 230 includes a plurality of microlenses, and the plurality of microlenses are in one-to-one correspondence with the plurality of filter units in the filter unit array 210 and the plurality of pixel units in the pixel unit array 220.

In some embodiments, the pixel structure in the image sensor may be referred to as an on-chip lens (OCL) pixel structure.

As shown in fig. 4, in the embodiment of the present application, in the microlens array, two sizes of microlenses are included, wherein a radius of a first microlens corresponding above the white filter unit may be 2 times a radius of a second microlens corresponding above the color filter unit.

Optionally, as shown in fig. 4, in the image sensor 200, a dielectric layer 240 may be further included between the filter cell array 210 and the pixel cell array 220 for connecting the pixel cell array 220 and the filter cell array 210.

In addition, the filter cell array 210 may further include a dielectric 215 and a reflective grid 216 located at the periphery; the reflection grid is used for reflecting the optical signals incident at large angles and preventing the optical signals from being lost.

The pixel unit array 220 may include a semiconductor substrate 221 and a photosensitive element 222, wherein the photosensitive element 222 is located in the semiconductor substrate 221, and the photosensitive element 222 includes, but is not limited to, a Photodiode (PD). Optionally, the pixel unit array 220 may further include an isolation region 223 between two photosensitive elements 222 to prevent electrical signal interference between two adjacent photosensitive elements.

Further, as can be seen from fig. 4, the photosensitive area of the photosensitive element in the white pixel unit corresponding to the first microlens is larger than the photosensitive area of the photosensitive element in the color pixel unit corresponding to the second microlens. I.e. the light sensitive area of the light sensitive element in the white pixel unit may be 4 times the light sensitive area of the light sensitive element in the color pixel unit.

It is understood that the image sensor 200 may include other stacked structures besides the basic structure shown in fig. 4, such as at least one metal interconnection layer to electrically connect a plurality of pixel units in a pixel unit array, and the like, and the structure of the image sensor is not limited thereto in the embodiments of the present application.

Alternatively, the top view shown in fig. 2 is also a schematic arrangement diagram of the filter unit array 210 according to the embodiment of the present application.

As shown in fig. 2, in the embodiment of the present application, each filter unit in a filter unit group is a square filter unit, where 4 square color filter units form a square color filter unit set, and 2 color filter unit sets and 2 white filter units form a square filter unit group. Optionally, the 2 white filter units are arranged at the same top angle on one diagonal line of the filter unit group, and the 2 color filter units are arranged at the same top angle on the other diagonal line of the filter unit group.

It should be noted that, in the present application, the white filter unit refers to a filter or a filter material for transmitting white light, and in some embodiments, the white filter unit may also be a transparent material or an air gap for transmitting all optical signals including white light in the environment. In particular, the white light may be a mixture of colored lights. For example, light of three primary colors in the spectrum: blue, red and green, mixed in a certain proportion to obtain white light, or the mixture of all visible light in the spectrum is also white light.

Correspondingly, the color filter unit refers to a filter or a filter material for transmitting color light. Specifically, the colored light may be a light signal in any wavelength range in the visible light spectrum, for example, the red filter unit may be configured to transmit red light, which may be a light signal in the wavelength range of 620nm to 750nm in the visible light spectrum. Similarly, the color filter units of other colors are also used to transmit the light signals of the corresponding colors.

Through the scheme of this application embodiment, because in every filtering unit group, include white filtering unit and colored filtering unit simultaneously, wherein, the area of white filtering unit is the sum of 4 colored filtering unit areas, when guaranteeing image color information, can also promote image dynamic range and light inlet quantity, and then promote image sensor's performance. In addition, in the subsequent image processing process, the pixel data amount can be reduced, and the image processing speed can be increased to further improve the performance of the image sensor.

Alternatively, in some embodiments, the filter unit set may include three color filter units (a first color filter unit, a second color filter unit, and a third color filter unit), for example, three primary color filter units, that is, red, green, and blue (RGB) filter units, or three complementary color filter units, that is, cyan, magenta, and yellow (CMY) filter units, or two complementary colors of one primary color, or two primary colors of one complementary color.

In the case that the filtering unit group includes filtering units of three colors, light signals of three colors passing through the filtering units of three colors may cover a visible light band, and the specific color of the filtering unit is not limited in the embodiment of the present application.

For convenience of description, the following description will be given by taking a color filter unit including three colors of red, green, and blue in the filter unit group 211 as an example, and when the filter unit group includes color filter units of other three colors, the following technical solutions may be referred to.

Fig. 5 is a schematic arrangement diagram of filter units in a filter unit group 211.

As shown in fig. 5, the filter unit group 211 includes a white filter unit 201, a red filter unit 202, a green filter unit 203, and a blue filter unit 204, and optionally, a ratio of the number of the white filter units 201, the number of the green filter units 203, the number of the red filter units 202, and the number of the blue filter units 204 is 1: 2: 1: 1.

optionally, the white filter cells 201 are located at the upper right corner and the lower left corner of the filter cell group 211, and the corresponding color filter cell sets are located at the upper left corner and the lower right corner of the filter cell group 211.

In some embodiments, in the filter unit group 211, each color filter unit set includes 2 color filter units.

For example, the first color filter unit located at the upper left corner of the filter unit group includes a green filter unit 203 and a red filter unit 202; the second color filter unit located at the lower right corner of the filter unit group includes a green filter unit 203 and a blue filter unit 204.

Optionally, in each color filter set, the color filters of the same color are arranged at a common corner.

For example, in fig. 5, 2 green filter units and 2 red filter units in the first color filter unit set are arranged at a common vertex angle, and 2 green filter units and 2 blue filter units in the second color filter unit set are also arranged at a common vertex angle.

Further, in the two color filter sets, the relative positional relationship of the color filters of the same color is the same.

For example, in fig. 5, 2 green filter cells in the first color filter cell set are arranged at a 45 ° common vertex angle, and 2 green filter cells in the second color filter cell set are also arranged at a 45 ° common vertex angle. In the two color filter sets, the relative position relationship of the green filter units is the same.

Alternatively, the positions of the color filter units of different colors in fig. 5 may be changed based on the unchanged position of the white filter unit 201 in fig. 5.

Alternatively, on the basis of fig. 5, positions of the red filter unit and the blue filter unit are changed while positions of the white filter unit and the green filter unit are kept unchanged.

For example, fig. 6 shows a schematic arrangement of several transformed sets of filter elements.

In the several filter groups shown in fig. 6, in addition to the diagram (a) in fig. 6, in the several filter groups shown in the diagrams (b) and (c) in fig. 6, each color filter set includes 2 green filter units, 1 red filter unit, and 1 blue filter unit, and in the two color filter unit sets, the relative position relationship of the red filter units is different, and the relative position relationship of the blue filter units is also different.

Alternatively, on the basis of fig. 5, positions of the green filter unit, the red filter unit, and the blue filter unit may be further changed while maintaining the positions of the white filter unit, resulting in more arrangement forms of the filter unit groups.

For example, fig. 7 shows a schematic arrangement of several transformed sets of filter elements.

As shown in (a) to (b), (d) to (e), (g) to (h) of fig. 7, one color filter set includes 2 common-vertex green filter elements and 2 common-vertex red filter elements, and the other color filter set includes 2 common-vertex green filter elements and 2 common-vertex blue filter elements.

In (d) to (f) of fig. 7, 4 green filter units are disposed on one diagonal line of the filter unit group, and 2 white filter units are disposed on the other diagonal line.

In this embodiment, the green filter unit and the white filter unit are arranged on the diagonal of the filter unit group, which is beneficial to restoring image details/image texture information in image processing and improving image quality.

As in (c), (f), and (i) of fig. 7, each color filter unit includes 2 green filter units, 1 red filter unit, and 1 blue filter unit.

It will be appreciated that for each of the sets of filter elements of the present application, it is within the scope of the present application to have the set of filter elements undergo geometric transformations, such as rotations.

For example, after rotating the filter unit group shown in fig. 5 by 90 ° clockwise or rotating it by 90 ° counterclockwise, two filter unit groups shown in fig. 8 can be formed.

As shown in fig. 8, the white filter cells 201 are located at the upper left corner and the lower right corner of the filter cell group 211, and the corresponding color filter cell sets are located at the upper right corner and the lower left corner of the filter cell group 211.

It should also be understood that, in addition to fig. 5, the filter unit group formed by rotating any one of the filter unit groups in fig. 6 to 7 also belongs to the protection scope of the present application, and is not shown by way of example.

The filter unit groups of the above embodiments are obtained by performing conversion based on fig. 5, and in addition to the above-mentioned structures, the filter unit group of the present application may be the filter unit group 211 shown by a dashed box in fig. 9.

It is understood that the filter unit group 211 in fig. 9 and the filter unit group 211 in fig. 2 may form a filter unit array having the same central area, and the difference is only in the arrangement form of the filter units of one or two circles at the outermost periphery of the filter unit array 210, and therefore, the filter unit array formed by the filter unit group 211 in fig. 2 and 9 may be equivalent to the same filter unit array.

Therefore, in the present application, any adjacent 2 white filter cells and 2 color filter cell sets in the filter cell array 210 in fig. 2 may be divided into one filter cell group, and the filter cell group in any division case is within the protection scope of the present application.

Similarly, the filter unit array formed by any one of the filter unit groups in the present application is also within the protection scope of the present application.

Referring to the filter unit group shown in fig. 9, in the filter unit group 211 shown in fig. 9, the white filter units 201 are located at the upper left corner and the lower right corner of the filter unit group 211, and the corresponding color filter unit sets are located at the upper right corner and the lower left corner of the filter unit group 211.

Alternatively, in the filter unit group 211, each color filter unit set includes color filter units of 2 colors.

For example, in fig. 9, the first color filter unit in the upper right corner includes a green filter unit 203 and a blue filter unit 204; the second color filter unit at the lower left corner includes a green filter unit 203 and a red filter unit 202.

Furthermore, 2 green filter units in the first color filter unit set and the second color filter unit set are arranged at a common vertex angle of 45 degrees. In addition, 2 blue filter units in the first color filter unit set are arranged at-45 degrees with a common vertex angle, and 2 red filter units in the second color filter unit set are arranged at-45 degrees with a common vertex angle.

Similarly, the positions of the color filter units of different colors in fig. 9 may be changed based on the position of the white filter unit 201 in fig. 9 being unchanged.

Alternatively, on the basis of fig. 9, positions of the red filter unit and the blue filter unit may be changed while positions of the white filter unit and the green filter unit are kept unchanged.

Alternatively, on the basis of fig. 9, the positions of the white filter units may be kept unchanged, and the positions of the green filter units, the red filter units, and the blue filter units may be further changed to obtain more different arrangement forms of the filter unit groups.

For example, fig. 10 shows several arrangements of the transformed sets of filter elements.

For a specific transformation, reference may be made to the schematic diagrams of fig. 6 and fig. 7, which are not described herein again.

In the embodiments of the above application, the color filter units of the same color in each color filter unit set have common vertex angles.

In addition to this, in each color filter set, color filters of the same color may be arranged in common.

Fig. 11 is a schematic arrangement diagram of filter cells in another filter cell array 210 and its filter cell group 211.

In fig. 11, 2 green filter units and 2 red filter units in the first color filter unit set located at the upper left corner of the filter unit group are arranged in a common-edge manner, and 2 green filter units and 2 blue filter units in the second color filter unit set are also arranged in a common-edge manner.

Optionally, in the first color filter set and the second color filter set, the relative positional relationship of the color filters of the same color is different.

For example, as shown in fig. 11, in the filter unit group, 2 green filter units are located in the second row in the first color filter unit set, 2 green filter units are located in the first row in the second color filter unit set, and 2 green filter units are arranged at the same vertex in the center of the filter unit group.

In the filter unit group shown in fig. 11, 1 red filter unit, 2 green filter units and 1 blue filter unit adjacent to each other at a common vertex angle are sequentially disposed on one diagonal line, and 2 white filter units are disposed on the other diagonal line.

Alternatively, the positions of the color filter units of different colors in fig. 11 may be changed based on the position of the white filter unit 201 in fig. 11 being unchanged.

Fig. 12 shows a schematic arrangement of several transformed sets of filter elements.

As shown in fig. 12, the positions of the white filter unit and the green filter unit in the filter unit groups are the same as those in fig. 11, and the positions of the red filter unit and the blue filter unit are changed.

Alternatively, on the basis of fig. 11, the position of the green filter unit may be further changed, so as to obtain more arrangement forms of the filter unit groups.

Fig. 13 is a schematic arrangement diagram of another modified filter unit group.

As shown in fig. 13, the positions of the white filter elements in the several filter element groups are the same as those in fig. 11, and the positions of the green filter elements, the red filter elements, and the blue filter elements therein are changed.

Alternatively, in fig. 13, several filter unit groups are shown, each color filter unit set includes filter units of 2 colors, and the filter units of the same color in each color filter unit set are arranged adjacently on the same side.

Alternatively, in addition to several filter unit groups shown in fig. 13, filter units of 3 colors, that is, 2 green filter units, 1 red filter unit, and 1 blue filter unit may be included in each color filter unit set while maintaining positions of the white filter units and the green filter units unchanged.

It is understood that, for each of the filter unit groups in fig. 11 to 13, the filter unit group after being rotated or symmetrically transformed is also within the protection scope of the present application.

For example, fig. 14 shows a filter unit group obtained by symmetrically transforming the filter unit group with the vertical direction as the axis of symmetry.

In the embodiment of the above application, each color filter set includes 2 color filters.

In addition to this embodiment, in the embodiment of the present application, one color filter set includes color filters of 2 colors, and the other color filter set includes color filters of the same color.

Fig. 15 is a schematic arrangement diagram of filter cells in another filter cell array 210 and its filter cell group 211.

In fig. 15, the first color filter unit located at the upper left corner of the filter unit group includes 4 green filter units 203; the second color filter unit located at the lower right corner of the filter unit group includes 2 red filter units 202 and 2 blue filter units 204.

Optionally, in the second color filter set, color filters of the same color are arranged adjacent to each other.

For example, as shown in fig. 15, in the second color filter unit, 2 red filter units are adjacent to each other, and 2 blue filter units are adjacent to each other.

In the filter unit group shown in fig. 15, 2 green filter units, 1 red filter unit, and 1 blue filter unit adjacent to each other at a common vertex angle are sequentially disposed on one diagonal line, and 2 white filter units are disposed on the other diagonal line.

Alternatively, on the basis of fig. 15, the positions of the red filter unit and the blue filter unit are changed while keeping the positions of the white filter unit and the green filter unit unchanged, or the positions of the green filter unit, the red filter unit, and the blue filter unit are changed while keeping the positions of the white filter unit unchanged.

For example, fig. 16 shows a schematic arrangement diagram of several transformed sets of filter units.

It is understood that, for each of the filter unit sets in fig. 15 to 16, the filter unit set after being rotated or symmetrically transformed is also within the protection scope of the present application.

For example, fig. 17 shows a filter unit group obtained by symmetrically transforming the filter unit group with the vertical direction as the axis of symmetry.

It is to be understood that, in addition to the filter unit group 211 shown in fig. 11 and 15, the filter unit group in the present application may also be a set of 2 adjacent white filter units and 2 color filter units in the filter unit array 210 in fig. 11 and 15, which are divided into one filter unit group, and the filter unit group in any division case is within the protection scope of the present application.

The basic structure of the image sensor 200 and the arrangement of the various filter unit groups 211 therein in the present application are described above with reference to fig. 2 to 17, and the image processing method for the image sensor 200 in the present application is described below with reference to fig. 18 and 19.

Fig. 18 shows a schematic flow chart of an image processing method. Fig. 19 shows a schematic image through the image processing method of fig. 18.

As shown in fig. 18, the image processing method 10 includes:

s110: an image formed by the pixel unit array is sub-sampled, and a first sampling diagram comprising color pixel values and a second sampling diagram comprising white pixel values are obtained.

As an example, a diagram # 1 in fig. 19 is an image generated by one pixel cell group corresponding to one filter cell group in the pixel cell array.

In diagram # 1 in fig. 19, a white pixel value 101 is a pixel value generated after a white pixel unit receives a white light signal, and a white filter unit 201 is correspondingly arranged above the white pixel value, and a red pixel value 102 is a pixel value generated after a red pixel unit receives a red light signal, and a red filter unit 202 is correspondingly arranged above the red pixel value; similarly, the green pixel value 103 is a pixel value generated by the green pixel unit receiving the green light signal, and the green filter unit 203 is correspondingly disposed above the green pixel value, and the blue pixel value 104 is a pixel value generated by the blue pixel unit receiving the red light signal, and the blue filter unit 204 is correspondingly disposed above the blue pixel value.

Fig. 2# in fig. 19 is a first sampling diagram obtained by sub-sampling the red pixel value, the green pixel value, and the blue pixel value in fig. 1# and fig. 3# in fig. 19 is a second sampling diagram obtained by sub-sampling the white pixel value in fig. 1 #. In the first and second sampling maps, the relative positional relationship of the pixel values coincides with the relative positional relationship of the pixel values in the original image 1# map.

S120: and performing data processing on the first sampling image and the second sampling image to obtain a first image and a second image.

As an example, fig. 4# in fig. 19 is a first image of the first sampling chart (fig. 2 #) after data processing, and it can be seen that the first image obtained after data processing is a Bayer (Bayer) format data image.

Because the first image after data processing is a Bayer format which is commonly used in the field of image processing at present, the first image can be suitable for more types of Image Signal Processors (ISPs), so that the image sensor can be adapted to more ISPs in the application and is suitable for more application scenes.

Alternatively, 4 green pixel values in the first sampling diagram may be processed to obtain 2 green pixel values arranged diagonally in the bayer data image, 2 red pixel values in the first sampling diagram may be processed to obtain 1 red pixel value in the bayer data image, and 2 blue pixel values in the first sampling diagram may be processed to obtain 1 blue pixel value in the bayer data image. After the data processing in the step, the pixel value of the image is reduced, and the resolution of the image is half of the resolution of the image, so that the processing speed of the image data is facilitated, and the user experience is optimized.

Fig. 5# in fig. 19 is a second image of the second sampling chart (fig. 3 #) after data processing, and alternatively, the second image may include 4 white pixel values corresponding to 4 color pixel values in the first sampling chart (fig. 4 #). The interpolation algorithm may be used to interpolate 2 white pixel values in the second sampling map (3# map) to obtain 4 white pixel values of the second image. The interpolation process may use any interpolation algorithm in the prior art, which is not limited in the embodiment of the present application.

Alternatively, the second image may only include one white pixel value, the one white pixel value corresponding to 4 color pixel values in the first sampling diagram, and the one white pixel value may be obtained from 2 white pixel values in the second sampling diagram, which includes but is not limited to a mean value or a maximum value of 2 white pixel values in the second sampling diagram.

It will be appreciated that the second sample map and the second image are not used to characterize color information in the image, but rather to enhance the brightness of the image. In the embodiment of the application, the white pixel values are uniformly distributed in the image, and the spatial sampling rate is higher, which is beneficial to improving the resolution and the image quality of the image.

S130: and fusing the first image and the second image to obtain an optimized color image.

Optionally, the resolution of the first image is the same as the resolution of the second image.

As an example, image fusion is performed on the 4# image and the 5# image in fig. 19 to obtain an optimized color image 6#, and the color image after image fusion can better maintain brightness information while ensuring image color information, so that image quality under low-light conditions can be effectively improved.

It is understood that the above-mentioned image processing method 10 can be implemented by a processor or a processing circuit, in other words, optionally, in the above-mentioned image sensor, a processing unit for implementing the above-mentioned image processing method 10 can also be included.

In addition to the image sensor 200 provided in the embodiments of the application, the present application also provides an electronic device, which may include the image sensor 200 in any of the embodiments described above.

The electronic device may be any electronic device having an image capturing function, for example, the electronic device may specifically be a mobile terminal such as a mobile phone and a computer, a shooting device such as a camera and a video camera, an Automatic Teller Machine (ATM), and the like.

Alternatively, the processing unit for executing the image processing method 10 may be located in a processing unit in the electronic device where the image sensor 200 is located, instead of the image sensor 200, for example, if the electronic device is a mobile phone, the processing unit may be an image signal processing unit in a processor in the mobile phone, or the processing unit may also be a separate image signal processing chip in the mobile phone, and the embodiment of the present application does not limit the specific hardware form of the processing unit.

It should be understood that the processor or processing unit of the embodiments of the present application may be an integrated circuit chip having signal processing capability. In implementation, the steps of the above method embodiments may be performed by integrated logic circuits of hardware in a processor or instructions in the form of software. The Processor may be a general purpose Processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), an off-the-shelf Programmable Gate Array (FPGA) or other Programmable logic device, discrete Gate or transistor logic device, or discrete hardware components. The various methods, steps, and logic blocks disclosed in the embodiments of the present application may be implemented or performed. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like. The steps of the method disclosed in connection with the embodiments of the present application may be directly implemented by a hardware decoding processor, or implemented by a combination of hardware and software modules in the decoding processor. The software module may be located in ram, flash memory, rom, prom, or eprom, registers, etc. storage media as is well known in the art. The storage medium is located in a memory, and a processor reads information in the memory and completes the steps of the method in combination with hardware of the processor.

It will be appreciated that the image sensor of embodiments of the application may also include memory, which may be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. It should be noted that the memory of the systems and methods described herein is intended to comprise, without being limited to, these and any other suitable types of memory.

Those of ordinary skill in the art will appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.

It is clear to those skilled in the art that, for convenience and brevity of description, the specific working processes of the above-described systems, apparatuses and units may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.

In the several embodiments provided in the present application, it should be understood that the disclosed system, apparatus and method may be implemented in other ways. For example, the above-described apparatus embodiments are merely illustrative, and for example, the division of the units is only one logical division, and other divisions may be realized in practice, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.

The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.

In addition, functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit.

The functions, if implemented in the form of software functional units and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application may be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the methods described in the embodiments of the present application. And the aforementioned storage medium includes: various media capable of storing program codes, such as a usb disk, a removable hard disk, a read-only memory (ROM), a Random Access Memory (RAM), a magnetic disk, or an optical disk.

The above description is only for the specific embodiments of the present application, but the scope of the present application is not limited thereto, and any person skilled in the art can easily conceive of the changes or substitutions within the technical scope of the present application, and shall be covered by the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims (27)

1. An image sensor, comprising:

the filter unit array comprises a plurality of filter unit groups, each filter unit group in the plurality of filter unit groups comprises 2 white filter units and 2 color filter unit sets, each color filter unit set is equal to the area of each white filter unit, and each color filter unit set comprises 4 color filter units;

the pixel unit array comprises a plurality of pixel units, the pixel unit array is positioned below the light filtering unit array, and the plurality of pixel units in the pixel unit array correspond to the plurality of light filtering units in the light filtering unit array one to one.

2. The image sensor according to claim 1, wherein the filter unit group includes first, second, and third color filter units of different colors, and a number of the first color filter units is equal to a sum of numbers of the second and third color filter units.

3. The image sensor according to claim 2, wherein the number of the second color filter units is equal to the number of the third color filter units in the filter unit group.

4. The image sensor of claim 3, wherein in the set of filter units, the 2 white filter units are disposed adjacent to each other at a common vertex, and the 2 sets of color filter units are disposed adjacent to each other at a common vertex.

5. The image sensor of claim 4, wherein, among the 2 sets of color filter units, a first set of color filter units includes 2 of the first color filter units and 2 of the second color filter units, and a second set of color filter units includes 2 of the first color filter units and 2 of the third color filter units.

6. The image sensor of claim 5, wherein in each color filter set, color filters of the same color are disposed adjacent to each other at a common corner.

7. The image sensor according to claim 6, wherein in the 2 sets of color filter elements, the relative positional relationship of the color filter elements of the same color is the same.

8. The image sensor according to claim 7, wherein 4 first color filter units are arranged on one diagonal line and 2 white filter units are arranged on the other diagonal line in the filter unit group.

9. The image sensor according to claim 7, wherein in the filter unit group, 2 second color filter units adjacent to each other at a common vertex angle and 2 third color filter units adjacent to each other at a common vertex angle are disposed on one diagonal line, and 2 white filter units are disposed on the other diagonal line.

10. The image sensor of claim 5, wherein in each color filter set, color filters of the same color are arranged side-by-side adjacently.

11. The image sensor according to claim 10, wherein in the 2 sets of color filter elements, the relative positional relationship of the color filter elements of the same color is different.

12. The image sensor according to claim 11, wherein 1 second color filter unit, 2 first color filter units adjacent to a common vertex angle, and 1 third color filter unit are sequentially disposed on one diagonal line, and 2 white color filter units are disposed on the other diagonal line in the filter unit group.

13. The image sensor of claim 4, wherein, among the 2 sets of color filter units, a first set of color filter units comprises 4 of the first color filter units, and a second set of color filter units comprises 2 of the second color filter units and 2 of the third color filter units.

14. The image sensor of claim 13, wherein in the second set of color filter elements, color filter elements of the same color are adjacently disposed on a common side.

15. The image sensor according to claim 14, wherein 2 first color filter units, 1 second color filter unit and 1 third color filter unit adjacent to each other at a common vertex angle are sequentially disposed on one diagonal line, and 2 white filter units are disposed on the other diagonal line in the filter unit group.

16. The image sensor according to any one of claims 2 to 15, wherein the first color filter unit, the second color filter unit, and the third color filter unit are configured to pass light signals of three colors respectively, and the wavelength bands of the light signals of the three colors cover a visible light wavelength band.

17. The image sensor of claim 16, wherein the first color filter unit, the second color filter unit, and the third color filter unit are respectively three colors of red, green, blue, cyan, magenta, and yellow.

18. The image sensor of claim 17, wherein the first color filter unit is a green filter unit, the second color filter unit and the third color filter unit are a red filter unit and a blue filter unit, respectively.

19. The image sensor of any one of claims 1 to 15, further comprising:

the micro-lens array comprises a plurality of micro-lenses, is positioned above the light filtering unit array and is used for converging light signals returned by a shooting object to the light filtering unit array, wherein the plurality of micro-lenses in the micro-lens array correspond to the plurality of light filtering units in the light filtering unit array one by one.

20. The image sensor of claim 19, wherein a first microlens in the microlens array corresponds to a white filter cell in the filter cell array, and a second microlens in the microlens array corresponds to a color filter cell in the filter cell array;

wherein a radius of the first microlens is 2 times a radius of the second microlens.

21. The image sensor according to any one of claims 1 to 15, wherein a white pixel cell in the pixel cell array corresponds to a white filter cell in the filter cell array, and a color pixel cell in the pixel cell array corresponds to a color filter cell in the filter cell array;

wherein the area of the first pixel unit is 4 times of the area of the second pixel unit.

22. The image sensor according to any one of claims 1 to 15, wherein pixel values of color pixel cells in the pixel cell array are used to generate first image data of the photographic subject, pixel values of white pixel cells in the pixel cell array are used to generate second image data of the photographic subject, and the first image data and the second image data are used to synthesize a target image of the photographic subject;

the white pixel unit is a pixel unit corresponding to the white filter unit, and the color pixel unit is a pixel unit corresponding to the color filter unit.

23. The image sensor of claim 22, wherein pixel values of color pixel cells in the pixel cell array are used to generate first image data in bayer format by data processing.

24. The image sensor of claim 22, wherein the resolution of the first image data is half the resolution of the array of pixel cells.

25. The image sensor of claim 22, wherein the second image data and the first image data are of the same resolution; alternatively, the first and second electrodes may be,

the second image data has a resolution half of a resolution of the first image data.

26. The image sensor of any one of claims 1 to 15, wherein the image sensor is a Complementary Metal Oxide Semiconductor (CMOS) image sensor or a Charge Coupled Device (CCD) image sensor.

27. An electronic device, comprising:

the image sensor of any one of claims 1 to 26.

Images